CN113747078A

CN113747078A - Display device and focus control method

Info

Publication number: CN113747078A
Application number: CN202111134859.8A
Authority: CN
Inventors: 孟祥菲
Original assignee: Hisense Visual Technology Co Ltd
Current assignee: Hisense Visual Technology Co Ltd
Priority date: 2021-09-18
Filing date: 2021-09-18
Publication date: 2021-12-03
Anticipated expiration: 2041-09-18
Also published as: WO2023040288A1; CN113825002A; CN113747078B; CN113825002B; CN117918057A

Abstract

The application relates to a display device and a focal length control method, which relate to the technical field of televisions, and the method comprises the following steps: responding to the focal length adjusting instruction, updating a focal length display range of the display, enabling the camera to detect a gesture instruction, obtaining a gesture rectangular region, determining that the gesture rectangular region is in the focal length display range based on the focal length display range and the gesture rectangular region, executing the gesture instruction, determining that the gesture rectangular region is not in the focal length display range based on the focal length display range and the gesture rectangular region, and not executing the gesture instruction. According to the display device and the focal length control method, only the gesture command in the focal length display range visible for the user is executed, and the user experience is improved.

Description

Display device and focus control method

The present application is a divisional application of chinese patent application 202111104825.4 entitled "display device and focal length control method" filed on 9/18/2021.

Technical Field

The application relates to the technical field of smart televisions, in particular to a display device and a focal length control method.

Background

With the rapid development of display devices, the functions of the display devices are increasing, for example, the display devices are equipped with cameras to provide functions of taking pictures, mirrors and the like for users.

However, in practical applications, the user does not see the corresponding gesture in the display interface of the display device, and the display device gives the photographing feedback.

Disclosure of Invention

The application provides an image processing method for a display device and the display device, only executing gesture instructions in a focal distance display range visible for a user, and improving the use experience of the user.

In a first aspect, the present application provides a display device comprising:

a display;

a camera;

a controller connected to the camera and the display, respectively, the controller configured to:

updating a focal length display range of the display in response to a focal length adjustment instruction;

the camera detects a gesture instruction to acquire a gesture rectangular area;

determining that the gesture rectangular area is within the focus display range based on the focus display range and the gesture rectangular area, and executing the gesture instruction;

and determining that the gesture rectangular area is not in the focus display range based on the focus display range and the gesture rectangular area, and not executing the gesture instruction.

In some embodiments of the present application, the controller is further configured to:

determining that the camera does not adjust the focal length based on the focal length adjusting instruction;

and the camera detects a gesture instruction and executes the gesture instruction.

In some embodiments of the present application, the controller, when executing the updating of the focal length display range of the display in response to the focal length adjustment instruction, is further configured to:

determining that the camera has performed focal length adjustment based on the focal length adjustment instruction, and acquiring focal length coordinate information;

updating the focal length display range of the display based on the focal length coordinate information.

In some embodiments of the present application, the controller, in executing the obtaining the rectangular region of the gesture, is further configured to:

acquiring a plurality of key points of the gesture instruction;

acquiring key point coordinate information corresponding to the plurality of key points;

determining a minimum horizontal and vertical coordinate value and a maximum horizontal and vertical coordinate value based on the coordinate information of the key points;

and acquiring the gesture rectangular area based on the minimum horizontal and vertical coordinate value and the maximum horizontal and vertical coordinate value.

In some embodiments of the application, the controller, in performing determining that the gesture rectangular area is within the focus display range based on the focus display range and the gesture rectangular area, executes the gesture instruction, and is further configured to:

acquiring target coordinate information of the focal length display range;

acquiring gesture coordinate information of the gesture rectangular area;

and determining that the gesture rectangular area is in the focal length display range based on the target coordinate information and the gesture coordinate information, and executing the gesture instruction.

In some embodiments of the present application, the controller, in performing determining that the gesture rectangular area is not within the focus display range based on the focus display range and the gesture rectangular area, and not performing the gesture instruction, is further configured to:

and determining that the gesture rectangular area is not in the focal length display range based on the target coordinate information and the gesture coordinate information, and not executing the gesture instruction.

In a second aspect, the present application provides another display device, comprising:

a display;

a camera;

responding to a control signal for starting the camera, and displaying an image collected by the camera in the display;

responding to a control signal indicating face focus, and acquiring a face position rectangular area based on the image;

determining an adjustment central point based on the face position rectangular area;

and adjusting the image according to the adjustment central point and a preset adjustment value to obtain a target image, and displaying the target image on the display.

acquiring a human body position rectangular region based on the image;

correspondingly, the controller, in performing the determining of the adjustment center point based on the face position rectangular region, is further configured to:

and determining the adjustment central point based on the face position rectangular region and the human body position rectangular region.

In some embodiments of the present application, in executing the determining of the adjustment center point based on the face position rectangular region, the controller is further configured to:

acquiring coordinate information of the face position rectangular area based on the face position rectangular area;

and calculating and processing based on the coordinate information of the face position rectangular area, and acquiring the coordinate of a central point as the adjustment central point.

In some embodiments of the present application, the controller, in performing the determining the adjustment center point based on the face position rectangular region and the body position rectangular region, is further configured to:

acquiring coordinate information of the human body position rectangular area based on the human body position rectangular area;

calculating and processing the coordinate information of the rectangular area of the face position to obtain a first central point coordinate;

calculating and processing the coordinate information of the rectangular area of the human body position to obtain a second central point coordinate;

and calculating and processing based on the first central point coordinate and the second central point coordinate, and acquiring a target central point coordinate as the adjustment central point.

In a third aspect, a method for controlling a focal length includes:

the camera detects a gesture instruction to acquire a gesture rectangular area;

In a fourth aspect, another focus control method of the present application includes:

Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages:

responding to the focal length adjusting instruction, updating a focal length display range of the display, enabling the camera to detect a gesture instruction, obtaining a gesture rectangular region, determining that the gesture rectangular region is in the focal length display range based on the focal length display range and the gesture rectangular region, executing the gesture instruction, determining that the gesture rectangular region is not in the focal length display range based on the focal length display range and the gesture rectangular region, and not executing the gesture instruction. Therefore, only the gesture instruction in the focal length display range visible for the user is executed, and the use experience of the user is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the implementation manner in the related art, a brief description will be given below of the drawings required for the description of the embodiments or the related art, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art according to the drawings.

FIG. 1 illustrates an operational scenario between a display device and a control apparatus according to some embodiments;

fig. 2 illustrates a hardware configuration block diagram of the control apparatus 100 according to some embodiments;

fig. 3 illustrates a hardware configuration block diagram of the display apparatus 200 according to some embodiments;

FIG. 4 illustrates a software configuration diagram in the display device 200 according to some embodiments;

FIG. 5a illustrates an example diagram of displaying an image in a display device 200 according to some embodiments;

FIG. 5b illustrates an example diagram of camera range versus focal range in accordance with some embodiments;

FIG. 6 illustrates a flow chart of a focus control method according to some embodiments;

FIG. 7a illustrates an example diagram of gesture control in a display device 200 according to some embodiments;

FIG. 7b illustrates an example diagram of gesture control in another display device 200 according to some embodiments;

FIG. 8 illustrates a flow diagram of another focus control method according to some embodiments;

FIG. 9 illustrates an exemplary diagram of gesture regions in a display device 200 according to some embodiments;

FIG. 10a illustrates an exemplary diagram of a rectangular area of face positions in a display device 200 according to some embodiments;

FIG. 10b illustrates an exemplary determination of an adjusted center point for a rectangular region of face positions in display device 200 according to some embodiments;

FIG. 10c illustrates an exemplary determination of an adjusted center point for a rectangular region of face positions in another display device 200 according to some embodiments;

FIG. 10d illustrates an example diagram of displaying a target image in a display device 200 according to some embodiments;

FIG. 11 illustrates a flow diagram of yet another focus control method according to some embodiments;

FIG. 12 illustrates an exemplary determination of an adjusted center point for a rectangular region of face positions in another display device 200 according to some embodiments;

FIG. 13 illustrates a flow diagram of yet another focus control method according to some embodiments;

FIG. 14 illustrates an example diagram of displaying a target image in another display device 200 according to some embodiments.

Detailed Description

To make the purpose and embodiments of the present application clearer, the following will clearly and completely describe the exemplary embodiments of the present application with reference to the attached drawings in the exemplary embodiments of the present application, and it is obvious that the described exemplary embodiments are only a part of the embodiments of the present application, and not all of the embodiments.

It should be noted that the brief descriptions of the terms in the present application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of the present application. These terms should be understood in their ordinary and customary meaning unless otherwise indicated.

The terms "first," "second," "third," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between similar or analogous objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

As used in this application, the terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements expressly listed but may include other elements not expressly listed or inherent to such product or apparatus.

The term "gesture" as used in this application refers to a user's behavior through a change in hand shape or an action such as hand motion to convey a desired idea, action, purpose, or result.

Fig. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in fig. 1, a user may operate the display apparatus 200 through the smart device 300 or the control device 100.

In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes an infrared protocol communication or a bluetooth protocol communication, and other short-distance communication methods, and controls the display device 200 in a wireless or wired manner. The user may input a user instruction through a key on a remote controller, voice input, control panel input, etc., to control the display apparatus 200. For example, the user inputs a corresponding control command through a volume up/down key, a channel control key, a number key, etc. on the remote controller, to implement the function of controlling the display device 200.

In some embodiments, the smart device 300 (e.g., mobile terminal, tablet, computer, laptop, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application program running on the smart device.

In some embodiments, the display device 200 may also be controlled in a manner other than the control apparatus 100 and the smart device 300, for example, the voice command control of the user may be directly received by a module configured inside the display device 200 to obtain a voice command, or may be received by a voice control device provided outside the display device 200.

In some embodiments, the display device 200 is also in data communication with a server 400. The display device 200 may be allowed to be communicatively connected through a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display apparatus 200. The server 400 may be a cluster or a plurality of clusters, and may include one or more types of servers.

Fig. 2 exemplarily shows a block diagram of a configuration of the control apparatus 100 according to an exemplary embodiment. As shown in fig. 2, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control apparatus 100 may receive an input operation instruction from a user and convert the operation instruction into an instruction recognizable and responsive by the display device 200, serving as an interaction intermediary between the user and the display device 200.

Fig. 3 shows a hardware configuration block diagram of the display apparatus 200 according to an exemplary embodiment.

In some embodiments, the display apparatus 200 includes at least one of a tuner demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, a user interface.

In some embodiments the controller comprises a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, a first interface to an nth interface for input/output.

In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving a focus control, a component for receiving an image signal from the controller output, performing display of video content, image content, and a menu manipulation interface, and a user manipulation UI interface.

In some embodiments, the display 260 may be a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

In some embodiments, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display apparatus 200 may establish transmission and reception of control signals and data signals with the external control apparatus 100 or the server 400 through the communicator 220.

In some embodiments, the user interface may be configured to receive control signals for controlling the apparatus 100 (e.g., an infrared remote control, etc.).

In some embodiments, the detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; alternatively, the detector 230 includes an image collector, such as a camera, which may be used to collect external environment scenes, attributes of the user, or user interaction gestures, or the detector 230 includes a sound collector, such as a microphone, which is used to receive external sounds.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, and the like. The interface may be a composite input/output interface formed by the plurality of interfaces.

In some embodiments, the tuner demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio/video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

In some embodiments, the controller 250 and the modem 210 may be located in different separate devices, that is, the modem 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any one of selectable objects, such as a hyperlink, an icon, or other actionable control. The operations related to the selected object are: displaying an operation connected to a hyperlink page, document, image, or the like, or performing an operation of a program corresponding to the icon.

In some embodiments the controller comprises at least one of a Central Processing Unit (CPU), a video processor, an audio processor, a Graphics Processing Unit (GPU), a RAM Random Access Memory (RAM), a ROM (Read-Only Memory), a first to nth interface for input/output, a communication Bus (Bus), and the like.

A CPU processor. For executing operating system and application program instructions stored in the memory, and executing various application programs, data and contents according to various interactive instructions receiving external input, so as to finally display and play various audio-video contents. The CPU processor may include a plurality of processors. E.g. comprising a main processor and one or more sub-processors.

In some embodiments, a graphics processor for generating various graphics objects, such as: icons, operation menus, user input instruction display graphics, and the like. The graphic processor comprises an arithmetic unit, which performs operation by receiving various interactive instructions input by a user and displays various objects according to display attributes; the system also comprises a renderer for rendering various objects obtained based on the arithmetic unit, wherein the rendered objects are used for being displayed on a display.

In some embodiments, the video processor is configured to receive an external video signal, and perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to a standard codec protocol of the input signal, so as to obtain a signal that can be displayed or played on the direct display device 200.

In some embodiments, the video processor includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like. The demultiplexing module is used for demultiplexing the input audio and video data stream. And the video decoding module is used for processing the video signal after demultiplexing, including decoding, scaling and the like. And the image synthesis module is used for carrying out superposition mixing processing on the GUI signal input by the user or generated by the user and the video image after the zooming processing by the graphic generator so as to generate an image signal for display. And the frame rate conversion module is used for converting the frame rate of the input video. And the display formatting module is used for converting the received video output signal after the frame rate conversion, and changing the signal to be in accordance with the signal of the display format, such as an output RGB data signal.

In some embodiments, the audio processor is configured to receive an external audio signal, decompress and decode the received audio signal according to a standard codec protocol of the input signal, and perform noise reduction, digital-to-analog conversion, and amplification processing to obtain an audio signal that can be played in the speaker.

In some embodiments, a user may enter user commands on a Graphical User Interface (GUI) displayed on display 260, and the user input interface receives the user input commands through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through the sensor. In some embodiments, a "user interface" is an application or operating system

The media interface for interaction and information exchange with a user enables conversion between an internal form of information and a user-acceptable form. A commonly used presentation form of the User Interface is a Graphical User Interface (GUI), which refers to a User Interface related to computer operations and displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in the display screen of the electronic device, where the control may include a visual interface element such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.

In some embodiments, a system of a display device may include a Kernel (Kernel), a command parser (shell), a file system, and an application program. The kernel, shell, and file system together make up the basic operating system structure that allows users to manage files, run programs, and use the system. After power-on, the kernel is started, kernel space is activated, hardware is abstracted, hardware parameters are initialized, and virtual memory, a scheduler, signals and interprocess communication (IPC) are operated and maintained. And after the kernel is started, loading the Shell and the user application program. The application program is compiled into machine code after being started, and a process is formed.

Referring to fig. 4, in some embodiments, the system is divided into four layers, which are an Application (Applications) layer (abbreviated as "Application layer"), an Application Framework (Application Framework) layer (abbreviated as "Framework layer"), an Android runtime (Android runtime) and system library layer (abbreviated as "system runtime library layer"), and a kernel layer from top to bottom.

In some embodiments, at least one application program runs in the application program layer, and the application programs may be windows (windows) programs carried by an operating system, system setting programs, clock programs or the like; or an application developed by a third party developer. In particular implementations, the application packages in the application layer are not limited to the above examples.

The framework layer provides an Application Programming Interface (API) and a programming framework for the application. The application framework layer includes a number of predefined functions. The application framework layer acts as a processing center that decides to let the applications in the application layer act. The application program can access the resources in the system and obtain the services of the system in execution through the API interface.

As shown in fig. 4, in the embodiment of the present application, the application framework layer includes a manager (Managers), a Content Provider (Content Provider), and the like, where the manager includes at least one of the following modules: an Activity Manager (Activity Manager) is used for interacting with all activities running in the system; the Location Manager (Location Manager) is used for providing the system service or application with the access of the system Location service; a Package Manager (Package Manager) for retrieving various information related to an application Package currently installed on the device; a Notification Manager (Notification Manager) for controlling display and clearing of Notification messages; a Window Manager (Window Manager) is used to manage the icons, windows, toolbars, wallpapers, and desktop components on a user interface.

In some embodiments, the activity manager is used to manage the lifecycle of the various applications as well as general navigational fallback functions, such as controlling exit, opening, fallback, etc. of the applications. The window manager is used for managing all window programs, such as obtaining the size of a display screen, judging whether a status bar exists, locking the screen, intercepting the screen, controlling the change of the display window (for example, reducing the display window, displaying a shake, displaying a distortion deformation, and the like), and the like.

In some embodiments, the system runtime layer provides support for the upper layer, i.e., the framework layer, and when the framework layer is used, the android operating system runs the C/C + + library included in the system runtime layer to implement the functions to be implemented by the framework layer.

In some embodiments, the kernel layer is a layer between hardware and software. As shown in fig. 4, the core layer includes at least one of the following drivers: audio drive, display driver, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (like fingerprint sensor, temperature sensor, pressure sensor etc.) and power drive etc..

In some embodiments, taking the magic mirror application (photographing application) in fig. 4 as an example, when the remote control receiving device receives an input operation of the remote controller, it is determined that a control corresponding to the input operation is a control corresponding to an icon of the magic mirror application, the magic mirror application calls an interface of the application framework layer, starts the magic mirror application, calls the kernel layer to start the camera driver, and thus, the camera obtains an image.

When the user utilizes the display device 200 to take a picture, the photographing application has a focus adjusting function and a gesture photographing function, the gesture photographing function refers to that a corresponding gesture is put out in the photographing range of the camera to automatically take a picture, and the focus adjusting function refers to that the focus display range of the camera can be adjusted through the related keys of the remote controller. If the user adjusts the focal length, the whole camera shooting area can be divided into a display part and a non-display part, the current realization effect is that the part which is not displayed on the display can be shot by putting out the gesture, actually, the user does not see the corresponding gesture in the camera display range, and the display equipment gives the shooting feedback to influence the user experience.

The following describes in detail the procedure of the focus control of the display device in this embodiment.

Specifically, fig. 5a shows an exemplary diagram of displaying an image in the display device 200 according to some embodiments, as shown in fig. 5a, an image is displayed in the display, the image is obtained by shooting through a camera through focus adjustment, it should be noted that the image displayed by the display after the camera is subjected to focus adjustment is different from the image corresponding to the shooting range of the camera, as shown in fig. 5b, the shooting range of the camera shooting the corresponding image is greater than the shooting range corresponding to the display image shown in fig. 5a, that is, the focus display range described in the embodiment of the present application.

In the above embodiment, the user opens the photographing application installed in the display device 200 by operating the control device, and then displays the original image, that is, the image corresponding to the photographing range of the camera, in the display, and then the user adjusts the focal length by operating the control device, and then displays the current image after the focal length adjustment, that is, the image corresponding to the focal length display range, in the display, where the focal length display range is smaller than the photographing range of the camera.

FIG. 6 illustrates a flow diagram of a focus control method according to some embodiments. In a display device provided in an embodiment of the present application, when executing a gesture instruction according to a gesture position and a focal length display range, a controller is configured to execute the focal length control method described in fig. 6, including:

step 601, responding to the focal length adjusting instruction, and updating the focal length display range of the display.

In some embodiments of the present application, the focal length adjustment instruction refers to an instruction (for example, an instruction indicating focal length adjustment of a camera or an image magnification instruction, etc.) for adjusting a focal length display range of a display, and may be input by a user through an operation control device (for example, a remote controller), or input by a user through a sound collector (for example, a microphone) of a display device, and specifically selects a setting according to an application scenario, which is not limited in this application.

In some embodiments of the present application, after the camera is started, a focal distance display range of the display is a camera shooting range, and in response to a focal distance adjustment instruction, the focal distance display range of the display is updated, specifically, a coordinate system is created, focal distance coordinate information after the focal distance adjustment is obtained, the focal distance display range of the display is updated according to the focal distance coordinate information, for example, a coordinate axis is established with a lower left corner of the camera shooting range as an original coordinate point, a lower boundary of the camera shooting range is a horizontal axis, a left boundary is a vertical axis, a coordinate system is created, and the focal distance coordinate information after the focal distance adjustment is read, so that the focal distance display range of the display is updated according to the focal distance coordinate information.

Step 602, the camera detects a gesture instruction, and acquires a gesture rectangular area.

In an embodiment of the application, the gesture instruction refers to a command transfer mode in which a camera detects a special gesture, for example, a series of photographing behaviors are completed through gesture recognition, the gesture recognition may be understood that a user may use a simple gesture to control or interact with a device, the camera acquires an image, and the detection image includes the special gesture, and then the gesture instruction is considered to be detected, wherein the user may make different gestures within a photographing range of the camera to send different instructions to a display device.

Specifically, the camera detects a gesture instruction, processes an image including the gesture instruction, acquires a gesture rectangular region, more specifically, identifies all key points of a gesture in the image region, determines a drawing rectangle of a minimum horizontal and vertical coordinate value and a maximum horizontal and vertical coordinate value according to coordinate information of all the key points, and acquires the gesture rectangular region.

Step 603, determining that the gesture rectangular area is in the focal length display range based on the focal length display range and the gesture rectangular area, and executing a gesture instruction.

And step 604, determining that the gesture rectangular area is not in the focal length display range based on the focal length display range and the gesture rectangular area, and not executing a gesture instruction.

In some embodiments of the present application, the focal length display range is taken as a standard, the gesture rectangular region executes the gesture instruction in the focal length display range, the gesture rectangular region does not execute the gesture instruction in the focal length display range, and only executes the gesture instruction in the focal length display range visible to a user, so that effectiveness and accuracy of control of the display device are improved.

In some embodiments of the application, as shown in fig. 7a, the gesture rectangular area is in the focal length display range, and executes a gesture instruction, and more specifically, the gesture instruction is a photographing instruction, and executes photographing operation to acquire and store an image, further, a user may further perform editing operation on the photographed image through a control device, a voice instruction, a gesture instruction, or the like, and it can be understood that the user may select to shoot again, and perform photographing again through the gesture instruction, so that various use requirements of the user are met, and use experience of the user is improved.

In some embodiments of the present application, as shown in fig. 7b, the gesture rectangular area is not in the focal length display range, and the gesture instruction is not executed, that is, although the gesture instruction is obtained in the camera shooting range, the gesture instruction is not in the user visible range, and the gesture instruction is not executed, so that the user experience is further improved.

In an embodiment of the application, whether a gesture rectangular area is in a focal length display range is judged through coordinate information of the focal length display range and coordinate information of the gesture rectangular area, more specifically, a coordinate axis is established by taking the lower left corner of a camera shooting range as an original coordinate point, a boundary below the camera shooting range is a horizontal axis, and a boundary on the left side is a vertical axis, a coordinate system is created, target coordinate information of the focal length display range is obtained, gesture coordinate information of the gesture rectangular area is obtained, and based on the target coordinate information and the gesture coordinate information, it is determined that the gesture rectangular area executes a gesture instruction in the focal length display range or it is determined that the gesture rectangular area does not execute the gesture instruction in the focal length display range.

In the above example, only the gesture instruction within the focal length display range visible to the user is executed, so that the user can perform adjustment and control operations according to the seen gesture, and the user experience is further improved.

FIG. 8 illustrates a flow diagram of another focus control method according to some embodiments. The present embodiment further optimizes the above focus control method based on the above embodiments. As shown in fig. 8, the method includes:

in some embodiments of the present application, it is determined whether the camera is to adjust the focal length.

In some embodiments of the present application, the focal length adjustment is a function of adjusting a display area on a photographing application of the display device, for example, a display range of a camera area on the display device is adjusted by a specific key of a remote controller, if the focal length is not adjusted, the camera detects a gesture instruction, executes the gesture instruction, if the focal length is adjusted, acquires focal length coordinate information after the focal length adjustment, and updates the focal length display range of the display based on the focal length coordinate information.

In some embodiments of the present application, the display range of the camera is adjusted by a control device, such as a related key of a remote controller, and in a case that the focal length is adjusted, the entire shooting area of the camera is divided into a display portion and a non-display portion, as shown in fig. 7a, the display portion is in the focal length display range, and the other portions are non-display portions.

In some embodiments of the present application, the camera detects a gesture instruction, and executes the gesture instruction.

In some embodiments of the present application, if the focal length is not adjusted by the camera, the area within the shooting range of the camera is consistent with the focal length display range of the display, an image is collected by the camera, and a gesture is included in the detected image, the gesture instruction is detected, and the gesture instruction is directly executed.

In some embodiments of the present application, a user inputs a focus adjustment instruction by operating a control device (e.g., a remote controller) or by a user inputting through a sound collector (e.g., a microphone) of a display device or the like. In response to the above instruction, the display device does not substantially adjust the focal length of the camera, that is, the camera maintains the original shooting range, for example, as shown in fig. 7a, the whole shooting range of the camera remains unchanged. But after the image collected by the camera is obtained, the image is amplified and then displayed in the display, so that the effect of amplifying the object can be achieved visually from the user.

In this case, the range of the image captured by the actual camera is large, and the display displays only a part of the enlarged image. In other words, the focal length display range on the display actually shows that a part of the image collected by the camera is enlarged, so that the focal length display range of the display is inconsistent with the shooting range of the camera, for example, as shown in fig. 7a, the whole shooting area of the camera is divided into a display part and a non-display part, the display part is in the focal length display range, and the non-display part is in other parts. At this time, the gesture rectangular area executes the gesture command within the focus display range with reference to the focus display range, and the gesture rectangular area does not execute the gesture command within the focus display range.

In this case, although it is possible that the user's gesture is still within the range captured by the camera, if it is not already within the range displayed by the display, the instruction is not recognized even though the gesture may be captured by the camera, or the instruction may be recognized but the instruction corresponding to the gesture is not responded to.

Therefore, the user can carry out adjustment and control operation according to the gesture intuitively seen on the display, and the effectiveness and the accuracy of the control of the display equipment are improved.

In some embodiments of the present application, focal distance coordinate information is obtained, and a focal distance display range of the display is updated based on the focal distance coordinate information.

In some embodiments of the present application, a coordinate axis is established with the lower left corner of the camera shooting range as an original coordinate point (0, 0), the lower boundary of the camera shooting range is a horizontal axis, and the left boundary is a vertical axis, a coordinate system is created, and focal length coordinate information after focal length adjustment, such as four coordinate points (x 1, y 1), (x 1, y 2), (x 2, y 1), (x 2, y 2), is read, so that a focal length display range is determined according to the four coordinate points.

In some embodiments of the application, the camera detects a gesture instruction, acquires a plurality of key points of the gesture instruction, acquires coordinate information of the key points corresponding to the plurality of key points, determines a minimum horizontal and vertical coordinate value and a maximum horizontal and vertical coordinate value, and acquires a gesture rectangular region based on the minimum horizontal and vertical coordinate value and the maximum horizontal and vertical coordinate value.

In the embodiment of the present application, as shown in fig. 9, a plurality of key points of a gesture command are obtained, and key point coordinate information corresponding to the plurality of key points is obtained, coordinate axes are established with the lower left corner of a camera shooting range as an original coordinate point (0, 0), a boundary below the camera shooting range is a horizontal axis, and a boundary on the left side is a vertical axis, a coordinate system is created, and the key point coordinate information is read, so as to determine a minimum horizontal and vertical coordinate value and a maximum horizontal and vertical coordinate value, for example, four coordinate points (m 1, n 1), (m 1, n 2), (m 2, n 1), (m 2, n 2), thereby determining a gesture rectangular region according to the four coordinate points.

In some embodiments of the present application, target coordinate information of a focal length display range is obtained, and gesture coordinate information of a gesture rectangular region is obtained.

In some embodiments of the present application, a gesture rectangular area is determined to be within the focus display range based on the target coordinate information and the gesture coordinate information, a gesture command is executed,

in some embodiments of the application, based on the target coordinate information and the gesture coordinate information, it is determined that the gesture rectangular area is not within the focus display range, and no gesture instruction is executed.

In some embodiments of the present application, target coordinate information of the focus display range such as (x 1, y 1), (x 1, y 2), (x 2, y 1), (x 2, y 2) described above is acquired, gesture coordinate information of the gesture rectangular region is acquired as (m 1, n 1), (m 1, n 2), (m 2, n 1), (m 2, n 2) described above, it is determined that the gesture rectangular region is within the focus display range according to the target coordinate information and the gesture coordinate information, a gesture instruction is executed, it is determined that the gesture rectangular region is not within the focus display range based on the target coordinate information and the gesture coordinate information, and the gesture instruction is not executed.

Therefore, the camera photographing interface can adjust the photographing display range through control equipment such as a remote controller, can photograph in the focus display range when the gesture, and cannot photograph in the focus display range when the gesture is not.

In the above example, only the gesture instruction in the focal length display range visible to the user is executed when the focal length is adjusted, so that the user can perform adjustment and control operations according to the viewed gesture, and the gesture instruction is detected to be directly executed in the camera shooting range when the focal length is not adjusted, thereby further improving the user experience.

In some embodiments, taking the magic mirror application (photographing application) in fig. 4 as an example, the artificial intelligence trial makeup is a trial makeup function on the photographing application, and by calculating the position of the face and drawing the makeup onto the corresponding position of the face, the trial makeup function includes a face focus effect, in the camera area, the area where the position of the face is located is focused, and the focus is changed as a person or an object approaches or moves away from the camera when a lens is photographed, so that the focus is maintained at an accurate focus. The face focus tracking of the display device is a focusing mode in which a camera can automatically identify the position of the face in a picture.

Since the user uses the face focus following function, when the face is at the lower corner (as shown in fig. 10 a), the human body detection position (0, 0, 0, 0) is used, and the calculated rectangular zoom area is as shown in fig. 10b, so that the center point is not accurately adjusted, resulting in poor focus following effect.

FIG. 11 illustrates a flow diagram of yet another focus control method according to some embodiments. In a display device provided in an embodiment of the present application, when a human face is in focus, a controller is configured to execute the focal length control method described in fig. 11, where the method includes:

and step 1101, responding to a control signal for starting the camera, and displaying the image collected by the camera in the display.

In some embodiments of the present application, a control signal for starting the camera is input through a key on the remote controller, a voice input, a control panel input, and the like, and an image obtained by the camera is displayed on the display.

In some embodiments of the present application, the image displayed in the display may include a face image or may not include a face image, and the present application mainly aims at processing including a face image, and the face image may be located at any position of the displayed image, such as the lower right corner of the image shown in fig. 10, or the center position shown in fig. 12, which is specifically determined according to the position where the user is located in the application scenario.

In step 1102, a face position rectangular region is obtained based on an image in response to a control signal indicating face focus.

In some embodiments of the present application, a control signal for a face to follow up is input through a key on a remote controller, a voice input, a control panel input, and the like, where a face region refers to a region where a face is located, so as to perform face recognition on an image, obtain a face position rectangular region, and recognize the face region of the image.

Step 1103, determining an adjustment center point based on the face position rectangular region.

And step 1104, adjusting the image according to the adjustment central point and a preset adjustment value, acquiring a target image, and displaying the target image on a display.

In some embodiments of the present application, the adjustment center point refers to a coordinate point in a face position rectangular region, and the adjustment center point is obtained by calculating through coordinate information of the face position rectangular region, specifically, coordinate information of the face position rectangular region is obtained, calculation processing is performed based on the face position rectangular region coordinate information, and the center point coordinate is obtained as the adjustment center point; the coordinate information refers to one or more coordinate points on the rectangular region of the face position and coordinate values corresponding to the coordinate points.

In some embodiments of the present application, the preset adjustment value may be selectively set according to the application scenario requirement.

In some embodiments of the present application, the adjustment center point is used as a center point, the pixel adjustment is performed on the image according to a preset adjustment value, a target image is obtained, and the target image is displayed on the display. Continuing with fig. 10a as an example, the obtained adjustment center point is calculated as the O point shown in fig. 10c, the preset adjustment value is one-fifth, the image in fig. 10c is adjusted, and the obtained target image is shown in fig. 10d, so that the focus following effect is ensured.

In the above example, the face focus is followed by calculating and adjusting the central point according to the face region, so that the focus following effect is ensured.

FIG. 13 illustrates a flow diagram of yet another focus control method according to some embodiments. The present embodiment further optimizes the above focus control method based on the above embodiments. As shown in fig. 11, the method includes:

in some embodiments of the present application, an image captured by a camera is displayed in a display, and a face position rectangular region is obtained based on the image in response to a control signal indicating that a face is in focus.

In some embodiments of the present application, the image displayed in the display may include a face image or not, and the present application is mainly directed to processing including a face image, and the face image may be in any position of the displayed image.

In some embodiments of the application, control signals of face focus tracking are input through keys on a remote controller, voice input, control panel input and the like, so that face recognition is performed on an image, and a face position rectangular region is obtained.

In some embodiments of the present application, an image collected by a camera is displayed in a display, and in response to a control signal indicating that a human face is in focus, after a human face position rectangular region is acquired based on the image, determining an adjustment center point based on the human face position rectangular region or acquiring the human body position rectangular region based on the image may be performed, and determining the adjustment center point based on the human face position rectangular region and the human body position rectangular region.

In some embodiments of the application, the adjustment center point is determined based on the human body position rectangular region which cannot be obtained from the image and based on the human face position rectangular region.

In some embodiments of the present application, it can be understood that the human body position rectangular region is a body region where four limbs and a trunk part are located, and after the human face position rectangular region is determined, a human body detection algorithm may be adopted to determine the human body position rectangular region, or a human body detection model is established, and then detection is performed based on the human body detection model to obtain the human body position rectangular region.

In some embodiments of the application, the human body position rectangular region is not obtained based on the image, the image only includes the human face position rectangular region, and the adjustment center point is determined based on the human face position rectangular region. That is, when only the face position rectangular region is detected and the body position rectangular region is not detected, only the face position rectangular region is used as the adjustment center point for calculating the rectangular zoom region.

In some embodiments of the present application, coordinate information of the face position rectangular region is obtained based on the face position rectangular region, calculation processing is performed based on the face position rectangular region coordinate information, and the center point coordinate is obtained as an adjustment center point, and more specifically, when the face focus following function is started, a face position rectangular area and a human body position rectangular area are detected, establishing coordinate axes by taking the lower left corner of the image as an original coordinate point (0, 0), taking the lower boundary of the shooting range of the camera as a horizontal axis and the left boundary as a vertical axis, creating a coordinate system, reading coordinate information of a rectangular area of the face position, such as four coordinate points (a 1, b 1), (a 1, b 2), (a 2, b 1), (a 2, b 2), thus, a face position rectangular region is determined from the four coordinate points, and the center point coordinates are further calculated as ((a 1+ a 2)/2, (b 1+ b 2)/2) as the adjustment center point.

In some embodiments of the present application, a human body position rectangular region is obtained based on an image, and an adjustment center point is determined based on the human face position rectangular region and the human body position rectangular region.

In some embodiments of the present application, coordinate information of a face position rectangular region is obtained based on the face position rectangular region, coordinate information of a body position rectangular region is obtained based on the body position rectangular region, calculation processing is performed based on the face position rectangular region coordinate information, a first central point coordinate is obtained, calculation processing is performed based on the body position rectangular region coordinate information, a second central point coordinate is obtained, calculation processing is performed based on the first central point coordinate and the second central point coordinate, and a target central point coordinate is obtained as an adjustment central point. More specifically, when the face focus function is turned on, a face position rectangular region and a body position rectangular region are detected, coordinate axes are established with the lower left corner of the image as an original coordinate point (0, 0), the lower boundary of the camera shooting range is a horizontal axis, and the left boundary is a vertical axis, a coordinate system is created, face position rectangular region coordinate information and body position rectangular region coordinate information, such as four coordinate points (a 1, b 1), (a 1, b 2), (a 2, b 1), (a 2, b 2), are read, so that the face position rectangular region is determined from the four coordinate points, and four coordinate points (c 1, d 1), (c 1, d 2), (c 2, d 1), (c 2, d 2), so that the body position rectangular region is determined from the four coordinate points.

Further, according to the determination that both the face position rectangular region and the body position rectangular region are detected at the same time when none of (a 1, b1, a2, b 2), (c 1, d1, c2, d 2) is zero, and the double determination is performed in combination with both, the adjustment center point is calculated from the body position rectangular region and the face position rectangular region, that is, the first center point coordinates (h 1, h 2) of the face position rectangular region are ((a 1+ a 2)/2, (b 1+ b 2)/2), and the second center point coordinates (k 1, k 2) of the body position rectangular region are ((c 1+ c 2)/2, (d 1+ d 2)/2), and the target center point coordinates are obtained by calculation processing according to the first center point coordinates and the second center point coordinates ((h 1+ k 1)/2, (h 2+ k 2)/2) as the adjustment center point.

In some embodiments of the present application, the image is adjusted according to the adjustment center point and a preset adjustment value, a target image is obtained, and the target image is displayed on a display.

In some embodiments of the present application, an adjustment center point is determined based on the face position rectangular region, specifically, coordinate information of the face position rectangular region is obtained, calculation processing is performed based on the face position rectangular region coordinate information, and a center point coordinate is obtained as the adjustment center point.

In some embodiments of the present application, the adjustment center point is used as a center point, the pixel adjustment is performed on the image according to a preset adjustment value, a target image is obtained, and the target image is displayed on the display. Continuing with fig. 10a as an example, the obtained adjustment center point is calculated as the point O shown in fig. 10a, the preset adjustment value is five times, the image in fig. 10a is adjusted, and the obtained image is shown in fig. 10c, so that the focus following effect is ensured. Continuing with fig. 12 as an example, the obtained adjustment center point is calculated as the O point shown in fig. 12, the preset adjustment value is one to five times, the image in fig. 12 is adjusted, and the obtained image is shown in fig. 14, so that the focus following effect is ensured.

Therefore, the camera displays the face focus following effect of the interface on the display, the face position rectangular area and the human body position rectangular area serve as double detection standards, when the face position rectangular area and the human body position rectangular area are detected, the face position rectangular area and the human body position rectangular area are used for calculating the rectangular scaling area, and when the human body position rectangular area is not detected, the face position rectangular area is used for calculating the rectangular scaling area. In addition, when the human body position rectangular area and the human face position rectangular area can not be detected, the rectangular scaling area is set to be null.

While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include such modifications and variations.

Claims

1. A display device, comprising:

a camera configured to acquire image data;

a display configured to present a user interface, wherein image data acquired by a camera is presented when a camera-related application is opened;

responding to an instruction for opening the camera related application, and controlling a display to present image data acquired by the camera;

in response to a user-entered instruction to trigger zooming in on the image data, zooming in on the image data and presenting at least a portion of the zoomed in image data on the display;

responding to a preset gesture acquired by the camera, acquiring a gesture rectangular area, and generating a gesture instruction corresponding to the preset gesture;

executing the gesture instruction if the gesture rectangular area is within a user interface presented by the display;

if the gesture rectangular area is not presented within the user interface, the gesture instruction is not executed.

2. The display device of claim 1, wherein the controller is further configured to:

determining a magnification factor by which the image data needs to be magnified in response to an instruction for triggering magnification of the image data input by a user;

and amplifying the image data according to the amplification factor.

3. The display device of claim 1, wherein the controller, in executing the instruction for triggering enlargement of the image data in response to user input, enlarging the image data, and presenting at least a portion of the enlarged image data on the display, is further configured to:

in response to a user-entered instruction to trigger enlarging of the image data, enlarging the image data,

presenting at least a portion of the magnified image data on the display, determining a focal display range based on the image data presented on the display; wherein the remaining portion of the enlarged image data is not displayed on the display.

4. The display device according to claim 1, wherein the controller, in executing the get gesture rectangular area, is further configured to:

acquiring a plurality of key points of the gesture instruction;

5. The display device according to claim 1, wherein the controller, in performing determining that the gesture rectangular region is within the focus display range based on the focus display range and the gesture rectangular region, performs the gesture instruction, and is further configured to:

acquiring target coordinate information of the focal length display range;

acquiring gesture coordinate information of the gesture rectangular area;

6. The display device according to claim 5, wherein the controller, in performing determining that the gesture rectangular region is not within the focus display range based on the focus display range and the gesture rectangular region, without performing the gesture instruction, is further configured to:

7. The display device according to claim 2, wherein the controller determines a magnification factor by which the image data needs to be magnified when executing an instruction for triggering magnification of the image data in response to a user input; amplifying the image data according to the amplification factor, specifically configured to:

when it is detected that a human object is in the image data acquired by the camera,

and amplifying the image data according to the amplification factor so that the amplified image data still contains the person.

8. The display device according to claim 2, wherein the controller determines a magnification factor by which the image data needs to be magnified when executing an instruction for triggering magnification of the image data in response to a user input; amplifying the image data according to the amplification factor, specifically configured to:

acquiring a face position rectangular area;

and adjusting the image according to the adjustment central point and the magnification factor to obtain a target image, and displaying the target image on the display.

9. The display device according to claim 7, wherein the controller, in performing the determining an adjustment center point based on the face position rectangular region, is further configured to:

10. A focus control method, comprising:

controlling a display to present image data acquired by a camera;

in response to an instruction for triggering the image data to be amplified, the image data is amplified, and at least a part of the amplified image data is presented on the display, wherein the focal distance display range is acquired based on the image data presented on the display;