KR102656929B1 - Intreactive White Board Having Facial recognition with multi-user security features - Google Patents

Abstract

The present invention is an interactive whiteboard with a face recognition function, which provides a camera unit for capturing a front image, a lecture screen, a display unit that displays the face image being captured by the camera unit on the screen when entering the instructor mode, and The interactive whiteboard is operated in one of administrator mode, public mode, and instructor mode. In administrator mode, user authentication is performed through NFC communication using a master key, and in instructor mode, user authentication is basically done through facial recognition. In public mode, user authentication is performed by entering a password, and when user authentication is performed in instructor mode, it includes a control section that controls the public application display section and a dedicated folder for the instructor to be displayed separately on the screen. .
According to the present invention as described above, the instructor can conveniently perform user authentication and use his/her own dedicated folder, making it easy for the instructor to manage classes efficiently and maintain the security of important data.

Description

{Intreactive White Board Having Facial recognition with multi-user security features}

The present invention relates to a technology that allows easy user authentication through facial recognition, identifies instructors through facial recognition, and configures a dedicated use environment for each instructor.

Interactive whiteboards, also known as electronic whiteboards, are products designed to perform various functions such as writing, data output, and screen sharing using a display panel and touch sensor. As interactive whiteboards have recently become popular, they are widely used in schools, businesses, and government offices. Interactive whiteboards are used to hold lectures and meetings in various places.

Interactive whiteboards are often used in academies, schools, etc. by multiple instructors sharing one interactive whiteboard. In this case, it is common for instructors to each be assigned an ID and perform user authentication by entering the assigned ID and password.

And since many instructors share one interactive whiteboard, the applications and materials that other instructors are using are also visible to other instructors. As a result, it is difficult to find your own materials, or other instructors accidentally delete the programs or materials you are using. Problems such as data being deleted may occur. In addition, there are cases where instructors are reluctant to share materials containing their own know-how or their own unique programs with other instructors.

1. Korean Patent No. 2492423 (Diy electronic whiteboard-based remote lecture provision system and method) 2. Korean Patent Publication No. 2433373 (ID generation device and method for touch input of electronic whiteboard)

The purpose of the present invention is to register the face of each instructor to enable easy user authentication through facial recognition, and to provide an instructor-only use environment such as a dedicated folder for each instructor during user authentication, making it convenient to use and keeping programs or materials used by each instructor private. So that it can be managed effectively.

Another object of the present invention is to increase face recognition accuracy by adjusting the brightness of the blank screen of the interactive whiteboard according to the surrounding brightness during face authentication, and to prevent the user from being glared by lighting by changing the brightness of the blank screen. It is to make it possible.

According to one aspect of the present invention for achieving the above object, a camera unit for capturing a front image and a lecture screen are provided, and when entering the lecture mode, a display that displays a face image being captured by the camera unit on the screen. First, the interactive whiteboard is operated in one of administrator mode, public mode, and instructor mode. In administrator mode, user authentication is performed through NFC communication using a master key, and in instructor mode, basically through facial recognition. User authentication is performed, and in public mode, user authentication is performed by entering a password. When user authentication is performed in instructor mode, it includes a control section that controls the public application display section and a dedicated folder for the instructor to be displayed on the screen separately. An interactive whiteboard with a face recognition function is provided.

Here, the control unit performs a function of managing the user authentication module that authenticates the user through one of the master key, face recognition, and password input depending on the operation mode, and the usage mode according to the usage authority and usage level of the interactive whiteboard. In the instructor mode, the mode management module grays out and deactivates the password menu as a default, and provides a user interface screen set according to the user mode. In the instructor mode, a public application in which applications commonly used by all users are displayed. It includes a UI management module that displays a section and a dedicated folder section where applications or documents used by the certified instructor are stored, and the dedicated folder section is preferably not shared between instructors.

In addition, it is more preferable that a sensor unit that detects ambient brightness information is further included, and the control unit includes a display control module that controls the blank brightness of the display unit based on the ambient brightness information detected through the sensor unit when capturing a face image. .

In addition, the display control module displays the face image being captured in the center of the screen, sets a brightness control area in the margins on both sides of the face image, and controls the brightness of the brightness control area to provide light of appropriate brightness to the user's face. possible.

In addition, a sensor unit that detects the distance between the screen and the user is further included, and the display control module can change the position and size of the brightness control area depending on the distance between the screen and the user.

In addition, the interactive whiteboard includes a face recognition application layer, a framework layer that provides the basic structure and common functions for face recognition application development, and a program to perform functions such as settings and control, face registration, face deletion, and face recognition. A native program layer including a network abstraction layer for transmission of video content, a V4L2 structure that is a plug-in for the V4L2 framework that supports camera capture and display, and a USB video class driver that streams video captured in the camera unit. It can be configured to include.

According to the present invention, instructors can conveniently perform user authentication and use their own dedicated folders, making it easier for instructors to manage classes efficiently and maintain the security of important data.

Figure 1 shows the configuration of an interactive whiteboard with a face recognition function according to the present invention.
Figure 2 shows the detailed configuration of the control unit.
Figure 3 shows an example of the initial screen of an interactive whiteboard with a face recognition function according to the present invention.
Figures 4a and 4b show the user authentication screen of the interactive whiteboard in instructor mode.
Figure 5 shows the instructor-only screen after user authentication is completed in instructor mode.
Figure 6 is a system hierarchy diagram showing the procedure in which face recognition is performed at each layer.

The embodiments described in the present invention and the configurations shown in the drawings are only preferred embodiments of the present invention and do not express the entire technical idea of the present invention. Therefore, the scope of rights of the present invention is limited to the embodiments and drawings described in the text. It should not be construed as limited by. In other words, since the embodiments can be modified in various ways and can have various forms, the scope of rights of the present invention should be understood to include equivalents that can realize the technical idea. In addition, the purpose or effect presented in the present invention does not mean that a specific embodiment must include all or only such effects, so the scope of the present invention should not be understood as limited thereby.

All terms used herein, unless otherwise defined, have the same meaning as commonly understood by a person of ordinary skill in the field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as consistent with the meaning in the context of the related technology, and cannot be interpreted as having an ideal or excessively formal meaning that is not clearly defined in the present invention.

Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the attached drawings.

Figure 1 shows the configuration of an interactive whiteboard with a face recognition function according to the present invention.

Referring to Figure 1, the interactive whiteboard with face recognition function according to the present invention includes a camera unit 10, a display unit 30, a power unit 30, a communication unit 40, a sensor unit 50, and a storage unit ( 60) and a control unit 70.

The camera unit 10 is installed in the interactive whiteboard main body in an embedded or externally mounted manner and captures images of the front of the interactive whiteboard. Filming of the front of the interactive whiteboard can be used for the purpose of filming the lecture scene and for instructor authentication through face authentication. In the present invention, the camera unit 10 is used for the purpose of authenticating the instructor's identity through face authentication.

The display unit 30 is a device for displaying user interface screens or lecture contents on a screen, and various light-emitting devices such as LED, LCD, and OLED may be used as a display device.

The power unit 30 is a part that supplies power used in the interactive whiteboard.

The communication unit 40 is a communication module for transmitting and receiving data for purposes such as an interface with an external PC or remote lectures or remote meetings via the Internet, and various wired and wireless communication devices can be used.

The sensor unit 50 is a sensor that detects the surrounding environment of the interactive whiteboard, and various sensors such as an infrared sensor to recognize the position of the instructor's hand and input written content, and an illuminance sensor to detect surrounding brightness can be used. Here, the illuminance sensor may be mounted integrally on the camera unit 10.

The storage unit 60 is an area where various data such as files or data required to run the interactive whiteboard, application programs used by instructors, and document data are stored.

The control unit 70 performs various control functions for the operation of the interactive whiteboard, and only the control functions related to the present invention will be separately described with reference to FIG. 2. Figure 2 shows the detailed configuration of the control unit.

Referring to FIG. 2, the control unit 70 includes a user authentication module 71, a mode management module 72, a UI management module 73, a folder management module 74, and a display control module 75. .

The user authentication module 71 performs the function of authenticating usage rights when a user wants to use the interactive whiteboard. As a user authentication method, a master key, password input, face recognition, etc. may be used depending on the user mode described later.

The mode management module 72 performs the function of managing the usage mode according to the usage authority and usage level of the interactive whiteboard. In the present invention, the user mode can be used in three types: administrator mode, public mode, and instructor mode. there is. Figure 3 shows an example of the initial screen of an interactive whiteboard with a face recognition function according to the present invention, in which a mode selection menu 21 is displayed on the screen of the display unit 20, and the mode selection menu 21 is It displays three usage modes: administrator mode, public mode, and instructor mode.

In administrator mode, user authentication is performed using a master key. Here, the master key may be in the form of a card equipped with an NFC module that recognizes the manager through NFC communication. Administrator mode has all setting rights for public mode and instructor mode, whether to set password authentication when entering public mode, permission to register/delete accounts to enter instructor mode, and files and folders created in instructor mode. Access rights are granted.

The public mode is a usage mode that allows the entire area of the storage unit 60 of the interactive whiteboard to be used as a shared space. After setting the password authentication entry function in the administrator mode, login can be performed by entering the password.

Instructor mode is a dedicated use mode for instructors who teach classes using an interactive whiteboard and can be accessed through face authentication or password authentication.

The user authentication screen in instructor mode of the interactive whiteboard is shown in FIG. 4A. Figure 4a is a screen when the password authentication entry function is set in the administrator mode. As a user authentication method, the password authentication menu 24 and the face display unit 23 where the user's captured face is located are displayed on the screen.

Figure 4b shows that in the default state in which the password authentication entry function is not set in the administrator mode, the password authentication menu 24 is grayed out so that only face recognition is performed.

The UI management module 73 performs the function of providing a user interface screen set according to the user mode.

Figure 5 shows the instructor-only screen after user authentication is completed in instructor mode. Referring to Figure 5, the instructor-only screen displays a public app section 26 and a dedicated folder section 27.

The public application section 26 is a section where applications that are commonly used by all users regardless of the instructor are displayed, and the same content is commonly displayed when the user enters the instructor mode.

The dedicated folder unit 27 is a folder dedicated to an instructor identified through user authentication, and applications or documents downloaded by the instructor are stored in the folder. The dedicated folder is not shared between users and is set as a secure area that can only be accessed through user authentication.

The folder management module 74 performs the function of separating and managing the public folder commonly used by all users and the instructor's private folder. The folder management module 74 divides the storage into a general area and a security area, sets a dedicated folder area within the security area, divides the dedicated folder area again by instructor, and starts the dedicated folder area before the partition information of the dedicated folder storage area. Add a flag that displays and store user identification or password information before or after the flag.

The display control module 75 controls the operation of the display unit 20, and in the present invention, operates in conjunction with the sensor unit 50.

When a user who wants to use the interactive whiteboard through the camera unit 10 selects the instructor mode in FIG. 3, the user's face is displayed on the face display unit 23 in FIG. 4A. If the surrounding brightness is dark or too bright, the accurate Face recognition may be difficult. In this case, the ambient brightness information detected through the sensor unit 50 is applied to the display control module 75, and the display control module 75 controls the blank space 22 of the display unit 20 when, for example, the ambient brightness is dark. By increasing the brightness, you can provide an optimal shooting environment.

In this case, separate lighting is not used and the screen brightness is adjusted using the blank space 22 outside the screen excluding the face display unit 23 displayed in the center of the screen of the display unit 20, so separate lighting is not required. This has the effect of maintaining appropriate shooting illumination without the user feeling glare.

As another method, as shown in FIG. 4A, the brightness control area 25 is set on both sides of the heat display unit 23 located in the center of the screen to control the brightness of the brightness control area 25 to more effectively control the brightness of the user's face. It is possible to provide light. The position and size of the brightness control area 25 can be appropriately controlled depending on the distance between the screen and the user. For example, when the user is standing close to the screen, the size of the brightness control area 25 is increased and the brightness change is reduced. Conversely, when the user is standing far from the screen, the size of the brightness control area 25 is reduced and the brightness change is reduced. You can position it and increase the screen brightness so that the light is well illuminated around the face.

Figure 6 is a system hierarchy diagram showing the procedure in which face recognition is performed at each layer.

The application layer 100 is a layer that provides user interface functions of an application program for face recognition.

The framework layer 110 is a software framework that provides basic structure and common functions when developing facial recognition applications. Here, the framework layer 110 consists of Java code running on the Java Virtual Machine (JVM), native applications (programs dependent on hardware and operating system platforms), and libraries written in other languages such as C, C++, and Assembly. It includes Java Native Interface (JNI), a programming framework that allows calling or vice versa.

The native program layer 120 is a program layer created in the programming language used to create the operating system, and includes programs to perform functions such as setting and control, face registration, face deletion, and face recognition.

The face detection program is a module that detects the face area from the image captured by the camera unit 10. The face detection algorithm was mainly based on selecting Haar-like features by boosting, but as deep learning methods have recently shown high performance, the entire image is reviewed using the Sliding Window method to determine the presence or absence of a face using AlexNet-based classification. Methods, a face detector that fine-tunes YOLO, a high-speed object detector, and a face detection algorithm that fine-tunes Faster R-CNN can be used.

Face recognition programs mainly use the interest point extraction algorithm, which is an algorithm that extracts feature points that can become features of the image by using pixels corresponding to a certain area of the two-dimensional image. At this time, feature points refer to corners and blobs. Feature points are also vectors, but each is assigned a unique scale and direction. Because feature points are constructed relative to this scale and direction, they can be robust to changes in scale or rotation.

Representative feature extraction algorithms include SIFT (Scale Invariant Feature Transform) and SURF (Speeded Up Robust Feature).

In feature point detection using SURF, the presence of a corner is determined using the determinant of the approximated Hessian matrix. At this time, a corner is a case where the value of the determinant is greater than 0, and is also called a maximum.

D, which constitutes the approximate Hessian matrix, means an approximation of the second order partial derivative of the Gaussian kernel, and each approximation is a convolution on the image in the form of a filter. do. At this time, the black area of the filter is given a weight of -2, the white area is given a weight of 1, and the gray area is given a weight of 0. Because the filters increase in scale at a constant rate, corners of various scales can be detected.

In feature point description, the direction of the feature point is calculated using pixel values around the location of the detected feature point, and the feature point whose direction is calculated is composed of a descriptor in the form of a vector, which is the final step.

The extracted feature points are used for feature point matching to calculate the matched feature points, and the matched feature points are used for feature point verification to calculate the number of verified feature points. Finally, the training image corresponding to the largest number of verified feature points becomes the face recognition result.

NAL (130: Network Abstraction Layer) was developed for efficient transmission of video content, provides an abstracted access method called a port to the network, and is part of the H.264/AVC and HEVC video coding standards.

The kernel layer 140 includes the V4L2 structure, a plug-in for the V4L2 framework that supports camera capture and display, and a UVC (USB Video Class) driver, which is a device that streams video from webcams and digital camcorders.

Although the present invention has been described in connection with the above-mentioned preferred embodiments, various modifications and variations can be made without departing from the spirit and scope of the invention. Accordingly, the scope of the appended claims will include such modifications or variations as fall within the gist of the present invention.

Claims (6)

An interactive whiteboard with facial recognition function,
A camera unit that captures front images;
It provides a lecture screen, comprising: a display unit that displays a face image being captured by the camera unit on the screen when entering the instructor mode;
A sensor unit that detects ambient brightness information;
The interactive whiteboard is operated in one of the user modes among administrator mode, public mode, and instructor mode. In administrator mode, user authentication is performed through NFC communication using a master key, and in instructor mode, user authentication is performed basically through facial recognition. Authentication is performed, and in public mode, user authentication is performed by entering a password. When user authentication is performed in instructor mode, it includes a control section that controls the public application display section and a dedicated folder for the instructor to be displayed separately on the screen. ,
The control unit controls the brightness of the blank space of the display unit based on the surrounding brightness information detected through the sensor unit when shooting a face image. The face image being captured is displayed in the center of the screen and sets a brightness control area in the margins on both sides of the face image. A facial recognition interactive whiteboard with a multi-user security function, comprising a display control module that controls the brightness of the brightness control area to provide light of appropriate brightness to the user's face.
According to claim 1,
The control unit
a user authentication module that authenticates the user through one of master key, face recognition, and password input depending on the user mode;
A mode management module that performs a function of managing the usage mode according to the usage authority and usage level of the interactive whiteboard, and grays out the password menu as a default in the instructor mode to disable it;
Provides a user interface screen set according to the user mode, but in the instructor mode, a public application section where applications commonly used by all users are displayed and a dedicated folder section where applications or documents used by the certified instructor are stored. UI management module; A facial recognition interactive whiteboard with a multi-user security function, wherein the dedicated folder portion is not shared between instructors.
delete delete According to claim 1,
A sensor unit that detects the distance between the screen and the user is further included,
The display control module is a facial recognition interactive whiteboard with a multi-user security function, wherein the display control module changes the position and size of the brightness control area depending on the distance between the screen and the user.
According to claim 1,
The interactive whiteboard is
Facial recognition application layer;
Framework layer that provides basic structure and common functions for developing facial recognition applications;
a native program layer that includes programs to perform functions such as setup and control, face registration, face deletion, and face recognition;
Network abstraction layer for transmission of video content; and
It has a multi-user security function, characterized by including a kernel layer including a V4L2 structure, which is a plug-in for the V4L2 framework that supports camera capture and display, and a USB video class driver that streams video captured in the camera unit. Facial recognition interactive whiteboard.