CN111767828B - Certificate image reproduction identification method and device, electronic equipment and storage medium - Google Patents
Certificate image reproduction identification method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111767828B CN111767828B CN202010597003.3A CN202010597003A CN111767828B CN 111767828 B CN111767828 B CN 111767828B CN 202010597003 A CN202010597003 A CN 202010597003A CN 111767828 B CN111767828 B CN 111767828B
- Authority
- CN
- China
- Prior art keywords
- image
- identified
- certificate
- frequency domain
- document image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000001228 spectrum Methods 0.000 claims abstract description 67
- 230000008569 process Effects 0.000 claims abstract description 17
- 230000001131 transforming effect Effects 0.000 claims abstract description 5
- 238000013527 convolutional neural network Methods 0.000 claims description 36
- 238000004891 communication Methods 0.000 claims description 17
- 238000001514 detection method Methods 0.000 claims description 14
- 238000010586 diagram Methods 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 238000012545 processing Methods 0.000 abstract description 9
- 238000004364 calculation method Methods 0.000 abstract description 3
- 238000004422 calculation algorithm Methods 0.000 description 9
- 230000003595 spectral effect Effects 0.000 description 7
- 230000003993 interaction Effects 0.000 description 4
- 238000007639 printing Methods 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 239000000969 carrier Substances 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The application relates to a certificate image reproduction identification method, a device, electronic equipment and a storage medium, which are applied to the technical field of image processing, wherein the method comprises the following steps: acquiring a certificate image to be identified; transforming the certificate image to be identified to obtain a spectrum image of the certificate image to be identified; and extracting the spatial domain characteristics of the certificate image to be identified and the frequency domain characteristics of the frequency spectrum image, judging whether the certificate image to be identified is a reproduction image according to the extracted spatial domain characteristics and the frequency domain characteristics, obtaining a judgment result, and taking the judgment result as a certificate image reproduction identification result. Therefore, the problems of complex calculation process and time and space resource consumption caused by limitation of the shooting process and shooting equipment in the prior art when certificate image recognition is carried out are solved.
Description
Technical Field
The present application relates to the field of image processing, and in particular, to a method and apparatus for identifying a document image by a roll-over, an electronic device, and a storage medium.
Background
The document image reproduction refers to a process of shooting a document object in a real scene by using an optical lens to obtain a first image, projecting the first image on other development carriers (paper printing, a screen and the like), and shooting by using the optical lens again to obtain a second image containing first image information. This practice can bring about the risk of fraudulent use of certificates, and can cause loss of personal information security. Therefore, in some business scenarios where document information needs to be audited, the second image that is flipped needs to be identified and intercepted.
In the related art, the certificate reproduction recognition technology mainly combines the spatial domain characteristics of the images with the interactive information, specifically, the technology needs to control the flash of a flash lamp and the flash of a gyroscope in the shooting process by means of external equipment, such as a mobile phone with the flash lamp and the gyroscope, collect the data of the gyroscope, and collect multi-frame images through interaction to obtain additional information.
However, this method has high requirements on the photographing device, and other devices (a flash lamp, a gyroscope, etc.) are required to be equipped on the photographing device at the same time; moreover, a photographer is required to shoot a video according to a specified interaction flow, so that the user experience is poor; in addition, the original data processed in this way is a video, and additional pre-algorithm output (shake judgment, hand-held judgment, frame selection) is required, which causes the consumption of space resources and time resources.
Disclosure of Invention
The application provides a certificate image reproduction identification method, a device, electronic equipment and a storage medium, which are used for solving the problems of limitation on a shooting process and shooting equipment, complex calculation process and time and space resource consumption in the prior art when certificate image identification is carried out.
In a first aspect, an embodiment of the present application provides a document image reproduction identification method, including:
acquiring a certificate image to be identified;
transforming the certificate image to be identified to obtain a frequency spectrum image of the certificate image to be identified;
and extracting the spatial domain characteristics of the certificate image to be identified and the frequency domain characteristics of the frequency spectrum image, judging whether the certificate image to be identified is a reproduction image or not according to the extracted spatial domain characteristics and the frequency domain characteristics, obtaining a judgment result, and taking the judgment result as a certificate image reproduction identification result.
Optionally, the acquiring the certificate image to be identified includes:
obtaining an original document image obtained by photographing a document, wherein the original document image comprises a background part and a document part to be identified;
and positioning and cutting the original document image to remove the background part in the original document image, thereby obtaining the document image to be identified.
Optionally, the positioning and cropping the original document image to remove the background portion in the original document image to obtain the document image to be identified includes:
positioning the original document image to obtain the center point and the size of a document part to be identified in the original document image;
and cutting the original document image according to the center point and the size of the document part to be identified so as to remove the background part in the original document image and obtain the document image to be identified.
Optionally, the positioning the original document image to obtain a center point and a size of a document part to be identified in the original document image includes:
and predicting the center point and the size of the certificate part to be identified in the original certificate image based on a target detection network, wherein the size is obtained by the target detection network in a regression mode after the center point is obtained.
Optionally, the extracting the spatial domain feature of the document image to be identified and the frequency domain feature of the spectrum image, and judging whether the document image to be identified is a flip image according to the extracted spatial domain feature and the frequency domain feature, to obtain a judgment result, including:
inputting the certificate image to be identified and the spectrum image into a convolutional neural network model;
extracting the airspace characteristics of N network levels of the certificate image to be identified and the frequency domain characteristics of N network levels of the spectrum image through N network levels in the convolutional neural network model, fusing the airspace characteristics of N network levels and the frequency domain characteristics of N network levels to obtain a characteristic diagram of the certificate image to be identified, judging whether the certificate image to be identified is a flip image according to the characteristic diagram, and outputting a judging result.
Optionally, extracting airspace features of N network levels of the document image to be identified and frequency domain features of N network levels of the spectrum image through N network levels in the convolutional neural network model, and fusing the airspace features of the N network levels and the frequency domain features of the N network levels to obtain a feature map of the document image to be identified, and outputting a judging result after judging whether the document image to be identified is a flip image according to the feature map, where the step of outputting the judging result includes:
carrying out group convolution on the certificate image to be identified and the frequency spectrum image by adopting the 1 st network level to obtain the airspace characteristic of the 1 st network level of the certificate image to be identified and the frequency domain characteristic of the 1 st network level of the frequency spectrum image;
carrying out group convolution on the feature map of the ith-1 network level by adopting the ith network level to obtain the airspace feature of the ith network level of the certificate image to be identified and the frequency domain feature of the ith network level of the frequency spectrum image, wherein the value of i is more than 1 and less than or equal to N;
after the airspace feature and the frequency domain feature of the Nth network level are obtained, fusing the airspace feature and the frequency domain feature of the Nth network level to obtain a feature map of the Nth network level, and performing downsampling and full connection on the feature map;
and judging whether the certificate image to be identified is a reproduction image or not through an activation function according to the fully connected result, and outputting a judging result.
Optionally, the flipping image includes: color-printed flip images and screen flip images.
In a second aspect, an embodiment of the present application provides a document image reproduction identifying apparatus, including:
the acquisition module is used for acquiring a certificate image to be identified;
the image conversion module is used for converting the certificate image to be identified to obtain a frequency spectrum image of the certificate image to be identified;
the certificate copying judging module is used for extracting the airspace characteristics of the certificate image to be identified and the frequency domain characteristics of the frequency spectrum image, judging whether the certificate image to be identified is a copying image according to the extracted airspace characteristics and the frequency domain characteristics, obtaining a judging result, and taking the judging result as a certificate image copying identification result.
Optionally, the acquiring module specifically includes:
the acquisition sub-module is used for acquiring an original document image shot by the document, wherein the original document image comprises a background part and a document part to be identified;
and the certificate detection and positioning module is used for positioning and cutting the original certificate image so as to remove the background part in the original certificate image and obtain the certificate image to be identified.
Optionally, the certificate detection positioning module includes:
the positioning module is used for positioning the original document image to obtain the center point and the size of the document part to be identified in the original document image;
and the clipping module is used for clipping the original document image according to the center point and the size of the document part to be identified so as to remove the background part in the original document image and obtain the document image to be identified.
Optionally, the positioning module includes:
the center point positioning module is used for predicting the center point of the certificate part to be identified in the original certificate image based on the target detection network;
and the size positioning module is used for obtaining the size of the certificate part to be identified in a regression mode.
Optionally, the document flap discriminating module includes:
the input module is used for inputting the certificate image to be identified and the spectrum image into a convolutional neural network model;
the judging module is used for extracting the airspace characteristics of the N network levels of the certificate image to be identified and the frequency domain characteristics of the N network levels of the frequency spectrum image through the N network levels in the convolutional neural network model, fusing the airspace characteristics of the N network levels and the frequency domain characteristics of the N network levels to obtain a characteristic diagram of the certificate image to be identified, judging whether the certificate image to be identified is a flip image according to the characteristic diagram, and outputting a judging result.
Optionally, the judging module is specifically configured to perform packet convolution on the document image to be identified and the spectrum image by adopting the 1 st network level, so as to obtain a spatial domain feature of the 1 st network level of the document image to be identified and a frequency domain feature of the 1 st network level of the spectrum image;
carrying out group convolution on the feature map of the ith-1 network level by adopting the ith network level to obtain the airspace feature of the ith network level of the certificate image to be identified and the frequency domain feature of the ith network level of the frequency spectrum image, wherein the value of i is more than 1 and less than or equal to N;
after the airspace feature and the frequency domain feature of the Nth network level are obtained, fusing the airspace feature and the frequency domain feature of the Nth network level to obtain a feature map of the Nth network level, and performing downsampling and full connection on the feature map;
and judging whether the certificate image to be identified is a reproduction image or not through an activation function according to the fully connected result, and outputting a judging result.
Optionally, the flipping image includes: color-printed flip images and screen flip images.
In a third aspect, an embodiment of the present application provides an electronic device, including: the device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
the memory is used for storing a computer program;
the processor is configured to execute the program stored in the memory, to implement the document image reproduction identification method according to the first aspect.
In a fourth aspect, an embodiment of the present application provides a computer readable storage medium storing a computer program, where the computer program when executed by a processor implements the document image reproduction identification method according to the first aspect.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: according to the method provided by the embodiment of the application, when the certificate image to be identified is identified, no additional interaction information is needed, the shooting equipment and the shooting process are not limited, the video stream is not required to be processed, and only the certificate image to be identified is required to be obtained for subsequent processing, so that the identification of the image can be realized, the flow of an algorithm is simplified, and space resources and time resources are saved; moreover, the user does not need to shoot a video, and can complete identification only by inputting a certificate image to be identified, so that the user experience is improved; in addition, the application not only utilizes the airspace characteristics of the certificate image, but also combines the frequency domain characteristics of the certificate image to identify, thereby improving the accuracy of image identification.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the application or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a flowchart of a document image reproduction identification method according to an embodiment of the present application;
FIG. 2 is a flowchart of a document image reproduction identification method according to another embodiment of the present application;
FIG. 3 is a flowchart of image positioning and cropping in a document image reproduction identification method according to an embodiment of the present application;
FIG. 4 is a spectrum diagram of a spectral image of different types of raw document images according to one embodiment of the present application;
FIG. 5 is a process diagram of convolutional neural network model recognition in a document image roll-over recognition method according to an embodiment of the present application;
FIG. 6 is a flowchart of training a convolutional neural network model in a document image roll-over recognition method according to an embodiment of the present application;
FIG. 7 is a block diagram of a document image reproduction identification apparatus according to an embodiment of the present application;
fig. 8 is a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The embodiment of the application provides a certificate image reproduction identification method which can be applied to any type of electronic equipment, such as a terminal or a server. As shown in fig. 1, the certificate image reproduction identification method includes:
and 101, acquiring a certificate image to be identified.
In some embodiments, the certificate image to be identified may be obtained by uploading the certificate image in the corresponding input box by the user, or may be obtained in the corresponding webpage. Of course, the certificate image to be identified can also be acquired directly.
Step 102, transforming the certificate image to be identified to obtain a spectrum image of the certificate image to be identified.
In some embodiments, the image to be identified is subjected to fourier transformation to obtain a spectrum image, and the identification result can be more accurate by performing double identification on the certificate image with the image to be identified and the spectrum image. Fourier transform transforms signals from the time domain to the frequency domain, and further studies the spectral structure and the change law of the signals. The fourier transform may, among other things, transform the document image to be identified using a fast fourier transform (Fast Fourier Transform, FFT), which is an efficient, fast computing method to compute the discrete fourier transform (Discrete Fourier Transform, DFT). The image can be transformed from a spatial domain to a frequency domain by 2D fast fourier transform, and a spectral image on the frequency domain is obtained.
Step 103, extracting the spatial domain characteristics of the certificate image to be identified and the frequency domain characteristics of the frequency spectrum image, judging whether the certificate image to be identified is a reproduction image according to the extracted spatial domain characteristics and frequency domain characteristics, obtaining a judging result, and taking the judging result as a reproduction identification result of the certificate image.
In some embodiments, further identification of the document image and the spectral image to be identified may be achieved by a convolutional neural network model. Specifically, the document image and the spectrum image to be identified can be input into the convolutional neural network model, and the spatial domain feature of the document image and the frequency domain feature of the spectrum image to be identified are gradually extracted through the network level in the convolutional neural network model, so that whether the document image to be identified is a flip image or not is judged based on the spatial domain feature and the frequency domain feature.
In the embodiment, the identification of the image can be realized by acquiring the identification image to be identified without additional interaction information, limiting the shooting equipment and the shooting process, processing video stream, acquiring the identification image to be identified and performing subsequent processing, so that the flow of an algorithm is simplified, space resources and time resources are saved, a user does not need to shoot a video, the identification can be completed by inputting the identification image to be identified, and the user experience is improved; and then carrying out Fourier transform on the certificate image to be identified to obtain a frequency spectrum image, and finally, identifying the certificate image to be identified by extracting the airspace characteristics of the certificate image to be identified and the frequency domain characteristics of the frequency spectrum image, so as to judge whether the certificate image to be identified is a flip image.
In another embodiment of the present application, as shown in fig. 2, a document image reproduction identification method is provided, which includes:
step 201, obtaining an original document image obtained by photographing a document, wherein the original document image comprises a background part and a document part to be identified.
In some embodiments, the original document image may be obtained in a variety of ways, such as by direct photographing by an electronic device that performs the document image reproduction identification method, or by obtaining the electronic device from another device, such as a document image acquisition device.
And 202, positioning and cutting the original document image to remove the background part in the original document image, thereby obtaining the document image to be identified.
Note that the document image to be recognized obtained by removing the background portion may be recorded as a focused image.
In some embodiments, the specific process of locating and cropping the image of the underlying document is as shown in FIG. 3:
step 301, positioning the original document image to obtain the center point and the size of the document part to be identified in the original document image.
Specifically, when the original document image is positioned, the center point and the size of the document part in the document image can be predicted based on the target detection network, wherein the size is obtained by the target detection network in a regression mode after the center point is obtained.
The types of the target detection network are various, and the target detection network can be specifically selected according to practical situations, for example, an R-CNN algorithm, a Fast R-CNN algorithm, a Mask R-CNN algorithm, a SSD (Single Shot MultiBox Defender) algorithm and a YOLO (You Only Look Once) algorithm.
Further, the size of the portion of the document to be identified may be, but is not limited to, the width and height of the portion of the document to be identified.
And 302, cutting the original document image according to the center point and the size of the part of the document to be identified so as to remove the background part in the original document image and obtain the document image to be identified.
Because the obtained original document image not only comprises the document part to be identified which needs to be identified, but also comprises the background part of the document placement position when the image is shot, in order to avoid the background part from interfering with the subsequent identification, in the embodiment, the original document image is cut first, the background part is cut off, the document image to be identified of the original document image, namely the focusing image, is obtained, the interference caused by the background part during the identification is avoided, and the accuracy of the identification result can be improved.
Specifically, after the center point, the width and the height of the certificate part to be identified are obtained through the steps, the original certificate image can be cut by taking the center point as the center and taking the width as a transverse cutting target value, and the original certificate image can be cut by taking the height as a longitudinal cutting target value, so that the certificate part to be identified, namely the focusing image, is obtained.
Step 203, performing fourier transform on the certificate image to be identified to obtain a spectrum image of the certificate image to be identified.
The fourier transform can transform the signal from the time domain to the frequency domain, so as to study the spectrum structure and the change rule of the signal.
Fig. 4 is a spectrum image of an original document of a different type according to an embodiment of the present application, and referring to fig. 4, in the spectrum image, a center point of the image represents a zero frequency component, and the represented frequency gradually increases from the center point to corner points around, and a brightness represents an amplitude of the frequency component. It is obvious that the frequency domain characteristics of the normally shot certificate image are mostly concentrated in the low frequency band (near the center point); the paper color printing and the flipping of the certificate images are carried out, and the spectrograms of the paper color printing and flipping of the certificate images are uniformly distributed between low frequency and high frequency due to the irregular graining property of the paper; the document image of the screen flip-flop will have a very high amplitude at the individual medium/high frequency points, due to the moire that the screen will produce. Since the certificate image and the flip image which are normally shot show obvious differences in the spectrum image, the accuracy of identification can be improved when the identification is carried out through the spectrum image.
Step 204, inputting the certificate image and the spectrum image to be identified into a convolutional neural network model.
In some embodiments, the convolutional neural network model input is split into two parts: the certificate image to be identified after certificate positioning cutting, namely a focusing image (RGB three channels) and a frequency spectrum image (F single channel) obtained by performing fast Fourier transformation on the focusing image. The focusing image and the spectrum image are simultaneously input into the convolutional neural network model, and the characteristics of the two images are simultaneously analyzed and identified, so that the identification accuracy is improved.
Step 205, extracting the spatial domain features of the N network levels of the document image to be identified and the frequency domain features of the N network levels of the spectrum image through the N network levels in the convolutional neural network model, fusing the spatial domain features of the N network levels and the frequency domain features of the N network levels to obtain a feature map of the document image to be identified, judging whether the document image to be identified is a flip image according to the feature map, and outputting a judging result.
In some embodiments, features of a document image (focused image) and a frequency domain image to be identified are extracted through a convolutional neural network model, and whether the document image is a flip image is identified, which comprises the following steps:
firstly, carrying out grouping convolution on a certificate image to be identified and a frequency spectrum image by adopting a 1 st network level to obtain the airspace characteristic of the 1 st network level of the certificate image to be identified and the frequency domain characteristic of the 1 st network level of the frequency spectrum image;
secondly, carrying out grouping convolution on the feature map of the ith-1 th network level by adopting the ith network level to obtain the airspace feature of the ith network level of the certificate image to be identified and the frequency domain feature of the ith network level of the frequency spectrum image, wherein the value of i is more than 1 and less than or equal to N;
thirdly, after the airspace feature and the frequency domain feature of the Nth network level are obtained, fusing the airspace feature and the frequency domain feature of the Nth network level to obtain a feature map of the Nth network level, and performing downsampling and full connection on the feature map;
fourthly, judging whether the certificate image to be identified is a reproduction image or not through an activation function according to the fully connected result, and outputting a judging result.
In the process, the spatial domain features of the certificate image (namely the focusing image) to be identified and the frequency domain features of the frequency spectrum image are extracted separately, so that the independence and the effectiveness of the spatial domain features and the frequency domain features are guaranteed, the spatial domain features and the frequency domain features are fused in the last layer of network level, and in the identification process, the spatial domain features and the frequency domain features are combined, so that the identification result is more accurate.
It should be noted that, the algorithm model specifically adopted by the convolutional neural network model is not limited herein. In one embodiment, the convolutional neural network model employs a packet convolutional neural network model. In the present embodiment, N is 4 in the N network levels set in the convolutional neural network model. The spatial domain features of the document image (focused image) to be identified and the frequency domain features of the spectrogram are extracted in 4 network levels respectively, and each layer of features has different feature map numbers on the RGB focused image and the F spectrum image. As shown in fig. 5, 32 spatial features may be extracted for the focused image and 16 frequency domain features may be extracted for the spectral image in a first network hierarchy; 64 spatial features can be extracted for the focused image and 32 frequency domain features can be extracted for the spectrum image in the second layer network hierarchy; 128 spatial features can be extracted for the focused image and 64 frequency domain features can be extracted for the spectral image in the third layer of network hierarchy; 256 spatial features may be extracted for the focused image and 128 frequency domain features may be extracted for the spectral image in the fourth network level. And then, respectively carrying out downsampling and full connection on the feature images in the fourth-layer network level, and obtaining a recognition result through an activation function in the convolutional neural network model.
Wherein, the reproduction image includes: the screen is flipped over and imaged and color printed.
It will be appreciated that, as shown in fig. 6, the convolutional neural network model described above may be obtained by training the following steps, specifically including:
step 601, obtaining a sample image set, wherein the sample image set comprises M sample images and a flip type identifier of each sample image, the flip type identifier is used for indicating whether the sample image is a flip image, and S sample images form a group of sample images;
wherein, the sample image comprises: and the certificate image to be identified and the spectrum image obtained by carrying out fast Fourier transform on the certificate image to be identified.
The following training procedure is performed separately for each group of sample images in the sample image set:
step 602, respectively performing the following processing on each sample image in a group of sample images, inputting the sample images into an initial convolutional neural network model, sequentially adopting N network levels, performing feature extraction on the sample images to obtain N network level features, and integrating the N network level features to obtain a feature map of a certificate in the sample images;
step 603, obtaining a probability value that the certificate image in the group of sample images is a flip image according to the feature image of the certificate in each sample image in the group of sample images;
step 604, calculating a loss function according to the probability value and the identifier of the flip type of the group of sample images, and according to the loss function, reversely propagating the gradient to each layer of the N network layers, and obtaining the next group of sample images from the sample image set after optimizing the parameters of the initial convolutional neural network model.
Steps 602 to 604 are repeatedly performed until the loss function tends to be stable, and the initial convolutional neural network model is taken as a final convolutional neural network model.
It will be appreciated that the above-described tap category identification may also be used to indicate an image tap category of a sample image, e.g., the tap category identification includes: non-flip images, screen flip images and color print flip images. By indicating the image type of the sample image by the type identification, the type of the image (namely, the screen image and the color printing image) can be identified through a convolutional neural network model obtained through training.
Based on the same conception, the embodiment of the present application provides a certificate image reproduction identification device, and the specific implementation of the device may be referred to the description of the embodiment of the method, and the repetition is omitted, as shown in fig. 7, where the device mainly includes:
an acquisition module 701, configured to acquire a document image to be identified;
the image transformation module 702 is configured to transform the document image to be identified, so as to obtain a spectrum image of the document image to be identified;
the document reproduction judging module 703 is configured to extract spatial features of the document image to be identified and frequency domain features of the spectrum image, judge whether the document image to be identified is a reproduction image according to the extracted spatial features and the frequency domain features, obtain a judgment result, and take the judgment result as a document image reproduction identification result.
Optionally, the acquiring module specifically includes:
the acquisition sub-module is used for acquiring an original document image shot by the document, wherein the original document image comprises a background part and a document part to be identified;
and the certificate detection and positioning module is used for positioning and cutting the original certificate image so as to remove the background part in the original certificate image and obtain the certificate image to be identified.
Optionally, the certificate detection positioning module includes:
the positioning module is used for positioning the original document image to obtain the center point and the size of the document part to be identified in the original document image;
and the clipping module is used for clipping the original document image according to the center point and the size of the document part to be identified so as to remove the background part in the original document image and obtain the document image to be identified.
Optionally, the positioning module includes:
the center point positioning module is used for predicting the center point of the certificate part to be identified in the original certificate image based on the target detection network;
and the size positioning module is used for obtaining the size of the certificate part to be identified in a regression mode.
Optionally, the document flap discriminating module includes:
the input module is used for inputting the certificate image to be identified and the spectrum image into a convolutional neural network model;
the judging module is used for extracting the airspace characteristics of the N network levels of the certificate image to be identified and the frequency domain characteristics of the N network levels of the frequency spectrum image through the N network levels in the convolutional neural network model, fusing the airspace characteristics of the N network levels and the frequency domain characteristics of the N network levels to obtain a characteristic diagram of the certificate image to be identified, judging whether the certificate image to be identified is a flip image according to the characteristic diagram, and outputting a judging result.
Optionally, the judging module is specifically configured to perform packet convolution on the document image to be identified and the spectrum image by adopting the 1 st network level, so as to obtain a spatial domain feature of the 1 st network level of the document image to be identified and a frequency domain feature of the 1 st network level of the spectrum image;
carrying out group convolution on the feature map of the ith-1 network level by adopting the ith network level to obtain the airspace feature of the ith network level of the certificate image to be identified and the frequency domain feature of the ith network level of the frequency spectrum image, wherein the value of i is more than 1 and less than or equal to N;
after the airspace feature and the frequency domain feature of the Nth network level are obtained, fusing the airspace feature and the frequency domain feature of the Nth network level to obtain a feature map of the Nth network level, and performing downsampling and full connection on the feature map;
and judging whether the certificate image to be identified is a reproduction image or not through an activation function according to the fully connected result, and outputting a judging result.
Optionally, the flipping image includes: color-printed flip images and screen flip images.
Based on the same concept, an embodiment of the present application provides an electronic device, as shown in fig. 7, where the electronic device mainly includes: a processor 801, a communication interface 802, a memory 803, and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 complete communication with each other through the communication bus 804. The memory 803 stores therein a program executable by the processor 801, and the processor 801 executes the program stored in the memory 803 to realize the following steps: acquiring a certificate image to be identified; transforming the certificate image to be identified to obtain a frequency spectrum image of the certificate image to be identified; and extracting the spatial domain characteristics of the certificate image to be identified and the frequency domain characteristics of the frequency spectrum image, judging whether the certificate image to be identified is a reproduction image or not according to the extracted spatial domain characteristics and the frequency domain characteristics, obtaining a judgment result, and taking the judgment result as a certificate image reproduction identification result.
The communication bus 804 mentioned in the above electronic device may be a peripheral component interconnect standard (Peripheral Component Interconnect, abbreviated to PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated to EISA) bus, or the like. The communication bus 804 may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, only one thick line is shown in fig. 8, but not only one bus or one type of bus.
The communication interface 802 is used for communication between the electronic device and other devices described above.
The memory 803 may include a random access memory (Random Access Memory, abbreviated as RAM) or may include a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor 801.
The processor 801 may be a general-purpose processor including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), a digital signal processor (Digital Signal Processing, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a Field programmable gate array (Field-Programmable Gate Array, FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware components.
In yet another embodiment of the present application, there is also provided a computer-readable storage medium having stored therein a computer program which, when run on a computer, causes the computer to perform the document image reproduction identification method described in the above embodiment.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, by a wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, microwave, etc.) means from one website, computer, server, or data center to another. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape, etc.), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk), etc.
It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing is only a specific embodiment of the application to enable those skilled in the art to understand or practice the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (9)
1. A document image roll-over identification method, comprising:
acquiring a certificate image to be identified;
transforming the certificate image to be identified to obtain a frequency spectrum image of the certificate image to be identified;
extracting the spatial domain characteristics of the certificate image to be identified and the frequency domain characteristics of the frequency spectrum image, judging whether the certificate image to be identified is a reproduction image or not according to the extracted spatial domain characteristics and the frequency domain characteristics, obtaining a judgment result, and taking the judgment result as a certificate image reproduction identification result;
the step of extracting the spatial domain feature of the document image to be identified and the frequency domain feature of the spectrum image, and judging whether the document image to be identified is a flip image or not according to the extracted spatial domain feature and the frequency domain feature to obtain a judgment result, wherein the step of obtaining the judgment result comprises the following steps:
inputting the certificate image to be identified and the spectrum image into a convolutional neural network model;
extracting the airspace characteristics of N network levels of the certificate image to be identified and the frequency domain characteristics of N network levels of the spectrum image through N network levels in the convolutional neural network model, fusing the airspace characteristics of N network levels and the frequency domain characteristics of N network levels to obtain a characteristic diagram of the certificate image to be identified, judging whether the certificate image to be identified is a flip image according to the characteristic diagram, and outputting a judging result.
2. The method for identifying a document image by flipping according to claim 1, wherein the step of acquiring the document image to be identified comprises:
obtaining an original document image obtained by photographing a document, wherein the original document image comprises a background part and a document part to be identified;
and positioning and cutting the original document image to remove the background part in the original document image, thereby obtaining the document image to be identified.
3. The method for document image reproduction identification according to claim 2, wherein the positioning and cropping the original document image to remove the background portion in the original document image to obtain the document image to be identified includes:
positioning the original document image to obtain the center point and the size of a document part to be identified in the original document image;
and cutting the original document image according to the center point and the size of the document part to be identified so as to remove the background part in the original document image and obtain the document image to be identified.
4. A document image reproduction identification method according to claim 3, wherein said positioning the original document image to obtain a center point and a size of a document portion to be identified in the original document image comprises:
and predicting the center point and the size of the certificate part to be identified in the original certificate image based on a target detection network, wherein the size is obtained by the target detection network in a regression mode after the center point is obtained.
5. The method for identifying a document image by a roll-over process according to claim 1, wherein the extracting, through N network levels in the convolutional neural network model, spatial features of the N network levels of the document image to be identified and frequency domain features of the N network levels of the spectrum image, and fusing the spatial features of the N network levels and the frequency domain features of the N network levels to obtain a feature map of the document image to be identified, and outputting a determination result after determining whether the document image to be identified is a roll-over image according to the feature map, includes:
carrying out group convolution on the certificate image to be identified and the frequency spectrum image by adopting the 1 st network level to obtain the airspace characteristic of the 1 st network level of the certificate image to be identified and the frequency domain characteristic of the 1 st network level of the frequency spectrum image;
carrying out group convolution on the feature map of the ith-1 network level by adopting the ith network level to obtain the airspace feature of the ith network level of the certificate image to be identified and the frequency domain feature of the ith network level of the frequency spectrum image, wherein the value of i is more than 1 and less than or equal to N;
after the airspace feature and the frequency domain feature of the Nth network level are obtained, fusing the airspace feature and the frequency domain feature of the Nth network level to obtain a feature map of the Nth network level, and performing downsampling and full connection on the feature map;
and judging whether the certificate image to be identified is a reproduction image or not through an activation function according to the fully connected result, and outputting a judging result.
6. The document image reproduction identification method according to claim 1, wherein the reproduction image includes: color-printed flip images and screen flip images.
7. A document image reproduction identification apparatus, comprising:
the acquisition module is used for acquiring a certificate image to be identified;
the image conversion module is used for converting the certificate image to be identified to obtain a frequency spectrum image of the certificate image to be identified;
the certificate copying judging module is used for extracting the airspace characteristics of the certificate image to be identified and the frequency domain characteristics of the frequency spectrum image, judging whether the certificate image to be identified is a copying image according to the extracted airspace characteristics and the frequency domain characteristics, obtaining a judging result, and taking the judging result as a certificate image copying identification result;
the step of extracting the spatial domain feature of the document image to be identified and the frequency domain feature of the spectrum image, and judging whether the document image to be identified is a flip image or not according to the extracted spatial domain feature and the frequency domain feature to obtain a judgment result, wherein the step of obtaining the judgment result comprises the following steps:
inputting the certificate image to be identified and the spectrum image into a convolutional neural network model;
extracting the airspace characteristics of N network levels of the certificate image to be identified and the frequency domain characteristics of N network levels of the spectrum image through N network levels in the convolutional neural network model, fusing the airspace characteristics of N network levels and the frequency domain characteristics of N network levels to obtain a characteristic diagram of the certificate image to be identified, judging whether the certificate image to be identified is a flip image according to the characteristic diagram, and outputting a judging result.
8. An electronic device, comprising: the device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
the memory is used for storing a computer program;
the processor is configured to execute a program stored in the memory to implement the document image reproduction identification method according to any one of claims 1 to 6.
9. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the document image reproduction identification method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010597003.3A CN111767828B (en) | 2020-06-28 | 2020-06-28 | Certificate image reproduction identification method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010597003.3A CN111767828B (en) | 2020-06-28 | 2020-06-28 | Certificate image reproduction identification method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111767828A CN111767828A (en) | 2020-10-13 |
| CN111767828B true CN111767828B (en) | 2023-12-08 |
Family
ID=72722452
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010597003.3A Active CN111767828B (en) | 2020-06-28 | 2020-06-28 | Certificate image reproduction identification method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111767828B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112927221B (en) * | 2020-12-09 | 2022-03-29 | 广州市玄武无线科技股份有限公司 | Image fine-grained feature-based reproduction detection method and system |
| CN112580621B (en) * | 2020-12-24 | 2022-04-29 | 成都新希望金融信息有限公司 | Identity card copying and identifying method and device, electronic equipment and storage medium |
| CN112767337B (en) * | 2021-01-12 | 2023-08-04 | 杭州海康威视数字技术股份有限公司 | Cigarette case detection method and device |
| CN114913565B (en) * | 2021-01-28 | 2023-11-17 | 腾讯科技（深圳）有限公司 | Face image detection method, model training method, device and storage medium |
| CN113033530B (en) * | 2021-05-31 | 2022-02-22 | 成都新希望金融信息有限公司 | Certificate copying detection method and device, electronic equipment and readable storage medium |
| CN113837310B (en) * | 2021-09-30 | 2023-05-23 | 四川新网银行股份有限公司 | Multi-scale fused certificate flap recognition method and device, electronic equipment and medium |
| CN114819142B (en) * | 2022-04-18 | 2024-09-06 | 支付宝(杭州)信息技术有限公司 | Screen shooting image recognition and training method and device for models thereof and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105118048A (en) * | 2015-07-17 | 2015-12-02 | 北京旷视科技有限公司 | Method and device for identifying copying certificate image |
| CN107909015A (en) * | 2017-10-27 | 2018-04-13 | 广东省智能制造研究所 | Hyperspectral image classification method based on convolutional neural networks and empty spectrum information fusion |
| CN108229499A (en) * | 2017-10-30 | 2018-06-29 | 北京市商汤科技开发有限公司 | Certificate recognition methods and device, electronic equipment and storage medium |
| CN109886275A (en) * | 2019-01-16 | 2019-06-14 | 深圳壹账通智能科技有限公司 | Reproduction image-recognizing method, device, computer equipment and storage medium |
| WO2019233341A1 (en) * | 2018-06-08 | 2019-12-12 | Oppo广东移动通信有限公司 | Image processing method and apparatus, computer readable storage medium, and computer device |
| EP3608918A1 (en) * | 2018-08-08 | 2020-02-12 | Tata Consultancy Services Limited | Parallel implementation of deep neural networks for classifying heart sound signals |
| CN111259915A (en) * | 2020-01-20 | 2020-06-09 | 中国平安人寿保险股份有限公司 | Method, device, equipment and medium for recognizing copied image |
| CN111275685A (en) * | 2020-01-20 | 2020-06-12 | 中国平安人寿保险股份有限公司 | Method, device, equipment and medium for identifying copied image of identity document |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100342399C (en) * | 2002-10-15 | 2007-10-10 | 三星电子株式会社 | Method and apparatus for extracting feature vector used for face recognition and retrieval |
-
2020
- 2020-06-28 CN CN202010597003.3A patent/CN111767828B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105118048A (en) * | 2015-07-17 | 2015-12-02 | 北京旷视科技有限公司 | Method and device for identifying copying certificate image |
| CN107909015A (en) * | 2017-10-27 | 2018-04-13 | 广东省智能制造研究所 | Hyperspectral image classification method based on convolutional neural networks and empty spectrum information fusion |
| CN108229499A (en) * | 2017-10-30 | 2018-06-29 | 北京市商汤科技开发有限公司 | Certificate recognition methods and device, electronic equipment and storage medium |
| WO2019233341A1 (en) * | 2018-06-08 | 2019-12-12 | Oppo广东移动通信有限公司 | Image processing method and apparatus, computer readable storage medium, and computer device |
| EP3608918A1 (en) * | 2018-08-08 | 2020-02-12 | Tata Consultancy Services Limited | Parallel implementation of deep neural networks for classifying heart sound signals |
| CN109886275A (en) * | 2019-01-16 | 2019-06-14 | 深圳壹账通智能科技有限公司 | Reproduction image-recognizing method, device, computer equipment and storage medium |
| CN111259915A (en) * | 2020-01-20 | 2020-06-09 | 中国平安人寿保险股份有限公司 | Method, device, equipment and medium for recognizing copied image |
| CN111275685A (en) * | 2020-01-20 | 2020-06-12 | 中国平安人寿保险股份有限公司 | Method, device, equipment and medium for identifying copied image of identity document |
Non-Patent Citations (1)
| Title |
|---|
| 基于空域与频域稳定特征融合的离焦虹膜识别;苑玮琦;刘博;;仪器仪表学报(第10期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111767828A (en) | 2020-10-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111767828B (en) | Certificate image reproduction identification method and device, electronic equipment and storage medium | |
| CN109325954B (en) | Image segmentation method and device and electronic equipment | |
| CN108229526B (en) | Network training method, network training device, image processing method, image processing device, storage medium and electronic equipment | |
| WO2020098250A1 (en) | Character recognition method, server, and computer readable storage medium | |
| US9521391B2 (en) | Settings of a digital camera for depth map refinement | |
| US11244157B2 (en) | Image detection method, apparatus, device and storage medium | |
| WO2019051941A1 (en) | Method, apparatus and device for identifying vehicle type, and computer-readable storage medium | |
| CN112602088B (en) | Method, system and computer readable medium for improving quality of low light images | |
| CN111476863B (en) | Method and device for coloring black-and-white cartoon, electronic equipment and storage medium | |
| CN108846814B (en) | Image processing method, image processing device, readable storage medium and computer equipment | |
| JP7212554B2 (en) | Information processing method, information processing device, and program | |
| CN110032931B (en) | Method and device for generating countermeasure network training and removing reticulation and electronic equipment | |
| CN110796000A (en) | Lip sample generation method and device based on bidirectional LSTM and storage medium | |
| CN113158773A (en) | Training method and training device for living body detection model | |
| CN115273123A (en) | Bill identification method, device and equipment and computer storage medium | |
| CN111597937B (en) | Fish gesture recognition method, device, equipment and storage medium | |
| CN112700376B (en) | Moire pattern removing method and device for image, terminal equipment and storage medium | |
| CN114650361A (en) | Shooting mode determining method and device, electronic equipment and storage medium | |
| CN112434717B (en) | Model training method and device | |
| CN110288560A (en) | A kind of image fuzzy detection method and device | |
| US20240127406A1 (en) | Image quality adjustment method and apparatus, device, and medium | |
| CN110335626A (en) | Age recognition methods and device, storage medium based on audio | |
| CN111340140A (en) | Image data set acquisition method and device, electronic equipment and storage medium | |
| CN116258906A (en) | Object recognition method, training method and device of feature extraction model | |
| CN110852242A (en) | Watermark identification method, device, equipment and storage medium based on multi-scale network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant after: Jingdong Technology Holding Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant before: Jingdong Digital Technology Holding Co.,Ltd. Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant after: Jingdong Digital Technology Holding Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant before: JINGDONG DIGITAL TECHNOLOGY HOLDINGS Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |