JP2006093880A - Image processing apparatus and control method thereof, computer program, and computer-readable storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate a data file, having a high compression ratio by maintaining the edge to be clear regardless of characters and handwritten images in a manuscript, and to maintaining the gradation properties of a natural image, when there is one. <P>SOLUTION: A region identifier 104 identifies a character region constituted of printing types in inputted image data from a non-character region, and generates each coordinates data. A handwritten region determinating section determines a handwritten region, based on information from a pixel determining section 109 for outputting character line drawing information and non-character region coordinate data. A painted-out section 106 paints out a character section, in a region shown by character region coordinates data and handwritten region coordinate data with a background color, and generates image data in a state without high-frequency components that include natural image as a whole. Then, the result is encoded by a JPEG encoder 108. The character region and the handwritten region are encoded by MMR encoders 112, 115 that maintain the edges of characters. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a technique for inputting, encoding, and outputting image data.

  In recent years, the digitization of documents has progressed with the spread of scanners. When a single A4 size document is scanned at 300 dpi resolution and full color, the image data will actually be 24 Mbytes in size, and memory will be strained for storage management, and mail etc. will be sent to other devices via the network. In case of transferring with, it becomes a problem of network traffic. Therefore, a compression technique for image data is important.

  For example, JPEG is known as full color image compression. JPEG is known to be very effective for compressing natural images such as photographs and to have good image quality. The use of such a compression technique alleviates the above-described data amount problem.

  However, on the other hand, JPEG causes image deterioration called mosquito noise for an image having a high frequency portion such as a character portion in a document, and the compression rate is also bad.

Therefore, a technique is known in which area division is performed, JPEG compression is performed on the background portion obtained by removing character components from an image in which a photographic area and character area are mixed, and the extracted character components are subjected to MMR compression as components indicating color information and character positions (Patent Documents 1 and 2).
JP 2002-77633 A JP 2003-309727 A

  In the above compression method, a printed matter created by a computer and printed by a printer is targeted for reading. Accordingly, when a character in the document is corrected by handwriting, the handwritten part is treated as a non-character and processed as a JPEG compression target. As described above, JPEG is a compression technique that is effective for natural images. However, since characters and line drawings are prone to image quality degradation, characters with a handwritten image superimposed on the surrounding characters ( In other words, the visual image quality may be inferior to that of a character on which a handwritten image is not superimposed.

  The present invention has been made in view of such problems, and when encoding an image in which characters, handwritten images, and natural images in a document are mixed, while maintaining the edges of the characters and handwritten images clearly, In the case where there is a natural image, it is intended to provide a technique for maintaining the gradation of the natural image and generating a data file with a high compression rate.

In order to solve this problem, for example, an image processing apparatus of the present invention has the following configuration. That is,
A character region determination means for determining a character region in the input image and generating character region coordinate information and non-character region coordinate information;
Handwritten region determination means for inputting attribute information indicating whether or not the target pixel is in a character / line drawing, determining a handwritten region based on the attribute information and the non-character region coordinate information, and generating handwritten region coordinate information When,
Based on the character area coordinate information and the handwritten area coordinate information, a painting means for painting a character and a handwritten part in the input image with surrounding colors;
First encoding means for encoding the entire image obtained by filling with the filling means with encoding means for a gradation image;
A second encoding means for encoding the character and the handwritten portion with an encoding means for character line drawing;
And output means for outputting each encoded data obtained by the first and second encoding means in one data file.

  According to the present invention, regardless of the character or handwritten image in the document, the edge is maintained clearly, and when there is a natural image, the gradation of the natural image is maintained and data with a high compression rate is obtained. A file can be generated.

  Embodiments according to the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a system configuration diagram in the embodiment. In the figure, reference numeral 100 denotes a multifunction peripheral (MFP) that functions as a network scanner, a network printer, and a copier. Reference numeral 200 denotes a file server for storing files such as document image data read by the multifunction peripheral 100, 301 and 302 denote client PCs such as personal computers, and 400 denotes a network for connecting these devices.

  In the above configuration, the client PC uses the multifunction peripheral 100 as a network printer, and stores a file shared by each client on the network in the file server 200.

  The multifunction device 100 according to the present embodiment functions as a network printer and a copying machine as described above, but these functions are not directly related to the present invention. The scanner function will be described.

  FIG. 2 shows an example of a document to be read. As shown in the figure, an example is shown in which a natural image and characters (sentences) are included, and a part of the sentence is corrected by handwriting. The text is not limited to black, and may be mixed with color characters such as red and blue that can be printed by a printer. Further, the drawing shows that the character string “new development technology” is handwritten and corrected with “new development technology” and a red pen. In this case, the user intends to correct the original character and is expressed as correction, but it may be a simple signature or marking. As long as the handwritten image is superimposed (written) in this way, it does not depart from the intention described in the present embodiment.

  Now, this document is read by the multi-function device 100 (256 gradations of 8 bits for each of R, G, and B) and compression-encoded, and whether or not it is a character is determined by a known character recognition process. Adopt the method used. That is, a horizontal and vertical dot histogram of the character image dots is created, the circumscribed rectangle of the character is detected, the characters arranged in the direction of the detected circumscribed rectangle are connected to each other, and the line spacing Is a method of determining a rectangular character region by connecting them as character lines if they are within a predetermined distance.

  In this case, if “newly developed technology” can be entered in the same order as the type, the character string is recognized as a character, but here it is assumed that it was not recognized as a character here (of course, part of the characters are determined as characters). May be.) In addition, the characters “new development technology” in the text are not judged as characters because handwritten line segments have been entered, and the characters that have been drawn through are also judged as characters. Suppose not.

  Now, the area determined as the character area in the above state is as shown in FIG. Since the character area is handled as a rectangle, six character areas are determined as shown in the figure. Note that. The character area is treated as a rectangle because it is convenient to manage the rectangular area with two coordinates of the upper left corner and the lower right corner on the diagonal line, but it is not limited to a rectangle (= tetragon). You may allow management with the above polygons.

  Now, when encoding the character area shown in FIG. 3, it is generally desired that the character image has a clear edge. Therefore, in the embodiment, the character region is encoded by applying lossless encoding such as MMR (other encoding techniques such as JBIG may be adopted). Since MMR is encoding of a binary image, the character region shown in FIG. 3 is simply binarized using an intermediate value (or 128 for 8 bits) of density (or luminance) and encoded. However, it is necessary to consider the color of each character in the character area. Therefore, when performing MMR encoding, color information indicating how many colors the pixels that become “1” when binarized are in the original image data is held separately. Although the color information may have values of R, G, and B, the number of characters cannot be as large as that of a natural image and is therefore assigned to a palette and stored. That is, the color distribution of the original image (multi-valued color image) at the position where it is binarized to “1” is examined to determine how many colors exist, and a palette number is assigned to each. For example, if the character color is black (R≈G≈B≈0), the palette number “0” is assigned. If the character color is red (for example, R ≧ 200 or more, G≈B≈0), the palette number “1” is assigned. Since adjacent characters are likely to have the same palette, if the palette number is also run-length encoded, it is possible to perform encoding that retains color information at a high compression rate. Of course, when decoding, a table indicating the relationship between the palette number and the actual RGB value is prepared separately in order to reproduce even the character color. That is, the pixel data of the significant pixel “1” obtained when the binary image is decoded is restored to the multi-value pixel data indicated by the palette number. A character may be accompanied by its background color. This background color will be described later.

  Next, encoding other than the character area will be described. Here, if correction (superimposition or writing) by the color pen is not taken into consideration, the area other than the character area is a natural image area. JPEG is known as an encoding technique excellent for natural images.

  If it is assumed that the outside of the character area is JPEG-encoded, the image of FIG. 4 including the handwritten corrected portion of FIG. 2 becomes the target for JPEG encoding. JPEG is an irreversible encoding and is excellent in gradation images, but if a character image is JPEG encoded, its edges will be blurred, so even if it is handwritten, its visibility is impaired. I will. Furthermore, since the type characters, line segments, and handwritten characters corrected by such handwriting contain many high-frequency components, a high compression rate cannot be obtained.

  Therefore, in the present embodiment, as shown in FIG. 5, an area that is not determined to be a character by handwriting (hereinafter referred to as a handwriting area) is extracted from other than the character area. To make it easier. The handwriting area is encoded as follows.

  First, the number of colors existing in the area determined to be a non-character area tends to be the same as or less than the number of colors in the character area. The reason is that the number of correction targets by handwriting will be at most several, and the pen ink colors used for correction are actually about one or two. Therefore, first, the color in the handwritten area is reduced. In the embodiment, each of R, G, and B is represented by 8 bits (256 gradations), so R, G, and B are represented (reduced color) by 2 bits. Simply, it is easy to understand if only the most significant 2 bits of each component are used.

Then, the color distribution of the image after color reduction is examined, and the number of colors is calculated. Since each of R, G, and B is 2 bits, the color is expressed by 2 + 2 + 2 = 6 bits, so that the maximum is expressed by 2 ⁶ = 64 colors. This is a sufficient number to represent the handwritten corrected character and the color of the pen ink used.

If it is determined by the color distribution that there are N colors, binary planes of C ₀ , C ₁ ,..., C _N-1 are generated, and MMR encoding is performed for each. A palette is assigned to associate each plane with a color.

  Next, JPEG encoding will be described.

  The JPEG encoding target in the embodiment is not only the natural image region shown in FIG. 2 but also the entire one page (one image). That is, the image is the same size as the originally input image. The reason for this is to consider the background color of characters or handwritten characters or handwritten line segments that existed in the character region or handwritten character region. When creating and printing the manuscript (excluding handwritten parts) shown in Fig. 2 with an application that runs on a computer, it is common to set the background color of characters, so include that background color as well. This is to make the original image faithful. It can also be said that such a background color is not an object to be encoded in the encoding of the character area and non-character area already described.

  The problem here is that JPEG has a low compression rate when it contains high-frequency components such as characters. Therefore, as shown in FIG. 7, when focusing on one character in the character area and the handwritten area, significant pixels (any color) 500 constituting the character are extracted from the original image, and the inside is extracted. By painting with the background color (501), only the background color or the background color is set. This process is performed for significant pixels in all character areas and handwritten areas. This result is shown in FIG. As shown in the drawing, the character area other than the natural image area or the handwritten area is made to have a substantially uniform luminance distribution, and then the entire page is subjected to JPEG encoding. Since encoding is performed with no high-frequency components in one page, the amount of encoded data per page does not become extremely large compared to the case where only a natural image area is JPEG encoded.

  Although the encoding process in the embodiment has been described above, it can be summarized as shown in FIG. FIG. 5 is a conceptual diagram of processing in the embodiment.

  The read image 500 includes a character area 501, a handwriting area 502, and a natural image area 503.

  Among these, for the character area 501, a palette is generated by detecting the color of each character included in the character area (process 504). Then, the character image area is binarized, and MMR encoding is performed (process 505) to generate character area encoded data 506. The character area encoded data is composed of the coordinate data of the character area, the palette, and the MMR encoded data.

  For the handwritten area 502, a color reduction process is performed to generate a palette (process 507). Then, MMR encoding is performed for each binary plane indicated by each palette (processing 508), and handwritten region encoded data 509 is generated. The handwritten region encoded data 509 is composed of the coordinates of each handwritten region, the palette, and MMR encoded data for each plane.

  On the other hand, for an entire page including a natural image, significant pixels (pixels that are not background pixels) in the character area and handwriting area are extracted, and the interior is filled with a nearby background color, and the character area corresponding to the character area is extracted. A page image 500 ′ including a background-only image 501 ′, a background-only image 502 ′ in a handwritten region, and a natural image region 503 is generated.

  Then, the page image 500 ′ is subjected to JPEG encoding to generate background & natural image region encoded data 511. Since the structure of JPEG encoded data may be as usual, it will not be described.

  In this way, the character area encoded data 506, the handwritten area encoded data 509, and the background & natural image area encoded data 511 are included, and one file 512 having a predetermined format is generated, and the file 512 is stored in the file server 200. Will be transferred.

  A specific configuration for realizing the processing described above is shown in FIG. 9, and the processing content will be described below.

  The original image 101 is image data read by an image scanner, and each pixel is data represented by 8 bits for each of RGB. The image binarization unit 102 extracts only the luminance component from the document image 101 (for example, an L component obtained by RGB → Lab conversion may be used), and sets the threshold value obtained according to the luminance distribution. Based on the binarization, binary image data 103 is output. Then, the area identifying unit 104 determines the character area and the non-character area by detecting the distribution of the circumscribed rectangle, combining the circumscribed rectangles, and the like from the obtained binary image data 103, as in normal character recognition. The character area coordinate data 117 is generated for the character area, and the non-character area coordinate data 105 is generated for the non-character area.

  The point to be noted here is that the non-character area coordinate data 105 generated by the area identification unit 104 does not identify the handwritten area and the natural image area, but is in a state where these coordinates are mixed.

  The fill unit 106 is determined to be a character region based on the character region coordinate data, and extracts (removes) the “1” pixel (significant pixel) of the binary image data 103 from the original image 101. A process of filling the extracted interior with a color of “0” in the vicinity of the binary image data is performed. Further, from the handwritten region coordinate data 119 described later, a pixel of the original image that is determined to be a handwritten region and is “1” pixel (significant pixel) of the binary image data 103 is extracted, and the extracted inside is expressed in binary. A process of painting with a color of “0” in the vicinity of the image data is performed.

  As a result, the data output from the filling unit 106 is multi-valued image data as shown in FIG.

  The character color extracting unit 113 determines that pixels in the region indicated by the character region coordinate data 117 and having “1” (significant pixel) in the secondary image data are pixels constituting the character, and color image data The extraction of the color information of the pixel at the corresponding position in the inside is repeated. As a result, the colors of all characters in the character area are detected, and the result is output as a palette. Then, binary image plane data for each color is generated and encoded by the MMR encoding unit 115. Then, the pallet, MMR encoded data, and information indicating the relationship between each plane and the pallet are output as encoded data 121. That is, the character color extraction unit 113 and the MMR encoding unit 115 encode the character region shown in FIG.

  In addition, the pixel determination unit 109 detects a pixel in which the change in the luminance component of the input color image data is steep (the difference between the luminance components of adjacent pixels is equal to or greater than a predetermined threshold value), and the result is the handwritten region determination unit To 110. This determination result signal is also a signal indicating whether or not the pixel is in the character / line drawing area.

  The handwritten region determination unit 110 can determine whether the pixel of interest is a character / line drawing or other than that based on the determination result of the pixel determination unit 109. Therefore, if only pixel data that matches the non-character area coordinate data 105 is output at the pixel “1” in the binary image data 103, the output pixel data constitutes character or line segment data corrected by handwriting. Since it becomes pixel data, the coordinate data is output as the coordinate data 119 of the handwritten region.

The color reduction & palette generation unit 111 reduces the color of pixel data in the handwritten area from the handwritten area 110 as described above (in the embodiment, R, G, and B are each converted to 2 bits), and the color after the color reduction is obtained. Count and determine the distribution (histogram) for each color. Then, it is divided at the valley positions of the histogram, and one pallet number is assigned to the color information at the center position between the divisions. As a result, for example, N pallets C ₀ , C ₁ ,... C _N-1 (N <64) are generated. Then, the MMR encoding unit 112 performs MMR encoding on the color pixels of the palette C ₀ as a binary image. Pallet C ₁ this process, ... similarly performed for C _N-1. As a result, each MMR encoded data is associated with the palette number, and the palette number and the color information after color reduction are output as palette & MMR encoded data 120.

  As a result, the data 117 to 121 are generated in each processing unit, and are output as a single file 130 that is synthesized into a predetermined format.

  Here, details of the color reduction & palette generation unit 111 and the MMR encoding unit 112 in the embodiment are shown in FIG.

  Character reduction / line drawing attribute information 1001, original image data 1002, and non-character area coordinate data 105 determined by the pixel determination unit 109 are supplied to the color reduction unit 1003.

  When the target pixel has the character / line drawing attribute and exists in the non-character area coordinates, the color reduction unit 1003 calculates R, G, B (each 8 bits) of the target pixel position in the input original image. Each color is reduced to 2 bits (6 bits in total), and the data after the color reduction is output to the color information counting unit 1004 and to the binarization unit 1008.

The color information counting unit 1004 has a built-in counter that counts the color represented by a total of 6 bits, 2 bits each for R, G, and B. When each counter is represented by CT (R (2 bits), G (2 bits), B (2 bits)) and the color of the target pixel after color reduction (Rr, Gr, Br),
CT (Rr, Gr, Br) ← CT (Rr, Gr, Br) +1
Will be calculated.

  If this process is performed until image input for one page is completed, the counter CT () prints characters, handwritten line segments, and handwritten characters that are not recognized as characters by handwriting (accurately, characters are recognized as characters). The frequency after the subtractive color reduction of the significant pixels constituting the handwritten character that did not exist) is stored.

  The sorting unit 1005 sorts the counter values obtained in the above manner in the three-dimensional color space of R, G, B from the ones with the highest frequency. Then, the palette table generation unit 1006 assigns “0”, “1,”,... As the palette numbers in the order of frequency for each color, and generates a palette table 1007 indicating the relationship between the palette number and the color. In the embodiment, since the subtractive color is 2 bits for each RGB, the palette number is “63” at the maximum, but the palette number is not assigned to the color with the frequency number 0.

  The binarization unit 1008 binarizes based on the color information of each pixel after color reduction from the color reduction unit 1003 and information from the palette table generation unit 1006. Specifically, when (1, 0, 0) is set as the color (R, G, B) of the palette number “0”, the output data from the color reduction unit 1003 is a pixel of (1, 0, 0). Only, that is, only pixels having a color matching the palette “0” are output as “1” (1 bit), and the others are output as “0”.

  By performing this process for all the generated pallet numbers, a binary image data group 1009 for each pallet number is generated.

  The MMR encoding unit 112 generates encoded data 1010 by performing MMR encoding on each binary image of the binary image data group 1009.

  As described above, according to this embodiment, when a document created on a computer and printed by a printer is subjected to handwriting correction (superimposition or writing) and the correction result is digitized, the printer originally For printed type characters and comparable characters, and parts that were corrected by handwriting and not recognized as characters, and handwritten line segments and handwritten characters are subject to natural image encoding (JPEG in the embodiment). However, since it is encoded by lossless encoding (MMR in the embodiment), handwritten characters and line segments, and furthermore, edges of type characters that are not recognized as characters by handwriting are retained. Good encoded image data can be obtained. In addition, the color of characters can be reproduced with the number of colors that almost cover the colors used normally, and even if it is not 100% faithful to the original image, it can be reproduced with colors close to it. Become.

<Second Embodiment>
FIG. 11 shows a configuration in place of FIG. 9 in the first embodiment. FIG. 11 differs from FIG. 9 in that a reduction unit 107 is disposed immediately before the JPEG encoding unit 108.

  The target for JPEG encoding is information including a natural image area and an area in which characters are extracted and painted with a background color, and does not include characters or line segments. In other words, this is an image in which gradation is prioritized over resolution. Therefore, if the reading resolution of the image scanner is sufficient, it can be said that even if the resolution is lowered, the influence on the image quality of the entire page is small. Therefore, immediately before encoding by the JPEG encoding unit 108, a reduction unit for reducing the image after filling is provided. As a result, the amount of JPEG encoded data can be significantly reduced. Of course, it is necessary to cope with the decoding process by storing information indicating the reduction ratio in the header (not shown) of the output file 130.

<Third Embodiment>
In some cases, the multifunction peripheral already includes a configuration corresponding to the pixel determination unit 109 described in the first embodiment in the document reading unit. In that case, it is only necessary to input the determination result, which can be realized with the configuration shown in FIG. The difference between FIG. 12 and FIG. 9 is that the attribute information 125, which is a corresponding determination result, is input instead of the pixel determination unit 109 being eliminated. Further, the configuration shown in the second embodiment may be applied to the configuration of the third embodiment.

  In the embodiment according to the present invention, the example applied to the multifunction machine has been described. However, on the computer connected to the image input device such as an image scanner, a program for executing processing substantially equivalent to the processing shown in FIG. 8 is executed. But it is clear that it can be realized. Further, since the computer program is usually stored in a computer-readable storage medium such as a CD-ROM and can be executed by setting it in the computer and copying or installing it in the system, of course, Such computer-readable storage media are also included in the scope of the present invention.

1 is an overall system configuration diagram according to an embodiment. 5 is a diagram illustrating an example of a document image to be read in the embodiment. FIG. It is a figure which shows the character area in a document image. It is a figure which shows the handwritten area | region and natural image area | region in a document image. It is a figure which shows the handwritten area | region in embodiment. It is a figure which shows the data format of the whole page which carries out JPEG encoding. It is a figure which shows the processing content of the filling part. It is a figure which shows the flow of a process of the image data in embodiment. It is a block block diagram regarding the digitization of image data in the first embodiment. It is a figure which shows the detailed structure of the color reduction & palette production | generation part in a 1st embodiment, and an MMR encoding part. It is a block block diagram regarding the digitization of the image data in 2nd Embodiment. It is a block block diagram regarding digitization of image data in the third embodiment.

Claims (8)

A character region determination means for determining a character region in the input image and generating character region coordinate information and non-character region coordinate information;
Handwritten region determination means for inputting attribute information indicating whether or not the target pixel is in a character / line drawing, determining a handwritten region based on the attribute information and the non-character region coordinate information, and generating handwritten region coordinate information When,
Based on the character area coordinate information and the handwritten area coordinate information, a painting means for painting a character and a handwritten part in the input image with surrounding colors;
First encoding means for encoding the entire image obtained by filling with the filling means with encoding means for a gradation image;
A second encoding means for encoding the character and the handwritten portion with an encoding means for character line drawing;
An image processing apparatus comprising: output means for including each encoded data obtained by the first and second encoding means in one data file and outputting the data. The second encoding means includes
Means for generating a binary image corresponding to each color and generating a palette indicating the color of the binary image;
Lossless encoding means for losslessly encoding the generated binary image,
The image processing apparatus according to claim 1, wherein the table indicating the relationship between the generated palette and color and the encoded data obtained by the lossless encoding unit are output as an encoding result.   The image processing apparatus according to claim 1, wherein the second encoding unit includes a color reduction unit, and encodes image data after color reduction by the color reduction unit.   2. The first encoding unit includes a reduction unit that reduces an entire image obtained by painting by the painting unit to a predetermined size, and encodes the image reduced by the reduction unit. An image processing apparatus according to 1.   2. The image according to claim 1, wherein the attribute determination unit outputs attribute information that is a character line image when a luminance difference or density difference between adjacent pixels of the input image is larger than a predetermined threshold. Processing equipment. A character region determination step for determining a character region in the input image and generating character region coordinate information as well as non-character region coordinate information;
A handwritten region determination step of inputting attribute information indicating whether or not the target pixel is in a character / line drawing, determining a handwritten region based on the attribute information and the non-character region coordinate information, and generating handwritten region coordinate information When,
Based on the character region coordinate information and the handwritten region coordinate information, a painting step of painting a character and a handwritten part in the input image with surrounding colors;
A first encoding step of encoding the entire image obtained by the filling step with an encoding means for a gradation image;
A second encoding step of encoding the character and the handwritten portion with an encoding means for character line drawing;
An image processing method comprising: an output step of outputting each encoded data obtained by the first and second encoding means in one data file. A computer program that is read and executed by a computer,
A character region determination means for determining a character region in the input image and generating character region coordinate information and non-character region coordinate information;
Handwritten region determination means for inputting attribute information indicating whether or not the target pixel is in a character / line drawing, determining a handwritten region based on the attribute information and the non-character region coordinate information, and generating handwritten region coordinate information When,
Based on the character area coordinate information and the handwritten area coordinate information, a painting means for painting a character and a handwritten part in the input image with surrounding colors;
First encoding means for encoding the entire image obtained by filling in the filling step with an encoding means for a gradation image;
A second encoding means for encoding the character and the handwritten portion with an encoding means for character line drawing;
A computer program that functions as output means for outputting each encoded data obtained by the first and second encoding means in one data file.   A computer-readable storage medium storing the computer program according to claim 7.

Images