JP2003091723A - Image processor, image processing method and medium with image processing control program recorded thereon - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate the arithmetic operation at image processing time when executing image processing by extracting the feature quantity of picture elements for constituting an actual photographing image. SOLUTION: A computer 21 receives image data with respective blocks of the actual photographing image divided into lateral 3 blocks and longitudinal 5 blocks, totalizes a histogram of luminance with respective blocks, and reevaluates the totalized luminance by using weighting imparted with respective blocks. For example, the luminance can be reevaluated by putting emphasis on the blocks of a central part of the image. The adjusting degree of contrast and lightness is determined on the basis of the luminance distribution after such reevaluation.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, an image processing method, and a medium in which an image processing control program is recorded, which automatically executes optimum image processing on real image data such as a digital photographic image. .

[0002]

2. Description of the Related Art Various types of image processing are performed on digital image data. For example, image processing such as expanding the contrast, correcting the color tone, or correcting the brightness. Usually, these image processes can be executed by a microcomputer, and the operator confirms the image on the monitor and selects the necessary image process, and determines the image process parameters and the like.

[0003]

In recent years, various types of image processing techniques have been proposed and are actually exerting their effects. However, when it comes to the extent of processing, humans must still be involved. This is because it was not possible to determine what is important in the digital image data to be image-processed.

For example, when considering the image processing for correcting the brightness, it is assumed that the automatic processing is performed so that the image is corrected to be bright if the average of the entire screen is dark and is corrected to be dark if the average of the entire screen is bright. Here, it is assumed that there is real image data of a person image taken at night. Although the background is almost black, it is assumed that the person was able to take good pictures. If this real shot image data is automatically corrected, the average of the entire image will be dark because the background is pitch dark, and the image will be corrected to be bright, resulting in a daytime image.

In this case, the fact that the background is dark if a human is involved is taken into consideration without giving much weight, and only the portion of the human figure is focused on. Then, if the person image is dark, it is corrected to be bright, and conversely, if the effect of flash or the like is too bright, the correction is made to be dark.

As described above, the conventional image processing cannot judge the importance according to each part in the photographed image data, so that there is a problem that a human must be involved.

On the other hand, even if the importance of the image can be judged by some method, since the judgment is made in pixel units, changing the importance in real time to proceed with the work leads to an increase in the amount of calculation. I will leave.

The present invention has been made in view of the above problems, and it is possible to automatically execute optimum image processing in consideration of importance in actual image data such as a digital photographic image. An image processing apparatus, an image processing method, and a medium in which an image processing control program is recorded are provided.

[0009]

In order to achieve the above-mentioned object, the invention according to claim 1 is an image processing apparatus for inputting real photographed image data composed of pixels in a dot matrix form and performing predetermined image processing. , A feature quantity equality extraction means for uniformly extracting the feature quantity of each pixel necessary for determining the image processing intensity over the entire screen, and a feature for re-evaluating the feature quantity extracted by the feature quantity equality extraction means by predetermined weighting It is configured to include an amount weighting re-evaluation means and a processing means for performing image processing by determining the image processing intensity based on the re-evaluated feature amount.

In the invention according to claim 1 configured as described above, the actually photographed image data is composed of pixels in a dot matrix form, and the feature quantity uniform extraction means is the feature quantity of each pixel necessary for judging the image processing intensity. Are extracted evenly over the entire screen. Then, the feature amount weighting re-evaluation means re-evaluates the feature amount extracted by the feature amount equalizing extraction means by predetermined weighting. And the processing means is
In this way, the image processing intensity is determined based on the feature amount obtained by the re-evaluation and image processing is performed.

That is, in the extraction stage, the whole screen is uniformly distributed, and a predetermined weighting is performed after the extraction.
The resulting features will differ from those obtained uniformly over the entire image.

The feature amount uniform extraction means extracts the feature amount of each pixel necessary for determining the image processing strength, and extracts it evenly over the entire screen. In this case, all pixels of the entire screen may be extracted,
Further, even if not all pixels are extracted, it may be extracted uniformly. As an example of the latter, the invention according to claim 2 is the image processing apparatus according to claim 1, wherein the feature amount equalizing extraction unit selects pixels that are thinned out evenly by a predetermined reference with respect to all pixels. The feature quantity is extracted.

In the invention according to claim 2 configured as described above, the number of pixels to be processed is reduced by uniformly thinning out all the pixels on the basis of a predetermined reference. Extract feature quantities.

In this case, the uniform thinning includes a thinning and selecting in a fixed cycle, and a random selecting and thinning.

The feature amount weighting re-evaluation means re-evaluates the extracted feature amount by a predetermined weighting. However, the extracted feature amount may be weighted in pixel units corresponding to pixel units. Alternatively, weighting may be performed for each suitable group. As an example of the latter, the invention according to claim 3 is the image processing apparatus according to claim 1 or 2, wherein the feature amount equalizing extraction means is
The feature amount is extracted for each region obtained by dividing the image according to a predetermined criterion, and the feature amount weighting re-evaluation means sets weighting for each region and re-evaluates the feature amount.

In the invention according to claim 3 configured as described above, it is premised that the image is divided on a region basis by dividing the image by a predetermined reference, and the feature quantity uniform extraction means calculates the feature quantity on a region basis. Extraction is performed, and the feature amount weighting re-evaluation means re-evaluates the feature amount for each area with the weighting set for each area.

The division of the area may be constant or may be changed for each image. In this case, the division method may be changed according to the content of the image.

Various weighting methods can be adopted, and any method can be used as long as re-evaluation is performed so as not to be merely uniform sampling.

As an example thereof, the invention according to claim 4 is the image processing apparatus according to any one of claims 1 to 3, wherein the feature amount weighting re-evaluation means is determined by the position of each pixel with respect to the image. The weighting is changed according to the correspondence.

When considering the composition of a photograph, the person image is often located in the center. Therefore, if the feature amount is uniformly extracted from the entire image and then the weight of the feature amount in the central portion is increased, the feature amount extracted from the pixels of the human image is highly evaluated.

In the invention according to claim 4 configured as described above, for example, when it is determined that the central portion of the image is weighted heavily and the surrounding weights are lightened, the feature amount weighting re-evaluation means Determines the position of each pixel with respect to the image and re-evaluates using the weighting that changes depending on this position.

As another example of the weighting method, the invention according to claim 5 is the image processing apparatus according to any one of claims 1 to 4, wherein the feature amount weighting re-evaluation means is In addition to obtaining the degree of change, the weighting is made heavier in the portion where the degree of change of the image is large.

In the invention according to claim 5 configured as described above, the degree of change of the image is obtained before the feature amount re-evaluation means performs the re-evaluation. It can be said that the degree of change of the image is the sharpness of the image, and the more the image is in focus, the more clearly the contour portion is, so the degree of change is large. On the contrary, if the image is out of focus, the image gradually changes in the contour portion of the image, and the degree of change becomes small. In a photograph or the like, the part in focus is the original subject, and the part out of focus is considered to be equivalent to the background. For this reason, it is considered that the place where the degree of change of the image is large is the original subject, and the feature amount weighting re-evaluation means evaluates by weighting the portion where the degree of change of the image is large to evaluate many features. The result is equivalent to extracting the quantity.

As still another example, claim 6
In the image processing device according to any one of claims 1 to 5, the feature amount weighting re-evaluation unit obtains the chromaticity of each pixel and extracts the feature amount with the same chromaticity. The number of pixels within the range of chromaticity of the target to be obtained is determined, and the weighting is made heavy in the portion where the number of pixels is large.

In the invention according to claim 6 configured as described above, the feature amount weighting re-evaluation means obtains the chromaticity of each pixel. In image processing, an object may be identified by a particular color. For example, a person may be determined as a target by searching for a skin-colored portion.
However, it is difficult to specify the skin color because the normal color image data also includes the brightness element. On the other hand, chromaticity represents an absolute ratio of color stimulus values and is not affected by brightness. Therefore, if it is within the range of chromaticity that the skin color can take, it can be determined that it is a pixel of a human image.
Of course, in addition to the skin color, the same applies to the range of the green range of the trees and the range of the blue color of the blue sky.

Under such a background, the feature weight re-evaluation means counts the number of pixels when the chromaticity obtained for each pixel falls within the range of the chromaticity of the target from which the feature is extracted, A portion having a large number of pixels is determined to be a target, weighted heavily, and a result similar to that obtained by extracting a large amount of feature amounts from the target is obtained.

Although the weighting method as described above is not necessarily an alternative method, the invention according to claim 7 is a preferred example in the case of overlapping application.
In the image processing device according to any one of the above, the feature amount weighting re-evaluation means obtains a temporary weighting coefficient individually based on a plurality of elements, and further adds these with weighting according to importance. The configuration is applied as a final weighting coefficient.

In the invention according to claim 7 configured as described above, the feature amount weighting re-evaluation means individually obtains a temporary weighting coefficient based on a plurality of elements. Then, these are added with weighting according to the importance, and the feature quantity extracted as the final weighting coefficient is re-evaluated. Therefore, even if a large weight is given at the stage evaluated by one weighting method, if the importance of the weighting method is low, a large weight may not be given as a result. In addition, there are many cases where final weighting is often performed on the weighting methods that have a large difference between the weighting methods and that are generally evaluated for weighting above the average.

The method of performing the weighting evenly over the entire screen in the extraction step and performing the predetermined weighting after extraction need not necessarily be limited to a substantial device. As an example, the invention according to claim 8 is the dot matrix form. An image processing method for inputting real shot image data consisting of pixels of a predetermined image processing, and extracting the feature amount of each pixel required for determining the image processing intensity evenly over the entire screen, The configuration is provided with a step of re-evaluating the extracted feature amount by predetermined weighting, and a step of determining image processing intensity based on the re-evaluated feature amount and performing image processing.

That is, it is not limited to a substantial device, and there is no difference in that the method is effective.

By the way, as described above, the image processing apparatus for re-evaluating the feature quantity by weighting and performing the image processing may exist alone, or may be used in a state of being incorporated in a certain device. The idea of the invention includes various aspects. Further, it can be changed as appropriate, such as being realized by hardware or software.

In the case of software for controlling the image processing apparatus as an example of embodying the idea of the invention, it naturally exists even on a recording medium recording such software,
There is no choice but to say that it will be used.

As an example thereof, the invention according to claim 9 is a medium in which an image processing control program for inputting actually photographed image data consisting of pixels in a dot matrix form to a computer to perform predetermined image processing is recorded. The feature amount of each pixel necessary for determining the image processing intensity is uniformly extracted over the entire screen, the extracted feature amount is re-evaluated by predetermined weighting, and the image processing is performed based on the re-evaluated feature amount. The intensity is determined and image processing is performed.

Of course, the recording medium may be a magnetic recording medium or a magneto-optical recording medium, and any recording medium developed in the future can be considered in exactly the same manner. In addition, the duplication stage of the primary duplication product, the secondary duplication product, and the like is absolutely the same. In addition, the present invention is used even when a communication line is used as a supply method, and the same applies to a case where the present invention is written in a semiconductor chip.

Further, even when a part is software and a part is realized by hardware, there is no difference in the idea of the invention, and it is necessary to store a part on a recording medium. It may be in such a form that it is read as appropriate.

[0036]

As described above, the present invention does not increase the amount of calculation processing because it is performed evenly over the entire screen in the extraction stage, and the extraction is simply performed uniformly by performing a predetermined weighting after extraction. It is possible to provide an image processing apparatus capable of automatically performing optimum image processing without performing irrelevant evaluation as in the case of the above.

According to the second aspect of the present invention, the pixels are thinned out at the time of evenly extracting the feature quantity.
The amount of processing can be reduced.

Further, according to the invention of claim 3,
Since the weighting is changed for each area, the calculation becomes relatively easy.

Further, according to the invention of claim 4,
Since the weighting is determined by the position of the pixel, the calculation becomes relatively easy.

Further, according to the invention of claim 5,
Since the weighting is changed according to the sharpness of the image, different targets can be accurately discriminated for each image and the feature amount can be extracted.

Further, according to the invention of claim 6,
Since a specific target can be selected according to chromaticity, different targets can be accurately discriminated from each image to extract a feature amount.

Further, according to the invention of claim 7,
A more suitable evaluation of the feature amount can be performed by appropriately combining a plurality of weighting methods.

Further, according to the invention of claim 8,
It is possible to provide an image processing method that requires a small amount of calculation and is capable of performing image processing with optimal evaluation. According to the invention of claim 9, a medium recording an image processing control program that can achieve the same effect. Can be provided.

[0044]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing an image processing system to which an image processing apparatus according to an embodiment of the present invention is applied, and FIG. 2 is a schematic block diagram showing a concrete hardware configuration example.

In FIG. 1, the image input device 10 outputs the photographed image data representing a photograph or the like as pixels in a dot matrix to the image processing device 20, and the image processing device 20.
Performs the image processing after determining the emphasis degree of the image processing through a predetermined process. The image processing apparatus 20 outputs the image-processed image data to the image output apparatus 30, and the image output apparatus 30 outputs the image-processed image by the dot matrix pixels. Here, in the image data output by the image processing device 20, after the feature amount is uniformly extracted from each pixel according to a predetermined reference, the feature amount is re-evaluated with a predetermined weight,
The image is processed with the degree of emphasis determined according to the re-evaluated feature amount. Therefore, the image processing device 20
In this way, the feature amount uniform extraction means for uniformly extracting the feature amount, the feature amount weighting re-evaluation means for re-evaluating the extracted feature amount with predetermined weighting, and the emphasis degree according to the re-evaluated feature amount. And processing means for image processing.

A specific example of the image input device 10 corresponds to the scanner 11, the digital still camera 12 or the video camera 14 in FIG. 2, and a specific example of the image processing device 20 is a computer 21, a hard disk 22, a keyboard 23 and a CD-ROM. A computer system including a drive 24, a flexible disk drive 25, a modem 26, and the like corresponds to the image output device 30, and a specific example of the image output device 30 includes a printer 31, a display 32, and the like. In the case of the present embodiment, since the object is found as the image processing and the appropriate image processing is performed, the photographed data such as a photograph is suitable as the image data. It should be noted that the modem 26 is connected to a public communication line and can be connected to an external network via the same public communication line so that software and data can be downloaded and introduced.

In the present embodiment, the image input device 10
The scanner 11 or the digital still camera 12 as an output outputs RGB (green, blue, red) gradation data as image data, and the printer 3 as an image output device 30.
1 requires CMY (cyan, magenta, yellow) or CMYK binary data in which black is added as input as gradation data, and the display 32 displays RGB.
The gradation data of is required as an input. On the other hand, the operating system 21a is running in the computer 21, and the printer driver 21b and the display driver 21c corresponding to the printer 31 and the display 32 are provided.
Is built in. The image processing application 21d is controlled by the operating system 21a to execute processing, and executes predetermined image processing in cooperation with the printer driver 21b and the display driver 21c as necessary. Therefore, the specific role of the computer 21 as the image processing apparatus 20 is to input RGB gradation data to create RGB gradation data that has been subjected to optimum image processing, and to display it through the display driver 21c to the display 32. And the printer driver 21
It is converted into CMY (or CMYK) binary data via b and printed by the printer 31.

As described above, in the present embodiment, a computer system is incorporated between image input / output devices to perform image processing. However, such a computer system is not necessarily required, and image data is not necessarily required. Any system may be used as long as it performs various kinds of image processing. For example, as shown in FIG.
It is also possible to have a system in which an image processing apparatus for judging an object and performing image processing is incorporated in 2a, and the converted image data is used to display on the display 32a or to print on the printer 31a. Further, as shown in FIG. 4, in the printer 31b that inputs and prints image data without using a computer system, the image data automatically input from the scanner 11b, the digital still camera 12b, the modem 26b, or the like. It is also possible to determine the object and process the image.

The above-mentioned determination of the object and the accompanying image processing are carried out in the computer 21 by an image processing program corresponding to the flowchart shown in FIG. In the flowchart shown in the figure, image processing for adjusting the contrast of an image is performed. In step S110, the luminance which is the feature amount is extracted while uniformly thinning out pixels from the entire image, and then in step S120. The same feature amount is re-evaluated by performing predetermined weighting, and image processing for adjusting the brightness is performed in steps S130 to S160.

In step S110, as shown in FIG. 6, the luminance of each pixel is obtained for the image data of the dot matrix in the vertical and horizontal directions to generate a histogram. In this case, it can be said that it is accurate if it is performed for all pixels, but as will be described later, since the totalized result is weighted and re-evaluated,
It does not necessarily have to be accurate. Therefore, it is possible to reduce the processing amount and speed up by thinning out the pixels from which the luminance is extracted to the extent that they are within a certain error range. According to the statistical error, the error with respect to the sample number N is about 1 / (N **
(1/2)). However, ** represents the power. Therefore, in order to perform processing with an error of about 1%, N =
It becomes 10,000.

Here, the bitmap screen shown in FIG. 6 has the number of pixels of (width) × (height), and the sampling period ratio is ratio = min (width, height) / A + 1 (1). Where min (width, heig
ht) is the smaller of width and height, and A is a constant. Further, the sampling cycle ratio referred to here indicates how many pixels are sampled, and the pixels marked with a circle in FIG.
The case where tio = 2 is shown. That is, one pixel is sampled every two pixels in the vertical direction and the horizontal direction, and sampling is performed every other pixel. The number of sampling pixels in one line when A = 200 is as shown in FIG.

As can be seen from the figure, when the width is 200 pixels or more, at least the number of samples is 100 pixels or more, except for the case where the sampling period ratio = 1 which is not sampled. Therefore, when the number of pixels is 200 or more in the vertical and horizontal directions, (100 pixels) × (100 pixels) = (10000 pixels)
Is ensured and the error can be reduced to 1% or less.

Where min (width, hei
ght) is used as a reference for the following reason.
For example, like the bitmap image shown in FIG.
If width >> height is satisfied, and if the sampling period ratio is determined by the longer width, as shown in FIG. 6B, only the upper and lower two lines of pixels are extracted in the vertical direction. It may happen that you are not done. However, min (widt
h, height), if the sampling period ratio is determined based on the smaller one, the same figure (c)
As shown in FIG. 5, it is possible to perform thinning out including the intermediate portion even in the smaller vertical direction. That is, it becomes possible to perform sampling while ensuring a predetermined number of extractions.

Luminance is obtained by thinning out pixels in this manner. As described above, in this embodiment, the computer 21 handles RGB gradation data and does not directly have a luminance value. L to obtain the brightness
It is possible to perform color conversion into the uv color space, but this is not a good idea due to problems such as the amount of calculation. For this reason, the following conversion equation for directly obtaining the luminance from RGB used in a television or the like is used.

Y = 0.30R + 0.59G + 0.11B (2) Further, the luminance histogram is not aggregated for one image at once, but as shown in FIG. The blocks are divided into a total of 15 blocks and counted individually. In the present embodiment, such 15 blocks are used, but it goes without saying that the block dividing method is arbitrary. In particular, a printer driver or the like receives image data for each block from an application, but area division for weighting may be performed using this block.

The reason why the blocks are totaled in this way is to reduce the processing amount. In the meaning of weighting and re-evaluating in step S120, it is not always necessary to add up for each block, and it is also possible to add up for each selected pixel and add up as a histogram. In addition, since it means that the weighting of the counting result is changed even for each block, it is possible to perform the counting with one histogram by using the weighting according to the block. FIG. 11 is a diagram showing an example of the luminance distribution of the block Bi.

When totaling each block,
In step S120, each area is weighted and re-evaluated. 12 and 13 show an example in which each block is weighted. Assuming a general photographic image, the composition is usually such that the original subject enters the central portion. In this sense, the feature amount should be evaluated with an emphasis on the image in the central portion of the image data. On the other hand, considering another example of taking a commemorative photo in front of a building,
The image of the person is located below the center. This is because the height of the ground is usually located at the bottom of the image. Therefore, in this case, it can be said that the feature amount should be evaluated by weighting the lower center of the image. The example of FIG. 12 corresponds to the former, and FIG. 13 corresponds to the latter.

The weighting of each block is Wi (i = 1 to 1).
5), the weighted and re-evaluated luminance distribution is DY,

[0060]

[Equation 1]

And   Ki = Wi / SP (4) Then,

[0062]

[Equation 2]

Is calculated as

When the histogram of the luminance distribution re-evaluated in this way is obtained, the intensity of image processing is obtained from this feature amount. That is, the width for expanding the contrast is determined. In determining the expansion width, it is considered to find both ends of the luminance distribution. The luminance distribution of the photographic image appears in a mountain shape as shown in FIG. Of course, there are various positions and shapes. The width of the luminance distribution is determined depending on where the ends are determined, but it is not possible to simply set the points at which the skirt extends and the distribution number becomes “0” as the ends. This is because the number of distributions may change in the vicinity of “0” in the tail part, and the number of distributions may approach “0” infinitely statistically.

Therefore, in the distribution range, the portions which are inward from the side having the highest brightness and the side having the lowest brightness by a certain distribution ratio are defined as the both ends of the distribution. In this embodiment, as shown in the figure, this distribution ratio is set to 0.5%. Of course, this ratio can be changed appropriately. In this way, by cutting the upper and lower ends by a certain distribution ratio, it is possible to ignore white points and black points caused by noise and the like. That is, if there is no white point or black point without such a process, it will become the both ends of the luminance distribution. Therefore, if the luminance value is 255 gradations, in most cases, the lowest edge is gradation. "0"
And the top end has a gradation of "255",
This is eliminated by setting the end portion to be a portion that is inside by 0.5% of the number of pixels from the upper end portion.

In the actual processing, 0.5% with respect to the number of pixels is calculated based on the histogram obtained by re-evaluation, and the luminance value at the upper end and the luminance value at the lower end in the reproducible luminance distribution are sequentially turned inward. However, the respective distribution numbers are accumulated to obtain a luminance value of 0.5%. Hereinafter, the upper end side is called ymax and the lower end side is called ymin.
Further, in the present embodiment, the brightness is corrected together with the expansion of the contrast, and the median ymed required for that purpose is also determined based on the re-evaluated histogram. The above process is performed in step S130.

The range of reproducible luminance is "0" to "25".
5 ”, the luminance Y of the conversion destination is calculated from the luminance y before conversion and the maximum value ymax and minimum value ymin of the luminance distribution range based on the following equation.

Y = ay + b (6) where a = 255 / (ymax-ymin) (7) b = -a.ymin or 255-a.ymax (8) Further, Y <0 in the above conversion formula. If so, Y = 0 and Y> 2
If 55, then Y = 255. Here, it can be said that a is an inclination and b is an offset. According to this conversion formula, as shown in FIG. 15, the luminance distribution having a certain narrow width can be expanded to a reproducible range. However,
If the maximum reproducible range is used to expand the luminance distribution, the highlight part may be white or the high shadow part may be black. In order to prevent this, the reproducible range is limited in this embodiment. That is, “5” is left as the brightness value as a range that does not expand to the upper and lower ends of the reproducible range. As a result, the parameters of the conversion formula are as follows.

A = 245 / (ymax-ymin) (9) b = 5-a · ymin or 250-a · ymax (10) In this case, in the range of y <ymin and y> ymax. Disables conversion.

However, if the same enlargement ratio (corresponding to a) is applied, a very large enlargement ratio may be obtained. For example, in the twilight state such as in the evening, it is natural that the width of the contrast from the brightest part to the darkest part is narrow, but as a result of trying to greatly expand the contrast of this image, it is converted like the daytime image. It can happen. Since such a conversion is not desired, a limit is set on the enlargement ratio, and a is 1.5 (~
2) Limit not to exceed the above. As a result, twilight will be expressed like twilight. In this case, processing is performed so that the center position of the brightness distribution does not change as much as possible. In step S140, the processing for obtaining the enlargement ratio a and the inclination b is performed in this manner.

By the way, it is irrational to execute the above conversion formula (Y = ay + b) every time the brightness is converted.
This is because the range of brightness y that can be taken is "0" to "25".
Since it can only be 5 ”, it is also possible to previously obtain the converted luminance Y corresponding to all possible values of the luminance y. Therefore, it is stored as a table as shown in FIG.

Since it is extremely effective not only to enhance the contrast by adjusting the brightness range but also to adjust the brightness in accordance with the brightness range, a parameter for correction is also generated by determining the brightness of the image. To do.

For example, when the peaks of the luminance distribution are generally on the dark side as shown by the solid line in FIG. 17, it is advisable to move the peaks to the bright side as shown by the broken line. On the contrary, when the peaks of the luminance distribution are generally on the bright side as shown by the solid line in FIG. 18, it is advisable to move the peaks to the dark side as shown by the wavy line.

As a result of various experiments, in the present embodiment, the median ymed in the luminance distribution is obtained, and when the median ymed is less than “85”, it is judged as a dark image and corresponds to the following γ value. Brightens with γ correction.

[0075]   γ = ymed / 85 (11) Alternatively,   γ = (ymed / 85) ** (1/2) (12) And

In this case, even if γ <0.7, γ =
Set to 0.7. This is because, if such a limit is not set, the night image becomes like the daytime image. Note that if the image is too bright, the image tends to be whitish as a whole, and the image tends to have a low contrast. Therefore, it is preferable to perform processing such as enhancing the saturation.

On the other hand, when the median ymed is larger than "128", it is judged as a bright image, and the image is made dark by the γ correction corresponding to the following γ value.

Γ = ymed / 128 (13) Alternatively, γ = (ymed / 128) ** (1/2) (14) In this case, even if γ> 1.3, γ = 1.3
Set a limit so that it does not get too dark.

The γ correction may be performed on the brightness distribution before conversion or on the brightness distribution after conversion. FIG. 19 shows the correspondence relationship when the γ correction is performed. If γ <1, the curve expands upward, and if γ> 1, the curve expands downward. Of course, the result of the γ correction may be reflected in the table shown in FIG. 16, and the same correction is performed on the table data. In step S150, processing for generating such a conversion table is executed.

After that, conversion based on the equation (6) is performed.
The conversion formula of the same formula can be applied also in the correspondence relationship with the RGB component values, and the component values (R0, G0,
The converted component values (R, G, B) for B0) can also be obtained as R = a · R0 + b (15) G = a · G0 + b (16) B = a · B0 + b (17) it can. Here, the luminance y and Y are from the gradation “0” to the gradation “255”, and the RGB component values (R0, G0, B0) and (R, G, B) are also in the same range. Therefore, it can be said that the above-mentioned conversion table of the luminances y and Y can be used as it is.

Therefore, in step S160, the image data (R0, G0, B0) of all pixels are set to (15) to (1).
The process of obtaining the converted image data (R, G, B) by referring to the conversion table corresponding to the expression 7) is repeated.

Of course, the processing means is constituted by the hardware configuration and software for executing these steps S130 to S160. In the present embodiment, the contrast enlargement process and the brightness correction process are executed as the image processing, but it goes without saying that the same can be applied to other image enhancement processes.

In the above processing, the feature amount is re-evaluated by weighting according to the position in the image. However, the weighting standard is not limited to this, and various modes are possible. FIG. 20 shows an example in which the original subject is detected from the degree of change in the image,
The flowchart when changing weighting is shown.

Step S210 is the above-mentioned step S210.
It replaces 110, and the luminance is aggregated for pixels that are evenly thinned. However, instead of counting only brightness,
The following edge amounts are also calculated.

It can be said that the background of the input image has a gradual change in shading, and the original subject is sharp, so that it can be said that the brightness changes drastically. Therefore, as shown in FIG. 21, one pixel is used as a pixel of interest, and a density difference from the peripheral pixels is obtained, and this density difference is used as the edge amount of the pixel of interest. This density difference can be calculated by applying a filter, and FIGS. 22 (a) to 22 (f) show several examples of such a filter. The weighting coefficient for weighted addition is shown. Here, in the case of FIG. 9A, since weighted addition is made for nine pixels, nine multiplications and eight additions are required to obtain the edge amount at each pixel. Since this amount of calculation cannot be ignored as the image becomes larger, it can be done by multiplying five times and adding four times in the figures (b) and (c), and three times in the figures (d) and (e). In this case, the multiplication is performed twice and the addition is performed once in FIG.

In these examples, the pixel of interest is compared only with the pixels that surround it, but the so-called unsharp mask is used to obtain the sharpness of the pixel of interest using the data of a wider range of pixels. It is possible to ask for However, the edge amount in the present embodiment is only for evaluating the weighting for each block, and thus the filters of these examples in which the calculation amount is reduced are also sufficient.

The edge amount may be totaled by calculating the edge amount obtained for each pixel for each block. If the absolute value of the edge amount is larger than a predetermined threshold value, it is determined as an edge pixel. However, the total number of edge pixels for each block may be totaled. E is the amount of edge in each block
If Ri (i = 1 to 15), the total amount SE is

[0088]

[Equation 3]

Therefore, the weighting coefficient KEi itself is expressed by KEi = ERi / SE (19). Therefore, the weighted and re-evaluated luminance distribution DY in this case is

[0090]

[Equation 4]

Is calculated as When the total number of edge pixels in each block is ENi (i = 1 to 15), the total amount SE is

[0092]

[Equation 5]

Therefore, the weighting coefficient KEi itself is represented by KEi = ENi / SE (22), and the reevaluated luminance distribution DY can be obtained by using the equation (20). In any case, in step S220, the brightness distribution DY is re-evaluated based on the arithmetic expression of Expression (20).

In this example, the luminance of the pixel determined as the edge pixel is not sampled and used, but the edge amount and the total number of edge pixels are used only for determining the weighting coefficient of the block. That is, the feature amounts of only pixels having a specific property (edge) are not aggregated, and an average feature amount that is not biased in the block can be obtained.

If the luminance distribution is re-evaluated in this way, the contrast may be expanded and the brightness may be corrected through the processes of steps S130 to S160 described above.

Further, considering that the original subject is often a human image, it is possible to re-evaluate by placing emphasis on a block having many pixels of skin color. FIG. 23 shows a flowchart for deciding the weighting coefficient of a block by paying attention to such a specific color.

Step S3 corresponding to step S110
In 10, the luminance is totaled by the same thinning-out process, and it is determined whether the pixel is a skin-colored pixel based on the chromaticity of each pixel, and the number of skin-colored pixels is totaled. For chromaticity, the xy chromaticity for each pixel is calculated. Now, the RGB gradation data in the RGB color system of the target pixel is (R, G,
B), r = R / (R + G + B) (23) g = G / (R + G + B) (24), and between chromaticity coordinates x and y in the XYZ color system. In, x = (1.1302 + 1.6387r + 0.6215g) / (6.7846-3.0157r-0.3857g) ... (25) y = (0.0601 + 0.9399r + 4.5306g) / (6.7846-3) .0157r-0.3857g) (26) The correspondence relationship is established. Here, since the chromaticity represents the absolute ratio of the color stimulus values without being influenced by the brightness, it can be said that it is possible to determine what kind of object the pixel is from the chromaticity. . In the case of skin color, since it is included in the range 0.35 <x <0.40 (27) 0.33 <y <0.36 (28), the chromaticity of each pixel was obtained. Sometimes, if it is within this range, the pixel is considered to be a pixel showing human skin, and the number of skin color pixels in the block is increased by one.

When the number of skin color pixels is obtained in this way,
In the next step S320, the weighting coefficient is determined in the same manner as in the case of the number of edge pixels described above, and the luminance distribution DY is re-evaluated. That is, the number of skin color pixels in each block is CN
If i (i = 1 to 15), the total amount SC is

[0099]

[Equation 6]

Therefore, the weighted KCi itself is expressed by KCi = CNi / SC (30), and the luminance distribution DY re-evaluated by weighting in this case is

[0101]

[Equation 7]

Is calculated as Also in this example, the luminance of the pixel determined to be the skin color pixel is not sampled and used, but the total number of skin color pixels is used only for determining the weighting coefficient of the block. Therefore, it is possible to obtain an average feature amount that is not biased in that block. Also in this case, if the luminance distribution is re-evaluated in this way, the contrast may be expanded and the brightness may be corrected through the processes of steps S130 to S160 described above. In the case of the photograph shown in FIG. 24, a girl appears in the vicinity of the center, and the pixels of the face and limbs are determined to be skin-color pixels. Of course, the chromaticity of other colors may be obtained and the number of pixels may be totaled.

By the way, the weighting coefficient has been determined by one factor so far, but it is also possible to apply the weighting factor in duplicate by taking into consideration the importance of each factor. When the weighting coefficient of each block Bi (i = 1 to 15) is Tji for each factor j (1: position in the image, 2: edge amount, 3: skin color pixel number), each block for each factor The weighted Tji distributed to is a temporary weighting,

[0104]

[Equation 8]

When Kji = Tji / Sj (33), the true weighting coefficient Ki in the block Bi is obtained.
Is

[0106]

[Equation 9]

It is represented by Here, Aj is a coefficient indicating the degree of importance of each factor, and is appropriately determined within the range where the total number is 1. As an example, if the skin color is emphasized, A1
= 0.2, A2 = 0.2, A3 = 0.6, etc. can be set.

Next, the operation of this embodiment having the above configuration will be described. First, a description will be given according to the above embodiment.

It is assumed that a photographic image is read by the scanner 11 and printed by the printer 31. Then first,
Operating system 21a on computer 21
Image processing application 21
d is activated to cause the scanner 11 to start reading a photo. When the read image data is loaded into the image processing application 21d via the operating system 21a, the luminance of each pixel is totalized while thinning out in step S110. The aggregated luminance distribution dYi is shown in FIG.
Re-evaluation is performed based on the weighting determined corresponding to the position of each block, and based on the re-evaluated luminance distribution DY, ymax, ymin, and yme are calculated in step S130.
Find d.

In the next step S140, the slope a and the offset b, which are the emphasis parameters, are calculated based on the equation (9) or the equation (10), and the equation (11)-
The γ value of the γ correction required for the brightness correction is obtained based on the equation (14), and the conversion table shown in FIG. 6 is created in step S150. Then, finally, in step S160, the image data for all pixels is converted by referring to the same conversion table.

When the weighting shown in FIG. 12 is used, the closer to the center the block is, the heavier the weight is. Therefore, the closer the brightness distribution is to the center, the higher the evaluation is. For example, assume that a person is photographed using a flash at night. The brightness distribution of the person image is dark due to the effect of flash, and even if a generally good brightness distribution is obtained as shown in FIG. 25A, the surroundings of the person are dark and dark as shown in FIG. A luminance distribution biased to the side is obtained. In this case, if the values are simply averaged, a luminance distribution biased to the dark side as a whole is obtained as shown in FIG. Therefore, if the contrast and the brightness are corrected as they are, the dark image is forcibly made bright and a good image cannot be obtained.

However, if the central block is weighted as shown in FIG.
As shown in FIG. 5D, a luminance distribution DY strongly influenced by the luminance distribution obtained in the central portion of the image is obtained. Therefore, the intensity of the image processing determined based on the brightness distribution is not such that the contrast is excessively enlarged or the brightness is corrected.

On the contrary, when a person is photographed in a backlit state, the face becomes dark and the background becomes bright, but this may result in a good luminance distribution in the entire image. However, even in such a case, as shown in FIG. 12, when the luminance distribution in the central block in which a dark face is shown is weighted and evaluated, the dark luminance distribution is reflected, and the contrast is increased or the image is enlarged. It becomes possible to perform image processing such as brightening.

By the above processing, the image data of the photograph read by the scanner 11 is automatically subjected to the image processing with the optimum intensity and displayed on the display 32.
It is printed by the printer 31.

As described above, the computer 21, which is the center of the image processing, tabulates the distribution of the luminance, which is the feature amount, for each area while selecting pixels evenly in step S110,
In step S120, the luminance distribution strongly influenced by the original luminance distribution of the subject can be obtained by performing the reevaluation with the weighting determined for each area, while performing the uniform sampling, and in steps S130 to S150. In order to convert the image data in step S160 after determining the intensity of image processing based on the brightness distribution,
Image processing can be executed with optimum intensity while lightening the processing.

[Brief description of drawings]

FIG. 1 is a block diagram of an image processing system to which an image processing apparatus according to an embodiment of the present invention is applied.

FIG. 2 is a block diagram of concrete hardware of the image processing apparatus.

FIG. 3 is a schematic block diagram showing another application example of the image processing apparatus of the present invention.

FIG. 4 is a schematic block diagram showing another application example of the image processing apparatus of the present invention.

FIG. 5 is a flowchart showing image processing in the image processing apparatus of the present invention.

FIG. 6 is a diagram showing a state in which processing target pixels are moved.

FIG. 7 is a diagram showing a sampling cycle.

FIG. 8 is a diagram showing the number of sampling pixels.

FIG. 9 is a diagram showing a relationship between a conversion source image and sampled pixels.

FIG. 10 is a diagram showing an arrangement of blocks obtained by dividing an image into regions.

FIG. 11 is a diagram showing an example of a luminance distribution in each block.

FIG. 12 is a diagram showing an example of weighting for each block.

FIG. 13 is a diagram showing another example of weighting for each block.

FIG. 14 is a diagram showing edge processing of luminance distribution and edges obtained by the edge processing.

FIG. 15 is a diagram showing an expansion of a luminance distribution and a reproducible luminance range.

FIG. 16 is a diagram showing a conversion table when enlarging a luminance distribution.

FIG. 17 is a diagram showing a concept of brightening by γ correction.

FIG. 18 is a diagram showing a concept of darkening by γ correction.

FIG. 19 is a diagram showing a correspondence relationship of luminance changed by γ correction.

FIG. 20 is a flowchart in the case of re-evaluating the feature amount based on the edge amount.

FIG. 21 is a diagram showing a relationship between a target pixel and a peripheral pixel for determining an edge amount.

FIG. 22 is a diagram showing an example of a filter for calculating an edge amount.

FIG. 23 is a flowchart in the case of re-evaluating a feature amount based on the number of pixels of a predetermined color.

FIG. 24 is a diagram showing an example of a photographic image.

FIG. 25 is a diagram showing a luminance distribution of a photographic image taken at night.

[Explanation of symbols]

10 ... Image input device 20 ... Image processing device 21 ... Computer 21a ... Operating system 21b ... printer driver 21c ... Display driver 21d ... Image processing application 25 ... Flexible disk drive 30 ... Image output device

   ─────────────────────────────────────────────────── ─── Continued front page    F term (reference) 5B057 CA08 CA12 CA16 CB08 CB12                       CB16 CC01 CE03 CE11 CH08                 5C077 LL04 MP01 MP08 PP03 PP15                       PQ12 PQ18 PQ19 TT02

Claims (9)

[Claims] 1. An image processing apparatus for inputting real shot image data composed of pixels in a dot matrix and performing a predetermined image processing, wherein the feature amount of each pixel required for determining the image processing intensity is displayed over the entire screen. Feature amount equality extraction means for extracting evenly, feature amount weighting re-evaluation means for re-evaluating the feature amount extracted by the feature amount equality extraction means by predetermined weighting, and image processing based on the re-evaluated feature amount An image processing apparatus comprising: a processing unit that determines intensity and performs image processing. 2. The image processing device according to claim 1, wherein the feature amount equality extraction means extracts the feature amount from pixels selected by thinning out all pixels evenly according to a predetermined criterion. An image processing device characterized by: 3. The image processing apparatus according to claim 1 or 2, wherein the feature amount uniform extraction means extracts the feature amount in units of regions obtained by dividing an image by a predetermined reference. The image processing apparatus, wherein the feature amount weighting re-evaluation means sets a weight for each area and re-evaluates the feature amount. 4. The image processing device according to any one of claims 1 to 3, wherein the feature amount weighting re-evaluation means changes the weighting in a correspondence relationship determined by the position of each pixel with respect to the image. An image processing device characterized by: 5. The image processing apparatus according to any one of claims 1 to 4, wherein the feature amount weighting re-evaluation means obtains the degree of change of the image, and at the portion where the degree of change of the image is large, An image processing device characterized by increasing weighting. 6. The image processing device according to claim 1, wherein the feature amount weighting re-evaluation means obtains the chromaticity of each pixel and extracts the feature amount with the same chromaticity. The image processing apparatus is characterized in that the number of pixels within the range of chromaticity of the target is obtained, and the weighting is made heavy in a portion having a large number of pixels. 7. The image processing apparatus according to claim 1, wherein the feature amount weighting re-evaluation means individually obtains a temporary weighting coefficient based on a plurality of elements, and further, ,
An image processing apparatus characterized in that these are added with weighting according to importance and applied as a final weighting coefficient. 8. An image processing method for inputting real shot image data composed of pixels in a dot matrix form and performing predetermined image processing, wherein the feature amount of each pixel required for determining the image processing intensity is provided over the entire screen. A step of uniformly extracting, a step of re-evaluating the extracted feature amount by predetermined weighting, and a step of determining an image processing intensity based on the re-evaluated feature amount and performing image processing. Characterized image processing method. 9. A medium in which an image processing control program for inputting real shot image data composed of pixels in a dot matrix form to a computer and performing predetermined image processing is recorded. The feature amount of the pixel is uniformly extracted over the entire screen, and the extracted feature amount is re-evaluated by predetermined weighting,
A medium having an image processing control program recorded therein, characterized in that the image processing intensity is determined based on the re-evaluated characteristic amount and image processing is performed.