JP4022638B2 - Image pickup device and image pickup apparatus using the same - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an imaging element and an imaging apparatus using the imaging element, and more particularly to an imaging element capable of imaging a color image and a high resolution monochrome image and an imaging apparatus using the imaging element.
[0002]
[Prior art]
Conventionally, imaging that captures a color image through primary color filters of red (R), green (G), and blue (G) or complementary color filters of cyan (C), magenta (M), and yellow (Y) When attempting to obtain a black and white image using an element, a predetermined arithmetic expression for converting the color system is applied to the primary color system or complementary color system element value for each pixel constituting the acquired color image. . That is, in the same way as when obtaining a color image, the image sensor outputs an image signal, and the output result is converted to a black and white image afterwards by a predetermined conversion formula by a signal processing circuit or signal processing software. And output.
[0003]
For example, when a color image composed of R, G, and B elements is converted to a black and white image, it is converted to a luminance Y of the YIQ color system adopted by the NTSC system. This conversion is
[Expression 1]
Y = 0.299R + 0.587G + 0.114B
It was performed by a signal processing circuit or signal processing software.
[0004]
[Problems to be solved by the invention]
However, when the signal processing circuit converts the above-described color image into a black and white image, the color imaging signal needs to be temporarily converted into a digital signal by A / D conversion and temporarily stored in the memory. There is a problem in that it takes time for signal processing operations of a signal processing circuit for converting to a black and white image or signal processing operations by signal processing software, and a black and white image output cannot be obtained at high speed.
[0005]
Further, in the case of an electronic camera, there is a problem in that a monochrome image cannot be output at a high speed, and as a result, the imaging interval of the monochrome image is limited and high-speed continuous shooting cannot be performed.
[0006]
Therefore, the present invention eliminates such a problem, and an image pickup device capable of picking up both a color image and a black and white image with the same image pickup device and capable of picking up a black and white image at high speed, and the same are used. An object is to provide an imaging device.
[0007]
[Means for Solving the Problems]
The first invention includes a plurality of pixels each having a photoelectric conversion element and arranged in a matrix, and when a is a natural number, among the 4a pixels adjacent vertically and horizontally, 2a pixels are green (G), When a color pixel matrix having a blue (B) pixel is mounted on the a pixel and a red (R) color filter is mounted on the other a pixel, and a color image to be stored in the nonvolatile memory is captured, It is possible to sequentially read out the charges of the 4a pixels adjacent to the top, bottom, left and right without adding, and when obtaining the luminance signal in the color pixel matrix , the charges of the 4a pixels adjacent to the top, bottom, left and right can be added and sequentially read out. It is characterized by that.
[0008]
In the first invention, the 4a pixels adjacent in the vertical and horizontal directions are output without being added or added , so that not only the image pickup signal for color image processing with the same image pickup device but also luminance, for example, A monochrome image signal such as a signal can be directly output on the image sensor.
[0009]
The second invention includes a plurality of pixels each having a photoelectric conversion element and arranged in a matrix, and when a is a natural number, among the 4a pixels adjacent vertically and horizontally, 2a pixels are green (G), When a color pixel matrix having a blue (B) pixel is mounted on the a pixel and a red (R) color filter is mounted on the other a pixel, and a color image to be stored in the nonvolatile memory is captured, It is possible to sequentially read out the charges of the 4a pixels adjacent to the top, bottom, left and right without adding, and when obtaining the luminance signal in the color pixel matrix , the charges of the 4a pixels adjacent to the top, bottom, left and right can be added and sequentially read out. and an imaging device relative to said imaging device, when imaging the color image, and supplies a drive signal in order to read without adding the charge of a 4a pixels adjacent to the vertically and horizontally, the luminance signal In obtaining it is characterized by comprising, a drive circuit for supplying a drive signal to adding and reading charges of four pixels adjacent to the vertically and horizontally.
[0010]
In the second invention, since the image pickup device of the first invention and the drive circuit for adding and outputting the 4a pixels adjacent to the image pickup device in the vertical and horizontal directions are added . For example, a monochrome image signal such as a luminance signal can be obtained directly from the image sensor, and there is no need to perform signal processing for converting a signal corresponding to a color image into a signal corresponding to a monochrome image. Such load and time can be reduced, the time required for monochrome image output can be shortened, the photographing interval can be shortened, and high-speed continuous shooting can be performed.
[0011]
According to a third aspect, in the second aspect, when m, n, and b are natural numbers, the color pixel matrix includes horizontal bm pixels and vertical bn pixels, and when obtaining the luminance signal, It is characterized in that it is possible to read out the pixel signals of m horizontal pixels and n vertical pixels by adding and reading out charges of adjacent 2b pixels by the driving circuit.
[0012]
In the third invention, when the luminance signal is obtained, the charge amount is increased as compared with the case where a color image is output, so that a highly sensitive image can be obtained.
[0013]
According to a fourth aspect, in the third aspect, the luminance signal acquisition mode for reading out the m horizontal and vertical n pixel signals and the color reading without adding the 2 m horizontal and 2n vertical pixels are added. characterized in that it comprises means for switching the photographing mode of the image.
[0014]
In the fourth aspect of the invention, the switching means switches, for example, between a color image photographing mode corresponding to color image output, which is read out without addition, and a luminance signal acquisition mode, which corresponds to monochrome image output, which is added and read out. Therefore, a multifunctional imaging device can be realized using the same imaging element, and a flexible imaging device can be realized.
[0015]
According to a fifth aspect, in the second aspect, when the luminance signal is obtained, a is a natural number, and the luminance signal is read out by adding the charges of the 4a pixels adjacent in the vertical and horizontal directions. .
As a result, a monochrome image corresponding to the luminance signal can be output with high sensitivity, and since it is not necessary to generate a luminance signal by signal processing calculation, high-speed output is possible and high-speed continuous shooting can be realized.
[0016]
According to a sixth aspect of the present invention, in the second aspect, when the color temperature of the light source differs from the color temperature of the reference light source depending on the shooting environment, a color correction filter is used, and when a is a natural number, It is characterized in that the correct luminance signal can be read out by correcting the ratio of each color signal of the 4a pixel adjacent to the left and right.
Thereby, an accurate luminance signal corresponding to an appropriate color temperature can be obtained, and as a result, an accurate monochrome image can be obtained.
In the seventh invention according to the first invention, the spectral characteristics of each color of the green (G), blue (B), and red (R) color filters are such that the output of each color component of each color pixel is “ R: G: B = 0.299: 0.587: 0.144 ”. The eighth invention is characterized in that in any of the second to sixth inventions, a black and white image is picked up by obtaining the luminance signal. Further, in a ninth invention according to any one of the second to sixth or eighth inventions, spectral characteristics of each color of the green (G), blue (B), and red (R) color filters of the imaging device are as follows. The output of each color component of the pixels of each color is set to be “R: G: B = 0.299: 0.587: 0.144”.
[0017]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is an explanatory diagram showing a configuration of an image sensor according to an embodiment of the present invention. The image sensor 10 constitutes a color pixel matrix in which a plurality of pixels 1 of 2m in the horizontal direction (row direction) and 2n in the vertical direction (column direction) are arranged. The color pixel matrix has R, G, and B color pixels arranged in an RGB Bayer array, and is composed of a plurality of color pixel groups 6 of m in the horizontal direction and n in the vertical direction. One color pixel group 6 in this RGB Bayer array is composed of four pixels 1 of 2 × 2 pixels, for example, G in 1 row and 1 column, R in 1 row and 2 columns, and 2 rows and 1 column. Each color pixel of G is adjacent to B and 2 rows and 2 columns. These pixels 1 as photoelectric conversion elements such as photodiodes function as any of R, G, and B color pixels, and this function allows each R, G, and B light to be applied to the upper surface of each pixel 1. A transparent color filter is applied. Therefore, for example, the pixel 1 in 1 row and 1 column transmits only G (green) light, receives only the light component of G, and performs photoelectric conversion. Note that m and n are natural numbers.
[0018]
When the image sensor 10 captures a color image, the charge accumulated in each vertical pixel 1 is output in synchronization with the vertical transfer circuit 2, and the charge output to the vertical transfer circuit 2 is output in each horizontal direction. For each charge corresponding to the pixel 1, one stage is sequentially transferred in synchronization with the horizontal transfer circuit 3, and the horizontal transfer circuit 3 outputs the charge corresponding to the pixel 1 in the horizontal direction one by one through the output amplifier 4. The signal 5 is transferred and output, and all charges accumulated in all the pixels 1 are swept out. As a result, a series of charges output as an output signal are charges corresponding to B, G, B, G,... Pixels in the first stage, and G, R, G, R,. , And the charges corresponding to the pixels B, G, B, G,... Are repeatedly output in the next stage, and the charges corresponding to the pixels G, R, G, R,. Therefore, R, G, B color images corresponding to the arrangement of the color pixel group 6 are generated.
[0019]
In particular, such an RGB Bayer arrangement facilitates the conversion from the RGB color system to the YIQ color system adopted by the NTSC system. That is, the YIQ color system is expressed by the luminance Y corresponding to the monochrome image and the color differences I and Q, and the RGB color system has the following conversion relationship. That is,
[Expression 2]
Y = 0.299R + 0.587G + 0.144B
I = 0.596R-0.274G-0.322B
Q = 0.211R-0.522G + 0.311B
However, the color differences I and Q can be further modified as follows:
[Equation 3]
I = 0.596 (RG) +0.322 (GB)
Q = 0.211 (RG) -0.311 (GB)
Therefore, if the difference between R and G and G and B can be calculated, the color differences I and Q can be easily obtained. Here, in the RGB Bayer array, B and G and G and R are adjacent to each other in the color pixel group 6 and are transferred and output adjacent to each other, so that the above-described calculation can be easily performed.
[0020]
On the other hand, when the image pickup device 10 picks up a black and white image that is a feature of the present invention, as shown in FIG. 2, the charges accumulated in the four pixels in each color pixel group 6 are transferred on the image pickup device 10. The added one charge is output as an output signal 5 in the form of being transferred in sequence.
[0021]
Further, for each of the four pixels in each color pixel group 6, the output of each color component of one R pixel: two G pixels: one B pixel is 0.299: 0.587: 0. The spectral characteristics of the R, G, B color filters are set so as to be 144. Thus, the charge added by each color pixel group 6 corresponds to the luminance Y of the YIQ color system, and the output signal 5 from the image sensor 10 outputs a luminance (Y) signal in the arrangement of each color pixel group 6. Thus, the addition pixel 8 of the m × n matrix represents each pixel of the black and white image (see FIG. 3).
[0022]
In this case, since the charge amount of the addition pixel 8 increases, a highly sensitive monochrome image can be obtained.
[0023]
Here, there are the following addition modes on the image sensor 10.
First, two vertical columns of pixels (G and B or R and G) in the color pixel group 6 are added on the vertical transfer circuit 2, and the added pixels are added on the output amplifier 4. It is what I did.
[0024]
The second is that two pixels in one vertical column in the color pixel group 6 are added when transferred by the horizontal transfer circuit 3, and the added pixels are further added on the output amplifier 4. .
[0025]
Third, the charge of 4 pixels is output only to the vertical transfer circuit 2 existing between 2 vertical columns of 4 pixels in the color pixel group 6, and the horizontal row of pixels is added on the vertical transfer circuit 2. In addition, one vertical column of pixels is added on the vertical transfer circuit 2.
[0026]
Such addition on the image sensor 10 can be easily achieved by controlling the drive of charge transfer to the image sensor 10.
[0027]
In this way, at the stage of output as the output signal 5 on the image sensor 10, the four pixels in each color pixel group 6 are added, and the virtual addition pixel 8 corresponding to the luminance signal is directly output as the output signal 5. As a result, a monochrome image can be obtained immediately without performing any computation on the R, G, and B color components in each color pixel group 6.
[0028]
Note that the color image matrix in the image sensor 10 described above is configured as a Bayer array. However, the arrangement is not limited to this, and the arrangement of the color components R, G, and B pixels in each color pixel group 6 is arbitrary. Good.
[0029]
The above-described imaging device 10 has been described on the assumption of a CCD solid-state imaging device. However, the imaging device 10 is not limited to this, and is an amplification type solid-state imaging device that sweeps out charges accumulated in each pixel 1 by switching processing. Obviously, it is also applicable.
[0030]
Next, an electronic camera as an imaging apparatus using the imaging device 10 shown in FIG. 1 will be described with reference to FIGS.
FIG. 4 is a block diagram illustrating a configuration of an electronic camera using the image sensor 10. In FIG. 4, the image sensor 10 converts an image of a subject 28 input via the optical system 11 into an electrical signal. The optical system 11 has an infrared cut filter.
[0031]
The CDS / AGC circuit 12 performs A / D conversion on the output signal from the image sensor 10 by performing a CDS action for reducing noise components by correlated double sampling and an AGC action for performing automatic amplification according to sensitivity. Output to the device 13.
[0032]
The A / D converter 13 converts the analog signal from the CDS / AGC circuit 12 into a 10-bit digital signal and outputs it to a digital signal processing unit (DSP) 14. Needless to say, the A / D converter 13 may convert the analog signal from the CDS / AGC circuit 12 into a digital signal of 10 bits or more.
[0033]
The DSP 14 performs image interpolation processing, black level adjustment, gamma correction, knee correction, and the like on the input 10-bit digital data, and performs a matrix, an outline on the digital data converted from 10 bits to 8 bits. This is a signal processing circuit that performs processing such as correction and performs processing such as generation of 16-bit digital data composed of 8-bit luminance components and 8-bit color difference components, and is a one-chip LSI for digital signal processing. The DSP 14 also performs timing pulse generation processing for driving the image sensor 10.
[0034]
The compression / decompression unit 15 performs compression / decompression processing based on the JPEG standard, which is an international standard for still images. Specifically, logical processing such as discrete cosine transform (DCT), inverse DCT, Huffman encoding / decoding, etc. Is a one-chip decoder. In addition, the compression / decompression unit 15 performs data fetching and data access to the buffer memory 16 and also refreshes the buffer memory 16 composed of DRAM.
[0035]
The buffer memory 16 is a memory that temporarily holds one frame of image data before being compressed by the compression / decompression unit 15, and is configured by a DRAM as described above.
[0036]
The SRAM 22 adds header information as a JPEG file to the image data compressed by the compression / decompression unit 15 and has a function as a buffer memory before being stored in the flash memory 26.
[0037]
The flash memory 26 is a non-volatile memory that finally stores an image file that is a JPEG file to which header information is added.
[0038]
The external interface 27 is an interface for transferring data between an external processing device such as a personal computer and the electronic camera body.
[0039]
The digital encoder 17 is a chip that modulates digital data into an analog video signal.
[0040]
The display 18 is realized by an LCD or the like, and displays and outputs the video signal generated by the digital encoder 17.
[0041]
The speedlight unit 24 has a function of independently controlling external light control. That is, the speedlight unit 24 is controlled to emit light, charge, and the like by a CPU 21 to be described later, and the light emission control is performed by external light control by the speedlight unit alone.
[0042]
The LCD 23 displays various shooting modes, remaining frames, erase (erase), battery detection, and other states on a liquid crystal display.
[0043]
The CPU 21 performs overall control of the above-described units configured by, for example, a microprocessor.
[0044]
The timing generator 20 generates various pulses for driving the image sensor 10 and various timing pulses for the above-described units.
[0045]
The image sensor 10 is controlled by the driving timing pulse from the DSP 14 described above. A horizontal transfer pulse for horizontal charge transfer of the image sensor 10 directly drives the image sensor 10 from the DSP 14 via the timing generator 20. A vertical transfer pulse for vertical charge transfer is input to the timing generator 20, and drives the image sensor 10 by a voltage-converted signal via the drive unit 19.
[0046]
Here, the operation unit 25 includes a shooting mode changeover switch for switching various shooting modes and a command dial for setting various commands.
[0047]
That is, the shooting mode changeover switch of the operation unit 25 instructs to switch between the color image shooting mode (color mode) and the monochrome image shooting mode (monochrome mode) described above. When the color mode is instructed, the CPU 21 instructs the DSP 14 that the color mode has been set, and the DSP 14 applies the four pixels in each color pixel group 6 in the horizontal pixel group according to normal charge transfer control. Are sequentially transferred one by one to drive the image pickup device 10 and drive the image pickup device 10 and perform signal processing corresponding to the color image output on the output signal input via the A / D converter 13.
[0048]
The CPU 21 also performs instruction control for other units corresponding to the color mode. On the other hand, when the monochrome mode is instructed, the CPU 21 instructs the DSP 14 that the monochrome mode has been set, and the DSP 14 generates a driving timing pulse for adding four pixels in each color pixel group 6, While driving the image sensor, the output signal input through the A / D converter 13 is subjected to signal processing corresponding to the monochrome image output. The CPU 21 also performs instruction control for other units corresponding to the monochrome mode. However, in the monochrome mode, only the luminance signal Y needs to be processed, so the load on the processing of the DSP 14 and the subsequent processing of the signal output from the DSP 14 is smaller than that in the color mode.
[0049]
In this electronic camera, the color mode and the black and white mode are manually switched via the operation unit 25. However, the present invention is not limited to this, and automatic switching is performed based on the light amount detected by the light amount detection unit (not shown) by the CPU 21. You may make it perform control to perform. For example, when the shooting environment is too dark to obtain a sufficient amount of light, it is preferable to switch to the high-sensitivity black and white mode, and when the shooting environment is bright, switch to the color mode of light resolution.
[0050]
When the color temperature of the light source is different from the reference color temperature depending on the shooting environment, the color temperature may be corrected using a color correction filter. In this case, there is almost no influence on the processing load of color correction by the DSP 14 when the color mode is set.
[0051]
Here, FIG. 5 shows a plan view of the electronic camera described above. The electronic camera body 31 includes a photographing mode changeover switch 32, a command dial 33, an LCD 34, and a release switch 35. The shooting mode changeover switch 32, the command dial 33, and the release switch 35 are a part of the operation unit 25 in FIG. 4, and the LCD 34 corresponds to the LCD 23 in FIG. The shooting mode switch 32 is a switch for switching between a color mode and a monochrome mode, and the mode state is displayed in the LCD 34. The command dial 33 is a dial used for setting operations such as shutter speed and aperture value, and the setting result and the like are displayed in the LCD 34.
[0052]
【The invention's effect】
As described above in detail, in the first invention, since the output signals corresponding to the color image and the black and white image can be directly output by the same image sensor, the subsequent processing load and processing time can be reduced. In addition, the black-and-white image has an effect that a high-sensitivity signal output is obtained because charges are added to the monochrome image.
[0053]
Further, in the second to sixth inventions, since the image sensor according to the first invention is used, a color image can be obtained by the same image sensor, and calculation processing load and computation for outputting a black and white image The processing time can be shortened, and as a result, the continuous shutter interval for black and white images can be remarkably shortened, and a high-speed continuous shooting function for black and white images can be realized. For example, the shutter interval, which previously took 6 to 7 seconds, can be reduced to 2 seconds.
[0054]
Furthermore, since the black and white image is a high-sensitivity image, there is an effect that an appropriate image can be obtained even when the shooting environment is dark.
[Brief description of the drawings]
FIG. 1 is an explanatory diagram showing a configuration of an image sensor according to an embodiment of the present invention.
FIG. 2 is an explanatory diagram showing the principle of obtaining a monochrome image by adding color pixel matrices.
FIG. 3 is an explanatory diagram showing a black and white image by addition of a color pixel matrix.
4 is a block diagram showing a configuration of an electronic camera using the image sensor shown in FIG. 1. FIG.
5 is a schematic plan view of the electronic camera shown in FIG.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Pixel 2 Vertical transfer circuit 3 Horizontal transfer circuit 4 Output amplifier 5 Output signal 6 Color pixel group 8 Addition pixel 10 Image sensor 14 Digital signal processing part (DSP)
19 Drive unit 20 Timing generator 21 CPU
25 Operation unit

Claims (9)

Each pixel has a plurality of pixels arranged in a matrix with photoelectric conversion elements, and a is a natural number, out of 4a pixels adjacent vertically and horizontally, 2a pixel is green (G), a pixel is blue (B ), A color pixel matrix configured by mounting a red (R) color filter on the other a pixels,
When capturing a color image to be stored in a non-volatile memory, it is possible to sequentially read without adding charges of the 4a pixels adjacent to the top, bottom, left and right, and when obtaining a luminance signal in the color pixel matrix, the top and bottom An image pickup device characterized in that the charges of 4a pixels adjacent to the left and right are added and sequentially read out. Each pixel has a plurality of pixels arranged in a matrix with photoelectric conversion elements, and a is a natural number, out of 4a pixels adjacent vertically and horizontally, 2a pixel is green (G), a pixel is blue (B 4a, which has a color pixel matrix configured by mounting a red (R) color filter on the other a pixels, and is adjacent to the upper, lower, left, and right sides when capturing a color image to be stored in the nonvolatile memory. An image sensor that can sequentially read without adding the charges of the pixels, and can obtain the luminance signal in the color pixel matrix by sequentially adding the charges of the 4a pixels adjacent in the vertical and horizontal directions;
When capturing the color image to the image sensor, a drive signal is supplied for reading without adding the charges of the 4a pixels adjacent in the vertical and horizontal directions, and when obtaining the luminance signal, a drive circuit for supplying a drive signal to adding and reading charges of 4 pixels adjacent vertically and horizontally,
An imaging apparatus comprising: When m, n, and b are natural numbers, the color pixel matrix includes bm horizontal pixels and bn vertical pixels,
3. The luminance signal is obtained by adding and reading out charges of adjacent 2b pixels by the driving circuit and reading out horizontal m pixel signals and vertical n pixel signals. The imaging device described in 1. Means for switching between the luminance signal acquisition mode for reading out the horizontal m and vertical n pixel signals and the color image photographing mode for reading out without adding the horizontal 2m and vertical 2n pixels; The imaging apparatus according to claim 3. 3. The imaging apparatus according to claim 2, wherein when obtaining the luminance signal, the luminance signal is read out by setting a as a natural number and adding charges of the 4 a pixels adjacent in the vertical and horizontal directions.   When the color temperature of the light source differs from the reference color temperature of the light source depending on the shooting environment, by using a color correction filter, when a is a natural number, the ratio of each color signal of the 4a pixels adjacent to the top, bottom, left, and right The imaging apparatus according to claim 2, wherein correction is made and an accurate luminance signal can be read out. The spectral characteristics of the respective colors of the green (G), blue (B), and red (R) color filters indicate that the output of each color component of each color pixel is “R: G: B = 0.299: 0.587: The imaging device according to claim 1, wherein the imaging element is set to be 0.144 ”. The imaging apparatus according to claim 2, wherein a black and white image is captured by obtaining the luminance signal. The spectral characteristics of each color of the green (G), blue (B), and red (R) color filters of the image sensor are such that the output of each color component of the pixel of each color is “R: G: B = 0.299. The imaging apparatus according to claim 2, wherein the imaging apparatus is set to be “0.587: 0.144”.

Images