Picture reading method, program and device, principal color extracting method, program and device, and image dividing method, program and device Number:7,522,766 from the United States Patent and Trademark Office (PTO) owispatent

Title: Picture reading method, program and device, principal color extracting method, program and device, and image dividing method, program and device

Abstract: A method for reading a picture is provided which is capable of removing adverse effects caused by lighting conditions or a like. A principal color extracting section extracts principal colors from a color image input by an image inputting section based on a local peak value shown in a color histogram obtained from the color image and divides the color image into principal color images each corresponding to each of the extracted principal colors. A combination creating section creates a combined image by combining separated principal color images. A picture region extracting section extracts character region candidates from the principal color image and combined image. A picture recognizing section performs character recognizing processing on all extracted character region candidates to obtain the best character reading result according to a character recognizing confidence of character recognizing processing results.

Patent Number: 7,522,766 Issued on 04/21/2009 to Ishidera

---

Inventors:	Ishidera; Eiki (Tokyo, JP)
Assignee:	NEC Corporation (Tokyo, JP)
Appl. No.:	11/110,892
Filed:	April 21, 2005

---

Foreign Application Priority Data

---

Apr 22, 2004 [JP]			2004-126675

Current U.S. Class:	382/165 ; 382/162; 382/166; 382/173
Current International Class:	G06K 9/00 (20060101)
Field of Search:	382/166,165,173,181,162 345/473 348/652,654,E9.04 358/539 375/E7.166

---

References Cited [Referenced By]

---

U.S. Patent Documents

5787192	July 1998	Takaichi et al.

Foreign Patent Documents

	07-73313		Mar., 1995		JP
	2000-251016		Sep., 2000		JP
	2001-222680		Aug., 2001		JP
	2003-85535		Mar., 2003		JP

Primary Examiner: Do; Anh Hong
Attorney, Agent or Firm: Sughrue Mion, PLLC

---

Claims

---

What is claimed is:

1. A picture reading method comprising: extracting color components having a high frequency of occurrence in an input image, as principal colors using a principal color extracting section; separating the input image into images each corresponding to each of the extracted principal colors using an image separating section; and combining the separated images, out of the separated images, whose principal colors have specified relation with one another using an image combining section; and reading a picture based on the separated images and the combined images using a picture reading section.

2. The picture reading method according to claim 1, wherein the extraction of the principal colors is performed by separating two or more color components contained in a colorimetric system representing the input image into a specified number of color component zones, by using a center of a first color component zone, as a temporary principal color center, when a color component value within the first color component zone forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value, by calculating a mean value of the color components having a value representing similarity of a color between the color components that is larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center being used as a center and using the mean value as a principal color center, and by making each of the color components having a value representing similarity of a color between the color components that is larger than a predetermined third threshold value be contained in the principal colors, out of the color components being distributed with the principal color center being used as a center.

3. The picture reading method according to claim 1, wherein the separation of said images is performed by preparing an image for every extracted principal color, by judging as to whether a pixel for every prepared image input belongs to any one of the extracted principal colors, and by using as a first pixel value the pixel belonging to any one of the extracted principal colors, and on the other hand, by using as a second pixel value the pixel belonging to no one of the principal colors.

4. The picture reading method according to claim 1, wherein the reading of a picture is performed by extracting picture region candidates from the separated images and combined images, by performing picture recognizing processing on all the extracted picture region candidates, and by performing picture reading processing according to a result from said picture recognizing processing.

5. The picture reading method according to claim 4, wherein the extraction of picture region candidates is performed by excluding, from said picture region candidates, out of said picture region candidates in which pixel values representing the separated images and combined images occur continuously in horizontal and/or vertical directions, said picture region candidates in which the number of pixel values occurring continuously in horizontal and/or vertical directions is larger than a predetermined ratio of the number of pixels in the horizontal and vertical directions in the input image and by comparing the number of pixel values occurring continuously in the horizontal and/or vertical directions in coordinates serving as a center of the non-excluded picture region candidate with the number of the pixel occurring in the horizontal and vertical directions in the input image and by judging, based on results from the comparison, whether the non-excluded picture region candidate is used as said picture region candidates.

6. A picture reading method comprising: extracting color components having a high frequency of occurrence in an input image, as principal colors using a principal color extracting section; separating the input image into images each corresponding to each of the extracted principal colors using an image separating section; comparing a degree of difference in color among the principal colors making up each of the separated images using a comparing section; combining the separated images, out of the separated images, whose principal colors have specified relation with one another, based on a result from the comparison using an image combining section; and reading a picture based on the separated images and the combined images using a picture reading section.

7. The picture reading method according to claim 6, wherein the extraction of the principal colors is performed by separating two or more color components contained in a calorimetric system representing the input image into a specified number of color component zones, by using a center of a first color component zone, as a temporary principal color center, when a color component value within the first color component zone forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value, by calculating a mean value of the color components having a value representing similarity of a color between the color components that is larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center being used as a center and using the mean value as a principal color center, and by making each of the color components having a value representing similarity of a color between the color components that is larger than a predetermined third threshold value be contained in the principal colors, out of the color components being distributed with the principal color center being used as a center.

8. The picture reading method according to claim 6, wherein the separation of said images is performed by preparing an image for every extracted principal color, by judging as to whether a pixel for every prepared image input belongs to any one of the extracted principal colors, and by using as a first pixel value the pixel belonging to any one of the extracted principal colors, and on the other hand, by using as a second pixel value the pixel belonging to no one of the principal colors.

9. The picture reading method according to claim 6, wherein the reading of a picture is performed by extracting picture region candidates from the separated images and combined images, by performing picture recognizing processing on all the extracted picture region candidates, and by performing picture reading processing according to a result from said picture recognizing processing.

10. The picture reading method according to claim 9, wherein the extraction of picture region candidates is performed by excluding, from said picture region candidates, out of said picture region candidates in which pixel values representing the separated images and combined images occur continuously in horizontal and/or vertical directions, said picture region candidates in which the number of pixel values occurring continuously in horizontal and/or vertical directions is larger than a predetermined ratio of the number of pixels in the horizontal and vertical directions in the input image and by comparing the number of pixel values occurring continuously in the horizontal and/or vertical directions in coordinates serving as a center of the non-excluded picture region candidate with the number of the pixel occurring in the horizontal and vertical directions in the input image and by judging, based on results from the comparison, whether the non-excluded picture region candidate is used as said picture region candidates.

11. A principal color extracting method comprising: separating two or more color components contained in a colorimetric system representing an input image into a specified number of color component zones using a separating section; using a center of a first color component zone, as a temporary principal color center by a temporary principal color center determining section when a color component value within the first color component zone forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value; calculating a mean value of the color components, using a principal color center determining section, having a value representing similarity of a color between the color components that is larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center being used as a center and using the mean value as a principal color center, and extracting as principal colors, using an extracting section, the color components each having a value representing similarity of a color between the color components that is larger than a predetermined third threshold value, out of the color components being distributed with the principal color center being used as a center.

12. An image dividing method comprising: separating two or more color components contained in a colorimetric system representing an input image into a specified number of color component zones using a separating section; using a center of a first color component zone, as a temporary principal color center, by a temporary principal color center determining section, when a color component value within the first color component zone forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value; calculating a mean value of the color components, using a principal color center determining section, having a value representing similarity of a color between the color components that is larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center being used as a center and using the mean value as a principal color center; extracting as principal colors, using an extracting section, the color components each having a value representing similarity of a color between the color components being larger than a predetermined third threshold value, out of the color components being distributed with the principal color center being used as a center; preparing images each corresponding to each of the extracted principal colors using a preparing section, judging as to whether a pixel for every prepared image belongs to any one of the extracted principal colors using a judging section; and separating an input image into images, using a principal color image outputting section, each corresponding to each of principal colors by using as a first pixel value the pixel belonging to any one of the extracted principal colors, and on the other hand, by using as a second pixel value the pixel belonging to no one of the principal colors, as obtained from the result of the judgment.

13. A picture reading device comprising: a principal color extracting section to extract color components having a high frequency of occurrence in an input image, as principal colors; an image separating section to separate the input image into images each corresponding to each of the extracted principal colors extracted by said principal color extracting section; an image combining section to combine the separated images, out of the separated images obtained by the separating operation of said image separating section, whose principal colors have specified relation with one another; and a picture reading section to read a picture based on said images obtained by the separation by said image separating section and said images obtained by the combination by said image combining section.

14. The picture reading device according to claim 13, wherein said principal color extracting section comprises: a separating section to separate two or more color components contained in a colorimetric system representing the input image into a specified number of color component zones; a temporary principal color center determining section to determine a center of a first color component zone, as a temporary principal color center, when a color component value within the first color component zone obtained by operation of said separating section forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value; a principal color center determining section to calculate a mean value of the color components having a value representing similarity of a color between the color components that is larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center being used as a center and to determine the calculated mean value as a principal color center; and an extracting section to extract as principal colors the color components each having a value representing similarity of a color between the color components that is larger than a predetermined third threshold value, out of the color components being distributed with the principal color center determined by said temporary principal color center determining section being used as a center.

15. The picture reading device according to claim 13, wherein said image separating section comprises: an inputting section to input an image for every principal color extracted by said principal color extracting section; and a creating section to create an image made up of principal colors by judging as to whether a pixel for every image input from said inputting section belongs to any one of the extracted principal colors, and by using as a first pixel value the pixel belonging to any one of the extracted principal colors, and on the other hand, by using as a second pixel value the pixel belonging to no one of the principal colors.

16. The picture reading device according to claim 13, wherein said picture reading section comprises: a candidate extracting section to extract picture region candidates from said images obtained by the separation by said image separating section and the combined images obtained by the combination by said image combining section; a picture recognizing section to perform picture recognizing processing on all picture region candidates extracted by said candidate extracting section; and a reading section to read a picture based on a result from said picture recognizing processing obtained by the recognition by said picture recognizing section.

17. The picture reading device according to claim 16, wherein said candidate extracting section comprises: a candidate excluding section to exclude, from said picture region candidates, out of said picture region candidates in which pixel values representing said images obtained by the separation by said image separating section and said images obtained by the combination by said image combining section occur continuously in horizontal and/or vertical directions, said picture region candidates in which the number of pixel values occurring continuously in the horizontal and/or vertical directions is larger than a predetermined ratio of the number of pixels in the horizontal and vertical directions in the input image; a comparing section to compare the number of pixel values occurring continuously in the horizontal and/or vertical directions in coordinates serving as a center of the picture region candidate not having been excluded by said candidate excluding section with the number of pixels in the horizontal and vertical directions in the input image; and a judging section to judge, based on results from the comparison by said comparing section, whether said picture region candidates not having been excluded are used as said picture region candidates.

18. A picture reading device comprising: a principal color extracting section to extract color components having a high frequency of occurrence in an input image, as principal colors; an image separating section to separate the input image into images each corresponding to each of the principal colors extracted by said principal color extracting section; a comparing section to compare a degree of difference in color among the principal colors making up the images separated by said image separating section; an image combining section to combine the separated images, out of the separated images, whose principal colors have specified relation with one another, based on a result from the comparison by said comparing section; and a picture reading section to read a picture based on images obtained by the separating operation of said image separating section and said images obtained by the combination by said image combining section.

19. The picture reading device according to claim 18, wherein said principal color extracting section comprises: a separating section to separate two or more color components contained in a colorimetric system representing the input image into a specified number of color component zones; a temporary principal color center determining section to determine a center of a first color component zone, as a temporary principal color center, when a color component value within the first color component zone obtained by operation of said separating section forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value; a principal color center determining section to calculate a mean value of the color components having a value representing similarity of a color between the color components that is larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center being used as a center and to determine the calculated mean value as a principal color center; and an extracting section to extract as principal colors the color components each having a value representing similarity of a color between the color components that is larger than a predetermined third threshold value, out of the color components being distributed with the principal color center determined by said temporary principal color center determining section being used as a center.

20. The picture reading device according to claim 18, wherein said image separating section comprises: an inputting section to input an image for every principal color extracted by said principal color extracting section; and a creating section to create an image made up of principal colors by judging as to whether a pixel for every image input from said inputting section belongs to any one of the extracted principal colors, and by using as a first pixel value the pixel belonging to any one of the extracted principal colors, and on the other hand, by using as a second pixel value the pixel belonging to no one of the principal colors.

21. The picture reading device according to claim 18, wherein said picture reading section comprises: a candidate extracting section to extract picture region candidates from said images obtained by the separation by said image separating section and the combined images obtained by the combination by said image combining section; a picture recognizing section to perform picture recognizing processing on all picture region candidates extracted by said candidate extracting section; and a reading section to read a picture based on a result from said picture recognizing processing obtained by the recognition by said picture recognizing section.

22. The picture reading device according to claim 21, wherein said candidate extracting section comprises: a candidate excluding section to exclude, from said picture region candidates, out of said picture region candidates in which pixel values representing said images obtained by the separation by said image separating section and said images obtained by the combination by said image combining section occur continuously in horizontal and/or vertical directions, said picture region candidates in which the number of pixel values occurring continuously in the horizontal and/or vertical directions is larger than a predetermined ratio of the number of pixels in the horizontal and vertical directions in the input image; a comparing section to compare the number of pixel values occurring continuously in the horizontal and/or vertical directions in coordinates serving as a center of the picture region candidate not having been excluded by said candidate excluding section with the number of pixels in the horizontal and vertical directions in the input image; and a judging section to judge, based on results from the comparison by said comparing section, whether said picture region candidates not having been excluded are used as said picture region candidates.

23. A principal color extracting device comprising: a separating section to separate two or more color components contained in a colorimetric system representing an input image into a specified number of color component zones; a temporary principal color center determining section to determine a center of a first color component zone, as a temporary principal color center, when a color component value within the first color component zone obtained by operation of said separating section forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value; a principal color center determining section to calculate a mean value of the color components having a value representing similarity of a color between the color components that is larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center determined by said temporary principal color center determining section being used as a center and to determine the calculated mean value as a principal color center; and an extracting section to extract, as principal colors, the color components each having a value representing similarity of a color between the color components that is larger than a predetermined third threshold value, out of the color components being distributed with the principal color center determined by said temporary principal color center determining section being used as a center.

24. An image dividing device comprising: a separating section to separate two or more color components contained in a colorimetric system representing the input image into a specified number of color component zones; a temporary principal color center determining section to determine a center of a first color component zone, as a temporary principal color center, when a color component value within the first color component zone obtained by operation of said separating section forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value; a principal color center determining section to calculate a mean value of the color components having a value representing similarity of a color between the color components being larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center determined by said temporary principal color center determining section being used as a center and to determine the calculated mean value as a principal color center; an extracting section to extract, as principal colors, the color components each having a value representing similarity of a color between the color components that is larger than a predetermined third threshold value, out of the color components being distributed with the principal color center determined by said temporary principal color center determining section being used as a center; an inputting section to input an image for every said principal color extracted by said extracting section; and a principal color image outputting section to output an image made up of principal colors by judging as to whether a pixel for every image input from said inputting section belongs to any one of the extracted principal colors, and by using as a first pixel value the pixel belonging to any one of the extracted principal colors, and on the other hand, by using as a second pixel value the pixel belonging to no one of the principal colors.

25. A computer readable medium storing a controlling program to make a computer execute a picture reading method, the method comprising: extracting color components having a high frequency of occurrence in an input image, as principal colors using an extracting section; separating the input image into images each corresponding to each of the extracted principal colors using a separating section; and combining the separated images, out of the separated images, whose principal colors have specified relation with one another using an image combining section; and reading a picture based on the separated images using a picture reading section and the combined images.

26. A computer readable medium storing a controlling program to make a computer execute a picture reading method, said method comprising: extracting color components having a high frequency of occurrence in an input image, as principal colors using a principal color extracting section; separating the input image into images each corresponding to each of the extracted principal colors using an image separating section; comparing a degree of difference in color among the principal colors making up each of the separated images using a comparing section; combining the separated images, out of the separated images, whose principal colors have specified relation with one another, based on a result from the comparison using an image combining section; and reading a picture based on the separated images and the combined images using a picture reading section.

27. A computer readable medium storing a controlling program to make a computer execute a principal color extracting method, said method comprising: separating two or more color components contained in a colorimetric system representing an input image into a specified number of color component zones using a separating section; using a center of a first color component zone, as a temporary principal color center by a temporary principal color center determining section, when a color component value within the first color component zone forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value; calculating a mean value of the color components, using a principal color center determining section, having a value representing similarity of a color between the color components that is larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center being used as a center and using the mean value as a principal color center, and extracting as principal colors, using an extracting section, the color components each having a value representing similarity of a color between the color components that is lamer than a predetermined third threshold value, out of the color components being distributed with the principal color center being used as a center.

28. A computer readable medium storing a controlling program to make a computer execute a image dividing method, said method comprising: separating two or more color components contained in a colorimetric system representing an input image into a specified number of color component zones using a separating section; using a center of a first color component zone, as a temporary principal color center by a temporary principal color center determining section, when a color component value within the first color component zone forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value; calculating a mean value of the color components, using a principal color center determining section, having a value representing similarity of a color between the color components that is larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center being used as a center and using the mean value as a principal color center; extracting as principal colors, using an extracting section, the color components each having a value representing similarity of a color between the color components being larger than a predetermined third threshold value, out of the color components being distributed with the principal color center being used as a center; preparing images each corresponding to each of the extracted principal colors using a preparing section, judging as to whether a pixel for every prepared image belongs to any one of the extracted principal colors using a judging section; and separating an input image into images, using a principal color image outputting section, each corresponding to each of principal colors by using as a first pixel value the pixel belonging to any one of the extracted principal colors, and on the other hand, by using as a second pixel value the pixel belonging to no one of the principal colors, as obtained from the result of the judgment.

29. A computer readable medium storing a controlling program to make a computer serve as a picture reading device, said device comprising: a principal color extracting section to extract color components having a high frequency of occurrence in an input image, as principal colors; an image separating section to separate the input image into images each corresponding to each of the extracted principal colors extracted by said principal color extracting section; an image combining section to combine the separated images, out of the separated images obtained by the separating operation of said image separating section, whose principal colors have specified relation with one another; and a picture reading section to read a picture based on said images obtained by the separation by said image separating section and said images obtained by the combination by said image combining section.

30. The computer readable medium of claim 29, wherein said image separating section comprises: an inputting section to input an image for every principal color extracted by said principal color extracting section; and a creating section to create an image made up of principal colors by judging as to whether a pixel for every image input from said inputting section belongs to any one of the extracted principal colors, and by using as a first pixel value the pixel belonging to any one of the extracted principal colors, and on the other hand, by using as a second pixel value the pixel belonging to no one of the principal colors.

31. The computer readable medium of claim 29, wherein said picture reading section comprises: a candidate extracting section to extract picture region candidates from said images obtained by the separation by said image separating section and the combined images obtained by the combination by said image combining section; a picture recognizing section to perform picture recognizing processing on all picture region candidates extracted by said candidate extracting section; and a reading section to read a picture based on a result from said picture recognizing processing obtained by the recognition by said picture recognizing section.

32. A computer readable medium storing a controlling program to make a computer serve as a picture reading device, the device comprising: a principal color extracting section to extract color components having a high frequency of occurrence in an input image, as principal colors; an image separating section to separate the input image into images each corresponding to each of the principal colors extracted by said principal color extracting section; a comparing section to compare a degree of difference in color among the principal colors making up the images separated by said image separating section; an image combining section to combine the separated images, out of the separated images, whose principal colors have specified relation with one another, based on a result from the comparison by said comparing section; and a picture reading section to read a picture based on images obtained by the separating operation of said image separating section and said images obtained by the combination by said image combining section.

---

Description

---

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a picture reading method, program and device, a principal color extracting method, program and device, and an image dividing method, program and device, each being capable of reading pictures in an input image with high accuracy.

The present application claims priority of Japanese Patent Application No. 2004-126675 filed on Apr. 22, 2004, which is hereby incorporated by reference.

2. Description of the Related Art

Conventionally, various types of picture readers (picture reading devices) are proposed that automatically recognize pictures contained in an image picked up by an image pickup device, for example, a CCD (Charge-Coupled Device) camera or a like. As examples, a "number plate recognizing device" is disclosed in Japanese Patent Application Laid-open No. 2000-251016 and a "number reader (number reading device)" is disclosed in Japanese Patent Application Laid-open No. 2001-222680. These devices employ a method of recognizing a character being a picture by using a gray-scale image.

In Japanese Patent Application Laid-open No. 2003-85535, a "position recognizing method for a road guiding sign" is disclosed. In this method, a component of a color registered in advance is extracted from a color image of the road guiding sign and a character region is then extracted by the extraction of the component of the color to perform character recognizing processing.

Furthermore, an "image separating method and its device" is disclosed in Japanese Patent Application Laid-open No. Hei 07-73313. In the disclosed technology, an image region is over-separated according to a color and the over-separated regions are combined into character and background regions so that the character region is extracted in a stable manner without being affected by a shadow and/or a glare (highlight).

An image of a subject to be picked up by an image pickup device is multifarious. For example, there is a great difference in a displaying method between a number plate to display a registration number of a vehicle being employed in Japan (simply called a number plate) and a license plate to display a registration number of a vehicle being employed in the North America (simply called a license plate). The displaying method of the number plate is comparatively simple, however, the displaying method of the license plate is complicated when compared with the number plate. More specifically, in the case of the displaying method of the license plate, a design and/or format including a color to be used for its background are diversified and a font and its color to be used for a character are also various.

For example, FIG. 23 is an example of an image of a license plate in which a word "California" is written in a red color in its upper portion and characters "123" representing a license number are written in a bluish green color in its center portion. As shown in FIG. 23, there is a background portion, in which gradation processing is performed using a dark blue color in an upper half of the license plate, being changed gradually to be a white color toward a center portion from its upper portion and there is a background portion, in which gradation processing is performed using a red color in a lower half of the license plate, being changed gradually to be a white color toward a center from its lower portion.

FIG. 24 is a diagram showing an example of an image obtained by displaying the color image of FIG. 23 on a gray scale. In this case, since color gradation processing is performed in the character portion, in some cases, pixel values in the character regions are not always a maximum or a minimum in an input image.

As a result, the above conventional technologies have technical problems. That is, even when threshold processing on a pixel value to binarize the pixel is performed on an image represented on a gray scale by using the technology disclosed in the Japanese Patent Application Laid-open Nos. 2000-251016 and 2001-222680 and a character region is extracted using the above pixel, the character region such as "California" and/or "123" cannot be extracted with high accuracy.

Due to these technological problems, when the conventional method using the gray-scale image is used, the character region cannot be separated accurately from its background in the license plate and, therefore, accurate and automatic reading of a license number is difficult.

As one technological means to solve these problems, a method may be provided in which a character region is obtained by extracting a region having a color corresponding to a license number from a color license plate image acquired by picking up the license plate image using a CCD camera.

For example, as one of the technologies described above, by using the method disclosed in the Japanese Patent Application Laid-open No. 2003-855353, a method is available in which a component of a color registered in advance is extracted from a color image of a license plate and, by using the extracted component of a color, a character region is extracted and character recognizing processing is performed.

However, the technology disclosed in the Japanese Patent Application Laid-open No. 2003-855353 has a following problem. That is, when character recognizing processing is performed, by using the technology disclosed in the Japanese Patent Application Laid-open No. 2003-855353, on a license plate which employs very diversified colors, it is difficult to register, in advance, colors that may be used for the character. As a result, when the method for obtaining a character region by extracting colors registered in advance is used, the character region and background region can not be accurately separated from the license plate and automatic reading of a license number is difficult.

Furthermore, a factor that interferes with successful separation of a character region and background region is that, when the technology disclosed in the Japanese Patent Application Laid-open No. 2003-855353 is used outside, lighting conditions differ depending on environments in which an image is picked up during daytime, in evening, or at night and, therefore, even if an image of the same plate is picked up, color information is not uniform depending on the environment in which the image is picked up. For example, in the case where a license plate has a sky blue background and dark blue characters, when an image of the license plate is picked up in a state in which the license plate is lighted by the afternoon sun, a red component in an entire image becomes very intense and, therefore, when the image of the license is picked up during the daytime, even in the region in which a blue component becomes most intense, the red component becomes most intense in some cases.

Therefore, accurate extraction of a character region is difficult only by using the technology disclosed in the Japanese Patent Application Laid-open No. 2003-855353, which makes it difficult to automatically read a license number with high accuracy. That is, the above technology presents a problem in that, when a character region is extracted from a license plate, what color is used for a character in the character region is unknown and, therefore, it is difficult to extract a character region in a stable manner by using the method in which a component of a color having been registered in advance is extracted from a color image. As described above, in an outdoor environment, color components making up a character region are separated into a plurality of color components due to adverse effects caused by shadow areas or glare (highlight) areas on the license plate, which makes it more difficult to achieve stable extraction of a character region.

To solve such technological problems as described above, such technology disclosed in Japanese Patent Application Laid-open No. Hei 07-73313 is available. In the disclosed technology, by over-dividing a region of an image based on a color and combining the over-separated region, a character region can be extracted in a stable manner without being affected by shadow areas and/or glare (highlight) areas. However, in the technology disclosed in the Japanese Patent Application Laid-open No. Hei 07-73313, in the case of an image of a subject to be picked up which is made up of a color and shape having been defined in advance such as a road sign, it is possible to judge whether or not a color and shape is to be combined by comparing the color and shape with those contained in a dictionary stored in advance; however, when the above technology is applied to a license plate having a variety of colors and designs, it is necessary that an enormous amount of data about colors and shapes has to be stored in advance, which is not realistic.

Moreover, a region to be combined according to the technology disclosed in the Japanese Patent Application Laid-open No. Hei 07-73313 is not always a character region which is desired to be extracted. That is, there is a problem that, whether or not a character is extracted exactly is not judged substantially unless whether or not an extracted object is recognized as a character. Therefore, the above technology has a problem in that, since only a region extracted according to a reference which does not always coincide with a reference for character recognition defined in the above technology is extracted as a candidate for a character, accurate extraction of a character region is difficult. Therefore, a technological problem remains still unsolved in that accurate and automatic reading of a license number is difficult.

SUMMARY OF THE INVENTION

In view of the above, it is an object of the present invention to provide a picture reading method, program and device, a principal color extracting method, program and device, and an image dividing method, program and device, each being capable of reading pictures in an input image with high accuracy by extracting a picture region, in a stable manner, in an object to be read which has various colors, designs, or a like.

According to a first aspect of the present invention, there is provided a picture reading method including:

a step of extracting color components having a high frequency of occurrence in an input image, as principal colors;

a step of separating the input image into images each corresponding to each of the extracted principal colors; and

a step of combining the separated images, out of the separated images, whose principal colors have specified relation with one another; and

a step of reading a picture based on the separated images and the combined images.

According to a second aspect of the present invention, there is provided a picture reading method including:

a step of extracting color components having a high frequency of occurrence in an input image, as principal colors;

a step of separating the input image into images each corresponding to each of the extracted principal colors;

a step of comparing a degree of difference in color among the principal colors making up each of the separated images;

a step of combining the separated images, out of the separated images, whose principal colors have specified relation with one another, based on a result from the comparison; and

a step of reading a picture based on the separated images and the combined images.

In the foregoing first and second aspects, a preferable mode is one wherein the extraction of the principal colors is performed by separating two or more color components contained in a calorimetric system representing the input image into a specified number of color component zones, by using a center of a first color component zone, as a temporary principal color center, when a color component value within the first color component zone forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value, by calculating a mean value of the color components having a value representing similarity of a color between the color components that is larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center being used as a center and using the mean value as a principal color center, and by making each of the color components having a value representing similarity of a color between the color components that is larger than a predetermined third threshold value be contained in the principal colors, out of the color components being distributed with the principal color center being used as a center.

Another preferable mode is one wherein the separation of said images is performed by preparing an image for every extracted principal color, by judging as to whether a pixel for every prepared image input belongs to any one of the extracted principal colors, and by using as a first pixel value the pixel belonging to any one of the extracted principal colors, and on the other hand, by using as a second pixel value the pixel belonging to no one of the principal colors.

Still another preferable mode is one wherein the reading of a picture is performed by extracting picture region candidates from the separated images and combined images, by performing picture recognizing processing on all the extracted picture region candidates, and by performing picture reading processing according to a result from said picture recognizing processing.

A further preferable mode is one wherein the extraction of picture region candidates is performed by excluding, from said picture region candidates, out of said picture region candidates in which pixel values representing the separated images and combined images occur continuously in horizontal and/or vertical directions, said picture region candidates in which the number of pixel values occurring continuously in horizontal and/or vertical directions is larger than a predetermined ratio of the number of pixels in the horizontal and vertical directions in the input image and by comparing the number of pixel values occurring continuously in the horizontal and/or vertical directions in coordinates serving as a center of the non-excluded picture region candidate with the number of the pixel occurring in the horizontal and vertical directions in the input image and by judging, based on results from the comparison, whether the non-excluded picture region candidate is used as said picture region candidates.

According to a third aspect of the present invention, there is provided a principal color extracting method comprising:

a step of separating two or more color components contained in a calorimetric system representing an input image into a specified number of color component zones;

a step of using a center of a first color component zone, as a temporary principal color center, when a color component value within the first color component zone forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value;

a step of calculating a mean value of the color components having a value representing similarity of a color between the color components that is larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center being used as a center and using the mean value as a principal color center, and

a step of extracting as principal colors the color components each having a value representing similarity of a color between the color components that is larger than a predetermined third threshold value, out of the color components being distributed with the principal color center being used as a center.

According to a fourth aspect of the present invention, there is provided an image dividing method including:

a step of separating two or more color components contained in a colorimetric system representing an input image into a specified number of color component zones;

a step of using a center of a first color component zone, as a temporary principal color center, when a color component value within the first color component zone forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value;

a step of calculating a mean value of the color components having a value representing similarity of a color between the color components that is larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center being used as a center and using the mean value as a principal color center;

a step of extracting as principal colors the color components each having a value representing similarity of a color between the color components being larger than a predetermined third threshold value, out of the color components being distributed with the principal color center being used as a center;

a step of preparing images each corresponding to each of the extracted principal colors,

a step of judging as to whether a pixel for every prepared image belongs to any one of the extracted principal colors; and

a step of separating an input image into images each corresponding to each of principal colors by using as a first pixel value the pixel belonging to any one of the extracted principal colors, and on the other hand, by using as a second pixel value the pixel belonging to no one of the principal colors, as obtained from the result of the judgment.

According to a fifth aspect of the present invention, there is provided a picture reading device including:

a principal color extracting section to extract color components having a high frequency of occurrence in an input image, as principal colors;

an image separating section to separate the input image into images each corresponding to each of the extracted principal colors extracted by said principal color extracting section;

an image combining section to combine the separated images, out of the separated images obtained by the separating operation of said image separating section, whose principal colors have specified relation with one another; and

a picture reading section to read a picture based on said images obtained by the separation by said image separating section and said images obtained by the combination by said image combining section.

According to a sixth aspect of the present invention, there is provided a picture reading device including:

a principal color extracting section to extract color components having a high frequency of occurrence in an input image, as principal colors;

an image separating section to separate the input image into images each corresponding to each of the principal colors extracted by said principal color extracting section;

a comparing section to compare a degree of difference in color among the principal colors making up the images separated by said image separating section;

an image combining section to combine the separated images, out of the separated images, whose principal colors have specified relation with one another, based on a result from the comparison by said comparing section; and

a picture reading section to read a picture based on images obtained by the separating operation of said image separating section and said images obtained by the combination by said image combining section.

In the foregoing fifth and sixth aspects, a preferable mode is one wherein said principal color extracting section comprises:

a separating section to separate two or more color components contained in a calorimetric system representing the input image into a specified number of color component zones;

a temporary principal color center determining section to determine a center of a first color component zone, as a temporary principal color center, when a color component value within the first color component zone obtained by operation of said separating section forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value;

a principal color center determining section to calculate a mean value of the color components having a value representing similarity of a color between the color components that is larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center being used as a center and to determine the calculated mean value as a principal color center; and

an extracting section to extract as principal colors the color components each having a value representing similarity of a color between the color components that is larger than a predetermined third threshold value, out of the color components being distributed with the principal color center determined by said temporary principal color center determining section being used as a center.

Another preferable mode is one wherein said image separating section comprises:

an inputting section to input an image for every principal color extracted by said principal color extracting section; and

a creating section to create an image made up of principal colors by judging as to whether a pixel for every image input from said inputting section belongs to any one of the extracted principal colors, and by using as a first pixel value the pixel belonging to any one of the extracted principal colors, and on the other hand, by using as a second pixel value the pixel belonging to no one of the principal colors.

A further preferable mode is one wherein said picture reading section comprises:

a candidate extracting section to extract picture region candidates from said images obtained by the separation by said image separating section and the combined images obtained by the combination by said image combining section;

a picture recognizing section to perform picture recognizing processing on all picture region candidates extracted by said candidate extracting section; and

a reading section to read a picture based on a result from said picture recognizing processing obtained by the recognition by said picture recognizing section.

A still further preferable mode is one said candidate extracting section comprises:

a candidate excluding section to exclude, from said picture region candidates, out of said picture region candidates in which pixel values representing said images obtained by the separation by said image separating section and said images obtained by the combination by said image combining section occur continuously in horizontal and/or vertical directions, said picture region candidates in which the number of pixel values occurring continuously in the horizontal and/or vertical directions is larger than a predetermined ratio of the number of pixels in the horizontal and vertical directions in the input image;

a comparing section to compare the number of pixel values occurring continuously in the horizontal and/or vertical directions in coordinates serving as a center of the picture region candidate not having been excluded by said candidate excluding section with the number of pixels in the horizontal and vertical directions in the input image; and

a judging section to judge, based on results from the comparison by said comparing section, whether said picture region candidates not having been excluded are used as said picture region candidates.

According to a seventh aspect of the present invention, there is provided a principal color extracting device including:

a separating section to separate two or more color components contained in a calorimetric system representing an input image into a specified number of color component zones;

a temporary principal color center determining section to determine a center of a first color component zone, as a temporary principal color center, when a color component value within the first color component zone obtained by operation of said separating section forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value;

a principal color center determining section to calculate a mean value of the color components having a value representing similarity of a color between the color components that is larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center determined by said temporary principal color center determining section being used as a center and to determine the calculated mean value as a principal color center; and

an extracting section to extract, as principal colors, the color components each having a value representing similarity of a color between the color components that is larger than a predetermined third threshold value, out of the color components being distributed with the principal color center determined by said temporary principal color center determining section being used as a center.

According to an eighth aspect of the present invention, there is provided an image dividing device including:

a separating section to separate two or more color components contained in a colorimetric system representing the input image into a specified number of color component zones;

a temporary principal color center determining section to determine a center of a first color component zone, as a temporary principal color center, when a color component value within the first color component zone obtained by operation of said separating section forms a peak against color component values within second color component zones being adjacent to the first color component zone and exceeds a first pre-determined threshold value;

a principal color center determining section to calculate a mean value of the color components having a value representing similarity of a color between the color components being larger than a predetermined second threshold value, out of the color components being distributed with the temporary principal color center determined by said temporary principal color center determining section being used as a center and to determine the calculated mean value as a principal color center;

an extracting section to extract, as principal colors, the color components each having a value representing similarity of a color between the color components that is larger than a predetermined third threshold value, out of the color components being distributed with the principal color center deter