Image decoding technique for suppressing tile boundary distortion Number:7,522,778 from the United States Patent and Trademark Office (PTO) owispatent An image decoding device for decoding a hierarchically encoded compressed code obtained by dividing an image into a plurality of tiles and performing discrete wavelet transform on the pixel values of the image tile by tile includes a tile boundary smoothing part that performs smoothing of tile boundary distortion on the image after the decoding by application of a low-pass filter. The tile boundary smoothing part controls the degree of smoothing of the low-pass filter according to the ratio of the decoding quantity to the entire quantity of the compressed code. The decoding quantity is the portion of the compressed code which portion is to be decoded.<H suppressing, distortion, Image, decoding, , 7522778.html, Patent, pto, 7522778

Title: Image decoding technique for suppressing tile boundary distortion

Abstract: An image decoding device for decoding a hierarchically encoded compressed code obtained by dividing an image into a plurality of tiles and performing discrete wavelet transform on the pixel values of the image tile by tile includes a tile boundary smoothing part that performs smoothing of tile boundary distortion on the image after the decoding by application of a low-pass filter. The tile boundary smoothing part controls the degree of smoothing of the low-pass filter according to the ratio of the decoding quantity to the entire quantity of the compressed code. The decoding quantity is the portion of the compressed code which portion is to be decoded.

Patent Number: 7,522,778 Issued on 04/21/2009 to Suino, et al.

Inventors: Suino; Tooru (Kanagawa, JP), Sakuyama; Hiroyuki (Tokyo, JP)
Assignee: Ricoh Company, Ltd. (Tokyo, JP)

Appl. No.: 11/980,580

Filed: October 31, 2007

Related U.S. Patent Documents


Application Number	Filing Date	Patent Number	Issue Date
10600333	Jun., 2003	7330596

Foreign Application Priority Data


Jul 17, 2002 [JP]			2002-208107
Jul 17, 2002 [JP]			2002-208156
Sep 13, 2002 [JP]			2002-267692

Current U.S. Class: 382/240 ; 382/232; 382/233; 382/236; 382/245; 382/246

Current International Class: G06K 9/36 (20060101); G06K 9/46 (20060101)

Field of Search: 382/232,233,236,238,239,240,245,246,248,250,251,275

References Cited [Referenced By]

U.S. Patent Documents


5166810	November 1992	Sorimachi et al.
5214741	May 1993	Akamine et al.
5224062	June 1993	McMillan et al.
5625714	April 1997	Fukuda
5787204	July 1998	Fukuda
6137595	October 2000	Sakuyama et al.
6226011	May 2001	Sakuyama et al.
6859558	February 2005	Hong
6904096	June 2005	Kobayashi et al.
7013049	March 2006	Sakuyama
7016546	March 2006	Fukuhara et al.
7027661	April 2006	Estevez et al.
7136538	November 2006	Kitagawa
7174091	February 2007	Umeda
7330596	February 2008	Suino et al.
2003/0007186	January 2003	Suino et al.
2003/0218776	November 2003	Morimoto et al.

Foreign Patent Documents


58-125171	Jul., 1983	JP
5-227518	Sep., 1993	JP
5-316361	Nov., 1993	JP
9-307855	Nov., 1997	JP
10-191335	Jul., 1998	JP
2839987	Dec., 1998	JP
11-98505	Apr., 1999	JP
2000-115542	Apr., 2000	JP
2001-245294	Sep., 2001	JP
2001-346208	Dec., 2001	JP
2001-359048	Dec., 2001	JP

Other References

JX. Wei, et al., SPIE, SPIE Visual Communications and Image Processing, vol. 4067, pp. 1290-1295, "A New Method for Reducing Boundary Artifacts in Block-Based Wavelet Image Compression", Jun. 20-23, 2000. cited by other.

Primary Examiner: Bella; Matthew C
Assistant Examiner: Bayat; Ali
Attorney, Agent or Firm: Oblon, Spivak, McClelland, Maier & Neustadt, P.C.

Parent Case Text

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of and is based upon and claims the benefit of priority under 35 U.S.C. .sctn.120 for U.S. Ser. No. 10/600,333, filed Jun. 23, 2003, now U.S. Pat. No. 7,330,596 and claims the benefit of priority under 35 U.S.C. .sctn. 119 from Japanese Patent Application No. 2002-208107, filed Jul. 17, 2002, Japanese Patent Application No. 2002-208156, filed Jul. 17, 2002, and Japanese Patent Application No. 2002-267692, filed Sep. 13, 2002, the entire contents of each which are incorporated herein by reference.

Claims

What is claimed is:

1. An image decoder successively decoding a plurality of frames of an image, the frames each being hierarchically compressed and encoded through discrete wavelet transform on pixel values of each of tiles into which the frame is divided, the image decoder comprising: a tile boundary smoothing part configured to smooth a distortion of a tile boundary in each of the decoded frames; a mode selection part configured to select one of a first mode giving priority to image quality and a second mode giving priority to processing speed in the smoothing of the distortion of the tile boundary by the tile boundary smoothing part; and a tile boundary smoothing switching part configured to switch between the first mode and the second mode based on the selection by the mode selection part in the smoothing of the distortion of the tile boundary by the tile boundary smoothing part.

2. The image decoder as claimed in claim 1, wherein the tile boundary smoothing part is configured to perform low-pass filtering on a peripheral pixel of the tile boundary in each of the decoded frames in the second mode.

3. The image decoder as claimed in claim 1, wherein the tile boundary smoothing part is configured to perform low-pass filtering on a peripheral pixel of the tile boundary in each of the decoded frames based on a distance between the peripheral pixel and the tile boundary and an edge amount of the peripheral pixel in the first mode.

4. The image decoder as claimed in claim 3, wherein the edge amount of the peripheral pixel of the tile boundary is calculated in a diagonal direction.

5. The image decoder as claimed in claim 1, wherein the tile boundary smoothing part is configured to perform low-pass filtering on a peripheral pixel of the tile boundary in each of the decoded frames based on a distance between the peripheral pixel and the tile boundary, an edge amount of the peripheral pixel, and a ratio of a quantity of decoded codes to a quantity of all codes in the first mode.

6. The image decoder as claimed in claim 5, wherein the edge amount of the peripheral pixel of the tile boundary is calculated in a diagonal direction.

7. An image decoding method successively decoding a plurality of frames of an image, the frames each being hierarchically compressed and encoded through discrete wavelet transform on pixel values of each of tiles into which the frame is divided, the image decoding method comprising: using a processor to perform the steps of: smoothing a distortion of a tile boundary in each of the decoded frames, wherein one of a first mode giving priority to image quality and a second mode giving priority to processing speed is selected in the smoothing of the distortion of the tile boundary; and a processing mode is switched between the first mode and the second mode based on said selection in the smoothing of the distortion of the tile boundary.

8. The image decoding method as claimed in claim 7, wherein the low-pass filtering is performed on a peripheral pixel of the tile boundary in each of the decoded frames in the second mode.

9. The image decoding method as claimed in claim 7, wherein low-pass filtering is performed on a peripheral pixel of the tile boundary in each of the decoded frames based on a distance between the peripheral pixel and the tile boundary and an edge amount of the peripheral pixel in the first mode.

10. The image decoding method as claimed in claim 9, wherein the edge amount of the peripheral pixel of the tile boundary is calculated in a diagonal direction.

11. The image decoding method as claimed in claim 7, wherein low-pass filtering is performed on a peripheral pixel of the tile boundary in each of the decoded frames based on a distance between the peripheral pixel and the tile boundary, an edge amount of the peripheral pixel, and a ratio of a quantity of decoded codes to a quantity of all codes in the first mode.

12. The image decoding method as claimed in claim 11, wherein the edge amount of the peripheral pixel of the tile boundary is calculated in a diagonal direction.

13. A computer-readable recording medium containing a program for causing a computer to execute the image decoding method as set forth in claim 7.

14. The computer-readable recording medium as claimed in claim 13, wherein the low-pass filtering is performed on a peripheral pixel of the tile boundary in each of the decoded frames in the second mode.

15. The computer-readable recording medium as claimed in claim 13, wherein low-pass filtering is performed on a peripheral pixel of the tile boundary in each of the decoded frames based on a distance between the peripheral pixel and the tile boundary and an edge amount of the peripheral pixel in the first mode.

16. The computer-readable recording medium as claimed in claim 15, wherein the edge amount of the peripheral pixel of the tile boundary is calculated in a diagonal direction.

17. The computer-readable recording medium as claimed in claim 13, wherein low-pass filtering is performed on a peripheral pixel of the tile boundary in each of the decoded frames based on a distance between the peripheral pixel and the tile boundary, an edge amount of the peripheral pixel, and a ratio of a quantity of decoded codes to a quantity of all codes in the first mode.

18. The computer-readable recording medium as claimed in claim 17, wherein the edge amount of the peripheral pixel of the tile boundary is calculated in a diagonal direction.

Description

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally relates to image processing techniques for controlling the distortion of an inter-unit boundary in the case of decompressing, unit by unit, a compressed image divided into predetermined units, and more particularly to an image decoding device, an image processing apparatus, and an image processing method suitable for suppressing a tile boundary distortion on an image decompressed after being compressed according to JPEG2000. Such an image decoding device may be employed in personal computers (PCs), personal digital assistants (PDAs), mobile phones, and digital cameras, for example, which are apparatuses for handling images such as Internet-based images, medical images, and satellite communication images. Such an image processing apparatus may be a personal computer, a PDA, a mobile phone, or a digital camera, for example, using such an image decoding device.

2. Description of the Related Art

In many cases, image data is temporarily compressed to be stored or transmitted, and the compressed image data is decompressed and output to be processed whenever or wherever necessary. For image data compression, an image compression method such as JPEG, which divides an image into predetermined division units called blocks and encodes the image using discrete cosine transform for each block, is widely used. Such an image compression method has a problem in that when the compressed image is decompressed, the decompressed image includes block boundary distortion. There are well-known methods that detect the block boundary distortion and perform low-pass filtering on the pixels of the block boundaries of the decompressed image in order to make the block boundary distortion less conspicuous. Such methods are disclosed in Japanese Laid-Open Patent Applications No. 5-316361 and No. 9-307855 and Japanese Patent No. 2839987, for instance.

Recently, improvements in image input and output technologies have greatly increased demand for high-definition images. In the case of digital cameras as image input apparatuses, for instance, high-performance charge coupled devices (CCDs) having 3,000,000 pixels or more have been reduced in price to be widely used in digital cameras in a popular price range. It is expected that products employing CCDs having 5,000,000 pixels or more will be commercially available in the near future. It is expected that this trend toward an increasing number of pixels will continue for a while.

On the other hand, there have also been remarkable developments in the high-definition property and significant progress in the price reduction of image output apparatuses and image display apparatuses such as hard-copy apparatuses including laser printers, ink-jet printers, and sublimation-type printers, and soft-copy apparatuses including flat panel displays made of CRTs, liquid crystal displays (LCDs), and plasma display panels (PDPs).

Due to the introduction of these high-performance, inexpensive image input and output apparatuses to the market, high-definition images have become popular. As a result, it is expected that there will be an increasing demand for high-definition images in various fields in the future. Actually, the developments in technologies related to PCs and networks including the Internet have accelerated such trends at an increasing rate. Particularly in recent years, mobile equipment such as mobile phones and notebook personal computers has become so popular that opportunities to transmit or receive high-definition images anywhere through communication means have increased rapidly.

It seems inevitable that, with these background trends, demand for improvement in the performance and multi-functioning of image compression and/or decompression technologies will become stronger in the future so that processing of high-definition images can be facilitated.

Therefore, in recent years, a new image compression method called JPEG2000, which can restore with high-quality an image compressed at a high compression rate, has been standardized as one of image compression techniques satisfying such demand. According to JPEG2000, by dividing an image into rectangular regions called tiles, compression and decompression can be performed on the image with a small memory capacity. That is, each individual tile serves as a basic unit in performing compression and decompression processes, so that the tiles can be subjected to the compression and decompression processes independent of one another.

SUMMARY OF THE INVENTION

The process of dividing an image into tiles in JPEG2000, which is referred to as tiling, is effective in reducing memory requirements and increasing processing speed. According to JPEG2000, however, as described in "A New Method for Reducing Boundary Artifacts in Block-Based Wavelet Image Compression," by J. X. Wei, M. R. Pickering, M. R. Frater, and J. F. Arnold, SPIE Visual Communications and Image Processing 2000 International Conference proceedings Vol. 4067, pp. 1290-1295, 20-23 Jun., 2000, Perth, Australia, there is a problem in that performing compression and decompression at a high compression rate results in a decompressed image with discontinuous tile boundaries. That is, such tile boundary discontinuity (or distortion) is likely to be conspicuous in the decompressed image.

In solving the problem of the tile boundary distortion, it is effective to overlap the boundaries of adjacent tiles at the time of performing processing. However, the basic specifications of JPEG2000 (JPEG2000 Part I) provide that no adjacent tile boundaries shall be overlapped. Therefore, it is not desirable to overlap adjacent tile boundaries in terms of compliance with the JPEG2000 provisions.

Further, in order to solve such a problem, there has been proposed a technique that makes tile boundaries less conspicuous by performing low-pass filtering evenly only on the periphery of the tile boundaries.

The above-described technique is effective in controlling the distortion of a tile boundary. However, if the edge degree is strong in the tile boundaries, the low-pass filtering blurs the edges around the tile boundaries, so that the degradation of image quality appears as stripes.

Further, such single-frame JPEG2000 images may be successively displayed at a predetermined frame rate (representing the number of frames reproduced per unit of time) as a moving image.

However, the above-described low-pass filtering requires a relatively large number of operations and takes time. This may delay an image reproduction process. Particularly, in the case of reproducing a moving image, delay in the image reproduction process may cause problems such as loss of synchronization with audio and dropped frames.

Accordingly, it is a general object of the present invention to provide an image decoding device, an image processing apparatus, a moving image display system, and an image decoding method in which the above-described disadvantages are eliminated, a program for causing a computer to execute such a method, and a computer-readable recording medium storing such a program.

A more specific object of the present invention is to provide an image decoding device, an image processing apparatus, a moving image display system, and an image decoding method that can produce an image with good quality by effectively suppressing tile boundary distortion in the image.

Another more specific object of the present invention is to provide an image decoding device, an image processing apparatus, and an image decoding method that can reduce processing time for suppressing tile boundary distortion.

Yet another more specific object of the present invention is to provide an image decoding device, an image processing apparatus, a moving image display system, and an image decoding method that can suppress tile boundary distortion while eliminating an undesirable effect such as dropped frames caused by decoding delay with respect to reproduction, by balancing the rate of decoding with the image quality realized by smoothing the tile boundary distortion.

Yet another more specific object of the present invention is to provide a program for causing a computer to execute such a method, and a computer-readable recording medium storing such a program.

The above objects of the present invention are achieved by an image decoding device for decoding a hierarchically encoded compressed code obtained by dividing an image into a plurality of tiles and performing discrete wavelet transform on pixel values of the image tile by tile, the image decoding device including a tile boundary smoothing part that performs smoothing of tile boundary distortion on the image after the decoding by application of a low-pass filter, the tile boundary smoothing part controlling a degree of smoothing of the low-pass filter according to a ratio of decoding quantity to the entire quantity of the compressed code, the decoding quantity being a portion of the compressed code which portion is to be decoded.

According to the above-described image decoding device, the low-pass filter is applied to a tile boundary in the image while controlling the degree of smoothing of the low-pass filter based on the ratio of the decoding quantity to the entire quantity of the compressed image. That is, the low-pass filter is optimized based on the decoding quantity of the compressed code. Accordingly, an image with good quality can be reproduced.

Additionally, in the above-described image decoding device, the tile boundary smoothing part may be prevented from performing the smoothing of tile boundary distortion when the ratio of the decoding quantity to the entire quantity of the compressed code exceeds a predetermined threshold.

According to the above-described configuration, with respect to an image having the ratio larger than the predetermined value, it is determined that the image is compressed at such a low compression rate that the tile boundary distortion is inconspicuous, and the tile boundary smoothing part is prevented from performing the smoothing of tile boundary distortion on the image. As a result, the processing time for suppressing tile boundary distortion can be reduced.

Additionally, in the above-described image decoding device, the image may be a moving image including a plurality of frames successively decodable by the image decoding device, and the tile boundary smoothing part may perform the smoothing of tile boundary distortion on each of the frames after the decoding. Further, the image decoding device may further comprise a mode selection part that makes selectable one of a first mode for giving priority to image quality and a second mode for giving priority to processing speed in the smoothing of tile boundary distortion by said tile boundary smoothing part, and a tile boundary smoothing switching part that switches a processing mode between the first mode and the second mode based on the selection by said mode selection part in the smoothing of tile boundary distortion on the frames after the decoding by the tile boundary smoothing part.

Accordingly, the first and second modes are switched according to the selection made by the mode selection part, and the tile boundary smoothing part performs the smoothing of tile boundary distortion on each frame after the decoding. This allows the tile boundary smoothing part to perform the smoothing of tile boundary distortion, suitably selecting one of the first and second modes. Therefore, by balancing the rate of decoding and the image quality realized by smoothing the tile boundary distortion, the tile boundary distortion may be suppressed while eliminating an undesirable effect such as dropped frames caused by decoding delay with respect to reproduction.

The above objects of the present invention are also achieved by an image processing apparatus including: a code stream storing part that stores a hierarchically encoded compressed code obtained by dividing an image into a plurality of tiles and performing discrete wavelet transform on pixel values of the image tile by tile; a decoding quantity specifying part that specifies decoding quantity of the compressed code, the decoding quantity being a portion of the compressed code which portion is to be decoded; an image decoding part that decodes the compressed code by the decoding quantity specified by the decoding quantity specifying part; and an image display part that causes a display unit to display the image based on the compressed code decoded by said image decoding part, wherein the image decoding part includes a tile boundary smoothing part that performs smoothing of tile boundary distortion on the image after the decoding by application of a low-pass filter, the tile boundary smoothing part controlling a degree of smoothing of the low-pass filter according to a ratio of the decoding quantity to the entire quantity of the compressed code.

Additionally, in the above-described image processing apparatus, the tile boundary smoothing part may be prevented from performing the smoothing of tile boundary distortion when the ratio of the decoding quantity to the entire quantity of the compressed code exceeds a predetermined threshold.

Additionally, in the above-described image processing apparatus, the image may be a moving image including a plurality of frames successively decodable by the image decoding part, the tile boundary smoothing part may perform the smoothing of tile boundary distortion on each of the frames after the decoding, and the image decoding part may further comprise a mode selection part that makes selectable one of a first mode for giving priority to image quality and a second mode for giving priority to processing speed in the smoothing of tile boundary distortion by said tile boundary smoothing part, and a tile boundary smoothing switching part that switches a processing mode between the first mode and the second mode based on the selection by said mode selection part in the smoothing of tile boundary distortion on the frames after the decoding by the tile boundary smoothing part.

The above-described image processing apparatus may produce the same effects as the above-described image decoding device.

The above objects of the present invention are also achieved by a moving image display system including an image input part acquiring a moving image composed of a plurality of frames, an image compression part that divides each of the frames into a plurality of tiles and performs discrete wavelet transform on pixel values of each of the frames tile by tile so as to hierarchically compress and encode the moving image, an image decoding part that successively decodes the compressed and encoded frames, and an image display part that causes a display unit to display the image based on the decoded frames, wherein the image decoding part includes a tile boundary smoothing part that performs smoothing of tile boundary distortion in each of the frames after the decoding, a mode selection part that makes selectable one of a first mode for giving priority to image quality and a second mode for giving priority to processing speed in the smoothing of tile boundary distortion by the tile boundary smoothing part, and a tile boundary smoothing switching part that switches a processing mode between the first mode and the second mode based on the selection by the mode selection part in the smoothing of tile boundary distortion on the frames after the decoding by the tile boundary smoothing part.

According to the above-described moving image display system, by balancing the rate of decoding and the image quality realized by smoothing the tile boundary distortion, the tile boundary distortion may be suppressed while eliminating an undesirable effect such as dropped frames caused by decoding delay with respect to reproduction.

The above objects of the present invention are also achieved by a method of decoding a hierarchically encoded compressed code obtained by dividing an image into a plurality of tiles and performing discrete wavelet transform on pixel values of the image tile by tile, the method including the step of (a) performing smoothing of tile boundary distortion on the image after the decoding by application of a low-pass filter, wherein step (a) controls a degree of smoothing of the low-pass filter according to a ratio of decoding quantity to the entire quantity of the compressed code, the decoding quantity being a portion of the compressed code which portion is to be decoded.

Additionally, in the above-described method, step (a) may be prevented from performing the smoothing of tile boundary distortion when the ratio of the decoding quantity to the entire quantity of the compressed code exceeds a predetermined threshold.

Additionally, in the above-described method, the image may be a moving image including a plurality of frames successively decodable by the method, and step (a) may perform the smoothing of tile boundary distortion on each of the frames after the decoding. Further, the method may further comprise the step of (b) making selectable one of a first mode for giving priority to image quality and a second mode for giving priority to processing speed in the smoothing of tile boundary distortion by step (a) so that a processing mode is switched between the first mode and the second mode based on the selection by step (b) in the smoothing of tile boundary distortion on the frames after the decoding by step (a).

The above-described method may produce the same effects as the above-described image decoding device.

The above objects of the present invention are also achieved by a computer-readable recording medium storing a program for causing a computer to execute a method of decoding a hierarchically encoded compressed code obtained by dividing an image into a plurality of tiles and performing discrete wavelet transform on pixel values of the image tile by tile, the method including the step of (a) performing smoothing of tile boundary distortion on the image after the decoding by application of a low-pass filter, wherein step (a) controls a degree of smoothing of the low-pass filter according to a ratio of decoding quantity to the entire quantity of the compressed code, the decoding quantity being a portion of the compressed code which portion is to be decoded.

The above objects of the present invention are further achieved by a program for causing a computer to execute a method of decoding a hierarchically encoded compressed code obtained by dividing an image into a plurality of tiles and performing discrete wavelet transform on pixel values of the image tile by tile, the method including the step of (a) performing smoothing of tile boundary distortion on the image after the decoding by application of a low-pass filter, wherein step (a) controls a degree of smoothing of the low-pass filter according to a ratio of decoding quantity to the entire quantity of the compressed code, the decoding quantity being a portion of the compressed code which portion is to be decoded.

BRIEF DESCRIPTION OF THE DRAWINGS

Other objects, features and advantages of the present invention will become more apparent from the following detailed description when read in conjunction with the accompanying drawings, in which:

FIG. 1 is a functional block diagram of a system realizing a hierarchical coding algorithm that forms the basis of JPEG2000, which algorithm is a precondition for the present invention;

FIG. 2 is a diagram for illustrating rectangular regions of each component of an original image according to JPEG2000;

FIG. 3 is a diagram for illustrating sub-bands at each decomposition level when the decomposition level is 3 according to JPEG2000;

FIG. 4 is a diagram for illustrating a precinct according to JPEG2000;

FIG. 5 is a diagram for illustrating a process for placing bit planes in order according to JPEG2000;

FIG. 6 is a schematic diagram showing a frame structure of code stream data according to JPEG2000;

FIG. 7 is a diagram showing a system including an image processing apparatus according to a first embodiment of the present invention;

FIG. 8 is a functional block diagram of the image processing apparatus according to the first embodiment of the present invention;

FIG. 9 is a diagram showing a two-dimensionally divided image according to the first embodiment of the present invention;

FIG. 10 is a diagram for illustrating a compressed code generated in accordance with the JPEG2000 algorithm based on the divided image of FIG. 9 according to the first embodiment of the present invention;

FIG. 11 is a block diagram showing a hardware configuration of the image processing apparatus according to the first embodiment of the present invention;

FIG. 12 is a functional block diagram of an image decompressor of the image processing apparatus according to the first embodiment of the present invention;

FIG. 13 is a diagram showing an image quality specifying screen displayed by a decoding quantity specifying part of the image decompressor according to the first embodiment of the present invention;

FIG. 14 is a diagram for illustrating a ratio of the decoding quantity to the entire quantity of a compressed code according to the first embodiment of the present invention;

FIG. 15 is a diagram for illustrating an operation of a tile boundary smoothing part of the image decompressor according to the first embodiment of the present invention;

FIG. 16 is a diagram for illustrating low-pass filtering on a vertical tile boundary according to the first embodiment of the present invention;

FIG. 17 is a diagram for illustrating low-pass filtering on a lateral tile boundary according to the first embodiment of the present invention;

FIG. 18 is a diagram for illustrating low-pass filtering on the periphery of the intersection of a vertical tile boundary and a lateral tile boundary according to the first embodiment of the present invention;

FIG. 19 is a functional block diagram of the image decompressor according to a second embodiment of the present invention;

FIGS. 20A and 20B are diagrams for illustrating a case of performing low-pass filtering only on the tile boundaries within an ROI according to the second embodiment of the present invention;

FIGS. 21A and 21B are diagrams for illustrating another case of performing low-pass filtering only on the tile boundaries within an ROI according to the second embodiment of the present invention;

FIG. 22 is a diagram for illustrating the concept of Motion JPEG2000;

FIG. 23 is a diagram showing a monitor camera system according to a third embodiment of the present invention;

FIG. 24 is a functional block diagram of the monitor camera system according to the third embodiment of the present invention;

FIG. 25 is a block diagram showing a hardware configuration of the monitor camera system according to the third embodiment of the present invention;

FIG. 26 is a functional block diagram of an image decompressor of a PC of the monitor camera system according to the third embodiment of the present invention;

FIG. 27 is a diagram showing an image quality specifying screen displayed by a mode selection part of the image decompressor according to the third embodiment of the present invention;

FIG. 28 is a diagram for illustrating operations of first and second tile boundary smoothing parts of the image decompressor according to the third embodiment of the present invention;

FIG. 29 is a diagram for illustrating low-pass filtering on a vertical tile boundary by the first tile boundary smoothing part according to the third embodiment of the present invention;

FIG. 30 is a diagram for illustrating low-pass filtering on a lateral tile boundary by the first tile boundary smoothing part according to the third embodiment of the present invention;

FIG. 31 is a diagram for illustrating low-pass filtering on the periphery of the intersection of a vertical tile boundary and a lateral tile boundary by the first tile boundary smoothing part according to the third embodiment of the present invention;

FIG. 32 is a diagram for illustrating low-pass filtering on a vertical tile boundary by the second tile boundary smoothing part according to the third embodiment of the present invention;

FIG. 33 is a diagram for illustrating low-pass filtering on a lateral tile boundary by the second tile boundary smoothing part according to the third embodiment of the present invention;

FIG. 34 is a diagram for illustrating low-pass filtering on the periphery of the intersection of a vertical tile boundary and a lateral tile boundary by the second tile boundary smoothing part according to the third embodiment of the present invention;

FIG. 35 is a diagram for illustrating a method of calculating the distance between a pixel and a tile boundary according to the third embodiment of the present invention;

FIG. 36 is a diagram showing an edge amount calculation filter according to the third embodiment of the present invention;

FIG. 37 is a functional block diagram of the image decompressor according to a fourth embodiment of the present invention;

FIG. 38 is a diagram for illustrating a start frame, a final frame, and a suspended frame according to the fourth embodiment of the present invention;

FIG. 39 is a functional block diagram of the image decompressor according to a fifth embodiment of the present invention;

FIG. 40 is a functional block diagram of the image decompressor according to a sixth embodiment of the present invention;

FIG. 41 is a diagram for illustrating mode selection based on decoding quantity according to the sixth embodiment of the present invention;

FIG. 42 is a functional block diagram of the image decompressor according to a seventh embodiment of the present invention;

FIGS. 43A and 43B are diagrams for illustrating a case of performing low-pass filtering only on the tile boundaries within an ROI according to the seventh embodiment of the present invention;

FIGS. 44A and 44B are diagrams for illustrating another case of performing low-pass filtering only on the tile boundaries within an ROI according to the seventh embodiment of the present invention;

FIG. 45 is a diagram showing an original image and a coordinate system therefor for illustrating 5.times.3 wavelet transform employed in JPEG2000;

FIG. 46 is a diagram showing a coefficient array obtained by vertically performing a one-dimensional wavelet transform operation on the original image of FIG. 45;

FIG. 47 is a diagram showing a coefficient array obtained by laterally performing a one-dimensional wavelet transform operation on the coefficient array of FIG. 46;

FIG. 48 is a diagram showing a coefficient array into which the coefficients of FIG. 47 are rearranged;

FIG. 49 is a diagram showing a coefficient array into which the coefficients obtained by two-dimensional wavelet transform at decomposition level 2 are rearranged;

FIG. 50 is a diagram for illustrating mirroring of pixel values at a tile boundary;

FIGS. 51A and 51B are graphs showing distributions of the mean square errors of pixel values generated in the pixels within a tile;

FIGS. 52A through 52C are diagrams showing symmetric low-pass filters applied to vertical and lateral tile boundaries;

FIG. 53 is a diagram showing a low-pass filter application in the case where a target pixel is located at an L coefficient position;

FIG. 54 is a block diagram showing a computer system according to an eighth embodiment of the present invention;

FIG. 55 is a flowchart for illustrating an image processing operation according to the eighth embodiment of the present invention;

FIG. 56 is a diagram for illustrating the distance from a tile boundary according to the eighth embodiment of the present invention;

FIG. 57 is a flowchart for illustrating a process for smoothing a tile boundary according to the eighth embodiment of the present invention;

FIG. 58 is a diagram showing a configuration of a low-pass filter for application to a vertical tile boundary according to the eighth embodiment of the present invention;

FIG. 59 is a diagram showing a configuration of the low-pass filter for application to a lateral tile boundary according to the eighth embodiment of the present invention;

FIGS. 60A through 60D are diagrams showing configurations of the low-pass filter for application to a tile boundary intersection according to the eighth embodiment of the present invention;

FIG. 61 is a diagram showing another configuration of the low-pass filter for application to the vertical tile boundary according to the eighth embodiment of the present invention;

FIG. 62 is a diagram showing yet another configuration of the low-pass filter for application to the vertical tile boundary according to the eighth embodiment of the present invention;

FIG. 63 is a diagram showing yet another configuration of the low-pass filter for application to the vertical tile boundary according to the eighth embodiment of the present invention;

FIG. 64 is a diagram showing an edge amount calculation filter according to the eighth embodiment of the present invention; and

FIGS. 65A through 65C are diagrams showing configurations of the low-pass filter for application to tile boundaries according to the eighth embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A description will be given, with reference to the accompanying drawings, of embodiments of the present invention.

First, a description will be given schematically of the "hierarchical coding algorithm" and the "JPEG2000 algorithm," which are the premises of the embodiments of the present invention.

FIG. 1 is a functional block diagram of a system realizing the hierarchical coding algorithm that forms the basis of JPEG2000. This system includes a color space conversion and inverse conversion part 101, a two-dimensional (2D) wavelet transform and inverse transform part 102, a quantization and inverse quantization part 103, an entropy coding and decoding part 104, and a tag processing part 105.

One of the major differences between this system and the conventional JPEG algorithm is the transform method. JPEG employs discrete cosine transform (DCT) while the hierarchical coding algorithm employs discrete wavelet transform (DWT) in the 2D wavelet transform and inverse transform part 102. Compared with DCT, DWT enjoys the advantage of excellent image quality in a highly compressed region. This advantage is one of the major reasons DWT is employed in JPEG2000, which is a successor algorithm to JPEG.

Another major difference is that the hierarchical coding algorithm additionally includes a functional block called the tag processing part 105 at the final stage of the system so as to form codes. The tag processing part 105 generates compressed data as code stream data at the time of compression and interprets code stream data necessary for decompression at the time of decompression. The code stream data allows JPEG2000 to realize a variety of convenient functions. For instance, as shown in FIG. 3, the compression and decompression of a still image can be stopped freely at any hierarchy (decomposition level) corresponding to the octave division in block-based DWT.

The part for inputting and outputting an original image is often connected to the color space conversion and inverse conversion part 101 of FIG. 1. For instance, the color space conversion and inverse conversion part 101 converts the RGB calorimetric system made up of primary color system components of red (R), green (G), and blue (B) or the YMC calorimetric system made up of complementary color system components of yellow (Y), magenta (M), and cyan (C) to the YUV or YCbCr calorimetric system, or performs the inverse conversion thereof.

Next, a description will be given of the JPEG2000 algorithm.

Generally, in a color image, each component 111 (RGB primary color system in this case) of the original image is divided into rectangular regions 112 as shown in FIG. 2. Generally, the rectangular regions 112 are referred to as blocks or tiles. Since the rectangular regions 112 are generally referred to as tiles in JPEG2000, the rectangular regions 112 are hereinafter referred to as tiles. In the case of FIG. 2, each component 111 is divided into 16 (4.times.4) rectangular tiles 112. Each of the tiles 112 (R00, R01, . . . , R15, G00, G01, . . . , G15, B00, B01, . . . , B15 in FIG. 2) becomes a basic unit in the image data compression and decompression process. Accordingly, the compression and decompression of image data is performed independently for each component 111 and each tile 112.

At the time of encoding the image data, the data of each tile 112 of each component 111 is input to the color space conversion and inverse conversion part 101 and subjected to color space conversion.

Thereafter, the data is subjected to 2D wavelet transform (forward transform) in the 2D wavelet transform and inverse transform part 102 and spatially divided into frequency bands.

FIG. 3 is a diagram showing the sub-bands of each decomposition level in the case where the decomposition level is 3. That is, the 2D wavelet transform is performed on the tile original image (0LL) of decomposition level 0 obtained by dividing the original image into tiles, so that the sub-bands (1LL, 1HL, 1LH, and 1HH) shown at decomposition level 1 are separated. Successively thereafter, the 2D wavelet transform is performed on the low-frequency component of 1LL at this level so that the sub-bands (2LL, 2HL, 2LH, and 2HH) shown at decomposition level 2 are separated. Similarly, the 2D wavelet transform is performed on the low-frequency component of 2LL so that the sub-bands (3LL, 3HL, 3LH, and 3HH) shown at decomposition level 3 are separated. In FIG. 3, the sub-bands to be subjected to encoding are indicated by hatching at each decomposition level. For instance, at decomposition level 3, the hatched sub-bands (3HL, 3LH, 3HH, 2HL, 2LH, 2HH, 1HL, 1LH, and 1HH) are to be subjected to encoding and the 3LL sub-band is not to be encoded.

Next, the target bits to be encoded are determined in a specified encoding order, and context is generated from the peripheral bits of each target bit in the quantization and inverse quantization part 103.

The quantized wavelet coefficients are divided into non-overlapping rectangles called "precincts" sub-band by sub-band. The precincts are introduced to effectively utilize memory upon implementation. As shown in FIG. 4, each precinct is composed of three spatially matching rectangular regions. Further, each precinct is divided into non-overlapping rectangular "code blocks." Each code block becomes a basic unit in performing entropy coding.

The coefficient values after the wavelet transform may directly be quantized and encoded. In order to improve encoding efficiency, however, JPEG2000 decomposes the coefficient values into units called "bit planes," which may be placed in order in each pixel or code block.

FIG. 5 is a diagram for illustrating a process for placing the bit planes in order. As shown in FIG. 5, an original image of 32.times.32 pixels is divided into four tiles each of 16.times.16 pixels in this case. The sizes of each precinct and each code block at decomposition level 1 are 8.times.8 and 4.times.4 pixels, respectively. The precincts and the code blocks are respectively numbered according to a raster sequence.

In this case, numbers 0 to 3 are assigned to the precincts and numbers 0 to 3 are assigned to the code blocks. A mirroring method is employed in pixel expansion beyond a tile boundary, and wavelet transform is performed with a reversible (5, 3) integer transform filter so that the wavelet coefficients of decomposition level 1 are obtained.

Further, FIG. 5 also shows a conceptual typical "layer" structure with respect to the tile 0, precinct 3, and code block 3. The transformed code block 3 is divided into sub-bands (1LL, 1HL, 1LH, and 1HH), and the sub-bands are allocated their respective wavelet coefficient values.

The layer structure is easier to understand when the wavelet coefficient values are viewed horizontally along the bit planes. One layer is composed of an arbitrary number of bit planes. In this case, the layers 0, 1, 2, and 3 are composed respectively of one, three, one, and three bit planes. The layer including a bit plane closer to the LSB (least significant bit) bit plane is subjected to the quantization earlier, and the layer including a bit plane closer to the MSB (most significant bit) bit plane is subjected to the quantization later. The method of discarding a layer closer to the LSB bit plane first is called truncation, by which the rate of quantization can be finely controlled.

The entropy coding and decoding part 104 of FIG. 1 performs encoding on the tiles 112 of each component 111 by probability estimation from the context and the target bits. Thus, the encoding is performed in units of the tiles 112 for each component 111 of the original image. Finally, the tag processing part 105 connects all the coded data supplied from the entropy coding and decoding part 104 into a single coded data stream (code stream data), and adds a tag thereto.

FIG. 6 is a schematic diagram showing a frame structure of the code stream data. Tag information called a main header is added to the head of the code stream data, and tag information called a tile-part header is added to the head of each of the coded data (bit stream) of each tile 112. The tile-part header indicates a tile boundary position and a tile boundary direction and is followed by the coded data of the corresponding tile 112. Coding and quantization parameters are written to the main header. Another tag (end of code stream) is added to the terminal end of the code stream data.

On the other hand, at the time of decoding the coded data, image data is generated from the code stream data of the tiles 112 of each component 111, which is the reverse of the process at the time of encoding the image data. In this case, the tag processing part 105 interprets the tag information added to the code stream data input from the outside. Then, the tag processing part 105 decomposes the input code stream data into the code stream data of the tiles 112 of each component 111, and decodes (decompresses) the code stream data in units of the tiles 112 for each component 111. At this point, the positions of the target bits to be subjected to the decoding are determined according to the order based on the tag information within the code stream data, and the quantization and inverse quantization part 103 generates context from the arrangement of the peripheral bits (already decoded) of the position of each target bit. The entropy coding and decoding part 104 performs decoding based on probability estimation from the context and the code stream data so as to generate the target bits, and writes the target bits to their respective positions. The thus decoded data is spatially divided in every frequency band. Therefore, each tile 112 of each component 111 of the image data can be restored by subjecting the decoded data to 2D wavelet inverse transform in the 2D wavelet transform and inverse transform part 102. The color space conversion and inverse conversion part 101 converts the restored data to the image data of the original calorimetric system.

First Embodiment

Next, a description will be given of a first embodiment of the present invention.

FIG. 7 is a diagram showing a system including an image processing apparatus 1 according to the first embodiment of the present invention. FIG. 8 is a functional block diagram of the image processing apparatus 1. As shown in FIG. 7, the image processing apparatus 1 is a personal computer, for instance, and is connectable via a network 5, which may be the Internet, to a server computer S storing and retaining a variety of image data.

In this embodiment, the image data stored in the server computer S are compressed codes generated in accordance with the JPEG2000 algorithm. More specifically, a two-dimensionally divided image as shown in FIG. 9 is subjected to compression coding and arranged one-dimensionally, so that a compressed code as shown in FIG. 10 is generated. In FIG. 10, SOC is a marker segment indicating the start of a code stream. MH refers to a main header storing values common to the entire code stream. The recorded values common to the entire code stream include tile lateral quantity, tile vertical quantity, image lateral quantity, and image vertical quantity. The coded data of each file follows the MH. The data of the tiles of FIG. 9 are compressed in the main scanning direction and the sub scanning direction according to the tile numbers, and arranged as shown in FIG. 10. An EOC marker at the end of the compressed code is a marker segment indicating the end of a compressed code.

As shown in FIG. 8, the image processing apparatus 1 includes an image decompressor 2, an image display unit 3, and a code stream storage part 4. The image decompressor 2 is an image decoding device that decompresses (decodes) code stream data (JPEG2000 data) output via the network 5 to the image processing apparatus 1 into the data of an image. The image display unit 3 displays the image based on the decompressed image data. The code stream storage part 4 stores the code stream data (JPEG2000 data) output via the network 5 to the image processing apparatus 1. The code stream storage part 4, which functions as a common buffer or storage for the code stream data of images, is used differently for the different purposes.

FIG. 11 is a block diagram showing a hardware configuration of the image processing apparatus 1. As shown in FIG. 11, the image processing apparatus 1 includes a CPU (central processing unit) 6 that is an important part of the computer and performs centralized control of each part of the computer. The CPU 6 is connected via a bus 9 to a ROM 7 that is a read-only memory storing a BIOS (basic input/output system) and a RAM (random access memory) 8 that rewritably stores a variety of data. The RAM 8, which has the characteristic of rewritably storing a variety of data, functions as a work area for the CPU 6, serving as an input buffer, for instance.

The bus 9 is further connected via input/output (I/O) parts (not shown in the drawing) to an HDD (hard disk drive) 10 functioning as the code stream storage part 4, a CD-ROM drive 12 for reading a CD-ROM 11 as a mechanism for reading computer software that is a distributed program, a communication control unit 13 controlling communication between the image processing apparatus 1 and the network 5, an input device 14 such as a keyboard or a mouse, and a display device 15 such as a CRT (cathode ray tube) or an LCD (liquid crystal display).

The compressed code (see FIG. 10) downloaded from the server computer S via the network 5 is stored in the HDD 10 functioning as the code stream storage part 4.

Further, the CD-ROM 11 shown in FIG. 11, which realizes a storage medium according to the present invention, stores an operation system (OS) and a variety of computer software. The CPU 6 reads the computer software stored in the CD-ROM 11 in the CD-ROM drive 12, and installs the read computer software in the HDD 10.

The computer software stored in the HDD 10 of the above-described image processing apparatus 1 includes an image processing program for processing images. The image processing program realizes a program according to the present invention. The function of the image decompressor 2 is realized by the processing performed by the CPU 6 based on the image processing program.

The CD-ROM 11 is not the only storage medium, but media of a variety of types, such as optical disks including DVDs, magneto-optical disks, magnetic disks including flexible disks, and semiconductor memory devices, may be employed as storage media according to the present invention. Alternatively, the computer software may be downloaded via the communication control unit 13 from the network 5 so as to be installed in the HDD 10. In this case, the storage device that stores the computer software on the transmitter-side server is also a storage medium of the present invention. The computer software may operate on a predetermined OS. In this case, part of later-described operations may be executed by the OS. Alternatively, the computer software may be included in a group of program files composing a predetermined application or OS.

A brief description will be given, with reference to FIG. 8, of the operation of each part of the image processing apparatus 1. Code stream data (JPEG2000 data) output via the network 5 to the image processing apparatus 1 is stored in the code stream storage part 4 and decompressed in the image decompressor 2. The data of an image generated by decompressing the code stream data in the image decompressor 2 is output to the image display unit 3, where the image based on the decompressed image data is displayed on the display device 15.

Next, an expatiation will be given of the image decompressor 2, which forms an important part of the present invention. FIG. 12 is a functional block diagram of the image decompressor 2. As shown in FIG. 12, by the CPU 6 operating based on the computer software (image processing program), the image decompressor 2 realizes the functions of a decoding quantity specifying part 20, a tag processing part 21, an entropy decoding part 22, an inverse quantization part 23, a two-dimensional (2D) wavelet inverse transform part 24, a color space inverse conversion part 25, a tile boundary control part 26, and a tile boundary smoothing part 27. The functions realized by the processing part 21, the entropy decoding part 22, the inverse quantization part 23, the 2D wavelet inverse transform part 24, and the color space inverse conversion part 25 are described with reference to the color space conversion and inverse conversion part 101, the 2D wavelet transform and inverse transform part 102, the quantization and inverse quantization part 103, the entropy coding and decoding part 104, and the tag processing part 105, respectively, of FIG. 1. Therefore, a description thereof is omitted.

The decoding quantity specifying part 20 specifies the portion of a compressed code which is to be decoded. Hereinafter, this portion may be referred to as decoding quantity. Specifically, the decoding quantity specifying part 20 displays an image quality specifying screen X as shown in FIG. 13 on the display device 15. The image quality specifying screen X includes radio buttons B for selecting the image quality of the image to be displayed on the display device 15 based on a predetermined compressed code from three image quality levels (High, Normal, and Low). When the operator operates the input device 14 to specify one of the three radio buttons B, a