Integrative encoding system and adaptive decoding system Number:7,072,491 from the United States Patent and Trademark Office (PTO) owispatent An integrative encoding system for encoding and transmitting a plurality of video signals having different resolutions that correspond to a plurality of display units. The integrative encoding system has a compression processor, an editing processor, and an integrated services digital broadcasting (ISDB) transmitter. In a first embodiment, the compression processor performs adaptive dynamic range coding (ADRC) to compress each of said plurality of video signals on a block basis by reducing the dynamic range. In another embodiment,the compression processor performs hierarchical encoding on the plurality of video signals by selectively replacing pixels of a higher resolution level with pixels from a lower resolution level calculated by combining pixels from said higher resolution level; thereby encoding a hierarchy of resolution levels within the plurality of video signals without increasing the amount of data. An adaptive decoding system receives and decodes the transmitted plurality of video signals for display by the plurality of display units.<H Integrative, decoding, Integrative, enc, 7072491.html, Patent, pto, 7072491

Title: Integrative encoding system and adaptive decoding system

Abstract: An integrative encoding system for encoding and transmitting a plurality of video signals having different resolutions that correspond to a plurality of display units. The integrative encoding system has a compression processor, an editing processor, and an integrated services digital broadcasting (ISDB) transmitter. In a first embodiment, the compression processor performs adaptive dynamic range coding (ADRC) to compress each of said plurality of video signals on a block basis by reducing the dynamic range. In another embodiment,the compression processor performs hierarchical encoding on the plurality of video signals by selectively replacing pixels of a higher resolution level with pixels from a lower resolution level calculated by combining pixels from said higher resolution level; thereby encoding a hierarchy of resolution levels within the plurality of video signals without increasing the amount of data. An adaptive decoding system receives and decodes the transmitted plurality of video signals for display by the plurality of display units.

Patent Number: 7,072,491 Issued on 07/04/2006 to Kondo

Inventors: Kondo; Tetsujiro (Kanagawa, JP)
Assignee: Sony Corporation (Tokyo, JP)

Appl. No.: 602231

Filed: June 24, 2003

Related U.S. Patent Documents


Application Number	Filing Date	Patent Number	Issue Date
10160418	May., 2002	6636640
09254536	Jul., 1998	6480630
PCT/JP98/03116	Jul., 1998

Foreign Application Priority Data


Jul 11, 1997 [JP]			9-186144

Current U.S. Class: 382/100

Current International Class: G06K 9/00 (20060101)

Field of Search: 382/232,233,236,238,239,249,298,309 358/537,539 348/404.1,409.1 375/240.02,240.03,240.12,240.16,240.21,240.25,240.27,240.28 341/51

References Cited [Referenced By]

U.S. Patent Documents


5068726	November 1991	Kondo et al.
5276531	January 1994	Chen et al.
5436665	July 1995	Ueno et al.
5438635	August 1995	Richards
5517588	May 1996	Kondo
5539586	July 1996	Inoue et al.
5559553	September 1996	Bitek
5598214	January 1997	Kondo et al.
5712689	January 1998	Yasuki et al.
5796858	August 1998	Zhou et al.
5818974	October 1998	Kimura et al.
6072831	June 2000	Chen

Foreign Patent Documents


690 28078	Jun., 1991	DE
693 16440	Nov., 1993	DE
0 373 924	Jun., 1990	EP
0 419 141	Mar., 1991	EP
0 430 631	Jun., 1991	EP
0 571 180	Nov., 1993	EP
0 618 727	Oct., 1994	EP
2 285 361	Jul., 1995	GB
1-213088	Aug., 1989	JP
3-167985	Jul., 1991	JP
5-328185	Dec., 1993	JP
7-87327	Mar., 1995	JP

Other References

Patent Abstracts of Japan vol. 1995, No. 06, Jul. 31, 1995 & JP 07 087327 A Mar. 31, 1995. cited by other .
Patent Abstracts of Japan vol. 015, No. 200, May 22, 1991 & JP 03 053778 A Mar. 7, 1991. cited by other .
Patent Abstracts of Japan vol. 1997, No. 08, Aug. 29, 1997 & JP 09 093285 A Apr. 4 1997. cited by other .
Patent Abstracts of Japan vol. 1997, No. 06, Jun. 30, 1997 & JP 09 055673 A Feb. 25, 1997. cited by other.

Primary Examiner: Couso; Jose
Attorney, Agent or Firm: Frommer Lawrence & Haug LLP Frommer; William S.

Parent Case Text

Divisional of prior application Ser. No. 10/160,418, filed May 31, 2002 now U.S. Pat. No. 6,636,640, which is a continuation of application Ser. No. 09/254,536, filed Mar. 9, 1999, now U.S. Pat. No. 6,480,630, issued Nov. 12, 2002, which is a 371 of PCT/JP98/03116, filed 10 Jul. 1998.

Claims

The invention claimed is:

1. An integrative encoding system for encoding and transmitting a plurality of video signals having different resolutions, comprising: a compression processor for performing hierarchical encoding on the plurality of video signals by selectively replacing pixels of a higher resolution level with pixels from a lower resolution level calculated by combining pixels from said higher resolution level; thereby encoding a hierarchy of resolution levels within the plurality of video signals without increasing the amount of data; an editing processor for editing the hierarchically encoded plurality of video signals into a bit stream; and an integrated services digital broadcasting (ISDB) transmitter having: a time code generator for generating a time code synchronized to the bit stream; an additive information generator for generating additive information in synch with said synchronous signal; and a multiplexer for multiplexing said bit stream, said additive information, and said time code into ISDB data for transmission.

2. The integrative encoding system according to claim 1, wherein said additive information includes computer graphics and network data.

3. The integrative encoding system according to claim 1, wherein said additive information comprises information of local area interest to a viewer and is identified by an area code.

4. The integrative encoding system according to claim 1, wherein said plurality of video signals comprises a high definition video signal and a standard definition video signal.

Description

FIELD OF THE ART

The present invention relates to a video encoder, video decoder, video processor and methods thereof, and more particularly to such methods and apparatuses capable of efficiently encoding a picture and further obtaining a decoded picture in an appropriate form to a monitor for displaying the picture.

BACKGROUND ART

FIG. 16 shows a structural example of a video transmission system for transmitting a picture from its transmitting side to receiving side.

On the transmitting side, in a high definition video camera 201, an object is imaged and a high definition picture formed of 1920.times.1035 pixels in horizontal by longitudinal in the 16:9 aspect ratio (hereinafter, it is preferably referred to as HD picture) is outputted, for example. The HD picture for approximately 30 frames will be normally outputted per second from the video camera 201. But here the HD picture is subjected to interlaced scanning. Therefore, the HD picture for approximately 60 fields is outputted per second from the video camera 201.

On the transmitting side, in a standard or low definition video camera 202, an object is imaged and a standard or low definition picture in the 4:3 aspect ratio (hereinafter, it is preferably referred to as SD picture) is outputted, for example. If the video camera 202 is in the national television system committee (NTSC) system or the phase alternation by line (PAL) system, one frame is formed of 720.times.480 pixels or 720.times.576 pixels respectively and the number of frames per second (the number of fields) is 30 (60) or 25 (50) respectively, for example.

On the transmitting side, in a progressive imager 203, an object is scanned and the scanned picture (hereinafter, it is preferably referred to as progressive picture) is sequentially outputted. And on the transmitting side, in a computer 204, a picture formed of, e.g., 640.times.480 pixels is generated and outputted as computer graphics (CG).

The ratio in horizontal to longitudinal of pixels that compose pictures to be outputted by the video cameras 201 and 202 is approximately 1:1.1. The ratio in horizontal to longitudinal of pixels that compose a picture to be outputted by the computer 204 is 1:1.

As the above, pictures different in its aspect ratio, the number of pixels, its scanned method, the ratio of pixels in horizontal and longitudinal, etc., will be outputted from the video cameras 201 and 202, progressive imager 203 and computer 204.

These pictures are inputted to an editor 205. In the editor 205, pictures from the video cameras 201 and 202, progressive imager 203 and computer 204 are edited respectively. All the edited pictures are converted into progressive pictures formed of, e.g., 1920.times.1080 pixels and outputted to a source encoder 206. In the source encoder 206, the pictures outputted from the editor 205 is coded (MPEG-coded) based on the moving picture experts group (MPEG) standard for example, and thus obtained coded data is supplied to a channel encoder 207.

The adoption of that converting all of pictures different in the number of pixels, its scanned method, etc., into 1920.times.1080-pixel progressive pictures by advanced television (ATV) has been planned.

In the channel encoder 207, channel coding is performed to improve the reliability of the coded data in transmission. That is, in the channel encoder 207, e.g., error correcting codes (ECCS) are added to the coded data as processing for error correction, and further it is subjected to prescribed modulation or the like. The transmit data obtained by the processing in the channel encoder 207 is transmitted via a transmission line 211.

On the receiving side, the transmit data transmitted from the transmitting side as the above is received. This transmit data is supplied to a channel decoder 208 to be channel-decoded. Specifically, prescribed demodulation is performed and further error correction using the ECCs or the like is performed, for example.

The coded data obtained as a result of the processing in the channel decoder 208 is supplied to a source decoder 209. In the source decoder 209, the coded is expanded by, for example, decoded (MPEG-decoded) based on the MPEG standard, and thus obtained picture data is supplied to a processor 210.

In the processor 210, the picture data from the source decoder 209 is processed to be matched to the format of an output device to output the picture data. That is, in the case where the picture data is displayed in an HD display device 221 for displaying HD pictures, in the processor 210, the picture data outputted by the source decoder 209 is processed into an interlace-scanned HD picture composed of, e.g., 1920.times.1035 pixels in the 16:9 aspect ratio. In the case where the picture data displayed in an SD display device 222 for displaying SD pictures, in the processor 210, the picture data outputted by the source decoder 209 is processed into an SD picture in the NTSC system or the PAL system, composed of 720.times.480 pixels or 720.times.576 pixels for example. In the case where the picture data is printed out by a printer 223, in the processor 210, the picture outputted by the source decoder 209 is converted into a picture of which the ratio of pixels in horizontal to longitudinal is corresponded to the printer 223. On the other hand, in the case where the picture data is displayed on a computer display 224, in the processor 210, the picture data outputted by the source decoder 209 is process d into a picture composed of 640.times.480 pixels for example.

In the HD display device 221, the SD display device 222, the printer 223 and the computer display 224, the picture from the processor 210 is displayed or printed out.

By the way, heretofore, the editing processing by the editor 205, the compression processing by the source encoder 206 and the channel coding processing by the channel encoder 207 on the transmitting side have quasi conducted respectively and independently.

For example, the compressed data has less information amount than the data before compression processing. Thus, if the compressed data can be set to be edited, a load on the editor 205 can be reduced. However, if the picture is MPEG-coded in the source encoder 206 as described above, the bit stream obtained as its result becomes difficult to be edited unless in a group of picture (GOP) unit, and editing of that is limited to so-called cut editing by only connecting the GOPs. Since compression processing regardless of the editing processing by the editor 205 is performed in the source encoder 206, the compressed data cannot be edited in a frame unit and it is difficult to give various effects on it.

Moreover, for example, in the source encoder 206, compression processing is not performed in consideration of the addition of ECCs by the channel encoder 207. Therefore, for example, if ECCs are added to the coded data obtained by the compression processing, sometimes compressibility as the entire data after the addition of ECCs has deteriorated.

As the above, heretofore, since the processing necessary for picture coding, e.g., the editing processing, compression processing and channel coding processing, etc., has not performed in consideration of the other processing, it has been difficult to perform efficient processing.

DISCLOSURE OF INVENTION

The present invention is provided considering the above aspects, which enable it to perform efficient processing.

A video encoder according to the present invention is characterized by including a processing means for performing one or plural processing necessary to encode a picture considering the other processing.

A method for encoding a picture according to the present invention is characterized by performing one or more processing among the plural processing necessary to encode a picture considering the other processing.

A video decoder according to the present invention is characterized by including a generation means for generating a decoded picture corresponding to the resolution of an output device for outputting the picture by linearly coupling transmit data to prescribed coefficients.

A method for decoding a picture according to the present invention is characterized by generating a decoded picture corresponding to the resolution of an output device for outputting the picture by linearly coupling the transmit data to the prescribed coefficients.

A video processor according to the present invention is characterized by including a processing means for performing one or more processing among the plural processing necessary to encode a picture considering the other processing, and a generation means for generating a decoded picture corresponding to the resolution of an output device for outputting the picture by linearly coupling the data obtained as the result of processing by the processing means to the prescribed coefficients.

A method for processing a picture according to the present invention is characterized by performing one or more processing among the plural processing necessary to encode a picture considering the other processing, and generating a decoded picture corresponding to the resolution of an output device for outputting the picture by linearly coupling thus obtained data to the prescribed coefficients.

In the video encoder according to the present invention, the processing means performs one or more processing among the plural processing necessary to encode the picture considering the other processing.

In the method of encoding a picture according to the present invention, one or more processing among the plural processing necessary to encode the picture are performed considering the other processing.

In the video decoder according to the present invention, the generation means generates a decoded picture corresponding to the resolution of an output device for outputting the picture by linearly coupling transmit data to prescribed coefficients.

In the method for decoding a picture according to the present invention, a decoded picture corresponding to the resolution of an output device for outputting the picture is generated by linearly coupling the transmit data to the prescribed coefficients.

In the video processor according to the present invention, the processing means performs one or more processing among the plural processing necessary to encode the picture considering the other processing, and the generation means generates a decoded picture corresponding to the resolution of an output device for outputting the picture by linearly coupling the data obtained as the result of processing by the processing means to the prescribed coefficients.

In the method for processing a picture according to the present invention, one or more processing among the plural processing necessary to encode the picture is performed considering the other processing, and a decoded picture corresponding to the resolution of an output device for outputting the picture is generated by linearly coupling thus obtained data to the prescribed coefficients.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing an embodiment of a video transmission system in which the present invention is applied.

FIGS. 2A and 2B are diagrams explaining the ADRC processing.

FIG. 3 is a block diagram showing a structural example of ADRC processing circuit.

FIG. 4 is a block diagram showing a structural example of sync block forming circuit.

FIG. 5 is a diagram showing the format of a sync block.

FIGS. 6A to 6C are diagrams explaining hierarchical coding.

FIG. 7 is a block diagram showing a structural example of hierarchical encoding circuit.

FIG. 8 is a block diagram showing a structural example of ISDB transmitter.

FIG. 9 is a block diagram showing a structural example of ADRC decoding circuit.

FIG. 10 is a block diagram showing another structural example of ADRC decoding circuit.

FIG. 11 is a block diagram showing a structural example of ISDB receiver.

FIG. 12 is a block diagram showing a structural example of resolution creating circuit.

FIG. 13 is a diagram showing a classificatory block and a predictive value calculating block.

FIGS. 14A and 14B are diagrams explaining classification processing.

FIG. 15 is a block diagram showing a structural example of for casting circuit 134 of FIG. 12.

FIG. 16 is a block diagram showing a structural example of video transmission system for transmitting a picture from its transmitting side to receiving side.

BEST MODE FOR CARRYING OUT THE INVENTION

An embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 shows a structural example of the embodiment of video transmission system in which the present invention is applied.

On its transmitting side, pictures composed of the maximum number of pixels capable of being outputted by an extended definition video camera 1, a video camera 2 in low definition, etc., are outputted from them. Specifically, for example, a progressive HD picture which is composed of 1920.times.960 pixels, and of which the aspect ratio is 16:9 and the frame rate is approximately 30 frame/sec, is outputted from the video camera 1. And for example, a progressive SD picture which is composed of 640.times.480 pixels, and of which the aspect ratio is 4:3 and the frame rate is approximately 30 frame/sec, or a progressive SD picture which is composed of 720.times.480 pixels, and of which the aspect ratio is the 4:3 and the frame rate is approximately 15 frame/sec is outputted from the video camera 2.

On its receiving side, a progressive imager 3 scans an object and outputs for example, a progressive picture of which the longitudinal number of pixels is integral times of 480. And a computer 4 generates and outputs for example, a picture which is composed of 640.times.480 pixels and of which a ratio in horizontal to longitudinal is 1:1, as computer graphics (CG).

Such pictures outputted by the video cameras 1 and 2, progressive imager 3 and computer 4 are supplied to an integrative encoding system 6 (the processing means).

In addition, for example, a progressive picture of which the longitudinal number of pixels is integral times of 480 is supplied from a network 5, e.g., an internet or the like, to the integrative encoding system 6. From the network 5, a picture of which the ratio in horizontal to longitudinal of each pixel is 1:1 is supplied similarly to the computer 4.

Here, the video cameras 1 and 2 are made to output the pictures composed of the maximum number of pixels capable of being outputted by them, because if aiming at such pictures for the processing, decoded pictures superior in image quality can be generally obtained comparing with the case of aiming at pictures having less number of pixels in one picture, e.g., pictures in an interlace system.

Moreover, all the longitudinal number of pixels of the pictures to be supplied to the integrative encoding system 6 is the pr scribed value, i.e., integral times of 480 here for example. And their frame rate is also the prescribed value, integral times of 15. This is because when an SD picture is generated by thinning out the pixels forming an HD picture in the spatial direction or temporal direction or when an HD picture is generated by interpolating the pixels forming an SD picture in the spatial direction or temporal direction, the deterioration in the image quality of the SD picture or HD picture to be generated can be reduced.

The longitudinal number of pixels of the HD picture to be outputted to the video camera 1 is set as 960 pixels, because when the longitudinal number of pixels is integral times of 480, "960" most approximates to "1035" being the longitudinal number of pixels of existing HD pictures (FIG. 16), and thus, the deterioration in its image quality can be reduced when in performing crossing over.

The longitudinal number of pixels of the SD picture to be outputted to the video camera 2 is set as 480 pixels, because that value is the integral number of 480 most approximate to the number of pixels adopted in the present NTSC system or PAL system or the like.

If the horizontal number of pixels of the SD picture of which the aspect ratio is 4:3 to be outputted to the video camera 2 is set to 640, the ratio in horizontal to longitudinal of the pixel becomes 1:1 (=4.times.480:3.times.640). As a result, compatibility with the picture to be outputted by the computer 4 or the picture to be supplied from the network 5 is apt to be obtained.

If the horizontal number of pixels of the SD picture of which the aspect ratio is 4:3 to be outputted to the video camera 2 is set to 720, the ratio in horizontal to longitudinal of the pixel becomes 8:9 (=4.times.480:3.times.720). This is equal to the ratio in horizontal to vertical (8:9=16.times.960:9.times.1920) of the pixels that form the HD picture to be outputted by the video camera 1. Therefore, in this case, if the horizontal and vertical numbers of pixels of the SD picture to be outputted by the video camera 2 are doubled and the longitudinal number of pixels is set to 960 pixels that is equal to the longitudinal number of pixels of the HD picture to be outputted by the video camera 1, quasi roundness can be kept (that is, the pictures can avoid being long in the horizontal direction or the vertical direction).

In the integrative encoding system 6, one or more processing necessary for its encoding, e.g., editing processing, compression pressing, channel coding processing or the like, are performed on pictures (digital video signals) supplied thereto considering the other processing. Here, the editing processing includes not only quasi video editing processing in which cut editing is performed or effects are given but also adding information to realize an integrated services digital broadcasting (ISDB: it is interactive broadcasting that digitalizes and transmits various information) and information to attach other values added (for example, information necessary to obtain a decoded picture in further high image quality), and linking a certain picture to another picture, and the like, for example.

Transmit data obtained by the processing in the integrative encoding system 6 is transmitted to the receiving side via a transmission line 13. As the transmission line 13, other than communication lines such as a satellite line, terrestrial wave, CATV network, public network, internet, etc., the process of magnetic recording/reproducing and also recording mediums such as a magnetic disk, optical disk, magnetic tape, magneto-optical disk, and other recording mediums are included.

The transmit data transmitted via the transmission line 13 is received on the receiving side and supplied to an adaptive decoding system 7 (the generation means). The adaptive decoding system 7 is connected to output devices for outputting a picture, e.g., a high definition display unit 9 for displaying the HD picture, a display unit 10 in standard definition or low definition for displaying the SD picture, a printer 11 for printing out a picture, and a computer display 12 connected to a computer (however, the printer 11 is connected via a converter for aspect ratio of pixel 8). The adaptive decoding system 7 generates decoded pictures corresponding to the respective resolution of the display units 9 and 10, printer 11 and computer display 12 and outputting these pictures to them respectively, by linearly coupling the transmit data to the prescribed coefficients.

In the display units 9 and 10 and the computer display 12, the decoded picture from the adaptive decoding system 7 is displayed, and in the printer 11 the decoded picture from the adaptive decoding system 7 is printed out.

Note that, the difference between the ratio in horizontal to longitudinal of each pixel in the display units 9 and 10 and the computer display 12 (hereinafter, it is preferably referred to as aspect ratio of pixel) and the aspect ratio of pixel forming the decoded picture is absorbed by horizontal scanning in each of the display units 9 and 10 and the computer display 12.

On the other hand, in the printer 11, since the difference of the aspect ratio of pixel cannot be absorbed by such horizontal scanning, the converter for aspect ratio of pixel 8 is provided in the preceding stage. There the aspect ratio of pixel of the decoded picture is converted into a value applicable to the printer 11.

Then, processing in the integrative encoding system 6 will be described.

In the integrative encoding system 6, for example, the compression processing is performed considering the editing processing.

In the integrative encoding system 6, an adaptive dynamic range coding (ADRC) processing is performed as the compression processing, for example.

Here, the ADRC processing will be described briefly.

To simplify the description, if considering a block composed of four pixels in line, in the ADRC processing, the maximum value MAX and the minimum value MIN of their pixel values are detected as shown in FIG. 2A. And DR=MAX-MIN is set as the local dynamic range of the block, and the pixel values of the pixels forming the block are requantized into K bit.

Specifically, the minimum value MIN is subtracted from each pixel value in the block. The subtracted values are divided into DR/2.sup.K. And the pixel values are converted into codes (ADRC codes) corresponding to thus obtained divided values. More specifically, if assuming K=2 for example, it is determined that the divided values belong to which range among ranges obtained by dividing the dynamic range DR into four (=2.sup.2) as shown in FIG. 2B. If the divided value belongs to the range of, e.g., the lowest level, the second level from low, the third level from low, or the top level, it is coded into two bits, e.g., 00B, 01B, 10B or 11B (B shows a binary number).

In the ADRC, a smaller value than the number of bits assigned to the pixel is used as the number of bits K at the time of requantizing. Therefore, each pixel is compressed into such small number of bits (however, the minimum value MIN and the dynamic range DR are generated for each block other than that).

Note that, its decoding can be performed by converting the ADRC code 00B, 01B, 10B or 11B into, e.g., the central value L.sub.00 of the lowest level range obtained by dividing the dynamic range DR into four, the central value L.sub.01 of the second level range from low, the central value L.sub.10 of the third level range from low or the central value L.sub.11 of the top level range, and adding the minimum value MIN to the value.

FIG. 3 shows a structural example of ADRC processing circuit that performs the ADRC processing.

Video data is supplied to a blocking circuit 21 and divided there into blocks in prescribed size. That is, the blocking circuit 21 divides the video data into blocks of, e.g., horizontal 4 by longitudinal 4 pixels, and supplies these blocks to a minimum value detecting circuit 22, a maximum value detecting circuit 23 and a computing unit 25.

In the minimum value detecting circuit 22, the minimum value MIN is detected from among 16 (=4.times.4) pixels forming the block from the blocking circuit 21. This minimum value MIN is outputted as one of the signals obtained by the ADRC processing as well as supplied to a computing unit 24 and the computing unit 25.

At the same time, in the maximum value detecting circuit 23, the maximum value MAX is detected from among 16 pixels forming the block from the blocking circuit 21 and supplied to the computing unit 24.

In the computing unit 24, the minimum value MIN is subtracted from the maximum value MAX to obtain the dynamic range DR of the block. This dynamic range DR is outputted as one of the signals obtained by the ADRC processing as well as supplied to a quantizing circuit 26.

In the computing unit 25, the minimum value MIN of the block is subtracted from each of the 16 pixels forming the block, and the subtracted values are supplied to the quantizing circuit 26. In the quantizing circuit 26, the subtracted values from the computing unit 25 are quantized in a quantization step corresponding to the dynamic range DR from the computing unit 24. That is, in the quantizing circuit 26, for example, the output of the computing unit 25 is divided into DR/2.sup.K, and a value which the decimal fractions has omitted is outputted as an ADRC code CODE (requantized result of pixel).

In this embodiment, the number of bits K in requantizing is determined, for example, corresponding to the size of the dynamic range DR of each block and the ADRC code is variable length.

For example, it is assumed that four threshold values T1, T2, T3 and T4 are now set and their size relationship is 0<T1<T2<T3<T4 and also eight bits are assigned to the pixels of an original picture (thus, T4 is less than 2.sup.8).

In this case, in the quantizing circuit 26, it is determined whether or not the dynamic range DR is within one of 0 or more and less than T1, T1 or more and less than T2, T2 or more and less than T3, T3 or more and less than T4 and T4 or more and less than 2.sup.8. And if the dynamic range DR is within one of those, for example, 0 through 4 bits are respectively assigned as the number of bits K in requantizing. Therefore, in this case, the ADRC code becomes 0 bit at the minimum and 4 bit at the maximum.

When the ADRC code is variable length, the number of bits K of the ADRC code is necessary for its decoding. Therefore, the quantizing circuit 26 outputs a threshold code showing that the dynamic range DR is within which range among the above ranges. This threshold code is outputted as the result of ADRC processing together with the minimum value MIN, dynamic range DR and ADRC code.

Note that, the minimum value MIN, dynamic range DR and threshold code are set in variable length for example.

In the above case, the ADRC cod is set in variable length, however, the ADRC cod can be set in fixed length provided that the number of bits K in requantizing is set to a fixed value irrespective of the dynamic range DR of the block.

The ADRC code obtained by the above ADRC processing becomes less than the number of bits assigned to the original pixels. On the other hand, the minimum value MIN, dynamic range DR and ADRC code obtained by the ADRC processing can be used in a block unit, thus the ADRC-processed picture can be edited in a frame unit for example.

From the above, by conducting the ADRC processing as the compression processing and setting the result of the ADRC processing as an object of various editing processing, the almost same editing processing as the case where the original picture before ADRC processing is set an object can be performed, and at the same time, a load of the processing can be reduced comparing with the case where the original picture before ADRC processing is set as the object.

Thus it can be said that the ADRC processing as compression processing is performed considering the editing processing, and the editing processing can be executed efficiently.

The detail of the ADRC has been disclosed in the patent laid-open No. Hei3(1991)-53778 previously filed by the present applicant, or the like, for example.

The ADRC can be referred to as block coding since it performs coding in a block unit. In addition to the ADRC, however, the block coding includes coding which obtains the mean value and the standard deviation of pixels forming a block and also 1-bit flag showing the size relationship between each pixel and the mean value, and the like. Such block coding may be adopted as the compression processing by the integrative encoding system 6.

In the above case, the minimum value MIN and the dynamic range DR are included in the result of the ADRC processing. In the result of the ADRC, also, the minimum value MIN and the maximum value MAX of the block, or the dynamic range DR and the maximum value MAX of the block can be included in addition to that.

Moreover, in the above case, a block is composed of one frame of horizontal 4 pixel by longitudinal 4 pixel. However, the block can be composed of pixels forming plural frames that temporary continue.

By the way, as a method for transmitting the minimum value MIN, dynamic range DR, threshold code and ADRC code for each block obtained by the ADRC processing, for example, there is a method which forms a block disposes a prescribed amount of data of the ADRC result (hereinafter, it is preferably referred to as sync block) following a synchronization pattern for matching and performing transmission in such sync block unit.

When transmission is performed in a sync block unit, the minimum value MIN, dynamic range DR and threshold code are fixed length as the above. Therefore, if these data are disposed at fixed positions in a sync block, even if one sync block cannot be obtained by failure, it does not affect minimum values MIN, dynamic ranges DR and threshold codes disposed in other sync blocks.

Since the ADRC code is variable length, however, in the case where an ADRC code is separately disposed in plural sync blocks because it cannot be contained in one sync block for example, the failure of the one sync block sometimes affects the other sync blocks. Specifically, if the first sync block has failed among the plural sync blocks, it is unknown that the ADRC code disposed at the top of the second sync block is corresponding to which position's pixel in the block, further it is unknown that the bit disposed as the ADRC cod is whether a bit forming the following (a part of) ADRC code disposed at the end of the first sync block or the first bit of the ADRC code. As a result, even an ADRC code disposed in a sync block after the second cannot be fetched. As the above, a certain sync block error propagates to the other sync blocks.

By the way, even if all the ADRC codes of a certain block has lost, if the minimum value MIN has known, the block in which all the pixels have the minimum value MIN as a pixel value can be reproduced. However, since this block has the same pixel value, practically flat, the reproducibility of the original picture is low.

To realize its further higher reproducibility, it can be considered that the ADRC code is separated into, for example, the most significant bit (MSB) and others (hereinafter, it is preferably referred to as remaining bits) and disposing also the MSB at a fixed position in the sync block similarly to the minimum value MIN, dynamic range DR and threshold code. In this case, even if the remaining bit has lost, the block formed of binary can be obtained by inversely quantizing the MSB based on the dynamic range DR. Thus, the picture with higher reproducibility can be obtained comparing with the case where all the ADRC codes have lost.

FIG. 4 shows a structural example of sync block forming circuit that performs sync block processing to form the above sync block.

The minimum value MIN, dynamic range DR and threshold code outputted from the ADRC processing circuit (FIG. 3) are supplied to a multiplexer 32 and the ADRC code CODE is supplied to a separator 31. In the separator 31, the ADRC code is separated into the MSB and the remaining bits and both are supplied to the multiplexer 32.

The synchronization pattern has supplied to the multiplexer 32 in addition to the above data. The multiplexer 32 performs time division multiplexing on the data supplied thereto and forms and outputs a sync block such as shown in FIG. 5.

That is, as shown in FIG. 5, the synchronization pattern having a fixed length is disposed at the top of the sync block, and added data having a fixed length is disposed following it. Here, the added data is formed only fixed length data such as the threshold cod or the like. Following the added data, the remaining bits are disposed by a prescribed number of bytes, that is, after the added data, the remaining bits are disposed from the head to an N.sub.1-1-th byte.

The dynamic range DR, MSB and minimum value MIN are disposed after from the top to an N.sub.1-th byte in the order of DR, MSB, MIN, MSB, DR, . . . , for example. Following the prescribed number of the disposed dynamic range DR, MSB and minimum value MIN, the remaining bits are disposed again. Then, after an N.sub.2-th byte from the top, the prescribed number of the dynamic range DR, MSB and minimum value MIN are disposed again in the aforementioned order. Hereinafter, the similar disposition is repeated until the end of the sync block.

Since the dynamic range DR, the MSB and the minimum value MIN are disposed from the determined position as from the N.sub.1-th byte, N.sub.2-th byte, . . . from the head of the sync block as the above and these data are fixed length, they are disposed at the fixed position in the sync block.

In the above sync block processing, even if an error is caused in the ADRC code (remaining bits), a decoded picture comparatively close to the original picture can be obtained. Furthermore, even if no bit remains, a decoded picture in high reproducibility can be obtained. Thus, extremely to say, it is unnecessary to add an ECC for error correction to the remaining bits, for example. In this case, a load against channel coding processing can be reduced. For this reason, it can be said that the sync block processing is performed considering channel coding processing.

Note that, the detail of the sync block processing has be n disclosed in the patent laid-open No. 2(1990)-162980 previously filed by the present applicant or the like.

The integrative encoding system 6 can also perform hierarchical coding processing as the compression processing in place of the ADRC processing, for example.

In the hierarchical coding, for example, high resolution picture data is set as the picture data of the lowest hierarchy or the first hierarchy, and the picture data of the second hierarchy (compressed picture) of which the number of pixels is less than the first hierarchy's is generated and the picture data of the third hierarchy of which the number of pixels is less than the second hierarchy's is generated. Similarly, picture data are generated until the most upper hierarchy. The picture data of each hierarchy is displayed on a monitor having a resolution (the number of pixels) corresponding to the hierarchy. Thus, on a user side, a picture of the same contents can be viewed by selecting picture data corresponding to the resolution of his monitor from among the hierarchically-coded picture data.

By the way, in the case where picture data having a certain resolution is set as the picture data of the lowest hierarchy (first hierarchy) and the picture data of an upper hierarchy is sequentially formed and all of them are stored or transmitted as they are, much storage capacity or transmission capacity is required by the picture data of the upper hierarchy comparing with the case of storing only the picture data of the lowest hierarchy.

For that reason, here, a hierarchical coding without such increasing of storage capacity or the like is adopted as the compression processing in the integrative encoding system 6.

For example, now the mean value of four pixels, 2.times.2 (horizontal by vertical) pixels in a lower hierarchy is set as the pixels (pixel value) of an upper hierarchy, and performing hierarchical coding in three hierarchies. In this case, considering 8.times.8 pixel as the picture of the lowest hierarchy as shown in FIG. 6A, the mean value m0 of four pixels h00, h01, h02 and h03 at the upper left of the second hierarchy is operated and set as the one pixel of the upper left of the second hierarchy. Similarly, the mean value m1 of four pixels h10, h11, h12 and h13 at the upper right of the picture of the lowest hierarchy, the mean value m2 of four pixels h20, h21, h22 and h23 at the lower left and the mean value m3 of four pixels h30, h31, h32 and h33 at the lower right are operated and set as the one pixel at the upper right, lower left and lower right of the second hierarchy respectively. Furthermore, the mean value (q) of 2.times.2 pixels of the second hierarchy, the four pixels m0, m1, m2 and m3, is operated and set as the pixel of the picture of the third hierarchy, i.e., the most upper hierarchy, here.

If all the above pixels h00 to h03, h10 to h13, h20 to h23, h30 to h33, m0 to m3 and (q) are stored as they are, much storage capacity or the like is required by the pixels m0 to m3 and (q) as described above.

Then, as shown in FIG. 6B, the pixel (q) of the third hierarchy is disposed at, e.g., the position of the pixel m3 at the lower right among the pixels m0 to m3 of the second hierarchy. Thus, the second hierarchy is composed of the pixels m0 to m2 and (q).

As shown in FIG. 6C, the pixel m0 of the second hierarchy is disposed at, e.g., the position of the pixel h03 at the lower right, among the pixels h00 to h03 of the third hierarchy that has used to obtain m0. Similarly, the remaining pixels m1, m2 and (q) of the second hierarchy are disposed in place of the pixels h13, h23 and h33 of the first hierarchy. Note that, the pixel (q) has not obtained directly from the pixels h30 to h33 but since it has been disposed in the second hierarchy in place of m3 obtained directly from them, the pixel (q) is disposed instead of disposing the pixel m3 at the position of the pixel h33.

By conducting as the above, as shown in FIG. 6C, the total of pixels becomes 16 pixel of 4.times.4. This is the same as the case of only the pixels of the lowest hierarchy shown in FIG. 6A. Therefore, in this case, an increase of storage capacity or the like can be prevented.

In this connection, the decoding of the pixel m3 changed into the pixel (q) and the pixels h03, h13, h23 and h33 changed into the pixels m0 to m3 respectively can be performed as follows.

Since (q) is the mean value of m0 to m3, the equation q=(m0+m1+m2+m3)/4 is satisfied. Thus, m3 can be obtained by the equation m3=4.times.q-(m0+m1+m2).

Since m0 is the mean value of h00 to h03, the equation m0=(h00+h01+h02+h03)/4 is satisfied. Thus, h03 can be obtained by the equation h03=4.times.m0-(h00+h01+h02). In similar manner, h13, h23 and h33 can be obtained.

FIG. 7 shows a structural example of hierarchical coding circuit that performs the above hierarchical coding processing. In this hierarchical coding circuit, the aforementioned hierarchical coding in three hierarchies is performed for example.

The first hierarchy (lowest hierarchy) picture data (her, it is progressive picture as described above) is supplied to a mean value calculating circuit 41 and a pixel extracting circuit 43.

In the mean value calculating circuit 41, with respect to the first hierarchy picture, for example, the mean value of 2.times.2 pixels the total four pixels as the above is computed, and the second hierarchy picture is generated. This second hierarchy picture is supplied to a mean value calculating circuit 42 and a pixel extracting circuit 44.

In the mean value calculating circuit 42, with respect to the second hierarchy picture, for example, the mean value of 2.times.2 pixels the total four pixels is computed, and the third hierarchy picture is generated. This third hierarchy picture is supplied to a pixel inserting circuit 45.

In the pixel extracting circuit 43, pixels which correspond to the pixels h03, h13 and h23 described in FIG. 6 are extracted from the first hierarchy picture and the remains are supplied to the pixel inserting circuit 45. In the pixel extracting circuit 44, pixels which correspond to the pixel m3 described in FIG. 6 is extracted and the remains are supplied to the pixel inserting circuit 45.

In the pixel inserting circuit 45, the pixels of the second hierarchy picture (e.g., the pixels m0 to m2) from the pixel extracting circuit 44 are inserted to positions corresponding to the pixels h03, h13 and h23 of the first hierarchy picture from the pixel extracting circuit 43, and the pixel of the third hierarchy picture (e.g., pixel (q)) from the mean value calculating circuit 42 is inserted to a position corresponding to the pixel h33 of the first hierarchy picture. In the above manner, the picture data described in FIG. 6C is formed and this is outputted as the result of the hierarchical coding.

If according to normal hierarchical coding, much storage capacity or transmission capacity is required by the picture data of upper hierarchies, however, according to the hierarchical coding described in FIGS. 6 and 7 (hereinafter, it is preferably referred to as improved hierarchical coding), the amount of data to be obtained as the result is the same as the picture of the lowest hierarchy. For this reason, the improved hierarchical coding can be said to be information compressing processing.

When the hierarchical coding is performed, the picture of a lower hierarchy can be obtained by performing, for example, interpolation or the like using an upper hierarchy picture (however, thus obtained picture is not the same picture as the lower hierarchy picture but a picture deteriorated in image quality.) Therefore, even in the worst, the pictures of all the hierarchies can be obtained provided that the picture of the most upper hierarchy can be restored, so that, for example, the addition of an ECC for error correction is sufficient only performing to the picture of the most upper hierarchy and it is unnecessary to perform to the pictures of all the hierarchies. In this case a load to channel coding processing can be reduced. For this reason, it can be said that the hierarchical coding processing is performed considering the channel coding.

Note that, in the above case, the upper hierarchy picture is gen rated with reducing the number of pixels in the spatial direction, but the upper hierarchy picture can be generated with reducing the number of pixels in the temporal direction for example.

Then, it will be described about the addition of information for realizing an ISDB, one of the editing processing in the integrative encoding system 6.

FIG. 8 shows a structural example of ISDB transmitter being a part of the integrative encoding system 6 that realizes the ISDB.

For example, an SD picture and audio attached thereto are inputted to an encoding part 51, and they are subjected there to the compression processing as the aforementioned ADRC processing. Thus obtained signal is outputted to a multiplexing part 57. Furthermore, the encoding part 51 outputs a synchronizing signal which represents the timing of the compression processing to a time code generating part 52. The time code generating part 52 generates a time cod or the like as additive information to be added to the output of the encoding part 51 synchronizing with the synchronizing signal from the encoding part 51 and outputs this to the multiplexing part 57.

Also in an encoding part 53 or a time code generating part 54, the processing similar to the encoding part 51 or the time code gen rating part 52, except for that an object of the processing is not SD picture but HD picture, is performed respectively. Both of the coded data obtained by the compression processing in the encoding part 51 and the time code outputted by the time code generating part 54 are supplied to the multiplexing part 57.

To an encoding part 55, for example, a computer program, data necessary to execute the program, facsimile data and also data for realizing multimedia such as local information are inputted. These data are compressed there and outputted to the multiplexing part 57 as coded data. Furthermore, the encoding part 55 outputs the synchronizing signal which represents the timing of the compression processing to an additive information generating part 56. The additive information generating part 56 generates additive information which represents a type of the data compressed in the encoding part 55, synchronizing with the synchronizing signal from the encoding part 55 and outputs this to the multiplexing part 57.

Here, the local information to be inputted to the encoding part 55 is information peculiar to each area, and it includes, for example, a weather forecast, map, information on establishments (e.g., the service contents and business hours of restaurants, etc.) and advertisements in each area. With respect to such local information, the additive information generating part 56 generates an area code which represents an area corresponding to each local information as added information.

In the multiplexing part 57, the outputs of the encoding part 51, time code generating part 52, encoding part 53, time code gen rating part 54, encoding part 55 and additive information generating part 56 are multiplexed and outputted.

Then, FIG. 9 shows a structural example of the adaptive decoding system 7 (FIG. 1) in the case where the ADRC processing is performed as the compression processing in the integrative encoding system 6. That is, FIG. 9 shows a structural example of ADRC decoding circuit being a part of the adaptive decoding system 7 that performs ADRC decoding processing when the result of the ADRC processing is decoded.

To a demultiplexer 101, a bit stream in which the result of the ADRC processing is disposed is inputted as transmit data transmitted via the transmission line 13 (FIG. 1), and there the minimum value MIN, dynamic range DR and ADRC code are separated from the transmit data. Note that, in the demultiplexer 101, the separation of the ADRC code is performed by separating the threshold code from the transmit data and recognizing the number of assigned bits to the ADRC code (the above K) based on the threshold code.

The minimum value MIN is supplied to an arithmetic unit 103, and the dynamic range DR and the ADRC code are supplied to an inverse quantizing circuit 102, respectively. In the inverse quantizing circuit 102, the ADRC code is inversely quantized in a quantization step corresponding to the dynamic range DR, and thus obtained inversely-quantized value is supplied to the arithmetic unit 103. In the arithmetic unit 103, the inversely-quantized value from the inverse quantizing circuit 102 is added to the minimum value MIN. Thereby, the pixels are decoded.

If obtaining a pixel for one block, the arithmetic unit 103 supplies this to a frame forming circuit 104. The frame forming circuit 104 sequentially stores the pixels which have supplied in a block unit and outputting them every storing of it.

In the case where the ADRC code has separated into the MSB and the remaining bits as described above, the demultiplexer 101 also performs processing for storing the original ADRC code by combining the MSB with the remaining bits. Furthermore, in the case where the ADRC code has separated into the MSB and the remaining bits, if an error has occurred in the remaining bits, the demultiplexer 101 outputs the MSB to the inverse quantizing circuit 102 as the ADRC code.

By the way, in the ADRC decoding, even if the remaining bits are in error, if the MSB, minimum value MIN and dynamic range DR exist, a decoded picture having fine reproducibility in a certain degree (decoded picture similar to the original picture) can be obtained, as described above. However, if the minimum value MIN or the dynamic range DR is in error, it becomes difficult to decode the block.

FIG. 10 shows a structural example of ADRC decoding circuit capable of decoding a block with relatively fine precision even if the minimum value MIN or the dynamic rang DR is in error. Note that, in FIG. 10, the same reference numerals are added to corresponding parts of FIG. 9, and its description will be omitted hereinafter. That is, this ADRC decoding circuit is basically similarly configured to FIG. 9 except that selectors 105 and 106, a memory 107 and a restoring circuit 108 are newly provided.

To the selector 105, the dynamic range DR outputted by the demultiplexer 101 and a predicted value of the dynamic range DR' outputted by the restoring circuit 108 are supplied. To the selector 106, the minimum value MIN outputted by the demultiplexer 101 and a predicted value of the minimum value MIN' outputted by the restoring circuit 108 are supplied. The demultiplexer 101 here detects whether an error is occurred in the minimum value MIN and the dynamic range DR included in the transmit data, and if an error is caused, it outputs an error signal to the selectors 105 and 106.

The selector 105, when not receiving the error signal, that is, no error is caused in the minimum value MIN and the dynamic rang DR, selects the dynamic range DR outputted by the demultiplexer 101 and outputs this to the inverse quantizing circuit 102. Similarly, the selector 106, when not receiving the error signal, selects the minimum value MIN outputted by the demultiplexer 101 and outputs this to the arithmetic unit 103.

Therefore, in this case, the ADRC decoding processing is performed similar to the case of FIG. 9.

On the other hand, the decoded values of the pixels outputted by the arithmetic unit 103 are supplied to not only the frame forming circuit 104 but also the memory 107. In the memory 107, the decoded values of the pixels from the arithmetic unit 103 are stored in each corresponding address.

Then, in the restoring circuit 108, the decoded values of pixels in the circumference of a block now being an object of the ADRC decoding processing are read out from the memory 107 by the same number as the number of pixels forming a block, that is, 16 pieces in this embodiment as described above. Furthermore, the restoring circuit 108 detects the minimum value and the dynamic range (difference between the maximum value and the minimum value) of that sixteen pixels, and outputs each of them to the selectors 106 and 105 respectively as the predicted value of the minimum value MIN' and the predicted value DR' of the dynamic range of the block now being the object of the ADRC decoding.

The selector 105 or 106, if receiving the error signal from the demultiplexer 101, that is, an error is caused in the minimum value MIN or the dynamic range DR, selects the predicted value of the dynamic range DR' or the predicted value of the minimum value MIN' from the restoring circuit 108, and outputs them to the inverse quantizing circuit 102 or the arithmetic unit 103 respectively.

Thus, in this case, in the inverse quantizing circuit 102, inverse quantization is performed using the predicted value of the dynamic range DR', and in the arithmetic unit 103, the pixels are decoded using the predicted value of the minimum value MIN'.

Marking a certain block, normally a large relationship exists between pixels forming the marked block and pixels in the circumference of the marked block. Therefore, according to the pixels having such correlation, the dynamic range and the minimum value of the marked block can be predicted in relatively fine precision. As a result, using thus predicted values enable to obtain a decoded picture almost similar to the case of using the true minimum value MIN and dynamic range DR.

The detail of the above ADRC decoding processing has been disclosed in the patent laid-open No. S63(1988)-257390 previously filed by the present applicant, for example.

When the ADRC processing is performed, even if an error has caused in the minimum value MIN or the dynamic range DR a decoded picture in a certain degree can be obtained as described above. Moreover, by performing sync block processing in addition to the ADRC processing, to cope with an error in the remaining bits is enabled as described above. Furthermore, also in the case of performing hierarchical coding, it is sufficient to perform the processing for error correction to the picture of the most upper hierarchy at least as described above, and it is not required to always perform the proces