Motion video signal encoder and encoding method Number:7,072,396 from the United States Patent and Trademark Office (PTO) owispatent A motion video signal encoder maximizes image quality without exceeding transmission bandwidth available to carry the encoded motion video signal by comparing encoded frames of the motion video signal to a desired size of frame. If the size of encoded frames differ from the desired size, encoding is adjusted to produce encoded frames closer in size to the desired size. In addition, a cumulative bandwidth error records an accumulated amount of available bandwidth. The cumulative bandwidth error is adjusted as time elapses to add to the available bandwidth and as each frame is encoded to thereby consume bandwidth. As the cumulative bandwidth error grows in magnitude above or below zero, encoding is adjusted as needed to either improve image quality to more completely consume available bandwidth or to reduce image quality to thereby consume less bandwidth and to thereby cause the cumulative bandwidth error to move toward zero. Rapid changes in the amount of change or motion in the motion video signal are detected by comparing the amount of change between two consecutive frames and filtering the amount of change with previously measured amounts of change. Encoding is pre-compensated according to the filtered measurement of rapid change.<H encoding, encoder, Motion, video, si, 7072396.html, Patent, pto, 7072396

Title: Motion video signal encoder and encoding method

Abstract: A motion video signal encoder maximizes image quality without exceeding transmission bandwidth available to carry the encoded motion video signal by comparing encoded frames of the motion video signal to a desired size of frame. If the size of encoded frames differ from the desired size, encoding is adjusted to produce encoded frames closer in size to the desired size. In addition, a cumulative bandwidth error records an accumulated amount of available bandwidth. The cumulative bandwidth error is adjusted as time elapses to add to the available bandwidth and as each frame is encoded to thereby consume bandwidth. As the cumulative bandwidth error grows in magnitude above or below zero, encoding is adjusted as needed to either improve image quality to more completely consume available bandwidth or to reduce image quality to thereby consume less bandwidth and to thereby cause the cumulative bandwidth error to move toward zero. Rapid changes in the amount of change or motion in the motion video signal are detected by comparing the amount of change between two consecutive frames and filtering the amount of change with previously measured amounts of change. Encoding is pre-compensated according to the filtered measurement of rapid change.

Patent Number: 7,072,396 Issued on 07/04/2006 to Wang

Inventors: Wang; Albert S. (Palo Alto, CA)
Assignee: Microsoft Corporation (Redmond, WA)

Appl. No.: 803068

Filed: March 16, 2004

Related U.S. Patent Documents


Application Number	Filing Date	Patent Number	Issue Date
09653124	Aug., 2000	6707852
08841838	May., 1997	6115420
08819507	Mar., 1997	6118817

Current U.S. Class: 375/240.12

Current International Class: H04N 7/12 (20060101); H04B 1/66 (20060101)

Field of Search: 375/240.12,240.01,240.02,240.03,240.04,240.05,240.06,240.07,240.13-240.18 348/699,700 382/238,239,248,251,253

References Cited [Referenced By]

U.S. Patent Documents


4259740	March 1981	Snell et al.
4394774	July 1983	Widergren et al.
4816914	March 1989	Ericsson
4862264	August 1989	Wells et al.
5050161	September 1991	Golestani
5088107	February 1992	Piasecki et al.
5144425	September 1992	Joseph
5247355	September 1993	Frederiksen
5313454	May 1994	Bustini et al.
5351095	September 1994	Kerdranvat
5414469	May 1995	Gonzales et al.
5434848	July 1995	Chimento, Jr. et al.
5453801	September 1995	Kim
5467413	November 1995	Barrett
5473379	December 1995	Horne
RE35158	February 1996	Sugiyama
5490252	February 1996	Macera et al.
5502492	March 1996	Jung
5504744	April 1996	Adams et al.
5512952	April 1996	Iwamura
5519701	May 1996	Colmant et al.
5533021	July 1996	Branstad et al.
5537155	July 1996	O'Connell et al.
5537408	July 1996	Branstad et al.
5557341	September 1996	Weiss et al.
5566175	October 1996	Davis
5574724	November 1996	Bales et al.
5576767	November 1996	Lee et al.
5583980	December 1996	Anderson
5600775	February 1997	King et al.
5606373	February 1997	Dopp et al.
5621660	April 1997	Chaddha et al.
5623312	April 1997	Yan et al.
5623313	April 1997	Naveen
5623690	April 1997	Palmer et al.
5663810	September 1997	Kirk
5699484	December 1997	Davis
5729295	March 1998	Okada
5734419	March 1998	Botsford et al.
5751378	May 1998	Chen et al.
5784115	July 1998	Bozdagi
5815217	September 1998	Kumazawa et al.
5825473	October 1998	Kodaira
5835163	November 1998	Liou et al.
5859667	January 1999	Kondo et al.
5875003	February 1999	Kato et al.
5892535	April 1999	Allen et al.
5929916	July 1999	Legall et al.
5936940	August 1999	Marin et al.
5978544	November 1999	Shimada et al.
6011589	January 2000	Matsuura et al.
6055025	April 2000	Shahraray
6081551	June 2000	Etoh
6115420	September 2000	Wang
6163574	December 2000	Yagasaki
6335990	January 2002	Chen et al.
6456648	September 2002	Bultan et al.
6480670	November 2002	Hatano et al.
6678324	January 2004	Yamauchi
6833846	December 2004	Hasser

Other References

"Video Coding for Low Bitrate Communication", ITU-T, Draft H.263: Line Transmission of Non-Telephone Signals, Int'l Telecommunications Union, (May 2, 1996). cited by other .
Chaddha, N., et al., "An end to end software only scalable video delivery system", Proceedings Networks and Operating System Support for Digital Audio and Video, pp. 130-141, (Apr. 21, 1995). cited by other .
Chaddha, N., et al., "Hierarchical Vector Quantization of Perceptually Weighted Block Transforms", IEEE, pp. 3-12, (1995). cited by other .
Linde, Y., et al., "An Algorithm for Vector Quantizer Design", IEEE Transactions on Communications, vol. COM-28, No. 1, pp. 84-95, (Jan. 1980). cited by other .
Moura, J.M., et al., "Retrieving Quality Video Across Hetergeneous Networks--Video Over Wireless", IEEE Personal Communications, 44-54, (Feb. 1996). cited by other.

Primary Examiner: Diep; Nhon
Attorney, Agent or Firm: Lee & Hayes, PLLC

Parent Case Text

CROSS REFERENCE TO RELATED APPLICATIONS

This is a continuation of U.S. patent application Ser. No. 09/653,124, filed Aug. 31, 2000, now U.S. Pat. No. 6,707,852, which is a divisional of U.S. patent application Ser. No. 08/841,838, filed on May 5, 1997, now U.S. Pat. No. 6,115,420, which is a continuation-in-part of U.S. patent application Ser. No. 08/819,507, filed Mar. 14, 1997, now U.S. Pat. No. 6,118,817, each of which are incorporated hereby reference.

Claims

What is claimed is:

1. A method for encoding a motion video signal, the method comprising: comparing first and second frames of the motion video signal to one another to determine a current absolute pixel difference between the first and second frames; determining, based at least in part on comparing the current absolute pixel difference to a filtered previous absolute pixel difference, whether the second frame represents a scene change in a motion video image represented by the motion video image; encoding the second frame as an independent frame upon a condition in which the second frame represents the scene change in the motion video image; and encoding the second frame as a motion-compensated frame upon a condition in which the second frame does not represent the scene change in the motion video image.

2. A computer readable medium useful in association with a computer which includes a processor and a memory, the computer readable medium including computer instructions which are configured to cause the computer to encode a motion video signal by performing the steps of: comparing first and second frames of the motion video signal to one another to determine a current absolute pixel difference between the first and second frames; determining, based at least in part on comparing the current absolute pixel difference to a filtered previous absolute pixel difference, whether the second frame represents a scene change in a motion video image represented by the motion video image; encoding the second frame as an independent frame upon a condition in which the second frame represents the scene change in the motion video image; and encoding the second frame as a motion-compensated frame upon a condition in which the second frame does not represent the scene change in the motion video image.

3. A computer system comprising: a processor; a memory operatively coupled to the processor; and a motion video signal encoder which executes in the processor from the memory and which, when executed by the processor, causes the computer system to encode a motion video signal by performing the steps of comparing first and second frames of the motion video signal to one another to determine a current absolute pixel difference between the first and second frames; determining, based at least in part on comparing the current absolute pixel difference to a filtered previous absolute pixel difference, whether the second frame represents a scene change in a motion video image represented by the motion video image; encoding the second frame as an independent frame upon a condition in which the second frame represents the scene change in the motion video image; and encoding the second frame as a motion-compensated frame upon a condition in which the second frame does not represent the scene change in the motion video image.

4. A computer readable medium comprising instructions which, when executed by a computer, performs the method of claim 1.

Description

FIELD OF THE INVENTION

The present invention relates to digital video signal compression and, in particular, to a particularly efficient signal encoding mechanism for encoding digital video signals according to digital video standards such as the ITU standard H.263.

BACKGROUND OF THE INVENTION

With the advent of digital video products and services, such as Digital Satellite Service (DSS) and storage and retrieval of video streams on the Internet and, in particular, the World Wide Web, digital video signals are becoming ever present and drawing more attention in the marketplace. Because of limitations in digital signal storage capacity and in network and broadcast bandwidth limitations, compression of digital video signals has become paramount to digital video storage and transmission. As a result, many standards for compression and encoding of digital video signals have been promulgated. For example, the International Telecommunication Union (ITU) has promulgated the H.261 and H.263 standards for digital video encoding. Additionally, the International Standards Organization (ISO) has promulgated the Motion Picture Experts Group (MPEG), MPEG-1, and MPEG-2 standards for digital video encoding.

These standards specify with particularity the form of encoded digital video signals and how such signals are to be decoded for presentation to a viewer. However, significant discretion is left as to how the digital video signals are to be transformed from a native, uncompressed format to the specified encoded format. As a result, many different digital video signal encoders currently exist and many approaches are used to encode digital video signals with varying degrees of compression achieved.

In general, greater degrees of compression are achieved at the expense of video image signal loss and higher quality motion video signals are achieved at the expense of lesser degrees of compression and thus at the expense of greater bandwidth requirements. It is particularly difficult to balance image quality with available bandwidth when delivery bandwidth is limited. Such is the case in real-time motion video signal delivery such as video telephone applications and motion video on demand delivery systems. It is generally desirable to maximize the quality of the motion video signal as encoded without exceeding the available bandwidth of the transmission medium carrying the encoded motion video signal. If the available bandwidth is exceeded, some or all of the sequence of video images are lost and, therefore, so is the integrity of the motion video signal. If an encoded motion video signal errs on the side of conserving transmission medium bandwidth, the quality of the motion video image can be compromised significantly.

The format of H.263 encoded digital video signals is known and is described more completely in "ITU-T H.263: Line Transmission of Non-Telephone Signals, Video Coding for Low Bitrate Communication" (hereinafter "ITU-T Recommendation H.263"). Briefly, in H.263 and other encoded video signal standards, a digital motion video image signal, which is sometimes called a video stream, is organized hierarchically into groups of pictures which include one or more frames, each of which represents a single image of a sequence of images of the video stream. Each frame includes a number of macroblocks which define respective portions of the video image of the frame. An I-frame is encoded independently of all other frames and therefore represents an image of the sequence of images of the video stream without reference to other frames. P-frames are motion-compensated frames and are therefore encoded in a manner which is dependent upon other frames. Specifically, a P-frame is a predictively motion-compensated frame and depends only upon one I-frame or, alternatively, another P-frame which precedes the P-frame in the sequence of frames of the video image. The H.263 standard also describes BP-frames; however, for the purposes of description herein, a BP-frame is treated as a P-frame.

All frames are compressed by reducing redundancy of image data within a single frame. Motion-compensated frames are further compressed by reducing redundancy of image data within a sequence of frames. Since a motion video signal includes a sequence of images which differ from one another only incrementally, significant compression can be realized by encoding a number of frames as motion-compensated frames, i.e., as P-frames. However, errors from noise introduced into the motion video signal or artifacts from encoding of the motion video signal can be perpetuated from one P-frame to the next and therefore persist as a rather annoying artifact of the rendered motion video image. It is therefore desirable to periodically send an I-frame to eliminate any such errors or artifacts. Conversely, I-frames require many times more bandwidth, e.g., on the order of ten times more bandwidth, than P-frames, so encoding I-frames too frequently consumes more bandwidth than necessary. Accordingly, determining when to include an I-frame, rather than a P-frame, in an encoded video stream is an important consideration when maximizing video image quality without exceeding available bandwidth.

Another important consideration when maximizing video image quality within limited signal bandwidth is the compromise between image quality of and bandwidth consumed by the encoded video signal as represented by an encoding parameter .lamda.. In encoding a video signal, a particular value of encoding parameter .lamda. is selected as a representation of a specific compromise between image detail and the degree of compression achieved. In general, a greater degree of compression is achieved by sacrificing image detail, and image detail is enhanced by sacrificing the degree of achievable compression of the video signal. In the encoding standard H.263, a quantization parameter Q effects such a comprise between image quality and consumed bandwidth by controlling a quantization step size during quantization in an encoding process.

However, a particular value of encoding parameter .lamda. which is appropriate for one motion video signal can be entirely inappropriate for a different motion video signal. For example, motion video signals representing a video image which changes only slightly over time, such as a news broadcast (generally referred to as "talking heads"), can be represented by relatively small P-frames since successive frames differ relatively little. As a result, each frame can include greater detail at the expense of less compression of each frame. Conversely, motion video signals representing a video image which changes significantly over time, such as fast motion sporting events, require larger P-frames since successive frames differ considerably. Accordingly, each frame requires greater compression at the expense of image detail.

Determining an optimum value of encoding parameter .lamda. for a particular motion video signal can be particularly difficult. Such is especially true for some motion video signals which include both periods of little motion and periods of significant motion. For example, in a motion video signal representing a football game includes periods where both teams are stationary awaiting the snap of the football from the center to the quarterback and periods of sudden extreme motion. Selecting a value of encoding parameter .lamda. which is too high results in sufficient compression that frames are not lost during high motion periods but also in unnecessarily poor image quality during periods were players are stationary or moving slowly between plays. Conversely, selecting a value of encoding parameter .lamda. which is too low results in better image quality during periods of low motion but likely results in loss of frames due to exceeded available bandwidth during high motion periods.

A third factor in selecting a balance between motion video image quality and conserving available bandwidth is the frame rate of the motion video signal. A higher frame rate, i.e., more frames per second, provides an appearance of smoother motion and a higher quality video image. At the same time, sending more frames in a given period of time consumes more of the available bandwidth. Conversely, a lower frame rate, i.e., fewer frames per second, consumes less of the available bandwidth but provides a motion video signal which is more difficult for the viewer to perceive as motion between frames and, below some threshold, the motion video image is perceived as a "slide show," i.e., a sequence of discrete, still, photographic images. However, intermittent loss of frames resulting from exceeding the available threshold as a result of using an excessively high frame rate provides a "jerky" motion video image which is more annoying to viewers than a regular, albeit low, frame rate.

I-frame placement and encoding parameter .lamda. value selection combine to represent a compromise between motion video image quality and conservation of available bandwidth. However, to date, conventional motion video encoders have failed to provide satisfactory motion video image quality within the available bandwidth.

SUMMARY OF THE INVENTION

In accordance with the present invention, a primary open loop rate control selects an optimized encoding parameter .lamda. by determining a desired size for an individual frame and comparing the size of the frame as encoded to the desired size. Encoding parameter .lamda. represents a compromise between the distortion introduced into a motion video signal as a result of encoding the motion video signal and the amount of data required to represent the motion video signal as encoded and therefore the amount of bandwidth consumed in delivering the encoded motion video signal. A specific value of encoding parameter .lamda. represents a specific compromise between image quality and consumed bandwidth. Encoding a motion video signal in accordance with encoding parameter .lamda. effects the compromise between consumed bandwidth and video image quality represented by encoding parameter .lamda.. If the encoded frame size is greater than the desired size, encoding parameter .lamda. is increased to reduce the size of subsequently encoded frames to consume less bandwidth at the expense of image quality. Conversely, if the encoded frame size is less than the desired size, encoding parameter .lamda. is reduced to increase the size of subsequently encoded frames to improve image quality and to fully consume available bandwidth. As a result, each frame is encoded in a manner which maximizes image quality while approaching full consumption of available bandwidth and guarding against exceeding available bandwidth.

Further in accordance with the present invention, a secondary close loop rate control ensures that overall available bandwidth is never exceeded. Encoding parameter .lamda. is selected by accumulating a cumulative bandwidth error which represents the amount by which bandwidth consumed by encoding a motion video signal deviates from the amount of bandwidth which is available for encoding of the motion video signal. The cumulative bandwidth error accumulates as time passes and is consumed by encoded frames which are transmitted through the communication medium whose bandwidth is measured. Encoding frames which are consistently slightly too large results in incremental reductions in the cumulative bandwidth error which can have a negative value and which can grow in magnitude as a result of such reductions. In response to the reduction of the cumulative bandwidth error, encoding parameter .lamda. is increased to reduce the size of subsequently encoded frames to consume less bandwidth at the expense of image quality. Encoding frames which are consistently slightly too small results in a incremental increases in the cumulative bandwidth error. In response to the increases in the cumulative bandwidth error, encoding parameter .lamda. is decreased to increase the size of subsequently encoded frames to improve image quality and to fully consume available bandwidth. As a result, gradual trends of the primary open loop rate control which allow available bandwidth to accumulate or to be exceeded are thwarted. In addition, secondary closed loop rate control contributes to selecting an optimum compromise between image quality and available bandwidth.

Further in accordance with the present invention, motion video images which change from a slow changing scene to a rapidly changing scene are detected and encoding parameter .lamda. is adjusted to more quickly adapt to the changing motion video signal and to continue to provide a particularly desirable compromise between image quality and available bandwidth. In particular, the absolute pixel difference between two consecutive frames is measured. Previously measured absolute pixel differences corresponding to previously encoded frames of the motion video signal are filtered to form a filtered previous absolute pixel difference. Encoding parameter .lamda. is adjusted in accordance with the absolute pixel difference and the filtered previous absolute pixel difference independently of changes to encoding parameter .lamda. as determined by the primary open loop rate control and secondary closed loop rate control described above. In particular, if the current absolute pixel difference is greater than the filtered previous absolute pixel difference, showing an increase in the rate of change between frames, encoding parameter .lamda. is increased to reduce the size of subsequently encoded frames and to thereby make additional bandwidth available for such encoded frames. Conversely, if the current absolute pixel difference is less than the filtered previous absolute pixel difference, a decrease in the rate of change between frames is detected and encoding parameter .lamda. is decreased to improve image quality and to more fully consume available bandwidth. As a result, the optimum compromise achieved by the primary open loop rate control and the secondary closed loop rate control is more stable, i.e., reaches equilibrium more quickly, when the rate of change between frames of a motion video image changes significantly and rapidly.

Further in accordance with the present invention, a scene change between frames of a motion video signal are detected and the first frame of the new scene is encoded as an I-frame. As a result, the encoded frame is only slightly larger than an equivalent P-frame since a scene change represents a particularly large change between the current frame and the previous frame. In addition, the encoding of the next I-frame is postponed until the expiration of a full I-frame interval which starts with the encoding of the scene change I-frame, even if the previous I-frame interval had partially elapsed but had not expired prior to encoding of the I-frame. A scene change is detected by measuring the absolute pixel difference between the current frame and the previous frame, filtering the absolute pixel difference with a previously filtered absolute pixel difference, and comparing the newly filtered absolute pixel difference to a threshold. The threshold is proportional to the previously filtered absolute pixel difference. If the newly filtered absolute pixel difference is greater than the threshold, the current frame is determined to be the first frame of a new scene and is therefore encoded as an I-frame.

Each of these mechanisms represents a significant improvement over the prior art and enhances the quality of a motion video image without exceeding available bandwidth. These mechanisms can be used individually or in combination.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a video signal encoder according to the present invention.

FIG. 2 is a block diagram of the encoding parameter .lamda. adjuster of FIG. 1.

FIG. 3 is a logic flow diagram illustrating the processing of the primary open loop rate control of FIG. 2.

FIG. 4 is a logic flow diagram illustrating the processing of the secondary close loop rate control of FIG. 2.

FIG. 5 is a logic flow diagram illustrating the processing of the .lamda. pre-compensator of FIG. 2.

FIG. 6 is a logic flow diagram illustrating the processing of the I/P framer of FIG. 1.

FIG. 7 is a logic flow diagram illustrating the determination by the I/P framer of FIG. 1 that the current frame of a motion video signal represents a scene change.

FIG. 8 is a logic flow diagram illustrating the processing of the frame rate controller of FIG. 1.

FIG. 9 is a block diagram of a server computer system, which includes the video signal encoder of FIG. 1, and a client computer system which includes a video signal decoder and a computer network which connects the two computers.

FIG. 10 is a block diagram of the server computer system of FIG. 9.

FIG. 11 is a block diagram of the client computer system of FIG. 9.

DETAILED DESCRIPTION

In accordance with the present invention, a video signal encoder 100 (FIG. 1) maximizes image quality without exceeding bandwidth available for transmitting the encoded motion video signal. Video signal encoder 100 receives a frame of a video signal from a video source (not shown in FIG. 1) which can include, for example, a video camera, a video cassette player, a video laser disk player, or similar video source. Video signal encoder 100 stores the frame in buffer 102 after moving any frame previously stored in buffer 102 into buffer 104. Thus, video signal encoder 100 stores two consecutive frames in buffers 102 and 104. The frame stored in buffer 102 is sometimes referred to herein as the current frame, and the frame stored in buffer 104 is sometimes referred to herein as the previous frame. I/P framer 106 of video signal encoder 100 includes a motion estimator/compensator 108 which retrieves the current frame from buffer 102 and a reconstructed previous frame from a buffer 128 and derives motion vectors which represent motion between the current and reconstructed previous frames. The reconstructed previous frame is reconstructed from a previously encoded frame as described more completely below. For each of one or more macroblocks of the current frame, motion estimator/compensator 108 derives a motion vector which specifies a portion of the reconstructed previous frame which the macroblock corresponds and an associated motion vector error signal. A motion vector specifies a motion-compensated macroblock in terms of a vector to an equal-sized portion of another frame. A macroblock specified by a motion vector of a particular macroblock is sometimes referred to herein as a macroblock which is temporally displaced from the particular macroblock. A motion vector error signal represents an amount of variation between the macroblock and a temporally displaced macroblock of the macroblock.

Motion estimator/compensator 108 produces a current motion-compensated frame from the motion vectors and the current and reconstructed previous frames received from buffers 102 and 128. Motion estimator/compensator 108 passes the motion-compensated frame to transform coder 110 which performs a transformation, e.g., a direct cosine transformation (DCT), on the motion-compensated macroblocks of the motion-compensated frame to produce a transformed frame. Transform coder 110 passes the transformed frame to a quantizer 112. Quantizer 112 quantizes coefficients used in transform coder 110 and these coefficients are then used later for Huffman coding the transformed frame to complete compression of the current frame retrieved from buffer 102. Huffman coding is described more completely in copending U.S. patent application Ser. No. 08/818,805 for "Method and Apparatus for Implementing Motion Detection and Estimation in Video Compression" filed on Mar. 14, 1997, and that description is incorporated herein by reference.

As described briefly above, a reconstructed previous frame is used to estimate motion between consecutive frames. The reconstructed previous frame is formed as follows. A dequantizer 120 receives the encoded current frame from quantizer 112 and performs the inverse of the quantization performed by quantizer 112. The dequantized frame is transferred from dequantizer 120 to a transform decoder 122 which performs an inverse transformation of the transformation performed by transform coder 110. A frame reconstructor 124 receives the transformed frame and reconstructs a reconstructed current frame therefrom. Specifically, frame reconstructor 124 reconstructs motion-compensated macroblocks of the frame received from transform decoder 122 by reference to a previously reconstructed frame stored in buffer 128. The reconstructed current frame is stored in a buffer 126 and the reconstructed frame which is previously stored in buffer 126 is moved to buffer 128. Therefore buffer 128 stores a reconstructed previous frame which is reconstructed from the previously encoded frame. Dequantizer 120, transform decoder 122, and frame reconstructor 124 are conventional.

Quantization by quantizer 112 is important in the encoding of a frame because a significant loss of signal for the sake of better compression can happen during quantization of the transform parameters. Quantization of coefficients used in transform coder 112 is known and is described, for example, in ITU-T Recommendation H.263 and that discussion is incorporated herein by reference. During quantization, and during other steps in the encoding of a motion video signal, an expression, D+.lamda.R, represents a selected compromise between motion video image quality and the degree of compression and thus consumed bandwidth. Specifically, D represents the distortion of a particular block of a frame of the motion video signal, i.e., the amount by which the block as encoded deviates from the block prior to encoding, and R represents the rate, i.e., amount of data required to encode the block. .lamda. 114 (FIG. 1) stores a value of the encoding parameter .lamda. which represents a specific trade-off between distortion and rate.

The expression, D+.lamda.R, is known and is used to select among various options when encoding a motion video signal. Generally, the expression, D+.lamda.R, is evaluated for each of a number of encoding options and the option for which the expression has a minimized value is selected as having the best compromise between image quality and available bandwidth. In particular, such selections are typically made during quantization where the bulk of image quality sacrifices for better compression are made. In general, a larger value stored in .lamda. 114 results in a greater degree of compression, and thus less consumed bandwidth, at the expense of greater signal loss. Conversely, a smaller value stored in .lamda. 114 generally results in less signal loss at the expense of a smaller degree of compression and thus greater consumed bandwidth.

As described above in greater detail, the appropriate value of .lamda. 114 for a given motion video signal depends on the particular subject matter of the particular motion video signal and, in fact, can change dramatically within a given motion video signal. Accordingly, .lamda. 114 is controlled by a .lamda. adjuster 116. .lamda. adjuster 116 is shown in greater detail in FIG. 2. .lamda. adjuster 116 includes generally two .lamda. adjustment mechanisms. The first includes a primary open loop rate control 202 and a secondary closed loop rate control 204. The second includes a .lamda. pre-compensator 206.

In general, primary open loop rate control 202 (FIG. 2) adjusts .lamda. 114 for each P-frame to achieve a desired size of encoded P-frame. Processing of primary open loop rate control 202 is illustrated generally by logic flow diagram 300 (FIG. 3) in which processing begins with loop step 302. Loop step 302, in conjunction with next step 314, defines a loop in which each P-frame is processed according to steps 304 312. In step 304, primary open loop rate control 202 (FIG. 2) determines a target size for the current P-frame. In general, the target size represents an ideal size for the current P-frame such that exactly the entire available bandwidth is completely consumed by the motion video stream produced by video signal encoder 100 (FIG. 1). First, the amount of total bandwidth occupied by I-frames is determined and subtracted from the total available bandwidth to determine the amount of bandwidth available for P-frames. In one embodiment, an I-frame is encoded every 6.5 seconds in a frame rate of 10 frames per second, and I-frames occupy about 10 15% of the available bandwidth. Accordingly, 85 90% of the total available bandwidth is available for P-frames. The target frame size for the current P-frame is determined from the time elapsed between the current P-frame and the previous frame and the amount of total available bandwidth for P-frames. If P-frames are encoded in such a way that each P-frame is smaller than the target size, then additional bandwidth is available and video image quality is unnecessarily poor. Conversely, if P-frames are encoded in such a way that each P-frame is larger than the target size, then the available bandwidth will eventually be exceeded.

In test step 306 (FIG. 3), primary open loop rate control 202 (FIG. 2) determines whether the current P-frame, i.e., the P-frame most recently encoded by I/P framer 106 (FIG. 1) is larger than the target size. If the current P-frame is larger than the target size, processing transfers from test step 306 (FIG. 3) to step 308. In step 308, primary open loop rate control 202 (FIG. 2) increases .lamda. 114 to thereby cause subsequent frames to be encoded with an increased degree of compression and a commensurate degradation of motion video image quality. By increasing the degree of compression of the motion video signal, exceeding the available bandwidth is avoided.

In one embodiment, .lamda. 114 is increased according to the following equation. d.lamda.=.lamda.(surplus/target)damping_factor (1)

In equation (1), d.lamda. represents the amount by which .lamda. 114 is changed in step 308 (FIG. 3). Target represents the target size, and surplus represents the amount by which the current frame exceeds the target size. Thus, .lamda. 114 is increased in step 308 by an amount which is proportional to the percentage of the target size which is exceeded by the current P-frame. Damping_factor represents a percentage of the correction, i.e., surplus/target, which is applied to .lamda. 114 in a single performance of step 308 and is 0.1 (i.e., 10%) in one embodiment. Application of a full correction of .lamda. 114 in each and every performance of step 308 causes .lamda. 114 to vary widely and frequently such that the encoded motion video signal varies significantly and frequently in quality and the visual effect of such is annoying to viewers. By setting damping_factor to 0.1, changes in .lamda. 114 and, accordingly, in the motion video image quality are smoothed.

In general, it is preferred that .lamda. 114 (FIGS. 1 and 2) changes sufficiently to quickly converge to a relatively optimum value such that image quality is maximized while available bandwidth is not exceeded. However, configuring .lamda. adjuster 116 to adjust .lamda. 114 excessively to converge too quickly can cause the value of .lamda. 114 to be over-adjusted such that correction in the reverse direction is required for subsequent frames. As a result, the value of .lamda. 114 oscillates, and such oscillation can produce perceptible and undesirable artifacts in the decoded motion video signal. Therefore, it is preferred that .lamda. 114 changes quickly enough to converge quickly to a relatively optimum value but changes slowly enough to avoid oscillation about the relatively optimum value. The adjustments to .lamda. 114 described herein have been determined to provide acceptable results.

After step 308 (FIG. 3), processing of the current P-frame by primary open loop rate control 202 (FIG. 2) is complete.

If, in test step 306 (FIG. 3), primary open loop rate control 202 (FIG. 2) determines that the size of the current P-frame is not larger than the target size, processing transfers to test step 310 (FIG. 3). In test step 310, primary open loop rate control 202 (FIG. 2) determines whether the size of the current P-frame is smaller than the target size. If the size of the current frame is not smaller than the target size, processing of the current P-frame by primary open loop rate control 202 is complete. Thus, if the size of the current P-frame is equal to the target size, .lamda. 114 is not adjusted by primary open loop rate control 202. Conversely, if the size of the current frame is smaller than the target size, processing transfers from test step 310 (FIG. 3) to step 312.

In step 312, primary open loop rate control 202 (FIG. 2) decreases .lamda. 114 to increase the image quality of subsequent P-frames and to more completely utilize the bandwidth available for encoding of P-frames. In one embodiment, .lamda. 114 is decreased according to the following equation. d.lamda.=.lamda.(deficit/current_size)damping_factor (2)

In equation (2), d.lamda. represents the amount by which .lamda. 114 is changed in step 312. Current_size represents the size of the current P-frame, and deficit represents the amount by which the target size exceeds the size of the current frame. Thus, .lamda. 114 is decreased in step 312 by an amount which is proportional to the percentage of the size of the current P-frame which is exceeded by the target size. Damping_factor is directly analogous to the damping factor described above with respect to equation (1). In one embodiment, the damping factors in equations (1) and (2) are equivalent.

It is important to prevent .lamda. 114 from decreasing too rapidly in which case available bandwidth can easily and quickly be exceeded. Therefore, primary open loop rate control 202 (FIG. 2) prevents .lamda. 114 from being decreased by more than one-half of the previous value of .lamda. 114 in a single performance of step 312 (FIG. 3) in one embodiment. After step 312, processing of the current P-frame by primary open loop rate control 202 (FIG. 2) is complete.

Thus, primary open loop rate control 202 determines an appropriate and relatively optimum compromise between image quality and bandwidth availability by comparing the size of the current encoded P-frame to a target, theoretically optimum, encoded P-frame size. However, use of primary open loop rate control 202 alone does not guarantee that the total available bandwidth will not be exceeded. For example, if P-frames are consistently slightly larger than the target size, available bandwidth can be eventually exceeded. Therefore, secondary closed loop rate control 204 uses a cumulative bandwidth buffer to ensure that the total available bandwidth is never exceeded.

Secondary closed loop rate control 204 monitors a cumulative bandwidth error to ensure that small cumulative excesses of bandwidth overlooked by primary open loop rate control 202 do not result in the encoded motion video signal exceeding the overall available bandwidth. Specifically, if the cumulative bandwidth error grows too large in magnitude, adjustments to .lamda. 114 by secondary closed loop rate control 204 are large enough to compensate for any adjustments to .lamda. 114 by primary open loop rate control 202.

Processing by secondary closed loop rate control 204 is illustrated in logic flow diagram 400 (FIG. 4) in which processing begins in step 402. In step 402, secondary closed loop rate control 204 initializes a cumulative bandwidth error to represent an initial cumulative deviation from the available bandwidth when secondary closed loop rate control 204 initially begins processing. In one embodiment, the cumulative bandwidth error has an initial value of zero to indicate that the available bandwidth is not exceeded and is completely consumed. In step 404, secondary closed loop rate control 204 determines a maximum allowable cumulative bandwidth error, which is one (1) second in one embodiment. Specifically, a maximum allowable error is established and represents the amount of data which can be transferred through a delivery medium in one second. As described more completely below, secondary closed loop rate control 204 adjusts .lamda. 114 to prevent the cumulative bandwidth error from growing in magnitude, in either a positive direction or a negative direction, from zero. An excessively high positive cumulative bandwidth error indicates that too much bandwidth is being consumed and that delivery of the encoded motion video signal can be adversely affected. Conversely, an excessively high-magnitude negative cumulative bandwidth error indicates that more bandwidth is available than is being consumed and that the available bandwidth can support greater motion video image quality.

Loop step 406 and next step 418 define a loop in which each frame, both I-frames and P-frames, are processed according to steps 408 416. In step 408, secondary closed loop rate control 204 adjusts the cumulative bandwidth error according to the size of the current encoded frame, i.e., the most recently encoded frame received from I/P framer 106 (FIG. 1) whether an I-frame or a P-frame. In particular, secondary closed loop rate control 204 adds to the cumulative bandwidth error the amount of data which can by carried by the transmission medium in the time which elapses between the previous encoded frame and the current encoded frame. Secondary closed loop rate control 204 subtracts from the cumulative bandwidth error the amount of data used to encode the current frame. A particularly large encoded frame, such as an I-frame for example, consumes more bandwidth than allocated for the amount of time which elapses between the current encoded frame and the preceding encoded frame. Accordingly, secondary closed loop rate control 204 notes a decrease in the cumulative bandwidth error or an increase in the magnitude of the cumulative bandwidth error if the cumulative bandwidth error is negative. Conversely, a particularly small frame consumes less bandwidth than allocated for the time which elapses between the current encoded frame and a preceding encoded frame and results in an increase in the cumulative bandwidth error or a decrease in the magnitude of the cumulative bandwidth error if the cumulative bandwidth error is negative.

In test step 410, secondary closed loop rate control 204 determines whether the cumulative bandwidth error is less than zero, i.e., indicates that more than the available bandwidth is being consumed. If the cumulative bandwidth error is not less than zero, processing transfers to test step 414 which is described more completely below. Conversely, if the cumulative bandwidth error is less than zero, bandwidth consumption is exceeding the available bandwidth and processing transfers to step 412 in which secondary closed loop rate control 204 increases .lamda. 114. Accordingly, image quality is sacrificed to conserve bandwidth used by subsequent frames. In one embodiment, .lamda. 114 is adjusted according to the following equation. d.lamda.=.lamda.(error_percentagedamping_factor+0.02) (3)

In equation (3), d.lamda. represents the amount by which .lamda. 114 is adjusted in step 412. Error_percentage represents the percentage of the maximum allowable bandwidth error indicated by the cumulative bandwidth error. Damping_factor represents a percentage of error_percentage which is to be applied to .lamda. 114 for any one frame and is fixed at 10% in one embodiment. The constant 0.02 in equation (3) represents a minimum correction of .lamda. 114 whenever a negative cumulative bandwidth error is detected by secondary closed loop rate control 204. In equation (3), the minimum correction is 2% of the current value of .lamda. 114. Without a minimum correction factor of 0.02, small cumulative bandwidth errors would be permitted to grow significantly before the error percentage of equation (3) grows large enough to correct .lamda. 114. After step 412, processing of the current frame by secondary closed loop rate control 204 completes.

In test step 414, secondary closed loop rate control 204 (FIG. 2) determines whether the cumulative bandwidth error is greater than zero. If the cumulative bandwidth error is equal to zero, processing of the current frame by secondary closed loop rate control 204 completes and .lamda. 114 is not adjusted since precisely the right amount of bandwidth is being consumed by the encoded motion video signal. Conversely, if the cumulative bandwidth error is greater than zero, unused bandwidth is accumulating and processing transfers to step 416 in which secondary closed loop rate control 204 decreases .lamda. 114. Accordingly, video image quality is increased at the expense of increased bandwidth consumed by subsequent frames. This is appropriate since unused accumulating bandwidth is detected and using such bandwidth improves the overall perceived quality of the motion video image. Therefore, small excesses in consumed bandwidth which are undetected by primary open loop rate control 202 but which accumulate over time are detected by secondary closed loop rate control 204 and available bandwidth is not exceeded. In one embodiment, .lamda. 114 is adjusted in accordance with equation (3) as described above with respect to step 412. In the context of step 416, d.lamda. of equation (3) is negative to result in a decrease in .lamda. 114. After step 416, processing of the current encoded frame by secondary closed loop rate control 204 completes.

Thus, primary open loop rate control 202 adjusts .lamda. 114 for each encoded frame to reach an optimum compromise between image quality and conserved bandwidth while secondary closed loop rate control 204 ensures that small excessive uses of bandwidth do not accumulate such that frames are ultimately lost as a result of exceeding available bandwidth. It should be noted that adjustments to .lamda. 114 in steps 412 (FIG. 4) and 416 are in addition to those made in steps 308 (FIG. 3) and 312. Accordingly, significant cumulative buffer errors resulting from small, incremental deviations from the target frame size permitt