Parametric compression/decompression modes for quantization matrices for digital audio Number:7,143,030 from the United States Patent and Trademark Office (PTO) owispatent Quantization matrices facilitate digital audio encoding and decoding. An audio encoder generates and compresses quantization matrices; an audio decoder decompresses and applies the quantization matrices. For example, the audio encoder generates a quantization matrix including weighting factors and processes a set of weighting factors according to a parametric model to switch between a direct representation and a parametric representation of the set of weighting factors, where the parametric representation of the set of weighting factors accounts for audibility of distortion according to a model of human auditory perception. In another example, an audio encoder receives a band weight representation of a ciuantization matrix and compresses the band weight representation of the quantization matrix using linear predictive coding, wherein the compressing includes computing pseudo-autocorrelation values for the quantization matrix. For example, a matrix includes weights for bands of a group of frequency coefficients.<H decompression, quantization, Parametric, comp, 7143030.html, Patent, pto, 7143030

Title: Parametric compression/decompression modes for quantization matrices for digital audio

Abstract: Quantization matrices facilitate digital audio encoding and decoding. An audio encoder generates and compresses quantization matrices; an audio decoder decompresses and applies the quantization matrices. For example, the audio encoder generates a quantization matrix including weighting factors and processes a set of weighting factors according to a parametric model to switch between a direct representation and a parametric representation of the set of weighting factors, where the parametric representation of the set of weighting factors accounts for audibility of distortion according to a model of human auditory perception. In another example, an audio encoder receives a band weight representation of a ciuantization matrix and compresses the band weight representation of the quantization matrix using linear predictive coding, wherein the compressing includes computing pseudo-autocorrelation values for the quantization matrix. For example, a matrix includes weights for bands of a group of frequency coefficients.

Patent Number: 7,143,030 Issued on 11/28/2006 to Chen, et al.

Inventors: Chen; Wei-Ge (Issaquah, WA), Lee; Ming-Chieh (Bellevue, WA), Thumpudi; Naveen (Sammamish, WA)
Assignee: Microsoft Corporation (Redmond, WA)

Appl. No.: 11/061,011

Filed: February 17, 2005

Related U.S. Patent Documents


Application Number	Filing Date	Patent Number	Issue Date
10017702	Dec., 2001	6934677

Current U.S. Class: 704/219 ; 704/221

Current International Class: G10L 19/04 (20060101)

References Cited [Referenced By]

U.S. Patent Documents


5079547	January 1992	Fuchigama et al.
5260980	November 1993	Akagiri et al.
5388181	February 1995	Anderson et al.
5524054	June 1996	Spille
5627938	May 1997	Johnston
5629780	May 1997	Watson
5661755	August 1997	Van De Kerkhof et al.
5661823	August 1997	Yamauchi et al.
5682152	October 1997	Wang et al.
5686964	November 1997	Tabatabai et al.
5701346	December 1997	Herre et al.
5812971	September 1998	Herre
5835030	November 1998	Tsutsui et al.
5845243	December 1998	Smart et al.
5956674	September 1999	Smyth et al.
5974380	October 1999	Smyth et al.
5995151	November 1999	Naveen et al.
6029126	February 2000	Malvar
6041295	March 2000	Hinderks
6058362	May 2000	Malvar
6064954	May 2000	Cohen et al.
6115688	September 2000	Brandenburg et al.
6115689	September 2000	Malvar
6182034	January 2001	Malvar
6240380	May 2001	Malvar
6249614	June 2001	Kolesnik et al.
6445739	September 2002	Shen et al.
6658162	December 2003	Zeng et al.
6738074	May 2004	Rao et al.
6766293	July 2004	Herre et al.
6771777	August 2004	Gbur et al.
6934677	August 2005	Chen et al.
2002/0143556	October 2002	Kadatch
2004/0044527	March 2004	Thumpudi et al.

Foreign Patent Documents


0597649	May., 1994	EP
0669724	Aug., 1995	EP
0910927	Apr., 1999	EP
0931386	Jul., 1999	EP
W/O 99/43110	Aug., 1999	WO

Other References

US. Appl. No. 09/771,371, filed Jan. 26, 2001, Kadatch. cited by other .
U.S. Appl. No. 10/017,694, filed Dec. 14, 2001, Chen. cited by other .
U.S. Appl. No. 10/017,861, filed Dec. 14, 2001, Chen. cited by other .
U.S. Appl. No. 10/020,708, filed Dec. 14, 2001, Chen. cited by other .
U.S. Appl. No. 10/016,918, filed Dec. 14, 2001, Chen. cited by other .
Gibson et al., Digital Compression for Multimedia, Title Page, Contents, "Chapter 7: Frequency Domain Coding," Morgan Kaufman Publishers, Inc., pp. iii, v-xi, and 227-262 (1998). cited by other .
H.S. Malvar, Signal Processing with Lapped Transforms, Artech House, Norwood, MA, pp. iv, vii-xi, 175-218, and 353-357 (1992). cited by other .
H.S. Malvar, "Lapped Transforms for Efficient Transform/Subband Coding," IEEE Transactions on Acoustics, Speech and Signal Processing, vol. 38, No. 6, pp. 969-978 (1990). cited by other .
Seymour Schlien, "The Modulated Lapped Transform, Its Time-Varying Forms, and Its Application to Audio Coding Standards," IEEE Transactions on Speech and Audio Processing, vol. 5, No. 4, pp. 359-366 (Jul. 1997). cite- d by other .
de Queiroz et al., "Time-Varying Lapped Transforms and Wavelet Packets," IEEE Transactions on Signal Processing, vol. 41, pp. 3293-3305 (1993). cited by other .
Herley et al., "Tilings of the Time-Frequency Plane: Construction of Arbitrary Orthogonal Bases and Fast Tiling Algorithms," IEEE Transactions on Signal Processing, vol. 41, No. 12, pp. 3341-3359 (1993). cited by oth- er .
ISO/IEC 11172-3, Information Technology--Coding of Moving Pictures and Associated Audio for Digital Storage Media at Up to About 1.5 Mbit/s--Part 3: Audio, 154 pp. (1993). cited by other .
Dolby Laboratories, "AAC Technology," 4 pp. [Downloaded from the web site aac-audio.com on World Wide Web on Nov. 21, 2001.] cited by other .
Srinivasan et al., "High-Quality Audio Compression Using an Adaptive Wavelet Packet Decomposition and Psychoacoustic Modeling," IEEE Transactions on Signal Processing, vol. 46, No. 4, pp. 1085-1093 (Apr. 1998). cited by other .
Caetano et al., "Rate Control Strategy for Embedded Wavelet Video Coders," Electronics Letters, pp. 1815-1817 (Oct. 14, 1999). cited by other .
Ribas Corbera et al., "Rate Control in DCT Video Coding for Low-Delay Communications," IEEE Transactions on Circuits and Systems for Video Technology, vol. 9, No. 1, pp. 172-185 (Feb. 1999). cited by other .
Zwicker et al., Das Ohr als Nachrichtenempfanger, Title Page, Table of Contents, "I: Schallschwingungen," Index, Hirzel-Verlag, Stuttgart, pp. III, IX-XI, 1-26, and 231-232 (1967). cited by other .
Terhardt, "Calculating Virtual Pitch," Hearing Research, 1:155-182 (1979). cited by other .
Lufti, "Additivity of Simultaneous Masking," Journal of Acoustic Society of America, 73:262-267 (1983). cited by other .
Jesteadt et al., "Forward Masking as a Function of Frequency, Masker Level, and Signal Delay," Journal of Acoustical Society of America, 71:950-962 (1982). cited by other .
ITU, Recommendation ITU-R BS 1387, Method for Objective Measurements of Perceived Audio Quality, 89 pp. (1998). cited by other .
ITU, Recommendation ITU-R BS 1115, Low Bit-Rate Audio Coding, 9 pp. (1994). cited by other .
Beerends, "Audio Quality Determination Based on Perceptual Measurement Techniques," Applications of Digital Signal Processing to Audio and Acoustics, Chapter 1, Ed. Mark Kahrs, Karlheinz Brandenburg, Kluwer Acad. Publ., pp. 1-38 (1998). cited by other .
Zwicker, Psychoakustik, Title Page, Table of Contents, "Teil I: Einfuhrung," Index, Springer-Verlag, Berlin Heidelberg, New York, pp. II, IX-XI, 1-30, and 157-162 (1982). cited by other .
Solari, Digital Video and Audio Compression, Title Page, Contents, "Chapter 8: Sound and Audio," McGraw-Hill, Inc., pp. iii, v-vi, and 187-211 (1997). cited by other .
A.M. Kondoz, Digital Speech: Coding for Low Bit Rate Communications Systems, "Chapter 3.3: Linear Predictive Modeling of Speech Signals" and "Chapter 4: LPC Parameter Quantisation Using LSFs," John Wiley & Sons, pp. 42-53 and 79-97 (1994). cited by other .
Wragg et al., "An Optimised Software Solution for an ARM Powered.TM. MP3 Decoder," 9 pp. [Downloaded from the World Wide Web on Oct. 27, 2001.] cited by other .
Fraunhofer-Gesellschaft, "MPEG Audio Layer-3," 4 pp. [Downloaded from the World Wide Web on Oct. 24, 2001.] cited by other .
Fraunhofer-Gesellschaft, "MPEG-2 AAC," 3 pp. [Downloaded from the World Wide Web on Oct. 24, 2001.] cited by other .
OPTICOM GmbH, "Objective Perceptual Measurement," 14 pp. [Downloaded from the World Wide Web on Oct. 24, 2001.] cited by other .
De Luca, "AN1090 Application Note: STA013 MPEG 2.5 Layer III Source Decoder," STMicroelectronics, 17 pp. (1999). cited by other .
Phamdo, "Speech Compression," 13 pp. [Downloaded from the World Wide Web on Nov. 25, 2001.] cited by other .
Malvar, "Biorthogonal and Nonuniform Lapped Transforms for Transform Coding with Reduced Blocking and Ringing Artifacts," appeared in IEEE Transactions on Signal Processing, Special Issue on Multirate Systems, Filter Banks, Wavelets, and Applications, vol. 46, 29 pp. (1998). cited by other .
Advanced Television Systems Committee, ATSC Standard: Digital Audio Compression (AC-3), Revision A, 140 pp. (1995). cited by other .
Brandenburg, "ASPEC Coding", AES 10.sup.th International Conference, pp. 81, 83, 85, 86, and 89, (1991). cited by other .
"ISO/IEC 13818-7, Information Technology--Generic Coding of Moving Pictures and Associated Audio Information--Part 7: Advanced Audio Coding (AAC)," 152 pp. (1997). cited by other .
"ISO/IEC 13818-7, Information Technology--Generic Coding of Moving Pictures and Associated Audio Information--Part 7: Advanced Audio Coding (AAC), Technical Corrigendum 1" 12 pp. (1998). cited by other .
Bosi et al., "ISO/IEC MPEG-2 Advanced Audio Coding," Journal of the Audio Engineering Society, Audio Engineering Society, vol. 45, No. 10, pp. 789-812 (1997). cited by other .
Davis, "The AC-3 Multichannel Coder," Dolby Laboratories, 9 pp. (Downloaded from the World Wide Web on Aug. 15, 2002). cited by other .
Edler et al., "Perceptual Audio Coding Using a Time-Varying Linear Pre- and Post-Filter," in AES 109th Convention, Los Angeles, California, 12 pp. (Sep. 2000). cited by other .
ISO/IEC 13818-7, Information technology--Generic coding of moving pictures and associated audio information--Part 7: Advanced Audio Coding (AAC), 150 pp. (1997). cited by other .
Kuo et al., "A Study of Why Cross Channel Prediction is Not Applicable to Perceptual Audio Coding," IEEE Signal Processing Letters, vol. 8, No. 9, 3 pp. (Sep. 2001). cited by other .
Meares, D.J., "Matrixed Surround Sound in an MPEG Digital World," Journal of the Audio Engineering Society, vol. 46, No. 4, 13 pp. (Apr. 1998). cit- ed by other .
"MPEG2 Audio for DVD: the Compromise Choice," 5 pp. (Oct. 1996). cited by other .
Search Report for European Patent Application No. 03 020 110.7. cited by other .
Search Report for European Patent Application No. 03 020 111.5. cited by other .
Stuart et al., "Lossless Compression for DVD-Audio," in AES 9th Regional Convention Tokyo, 4 pp. (1999). cited by other .
Vaidyanathan, Multirate Systems and Filter Banks, Prentice Hall Signal Processing Series, Cover page, pp. 745-751 (1992). cited by other .
Van Assche et al., "Lossless Compression of Pre-Press Image Using a Novel Color Decorrelation Technique," Proc. SPIE, Very High Resolution and Quality III. vol. 3308, 8 pp. (1998). cited by other .
Wang et al., "A Multichannel Audio Coding Algorithm for Inter-Channel Redundancy Removal," in AES 110th Convention, Amsterdam, the Netherlands, 6 pp. (May 2001). cited by other .
Wang et al., "EE225a Lecture 13: Karhunen Loeve Transform and Discrete Cosine Transform," Department of EECS, University of California at Berkley, 10 pp. (Mar. 2002). cited by other .
Yang et al., "Adaptive Karhunen-Loeve Transform for Enhanced Multichannel Audio Coding," Proc. SPIE vol. 4475, 13 pp., Mathematics of Data/Image Coding, Compression, and Encryption IV San Diego, CA. (Jul. 29-Aug. 3, 2001). cited by other .
Yang et al., "An Inter-Channel Redundancy Removal Approach for High-Quality Multichannel Audio Compression," in AES 109th Convention, Los Angeles, California, 8 pp. (Sep. 2000). cited by other.

Primary Examiner: Storm; Donald L.
Attorney, Agent or Firm: Klarquist Sparkman, LLP

Parent Case Text

RELATED APPLICATION INFORMATION

The present application is a divisional of U.S. patent application Ser. No. 10/017,702, filed Dec. 14, 2001, now U.S. Pat. No. 6,934,677 entitled "Quantization Matrices For Digital Audio," the disclosure of which is hereby incorporated by reference. The following concurrently filed U.S. patent applications relate to the present application: 1) U.S. patent application Ser. No. 10/020,708, entitled, "Adaptive Window-Size Selection in Transform Coding," filed Dec. 14, 2001, the disclosure of which is hereby incorporated by reference; 2) U.S. patent application Ser. No. 10/016,918, entitled, "Quality Improvement Techniques in an Audio Encoder," filed Dec. 14, 2001, the disclosure of which is hereby incorporated by reference; 3) U.S. patent application Ser. No. 10/017,694, entitled, "Quality and Rate Control Strategy for Digital Audio," filed Dec. 14, 2001, the disclosure of which is hereby incorporated by reference; and 4) U.S. patent application Ser. No. 10/017,861, entitled, "Techniques for Measurement of Perceptual Audio Quality," filed Dec. 14, 2001, the disclosure of which is hereby incorporated by reference.

Claims

We claim:

1. A computer-readable medium encoded with computer-executable instructions for causing a computer programmed thereby to perform a method comprising: processing at least one set of weighting factors according to a parametric model to switch between a direct representation and a parametric representation of the at least one set of weighting factors, wherein the parametric representation of the at least one set of weighting factors accounts for audibility of distortion according to a model of human auditory perception; and outputting a result of the processing.

2. The computer-readable medium of claim 1 wherein the processing comprises compression, and wherein the result is the parametric representation.

3. The computer-readable medium of claim 1 wherein the processing comprises decompression, and wherein the result is the direct representation.

4. The computer-readable medium of claim 1 wherein the parametric model uses linear predictive coding for the at least one set of weighting factors.

5. The computer-readable medium of claim 4 wherein the at least one set of weighting factors is for a block of audio data, and wherein pseudo-autocorrelation values used in the processing differ from autocorrelation values for the block due at least in part to processing of the block according to an auditory model.

6. The computer-readable medium of claim 4 wherein pseudo-autocorrelation values used in the processing differ from autocorrelation values for blocks of audio data due at least in part to joint channel coding of the blocks.

7. The computer-readable medium of claim 1 further comprising: processing another set of weighting factors according a direct mode.

8. The computer-readable medium of claim 7 wherein the processing the other set of weighting factors comprises compressing the other set of weighting factors.

9. The computer-readable medium of claim 7 wherein the processing the other set of weighting factors comprises decompressing the other set of weighting factors.

10. In an audio encoder, a method comprising: receiving a band weight representation of a quantization matrix; and compressing the band weight representation of the quantization matrix using linear predictive coding, wherein the compressing includes computing pseudo-autocorrelation values for the quantization matrix.

11. The method of claim 10 wherein the computing pseudo-autocorrelation values includes converting the band weight representation into an intermediate representation, and wherein the converting comprises: for each of plural bands in the band weight representation, repeating a weight by an expansion factor in the intermediate representation, wherein the expansion factor relates to size of the band.

12. The method of claim 11 wherein the converting further comprises: mirroring the intermediate representation.

13. The method of claim 12 wherein the converting further comprises: inverse frequency transforming the mirrored intermediate representation, thereby producing the pseudo-autocorrelation values for the quantization matrix.

14. The method of claim 10 wherein the computing pseudo-autocorrelation values comprises: inverse frequency transforming an intermediate representation based upon the band weight representation.

15. The method of claim 10 wherein the compressing further comprises: computing linear predictive coding parameters based upon the pseudo-autocorrelation values.

16. The method of claim 10 further comprising: receiving a second band weight representation of a second quantization matrix; and compressing the second band weight representation using a direct compression mode.

17. The method of claim 10 further comprising: receiving a second band weight representation of a second quantization matrix; and compressing the second band weight representation using either a direct compression mode or the linear predictive coding.

18. A computer-readable medium encoded with computer-executable instructions for causing a computer programmed thereby to perform a method comprising: receiving a parametric representation of a quantization matrix, the quantization matrix including weights for bands of a group of frequency coefficients, wherein the parametric representation accounts for audibility of distortion according to a model of human auditory perception; and decompressing the parametric representation of the quantization matrix, thereby producing a direct representation of the quantization matrix.

19. The computer-readable medium of claim 18 wherein the parametric representation is based at least in part upon linear predictive coding of pseudo-autocorrelation values for the quantization matrix.

20. The computer-readable medium of claim 18 wherein the method further comprises receiving and decompressing a compressed direct representation of a second quantization matrix.

Description

TECHNICAL FIELD

The present invention relates to quantization matrices for audio encoding and decoding. In one embodiment, an audio encoder generates and compresses quantization matrices, and an audio decoder decompresses and applies the quantization matrices.

BACKGROUND

With the introduction of compact disks, digital wireless telephone networks, and audio delivery over the Internet, digital audio has become commonplace. Engineers use a variety of techniques to process digital audio efficiently while still maintaining the quality of the digital audio. To understand these techniques, it helps to understand how audio information is represented in a computer and how humans perceive audio.

I. Representation of Audio Information in a Computer

A computer processes audio information as a series of numbers representing the audio information. For example, a single number can represent an audio sample, which is an amplitude value (i.e., loudness) at a particular time. Several factors affect the quality of the audio information, including sample depth, sampling rate, and channel mode.

Sample depth (or precision) indicates the range of numbers used to represent a sample. The more values possible for the sample, the higher the quality because the number can capture more subtle variations in amplitude. For example, an 8-bit sample has 256 possible values, while a 16-bit sample has 65,536 possible values.

The sampling rate (usually measured as the number of samples per second) also affects quality. The higher the sampling rate, the higher the quality because more frequencies of sound can be represented. Some common sampling rates are 8,000, 11,025, 22,050, 32,000,44,100, 48,000, and 96,000 samples/second.

Mono and stereo are two common channel modes for audio. In mono mode, audio information is present in one channel. In stereo mode, audio information is present in two channels usually labeled the left and right channels. Other modes with more channels, such as 5-channel surround sound, are also possible. Table 1 shows several formats of audio with different quality levels, along with corresponding raw bitrate costs.

TABLE-US-00001 TABLE 1 Bitrates for different quality audio information Sample Depth (bits/ Sampling Rate Raw Bitrate Quality sample) (samples/second) Mode (bits/second) Internet telephony 8 8,000 mono 64,000 Telephone 8 11,025 mono 88,200 CD audio 16 44,100 stereo 1,411,200 high quality audio 16 48,000 stereo 1,536,000

As Table 1 shows, the cost of high quality audio information such as CD audio is high bitrate. High quality audio information consumes large amounts of computer storage and transmission capacity.

Compression (also called encoding or coding) decreases the cost of storing and transmitting audio information by converting the information into a lower bitrate form. Compression can be lossless (in which quality does not suffer) or lossy (in which quality suffers). Decompression (also called decoding) extracts a reconstructed version of the original information from the compressed form.

Quantization is a conventional lossy compression technique. There are many different kinds of quantization including uniform and non-uniform quantization, scalar and vector quantization, and adaptive and non-adaptive quantization. Quantization maps ranges of input values to single values. For example, with uniform, scalar quantization by a factor of 3.0, a sample with a value anywhere between -1.5 and 1.499 is mapped to 0, a sample with a value anywhere between 1.5 and 4.499 is mapped to 1, etc. To reconstruct the sample, the quantized value is multiplied by the quantization factor, but the reconstruction is imprecise. Continuing the example started above, the quantized value 1 reconstructs to 1.times.3=3; it is impossible to determine where the original sample value was in the range 1.5 to 4.499. Quantization causes a loss in fidelity of the reconstructed value compared to the original value. Quantization can dramatically improves the effectiveness of subsequent lossless compression, however, thereby reducing bitrate.

An audio encoder can use various techniques to provide the best possible quality for a given bitrate, including transform coding, rate control, and modeling human perception of audio. As a result of these techniques, an audio signal can be more heavily quantized at selected frequencies or times to decrease bitrate, yet the increased quantization will not significantly degrade perceived quality for a listener.

Transform coding techniques convert data into a form that makes it easier to separate perceptually important information from perceptually unimportant information. The less important information can then be quantized heavily, while the more important information is preserved, so as to provide the best perceived quality for a given bitrate. Transform coding techniques typically convert data into the frequency (or spectral) domain. For example, a transform coder converts a time series of audio samples into frequency coefficients. Transform coding techniques include Discrete Cosine Transform ["DCT"], Modulated Lapped Transform ["MLT"], and Fast Fourier Transform ["FFT"]. In practice, the input to a transform coder is partitioned into blocks, and each block is transform coded. Blocks may have varying or fixed sizes, and may or may not overlap with an adjacent block. For more information about transform coding and MLT in particular, see Gibson et al., Digital Compression for Multimedia, "Chapter 7: Frequency Domain Coding," Morgan Kaufman Publishers, Inc., pp. 227 262 (1998); U.S. Pat. No. 6,115,689 to Malvar; H. S. Malvar, Signal Processing with Lapped Transforms, Artech House, Norwood, Mass., 1992; or Seymour Schlein, "The Modulated Lapped Transform, Its Time-Varying Forms, and Its Application to Audio Coding Standards," IEEE Transactions on Speech and Audio Processing, Vol. 5, No. 4, pp. 359 66, July 1997.

With rate control, an encoder adjusts quantization to regulate bitrate. For audio information at a constant quality, complex information typically has a higher bitrate (is less compressible) than simple information. So, if the complexity of audio information changes in a signal, the bitrate may change. In addition, changes in transmission capacity (such as those due to Internet traffic) affect available bitrate in some applications. The encoder can decrease bitrate by increasing quantization, and vice versa. Because the relation between degree of quantization and bitrate is complex and hard to predict in advance, the encoder can try different degrees of quantization to get the best quality possible for some bitrate, which is an example of a quantization loop.

II. Human Perception of Audio Information

In addition to the factors that determine objective audio quality, perceived audio quality also depends on how the human body processes audio information. For this reason, audio processing tools often process audio information according to an auditory model of human perception.

Typically, an auditory model considers the range of human hearing and critical bands. Humans can hear sounds ranging from roughly 20 Hz to 20 kHz, and are most sensitive to sounds in the 2 4 kHz range. The human nervous system integrates sub-ranges of frequencies. For this reason, an auditory model may organize and process audio information by critical bands. For example, one critical band scale groups frequencies into 24 critical bands with upper cut-off frequencies (in Hz) at 100, 200, 300, 400, 510, 630, 770, 920, 1080, 1270, 1480, 1720, 2000, 2320, 2700, 3150, 3700, 4400, 5300, 6400, 7700, 9500, 12000, and 15500. Different auditory models use a different number of critical bands (e.g., 25, 32, 55, or 109) and/or different cut-off frequencies for the critical bands. Bark bands are a well-known example of critical bands.

Aside from range and critical bands, interactions between audio signals can dramatically affect perception. An audio signal that is clearly audible if presented alone can be completely inaudible in the presence of another audio signal, called the masker or the masking signal. The human ear is relatively insensitive to distortion or other loss in fidelity (i.e., noise) in the masked signal, so the masked signal can include more distortion without degrading perceived audio quality. Table 2 lists various factors and how the factors relate to perception of an audio signal.

TABLE-US-00002 TABLE 2 Various factors that relate to perception of audio Factor Relation to Perception of an Audio Signal outer and Generally, the outer and middle ear attenuate higher middle frequency information and pass middle frequency ear transfer information. Noise is less audible in higher frequencies than middle frequencies. noise in the Noise present in the auditory nerve, together with auditory noise from the flow of blood, increases for low nerve frequency information. Noise is less audible in lower frequencies than middle frequencies. perceptual Depending on the frequency of the audio signal, hair frequency cells at different positions in the inner ear react, which scales affects the pitch that a human perceives. Critical bands relate frequency to pitch. excitation Hair cells typically respond several milliseconds after the onset of the audio signal at a frequency. After exposure, hair cells and neural processes need time to recover full sensitivity. Moreover, loud signals are processed faster than quiet signals. Noise can be masked when the ear will not sense it. detection Humans are better at detecting changes in loudness for quieter signals than louder signals. Noise can be masked in louder signals. simultaneous For a masker and maskee present at the same time, the masking maskee is masked at the frequency of the masker but also at frequencies above and below the masker. The amount of masking depends on the masker and maskee structures and the masker frequency. temporal The masker has a masking effect before and after than masking the masker itself. Generally, forward masking is more pronounced than backward masking. The masking effect diminishes further away from the masker in time. loudness Perceived loudness of a signal depends on frequency, duration, and sound pressure level. The components of a signal partially mask each other, and noise can be masked as a result. cognitive Cognitive effects influence perceptual audio quality. processing Abrupt changes in quality are objectionable. Different componentcs of an audio signal are important in different applications (e.g., speech vs. music).

An auditory model can consider any of the factors shown in Table 2 as well as other factors relating to physical or neural aspects of human perception of sound. For more information about auditory models, see: 1) Zwicker and Feldtkeller, "Das Ohr als Nachrichtenempfanger," Hirzel-Verlag, Stuttgart, 1967; 2) Terhardt, "Calculating Virtual Pitch," Hearing Research, 1:155 182, 1979; 3) Lufti, "Additivity of Simultaneous Masking," Journal of Acoustic Society of America, 73:262 267, 1983; 4) Jesteadt et al., "Forward Masking as a Function of Frequency, Masker Level, and Signal Delay," Journal of Acoustical Society of America, 71:950 962, 1982; 5) ITU, Recommendation ITU-R BS 1387, Method for Objective Measurements of Perceived Audio Quality, 1998; 6) Beerends, "Audio Quality Determination Based on Perceptual Measurement Techniques," Applications of Digital Signal Processing to Audio and Acoustics, Chapter 1, Ed. Mark Kahrs, Karlheinz Brandenburg, Kluwer Acad. Publ., 1998; and 7) Zwicker, Psychoakustik, Springer-Verlag, Berlin Heidelberg, New York, 1982. III. Generating Quantization Matrices

Quantization and other lossy compression techniques introduce potentially audible noise into an audio signal. The audibility of the noise depends on 1) how much noise there is and 2) how much of the noise the listener perceives. The first factor relates mainly to objective quality, while the second factor depends on human perception of sound.

Distortion is one measure of how much noise is in reconstructed audio. Distortion D can be calculated as the square of the differences between original values and reconstructed values: D=(u-q(u)Q).sup.2 (1), where u is an original value, q(u) is a quantized value, and Q is a quantization factor. The distribution of noise in the reconstructed audio depends on the quantization scheme used in the encoder.

For example, if an audio encoder uses uniform, scalar quantization for each frequency coefficient of spectral audio data, noise is spread equally across the frequency spectrum of the reconstructed audio, and different levels are quantized at the same accuracy. Uniform, scalar quantization is relatively simple computationally, but can result in the complete loss of small values at moderate levels of quantization. Uniform, scalar quantization also fails to account for the varying sensitivity of the human ear to noise at different frequencies and levels of loudness, interaction with other sounds present in the signal (i.e., masking), or the physical limitations of the human ear (i.e., the need to recover sensitivity).

Power-law quantization (e.g., .alpha.-law) is a non-uniform quantization technique that varies quantization step size as a function of amplitude. Low levels are quantized with greater accuracy than high levels, which tends to preserve low levels along with high levels. Power-law quantization still fails to fully account for the audibility of noise, however.

Another non-uniform quantization technique uses quantization matrices. A quantization matrix is a set of weighting factors for series of values called quantization bands. Each value within a quantization band is weighted by the same weighting factor. A quantization matrix spreads distortion in unequal proportions, depending on the weighting factors. For example, if quantization bands are frequency ranges of frequency coefficients, a quantization matrix can spread distortion across the spectrum of reconstructed audio data in unequal proportions. Some parts of the spectrum can have more severe quantization and hence more distortion; other parts can have less quantization and hence less distortion.

Microsoft Corporation's Windows Media Audio version 7.0 ["WMA7"] generates quantization matrices for blocks of frequency coefficient data. In WMA7, an audio encoder uses a MLT to transform audio samples into frequency coefficients in variable-size transform blocks. For stereo mode audio data, the encoder can code left and right channels into sum and difference channels. The sum channel is the averages of the left and right channels; the difference channel is the differences between the left and right channels divided by two. The encoder computes a quantization matrix for each channel: Q[c][d]=E[d] (2), where c is a channel, d is a quantization band, and E[d] is an excitation pattern for the quantization band d. The WMA7 encoder calculates an excitation pattern for a quantization band by squaring coefficient values to determine energies and then summing the energies of the coefficients within the quantization band.

Since the quantization bands can have different sizes, the encoder adjusts the quantization matrix Q[c][d] by the quantization band sizes:

.function..function..rarw..function..function..times..function. ##EQU00001## where Card{B[d]} is the number of coefficients in the quantization band d, and where u is an experimentally derived exponent (in listening tests) that affects relative weights of bands of different energies. For stereo mode audio data, whether the data is in independently (i.e., left and right) or jointly (i.e., sum and difference) coded channels, the WMA7 encoder uses the same technique to generate quantization matrices for two individual coded channels.

The quantization matrices in WMA7 spread distortion between bands in proportion to the energies of the bands. Higher energy leads to a higher weight and more quantization; lower energy leads to a lower weight and less quantization. WMA7 still fails to account for the audibility of noise in several respects, however, including the varying sensitivity of the human ear to noise at different frequencies and times, temporal masking, and the physical limitations of the human ear.

In order to reconstruct audio data, a WMA7 decoder needs the quantization matrices used to compress the audio data. For this reason, the WMA7 encoder sends the quantization matrices to the decoder as side information in the bitstream of compressed output. To reduce bitrate, the encoder compresses the quantization matrices using a technique such as the direct compression technique (100) shown in FIG. 1.

In the direct compression technique (100), the encoder uniformly quantizes (110) each element of a quantization matrix (105). The encoder then differentially codes (120) the quantized elements, and Huffman codes (130) the differentially coded elements. The technique (100) is computationally simple and effective, but the resulting bitrate for the quantization matrix is not low enough for very low bitrate coding.

Aside from WMA7, several international standards describe audio encoders that spread distortion in unequal proportions across bands. The Motion Picture Experts Group, Audio Layer 3 ["MP3"] and Motion Picture Experts Group 2, Advanced Audio Coding ["AAC"] standards each describe scale factors used when quantizing spectral audio data.

In MP3, the scale factors are weights for ranges of frequency coefficients called scale factor bands. Each scale factor starts with a minimum weight for a scale factor band. The number of scale factor bands depends on sampling rate and block size (e.g., 21 scale factor bands for a long block of 48 kHz input). For the starting set of scale factors, the encoder finds a satisfactory quantization step size in an inner quantization loop. In an outer quantization loop, the encoder amplifies the scale factors until the distortion in each scale factor band is less than the allowed distortion threshold for that scale factor band, with the encoder repeating the inner quantization loop for each adjusted set of scale factors. In special cases, the encoder exits the outer quantization loop even if distortion exceeds the allowed distortion threshold for a scale factor band (e.g., if all scale factors have been amplified or if a scale factor has reached a maximum amplification). The MP3 encoder transmits the scale factors as side information using ad hoc differential coding and, potentially, entropy coding.

Before the quantization loops, the MP3 encoder can switch between long blocks of 576 frequency coefficients and short blocks of 192 frequency coefficients (sometimes called long windows or short windows). Instead of a long block, the encoder can use three short blocks for better time resolution. The number of scale factor bands is different for short blocks and long blocks (e.g., 12 scale factor bands vs. 21 scale factor bands).

The MP3 encoder can use any of several different coding channel modes, including single channel, two independent channels (left and right channels), or two jointly coded channels (sum and difference channels). If the encoder uses jointly coded channels, the encoder computes and transmits a set of scale factors for each of the sum and difference channels using the same techniques that are used for left and right channels. Or, if the encoder uses jointly coded channels, the encoder can instead use intensity stereo coding. Intensity stereo coding changes how scale factors are determined for higher frequency scale factor bands and changes how sum and difference channels are reconstructed, but the encoder still computes and transmits two sets of scale factors for the two channels.

The MP3 encoder incorporates a psychoacoustic model when determining the allowed distortion thresholds for scale factor bands. In a path separate from the rest of the encoder, the encoder processes the original audio data according to the psychoacoustic model. The psychoacoustic model uses a different frequency transform than the rest of the encoder (FFT vs. hybrid polyphase/MDCT filter bank) and uses separate computations for energy and other parameters; In the psychoacoustic model, the MP3 encoder processes the blocks of frequency coefficients according to threshold calculation partitions at sub-Bark band resolution (e.g., 62 partitions for a long block of 48 kHz input). The encoder calculates a Signal to Mask Ratio ["SMR"] for each partition, and then converts the SMRs for the partitions into SMRs for the scale factor bands. The MP3 encoder later converts the SMRs for scale factor bands into the allowed distortion thresholds for the scale factor bands.

The encoder runs the psychoacoustic model twice (in parallel, once for long blocks and once for short blocks) using different techniques to calculate SMR depending on the block size.

For additional information about MP3 and MC, see the MP3 standard ("ISO/IEC 11172-3, Information Technology--Coding of Moving Pictures and Associated Audio for Digital Storage Media at Up to About 1.5 Mbit/s--Part 3: Audio") and the AAC standard.

Although MP3 encoding has achieved widespread adoption, it is unsuitable for some applications (for example, real-time audio streaming at very low to mid bitrates) for several reasons. First, MP3's iterative refinement of scale factors in the outer quantization loop consumes too many resources for some applications. Repeated iterations of the outer quantization loop consume time and computational resources. On the other hand, if the outer quantization loop exits quickly (i.e., with minimum scale factors and a small quantization step size), the MP3 encoder can waste bitrate encoding audio information with distortion well below the allowed distortion thresholds. Second, computing SMR with a psychoacoustic model separate from the rest of the MP3 encoder (e.g., separate frequency transform, computations of energy, etc.) consumes too much time and computational resources for some applications. Third, computing SMRs in parallel for long blocks as well as short blocks consumes more resources than is necessary when the encoder switches between long blocks or short blocks in the alternative. Computing SMRs in separate tracks also does not allow direct comparisons between blocks of different sizes for operations like temporal spreading. Fourth, the MP3 encoder does not adequately exploit differences between independently coded channels and jointly coded channels when computing and transmitting quantization matrices. Fifth, ad hoc differential coding and entropy coding of scale factors in MP3 gives good quality for the scale factors, but the bitrate for the scale factors is not low enough for very low bitrate applications.

IV. Parametric Coding of Audio Information

Parametric coding is an alternative to transform coding, quantization, and lossless compression in applications such as speech compression. With parametric coding, an encoder converts a block of audio samples into a set of parameters describing the block (rather than coded versions of the audio samples themselves). A decoder later synthesizes the block of audio samples from the set of parameters. Both the bitrate and the quality for parametric coding are typically lower than other compression methods.

One technique for parametrically compressing a block of audio samples uses Linear Predictive Coding ["LPC"] parameters and Line-Spectral Frequency ["LSF"] values. First, the audio encoder computes the LPC parameters. For example, the audio encoder computes autocorrelation values for the block of audio samples itself, which are short-term correlations between samples within the block. From the autocorrelation values, the encoder computes the LPC parameters using a technique such as Levinson recursion. Other techniques for determining LPC parameters use a covariance method or a lattice method.

Next, the encoder converts the LPC parameters to LSF values, which capture spectral information for the block of audio samples. LSF values have greater intra-block and inter-block correlation than LPC parameters, and are better suited for subsequent quantization. For example, the encoder computes partial correlation ["PARCOR"] or reflection coefficients from the LPC parameters. The encoder then computes the LSF values from the PARCOR coefficients using a method such as complex root, real root, ratio filter, Chebyshev, or adaptive sequential LMS. Finally, the encoder quantizes the LSF values. Instead of LSF values, different techniques convert LPC parameters to a log area ratio, inverse sine, or other representation. For more information about parametric coding, LPC parameters, and LSF values, see A. M. Kondoz, Digital Speech: Coding for Low Bit Rate Communications Systems, "Chapter 3.3: Linear Predictive Modeling of Speech Signals" and "Chapter 4: LPC Parameter Quantisation Using LSFs," John Wiley & Sons (1994).

WMA7 allows a parametric coding mode in which the audio encoder parametrically codes the spectral shape of a block of audio samples. The resulting parameters represent the quantization matrix for the block, rather than the more conventional application of representing the audio signal itself. The parameters used in WMA7 represent spectral shape of the audio block, but do not adequately account for human perception of audio information.

SUMMARY

The present invention relates to quantization matrices for audio encoding and decoding. The present invention includes various techniques and tools relating to quantization matrices, which can be used in combination or independently.

First, an audio encoder generates quantization matrices based upon critical band patterns for blocks of audio data. The encoder computes the critical band patterns using an auditory model, so the quantization matrices account for the audibility of noise in quantization of the audio data. The encoder computes the quantization matrices directly from the critical band patterns, which reduces computational overhead in the encoder and limits bitrate spent coding perceptually unimportant information.

Second, an audio encoder generates quantization matrices from critical band patterns computed using an auditory model, processing the same frequency coefficients in the auditory model that the encoder compresses. This reduces computational overhead in the encoder.

Third, blocks of data having variable size are normalized before generating quantization matrices for the blocks. The normalization improves auditory modeling by enabling temporal smearing.

Fourth, an audio encoder uses different modes for generating quantization matrices depending on the coding channel mode for multi-channel audio data, and an audio decoder can use different modes when applying the quantization matrices. For example, for stereo mode audio data in jointly coded channels, the encoder generates an identical quantization matrix for sum and difference channels, which can reduce the bitrate associated with quantization matrices for the sum and difference channels and simplify generation of quantization matrices.

Fifth, an audio encoder uses different modes for compressing quantization matrices, including a parametric compression mode. An audio decoder uses different modes for decompressing quantization matrices, including a parametric compression mode. The parametric compression mode lowers bitrate for quantization matrices enough for very low bitrate applications while also accounting for human perception of audio information.

Additional features and advantages of the invention will be made apparent from the following detailed description of an illustrative embodiment that proceeds with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing direct compression of a quantization matrix according to the prior art.

FIG. 2 is a block diagram of a suitable computing environment in which the illustrative embodiment may be implemented.

FIG. 3 is a block diagram of a generalized audio encoder according to the illustrative embodiment.

FIG. 4 is a block diagram of a generalized audio decoder according to the illustrative embodiment.

FIG. 5 is a chart showing a mapping of quantization bands to critical bands according to the illustrative embodiment.

FIG. 6 is a flowchart showing a technique for generating a quantization matrix according to the illustrative embodiment.

FIGS. 7a 7c are diagrams showing generation of a quantization matrix from an excitation pattern in an audio encoder according to the illustrative embodiment.

FIG. 8 is a graph of an outer/middle ear transfer function according to the illustrative embodiment.

FIG. 9 is a flowchart showing a technique for generating quantization matrices in a coding channel mode-dependent manner according to the illustrative embodiment.

FIGS. 10a 10b are flowcharts showing techniques for parametric compression of a quantization matrix according to the illustrative embodiment.

FIGS. 11a 11b are graphs showing an intermediate array used in the creation of pseudo-autocorrelation values from a quantization matrix according to the illustrative embodiment.

DETAILED DESCRIPTION

The illustrative embodiment of the present invention is directed to generation/application and compression/decompression of quantization matrices for audio encoding/decoding.

An audio encoder balances efficiency and quality when generating quantization matrices. The audio encoder computes quantization matrices directly from excitation patterns for blocks of frequency coefficients, which makes the computation efficient and controls bitrate. At the same time, to generate the excitation patterns, the audio encoder processes the blocks of frequency coefficients by critical bands according to an auditory model, so the quantization matrices account for the audibility of noise.

For audio data in jointly coded channels, the audio encoder directly controls distortion and reduces computations when generating quantization matrices, and can reduce the bitrate associated with quantization matrices at little or no cost to quality. The audio encoder computes a single quantization matrix for sum and difference channels of jointly coded stereo data from aggregated excitation patterns for the individual channels. In some implementations, the encoder halves the bitrate associated with quantization matrices for audio data in jointly coded channels. An audio decoder switches techniques for applying quantization matrices to multi-channel audio data depending on whether the channels are jointly coded.

The audio encoder compresses quantization matrices using direct compression or indirect, parametric compression. The indirect, parametric compression results in very low bitrate for the quantization matrices, but also reduces quality. Similarly, the decoder decompresses the quantization matrices using direct decompression or indirect, parametric decompression.

According to the illustrative embodiment, the audio encoder uses several techniques in the generation and compression of quantization matrices. The audio decoder uses several techniques in the decompression and application of quantization matrices. While the techniques are typically described herein as part of a single, integrated system, the techniques can be applied separately, potentially in combination with other techniques. In alternative embodiments, an audio processing tool other than an encoder or decoder implements one or more of the techniques.

I. Computing Environment

FIG. 2 illustrates a generalized example of a suitable computing environment (200) in which the illustrative embodiment may be implemented. The computing environment (200) is not intended to suggest any limitation as to scope of use or functionality of the invention, as the present invention may be implemented in diverse general-purpose or special-purpose computing environments.

With reference to FIG. 2, the computing environment (200) includes at least one processing unit (210) and memory (220). In FIG. 2, this most basic configuration (230) is included within a dashed line. The processing unit (210) executes computer-executable instructions and may be a real or a virtual processor. In a multi-processing system, multiple processing units execute computer-executable instructions to increase processing power. The memory (220) may be volatile memory (e.g., registers, cache, RAM), non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.), or some combination of the two. The memory (220) stores software (280) implementing an audio encoder that generates and compresses quantization matrices.

A computing environment may have additional features. For example, the computing environment (200) includes storage (240), one or more input devices (250), one or more output devices (260), and one or more communication connections (270). An interconnection mechanism (not shown) such as a bus, controller, or network interconnects the components of the computing environment (200). Typically, operating system software (not shown) provides an operating environment for other software executing in the computing environment (200), and coordinates activities of the components of the computing environment (200).

The storage (240) may be removable or non-removable, and includes magnetic disks, magnetic tapes or cassettes, CD-ROMs, CD-RWs, DVDs, or any other medium which can be used to store information and which can be accessed within the computing environment (200). The storage (240) stores instructions for the software (280) implementing the audio encoder that that generates and compresses quantization matrices.

The input device(s) (250) may be a touch input device such as a keyboard, mouse, pen, or trackball, a voice input device, a scanning device, or another device that provides input to the computing environment (200). For audio, the input device(s) (250) may be a sound card or similar device that accepts audio input in analog or digital form, or a CD-ROM reader that provides audio samples to the computing environment. The output device(s) (260) may be a display, printer, speaker, CD-writer, or another device that provides output from the computing environment (200).

The communication connection(s) (270) enable communication over a communication medium to another computing entity. The communication medium conveys information such as computer-executable instructions, compressed audio or video information, or other data in a modulated data signal. A modulated data signal is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media include wired or wireless techniques implemented with an electrical, optical, RF, infrared, acoustic, or other carrier.

The invention can be described in the general context of computer-readable media. Computer-readable media are any available media that can be accessed within a computing environment. By way of example, and not limitation, with the computing en