Encoding apparatus and method, and multiplexing apparatus and method Number:7,394,851 from the United States Patent and Trademark Office (PTO) owispatent

Title: Encoding apparatus and method, and multiplexing apparatus and method

Abstract: An encoder to encode video streams representing a moving picture is provided whose video encoder calculates a bit storage quantity an STD video buffer should have when input of a stream ending with a first picture to the STD video buffer ends, calculates an initial bit storage quantity of a second picture of a stream concatenated to the end of the first picture and starting with the second picture, in a VBV buffer, based on the calculated bit storage quantity of the STD buffer, and encodes the stream based on the bit storage quantity. The video encoder further calculates a bit storage quantity of the STD video buffer, for decoding a picture of the stream in the STD video buffer, calculates, based on the calculated bit storage quantity, a bit storage quantity the VBV buffer should have for ending decoding of a fourth picture of a second stream concatenated to the beginning of a third picture and ending with the fourth picture, and encodes the stream based on the calculated bit storage quantity. With such encoding, a part of video streams can be re-coded without referencing to information indicative of an initial status of a video buffer when editing the video streams.

Patent Number: 7,394,851 Issued on 07/01/2008 to Kato,   et al.

---

Inventors:	Kato; Motoki (Kanagawa, JP), Obata; Koji (Tokyo, JP), Igi; Nobuhiro (Kanagawa, JP), Takashima; Yoshikazu (Tokyo, JP)
Assignee:	Sony Corporation (Tokyo, JP)
Appl. No.:	10/861,748
Filed:	June 4, 2004

---

Related U.S. Patent Documents

---

	Application Number	Filing Date	Patent Number	Issue Date
	09583422	May., 2000	6795499

---

Foreign Application Priority Data

---

Jun 01, 1999 [JP]			P11-154402
Mar 27, 2000 [JP]			P2000-090769

Current U.S. Class:	375/240.01 ; 382/239
Current International Class:	H04N 7/12 (20060101); G06K 9/36 (20060101)
Field of Search:	375/240.01,240.16,240.02,240.26 382/239

---

References Cited [Referenced By]

---

U.S. Patent Documents

6219043	April 2001	Yogeshwar et al.
6301428	October 2001	Linzer
6396874	May 2002	Kato
6483945	November 2002	Kato
6560282	May 2003	Tahara et al.
6567471	May 2003	Yoshinari
6795499	September 2004	Kato et al.
7236526	June 2007	Kitamura

Foreign Patent Documents

	0 935 395		Aug., 1999		EP
	WO 00 05864		Feb., 1999		WO

Other References

Wee S J ET AL: "Splicing MPEG Video Streams in the Compressed Domain" IEEE Workshop on Multimedia Signal Processing. Proceedings of Signal Processing Society Workshop on Multimedia Signal Processing, XX, XX Jun. 23, 1997, pp. 225-230, XP000957700. cited by other . Hurst N ET AL: "MPEG Splicing: A New Standard for Television--SMPTE312M" SMPTE Journal, SMPTE Inc. Scarsdale, N.Y., US, vol. 107, No. 11, Nov. 1998, pp. 978-988, XP000804761 ISSN: 0036-1682. cited by other.

Primary Examiner: Philippe; Gims
Attorney, Agent or Firm: Oblon, Spivak, McClelland, Maier & Neustadt, P.C.

---

Parent Case Text

---

This application is a continuation of U.S. application Ser. No. 09/583,422, filed May 31, 2000 now U.S. Pat. No. 6,795,499.

---

Claims

---

What is claimed is:

1. A computer program recorded on a computer readable recordable medium for performing an encoding process, comprising the steps of: calculating, for a video buffer in a virtual system target decoder model for decoding re-encoded encoded streams, a first bit storage quantity which should be when input of a first picture of a first encoded stream ending with the first picture to the video buffer ends; calculating, based on the first bit storage quantity, a second bit storage quantity the video buffer in the virtual system target decoder should have for starting, according to the virtual system target model, decoding of a second picture of a second encoded stream concatenated to the end of the first picture and beginning with the second picture; re-encoding the second encoded stream based on the second bit storage quantity; and generating, based on the first bit storage quantity, a multiplexed stream including the re-encoded second encoded stream.

2. A computer program recorded on a computer reader recordable medium for performing an encoding process, comprising the steps of: calculating, for a video buffer in a virtual system target decoder model for decoding re-encoded encoded streams, a first bit storage quantity which should be when input of a first picture of a first encoded stream ending with the first picture to the video buffer ends; calculating, based on the first bit storage quantity, a second bit storage quantity the video buffer in the virtual system target decoder should have for starting, according to the virtual system target model, decoding of a second picture of a second encoded stream concatenated to the end of the first picture and beginning with the second picture; re-encoding the second encoded stream based on the second bit storage quantity; and changing a number of pictures included in the second encoded stream based on the second bit storage quantity to re-encode the second encoded stream whose number of pictures is changed.

3. A computer program recorded on a computer readable recordable medium for performing an encoding process, comprising the steps of: calculating a first bit storage quantity a video buffer in a virtual system target decoder model for decoding re-encoded encoded streams should have for decoding a first picture at the top of a first encoded stream; calculating, based on the first bit storage quantity, a second bit storage quantity the video buffer should have for ending, according to the virtual system target model, decoding of a second picture of a second encoded stream concatenated to the beginning of the first picture and ending with the second picture; re-encoding the second encoded stream based on the second bit storage quantity; and pg,75 changing a number of pictures included in the second encoded stream based on the second bit storage quantity to re-encode the second encoded stream whose number of pictures is changed.

---

Description

---

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a video stream encoder and video stream encoding method, and a video stream multiplexer and video stream multiplexing method, and more particularly, to a video stream encoder and video stream encoding method, and a video stream multiplexer and video stream multiplexing method, capable of re-encoding, for edition of a video stream, the video stream without reference to information indicative of an initial status of a video buffer when re-encoding a part of the video stream.

2. Description of the Related Art

There is available the MPEG (Moving Picture coding Experts Group) standard as one of the techniques for encoding video streams representing a moving picture. The MPEG standard prescribes a virtual decoder model for connection to the output of an encoder. More specifically, to prevent any underflowing and overflowing of a buffer in the decoder model, to which an encoded video stream is supplied, the MPEG standard limits the buffer capacity anticipated when encoding the video stream. By thus limiting the occupancy by stored data of the buffer included in the decoder model, the MPEG defines a limitation imposed on the encoding of a video stream. The virtual decoder model is called "VBV (Video Buffering Verifier)" and the buffer in the virtual decoder is called "VBV buffer".

For conformity with the requirements prescribed in the MPEG standard, a video stream should be encoded for neither underflowing nor overflowing of the VBV buffer. For example MP@ML (Main Profile at Main Level) of the MPEG-2 standard, the VBV buffer size is defined as 1.75 Mbits.

The encoder encodes a 16-bit field included in each picture and called "information indicative of an initial status of a video buffer" indicative of an initial status of the buffer at a random access. The information indicative of an initial status of a video buffer has a value representing a bit storage quantity (bit occupancy) the VBV buffer should have for decoding the picture. When decoding a picture, the virtual decoder reads a information indicative of an initial status of a video buffer appended to the picture, and then decodes the picture when the VBV buffer has a bit storage quantity represented by the information indicative of an initial status of a video buffer.

When editing two video streams encoded according to the MPEG standard to concatenate them with an accuracy of pictures, a few pictures before and after an edit point which is a concatenation between the two video streams are re-encoded in some cases. For a picture reproduction with a decoding of the edited video streams with no seam between them by the decoder, it is necessary that the video streams should have been edited and re-encoded for no overflowing and underflowing of the VBV buffer. To this end, an outpoint-side picture, in the edited video stream, positioned temporally before the edit point and an inpoint-side picture positioned temporally after the edit point are re-encoded as will be described below:

To re-encode an outpoint-side picture, the encoder first reads the information indicative of an initial status of a video buffer of a picture before a first picture to be re-encode. Next, the encoder uses the read information indicative of an initial status of a video buffer to calculate an initial bit storage quantity the VBV buffer should have for re-encoding the first picture to re-encoded by the VBV. Then, based on the computed initial bit storage quantity, the encoder re-encodes the pictures according to the VBV model.

Also, to re-encode an inpoint-side picture, the encoder first reads the information indicative of an initial status of a video buffer of a picture next to a last picture to re-encode. Next, the encoder uses the read information indicative of an initial status of a video buffer to calculate a bit storage quantity the VBV buffer should have for the last picture to re-encode by the VBV. Then, the encoder re-encodes the picture from the computed bit storage quantity according to the VBV model.

The MPEG-2 standard prescribes that it may optionally be selected whether or not information indicative of an initial status of a video buffer value should be encoded to be header information of a picture layer and included in a video stream. Therefore, in many cases, the above encoder encodes, into a value "0xFFFF", the 16-bit information indicative of an initial status of a video buffer field included in header information of a picture layer of a video stream having been encoded in conformity to the MPEG-2 standard. Thus, when the information indicative of an initial status of a video buffer value is optionally encoded, it is not possible to calculate a correct bit storage quantity of the VBV buffer for an arbitrary picture. Therefore, with the above encoder, a bit storage quantity cannot be computed and a picture cannot be re-encoded from any bit storage quantity when editing video streams.

OBJECT AND SUMMARY OF THE INVENTION

It is therefore an object of the present invention to overcome the above-mentioned drawbacks of the prior art by providing an encoding apparatus and method, and a multiplexing apparatus and method, adapted to re-encode a part of video streams in edition of them without the necessity of referencing to information indicative of an initial status of a video buffer included as a parameter of picture layer.

The above object can be attained by providing an encoder including, according to the present invention:

means for re-encoding a first encoded stream ending with a first picture and a second stream concatenated to the end of the first picture and starting with a second picture;

a first means for calculating, for a video buffer in a virtual system target decoder model for decoding encoded streams re-encoded by the encoding means, a first bit storage quantity which should be when input of the first picture of the first encoded stream to the video buffer ends;

a second means for calculating, based on the first bit storage quantity calculated by the first bit storage quantity calculating means, a second bit storage quantity the video buffer in the virtual system target decoder model should have for starting decoding of the second picture of the second encoded stream according to the virtual system target model; and

means for controlling, based on the second bit storage quantity calculated by the second bit storage quantity calculating means, re-encoding of the second encoded stream, effected by the encoding means.

The above encoder further includes means for multiplexing, based on the first bit storage quantity, multiplexed streams including the second encoded stream re-encoded by the encoding controlling means.

The virtual system target decoder used in the above encoder conforms to the MPEG (Moving Picture coding Experts Group) standard.

Also the above object can be attained by providing an encoding method including, according to the present invention, steps of:

calculating, for a video buffer in a virtual system target decoder model for decoding re-encoded encoded streams, a first bit storage quantity which should be when input of a first picture of a first encoded stream ending with the first picture to the video buffer ends;

calculating, based on the first bit storage quantity, a second bit storage quantity the video buffer in the virtual system target decoder should have for starting, according to the virtual system target model, decoding of a second picture of a second encoded stream concatenated to the end of the first picture and beginning with the second picture; and

re-encoding the second encoded stream based on the second bit storage quantity.

The above encoding method further includes a step of re-multiplexing, based on the first bit storage quantity, multiplexed streams including the second encoded stream re-encoded by the encoding controlling means.

The virtual system target decoder used in the above encoding method conforms to the MPEG (Moving Picture coding Experts Group) standard.

With the above encoding apparatus and method, it is possible to re-encode a part of video streams without the necessity of referencing to information indicative of an initial status of a video buffer included as a parameter of picture layer.

Also the above object can be attained by providing an encoder including according to the present invention:

means for re-encoding a first encoded stream beginning with a first picture and a second stream concatenated to the beginning of the first picture and ending with a second picture;

means for calculating a first bit storage quantity a video buffer in a virtual system target decoder model for decoding encoded streams re-encoded by the encoding means should have for decoding the first picture of the first encoded stream;

means for calculating, based on the first bit storage quantity calculated by the first bit storage quantity calculating means, a second bit storage quantity the video buffer in the virtual system target decoder model should have for ending decoding of the second picture of the second encoded stream according to the virtual system target model; and

means for controlling, based on the second bit storage quantity calculated by the second bit storage quantity calculating means, re-encoding of the second encoded stream, effected by the encoding means.

The virtual system target decoder used in the above encoder conforms to the MPEG (Moving Picture coding Experts Group) standard.

Also the above object can be attained by providing an encoding method including, according to the present invention, steps of:

calculating a first bit storage quantity a video buffer in a virtual system target decoder model for decoding re-encoded encoded streams should have for decoding a first picture at the top of a first encoded stream;

calculating, based on the first bit storage quantity, a second bit storage quantity the video buffer should have for ending, according to the virtual system target model, decoding of a second picture of a second encoded stream concatenated to the beginning of the first picture and ending with the second picture; and

re-encoding the second encoded stream based on the second bit storage quantity.

With the above encoding apparatus and method, it is possible to re-encode a part of video streams without the necessity of referencing to information indicative of an initial status of a video buffer included as a parameter of picture layer.

Also the above object can be attained by providing a multiplexer including according to the present invention:

means for re-encoding a first encoded stream beginning with a first picture and a second stream concatenated to the beginning of the first picture and ending with a second picture;

means for calculating a first bit storage quantity a video buffer in a virtual system target decoder model for decoding encoded streams re-encoded by the encoding means should have for decoding the first picture of the first encoded stream;

means for calculating, based on the first bit storage quantity calculated by the first bit storage quantity calculating means, a second bit storage quantity the video buffer in the virtual system target decoder model should have for decoding of the second picture of the second encoded stream according to the virtual system target model;

means for controlling, based on the second bit storage quantity calculated by the second bit storage quantity calculating means, re-encoding of the second encoded stream, effected by the encoding means;

a multiplexing means for generating a multiplexed stream including the second encoded stream using the second encoded stream re-encoded by the encoding means;

means for judging whether or not a third bit storage quantity the video buffer in the virtual system target decoder should have for concatenating the multiplexed stream re-multiplexed by the multiplexing means to a multiplexed stream including the first encoded stream presented after the re-multiplexed multiplexed stream from the multiplexing means, is within a predetermined range;

a multiplexing controlling means for controlling, based on the judgment result from the judging means, the time when the first picture of the multiplexed stream including the first encoded stream is to be supplied to the system target decoder model.

Also the above object can be attained by providing a multiplexing method including, according to the present invention, steps of:

calculating a first bit storage quantity a video buffer in a virtual system target decoder model for decoding re-encoded encoded streams should have for decoding a first picture at the top of a first encoded stream;

calculating, based on the first bit storage quantity, a second bit storage quantity the video buffer in the virtual system target decoder should have for ending, according to the virtual system target model, decoding of a second picture of a second encoded stream concatenated to the beginning of the first picture and ending with the second picture;

re-encoding the second encoded stream based on the second bit storage quantity;

generating a multiplexed stream including the second encoded stream using the re-encoded second encoded stream;

judging whether or not a third bit storage quantity the video buffer in the virtual system target decoder should have for concatenating the re-multiplexed multiplexed stream multiplexed to a multiplexed stream including the first encoded stream presented after the multiplexed stream, is within a predetermined range;

controlling, based on the judgment result, the time when the first picture of the multiplexed stream including the first encoded stream is to be supplied to the system target decoder model.

With the above multiplexing apparatus and method, it is possible to re-encode, for re-multiplexing, a part of video streams without the necessity of referencing to information indicative of an initial status of a video buffer included as a parameter of picture layer.

These objects and other objects, features and advantages of the present intention will become more apparent from the following detailed description of the preferred embodiments of the present invention when taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of the moving picture recording and/or writing apparatus according to the present invention;

FIG. 2a shows a video stream at the outpoint side;

FIG. 2b shows a video stream at the inpoint side;

FIG. 2c shows a portion of the outpoint-side video streams, re-encoded and re-multiplexed by the moving picture recording and/or reproducing apparatus in FIG. 1;

FIG. 2d shows a portion of the inpoint-side video streams, re-encoded and re-multiplexed by the moving picture recording and/or reproducing apparatus;

FIG. 2e shows an order in which the video streams re-encoded and re-multiplexed by the moving picture recording and/or reproducing apparatus are presented;

FIG. 3a shows multiplexed streams of an outpoint-side program re-multiplexed by the moving picture recording and/or reproducing apparatus;

FIG. 3b shows a bridge sequence generated by re-multiplexing;

FIG. 4a shows multiplexed streams of the inpoint-side program re-multiplexed by the moving picture recording and/or reproducing apparatus;

FIG. 4b shows a bridge sequence generated by re-multiplexing;

FIG. 5 shows structures of multiplexed streams, before and after an edit point, of the outpoint-side and inpoint-side programs, respectively;

FIG. 6 is a block diagram of a system target decoder to decode streams multiplexed by the moving picture recording and/or reproducing apparatus;

FIG. 7 is a timing chart of operations effected by the system target decoder when it is supplied with steams multiplexed by the moving picture recording and/or reproducing apparatus;

FIG. 8 shows a VBV model conforming to the MPEG standard;

FIG. 9 is a flow chart of operations effected by the moving picture recording and/or reproducing apparatus for re-encoding video streams and then re-multiplexing them;

FIG. 10 shows a relation between STC time for Clip-2 and bit storage quantity the video buffer (bit storage quantity for STD buffer) should have;

FIG. 11 shows a relation between time and bit storage quantity the video buffer (bit storage quantity for STD buffer) should have for encoding Clip- 1;

FIG. 12 shows a relation between time and bit storage quantity the video buffer (bit storage quantity for STD buffer) should have for encoding Clip-2;

FIG. 13 is a flow chart of operations effected by the moving picture recording and/or reproducing apparatus for re-encoding a video stream of the outpoint-side program;

FIG. 14 shows a relation between quantity of bits stored in the STD video buffer when the outpoint-side video stream ( . . . , B.sub.07) is supplied, and STC-1 according to SCR added to a video pack;

FIG. 15 shows a relation between quantity of bits stored in the VBV video buffer when the inpoint-side video stream (I.sub.12, . . . ) is supplied, and STC-1 according to SCR added to a video pack;

FIG. 16 shows a relation between quantity of bits stored in the STD video buffer when the outpoint-side video stream is supplied and then the inpoint-side video stream is supplied, and STC-1 and STC-2;

FIG. 17 is a flow chart of operations effected by the moving picture recording and/or reproducing apparatus when re-encoding and re-multiplexing video streams of the inpoint-side program;

FIG. 18 shows a relation between quantity of bits stored in the STD buffer when the inpoint-side video stream (I.sub.m2, I.sub.m0, . . . ) is supplied, and STC-2 according to SCR added to a video pack;

FIG. 19 explains how to determine an allocated bit quantity so that bit storage quantity Bend2 the VBV buffer should have at a time when a picture B.sub.n7x included in GOP(n-x) being a GOP to re-encode, becomes larger than a bit storage quantity Bj;

FIG. 20 shows a relation between STC-2 and bit storage quantity the STD video buffer should have when a video pack including a re-multiplexed inpoint-side GOP starts being supplied (SCR_video2_start) from a time scr(I.sub.n5x) and then video packs after the time scr(.sub.m2) are supplied to the STD video buffer;

FIG. 21 shows a relation between STC-2 and quantity of bits stored in the STD video buffer when a video pack including a re-multiplexed inpoint-side GOP starts being supplied (SCR_video2_start) from the time scr(I.sub.n5x) and then video packs after a time scr(I.sub.m2) are supplied to the STD video buffer;

FIG. 22 explains how to rewrite SCR added to a video pack so that an inpoint-side video data is supplied to the video buffer with a delay of a predetermined time;

FIGS. 23a to 23d show together another example of re-encoding of an outpoint-side video stream by the moving picture recording and/or reproducing apparatus, of which:

FIG. 23a shows an outpoint-side video stream including an outpoint picture;

FIG. 23b shows re-encoding and re-multiplexing of outpoint-side video streams by the moving picture recording and/or reproducing apparatus;

FIG. 23c shows another example of re-encoding and re-multiplexing of outpoint-side video streams by the moving picture recording and/or reproducing apparatus; and

FIG. 23d shows a still another example of re-encoding and re-multiplexing of outpoint-side video streams by the moving picture recording and/or reproducing apparatus;

FIGS. 24a to 24d show together another example of re-encoding of an inpoint-side video stream by the moving picture recording and/or reproducing apparatus, of which:

FIG. 24a shows an inpoint-side video stream including an inpoint picture;

FIG. 24b shows re-encoding and re-multiplexing of inpoint-side video streams by the moving picture recording and/or reproducing apparatus;

FIG. 24c shows another example of re-encoding and re-multiplexing of inpoint-side video streams by the moving picture recording and/or reproducing apparatus; and

FIG. 24d shows a still another example of re-encoding and re-multiplexing of inpoint-side video streams by the moving picture recording and/or reproducing apparatus;

FIG. 25 is a flow chart of operations effected by the moving picture recording and/or reproducing apparatus for re-encoding by the video encoder having been described above with reference to FIG. 23 and re-multiplexing by the multiplexer;

FIG. 26 explains an example that the bit storage quantity Bend1 is judged by the video encoder to be larger than the initial bit storage quantity Bvs;

FIG. 27 explains an example that the bit storage quantity Bend1 is judged by the video encoder to be smaller than the initial bit storage quantity Bvsd;

FIG. 28 explains how to change the bit storage quantity of the video buffer shown in FIG. 27;

FIG. 29 is a flow chart of operations effected by the moving picture recording and/or reproducing apparatus for re-encoding by the video encoder having been described above with reference to FIG. 24 and re-multiplexing by the multiplexer;

FIG. 30 explains an example that the bit storage quantity Bend2 is judged by the video encoder to be larger than the initial bit storage quantity Bj;

FIG. 31 explains an example that the bit storage quantity Bend2 is judged by the video encoder to be smaller than the initial bit storage quantity Bj;

FIG. 32 explains how to change the bit storage quantity of the video buffer shown in FIG. 31; and

FIG. 33 is a block diagram of a system target decoder to combine multiplexed streams multiplexed in the form of a transport stream by the moving picture recording and/or reproducing apparatus.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to FIG. 1, there is illustrated in the form of a schematic block diagram a moving picture recording and/or reproducing apparatus in which the encoder or encoding method and multiplexer or multiplexing method according to the present invention are employed. The moving picture recording and/or reproducing apparatus is generally indicated with a reference 1. The moving picture recording and/or reproducing apparatus 1 edits a moving image data having been compression-encoded according to the MPEG-2 standard to generate a moving image data which can be reproduced with no seam between two video streams concatenated by a decoder, and writes the moving picture data again to an optical disc 2.

The optical disc 2 used in the moving picture recording and/or reproducing apparatus 1 has packed therein a video data and audio data encoded according to the MPEG-2 standard, and has recorded therein multiplexed streams time-division multiplexed in units of a pack. A moving picture program including an outpoint picture in any of two video streams concatenated for reproduction is called "outpoint-side program" and a moving picture program including an inpoint picture in the video stream is called "inpoint-side program". A group of pictures (GOP) including the outpoint picture in the video stream reproduced is called "outpoint-side GOP" and a GOP including the inpoint picture in the video stream reproduced is called "inpoint-side GOP".

As shown in FIG. 1, the moving picture recording and/or reproducing apparatus 1 includes a read head 3 to read multiplexed streams from the optical disc 2, a demodulator 4 to demodulate the multiplexed streams read by the read head 3, an error correction circuit (ECC) 5 to make an error correction of the multiplexed streams demodulated by the demodulator 4, a buffer 6 to provisionally store the multiplexed streams whose error has been corrected by the ECC circuit 5, an error correction code (ECC) appending circuit 7 to append an error correction code (ECC) to multiplexed streams generated by edition, a modulator 8 to modulate the multiplexed streams to which the ECC has been appended by the ECC appending circuit 7, and a write head 9 to write to the optical disc 2 the multiplexed streams modulated by the modulator 8.

Further, the moving picture recording and/or reproducing apparatus 1 includes a demultiplexer 11 to separate multiplexed streams stored in the buffer 6 into a video stream and audio stream, a video decoder 12 for decoding the video stream separated by the demultiplexer 11 to generate video data, a video encoder 13 for re-encoding the video data decoded by the video decoder 12 to generate video stream, and a multiplexer 14 for making a time-division multiplexing of the video and audio streams to generate multiplexed streams.

Moreover, the moving picture recording and/or reproducing apparatus 1 includes an edit information input device 15 to supply the read head 3 with edit information necessary for reproduction of information such as inpoint picture information and outpoint picture information. The edit information input device 15 generates edit information from inpoint picture information and outpoint picture information designated according to an operation input command supplied from a keyboard or the like operated by the user for example, and supplies it to the read head 3.

Also, the moving picture recording and/or reproducing apparatus 1 includes an analysis controller 16 which analyzes edit information supplied from the edit information input device 15, multiplexed streams supplied from the demultiplexer 11, etc., generates editing information necessary for allowing the decoder to make a seamless reproduction, and controls the video encoder 13 and multiplexer 14. The analysis controller 16 provides the video encoder 13 with a method allowing the video encoder 13 to re-encode the a video stream, and supplies the multiplexer 14 with editing information suggesting a method by which the multiplexer 14 is allowed to re-multiplex the video stream.

Furthermore, the moving picture recording and/or reproducing apparatus 1 includes a select switch 17 to select a video stream route along which a video stream is supplied to the multiplexer 14. Under the control of the analysis controller 16, the select switch 17 makes a selection between two routes, along which a video stream separated by the demultiplexer 11 is decoded by the video decoder 12 and re-encoded by the video encoder 13 and then supplied to the multiplexer 14, and the video stream separated by the demultiplexer 11 is supplied directly to the multiplexer 14, respectively. An audio stream separated by the demultiplexer 11 is supplied directly to the multiplexer 14 without being decoded and re-encoded.

In the moving picture recording and/or reproducing apparatus 1 constructed as in the above, the analysis controller 16 analyzes multiplexed streams recorded in the optical disc 2 to control the read head 3, video decoder 12, video encoder 13, multiplexer 14 and select switch 17, thereby producing a bridge sequence in which video streams concatenated by the decoder are reproduced with no seam between the video streams, and records the bridge sequence into the optical disc 2.

Re-encoding of a video stream in the moving picture recording and/or reproducing apparatus 1 will be described below:

In the moving picture recording and/or reproducing apparatus 1, a video stream is re-encoded by the video encoder 13 under the control of the analysis controller 16 so that for skip reproduction of a part of a moving picture program in the decoder, an outpoint-side program being a program temporally before an outpoint picture at which the skip reproduction is to start and an inpoint-side program being a program temporally after an inpoint picture at which the skip reproduction is to arrive, can be concatenated with no seam between them.

A GOP being a unit of picture groups conforming to the MPEG-2 standard includes three kinds of encoded pictures: at least an I (intra) picture being a reference picture resulted from encoding of a picture with no predictive encoding from any other picture, P (predictive) pictures being forward predictive-encoded picture resulted from encoding of a picture with a forward predictive encoding in the order of presentation, and B (bidirectional) pictures being bidirectional predictive-encoded pictures resulted from encoding of a picture within both forward predictive encoding and backward predictive encoding.

Assume for example that as shown in FIG. 2a, an outpoint-side GOP including an outpoint picture (Pout) is GOP(1) and an outpoint picture is B.sub.14 which is a B picture included in the outpoint-side GOP. The GOP(1) is contiguous to GOP(0). Also assume for example that as shown in FIG. 2b, an inpoint-side GOP including an inpoint picture (Pin) is GOP(n) and an outpoint picture is B.sub.n4 which is a B picture included in the in point-side GOP. The GOP(n) is followed by GOP(m). Note that an I picture presented at the j-th position (namely, its temporal reference is "j") in the i-th GOP (namely, GOP(i)) will be annotated with "I.sub.ij", a P picture presented at the i-th position in the i-th GOP(i) be annotated with "P.sub.ij", and a B picture presented at the j-th position in the i-th GOP(i) be annotated with "B.sub.ij".

More specifically, the GOP(1) being an outpoint-side GOP includes pictures I.sub.12, B.sub.10, B.sub.11, P.sub.15, B.sub.13, B.sub.14, P.sub.18, B.sub.16 and B.sub.17 which are recorded in this order into the optical disc 2. Also, the GOP(n) being an inpoint-side GOP includes pictures I.sub.n2, B.sub.n0, B.sub.n1, P.sub.n5, B.sub.n3, B.sub.n4, P.sub.n8, B.sub.n6 and B.sub.n7 which are recorded in this order into the optical disc 2.

The re-encoding by the video encoder 13 in the moving picture recording and/or reproducing apparatus 1 will be described below concerning the outpoint-side GOP and inpoint-side GOP as shown in FIGS. 2a and 2b.

First, The GOP(1) being an outpoint-side GOP including an outpoint picture (Pout) is decoded. Then, the GOP(1) is re-encoded so that the outpoint picture (Pout) can be decoded without the necessity of predictive reference to any encoded pictures after the outpoint picture (Pout) in the order of presentation. If the picture B.sub.14 in the GOP(1) being an outpoint-side GOP as shown in FIG. 2a is an outpoint picture (Pout), the pictures B.sub.13 and B.sub.14 having been predictive-encoded based on the picture P.sub.15 can be made without predictive reference to the picture P.sub.15, thereby generating a GOP(1-x) being a new GOP as shown in FIG. 2c.

More particularly, to generate the GOP(1-x), first the pictures I.sub.12, B.sub.10, B.sub.11, P.sub.15, B.sub.13 and B.sub.14 are decoded and rendered back to non-coded video data by the video decoder 12, and then the picture B.sub.14 is re-encoded to a picture P.sub.14x of a P picture predictive-encoded based on the picture I.sub.12. Next, the picture B.sub.13 is re-encoded to a picture B.sub.13x of a B picture predictive-encoded based on the pictures I.sub.12 and P.sub.14x. The pictures I.sub.12, B.sub.10 and B.sub.12 are copied from the GOP(l) without being re-encoded. These pictures I.sub.12, B.sub.10 and B.sub.12 may be re-encoded. As a result of the re-encoding, there is generated the GOP(1-x) composed of the pictures I.sub.12, B.sub.10, B.sub.11, P.sub.14x and B.sub.13x and SQE (sequence-end-code) as shown in FIG. 2c.

Next, The GOP(n) being an inpoint-side GOP including an inpoint picture (Pin) is decoded. Then, the GOP(n) is re-encoded so that the outpoint picture (Pin) can be decoded without the necessity of predictive reference to any encoded pictures before the outpoint picture (Pin) in the order of presentation. If the picture B.sub.n4 in the GOP(n) being an inpoint-side GOP as shown in FIG. 2b is an inpoint picture (Pin), the picture B.sub.n4 having been predictive-encoded based on the picture I.sub.n2 can be made without predictive referencing, thereby generating a GOP(n-x) being a new GOP as shown in FIG. 2d.

More particularly, to generate the GOP(n-X), first the pictures I.sub.n2, B.sub.n0, B.sub.n1, P.sub.n5, B.sub.n3, B.sub.n4, P.sub.n8, B.sub.n6 and Bn.sub.7 are decoded and rendered back to non-encoded video data by the video decoder 12, and then the picture P.sub.n5 is re-encoded to a picture In5x of an I picture. For the pictures B.sub.n4, P.sub.n8, B.sub.n6 and B.sub.n7, pictures B.sub.n4x, P.sub.n8x, B.sub.n6x and B.sub.n7x of the same types as the pictures in consideration are re-encoded. As a result of the re-encoding, there is generated the GOP(n-x) composed of SH (sequence header) the pictures I.sub.n5x, B.sub.n4x, P.sub.n8x, B.sub.n6x and B.sub.n7x as shown in FIG. 2e.

In the moving picture recording and/or reproducing apparatus 1, the re-encoding of the inpoint-side and outpoint-side GOPs is effected by the video decoder 12, video encoder 13 and select switch 17 under the control of the analysis controller 16. By re-encoding the pictures ( . . . , I.sub.12, B.sub.10, B.sub.11 and B.sub.13) before the outpoint picture B.sub.14 in the order of presentation and pictures (P.sub.n5, P.sub.n8, B.sub.n6, B.sub.n7, . . . ) after the inpoint picture B.sub.n4 in the order of presentation, the moving picture recording and/or reproducing apparatus 1 generates a moving picture which is displayed in a presented order of . . . , B.sub.10, B.sub.11, I.sub.12, B.sub.13x, P.sub.14x, B.sub.n3x, B.sub.n4x, I.sub.n5x, B.sub.n6x, B.sub.n7x, P.sub.n8x, . . . .

Next, the re-multiplexing of multiplexed streams by the multiplexer 14 provided in the moving picture recording and/or reproducing apparatus 1 will be described herebelow:

For skip reproduction of a part of a moving picture program in the decoder, the analysis controller 16 controls to have the multiplexer 14 re-multiplex multiplexed streams so that an outpoint-side program being a program temporally before an outpoint picture at which the skip reproduction is to start and an inpoint-side program being a program temporally after an inpoint picture at which the skip reproduction is to arrive can be concatenated with no seam between them.

FIG. 3a shows a structure example of multiplexed streams of an outpoint-side program. In FIG. 3a, Clip-A is multiplexed streams including the outpoint-side program. For example, it is a program stream followed by a system clock reference (SCR) defined in the MPEG-2 system standard (ISO/IEC 13818-1). The Clip-A includes video streams and audio streams as time-division multiplexed in packs. In FIG. 3, each of video streams v0, v1, v2 and v3 is a video stream having the length of a GOP, and each of audio streams a0, a1, a2 and a3 is an audio stream having the length of a GOP. In the Clip-A, the streams v1 and a0 are time-division multiplexed in packs between byte positions Ba and Bjo, for example, of the Clip-A. Note that one pack has a size of 2,048 bytes for example.

As shown in FIG. 3a, in the Clip-A, the audio streams are located at byte positions a predetermined number of bytes (audio skew: AV multiplexing phase difference) off those of the video streams which will be reproduced synchronously with the audio streams. In the example shown in FIG. 3a, the audio skew is constant. It should be noted however that the audio skew may be variable in a program stream. In this example, the streams v0 and a0 are synchronous with each other. Similarly, the streams v1 and a1, v2 and a2, and v3 and a3 are synchronous with each other, respectively.

Assume here that an outpoint picture (Pout) is selected from a GOP of v3 in the Clip-A. In this case, the moving picture recording and/or reproducing apparatus 1 follows the procedure below to generate an outpoint-side bridge sequence in the following procedure. The bridge sequence is multiplexed streams acquired by re-multiplexing video streams generated by re-encoding video streams near the edit point.

First at the first step, the GOP of v3 including outpoint pictures is re-encoded by re-encoding the above video stream. In this case, a GOP of v3' is newly generated by re-encoding the GOP of video stream v3. The time length of this new video stream v3' is shorter than the video stream v3.

At the second step, one of video streams existing at byte points after a point Bjo for jump from the Clip-A to the outpoint-side bridge sequence and preceding the video stream v3, namely, video stream v2 in this case, is copied from the Clip-A. Also, audio streams at byte positions after the jump point Bjo and preceding an audio stream synchronous with the new video stream v3', that is, audio streams a1 and a2 in this case, are copied from the Clip-A. Next, the audio stream synchronous with the new video stream v3' is copied from inside the audio stream a3 to generate an audio stream a3'.

At the third step, the video streams and audio streams generated at the first and second steps are re-multiplexed. In this case, the video streams v2 and v3', and audio streams a1, a2 and a3' are re-multiplexed to generate a bridge sequence-A as shown in FIG. 3b and record it in the optical disc 2.

For reading the optical disc 2 having the bridge sequence-A recorded therein to concatenate the two video streams in the decoder for reproduction, the Clip-A is read down to the jump point Bjo for reproduction of the outpoint-side multiplexed streams, and then the bridge sequence-A is read.

Note that in the moving picture recording and/or reproducing apparatus 1, streams successively existing between the Clip-A down to the jump point Bjo and the bridge sequence-A have to be multiplexed to provide a program stream followed by an SCR.

FIG. 4a shows a structure example of multiplexed streams of an outpoint-side program. In FIG. 4a, Clip-B is multiplexed streams including the inpoint-side program. For example, it is a program stream followed by a system clock reference (SCR) defined in the MPEG-2 system. The Clip-B includes video streams and audio streams as time-division multiplexed in packs. In FIG. 4, each of video streams v5, v6, v7 and v8 is a video stream having the length of a GOP, and each of audio streams a5, a6 and a7 is an audio stream having the length of a GOP, as in FIG. 3. In the Clip-B, the streams v8 and a7 are time-division multiplexed in packs between byte positions Bji and Bb, for example, of the Clip-B. As shown in FIG. 4a, in the Clip-B, the audio streams are located at byte positions a predetermined number of bytes (audio) off those of the video streams which will be reproduced synchronously with the audio streams. Also in the example shown in FIG. 4a, the audio skew is constant. It should be noted however that the audio skew may be variable in a program stream. In this example, the streams v5 and a5 are synchronous with each other. Similarly, the streams v6 and a6, v7 and a7, and v8 and a8 are synchronous with each other, respectively.

Assume here that an inpoint picture (Pin) is selected from a GOP of v5 in the Clip-B. In this case, the moving picture recording and/or reproducing apparatus 1 follows the procedure below to generate an inpoint-side bridge sequence in the following procedure.

First at the first step, the GOP of v5 including inpoint pictures is re-encoded by re-encoding the above video stream. In this case, a GOP of v5' is newly generated by re-encoding the GOP of video stream v5. The time length of this new video stream v5' is shorter than the video stream v5.

At the second step, video streams existing after the video stream v5 and at byte points preceding a point Bji for jump from the inpoint-side bridge sequence to the Clip-B, namely, video streams v6 and v7 in this case, are copied from the Clip-B. Also, a one of audio streams existing after an audio stream synchronous with the new video stream v5' and at byte positions before the jump point Bji, that is, an audio stream a6 in this case, is copied from the Clip-B. Next, the audio stream synchronous with the new video stream v5' is copied from inside the audio stream a5 to generate an audio stream a5'.

At the third step, the video streams and audio streams generated at the first and second steps are re-multiplexed. In this case, the video streams v5', v6 and v7, and audio stream a5' and a6 are re-multiplexed to generate a bridge sequence-B as shown in FIG. 4b and record it in the optical disc 2.

For reading the optical disc 2 having the bridge sequence-B recorded therein to concatenate the two video streams in the decoder for reproduction, the bridge sequence-B is read for reproduction of the inpoint-side program, and then Clip-B is read from the jump point Bji.

Note that in the moving picture recording and/or reproducing apparatus 1, streams successively existing between the bridge sequence-B and Clip-B after the jump point Bji have to be multiplexed to provide a program stream followed by an SCR.

In the moving picture recording and/or reproducing apparatus 1, the above-mentioned re-multiplexing permits to generate the bridge sequence-A as shown in FIG. 3b and bridge sequence-B as shown in FIG. 4b.

FIG. 5 shows the structure of multiplexed streams presented before and after the edit point, in which it is assumed that multiplexed streams lasting from the Clip-A before the jump point Bjo to the bridge sequence-A are Clip-1 and a multiplexed stream lasting from the bridge sequence-B to the Clip-B after the jump point Bji are Clip-2. In the decoder, when the multiplexed stream lasting from Clip-1 to Clip-2 has successively been decoded, the video and audio streams have to be reproduced with no seam between them. In the moving picture recording and/or reproducing apparatus 1, for reproduction of the video and audio streams with no seam between them, the audio streams in Clip-1 and Clip-2 should be re-encoded and re-multiplexed with the following limits imparted to them:

In the moving picture recording and/or reproducing apparatus 1, the audio streams in Clip-1 and Clip-2 are limited so that no gap between times at which an audio stream is presented will exist at the boundary between the trailing end of Clip-1 and beginning end of Clip-2. More particularly, the audio streams of Clip-1 are re-multiplexed to include audio samples presented at a time when video streams of Clip-1 stops being presented, while the audio streams of Clip-2 are re-multiplexed to include audio samples presentation at a time when video streams of Clip-2 start being presented. Therefore, at the boundary between the trailing end of Clip-1 and beginning end of Clip-2, time lengths of presentation shorter than a time for less than two audio frames will possibly overlap each other. Note that in case of audio streams of Clip-1 in the MPEG-1 standard, one audio frame is an audio stream having a time length of presentation of 24 msec.

In FIG. 5, V1LBI, A1LBI, V2FBI and A2FBI in FIG. 5 are as follows:

V1LBI Last byte position of last pack in video-1 in Clip-1

A1LBI Last byte position of last pack in audio-1 in Clip-1

V2FBI First byte position of first pack in video-1 in Clip-2

A2FBI First byte position of first pack in audio-2 in Clip-2

These byte positions V1LBI and A1BLI, and V2FBI and A2FBI are in the following relations, respectively:

V1LBI<A1LBI

V2FBI<A2FBI

Other than the above relations are possible in the MPEG standard, but they rarely exist in practice.

FIG. 6 shows the construction of a system target decoder being a virtual decoder model intended to reproduce multiplexed streams Clip-1 and Clip-2 each including packs of audio streams and video streams re-encoded and re-multiplexed by the moving picture recording and/or reproducing apparatus 1 and recorded in the optical disc 2. The system target decoder is generally indicated with a reference 20.

Each of the multiplexed streams include a video pack having appended thereto as additional information SCR (system clock reference) indicative of a n output timing to the system target decoder 20, a decoding time stamp (DTS) indicative of a decoding timing and a presentation time stamp (PTS) indicative of an output timing of decoded data, and an audio pack having SCR, DTS and PTS appended thereto as additional information as in the video pack. The decoding timing and output timing are controlled with reference to these additional information.

As shown in FIG. 6, the system target decoder 20 includes a demultiplexer 21 to receive multiplexed streams Clip-1 and Clip-2 read from the optical disc 2 and separate the multiplexed streams into a video stream and audio stream, a video buffer 22 which provisionally stores the video stream separated by the demultiplexer 21, an audio buffer 23 which provisionally stores the audio stream separated by the demultiplexer 21, a video decoder 24 to extract, for decoding, the video stream stored in the video buffer 22, a re-order buffer 25 which provisionally stores decoded video data, an audio decoder 26 which extracts, for decoding, the audio stream stored in the audio buffer 23, and an output switch 27 which selects, for delivery as output, the video data decoded by the video decoder 24 or video data stored in the re-order buffer 25.

Further the system target decoder 20 includes a timing controller 28 to provide a system time clock (STC) used to control a timing of the selection between the video stream and audio stream by the demultiplexer 21, decoding timing of the video decoder 24, decoding and output timing of the audio decoder 26, and an output timing of the output switch 27.

Also the system target decoder 20 is provided with first to fourth STC switches SW1 to SW4 to make a selection between system time clocks STC-1 and STC-2 provided from the timing controller 28.

The demultiplexer 21 is supplied with packets forming together each multiplexed stream according to SCR appended to the multiplexed stream. Based on a STC supplied from the timing controller 28, the demultiplexer 21 separates the multiplexed stream into a video stream and audio stream. Ans, the demultiplexer 21 provides the video and audio streams to the video buffer 22 and audio buffer 23 at a predetermined output rate (program_mux_rate).

The video decoder 24 is provided to extract, for decoding, data of predetermined data from the video buffer 22 when DTS appended to the video stream coincides with STC supplied from the timing controller 28. Then, when PTS coincides with DTS, the video decoder 24 delivers the decoded data to outside via the output switch 27, or stores it once in the re-order buffer 25 and then delivers it to outside via the output switch 27.

The audio decoder 26 extracts, for decoding, the audio stream from the audio buffer 23 when DTS appended to the audio stream coincides with STC supplied from the timing controller 28. And the audio decoder 26 provides to the STC switch SW3 the audio data having been decoded when PTS and STC coincide with each other.

Note that the audio buffer 23 provided upstream of the audio decoder 26 should have a following size in view of the buffer size prescribed in the MPEG-2 CSPS=1:

additional-buffer-size=(program_mux-rate-Ra)*Ra/program-mux-rate where "Ra" is a maximum bit rate of the audio stream and "program-mux-rate" is either a maximum bit rate of the program stream Clip-1 or Clip-2 whichever is larger. For example, if program-mux-rate=10 Mbps and Ra=256 kbps, the additional-buffer-size of the audio buffer 23 is 0.249 Mbits.

The output switch 27 provides the decoded video data when PTS appended to the video pack coincides with STC supplied from the timing controller 28. Note that the output switch 27 provides as output the video data stored in the re-order buffer 25 as necessary.

When selecting, after the outpoint-side program, the inpoint-side program for concatenation of two video streams for reproduction, the timing controller 28 generates two STCs: one synchronous with SCR of the outpoint-side program and the other synchronous with SCR of the inpoint-side program.

The timing controller 28 includes an STC generator 28a to generate an STC and a subtractor 28b to subtract a predetermined offset value (STC_delta) from an STC generated by the STC generator 28a. Thus, the timing controller 28 generates two STCs. One of the STCs is an STC provided directly from the STC generator 28a and from which no offset value has been subtracted (this is synchronous with SCR of the outpoint-side program; will be referred to as "STC-1" hereinafter), and the other is a one derived from subtraction of the offset value (STC_delta) from an STC provided directly from the STC generator 28a (this is synchronous with SCR of the inpoint-side program; will be referred to as "STC-2" hereinafter).

The offset value (STC_delta) indicates an offset between the time bases of the multiplexed streams Clip-1 and Clip-2. Namely, the offset value is a difference between the time of Clip-1 on the time base when the video stream of Clip-1 disappears and the time of Clip-2 on the time base when the video stream of Clip-2 starts being presented.

On the assumption that PTS of an outpoint picture Pout on the time base of Clip-1 is PTS-Pout, presentation period of the outpoint picture Pout is Tpp and PTS of an inpoint picture Pin on the time base of Clip-2 is PTS-Pin, for example, the offset value "STC_delta" is given by the following expression (1): PTS-Pout-end=PTS-Pout+Tpp i STC-delta=PTS-Pout-end-PTS-Pin (1)

Any of the two STCs (STC-1 and STC-2) provided from the timing controller 28 is selected by the first to forth STC switches SW1 to SW4, and supplied to the demultiplexer 21, video decoder 24, audio decoder 26 and output switch 27. The first STC switch SW1 will be supplied at a terminal A thereof with STC-1 and at a terminal B thereof with STC-2. It selects one of these terminals A and B and supplies STC supplied at the selected terminal to the demultiplexer 21. The second STC switch SW2 will be supplied at a terminal A thereof with STC-1 and at a terminal B thereof with STC-2. It selects one of these terminals A and B and supplies STC supplied at the selected terminal to the video decoder 24. The third STC switch SW3 will be supplied at a terminal A thereof with STC-1 and at a terminal B thereof with STC-2. It selects one of these terminals A and B and supplies STC supplied at the selected terminal to the audio decoder 26. The fourth STC switch SW4 will be supplied at a terminal A thereof with STC-1 and at a terminal B thereof with STC-2. It selects one of these terminals A and B and supplies STC supplied at the selected terminal to the output switch 27.

The system target decoder 20 constructed as in the above functions as will be described herebelow:

FIG. 7 is a timing chart of operations effected by the system target decoder 20 when it is supplied with two multiplexed steams Clip-1 and Clip-2 contiguous to each other.

First, SCR appended to the first pack in Clip-1 is set as STC in the STC generator 28a. All the first to fourth STC switches SW1 to SW4 are connected to the terminals A, respectively, and thus STC-1 (STC synchronous with SCR of Clip-1) is supplied to the demultiplexer 21, video decoder 24, audio decoder 26 and output switch 27. Namely, all the functions work based on SCR appended to Clip-1.

Before a time T1 is reached, input to the demultiplexer 21 is made at a time when SCR appended to each pack in Clip-1 coincides with STC-1 which has not yet added thereto an offset value supplied from the terminal A of the first STC switch SW1. Next, at the time T1, the entry of the last video pack of Clip-1 to the demultiplexer 21 ends. For a time from the time T1 to a time T2, the demultiplexer 21 is supplied with each pack of Clip-1 at a maximum bit rate "program_mux_rate1" of Clip-1 regardless of SCR appended to each pack. The maximum bit rate "program_mux_rate1" may be a maximum transfer rate at which data is to be read from the optical disc 2 for example.

On the assumption that a data amount from a pack next to the last video pack in Clip-1 to the last pack in Clip-1 is "N1", the time ".DELTA.T1" from the time T1 to T2 is given by the following expression (2): .DELTA.T1=T2-T1=N1/program-mux-rate1 (2)

At the time T2, input of the last pack (audio pack) of Clip-1 to the demultiplexer 21 ends. At this time T2, the first STC switch SW1 is shifted to the terminal B and thus STC-2 (STC synchronous with SCR of Clip-2) is supplied to the demultiplexer 21. Therefore, the demultiplexer 21 starts functioning based on SCR appended to Clip-2.

For a time period between T2 and T3, the demultiplexer 21 is supplied with packs including from the first pack in Clip-2 to a pack before the first video pack in Clip-2 at the maximum bit rate program_mux_rate2 of Clip-2 regardless of SCR of each pack when the first packet in Clip-2 is not a video pack. The maximum bit rate program_mux_rate2 may be a maximum transfer rate for reading data from the optical disc 2, for example.

Assuming that the data amount from the first pack in Clip-2 to a pack before the first video pack in Clip-2 is "N2", the time ".DELTA.T2" from the time T2 to a time T3 is given by the following expression (3): .DELTA.T2=T3-T2=N2/program-mux-rate2 (3)

Note that since the first pack in an MPEG-2 program stream is usually a video pack, .DELTA.T2=0.

Further at the time T3, the entry of the first video pack in Clip-2 to the demultiplexer 21 starts. Subsequently, input to the demultiplexer 21 is made at a time when SCR of each pack in Clip-2 coincides with STC-1 which has added thereto an offset value supplied from t