Interlocking device for a receiver of compressed audio and video signals

The present invention relates to a device for processing compressed audio and video signals in a receiver.

It is known to use the MPEG protocol for a compressed video signal, which protocol is defined by the Moving Pictures EKperts Group of International

177 313

Standardization Organization. This protocol defines a universal signal standard that includes both in-frame coding and predicted motion-compensated coding. Due to coding format changes for different frames and image content changes, different frames have very different amounts of compressed data. As a result of different frames, when compressed with different amounts of data, there is a tendency to transmit data frames asynchronously.

It is known that audio signals are also compressed according to the MPEG protocol. The compressed audio signal may be associated with the video signal but transmitted independently. For transmission purposes, the compressed audio signal is segmented into packets, which are then multiplexed with time division together with the compressed video signal in an asynchronous manner.

Related, compressed acoustic and video components are not only independent asynchronously, but their mutual temporal relationship, i.e. synchronism, does not occur during transmission. Compressed audio and video components of the MPEG protocol may contain presentation PTS timestamps for determining the reference between the individual segments of the compressed signal and the reference clock signal. Acoustic and visual PTS timestamps are used by the receiving device to re-synchronize decompressed components and restore their mutual time relationships.

A particularly annoying feature of the reproduced acoustic and video signals is the loss of synchronization of sound with the image, observed on the basis of the movements of the lips, that is, between the movements of the lips and spoken words.

A method of blocking interference is known from U.S. Patent No. 5,381,186, in which the synchronization signal of a received audio signal frame is compared with a standard synchronization signal of a frame, the number of unequal synchronization bits per frame is detected, the number of uneven bits detected the next frame according to the detected number of uneven bits and the interference is blocked according to the detected number of uneven bits.

The interference blocking system used in the video signal decoder is known from the same patent description, containing a synchronization signal comparator for comparing an acoustic frame synchronization signal containing data bits in a received signal, with a standard frame synchronization signal containing a specified number of synchronization bits per frame and producing response signal comparison, detector of the number of uneven bits, which receives the comparison signal for detecting the number of synchronization bits per frame in the audio synchronization signal of the received signal frame and the standard frame synchronization signal that are not matched and the interference blocking controller for controlling interference blocking in response to the number of synchronization bits per frame that are not matched. The known decoder interference blocking system removes the interference generated when changing channels in a multi-channel radio receiver and completely eliminates excessive errors in the received signal and interference generated when changing channels.

The device according to the invention comprises subtraction systems connected to the outputs of acoustic and video signal decompressors, whereby the acoustic signal decompressor is connected via a latch flip-flop to an inverse transport processor and the video signal decompressor is connected via a latch flip-flop to an inverse transport processor, and a subtractor is attached to the subtractor systems threshold connected to the interference blocking transistor.

Preferably, the threshold detector has a first and second state control signal output connected to a interference blocking transistor to pass the decompressed acoustic signal in the second state and provide a replacement value in the first state.

Preferably a digital to analog converter is connected to the interference blocking transistor.

17*7313

Preferably, an element I connected to the acoustic signal decompressor and having a enable input connected to the threshold detector is connected to the interference blocking transistor.

Preferably a clock is included between the acoustic and video signal decompressors and the inversion transport processor, and the subtractor and threshold detector is connected to a filter and a processing system to which the output of the acoustic signal time base system is connected to provide a synchronization error signal.

Preferably, the next threshold detector is connected to the interference transistor and the OR element is connected between them to combine control signals from the threshold detector and the next threshold detector to synchronize sound with the image based on the movements of the mouth.

Preferably, the clock includes a voltage-controlled local clock signal generator, to which the input is connected a meter, and to the output is a clock controller, the input of which is connected to a voltage-controlled generator, for determining the time between the occurrence of time stamps and acoustic and video signals, and subtraction systems are attached to the clock, filter and processing system, for calculating the time stamp difference and generating the synchronization control signal and the threshold detector is adapted to generate an error signal when the synchronization control signal exceeds a certain value.

An advantage of the invention is to provide a processing device in the receiver that provides a return of synchronization between sound and audio in the event of synchronization errors caused by disturbances in audio and video signals.

The subject of the invention is shown in the embodiments in the drawing, in which Fig. 1 is a block diagram of a transmission device for compressing audio and video signals, Fig. 2 is a block diagram of a receiving device for decompressing audio and video signals according to the invention, Fig. 3 is a block diagram a system for providing a clock signal of a receiving device having the same frequency as the clock signal of a transmitting device, fig. 4. - a flow chart of the device of Fig. 2 and Fig. 5 a block diagram of the interference suppression system used in the device of Fig. 2.

Figure 1 shows a typical device for transmitting compressed digital audio and video signals. The signal from the source 10 of the audio and video signals is provided to the video signal compressor 11, which includes, for example, a motion-compensated prediction coder using discrete cosine transforms. The compressed video signal from the compressor 11 is fed to a formatting system 12 that formats the video signal and auxiliary data according to a signal protocol, for example MPEG protocol in the International Organization for Standardization. The standard signal is delivered to the transport processor 13, which splits the signal into data packets and adds organizational data to ensure interference immunity during transmission. Packets transmitted normally at a heterogeneous rate are delivered to a rate buffer 14 that provides output at a relatively constant rate adapted to a transmission channel with a relatively small bandwidth. The buffered data is fed to the forward modem transmitting the signal further.

Clock 22 provides a clock signal controlling a substantial portion of the device, including at least a transport processor 13. Clock 22 operates at a fixed frequency, e.g., 27 MHz, and is used here to generate timing information. The clock 22 is connected to the synchronization input of the counter 23, which is adapted, for example, to counting modulo 2³⁰. The output of the counting values ​​of the counter 23 is connected to two latches 24 and 25. The status of the latch flip 24 is determined by the source of 10 audio and video signals to block the count values ​​at the occurrence of certain frame times. These blocked values ​​correspond to the PTS ^ time stamps from the video presentation and are introduced into the compressed video signal by means of the formatting system 12 and are used by the receiver to ensure the synchronization of mouth movement on video with audio. The state of the latch flip-flop 25 is determined by the transport processor 13 or the system controller 21 to block the counting values ​​according to a particular rule. These counted SCR reference clock values ​​of clock 22 are input as auxiliary data to the transport auxiliary packages.

An audio signal associated with the video signal from the source 10 is provided to the compressor 18 of the audio signals. The compressor 18 provides frame sampling pulses, independent of the video frames, to control the latch trigger 19. According to the sampling pulses, the latch latch 19 registers the values ​​counted by the counter 23. These blocked values ​​correspond to PTS timestamps_akacoustic presentation and are introduced into the compressed audio signal provided by the compressor 18. The compressed audio signal is delivered to the transport processor 17, which divides the signal into data packets and adds organizational data to provide immunity to interference during transmission. The transported acoustic packets provided by the transport processor 17 are fed to the multiplexer 16, which multiplexes the transported audio and video packets with time division. In Fig. 1, separate transport processors are shown in the audio and video signal processing channels. For systems where the data speed is moderate, the functions of two 13.17 transport processors and 16 multiplexer can be combined into a single transport processor.

The system controller 21 is a variable state programmable to coordinate various processing elements. The system controller 21, compressors 11 and 18 of video and audio signals, transport processors 13 and 17, and speed buffer 14 operate synchronously through a common clock 22 as long as proper interaction between processing elements is ensured. Because both compressors 11 and 18 provide PTS timestamp values ​​from the same counter 23, accurate timing is ensured between both compressed signals in the compressed output signal.

Figure 2 shows a receiving device for decompressing audio and video signals according to the invention, in which the receiving modem 200 performs the opposite function to the transmitting modem 15 of Fig. 1 and the speed buffer 206 inversely functions than the speed buffer 14 of Fig. 1. Inverse transport processor 202 splits transport packets and allocates different packets of memory in speed buffer 206 to each packet packet. Individual packet packages are separated from auxiliary data, with auxiliary data being provided to system controller 210. In a different device, a separate transport processor is included in each processing channel and organized to recognize and process only data associated with each channel.

The compressed video data from the speed buffer 206 is passed to the video signal decompressor 214. Speed ​​buffer 206 receives the compressed video data with pulse or variable speed and provides data to the video decompressor 214 on demand. The video signal decompressor 214 produces, according to the compressed video signal, an uncompressed video signal for reproduction or storage in reproduction or memory circuits not shown.

The compressed audio data from the inversion transport processor 202 is provided to the speed buffer 206, which provides, according to the signal protocol, a compressed audio signal to the decompressor 212 of audio signals. The acoustic signal decompressor 212, in response to the compressed acoustic signal, produces an uncompressed acoustic signal for reproduction or storage in reproduction or memory circuits not shown.

Inverse transport processor 202 also provides SCR reference clock signals based on auxiliary data and control signals to clock 208. Clock 208 produces, in response to these signals, a clock signal synchronous with at least the operation of the transport processor. This clock signal is provided to system controller 210 to control the timing of processing elements.

Figure 3 shows in detail the system for providing the clock signal of the receiving device. Data from the receiving modem 200 is forwarded to the inversion

177 313 of the transport processor 202 'comprising a detector 31 auxiliary packets. The inverse transport processor 202 'separates header transport data from individual transport packet packets. In response to the header transport data, the processor 202 'demultiplexes packages of required, related audio and video components. Acoustic and video packages, and auxiliary packages are stored in separate speed buffer memory blocks 206. Each of the memory blocks acts as FIFO memory, saving data when it is available from the modem, and reading data when required by a component signal processor that is not shown. The reference signals for auxiliary vacancies are routed and stored in memory 34.

Auxiliary packet detector 31, which is, for example, a matched filter for recognizing the code words denoting the auxiliary transport packet, containing the SCR reference clock signal, generates a control pulse when transport packets containing such data occur. The control pulse is used to receive and store in the latch flip-flop 35 the count value that is displayed in real time by the counter 36 at a time exactly related to the detection time. The counter 36 counts the pulses provided by the voltage-controlled generator 37 of the local clock signal. Counter 36 counts modulo M, which may or may not be the same number as the number N in its counterpart, in counter 23. If M is different from N, the difference may be placed in the error equation.

The voltage-controlled generator 37 of the local clock signal is controlled by the error signal filtered in the low-pass filter 38 provided by the clock controller 39. The error signal is generated as follows. Let the reference clock signal SCR appearing at time n be marked SCR ", and the counted value obtained simultaneously in the latch 35 is marked LCR". The clock controller 39 reads successive SCR and LCR values ​​and produces an error signal E proportional to the difference:

E => | SCR - SCRn- | -1 LCRn - LCRm |

The error signal E is used to put the voltage-controlled generator 37 into a state with a frequency to compensate for differences. Negative differences, caused by cyclical counting by the modulo counter, can be neglected. The error signal generated by the clock controller 39 is in the form of a pulse width modulated pulse signal, transformed into an analog error signal using a low-pass filter 38 in an analog system.

The limitation of this system is that the counters at both ends of the system count the same frequency or even multiply it. This requires the rated frequency of the voltage-controlled generator to be almost equal to the clock's frequency.

The previous solution provides rather fast synchronization, but can introduce a long-lasting LTE error signal proportional to the difference as follows:

LTE => | LCRn - LCR0 | - | SCRn - SCRj where SCR0 and LCR0 are, for example, the first reference clock signal occurring and the corresponding blocked counter value of the receiver. Nominally, the E and LTE error signals change in discrete steps. When the system is synchronized, the error signal fluctuates by a unit around the zero point. The recommended synchronization method is to initiate the control of a voltage-controlled generator by using the E error signal until a unit deviation in the E error signal occurs, then turning on to use the LTE error signal to control the voltage-controlled generator.

The clock signal, provided by the voltage controlled generator 37, is used, for example, to control at least a transport processor and speed buffers. Since it is synchronized at least with the clock,

177 313 the possibility of an overflow or underflow of the speed buffer, due to clocking errors, generally does not exist.

Based on Fig. 2, audio and video synchronization can be explained. The PTSwz time stamp of the presentation is introduced into the compressed video signal associated with the specified video data. PTS tag_visas is an indicator of the relative time at which the video signal is to be played. Similarly, the compressed audio signal includes PTS time stamps_ak presentations related to the acoustic signal to be reproduced at times related to individual PTS-k time stamps. PTSa time stamps in the receiver_k and PTSwz cannot be compared directly to ensure audio and video synchronization because individual samples are determined at different times. Individual timestamp values ​​are compared with a continuous time base, which is the clock signal of the receiver, provided by the voltage-controlled generator 37. This is done by sampling the counted LCR values ​​generated by the clock 208 to obtain time stamps.

When there is data associated with the PTS time stamp, the counted LCR value is sampled. For example, the audio decompressor 212 sends a PTS time stamp_ak when an individual audio frame is output for playback. During this time, the control signal determines the state of the latch flip-flop 220 to sample the counted LCR value, the individual values ​​of which are designated by LAS for individual acoustic markers. Similarly, when a video decompressor 'provides a video frame for reproduction, it provides a PTSwz time stamp and a control pulse to determine the state of the latch flip-flop 222 to store the current counted LCR values ​​that are labeled by LVS for individual video markers.

The LAS and PTSak values ​​are fed to the input terminals of the subtractor 218 which produces the A signal_AND.p_TS according to the equation:

Δα-pts = PTSak - LAS

Whereas the values ​​of LVS and PTSwiz ^sh fed to the input terminals of subtractor 217, which produces signal A_v_pts according to the equation:

^ V-PTS ⁼ PTSwz - LVS.

The Av-ts and Δα-pts signals are fed to the input terminals of the next subtractor 219, which generates the ERRpts error signal for acoustic and video synchronization according to the equation:

ERRpts ~ ^ v-pts Δα-ts

Acoustic and video synchronization requires that the acoustic and video synchronization error be set to zero. Thus, when the difference in values ​​corresponding to the audio and video PTS time stamps is equal in time, in reference units, between the appearance of the respective PTS time stamps, the audio and video signals will be synchronized.

Two mechanisms based on the ERRptS error signal are used to control the acoustic and video synchronization, skipping and repeating data sections and changing the processing timing. Skipping fixed intervals, i.e. audio frames, speeds up the flow of audio data within a specified range relative to the video signal. Repeating or blocking without using data delays the flow of audio data in fixed intervals relative to the video signal. Skipping and repeating of acoustic frames is audible in various conditions and is therefore only used for coarse synchronization control. Even then, a short skip or repetition can be beneficial in determining acoustic and video synchronization errors. If the acoustic frames are

177 313 shorter than 40 ms, coarse adjustment by skipping and repeating may cause synchronization errors in the range of ± 20 ms, which is within the industry standards of acoustic and video synchronization. However, this synchronization will be worse if the time base of the audio signal does not match the corresponding source. When the synchronization is coarsely adjusted, the variation of the timing of the audio processing is determined to allow accurate audio and video synchronization.

ERR error signal_PTS synchronization is supplied to the filter and processing system 216. The filtering function smooths the ERR signal_PTS to minimize abnormal effects that could be caused by a disturbance signal. The processing function then examines the smoothed error signal and determines whether audio skipping and repetition should be used to cause coarse synchronization of the audio and video signals and / or whether audio frequency adjustment should be used to achieve accurate synchronization. If coarse synchronization is required, the processing system 216 provides the S / R control signal to the decompressor 212 of acoustic signals to put the decompressor into the bypass or repeat state of the decompressed current acoustic frame. Differently or in addition to coarse control, if fine control is required, the processing system 216 provides a control signal to the time base system 215 of the audio signal to control the frequency of the audio processing clock.

Figure 4 shows a flow diagram of the system. After initialization in step 400, the system controls in step 401 an acoustic decompressor as to the presence of a PTS time stamp_ak and if the PTS time stamp_ak is detected, it is read in step 403 and the local LAS clock reference is received and stored. If the PTS time stamp_akdid not appear, the system controls in step 402 a video compressor as to the presence of a PTS time stamp_{W-Z CAKE}. If the PTS time stamp_W1Z appeared, it is read in step 404 and the local LVS clock reference is received and stored. When both PTS time stamps_ak and PTS ^ were read, ERR error signal_PTSis calculated in step 405 according to the equation:

ERR ^ ts - Δ_ν._ργδ - Δ_α._ΡΤ5

The error signal value is examined at step 406 to determine if it is greater than half the duration of the audio frame. If it is greater than half the duration of the audio frame, the error signal is checked in polarity step 407. If the polarity is positive, the current audio frame is repeated in step 409. If the polarity is negative, the current audio frame is skipped in step 408. After skipping or repeating the frame, the system returns to the starting position, waiting for the next appearance of the PTS time stamp.

At step 406, if the error signal value is less than half the duration of the audio frame, an error is tested at step 410 to determine if it is greater than zero. If the error is greater than zero, it is checked in step 412 to determine if it is less than the previous error signal. If it is smaller than the previous error signal, it means that the system is trying to synchronize and the synchronization control parameters are not changed. The system returns to the starting position and waits for the next PTS timestamp. Conversely, if the error increased above the previous error signal, the timing of the audio processing is adjusted in step 414 to reduce the frequency.

In step 410, if the error is less than zero or negative, it is checked in step 411 to determine if it is greater than the previous error signal. If it is larger than the previous error signal, it also means that the system is attempting synchronization and the synchronization control parameters are not changed. Otherwise, when the current error signal is smaller than the previous error signal, the system has moved away from the synchronization state and the timing of the audio processing is increased in step 413. After the processing steps 412 and 413, the system returns to waiting for the next appearance of the PTS timestamp. The system only provides coarse adjustment by skipping or

177 313 repetition of audio frames until the error of audio and video synchronization is reduced to less than half the duration of the audio frame.

In a different embodiment, the filtered error signal is compared with a specific threshold associated with the size of the individual acoustic frames. If the error signal is smaller than the threshold value, it means that the error of the acoustic and video control in time is less than the acoustic frame, and the error signal is fed to the time base system 215 of the acoustic signal in which it is used to adjust the frequency of the processing of the acoustic signal when decompressing. Otherwise, if the error signal is greater than the threshold value, the error signal may be divided into the duration of the audio frame to determine the number of audio frames by which the audio and video signal is detached. The total part of the quotient is fed to the acoustic signal decompressor in order to put it into the state of skipping or repeating this number of acoustic frames. The error signal polarity will determine whether audio frames should be skipped or repeated. Nominally, the compressed data is placed in the buffer memory before decoding, hence skipping or repeating of audio frames is a simple way of controlled memory permission for read and write orders.

The fractional part of the quotient is fed to the time base system 215 of the audio signal, in which it is used to adjust the timing of the audio processing to fine-tune the audio and video synchronization.

The speed of producing PTS acoustic time stamps is proportional to the processing speed of the acoustic decompressor. The processing speed of the acoustic decompressor is directly proportional to the frequency of the clock signal used to control the acoustic decompressor. If the clock frequency of the acoustic decompressor is independent of the clock used to control the video decompressor and is precisely controlled, then the relative speed of the appearance of the PTS acoustic and video time stamps can be adjusted and synchronized accurately.

The decompressed acoustic signal is fed to the 227 D / A converter. The analog output signal from the 227 D / A converter is fed to the next analog processing circuit, which is not shown, through a 228 resistor. . The transistor control electrode is connected to the output of the threshold detector 225. A positive control voltage greater than the base-emitter voltage introduces the 229 transistor blocking interference into the stabilization state of the output acoustic signal from the 227 DAC on ground potential, blocking the interference of the acoustic signal.

Usually, a compressed audio signal has many components, such as the left channel and right channel signals. Figure 2 shows for simplicity only one audio output, although each audio channel includes an interference blocking circuit controlled by a common interference blocking control signal.

Audio interference blocking is used for a number of reasons, for example due to loss of audio-video synchronization. The device according to the invention blocks the interference of the audio signal on the basis of a lip synchronization error in the video relative to the sound.

as less than 20 ms, but greater than half of MPEG1, layer II of the acoustic frame, which is the Viewer, may notice errors of lip synchronization on the video relative to audio of the order of -20 ms or +40 ms. In the presented system, acoustic signal disturbances are blocked when the moss synchronization error in the audio vision exceeds about 13 ms. The threshold of 13 ms was selected 24 ms. The threshold value was chosen as slightly larger than half of the audio frame with omissions and repetitions of the audio and video synchronization frames at which the initial synchronization state in the middle of the frame at 12 ms is possible, and the threshold value of about 12 ms or less could cause it is also fed into the synchronized system , but with interference blocking. Extra value

177 A threshold of 313 equal to half of the frame results in intermittent blocking of interference, caused by slight deviations in the calculated errors of lip sync on audio with respect to voice, which were caused by the uncertainty of clocking and sampling of striking PTS time stamps.

The interference blocking control signal is generated by the threshold detector 225, which controls the audio and video synchronization error signal received from the subtractor 219. When the error signal exceeds the value corresponding to 13 ms, the interference blocking control value is generated. To exclude the generation of false blocking signals by interference or other pulse states, the subtractor system error signals 219 are filtered through the low pass filter before being fed to the threshold detector 225.

The dotted arrows coming from the threshold detector 225 and ending at the D / A 227 and acoustic signal decompressor 212 indicate various possibilities for blocking interference. For example, the interference blocking control signal may be designed to disable the output of the 227 D / A converter or the output of the decompressor 212 of acoustic signals. At any time, the shutdown function should be adapted to determine the state of the individual processing element so that the value of the signal amplitude at the output occurs in the middle of the dynamic range of the output signal. The device of Fig. 2 also shows that interference blocking or blanking of video signals can be implemented by controlling the video-analog converter 224 of video signals.

Figure 5 shows another embodiment of a interference blocking device. The S / R skip and repeat signal, produced by filter 216, is used for the primary control of interference blocking. The signal is fed to the interference suppression system through the OR 230 element. If in the system accurate control for adjusting the time base of the audio signal reaches a range covering at least half of the audio frame, the threshold detector 231 may be turned on to control the exact time base control signal of the audio signal and produce a secondary interference blocking control signal. This secondary control signal blocking interference by the OR element 230.

The apparatus of Fig. 5 shows a noise blocking system, comprising element I 226, for selectively feeding the decompressed acoustic signal from the decompressor 212 to the 227 digital-to-analog converter. Nominally, the acoustic signals are bipolar and fluctuate around zero. Element I, when in a disconnected state, provides a zero value at the output which is in the middle of the dynamic range of the signal.

Currently, satellite broadcast systems using direct decompressed acoustic and video signals are used, in which many packaged programs are multiplexed and transferred with time division by a single relay. The specified program can only contain acoustic signals, avoiding problems of lip sync on video and audio. However, unwanted audio signals can be played if clock synchronization is not working. Therefore, a further threshold detector 232 may be introduced to control the error signal generated by the clock controller 39, shown in Fig. 3. The threshold detector 232 generates an interference blocking control signal when the error signal generated by the controller represents a frequency deviation from a steady state, e.g. 0.2 . This interference blocking control signal is applied to the OR element 230 to cause interference blocking of the acoustic signals until the clock of the transmitting device is substantially synchronized with the clock of the receiving device. Similarly, a detection device may be provided to measure the frequency deviation of the time base system 215 of the audio signal to produce the next interference blocking signal that may be subjected to a logic function in the OR element 230.

17*7313

177 313 ο

τ ·

CM

<img file="PL177313B1_D0001.tif" />

177 313 system clock

200

<img file="PL177313B1_D0002.tif" />

INVERSION TRANSPORT PROCESSOR m 313

<img file="PL177313B1_D0003.tif" />

/413

FIG. 4

177 313 φ TO DISPLAY, (= o | DAC -ffi, ....., »OR ntsc encoder

21¾^

226

2Zt —Λ_

<img file="PL177313B1_D0004.tif" />

<img file="PL177313B1_D0005.tif" />

FIG. 5

232

177 313

<img file="PL177313B1_D0006.tif" />

UP Department of Publications. Circulation of 70 copies Price PLN 4.00