Improvements relating to video data transmission

Image data transmission system

The invention relates to an image data transmission system and is particularly useful in image data transmission using conditional replenishment.

A television picture usually consists of about 200,000 pixels, and if the data is transmitted digitally, this requires 8 bits in order to transmit the brightness information of each element. If the frame rate is 25 frames per second for a moving image, the digital transmission described above would require a transmission channel capacity of 40 megabits per second. This need can be greatly reduced by taking advantage of the great similarity between one image and the next image, due to the fact that only a small part of the image moves at any one time. Conditional replenishment involves only transferring the changes that have taken place, e.g., 16 non-linearly scattered quantized levels from image to image, with the achieved data reduction generally greater than the additional information to be transmitted to properly identify certain areas of the image to which the changes belong. Areas are identified by the line number and pixel address on this line. Although 9 bits are needed to identify the line number, savings are achieved by using only 3 bits and reporting the number as modulo 8. The field synchronization code is sent to identify the first line in which the field occurs, and this is the case with all line numbers, whether or not they have changed. The rate of data transfer required using conditional replenishment varies considerably because it depends on how much of the image is moving at any given time, which is why it is much more common to use a constant data rate. To overcome this difficulty, buffer memories are used at both ends of the transport channel, so it is important that these buffer memories are neither empty nor overflow. Using this method, it has proven possible to transfer the moving image satisfactorily along a 2 megabit / second channel.

If B (t) is the number of bits stored in the encoder buffer (encoder stage) at the channel transmission end at time t and B ^ (t) is the number of bits stored in the decoder buffer at the channel reception end at time t, it can be shown that

B (t - At) + B (t) = V .At E DR where V is the channel transmission capacity (bits / s) and is assumed to be constant, and At is the time delay for data to enter the encoder buffer memory to leave the decoder buffer memory Usually selected At: lie is the optimum value such that the total amount of data stored in the buffers = half of the total available buffer memory, and the decoder buffer control is based on the state of the encoder buffer for the time At earlier. This means that decoder buffer overflow and overflow can be anticipated and the required actions can be taken, either temporarily stopping decoding or deleting data to avoid corruption of the reproduced image due to suddenly unavailable or lost data.

Since the replenishment information is derived from an image that is browsed with a conventional television raster and the augmented image is updated line by line with the supplementary information, it follows that reading data from the decoder buffer must be followed by writing data to the encoder buffer. This places an additional requirement on reading from the decoder buffer and is handled by shifting the encoder buffer state B (t) to the decoder so that the E decoder buffer B (t) can be predicted based on the above equation.

In addition, four methods are used to reduce the speed of very large image data, which can be used if large image movement occurs. These four methods are:

1. The sensitivity of the motion detector with the encoder is reduced as the filling level of the encoder buffer increases, thereby reducing the number of areas of detected image changes.

2. A time-based field subsampling is performed. In this method, alternating data fields are removed and the decoder interpolates the missing data from two adjacent transmitted fields.

3. Element-based subsampling is performed so that certain moving area elements are removed by an encoder and interpolated by a decoder.

4. If the above methods are unable to prevent an imminent encoder buffer overflow, all movements of the transferred image are ignored until the encoder buffer state has returned to a safe value.

Reducing the sensitivity of the motion detector in the encoder results in small contrast changes being ignored, leading to a so-called dirty window. This can be allowed for a short time. Field subsampling results in motion vibration throughout the field. If the decoder has an interpolator, this can be used to reduce vibration, assuming that all movement occurs at the same speed. Subsampling of elements, which deals with the transfer of changes involving alternating elements, for example, allows the amount of data to be transferred to be reduced to a minimum in the event of a large amount of movement without using the image freeze method described in step 4 above. The disadvantage of subsampling an element is that part of the interpolation of image details can be undesirable because the interpolation assumes that the brightness of the interpolated point is the average of two adjacent points on the same line. This is clearly not always the case when the interpolated point is on the vertical edge of the dark and light area.

It is an object of the present invention to avoid this disadvantage of under-sampling of elements.

According to the present invention, an image communication system of the type in which the image is represented by digital values ​​corresponding to the brightness of the elements along parallel lines across the image and only the elements whose brightness changes from one image to another is transmitted from the transmitter to the receiver to update the digital representation of the stored image is used. reduce the speed of data transfer when many elements change in brightness, a particular isolated element is not shifted but is calculated at the receiver based on the change in brightness of adjacent elements, where the selection of each particular isolated element depends on no major differences in brightness of adjacent or nearly adjacent elements in the previous row and close to a particular isolated element.

The magnitude of the differences in brightness changes between two adjacent or nearly adjacent elements in a row depends on whether the image has a vertical moving edge of the image. A large difference indicates the presence of such an edge, while a small difference indicates that there is no such edge. Due to the effect of element subsampling, which does not shift the brightness changes of certain isolated elements but calculates them from neighboring elements, interpolation can reduce high-contrast vertical walls. This is avoided according to the invention because subsampling is not used if a vertical detail is present.

Element subsampling may involve shifting brightness changes only in alternating elements, achieving the maximum data rate reduction that can be achieved because all omitted elements, i.e., those whose brightness changes are not transferred, must have a shifted element on each side so that subsampling does not cause significant inaccuracy. Preferably, the omitted elements are arranged in diagonal rather than vertical rows.

The row or field synchronization codewords may carry information that the image information has been processed by element subsampling, and the receiver may be adapted to examine the received information to determine if moving vertical edges have been detected by the encoder producing transmission information and whether the received information includes a brightness change. Therefore, there is no need for a separate indication that the normally omitted element has actually been moved, as this information is included in the image data.

Unfortunately, although the system described above works well if there are no errors in image transfer, only the brightness change error of a single element is needed for the receiver to misjudge whether or not an element on the next line is omitted, which in turn leads to corresponding errors on element changes on subsequent lines. . To overcome this problem, the codes commonly used to transmit brightness changes to omitted elements may include an indication that the element is one that is generally omitted. One way to accomplish this is to reduce the number of quantized levels used in element subsampling compared to normal transfer, use codes that generally represent a limited number of quantized levels of transferred elements, and use the remaining codes for generally omitted elements that are transferred. Preferably, the codes used to transfer generally shifted elements and generally omitted elements to be shifted should be different enough so that the transmission error does not affect code corruption from one type of element to another. Here, too, the receiver may examine the elements in the preceding line to check for the presumed transfer of generally omitted elements and to search for codes for such elements.

In order that the invention may be fully understood and properly applied, it will now be explained with reference to the accompanying drawings, in which Figure 1 is a diagram used to explain conditional complementation and the creation of signals representing changes in various pixels; Figure 2 shows an image structure used in communication. For 8-level and 4-level quantization, Figure 4 is a diagram used to explain the criteria, which determine whether information relating to the change of a particular pixel is transmitted, Fig. 5 is a block diagram of an image data encoder, and Fig. 6 is a block diagram of an image data decoder.

In Fig. 1, the rectangle 1 represents the image area of ​​the television image and 2 represents a group of consecutive rows in Fig. 8. The eight lines are numbered according to modulo 8, the number being represented by a group of three bits from 000 to III. Along each line is an address assigned to the pixel, which depends on its position on the line. Each row has 256 elements. A group of 3 elements on line 011 is indicated and it is assumed that these elements have changed from the previous figure. The elements that make up group 3 are identified by the line number 011 and the group start address. Once this information is available, changes in the elements are presented sequentially using a variable-length Huffman code. At the end of the group, there is a code indicating the end of the group.

If the transmitter and receiver have image memories that store the brightness of the elements in digital form, which elements are browsed in sync with other image memories in normal television practice, information related to changes in element brightness can be transferred from one image memory to another, the receiving image memory reproducing transmitter information. Of course, this information needs a row and field synchronization codeword 76902 and to ensure that the browsing of the two images is synchronized. The line sync codeword also contains a three-bit line number.

There are two types of field synchronization codewords that recognize even and odd fields in normal interleaved browsing.

The data to be transmitted from the transmitter to the receiver is included in the standard CCITT G732 image structure. As shown in Figure 2, this structure consists of 32 sets of 8-bit time slots that form an image. 16 such images form a partial image. The odd image time slot 0 contains an image synchronizing word for the image structure in the form of a Barker queue, together with a bit that forms other odd time slots 0 with individual bits for the Barker queue sub-image, allowing the image structure timing to be identified by the receiver. Alarm and control information is transmitted in even time slots 0. Audio information and additional synchronization words occur in time slots 1 and 2. The remaining 29 time slots in each image are filled with image data containing row and field synchronization codewords representing changes to be transferred from the transmitter's image memory to the receiver's image memory. Because the number of images transferred at any one time also varies, the amount of data to be transferred from the transmitter's image memory to the receiver's image memory must be transmitted, but this information must be transmitted to keep the data in the receiver's image memory substantially the same as the transmitter's image memory. That is, the amount of data to be transferred during one field scan varies with the amount of motion of the image, and therefore the image information is asynchronous with the image structure of the G732-shaped signal in which it is transmitted.

The brightness of the pixel is represented by an 8-bit binary number in the incoming PCM data, so that the change in the brightness of the element can be in the range of -255 to +255. A change representation implemented on this principle would require 9 bits, and to reduce this, the changes are quantized to a 16-level. Figure 3 shows an 8-bit quantizer in which the positive input values ​​correspond to the positive output values. With a corresponding curve of negative values, this gives a 16-level quantizer. In one application, an 8-level quantizer is better than a 16-level quantizer when normally omitted elements in the subsampling field are shifted due to the presence of moving vertical edges. The dashed line in Figure 3 represents the 4-level quantization of the positive output value corresponding to the positive input value. If the dashed line covers the line, no dashed line is drawn. This is especially noticeable at very low levels, where the whole first step is common. As with 16-level quantization, for the 8-level quantization, the negative values ​​are exactly the same as the positive values ​​but are inverted.

When sub-sampling of elements is used, the row synchronization signal contains a code indicating this. In general, subsampling of elements means omitting codes that represent differences in brightness of alternating elements, assuming that each omitted element is surrounded by an element shifted on both sides. If this is not the case, an element that would not otherwise have been moved is moved, for example, at the end of the group. If the change in the image extends over several lines, the elements to be omitted are usually selected diagonally or in the form of a diagonal cross. Element subsampling causes a reduction in data rate and is used if the encoder buffer is exceeded above a certain threshold and methods 1 and 2, i.e., motion detector sensitivity reduction or field subsampling, are not sufficient to solve the problem. Reducing this data rate reduces the spatial response in the areas where it is used, and the present invention seeks to avoid the use of this reduction in areas with high vertical detail, resulting in only slight image distortion.

It is suggested that the decoder should be able to reconstruct for itself information about how the information about the element is encoded. In this case, this means that the decoder must be able to determine which elements have been subsampled and which have been transferred, because it is not desirable to transmit additional information.

An adaptive process is proposed, which depends on the information already encoded in the previous line as a criterion according to which an element that would normally be omitted in the sub-sampling is actually transferred. If the encoder operates on this principle, the decoder can operate on the same principle and infer for itself if a normally omitted element has been moved or not.

Referring to Figure 4, which shows the nine elements of three consecutive rows in the figure, the element X in the middle is the one from which it must be deduced whether or not it is to be moved. Assume that in the element subsampling mode, elements A and E would be movable and X would be omitted. The test for determining whether or not element X should be moved finds, for example, the magnitudes of the differences in brightness of elements B and D, which are in line n-1, which precedes line n, where X is located. If the difference exceeds a threshold that has been found experimentally to give good results, element X is moved because it is likely that there will be a substantially vertical edge at the nine elements in the image. On the other hand, if the difference is less than the threshold value, the element X is omitted and the value of its brightness difference is verified at the receiver by interpolation.

It would be desirable not to transfer elements B and D, in which case the values ​​obtained by interpolation between the transferred values ​​could be used, or elements outside B and D could be used instead.

Unfortunately, the above process collapses if there are transmission errors or malfunctions. For example, if in the encoder the brightness changes of elements B and D differ by an amount exceeding the threshold but the values ​​of the changes received by the receiver do not, the encoder correctly transmits the change of element X but the receiver interprets this change as belonging to element E because the change of element X is not filled in the receiver. This error causes significant distortion in the image, which may continue until the end of that field.

To overcome this difficulty, when the brightness change of an element that would normally be omitted is shifted, the coding for brightness changes in all those elements and also in those elements on the same row adjacent to the touched elements is subjected to 8-level quantization instead of 16-level quantization. and in addition, elements that would normally be omitted using codes are transferred, which differ from those used for normally transferred elements. 16-level quantization requires 16 different codes that can be numbered 1-16. When normally omitted elements are moved, codes 1-8 are used for normally moved elements and codes 9-16 are used for normally omitted elements, e.g.

Preferably, the code selected as 8-level quantization levels for normally shifted elements and normally omitted is such that a change in quantization level numbers from 16 to 8 would be obvious to the decoder, for example using Huffman codes, and the probability of code change for another type of element is maintained. as the code of the second type of element in the minimum.

Figure 5 shows an example of a block diagram of an encoder that can be used in the present invention. The 8-bit pulse code modulated (PCM) image data is applied to the encoder via a terminal 100 connected to the input of the separation pulse code modulated (DPCM) encoder 102 via a space-time filter 101. Filter 101 processes the image with space-time non-linear filtering and noise reduction to improve the performance of the next motion detector. The encoder 102 receives information as a second input image representing the previous image from the image memory 103, and the motion detector 104 provides the encoder 102 with information that allows the encoder 102 to detect those elements in the incoming image information that have undergone a change exceeding a threshold determined by the sensitivity control signal. has been exported to terminal 105. The output of encoder 102 is applied to port 106, which is controlled by a subsampling control circuit 108, which is responsible for the signal applied to terminal 107 and indicates whether or not the element is subsampled. If element subsampling is not used, the output of encoder 102 is applied to a 4-bit quantizer 109 and codes representing different levels are passed to multiplexer 110, where they are combined with image signals input to terminal 111 and a signal indicating whether or not element subsampling has been used. to provide an output signal at terminal 112 for use in the encoder buffer. In the case of element subsampling, alternating codes are applied at output of encoder 102 through port 106 to 4-bit quantizer 109 and then to terminal 112 as before, with control circuit 108 controlling port 106. Detector 113 is coupled to respond to port 106 input by providing output if image information indicates moving vertical the presence of an edge. The output of detector 113 is applied to control circuit 108 and port 116. As long as no such edges are identified, the subsampled signals are applied to a 4-bit quantizer 109 as described above. If a vertical edge is detected, the output of gate 106 is connected to an input of gate 116, which is controlled by the output of detector 113, so that normally shifted elements are applied to 3-bit quantizer 114 and normally omitted elements are applied to 3-bit quantizer 115. Quantifiers 114 and 115 are also connected to multiplexers 114 and 115. .

Examining Figure 5, it will be apparent how the subsampled signals described above are produced.

Fig. 6 shows an example of a decoder block diagram for decoding signals produced by, for example, the encoder of Fig. 5. The received signals arrive via the terminal 200 and are stored in the update memory 201 and also fed to a signal decoder 212, which detects the presence of subsampling. If subsampling is present, an indication is sent to subsampling control circuit 203, which controls port 204 so that information is transferred from memory 201 either via 4-bit dequantizer 205 to image memory 207 input circuit 206, or one of two 3-bit quantizers 208 and 209, depending on port 210 control. to which the output of port 204 is connected. The detector circuit 211 connected to the outputs of the update memory 201 and the image memory 207 searches for signals indicating that a moving vertical edge is present as mentioned above, and if such an edge is detected, produces a signal to control the port 210 and also provides a signal to the control circuit 203. Interpolator 212 is adapted to interpolate between the output of dequantizer 205 and the output signal of image memory 207 to generate a brightness change signal of a normally omitted element. Address circuits 213 perform image memory 207 browsing to provide image output to terminal 214 and also provide an address to update memory 201 so that update information regarding the element currently being browsed in image memory 207 is read to update image memory 207.

If there is no subsampling of the element in Fig. 6, the output of the update memory 201 is passed through the 4-bit dequantizer 205 to update the information of the image memory 207. If element subsampling is used, the output of the control circuit 203 activates the interpolator 212 so that update information of the omitted elements is produced for the elements transferred therein. If the detector 211 detects a moving vertical edge using the method described above, it provides signals to the subsampling control circuits 203 and gate 210 such that the update information is alternately applied through quantizers 208 and 209 to the input of image memory 207.