Moving picture cluster processing system

The present invention relates to a moving image cluster processing system, and is suitable for application when performing moving image processing using a plurality of personal computers, workstations, and other low-priced CPU boards (hereinafter referred to as personal computers).

In recent years, with the increase in performance and price of personal computers, a plurality of personal computers are interconnected via a network to share the processing (this is called cluster processing), so that an expensive computer has been required in the past. Complex processing can be performed at low cost. Applications of such cluster processing include cryptanalysis, genome analysis, and computer graphics rendering processing (see, for example, Patent Document 1).

FIG. 20 shows a configuration of a general cluster processing system using a personal computer, in which a host node personal computer 101 and a plurality of computing node personal computers 102 (102A to 102N) are connected via a network 103 to form a cluster processing system 100. Consists of. The host node personal computer 101 distributes the processing to the managed computing node personal computers 102A to 102N.

FIG. 21 shows the application structure in the cluster processing system 100, and the cluster application program 110 of the host node personal computer 101 controls the distributed job 113 on each computer node personal computer 102 via the cluster management program 111 and the communication layer 112.

The cluster management program 111 receives an instruction from the cluster application program 110 and executes data transmission / reception to another computer, start / end of the distributed job 113, and the like via the communication layer 112.<patcit num="1"><text>Japanese Patent Application Laid-Open No. 2000-30047</text></patcit>

<p> Here, it is also conceivable to apply the above-mentioned cluster processing to the compression coding and decompression / decoding processing (moving image compression / decompression processing) of the moving image. In such moving image compression / decompression processing, there are cases where real-time processing such as processing and transmitting input data in real time is required, and non-real-time processing such as storing the processed data in a storage medium such as a hard disk. It can be divided into two.</p><p> Since there is no limit to the processing time in non-real-time processing, simply increasing the number of personal computers (hereinafter referred to as nodes) that perform cluster processing can execute high-load compression / decompression processing at higher speed. Can be done.</p><p> On the other hand, in real-time processing, it is required that the processing is completed within a certain time, and if the processing is not completed within the certain time, a problem such as missing processing occurs. Therefore, in order to guarantee the real-time performance in cluster processing, measures are taken such as evaluating the processing load for each node in advance and allocating the processing according to each processing load to each node.</p><p> On the other hand, in each node of the cluster, the program that manages the cluster processing generally runs on a basic program (operating system) such as Linux, Unix (registered trademark), and Windows (registered trademark). ..</p><p> Here, the above-mentioned operating system does not have a mechanism for guaranteeing the end of processing within a certain period of time for an application program running under its control (hereinafter, this is referred to as a real-time OS). Further, in most cases, a plurality of programs (tasks) are executed at the same time on each node, and it does not guarantee the allocation of a predetermined computer resource to a specific task. Some operating systems, such as Unix (registered trademark), have a function to specify the priority of tasks, but this also does not guarantee that the predetermined processing will be completed within the predetermined time.</p><p> Therefore, when real-time moving image compression / decompression processing is performed using a cluster configured by a personal computer, real-time performance may be impaired due to fluctuations in the processing time of each node, which may cause a delay in data processing.</p><p> Further, when performing real-time moving image compression / decompression processing using a cluster configured by a personal computer as described above, if the number of nodes is increased, a delay in data processing at each node may occur.</p><p> In this way, when performing real-time moving image compression / decompression processing using a cluster composed of personal computers, data processing delays may occur at each node, which may cause defects in the coded data and decoded images. There was a problem of having sex.</p><p> The present invention has been made in consideration of the above points, and an object of the present invention is to propose a moving image cluster processing system capable of reliably correcting data or image loss due to node processing delay.</p>

<p> In order to solve such a problem, in the present invention, in the moving image cluster processing system including a plurality of information processing devices and a management device for distributing the moving image processing to each of the plurality of information processing devices, a plurality of information processing devices. A moving image processing instruction means for executing moving image processing for each picture, a processing result acquisition means for acquiring the processing result of moving image processing for each picture returned from each of a plurality of information processing devices, and an information processing device. When the return of the processing result from is delayed, the variation evaluation value indicating the degree of change in the pattern between the delayed picture including the delayed processing result and the other reference picture is compared with a predetermined threshold value, and the variation evaluation value is the threshold value. If it is less than, only the delayed processing result is replaced with the corresponding processing result of the reference picture, and if the variation evaluation value is equal to or more than the threshold value, the processing result of the entire delayed picture is replaced with the processing result of the reference picture. Was provided in the management device.</p>

<p> By deciding whether to replace only the delayed processing result or the processing result of the entire delayed picture according to the variation evaluation value between the delayed picture and the reference picture, the data due to the processing delay and the data due to the processing delay can be determined. Image defects can be reliably corrected according to the degree of change in the picture pattern.</p>

Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings.

(1) First Embodiment (1-1) Overall Configuration of Video Cluster Processing System In Fig. 1, 1 indicates the video cluster processing system of the first embodiment as a whole, and the host as a management device. The node personal computer 2 and the four computing node personal computers 3A to 3D as information processing devices are connected via the network 4.

This moving image cluster processing system 1 is adapted to perform real-time compression coding processing of an analog video signal supplied from the outside by a JPEG (Joint Photographic cording experts Group) 2000 method by cluster processing.

In the host node personal computer 2, the memory 12, the display unit 13, the hard disk drive 14, the network interface 15, and the video capture circuit 16 are connected to the CPU 11 that controls the host node personal computer 2 via the bus 17. ..

The hard disk drive 14 stores, for example, an operating system such as Windows (registered trademark) and various application programs such as a moving image cluster processing program. The CPU 11 reads the operating system from the hard disk drive 14 in response to the startup of the host node personal computer 2, expands it into the memory 12, and executes it.

Then, the CPU 11 as the moving image processing instruction means executes the moving image cluster processing program under the execution environment of the operating system, and performs distributed processing of the moving image compression coding and decompression decoding processing for the calculation node personal computers 3A to 3D. Let me.

That is, at the time of compression coding of the moving image, the video capture circuit 16 of the host node personal computer 2 digitally converts the analog video signal S1 supplied from the outside to generate the digital video signal D1 and supplies this to the CPU 11. The CPU 11 distributes the compression coding of the digital video signal D1 to the computing node personal computers 3A to 3D according to the moving image cluster processing program.

Here, in the JPEG2000 method, compression coding of a digital video signal is completed for each frame (or field) as a picture, and each frame is divided into a plurality of image blocks (hereinafter, this is referred to as tiles). Then, each tile can be individually compressed and encoded.

That is, as shown in FIG. 2, the CPU 11 divides each frame of the digital video signal S1 consisting of, for example, 720 × 480 pixels into four according to the number of calculation node personal computers 3A to 3D, and tiles 1 to tiles consisting of 360 × 240 pixels. Generate 4. Then, the CPU 11 sequentially supplies the data of the tiles 1 to 4 after the division to the corresponding calculation node personal computers 3A to 3D as the divided digital video data D2A to D2D (Fig. 3 (A)).

In the calculation node personal computers 3A to 3D, the memory 22, the display unit 23, the hard disk drive 24, and the network interface 25 are connected to the CPU 21 that controls the calculation node personal computer 3 via the bus 27.

Various application programs such as an operating system and a cluster application program are stored in the hard disk drive 24. The CPU 21 reads the operating system from the hard disk drive 24 in response to the startup of the calculation node personal computer 3, expands it into the memory 22, and executes it.

Then, the CPU 21 compresses and encodes the divided digital video data D2 (D2A to D2D) supplied from the host node personal computer 2 by the JPEG2000 method by executing the moving image cluster processing program under the execution environment of the operating system, and divides the code. Generates data D3 (D3A to D3D) and returns it to the host node computer 2 (Fig. 3 (B)).

The CPU 11 as a processing result acquisition means sequentially synthesizes the divided coded data D3A to D3D returned from each calculation node personal computer 3A to 3D in frame units, and a coded image consisting of a series of compressed coded frame data. Generate signal D4 and store it in hard disk drive 14 or send it to an external destination via network 4 (Fig. 3 (C)).

Thus, the moving image cluster processing system 1 performs real-time compression coding of the analog video signal S1 by cluster processing.

At the time of decompression and decoding of the moving image, the CPU 11 of the host node personal computer 2 follows the cluster application program and either the coded video signal D4 read from the hard disk drive 14 or the coded video signal supplied from the outside via the network 4. The decompression and decoding of D4 is distributed to the calculation node personal computers 3A to 3D.

That is, the CPU 11 divides the coded data for each frame of the coded video signal D4 into four parts corresponding to tiles 1 to 4 shown in FIG. 2 to generate divided coded data D3A to D3D, and corresponds to each of them. Compute node Supply to personal computers 3A to 3D (Fig. 3 (C)).

CPU21 of the calculation node personal computer 3A to 3D decompresses and decodes the divided coding data D3A to D3D supplied from the host node personal computer 2 according to the moving image cluster processing program, and divides digitally consisting of continuous images of tiles 1 to 4. Video data D2A to D2D are generated and returned to the host node computer 2 (Fig. 3 (D)).

The CPU 11 as a processing result acquisition means sequentially synthesizes the divided digital video data D2A to D2D returned from each calculation node personal computer 3A to 3D for each frame to generate a digital video signal D1, and displays this on the display unit 13. Or send it to an external destination via network 4 (Fig. 3 (E)).

Thus, the moving image cluster processing system 1 performs real-time decompression and decoding of the coded video signal D4 by cluster processing.

(1-2) Error Concealment at the Time of Compression Coding and Decompression Decoding In addition to this configuration, the host node personal computer 2 sets the coding processing time or decoding processing time for each tile in each of the calculation node personal computers 3A to 3D. I am constantly monitoring.

Then, when either the coding processing time or the decoding processing time exceeds a predetermined reference processing time (that is, a processing delay occurs in the calculation node), the host node personal computer 2 has a frame including a tile in which the processing delay has occurred. By interpolating using the frames before or after that, the loss of the decoded image due to the processing delay of the calculation node is prevented. Such interpolation processing is called error concealment.

First, the cluster coding processing procedure in the host node personal computer 2 for performing the moving image compression coding accompanied by the error concealment described above will be described in detail with reference to the flowchart shown in FIG.

The CPU 11 of the host node personal computer 2 enters from the start step of the routine RT1 and moves to the next step SP1. In step SP1, the CPU 11 as the moving image processing instruction means transmits various processing parameters such as the coding target rate to each of the calculation node personal computers 3A to 3D, and moves to the next step SP2.

In step SP2, the CPU 11 stores the image data for one frame of the digital video signal D1 in the memory 12, and moves to the next step SP3.

In step SP3, the CPU 11 divides the image data for one frame captured in the memory 12 corresponding to the four tiles to generate the divided digital video data D2A to D2D, and moves to the next step SP4.

In step SP4, CPU 11 calculates the difference value of the pixel value of each RGB pixel between each tile of the current frame and the tile of the corresponding previous frame (this is called a comparison target frame). ..

Then, the CPU 11 averages the absolute values of the calculated difference values for each tile, and stores each average value in the memory 12 as image variation evaluation values Diff1 to Diff4 indicating the degree of change in the pattern between the frames of tiles 1 to 4. Then move on to the next step SP5.

FIG. 5 shows an example of the transition of the image variation evaluation value with the change of the frame. For example, in frame 3, since a triangular object appears over tiles 1 and 3, the image variation evaluation value Diff1 of tile 1 of the frame 3 and the image variation evaluation value Diff3 of tile 3 are increasing. Further, since the frame 4 having a completely different pattern is sandwiched between the frame 3 and the frame 5, the image variation evaluation values Diff1 to Diff4 of the frame 4 and the frame 5 are both large values.

In step SP5, the CPU 11 as the moving image processing instruction means supplies the corresponding tiles 1 to 4 to each calculation node personal computer 3A to 3D, and instructs the start of compression coding for the tile, and then next Move to step SP6.

In step SP6, the CPU 11 as the processing result acquisition means starts listening for the divided coded data D3A to D3D returned from the calculation node personal computers 3A to 3D. Then, when the reference processing time elapses from the start of the standby, the CPU 11 moves to the next step SP7.

In step SP7, the CPU 11 determines whether or not there is a delay in the compression coding based on the return status of the divided coding data D3A to D3D returned from the calculation node personal computers 3A to 3D.

When all the divided coded data D3A to D3D are returned in step SP7, this means that there is no processing delay in all the calculation node personal computers 3A to 3D and no error concealment is required. At this time, the CPU 11 moves to step SP8, synthesizes all the returned divided coded data D3A to D3D (tiles) to generate a coded video signal D4, and returns to step SP2.

On the other hand, in step SP7, if any of the divided coded data D3A to D3D has not been returned yet, this causes a processing delay in any of the calculation node personal computers 3A to 3D, and error concealment is required. At this time, CPU 11 moves to step SP9.

In step SP9, the CPU 11 as the processing result replacement means compares the image variation evaluation value Diff of the tile in which the processing delay has occurred with the predetermined variation evaluation threshold Dlim, and based on the comparison result, an error concealment method (1). Select whether to replace part tiles or all tiles).

That is, in step SP9, when the image variation evaluation value Diff of the tile in which the processing delay has occurred is less than the variation evaluation threshold Dlim, this means that this is between the tile in which the processing delay has occurred and the tile in the corresponding frame to be compared. Even if the divided coded data D3A to D3D are combined using the tiles of the comparison target frame instead of the tiles in which the processing delay occurs due to the small difference in the pattern, it shows that there is little discomfort in the decoded image. At this time, CPU 11 moves to step SP10.

In step SP10, the CPU 11 as the processing result replacement means replaces the divided coded data D3n of the tile in which the processing delay has occurred with the divided coded data D3n of the tile of the comparison target frame, and then replaces the divided coded data D3A to D3D. The coded video signal D4 is generated by synthesizing, and the process returns to step SP2.

On the other hand, in step SP9, when the image variation evaluation value Diff of the tile in which the processing delay has occurred is equal to or greater than the variation evaluation threshold Dlim, this means that the pattern between the tile in which the processing delay has occurred and the tile in the comparison target frame. If the divided coded data D3A to D3D are combined using the tiles of the comparison target frame instead of the tiles in which the processing delay occurs due to the large difference between the two, the discomfort in the decoded image becomes large. CPU 11 moves to step SP11.

In step SP11, the CPU 11 as the processing result replacement means replaces the divided coded data D3A to D3D of all tiles including the tile in which the processing delay has occurred with the divided coded data D3A to D3D of the tiles of the comparison target frame. It is post-synthesized to generate a coded video signal D4, and the process returns to step SP2.

6 and 7 show examples of error concealment during coding. Here, the fluctuation evaluation threshold Dlim = 10 is set.

For example, as shown in FIG. 6 (B), when tile 1 (image variation evaluation value Diff1 = 4) of frame 3 is delayed in processing, the image variation evaluation value Diff1 of the tile 1 is less than the variation evaluation threshold Dlim. , CPU11 replaces only the tile 1 in which the processing delay occurs with the tile 1 of the frame 2 which is the comparison target frame. In this case, the tip of the triangle on the tile 1 is erased by replacement, but since the area is relatively small, there is little discomfort in the decoded image.

Further, as shown in FIG. 6 (C), when the tile 2 of the frame 3 (image variation evaluation value Diff1 = 0) is delayed in processing, the image variation evaluation value Diff1 of the tile 2 is less than the variation evaluation threshold Dlim. , CPU11 replaces only tile 2 with processing delay with tile 2 of frame 2.

On the other hand, as shown in FIG. 6 (D), when tile 3 (image variation evaluation value Diff1 = 30) of frame 3 is delayed in processing, the image variation evaluation value Diff1 of the tile 3 is equal to or higher than the variation evaluation threshold Dlim. Therefore, CPU 11 replaces all tiles with the tile of frame 2 which is the comparison target frame. In this case, two frames having the same pattern are consecutive, but the feeling of discomfort in the decoded image is less than in the case of replacing the tile 3.

Further, as shown in FIG. 7, when any of the tiles in frame 5 (FIG. 5 (E)) is delayed in processing, the image variation evaluation values Diff1 to Diff4 of each tile 1 to 4 of the frame 5 change. Since the evaluation threshold is Dlim or higher, CPU 11 replaces all tiles with tiles of frame 4, which is the frame to be compared.

Next, the cluster decoding processing procedure in the host node personal computer 2 for performing moving image decompression decoding with error concealment will be described in detail with reference to the flowchart shown in FIG.

The CPU 11 of the host node personal computer 2 enters from the start step of the routine RT2 and moves to the next step SP21. In step SP21, the CPU 11 as the moving image processing instruction means transmits various processing parameters such as the decoding level to each calculation node personal computer 3A to 3D, and moves to the next step SP22.

In step SP22, the CPU 11 stores the coded data for one frame of the coded video signal D4 in the memory 12, and moves to the next step SP23.

In step SP23, the CPU 11 divides the coded data for one frame taken into the memory 12 corresponding to the four tiles to generate the divided coded data D3A to D3D, and moves to the next step SP24.

In step SP24, the CPU 11 as the moving image processing instruction means supplies the corresponding divided coded data D3A to D3D to each calculation node personal computer 3A to 3D, and moves to the next step SP25.

In step SP25, the CPU 11 as the processing result acquisition means starts listening to the divided digital video data D2A to D2D returned from the computer nodes 3A to 3D of each calculation node. Then, when the reference processing time elapses from the start of the standby, the CPU 11 moves to the next step SP26.

In step SP26, the CPU 11 as the processing result replacement means determines whether or not there is a delay in decompression / decoding based on the return status of the divided digital video data D2A to D2D returned from each calculation node personal computer 3A to 3D. To do.

When all the divided digital video data D2A to D2D are returned in step SP26, this means that there is no processing delay in all the calculation node personal computers 3A to 3D and no error concealment is required. At this time, the CPU 11 moves to step SP26, synthesizes using all the returned divided digital video data D2A to D2D, and returns to step SP22.

On the other hand, in step SP26, if any of the divided digital video data D2A to D2D has not been returned yet, this causes a processing delay in any of the calculation node personal computers 3A to 3D, and error concealment is required. At this time, CPU 11 moves to step SP28.

Here, in the error concealment at the time of compression coding described above, the error concealment method is selected based on the image variation evaluation value Diff of the tile in which the coding processing delay has occurred. In the error concealment at the time of decoding, since the tile itself in which the decoding processing delay has occurred is not decoded, the image variation evaluation value Diff of the tile in which the processing delay has occurred cannot be obtained.

Therefore, in this case, the error concealment method is selected based on the average value of the image variation evaluation value Diff of the other tiles (tiles whose decoding is completed within the reference processing time) in which the processing delay does not occur.

That is, in step SP28, the CPU 11 as the processing result replacement means determines the image variation evaluation value Diff between each tile of the current frame for which decoding has been completed and the tile of the corresponding previous frame (comparison target frame). Calculate the average value of (this is called the average fluctuation evaluation value Dave), and move to the next step SP29.

In step SP29, CPU 11 compares the calculated average fluctuation evaluation value Dave with the fluctuation evaluation threshold Dlim.

In step SP29, when the average fluctuation evaluation value Dave is less than the fluctuation evaluation threshold Dlim, this means that there is little difference in the pattern between the tile in which the processing delay did not occur and the tile in the corresponding comparison frame. That is, it means that there is little difference in the pattern in the entire frame.

Therefore, even if only the tile in which the processing delay occurs is replaced with the tile of the comparison target frame, it is assumed that there is little discomfort in the decoded video. At this time, the CPU 11 as the processing result replacement means moves to step SP30. After replacing the divided digital video data D2n of the tile in which the processing delay has occurred with the divided digital video data D2n of the tile of the comparison target frame, the divided digital video data D2A to D2D are combined to generate the digital video signal D1 and step. Return to SP22.

On the other hand, in step SP29, when the average variation evaluation value Dave is equal to or greater than the variation evaluation threshold Dlim, this means that the pattern difference between the tile in which the processing delay did not occur and the tile in the corresponding comparison frame. Is large, that is, the difference in the pattern in the entire frame is large.

Therefore, if only the tiles in which the processing delay has occurred are replaced with the tiles in the frame to be compared, it is assumed that the uncomfortable feeling in the decoded video will increase. At this time, the CPU 11 as the processing result replacement means moves to step SP30. After replacing the divided digital video data D2A to D2D of all tiles including the tile in which the processing delay has occurred with the divided digital video data D2A to D2D of the frame to be compared, they are combined to generate the digital video signal D1 and return to step SP22. ..

9 and 10 show an example of error concealment during decoding. Here, the fluctuation evaluation threshold Dlim = 10 is set.

For example, as shown in Fig. 9 (B), when tile 1 of frame 3 is delayed in processing, the average fluctuation evaluation value Dave of tile 2, tile 3 and tile 4 in time for processing is (0 + 30 + 0) / 3 Since = 10 and the average fluctuation evaluation value Dave is equal to or greater than the fluctuation evaluation threshold Dlim, CPU 11 replaces all tiles with the tile of frame 2 which is the comparison target frame.

Also, as shown in Fig. 9 (C), when tile 2 of frame 3 is delayed in processing, the average fluctuation evaluation value Dave of tile 1, tile 3 and tile 4 in time for processing is (4 + 30 + 0) / 3 Since = 11.3 and the average fluctuation evaluation value Dave is equal to or higher than the fluctuation evaluation threshold Dlim, CPU 11 replaces all tiles with tiles in frame 2.

On the other hand, as shown in FIG. 9 (D), when the processing of tile 3 of frame 3 is delayed, the average fluctuation evaluation value Dave of tile 1, tile 2 and tile 4 in time for processing is (4 + 0 + 0). ) / 3 = 1.3, and since the average fluctuation evaluation value Dave is less than the fluctuation evaluation threshold Dlim, only the tile 3 in which the processing delay occurs is replaced with the tile 3 of the frame 2 which is the comparison target frame.

Further, as shown in FIG. 10, when any tile in frame 5 (FIG. 5 (E)) is delayed in processing, the average fluctuation evaluation value Dave is equal to or higher than the fluctuation evaluation threshold Dlim regardless of which tile is delayed in processing. Therefore, CPU 11 replaces all tiles with the tile of frame 2 which is the comparison target frame.

Next, the calculation processing procedure in the calculation node personal computer 3 (3A to 3D) for performing the above-mentioned video compression coding and decompression decoding processing will be described in detail with reference to the flowchart shown in FIG.

The CPU 21 of the calculation node personal computer 3 enters from the start step of the routine RT3 and moves to the next step SP41. In step SP41, the CPU 21 listens for various processing parameters such as the coding target rate and the decoding level transmitted from the host node personal computer 2, and when it receives the processing parameters, it moves to the next step SP42.

In step SP42, the CPU 21 listens for the divided digital video data D2n or the divided coded data D3n transmitted from the host node personal computer 2, and when it receives the data, it moves to the next step SP43.

In step SP43, the CPU 21 performs a coding process for the divided digital video data D2n or a decoding process for the divided coded data D3n received from the host node personal computer 2, and when the processing is completed, the process proceeds to the next step SP44.

In step SP44, the CPU 21 returns the divided coded data D3n or the divided digital video data D2n of the processing result to the host node personal computer 2, and returns to step SP42.

(1-3) Operation and effect In the above configuration, the host node personal computer 2 of the moving image cluster processing system 1 supports four tiles of image data for one frame of the digital video signal D1 during compression coding processing. The divided digital video data D2A to D2D divided in this way are supplied to the corresponding calculation node personal computers 3A to 3D for compression coding processing, and each tile of the current frame has a pattern between it and the tile of the comparison target frame. The image variation evaluation value Diff, which represents the degree of change in, is calculated and stored in the memory 12.

Then, the host node personal computer 2 compresses and encodes the moving image by synthesizing the divided coded data D3A to D3D that are compressed and coded by each computing node personal computer 3A to 3D and returned to generate a coded video signal D4. To cluster.

At this time, when the compression coding process is delayed in any of the calculation node personal computers 3A to 3D, the host node personal computer 2 compares the image variation evaluation value Diff of the tile in which the processing delay has occurred with the variation evaluation threshold Dlim. ..

When the image variation evaluation value Diff is less than the variation evaluation threshold Dlim, the host node personal computer 2 has a small change in the pattern between the tile in which the processing delay has occurred and the tile in the comparison target frame, and the tile in which the processing delay has occurred. Is replaced with the tile of the comparison target frame, but the tile in which the processing delay occurs is replaced with the tile of the comparison target frame to generate the encoded video signal D4, assuming that there is little discomfort in the video after decoding.

On the other hand, in the host node personal computer 2, when the image variation evaluation value Diff is equal to or greater than the variation evaluation threshold Dlim, the pattern change between the tile in which the processing delay has occurred and the tile in the comparison target frame is large, and the processing delay occurs. Assuming that replacing only the tiles that have been used will increase the sense of discomfort in the decoded video, the entire frame including the tiles in which the processing delay has occurred is replaced with the frame to be compared to generate the encoded video signal D4.

In addition, the host node personal computer 2 divides the image data for one frame of the coded video signal D4 into four tiles at the time of decompression / decoding processing, and divides the coded data D3A to D3D into corresponding calculation nodes. It is supplied to personal computers 3A to 3D for decompression and decoding processing.

Then, the host node personal computer 2 expands and decodes the moving image by synthesizing the divided digital video data D2A to D2D that has been decompressed and decoded by each of the calculation node personal computers 3A to 3D and returned to generate the digital video signal D1. Cluster processing of digitalization.

At this time, the host node personal computer 2 is the average of the image variation evaluation values Diff of the other tiles that did not cause the processing delay when the tile decompression / decoding processing is delayed in any of the calculation node personal computers 3A to 3D. The fluctuation evaluation value Dave is calculated, and the average fluctuation evaluation value Dave is compared with the fluctuation evaluation threshold Dlim.

When the average fluctuation evaluation value Dave is less than the fluctuation evaluation threshold Dlim, the host node personal computer 2 has a small change in the pattern between the frame containing the tile in which the processing delay has occurred and the frame to be compared, and the processing delay occurs. Even if the tiles of the comparison target frame are replaced with the tiles of the comparison target frame, the tiles in which the processing delay has occurred are replaced with the tiles of the comparison target frame to generate the digital video signal D1.

On the other hand, in the host node personal computer 2, when the average fluctuation evaluation value Dave is equal to or higher than the fluctuation evaluation threshold Dlim, the pattern change between the frame containing the tile in which the processing delay has occurred and the frame to be compared is large, and the processing is performed. If the tile in which the delay occurs is replaced with the tile in the comparison target frame, the discomfort in the video after decoding becomes large, and the entire frame including the tile in which the processing delay occurs is replaced with the comparison target frame to generate the digital video signal D1.

According to the above configuration, when a processing delay occurs in the moving image cluster processing, the image variation evaluation value Diff of the tile in which the processing delay has occurred, or the image variation evaluation value Diff of the tile other than the tile in which the processing delay has occurred. Based on the average value of, it is determined whether to replace only the tile in which the processing delay occurs with the tile of the comparison target frame or the entire frame, so that it is smooth according to the change of the pattern. Error concealment can be performed.

(1-4) Other Embodiments In the above-described embodiment, the case where the moving image cluster processing system 1 executes the moving image compression / decompression processing by the JPEG2000 method has been described, but the present invention is not limited to this. , For example, various coding methods such as MPEG (Moving Picture Experts Group) 2 method can be applied to the present invention.

Here, the MPEG2 method is different from the JPEG2000 method in which coding is completed for each frame, and motion compensation frame-to-frame coding using the block matching method is performed by referring to the previous and next frames to improve the coding efficiency. There is.

That is, in the MPEG2 method, an I picture that performs in-frame coding, a P picture that performs forward predictive coding by referring to the I picture, and a B picture that performs bidirectional predictive coding by referring to the previous and next I pictures and P pictures. Three picture types are selected as appropriate. In the MPEG2 system, as shown in FIG. 12, a frame is composed of a sequence of digital video signals before coding (Fig. 12 (A)) and a sequence of coded video signals after coding (Fig. 12 (B)). There is a difference in the order of.

Therefore, when applying the present invention to the MPEG2 method, it is necessary to change the frame to be compared according to the picture type of the frame to be encoded. For example, when the frame to be encoded is a B picture or a P picture, the I picture or P picture in the front direction thereof is set as the comparison target frame. When the frame to be encoded is an I picture, the closest I picture or P picture in the front direction is set as the comparison target frame.

Further, in the MPEG2 method, as shown in FIG. 13, the coding processing time and the decoding processing time change according to the picture type and the motion search level (the motion search target range of the block matching method). Therefore, when the present invention is applied to the MPEG2 method, it is necessary to change the reference processing time for determining the processing delay according to the picture type and the motion search level of the frame to be encoded.

Further, in the above-described embodiment, the CPU 11 of the host node personal computer 2 executes the moving image cluster processing program stored in the hard disk drive 14, but the present invention is not limited to this, and the moving image cluster processing program is not limited to this. By installing the program storage medium in which is stored in the host node personal computer 2, the above-mentioned moving image cluster processing may be executed.

In this case, the program storage medium for installing the moving image cluster processing program on the host node personal computer 2 is not limited to package media such as CD-ROM (Compact Disk-Read Only Memory) and DVD (Digital Versatile Disk). It may be realized by a semiconductor memory, a magnetic disk, or the like in which the program is temporarily or permanently stored. Further, as a means for storing the program in these program storage media, a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting may be used.

(2) Second Embodiment In FIG. 14, which is shown by assigning the same reference numerals to the parts corresponding to those in FIG. 1, 30 is the virtual reality cluster processing system of the second embodiment of the present invention as a whole (hereinafter, VR). It is called a cluster processing system), and the host node personal computer 2 and the three computing node personal computers 3A to 3C are connected via the network 4.

In this VR cluster processing system 30, a virtual reality image generated by computer graphics processing is arranged so as to surround the front and left and right of a virtual reality (virtual reality) experiencer (hereinafter referred to as a VR experiencer). To detect the line-of-sight direction and movement of the three displays 32A to 32C (front display 32A, right side display 32B and left side display 32C) for displaying (hereinafter referred to as VR images) and the VR experience person. The VR device 34 including the motion sensor 33 of the above is connected to the video interface 31 of the host node personal computer 2. The motion sensor 33 may be mounted on a human head, or may detect the line-of-sight direction and movement with a video camera or the like.

The host node personal computer 2 of the VR cluster processing system 30 executes the VR cluster processing program under the execution environment of the operating system, detects the line-of-sight direction and movement of the VR experiencer via the motion sensor 33, and responds to the detection result. The VR image data D5A to D5C are distributed and generated in the calculation node personal computers 3A to 3C.

That is, the calculation node personal computer 3A generates VR image data D5A which is the data of the VR image displayed on the front display 32A. Similarly, the calculation node personal computer 3B generates VR image data D5B which is the data of the VR image displayed on the right side display 32B, and the calculation node personal computer 3C is the data of the VR image displayed on the left side display 32C. Generate VR image data D5C.

Then, the CPU 11 of the host node personal computer 2 supplies the VR image data D5A to D5C generated by each calculation node personal computer 3A to 3C to the corresponding displays 32A to 32C and displays the VR image to the VR experience person. , The VR experience is designed to let you experience a changing virtual reality space where the display changes according to your own movements.

FIG. 16 shows an example of the display change of the VR image in the front display 32A, the right side display 32B, and the left side display 32C. In this case, the VR experience person is moving toward the front display 32A while keeping the line-of-sight direction toward the front display 32A.

In addition to this configuration, the host node personal computer 2 constantly monitors the processing time required to generate VR image data D5A to D5C in each of the calculation node personal computers 3A to 3D, and any of the VR image data D5A to D5C can be generated. When there is a delay (occurrence of processing delay), error concealment is performed using the VR image data of the previous frame to prevent the VR image from being lost due to the processing delay of the calculation node.

Next, the cluster VR processing procedure in the host node personal computer 2 for generating VR image data with the above-mentioned error concealment will be described in detail with reference to the flowchart shown in FIG.

The CPU 11 of the host node personal computer 2 enters from the start step of the routine RT4 and moves to the next step SP51. In step SP51, the CPU 11 as a moving image processing instruction means instructs each calculation node personal computer 3A to 3C to generate VR image data D5A to D5C in the initial state, and moves to the next step SP52.

In step SP52, the CPU 11 as the processing result acquisition means supplies the VR image data D5A to D5C returned from each calculation node personal computer 3A to 3C to the corresponding displays 32A to 32C to display the VR image, and the next step. Move to SP53.

In step SP53, the CPU 11 extracts the line-of-sight direction and movement of the VR experiencer based on the change in the image D6 of the VR experiencer supplied from the motion sensor 33, and moves to the next step SP54.

In step SP54, the CPU 11 determines whether or not the line-of-sight direction and movement of the extracted VR experience person have changed. If it is determined in step SP53 that the line-of-sight direction and movement of the VR experience person have not changed, the CPU 11 returns to step SP53 and extracts the line-of-sight direction and movement of the VR experience person again.

On the other hand, if it is determined in step SP53 that the line of sight and movement of the VR experiencer have changed, the CPU 11 moves to the next step SP55.

In step SP55, the CPU 11 as the moving image processing instruction means instructs each calculation node personal computer 3A to 3C to generate a new VR image according to the change in the line-of-sight direction and the operation of the extracted VR experience person. Move on to the next step SP56.

In step SP56, the CPU 11 as the processing result acquisition means starts listening to the VR image data D5A to D5C returned from the calculation node personal computers 3A to 3C. Then, when the reference processing time elapses from the start of the standby, the CPU 11 moves to the next step SP57.

In step SP57, the CPU 11 determines whether or not there is a delay in the generation of the VR image data based on the return status of the VR image data D5A to D5C returned from the calculation node personal computers 3A to 3C.

When all the VR image data is returned in step SP57, this means that there is no processing delay in all the calculation node computers 3A to 3C and no error concealment is required. At this time, CPU11 Moves to step SP58, supplies each of the returned VR image data D5A to D5C to the corresponding displays 32A to 32C to display the VR image, and returns to step SP53.

On the other hand, in step SP57, if any of the VR image data D5A to D5C has not been returned yet, this causes a processing delay in any of the calculation node personal computers 3A to 3C, and error concealment is required. At this time, CPU 11 moves to step SP59.

In step SP59, the CPU 11 as the processing result replacement means determines whether or not the VR image data D5 in which the processing delay has occurred is an image in the line-of-sight direction of the VR experience person. If it is determined in step SP59 that the VR image data D5 in which the processing delay has occurred is not an image in the line-of-sight direction, the CPU 11 moves to step SP60.

In this case, since the VR image data D5 in which the processing delay has occurred is not an image in the line-of-sight direction, even if the image is discontinuous due to error concealment, it is difficult for the VR experiencer to detect it. Therefore, in step SP60, the CPU 11 uniformly replaces only the VR image data D5 in which the processing delay has occurred with the VR image data D5 one frame before, and returns to step SP53.

The state of error concealment in this case is shown in FIG. It is assumed that a processing delay occurs in the left side image at the timing of T = 3. In this case, since the VR image in which the processing delay has occurred is not an image in the line-of-sight direction, the image with T = 2 is replaced.

On the other hand, in step SP59, when it is determined that the VR image data D5 in which the processing delay has occurred is an image in the line-of-sight direction, the CPU 11 moves to step SP61.

In this case, since the VR image data D5 in which the processing delay has occurred is an image in the line-of-sight direction, if the image is discontinuous due to error concealment, the VR experience person can easily detect this. Therefore, the CPU 11 as the processing result replacement means calculates the average fluctuation evaluation value Dave for the other two VR image data D5 that did not cause the delay in step SP61, and moves to the next step SP62 to calculate the average fluctuation evaluation value. Compare Dave with the variation evaluation threshold Dlim.

In step SP62, when the average fluctuation evaluation value Dave is less than the fluctuation evaluation threshold Dlim, this means that there is little change in the VR image in which no processing delay has occurred. Therefore, it is assumed that there is little discomfort even if only the VR image in which the processing delay has occurred is replaced with the VR image in the previous frame. At this time, the CPU 11 as the processing result replacement means moves to step SP63, and the processing delay occurs. Only the VR image data D5 is replaced with the VR image data D5 of the previous frame and displayed, and the process returns to step SP53.

The state of error concealment in this case is shown in FIG. It is assumed that a processing delay occurs in the front image at the timing of T = 2. In this case, since the VR image in which the processing delay occurs is a front image in the line-of-sight direction, the average fluctuation evaluation value Dave of the right side image and the left side image in which the processing delay does not occur is calculated, and the average fluctuation evaluation value Dave is calculated. Is less than the variation evaluation threshold Dlim, so replacement is performed on the image with T = 1.

On the other hand, in step SP62, when the average fluctuation evaluation value Dave is equal to or greater than the fluctuation evaluation threshold Dlim, this indicates that the change in the VR image without the processing delay is large. For this reason, it is assumed that replacing only the VR image in which the processing delay has occurred with the VR image of the previous frame will increase the sense of incongruity. At this time, the CPU 11 as the processing result replacement means moves to step SP64, and all the VR image data D5. Is replaced with the VR image data D5 of the previous frame and displayed, and the process returns to step SP53.

The state of error concealment in this case is shown in FIG. It is assumed that a processing delay occurs in the front image at the timing of T = 4. In this case, since the VR image in which the processing delay occurs is a front image in the line-of-sight direction, the average fluctuation evaluation value Dave of the right side image and the left side image in which the processing delay does not occur is calculated, and the average fluctuation evaluation value Dave is calculated. Is equal to or greater than the variation evaluation threshold Dlim, so that all images are replaced with images with T = 1.

In the above configuration, the host node personal computer 2 of the VR cluster processing system 30 has VR image data D5A ~ according to the line-of-sight direction and operation of the VR experiencer detected via the motion sensor 33 with respect to the calculation node personal computers 3A to 3C. D5C is generated by cluster processing, and the VR image data D5A to D5C are supplied to the corresponding displays 32A to 32C to display the VR image.

At this time, when the generation of VR image data other than the line-of-sight direction of the VR experience person is delayed, the host node personal computer 2 replaces only the VR image data in which the processing delay has occurred with the VR image data one frame before and displays the data.

On the other hand, when the generation of VR image data in the line-of-sight direction of the VR experience person is delayed, the host node personal computer 2 sets the average fluctuation evaluation value Dave and the fluctuation evaluation threshold Dlim of the other VR image data that did not cause the processing delay. In comparison, when the average fluctuation evaluation value Dave is less than the fluctuation evaluation threshold Dlim, only the VR image data in which the processing delay has occurred is replaced with the VR image data of the previous frame and displayed, whereas the average fluctuation evaluation value Dave fluctuates. When the evaluation threshold is Dlim or more, all VR image data is replaced with the VR image data of the previous frame and displayed.

According to the above configuration, when a processing delay occurs in VR cluster processing, only the VR image data in which the processing delay has occurred is selected based on the average variation evaluation value Dave of other VR image data that did not cause the processing delay. By deciding whether to replace or replace all the VR image data, smooth error concealment can be performed according to the change of the VR image.

In the above-described embodiment, the case where the present invention is applied to the VR cluster processing system 30 that generates a VR image by computer graphics processing has been described, but the present invention is not limited to this, and for example, a game screen in a computer game. The present invention can be applied to various computer graphics processing such as generation of.

Further, in the above-described embodiment, the CPU 11 of the host node personal computer 2 executes the VR cluster processing program stored in the hard disk drive 14, but the present invention is not limited to this, and the VR cluster processing program stores the VR cluster processing program. The above-mentioned moving image cluster processing may be executed by installing the program storage medium described above in the host node personal computer 2.

In this case, the program storage medium for installing the VR cluster processing program on the host node personal computer 2 is not limited to package media such as CD-ROM (Compact Disk-Read Only Memory) and DVD (Digital Versatile Disk). It may be realized by a semiconductor memory, a magnetic disk, or the like in which the program is temporarily or permanently stored. Further, as a means for storing the program in these program storage media, a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting may be used.

It can be applied when moving image processing is performed using a plurality of personal computers, workstations, personal computers, and the like.

<figref num="1">It is a block diagram which shows the whole structure of a moving image cluster processing system.</figref><figref num="2">It is a schematic diagram which shows the division state of a frame.</figref><figref num="3">It is a schematic diagram which shows the flow of moving image cluster processing.</figref><figref num="4">It is a flowchart which shows the cluster coding processing procedure.</figref><figref num="5">It is a schematic diagram which shows the relationship between the input image and the image variation evaluation value.</figref><figref num="6">It is a schematic diagram which shows the state of the error concealment at the time of coding.</figref><figref num="7">It is a schematic diagram which shows the state of the error concealment at the time of coding.</figref><figref num="8">It is a flowchart which shows the cluster decoding processing procedure.</figref><figref num="9">It is a schematic diagram which shows the state of the error concealment at the time of decoding.</figref><figref num="10">It is a schematic diagram which shows the state of the error concealment at the time of decoding.</figref><figref num="11">It is a flowchart which shows the calculation node personal computer processing procedure.</figref><figref num="12">It is a schematic diagram which shows the change of a sequence before and after coding.</figref><figref num="13">It is a table which shows the standard coding processing time of the MPEG2 system.</figref><figref num="14">It is a block diagram which shows the whole structure of a VR cluster processing system.</figref><figref num="15">It is a flowchart which shows the cluster VR processing procedure.</figref><figref num="16">It is a schematic diagram which shows the display example of a VR image.</figref><figref num="17">It is a schematic diagram which shows the state of the error concealment of a VR image.</figref><figref num="18">It is a schematic diagram which shows the state of the error concealment of a VR image.</figref><figref num="19">It is a schematic diagram which shows the state of the error concealment of a VR image.</figref><figref num="20">It is a block diagram which shows the structure of the conventional cluster processing system.</figref><figref num="21">It is a block diagram which shows the application structure of a cluster processing system.</figref>

Code description

1 ...... Video cluster processing system, 2 ...... Host node computer, 3A ~ 3D ...... Computation node computer, 4 ...... Network, 11, 21. ..... CPU11, 12, 22 ... Memory, 13, 23 ... Display indicator, 14, 24 ... Hard disk drive, 15, 25 ... ... network interface, 16 ... video capture circuit, 17, 27 ... bus, 30 ... VR cluster processing system, 31 ... video interface, 32A ~ 32C ...... Display, 33 ...... Motion sensor.