Motion image data compression system

BACKGROUND OF THE INVENTION

1. FIELD OF THE INVENTION:

0001The present invention relates to an apparatus for transmitting and receiving stream data which forms a three-dimensional virtual space represented by three-dimensional computer graphics (hereinafter, referred to as three-dimensional CG or 3DCG) based on a network such as the Internet.

2. DESCRIPTION OF THE RELATED ART:

0002In recent years, as the fields to which 3DCG is applicable, a virtual mall, electric commerce, and various home pages related thereto on the Internet such as a WWW (World Wide Web) have drawn attention. In particular, with the rapid development of the Internet, an environment is becoming available for easily handling 3DCG with a relatively high quality such as games and movies at home. In the conventional WWW, a machine called a server such as a personal computer and a work station is connected to a plurality of machines called clients such as personal computers. When a client requests, the client downloads data such as information of an image, a voice, a text, and an arrangement provided by the server, if required. The data is reconstructed on the client side, whereby a user can obtain required information. A communication method based on a TCP/IP (Transmission Control/Internet Protocol) is adopted for communication between the server and the client.

0003Conventionally, data provided by the server is mainly text data and image data. Recently, with the advancement in standardization of VRML (Virtual Reality Modeling Language), a browser of VRML, and the like, there is a tendency of transferring 3DCG data such as a shape and/or a scene.

0004Hereinafter, the VRML will be briefly described.

0005According to a conventional data format such as an image and a text as in HTML (Hyper Text Markup Language), a great amount of transfer time and transfer cost are required for transferring image data, in particular, animation data. Therefore, in the current system, there is a limit to the network traffic. In contrast, according to conventional 3DCG, visual point information and light source information including configurations are all processed by using three-dimensional data obtained by CG (Computer Graphics) data by calculation. On the receiving side, image data is generated based on three-dimensional data, and it is displayed.

0006With the advancement in CG technology, the quality of an image created by CG is rapidly improved. Therefore, it is becoming very efficient to transfer CG data as it is in view of a data amount.

0007Typically, when CG data is used, the data can be compressed at a data compression ratio of 1/100 or less, compared with the case where comparable image data is transferred. Thus, there is a tendency of standardizing a method for transferring 3DCG data through a network. As one solution, it is proposed to standardize 3 DCG data called VRML (VRML Ver. 2.0). In VRML Ver. 2.0, formats of data such as data having a configuration called a primitive, light source data, visual point data, and texture data, and a method for specifying a movement of a rigid body.

0008On the other hand, there is an animation technique for generating an image in real time, which has drawn attention recently in the CG field. By using an animation technique in real time, it is attempted to reproduce a real movement of a CG character particularly in CMs and movies. As an example, a complicated shape of a human is represented by a skeleton structure, and a motion amount of joints of a skeleton which change from moment to moment is defined, whereby a complicated motion can be naturally reproduced.

0009However, according to the conventional 3DCG modeling language on the Internet based on the VRML, a motion cannot be set in real time to a structure having a complicated shape such as a human.

0010Furthermore, it is impossible to transmit and receive motion data of a CG character (e.g., human) which performs real motions in real time through a network with a narrow band such as a telephone line.

SUMMARY OF THE INVENTION

0011A transmitter of the present invention includes a multi-dimensional stream data transmitting unit, for transmitting multi-dimensional stream data.

0012In one embodiment of the present invention, the transmitter includes: a multi-dimensional stream data identifying unit for classifying input data into a plurality of kinds of motion data: and a multi-dimensional stream data compressing unit for compressing the plurality of kinds of motion data on a block basis.

0013In another embodiment of the present invention, the multi-dimensional stream data compressing unit changes a compression ratio in accordance with the kind of the motion data.

0014In another embodiment of the present invention, the multi-dimensional stream data compressing unit selects a compression method in accordance with the kind of the motion data.

0015In another embodiment of the present invention, the multi-dimensional stream data compressing unit generates a node sequence which is time-series data, composed of at least one motion data among the plurality of kinds of motion data during a predetermined period, and extracts an arbitrary node from the node sequence to compress the plurality of kinds of motion data on a block basis.

0016In another embodiment of the present invention, the multi-dimensional stream data transmitting unit transmits interpolation information required for reproducing the extracted node and the node sequence.

0017In another embodiment of the present invention, the multi-dimensional stream data compressing unit reduces a high frequency component of the plurality of kinds of motion data to compress the plurality of kinds of motion data on a block basis.

0018In another embodiment of the present invention, the multi-dimensional stream data compressing unit subjects at least one motion data among the plurality of kinds of motion data to coordinate transformation to compress the plurality of kinds of motion data on a block basis.

0019In another embodiment of the present invention, the multi-dimensional stream data compressing unit transforms data represented by three-dimensional vector coordinates among the plurality of kinds of motion data to data represented by polar coordinates, thereby compressing the plurality of kinds of motion data on a block basis.

0020In another embodiment of the present invention, the multi-dimensional stream data includes header information or stream format describing data, describing a kind of the multi-dimensional stream data to be transmitted, an order of the plurality of kinds of motion data to be transmitted, and a length of the plurality of kinds of motion data to be transmitted.

0021In another embodiment of the present invention, the multi-dimensional stream data has time information and space information of a plurality of dimensions.

0022In another embodiment of the present invention, the space information of a plurality of dimensions includes position information, rotation information, and scale information.

0023In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene includes data representing a position, rotation and a scale of the CG scene.

0024In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene is multi-dimensional animation stream data.

0025In another embodiment of the present invention, the multi-dimensional stream data has a data portion having a plurality of channels and a header portion describing a stream data format of the data portion and, each of the plurality of channels has a plurality of pieces of information regarding a three-dimensional CG character at a certain time.

0026In another embodiment of the present invention, the three-dimensional CG character is capable of being represented by a hierarchical structure having at least one joint, and each of the plurality of motion data has at least one of data having a dimension of a position vector at a reference position of the three-dimensional CG character, data having a dimension of a direction vector representing a posture at the reference position, data having a dimension of an angle vector of the at least one joint and a rotation angle around the angle vector, and stream data having a dimension of a rotation angle around a local coordinates axis of the at least one joint.

0027In another embodiment of the present invention, the multi-dimensional stream data is a packet having a header portion and a data portion, and the header portion represents a kind of the packet, an order of the packet, and a length of the data portion.

0028In another embodiment of the present invention, the above-mentioned transmitter includes a multi-dimensional stream data retaining unit for retaining data compressed by the multi-dimensional stream data compressing unit in a multi-hierarchical structure, wherein the transmitter selects data retained in a certain hierarchical structure from the data retained in the multi-hierarchical structure, in accordance with a state of a transmission path through which the multi-dimensional stream data is to be transmitted and a reproduction speed at which a receiver receiving the multi-dimensional stream data reproduces the multi-dimensional stream data, and transmits the selected data.

0029In another embodiment of the present invention, the above-mentioned transmitter selects data at a hierarchy level, at which a reproducible data amount is the largest, from the data retained in the multi-hierarchical structure, in accordance with a state of a transmission path through which the multi-dimensional stream data is to be transmitted, and a reproduction speed at which the receiver receiving the multi-dimensional stream data reproduces the multi-dimensional stream data.

0030In another embodiment of the present invention, the above-mentioned transmitter further includes a multi-dimensional stream data retaining unit for retaining data compressed by the multi-dimensional stream data compressing unit in a file format of the multi-dimensional stream data.

0031According to another aspect of the present invention, a transmitting method includes a multi-dimensional stream data transmitting step of transmitting multi-dimensional stream data.

0032In one embodiment of the present invention, the above-mentioned transmitting method includes: a multi-dimensional stream data identifying step of classifying input data into a plurality of kinds of motion data; and a multi-dimensional stream data compressing step of compressing the plurality of kinds of motion data on a block basis.

0033In another embodiment of the present invention, the multi-dimensional stream data compressing unit changes a compression ratio in accordance with the kind of the motion data.

0034In another embodiment of the present invention, the multi-dimensional stream data compressing unit selects a compression method in accordance with the kind of the motion data.

0035In another embodiment of the present invention, in the multi-dimensional stream data compressing step, a node sequence is generated, which is time-series data, composed of at least one motion data among the plurality of kinds of motion data during a predetermined period, and extracts an arbitrary node from the node sequence to compress the plurality of kinds of motion data on a block basis.

0036In another embodiment of the present invention, the above-mentioned transmitting method includes the step of transmitting interpolation information required for reproducing the extracted node and the node sequence.

0037In another embodiment of the present invention, in the multi-dimensional stream data compressing step, a high frequency component of the plurality of kinds of motion data is reduced to compress the plurality of kinds of motion data on a block basis.

0038In another embodiment of the present invention, in the multi-dimensional stream data compressing step, at least one motion data among the plurality of kinds of motion data is subjected to coordinate transformation to compress the plurality of kinds of motion data on a block basis.

0039In another embodiment of the present invention, in the multi-dimensional stream data compressing step, data represented by three-dimensional vector coordinates among the plurality of kinds of motion data is transformed to data represented by polar coordinates, thereby compressing the plurality of kinds of motion data on a block basis.

0040In another embodiment of the present invention, the multi-dimensional stream data includes header information or stream format describing data, describing a kind of the multi-dimensional stream data to be transmitted, an order of the plurality of kinds of motion data to be transmitted, and a length of the plurality of kinds of motion data to be transmitted.

0041In another embodiment of the present invention, the multi-dimensional stream data has time information and space information of a plurality of dimensions.

0042In another embodiment of the present invention, the space information of a plurality of dimensions includes position information, rotation information, and scale information.

0043In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene includes data representing a position, rotation and a scale of the CG scene.

0044In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene is multi-dimensional animation stream data.

0045In another embodiment of the present invention, the multi-dimensional stream data has a data portion having a plurality of channels, and a header portion describing a stream data format of the data portion, and each of the plurality of channels has a plurality of pieces of information regarding a three-dimensional CG character at a certain time.

0046In another embodiment of the present invention, the three-dimensional CG character is capable of being represented by a hierarchical structure having at least one joint, and each of the plurality of motion data has at least one of data having a dimension of a position vector at a reference position of the three-dimensional CG character, data having a dimension of a direction vector representing a posture at the reference position, data having a dimension of an angle vector of the at least one joint and a rotation angle around the angle vector, and stream data having a dimension of a rotation angle around a local coordinates axis of the plurality of joints.

0047In another embodiment of the present invention, the multi-dimensional stream data is a packet having a header portion and a data portion, and the header portion represents a kind of the packet, an order of the packet, and a length of the data portion.

0048In another embodiment of the present invention, the above-mentioned transmitting method includes a multi-dimensional stream data retaining step of retaining data compressed by the multi-dimensional stream data compressing unit in a multi-hierarchical structure, wherein the transmitting method selects data retained in a certain hierarchical structure from the data retained in the multi-hierarchical structure, in accordance with a state of a transmission path through which the multi-dimensional stream data is to be transmitted and a reproduction speed for reproducing the multi-dimensional stream data, and transmits the selected data.

0049In another embodiment of the present invention, data at a hierarchy level, at which a reproducible data amount is the largest, is selected from the data retained in the multi-hierarchical structure, in accordance with a state of a transmission path through which the multi-dimensional stream data is to be transmitted, and a reproduction speed at which the receiver receiving the multi-dimensional stream data reproduces the multi-dimensional stream data.

0050In another embodiment of the present invention, the above-mentioned transmitting method further includes a multi-dimensional stream data retaining step of retaining data compressed in the multi-dimensional stream data compressing step in a file format of the multi-dimensional stream data.

0051According to another aspect of the present invention, a receiver includes a multi-dimensional stream data receiving unit for receiving multi-dimensional stream data.

0052In one embodiment of the present invention, the receiver includes: a multi-dimensional stream data identifying unit for identifying a data content of the multi-dimensional stream data from one of header information and stream format describing information of the multi-dimensional stream data; a multi-buffering unit for sequentially reading the multi-dimensional stream data into a plurality of multi-buffers on a block basis; and a multi-dimensional stream data decompressing unit for decompressing the multi-dimensional put in blocks on a block basis.

0053In another embodiment of the present invention, the multi-dimensional stream data identifying unit determines whether or not the received multi-dimensional stream data is a channel definition packet.

0054In another embodiment of the present invention, the multi-dimensional stream data identifying unit determines whether or not the received multi-dimensional stream data is a data packet.

0055In another embodiment of the present invention, in the case where the multi-dimensional stream data identifying unit determines that the received multi-dimensional stream data is a channel definition packet, the multi-dimensional stream data identifying unit identifies a time stamp, a total number of channels, and a compression ratio identifier representing a compression ratio of channel data from the received multi-dimensional stream data.

0056In another embodiment of the present invention, in the case where the multi-dimensional stream data identifying unit determines that the received multi-dimensional stream data is a data packet, the multi-dimensional stream data identifying unit identifies a time stamp and a total number of channels from the received multi-dimensional stream data.

0057In another embodiment of the present invention, the received multi-dimensional stream data contains compressed motion data and compression information.

0058In another embodiment of the present invention, the multi-dimensional stream data decompressing unit decompresses the compressed motion data based on the compression information.

0059In another embodiment of the present invention, the multi-dimensional stream data decompressing unit determines a decompression ratio based on the compression information.

0060In another embodiment of the present invention, the multi-dimensional stream data decompressing unit determines a decompression method based on the compression information.

0061In another embodiment of the present invention, in the case where the compressed motion data is generated by extracting arbitrary motion data from a plurality of motion data corresponding to a predetermined period, the multi-dimensional stream data decompressing unit recovers a plurality of motion data corresponding to the predetermined period based the extracted motion data and the compression information.

0062In another embodiment of the present invention, the compression information is interpolation information.

0063In another embodiment of the present invention, the multi-dimensional stream data decompressing unit decompresses the compressed motion data by subjecting the compressed motion data to coordinate transformation based on the compression information.

0064In another embodiment of the present invention, in the case where the compressed motion data is represented by polar coordinates, the multi-dimensional stream data decompressing unit decompresses the compressed motion data by converting the compressed motion data represented by polar coordinates into data represented by three-dimensional vector coordinates.

0065In another embodiment of the present invention, the multi-dimensional stream data has time information and space information of a plurality of dimensions.

0066In another embodiment of the present invention, the space information of a plurality of dimensions has position information, rotation information, and scale information.

0067In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene has data representing a position, a rotation, and a scale of the CG scene.

0068In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene is multi-dimensional animation stream data.

0069In another embodiment of the present invention, the multi-dimensional stream data has a data portion having a plurality of channels and a header portion describing a stream data format of the data portion, and each of the plurality of channels has a plurality of pieces of information regarding a three-dimensional CG character at a certain time.

0070In another embodiment of the present invention, the three-dimensional CG character is capable of being represented by a hierarchical structure having at least one joint, and each of the plurality of motion data has at least one of data having a dimension of a position vector at a reference position of the three-dimensional CG character, data having a dimension of a direction vector representing a posture at the reference position, data having a dimension of an angle vector of the at least one joint and a rotation angle around the angle vector, and stream data having a dimension of a rotation angle around a local coordinates axis of the plurality of joints.

0071In another embodiment of the present invention, the multi-dimensional stream data is a packet having a header portion and a data portion, and the header portion represents a kind of the packet, an order of the packet, and a length of the data portion.

0072In another embodiment of the present invention, the receiver indicates a compression ratio of the multi-dimensional stream data to a transmitter for transmitting the multi-dimensional stream data, in accordance with at least one of a state of a transmission path through which the multi-dimensional stream data is to be transmitted and a reproduction speed at which the receiver reproduces the multi-dimensional stream data.

0073According to another aspect of the present invention, a receiving method includes a multi-dimensional stream data receiving step of receiving multi-dimensional stream data.

0074In one embodiment of the present invention, the above-mentioned receiving method includes: a multi-dimensional stream data identifying step of identifying a data content of the multi-dimensional stream data from one of header information and stream format describing information of the multi-dimensional stream data; a multi-buffering step of sequentially reading the multi-dimensional stream data into a plurality of multi-buffers on a block basis; and a multi-dimensional stream data decompressing step of decompressing the multi-dimensional put in blocks on a block basis.

0075In one embodiment of the present invention, in the multi-dimensional stream data identifying step, it is determined whether or not the received multi-dimensional stream data is a channel definition packet.

0076In another embodiment of the present invention, in the multi-dimensional stream data identifying step, it is determined whether or not the received multi-dimensional stream data is a data packet.

0077In another embodiment of the present invention, in the case where the multi-dimensional stream data identifying unit determines that the received multi-dimensional stream data is a channel definition packet, in the multi-dimensional stream data identifying step, a time stamp, a total number of channels, and a compression ratio identifier representing a compression ratio of channel data are identified from the received multi-dimensional stream data.

0078In another embodiment of the present invention, in the case where the multi-dimensional stream data identifying unit determines that the received multi-dimensional stream data is a data packet, in the multi-dimensional stream data identifying step, a time stamp and a total number of channels are identified from the received multi-dimensional stream data.

0079In another embodiment of the present invention, the received multi-dimensional stream data contains compressed motion data and compression information.

0080In another embodiment of the present invention, the multi-dimensional stream data decompressing unit decompresses the compressed motion data based on the compression information.

0081In another embodiment of the present invention, in the multi-dimensional stream data decompressing step, a decompression ratio is determined based on the compression information.

0082In another embodiment of the present invention, in the multi-dimensional stream data decompressing step, a decompression method is determined based on the compression information.

0083In another embodiment of the present invention, in the case where the compressed motion data is generated by extracting arbitrary motion data from a plurality of motion data corresponding to a predetermined period, in the multi-dimensional stream data decompressing step, a plurality of motion data corresponding to the predetermined period are recovered based the extracted motion data and the compression information.

0084In another embodiment of the present invention, the compression information is interpolation information.

0085In another embodiment of the present invention, in the multi-dimensional stream data decompressing step, the compressed motion data is decompressed by subjecting the compressed motion data to coordinate transformation based on the compression information.

0086In another embodiment of the present invention, in the case where the compressed motion data is represented by polar coordinates, in the multi-dimensional stream data decompressing step, the compressed motion data is decompressed by converting the compressed motion data represented by polar coordinates into data represented by three-dimensional vector coordinates.

0087In another embodiment of the present invention, the multi-dimensional stream data has time information and space information of a plurality of dimensions.

0088In another embodiment of the present invention, the space information of a plurality of dimensions has position information, rotation information, and scale information.

0089In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene has data representing a position, a rotation, and a scale of the CG scene.

0090In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene is multi-dimensional animation stream data.

0091In another embodiment of the present invention, the multi-dimensional stream data has a data portion having a plurality of channels and a header portion describing a stream data format of the data portion and, each of the plurality of channels has a plurality of pieces of information regarding a three-dimensional CG character at a certain time.

0092In another embodiment of the present invention, the three-dimensional CG character is capable of being represented by a hierarchical structure having at least one joint, and each of the plurality of motion data has at least one of data having a dimension of a position vector at a reference position of the three-dimensional CG character, data having a dimension of a direction vector representing a posture at the reference position, data having a dimension of an angle vector of the at least one joint and a rotation angle around the angle vector, and stream data having a dimension of a rotation angle around a local coordinates axis of the at least one joint.

0093In another embodiment of the present invention, the multi-dimensional stream data is a packet having a header portion and a data portion, and the header portion represents a kind of the packet, an order of the packet, and a length of the data portion.

0094In another embodiment of the present invention, a compression ratio of the multi-dimensional stream data is indicated to a transmitter for transmitting the multi-dimensional stream data, in accordance with at least one of a state of a transmission path through which the multi-dimensional stream data is to be transmitted and a reproduction speed for reproducing the multi-dimensional stream data.

0095In another embodiment of the present invention, the multi-dimensional stream data contains motion data and voice data, the motion data has time stamp information, and the voice data has time stamp information, and the receiver reproduces the motion data and the voice data in synchronization, in accordance with the time stamp information of the motion data and the time stamp information of the voice data.

0096In another embodiment of the present invention, the multi-dimensional stream data contains motion data and voice data, the motion data has time stamp information, and the voice data has time stamp information, and the receiving method further comprises the step of reproducing the motion data and the voice data in synchronization, in accordance with the time stamp information of the motion data and the time stamp information of the voice data.

0097According to another aspect of the present invention, a recording medium capable of being read by a computer is provided, in which a program for allowing the computer to execute the above-mentioned transmitting method is recorded.

0098According to another aspect of the present invention, a recording medium capable of being read by a computer is provided, in which a program for allowing the computer to execute the above-mentioned receiving method is recorded.

0099According to another aspect of the present invention, a recording medium capable of being read by a computer is provided, in which a program for allowing the computer to execute the above-mentioned receiving method is recorded.

0100According to another aspect of the present invention, a communication system includes: a transmitter for transmitting multi-dimensional stream data; and a receiver for receiving the multi-dimensional stream data.

0101In one embodiment of the present invention, the transmitter includes one of an artificial satellite, a ground wave broadcasting station, and a wire broadcasting station, and the receiver is one of a digital TV and a set top box.

0102In the above-mentioned transmitter, the multi-dimensional stream data contains data capable of representing a CG scene.

0103In the above-mentioned transmitting method, the multi-dimensional stream data contains data capable of representing a CG scene.

0104In the above-mentioned receiver, the multi-dimensional stream data contains data capable of representing a CG scene.

0105In the above-mentioned receiving method, the multi-dimensional stream data contains data capable of representing a CG scene.

0106In the above-mentioned communication system, the multi-dimensional stream data contains data capable of representing a CG scene.

0107Thus, the invention described herein makes possible the advantage of providing an apparatus for transmitting and receiving multi-dimensional stream data, which is capable of transmitting and receiving motion data of a CG character in real time.

0108This and other advantages of the present invention will become apparent to those skilled in the art upon reading and understanding the following detailed description with reference to the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

0109<ul id="ul0001" list-style="none"><li>Figure <b>1</b> shows an example of a CG scene displayed by a display device.</li><li>Figure <b>2</b> shows a hierarchical structure representing a CG scene shown in Figure <b>1</b> as an example of multi-dimensional stream data.</li><li>Figure <b>3</b> shows a system which transmits and receives multi-dimensional stream data in Embodiment 1.</li><li>Figure <b>4</b> shows an apparatus for transmitting and receiving multi-dimensional stream data.</li><li>Figure <b>5</b> shows an example of a skeleton structure of a three-dimensional CG character.</li><li>Figure <b>6</b> shows an example of classification of input data.</li><li>Figure <b>7</b> shows an example of a block in a time axis direction of three-dimensional CG character motion data.</li><li>Figure <b>8</b> shows a relationship between the three-dimensional vector coordinates and the polar coordinates.</li><li>Figure <b>9A</b> shows 11 nodes extracted from 30 frame/block, and Figure <b>9B</b> shows 6 nodes extracted from 15 frame/block.</li><li>Figure <b>10</b> shows an example of a multi-dimensional stream data format.</li><li>Figure <b>11</b> shows an exemplary structure of multi-dimensional stream data in a time axis direction.</li><li>Figure <b>12</b> shows an example of stream data output from a transmitter in the case where data input to the transmitter is 30 frame/block.</li><li>Figure <b>13</b> shows reproduction of a motion stream and a voice stream in the case where a receiver is provided with a reproduction unit.</li><li>Figure <b>14</b> illustrates that a plurality of receivers are connected to the transmitter.</li><li>Figure <b>15</b> shows an example in which time stamps are inserted in a stream authoring unit.</li><li>Figure <b>16</b> is a flow chart illustrating the operation of the transmitter.</li><li>Figure <b>17</b> is a flow chart illustrating the operation of the receiver.</li></ul>

DESCRIPTION OF THE PREFERRED EMBODIMENTS

0110The present invention will be described by way of illustrative embodiments with reference to the drawings.

Embodiment 1

0111Hereinafter, Embodiment 1 will be described with reference to Figures <b>1</b> to <b>3</b>.

0112In Embodiment 1, multi-dimensional stream data is transmitted and/or received, whereby multi-dimensional stream data is reproduced with a stable quality at a high speed. Herein, the multi-dimensional stream data may contain data representing a CG scene. The data representing a CG scene contains geometric information such as object information, light source information, and viewing point information.

0113Figure <b>1</b> shows an example of a CG scene displayed by a display device. Figure <b>1</b> shows that a table is positioned in a room surrounded by a wall.

0114Figure <b>2</b> shows a hierarchical structure representing a CG scene shown in Figure <b>1</b> as an example of multi-dimensional stream data. As shown in Figure 2, the wall and the table are branched from the room which is a root. If there is a cup on the table, the cup is branched from the table. The room, wall, and table respectively contain data for forming these objects. In an example shown in Figure <b>2</b>, the room, wall and table respectively contain data such as a position, a rotation, and a scale.

0115Figure <b>3</b> shows a system <b>100</b> which transmits and receives multi-dimensional stream data in Embodiment 1.

0116The system <b>100</b> shown in Figure <b>3</b> is provided with a multi-dimensional stream data transmitting unit <b>101,</b> and a multi-dimensional stream data receiving unit <b>102.</b> The multi-dimensional stream data transmitting unit <b>101</b> transmits multi-dimensional stream data in time series, and the multi-dimensional stream data receiving unit <b>102</b> receives multi-dimensional stream data in time series. The multi-dimensional stream data receiving unit <b>102</b> may reproduce a CG scene based on received multi-dimensional stream data.

0117Herein, the multi-dimensional stream data includes time information and space information of a plurality of dimensions, and time and space are multi-dimensional. The space information of a plurality of dimensions may include position information, rotation information, and scale information of a CG character. Alternatively, the multi-dimensional stream data may include data representing a position, a rotation and a scale of a CG character. Furthermore, the multi-dimensional stream data may be multi-dimensional animation stream data representing animation of a CG character.

0118A transmission path through which multi-dimensional stream data is transmitted is not limited to a telephone line or the like, and may be wireless. Furthermore, the multi-dimensional stream data receiving unit <b>102</b> may be a personal computer, a digital TV, or the like, as long as it is provided with a display device.

0119In Embodiment 1, CG data is transmitted as stream data. Therefore, even when a part of stream data is lost, the original CG scene can be easily reproduced. An example of a data structure of the multi-dimensional stream data will be described later.

0120In Embodiment 1, the multi-dimensional stream data transmitting unit may include an artificial satellite, and the multi-dimensional stream data receiving unit may be a digital TV.

0121In Embodiment 1, the multi-dimensional stream data transmitting unit may include an artificial satellite, and the multi-dimensional stream data receiving unit may be a set top box.

0122The artificial satellite receives the multi-dimensional stream data from a ground station.

0123In Embodiment 1, the multi-dimensional stream data transmitting unit may include a ground wave broadcasting station, and the multi-dimensional stream data receiving unit may be a digital TV.

0124In Embodiment 1, the multi-dimensional stream data transmitting unit may include a ground wave broadcasting station, and the multi-dimensional stream data receiving unit may be a set top box.

0125In Embodiment 1, the multi-dimensional stream data transmitting unit may include a wire broadcasting station, and the multi-dimensional stream data receiving unit may be a digital TV.

0126In Embodiment 1, the multi-dimensional stream data transmitting unit may include a wire broadcasting station, and the multi-dimensional stream data receiving unit may be a set top box.

0127A program for allowing a computer to execute the above-mentioned multi-dimensional stream data transmitting method and/or receiving method may be recorded in a recording medium capable of being read by a computer.

Embodiment 2

0128Hereinafter, Embodiment 2 will be described with reference to Figures <b>4</b> to <b>16.</b>

0129In Embodiment 2, data representing a CG scene is compressed based on a tree structure of CG, and multi-dimensional stream data is transmitted at a higher speed.

0130Figure <b>4</b> shows an apparatus <b>10</b> for transmitting and receiving multi-dimensional stream data. A transmitter <b>1</b> transmits multi-dimensional stream data, and a receiver <b>2</b> receives the transmitted multi-dimensional stream data.

0131Herein, the multi-dimensional stream data includes time information and space information of a plurality of dimensions, and time and space are multi-dimensional. The space information of a plurality of dimensions may include position information, rotation information, and scale information of a CG character. Alternatively, the multi-dimensional stream data may include data representing a position, a rotation and a scale of a CG character. Furthermore, the multi-dimensional stream data may be multi-dimensional animation stream data representing animation of a CG character.

0132The apparatus <b>10</b> shown in Figure <b>4</b> includes a transmitter <b>1</b>, a receiver <b>2</b>, a multi-dimensional stream data storing unit <b>31</b>, and a multi-dimensional stream data storing unit <b>32</b>.

0133The transmitter <b>1</b>, and the receiver <b>2</b> are realized by a computer apparatus such as a work station, a personal computer, and a game machine.

0134The transmitter <b>1</b> may have a stream authoring unit (not shown), or the stream authoring unit may be provided outside the transmitter <b>1</b>. In accordance with the user's instruction, the steam authoring unit generates three-dimensional CG stream data representing a three-dimensional virtual space. The operation of the stream authoring unit will be described later.

0135Furthermore, the transmitter <b>1</b> includes a multi-dimensional stream data identifying unit <b>11,</b> a multi-dimensional stream data compressing unit <b>12,</b> and a multi-dimensional stream data transmitting unit <b>13.</b> The receiver <b>2</b> includes a multi-dimensional stream data identifying unit <b>21,</b> a multi-buffering unit <b>22,</b> a multi-dimensional stream data decompressing unit <b>23,</b> and a multi-dimensional stream data receiving unit <b>24.</b>

0136First, a series of processing in the transmitter <b>1</b> will be described.

0137An example of identifying multi-dimensional stream data in the transmitter <b>1</b> will be described. As an example of a multi-dimensional stream, the case of transmitting motion data of a three-dimensional CG character will be described.

0138Figure <b>5</b> shows an example of a skeleton structure of a three-dimensional CG character.

0139The skeleton of the three-dimensional CG character shown in Figure <b>5</b> corresponds to a humanoid skeleton, which has 22 joints. The first joint represents a root.

0140The multi-dimensional stream data identifying unit <b>11</b> classifies a plurality of data input to the transmitter <b>1</b>.

0141Figure <b>6</b> shows an example of classification of input data. The multi-dimensional stream data identifying unit <b>11</b> classifies the input data into a plurality of kinds of motion data of the three-dimensional CG character and voice data <b>SS.</b> A plurality of kinds of motion data of the three-dimensional CG character obtained by classifying the input data contain a position vector <b>MS1</b> of a reference position of the three-dimensional CG character in a three-dimensional space, a direction vector <b>MS2</b> representing the posture of the three-dimensional CG character at the reference position, and information <b>MS3</b> of an angle vector at each joint of the skeleton and a rotation angle around the angle vector.

0142In the case where the three-dimensional CG character has the above-mentioned skeleton, <b>MS1</b> may be a position vector of the root, <b>MS2</b> may be a direction vector and a rotation angle around the direction vector, and <b>MS3</b> may be a rotation axis vector at each joint and a rotation angle around the rotation axis vector. A general expression of a hierarchical structure of a three-dimensional CG character is described in detail in Japanese Patent Application No. 9-100458 filed by the Applicant. Furthermore, the multi-dimensional stream data identifying unit <b>11</b> may classify the input data into data other than motion data and voice data <b>SS.</b>

0143The multi-dimensional stream data compressing unit <b>12</b> in the transmitter <b>1</b> applies an individual compression method to the classified data <b>MS1</b> to <b>MS3</b> based on the tree structure of CG and/or the kind of motion data to be classified, and compresses the data <b>MS1</b> to <b>MS3</b> on a block basis. For example, the multi-dimensional stream data compressing unit <b>12</b> changes the compression ratio depending upon the kind of motion data.

0144A method for compressing multi-dimensional stream data by the multi-dimensional stream data compressing unit <b>12</b> will be described with reference to Figures <b>7</b> and <b>8</b>.

0145With respect to the data <b>MS1</b> to <b>MS3</b> of the three-dimensional CG character classified by the multi-dimensional stream data identifying unit <b>11,</b> the multi-dimensional stream data compressing unit <b>12</b> puts motion data in one block during an arbitrary period (time <b>t</b> to time <b>t'</b>) in a time axis direction, as shown in Figure <b>7</b>. Next, the multi-dimensional stream data compressing unit <b>12</b> applies a different compression method, depending upon the kind of the data <b>MS1</b> to <b>MS3.</b>

0146A block method for compressing the data <b>MS1</b> to <b>MS3</b> on a block basis by the multi-dimensional stream data compressing unit <b>12</b> will be described.

0147The position vector <b>MS1</b> at the reference position of the three-dimensional CG character has three floating points representing a position (x, y, z) in a three-axis direction. By reducing the precision of each value, data compression can be performed.

0148For example, in the case where each component of the position (x, y, z) is represented by 4 bytes (32 bits), the multi-dimensional stream data compressing unit <b>12</b> performs data compression by representing each component of the position by 2 bytes (16 bits).

0149When each component of the position is represented by 2 bytes, three floating points can be expressed in 65536 stages (-32768 to 32767) in a decimal number. By expressing the floating points of 4 bytes in 65536 stages, data of 4 bytes can be compressed to data of 2 bytes. For example, the multi-dimensional stream data compressing unit <b>12</b> may search for a maximum value and a minimum value of input floating point data during a predetermined period, divided the difference between the maximum value and the minimum value into 65536 stages, and express 4-byte data by 2 bytes.

0150In the above-mentioned example, data of 4 bytes is expressed by 2 bytes. However, the multi-dimensional stream data compressing unit <b>12</b> may express a component of an N-byte position by 0 to (N-1) byte.

0151The direction vector <b>MS2</b> representing the posture of the three-dimensional CG character contains 3 components (three-dimensional vector) and the information <b>MS3</b> of the angle vector at each joint and the rotation angle around the angle vector contains 4 components in total (3 components (three-dimensional vector) representing a rotation axis in a local coordinate system at each hierarchical level of the three-dimensional CG character and one component representing the rotation angle). The multi-dimensional stream data compressing unit <b>12</b> normalizes (sets the magnitude of a vector at 1) the three-dimensional vector representing the rotation axis, i.e., P (X, Y, Z) in Figure <b>8,</b> and converts rectangular coordinates into polar coordinates, thereby expressing three components (X, Y, Z) by two components ( τ , θ ). More specifically, the direction vector <b>MS2</b> can be compressed from 3 components to 2 components in a space axis direction. The information <b>MS3</b> can be compressed from 4 components to 3 components in a space axis direction.

0152Furthermore, the multi-dimensional stream data compressing unit <b>12</b> may compress the position vector <b>MS1</b> at the reference position of the three-dimensional CG character and the angle vector at each joint and the rotation angle <b>MS3</b> around the angle vector in a time axis direction. For example, the multi-dimensional stream data compressing unit <b>12</b> generates a node sequence from a plurality of kinds of motion data (e.g., at least one of the position vector <b>MS1</b> and the information <b>MS3</b>), and extracts an arbitrary node from the node sequence, thereby compressing a plurality of kinds of motion data on a block basis.

0153Figure <b>9A</b> shows 11 nodes extracted from 30 frame/block. Figure <b>9B</b> shows 6 nodes extracted from 15 frame/block.

0154The nodes shown in Figures <b>9A</b> and <b>9B</b> represent at least one of the position vector <b>MS1</b> and the information <b>MS3.</b> A node column represents at least one of the position vector <b>MS1</b> and the information <b>MS3</b> in time sequence.

0155As shown in Figure <b>9A,</b> in the case where the nodes in one block are denoted with Nos. 0 to 29 in 30 frame/block, 11 nodes of Nos. 0, 2, 5, 8, 11, 14, 18, 21, 24, 27, and 29 are extracted. Only the extracted nodes are output from the multi-dimensional stream data compressing unit <b>12,</b> and the other nodes are not output from the multi-dimensional stream data compressing unit <b>12.</b>

0156Furthermore, as shown in Figure <b>9B,</b> in the case where the nodes in one block are denoted with Nos. 0 to 14 in 15 frame/block, 6 nodes of Nos. 0, 2, 5. 9, 12, and 14 are extracted. Only the extracted nodes are output from the multi-dimensional stream data compressing unit <b>12</b>, and the other nodes are not output from the multi-dimensional stream data compressing unit <b>12</b>.

0157In the case where a node extracted from the multi-dimensional stream compressing unit <b>12</b> is output, the multi-dimensional stream compressing unit <b>12</b> may output interpolation information required for recovering an original node sequence from the extracted node.

0158The number of nodes to be extracted per block is not limited to 11 or 6. The number of nodes to be extracted is appropriately determined by a user or the like. When it is desired to enhance an image quality of a CG scene to be transmitted, the number of nodes to be extracted is increased. When a CG scene is transmitted at a high speed, the number of nodes to be extracted should be decreased.

0159The extracted nodes are subjected to data interpolation by three-dimensional spline interpolation in the receiver <b>2.</b> Other interpolation methods such as a linear interpolation may be used in place of the three-dimensional interpolation.

0160The multi-dimensional stream data compressing unit <b>12</b> may reduce at least one high frequency component of a plurality of kinds of motion data, and compress the motion data on a block basis.

0161Next, the multi-dimensional stream data storing unit <b>31</b> will be described in detail.

0162Stream data <b>MSC1</b> to <b>MSC3,</b> and <b>SSC</b> compressed on a block basis are stored, as multi-dimensional stream data, in a storage device in an ordinary computer apparatus in the multi-dimensional stream data storing unit <b>31</b>.

0163Hereinafter, a format of multi-dimensional stream data stored in the multi-dimensional stream data storing unit <b>31</b> will be described in detail.

0164The multi-dimensional stream data compressing unit <b>12</b> puts compressed motion data in a block, and adds header information for transmitting multi-dimensional stream data to the block. The block and the header information are stored in the multi-dimensional stream data storing unit <b>31</b>.

0165Figure <b>10</b> shows an example of a format of multi-dimensional stream data stored in the multi-dimensional stream data storing unit <b>31.</b>

0166The multi-dimensional stream data is composed of packets. Each packet has a header portion and a data portion. Depending upon the kind of stream data, one stream data will be referred to as a channel, and each channel is provided with a uniquely determined channel No. (There is no limit to the order of Nos. to be provided to the channels).

0167The multi-dimensional stream data stored in the multi-dimensional stream data storing unit <b>31</b> includes a channel definition packet and a data packet, as shown in Figure <b>10</b>. The channel definition packet refers to a stream defining the channels in the data packet which follows after the channel definition packet is transmitted. The channel definition packet and the data packet respectively have a header portion and a data portion.

0168The header portion of the channel definition packet includes, for example, a packet identifier which identifies itself as the channel definition packet, a time stamp which represents a time based on a certain reference time when the channel definition packet is generated, the total number <b>TC</b> of channels through which data can be transmitted in the channel definition packet, and a compression ratio identifier which represents the number of nodes when the nodes are extracted under compression on a block basis of the stream data. The data portion of the channel definition packet includes the total number <b>TC</b> of channels. Each channel of the channel definition includes a channel identifier which represents a channel No., a data size of the channel, a channel name, and a channel type which represents if the channel is a position vector or a rotation angle vector.

0169As described above, the data packet includes a header portion and a data portion.

0170The header portion of the data packet includes, for example, a packet identifier which identifies itself as a data packet, a time stamp which represents a time based on a certain reference time when the channel definition packet is generated, and the total number <b>DC</b> of channels (<b>TC ≧ DC</b>) through which data can be transmitted in the data packet.

0171The data portion of the data packet includes the total number <b>DC</b> of channels, and each channel includes a channel identifier which represents a channel No., channel data and compression ratio identifier (e.g., the number of extracted notes, etc.). It is noted that the number of compression identifiers is equal to that of the channel data. The data packet is equal to one block of multi-dimensional steam data.

0172Figure <b>11</b> shows an exemplary structure of the channel definition packet and the data packet. When the channel definition packet is first transmitted, only the data packets may be transmitted thereafter. In the case where it is desired to change the channel definition in the middle of a stream, the channel definition can be easily changed by transmitting the channel definition packet in the middle.

0173The multi-dimensional stream data is not limited to the packet shown in Figure <b>10</b>. For example, the multi-dimensional stream data may have header information or stream format describing data, describing the kind of multi-dimensional stream data, the order of a plurality of motion data, and the length of a plurality of motion data.

0174Next, the multi-dimensional stream data storing unit <b>31</b> stores a plurality of stream data having different compression ratios. In the case where an excessive or insufficient amount of data flows through a transmission path between the transmitter <b>1</b> and the receiver <b>2</b>, in order to effectively utilize the transmission path, the transmitter <b>1</b> selects stream data with an optimum compression ratio, in accordance with at least one of the state of the transmission path, the request from the receiver <b>2</b>, and the request from a user. This will be described below. To store a plurality of stream data having different compression ratios will be referred to as a multi-hierarchical storage method.

0175The transmitter <b>1</b> generates stream data which is compressed at a certain compression ratio on a block basis, and generates some kinds of stream data which are compressed at a different compression ratio from that of the compressed stream data. The stream data compressed at a different compression ratio are stored in the multi-dimensional stream data storing unit <b>31</b>.

0176Figure <b>12</b> shows an example of stream data output from the transmitter <b>1</b> in the case where data input to the transmitter <b>1</b> is 30 frame/block.

0177The highest hierarchical level shown in Figure <b>12</b> refers to stream data obtained by outputting input data in 30 frame/block as one block without compressing it. The second hierarchical level refers to stream data obtained by compressing input data in 30 frame/block into data in 11 frame/block. The third hierarchical level refers to stream data obtained by thinning out every other input data in 30 frame/block, thereby compressing the input data into data in 15 frame/block which is a half data amount. The lowest hierarchical level refers to stream data obtained by compressing stream data in 15 frame/block into data in 6 frame/block. Furthermore, the stream data at the second hierarchical level and the stream data at the lowest hierarchical level are complemented by three-dimensional spline interpolation.

0178Multi-hierarchical stream data is stored in the multi-dimensional stream data storing unit <b>31</b>, whereby stream data containing an information amount in accordance with the state of the transmission path can be transmitted.

0179Furthermore, according to the multi-hierarchical storage method, a compression ratio of stream data transmitted from the transmitter can be controlled to be optimum so as to be consistent with the ability of a control circuit (e.g., computer) included in the receiver <b>2</b>.

0180Next, the data transmission between the transmitter <b>1</b> and the receiver <b>2</b> will be described.

0181In the case where there is an instruction from a control portion of the transmitter <b>1</b> or there is a request from the receiver <b>2</b>, the transmitter <b>1</b> transmits multi-dimensional stream data stored in a storage device in a computer apparatus, i.e., the multi-dimensional stream data storing unit <b>31</b>. A medium for data transmission in this case is assumed to be a bi-directional communication system, i.e., the Internet. However, data can be similarly transmitted and received via a broadcasting system such as satellite broadcasting and ground wave broadcasting.

0182Hereinafter, a series of processing of the receiver <b>2</b> will be described.

0183The multi-dimensional stream data receiving unit <b>24</b> receives multi-dimensional stream data transmitted from the transmitter <b>1</b>.

0184Next, identification of multi-dimensional stream data in the receiver <b>2</b> will be exemplified. The case where multi-dimensional stream data of a three-dimensional CG character is received will be described as an example of a multi-dimensional stream. The multi-dimensional stream data includes the above-mentioned compressed motion data in a predetermined stream format. The motion data included in the multi-dimensional stream data may not be compressed.

0185First, the multi-dimensional stream data receiving unit <b>24</b> in the receiver <b>2</b> receives multi-dimensional stream data <b>MSC1</b> to <b>MSC3</b> transmitted from the transmitter <b>1</b>. In the case where the transmitted multi-dimensional stream data <b>MSC1</b> to <b>MSC3</b> has a format as shown in Figure <b>10,</b> the multi-dimensional stream data identifying unit <b>21</b> first receives a packet, identifier and analyzes it, thereby determining if the received stream data is a channel definition packet or a data packet.

0186In the case where the multi-dimensional stream data identifying unit <b>21</b> determines the transmitted multi-dimensional stream data <b>MSC1</b> to <b>MSC3</b> to be a channel definition packet, the multi-dimensional stream data identifying unit <b>21</b> then receives a time stamp, the total number of channels <b>TC</b>, and a compression ratio identifier, and analyzes them. Subsequently, the multi-dimensional stream data identifying unit <b>21</b> receives a data portion, i.e., channel identifiers, channel data sizes, channel names, and channel types respectively by the total number of channels <b>TC</b>, and analyze them. If the multi-dimensional stream data identifying unit <b>21</b> fails in data analysis, the receiver <b>2</b> requests the transmitter <b>1</b> to re-transmit data so as to obtain data again. Alternatively, the receiver <b>2</b> forcefully terminates a connection of the transmitter <b>1</b> and the receiver <b>2</b>. Herein, the compression ratio identifier means compression information such as a compression method and a compression ratio used in the compression method.

0187In the case where the multi-dimensional stream data identifying unit <b>21</b> determines that the transmitted multi-dimensional stream data <b>MSC1</b> to <b>MSC3</b> to be a data packet, the multi-dimensional stream data identifying unit <b>21</b> receives a time stamp, and the total number of channels <b>DC</b>, and analyzes them. Then, the multi-dimensional stream data identifying unit <b>21</b> receives a data portion, i.e., a channel identifier, channel data, and a compression ratio identifier, and analyzes them. This is repeated by the total number of channels <b>DC.</b> The multi-dimensional stream data identifying unit <b>21</b> assumes that only the position vector at the reference position and the direction vector representing a posture among motion data of a three-dimensional CG character have special channel names, and distinguishes these vectors from each other based on the channel names.

0188In the case where the multi-dimensional stream data identifying unit <b>21</b> fails in data analysis of the data packet, the receiver <b>2</b> requests the transmitter <b>1</b> to re-transmit data so as to obtain the data again. Alternatively, the receiver <b>2</b> forcefully terminates a connection of the transmitter <b>1</b> and the receiver <b>2</b>. Alternatively, the receiver <b>2</b> discards the data packet whose analysis has failed, and receives a subsequent packet, whereby the connection can be continued.

0189Next, the multi-buffering unit <b>22</b> in the receiver <b>2</b> will be described.

0190In the multi-buffering unit <b>22</b> in the receiver <b>2</b>, multi-dimensional stream data divided into blocks is buffered on a block basis at a multiplicity which is consistent with processing performance or an execution speed of a computer having the receiver <b>2</b>. More specifically, in the case where the processing performance and execution speed of the computer are large, multiplicity of buffering is set to be large. In the case where the processing performance and execution speed of the computer are small, multiplicity of buffering is set to be small. In this case, data storage for buffering is performed by the multi-dimensional stream data storing unit <b>32.</b> The multi-buffering will be described in more detail together with the later description of simultaneous reproduction.

0191Next, the multi-dimensional stream data decompressing unit <b>23</b> in the receiver <b>2</b> will be described.

0192In the multi-dimensional stream data decompressing unit <b>23</b> in the receiver <b>2</b>, compressed stream data <b>MSC1</b> to <b>MSC3</b> for one block of the data packet are decompressed by a different method depending upon compression information represented by the compression identifier. For example, the multi-dimensional stream data decompressing unit <b>23</b> may determine a decompression method based on compression information. The multi-dimensional stream data decompressing unit <b>23</b> may determine a decompression ratio for decompressing compressed motion data based on compression information.

0193In the case where compression in a time axis direction in one block and compression for decreasing precision of a value of each component (e.g., 4 bytes of data is compressed to 2 bytes of data) are performed in the position vector data <b>MSC1</b> at a reference position of a three-dimensional CG character, the multi-dimensional stream data decompressing unit <b>23</b> first performs decompression (2 bytes of data is decompressed to 4 bytes of data by reverse conversion of compression conversion) with respect to the compression for decreasing precision of a value of each component. Then, with respect to the compression in a time axis direction, the multi-dimensional stream data decompressing unit <b>23</b> performs decompression of one block of stream data by three-dimensional spline interpolation, linear interpolation, or the like, using a compression identifier in the header portion of the channel definition packet.

0194The multi-dimensional stream data decompressing unit <b>23</b> converts the direction vector data <b>MSC2</b> from the polar coordinates to the three-dimensional vector coordinates. For example, in the case where the direction vector data <b>MSC2</b> representing the posture of a three-dimensional CG character has two polar coordinates components representing a rotation axis and is received as stream data, the multi-dimensional stream data decompressing unit <b>23</b> may convert the position vector data <b>MSC2</b> from the polar coordinates to the three-dimensional vector coordinates to obtain a direction vector representing a rotation axis with a magnitude 1.

0195In the angle vector at each joint of a three-dimensional CG character and rotation angle around the angle vector <b>MSC3,</b> in the same way as in the conversion from the polar coordinates in the direction vector representing the posture to the three-dimensional vector coordinates, the multi-dimensional stream data decompressing unit <b>23</b> first obtains three three-dimensional vector coordinates from two polar coordinates. Then, in the same way as in the decompression of stream data in a time axis direction in the position vector <b>MSC1</b> at the reference position, the multi-dimensional stream data decompressing unit <b>23</b> performs decompression of one block of stream data by three-dimensional spline interpolation, linear interpolation, or the like, using a compression ratio identifier in a header portion of the channel definition packet.

0196After the decompression of multi-dimensional stream data is completed in the multi-dimensional stream data decompressing unit <b>23</b>, the multi-dimensional stream data storing unit <b>32</b> stores the decompressed multi-dimensional stream data. In the case where a three-dimensional CG reproduction unit provided outside requests stream data, stream data is supplied from the multi-dimensional stream data storing unit <b>32,</b> whereby the receiver <b>2</b> can reproduce three-dimensional CG scene in real time.

0197Multi-buffering and synchronous reproduction of multi-dimensional stream data and a method for reproducing data in real time in the receiver <b>2</b> will be described in detail.

0198Hereinafter, the case where one motion stream data and one voice stream data of a three-dimensional CG character are buffered when the receiver <b>2</b> is provided with a reproduction unit (not shown) will be described.

0199Figure <b>13</b> shows reproduction of a motion stream and a voice stream in the case where the receiver <b>2</b> is provided with a reproduction unit. A reproduction process and a buffering process are performed with respect to the motion stream and the voice stream.

0200According to the reproduction process for reproducing a motion stream during one block of period (time <b>t</b> to time <b>t'</b>), a reproduction unit reproduces the motion of a three-dimensional CG character from buffered motion stream data, in accordance with time stamps of the motion stream data. According to the buffering process of a motion stream, the receiver <b>2</b> receives motion stream data, for example, in a block following the block to which the motion stream data which is currently being reproduced belongs, i.e., in a block corresponding to a period (time <b>t'</b> to time <b>t''</b>). The multi-dimensional buffering unit <b>22</b> buffers the received motion stream data. The multi-dimensional stream data identifying unit <b>21</b> analyzes motion stream data, and the multi-dimensional stream data decompressing unit <b>23</b> decompresses motion stream data. In order to prepare for the reproduction of the motion stream data (data reproduction in the subsequent block), the multi-dimensional buffering unit <b>22</b> buffers the decompressed motion stream data again.

0201Voice stream data is also subjected to the above-mentioned reproduction process and buffering process. Furthermore, even in the case where multiplicity of buffering is increased, multi-buffering can be realized by a similar operation.

0202In the case where the transmission speed of stream data from the receiver <b>2</b> to the reproduction unit which reproduces three-dimensional CG data is higher than the buffering speed, i.e., in the case where stream data cannot be received and decompressed before being transmitted to the reproduction unit, the receiver <b>2</b> requests the transmitter <b>1</b> of compressed multi-dimensional stream data having a low data amount and an optimum compression ratio for data reproduction, and the receiver <b>2</b> receives data, whereby real time data reproduction can be maintained.

0203A minimum structure of an apparatus for transmitting and receiving multi-dimensional stream data (i.e., the case where one receiver <b>2</b> is connected to one transmitter <b>1</b>) has been described. However, as shown in Figure <b>14</b>, a plurality of receivers <b>2</b> may be connected to one transmitter <b>1</b> via a transmission portion.

0204As described above, the transmitter <b>1</b> may be provided with a stream authoring unit (not shown). Hereinafter, the basic operation of the stream authoring unit mentioned with reference to Figure <b>4</b> will be briefly described. The stream authoring unit cuts motion stream data and voice stream data of a three-dimensional CG character having an arbitrary data length into an arbitrary length required for synchronization. The data length of the motion stream data is determined based on the duration of an operation, and the data length of the voice stream data is determined based on the reproduction time of a voice. As shown in Figure <b>15,</b> the stream authoring unit gives time stamp information required for synchronization to the motion stream and the voice stream. Furthermore, the stream authoring unit connects the motion stream to the voice stream, if required, and generates a new stream. The stream authoring unit registers the stream data thus edited in the transmitter <b>1</b> as input data.

0205In the case where the receiver <b>2</b> shown in Figure <b>4</b> has a reproduction unit, the motion stream is preferably reproduced in synchronization with the reproduction of the voice stream, based on time stamp information of the motion stream and time stamp information of the voice stream. Furthermore, the voice stream may be reproduced in synchronization with the reproduction of the motion stream, based on each time stamp.

0206Hereinafter, the basic operation of the apparatus for transmitting and receiving multi-dimensional stream data in the present embodiment will be described with reference to Figures <b>16</b> and <b>17.</b>

0207Figure <b>16</b> is a flow chart illustrating the operation of the transmitter <b>1</b>, and Figure <b>17</b> is a flow chart illustrating the operation of the receiver <b>2</b>.

0208First, an exemplary basic operation of the transmitter <b>1</b> will be described.

0209At Step <b>S11</b> in Figure <b>16</b>, stream data <b>MS1</b> to <b>MS3</b> and <b>SS</b> created in the external stream data authoring unit are input to the transmitter <b>1</b>.

0210At Step <b>S12</b>, the transmitter <b>1</b> identifies the stream data <b>MS1</b> to <b>MS3</b> and <b>SS</b> input at Step <b>S11,</b> and compresses the stream data to obtain compressed stream data <b>MSC1</b> to <b>MSC3</b> and <b>SSC.</b> The transmitter <b>1</b> generates and stores multi-dimensional stream data packets.

0211At Step <b>S13,</b> the transmitter <b>1</b> transmits a channel definition packet to the receiver <b>2</b>.

0212At Step <b>S14,</b> the transmitter <b>1</b> determines whether or not the receiver <b>2</b> requests that the channel definition should be changed. In the case where the channel definition should be changed, the process returns to Step <b>S13.</b>

0213At Step <b>S15</b>, the transmitter <b>1</b> transmits a data packet to the receiver <b>2</b>.

0214At Step <b>S16</b>, the transmitter <b>1</b> determines whether or not the amount of data which flows through a transmission path is excessive or insufficient. In the case where the amount data to be transmitted exceeds an allowable amount in the transmission path, or in the case where the amount of data to be transmitted is below the allowable amount, or in the case where the receiver <b>2</b> requests that data to be transmitted should be changed, the process proceeds to Step <b>S17.</b> In the case where the transmitter <b>1</b> determines that the amount of data is not excessive or insufficient, the process proceeds to Step <b>S14</b>.

0215At Step <b>S17,</b> the transmitter <b>1</b> selects stream data with an optimum compression ratio consistent with the state of the transmission path, or stream data with an optimum compression ratio in accordance with a request from the receiver <b>2</b>. Then, the process returns to Step <b>S14.</b>

0216Next, an example of the operation of the receiver <b>2</b> will be described.

0217At Step <b>S21</b> in Figure <b>17,</b> the receiver <b>2</b> receives the multi-dimensional stream data packet from the transmitter <b>1</b>.

0218At Step <b>S22</b>, the receiver <b>2</b> determines whether or not the stream data packet received at Step <b>21</b> is a channel definition packet. In the case where the received packet is a channel definition packet, the process returns to Step <b>S21</b>.

0219At Step <b>S23</b>, the receiver <b>2</b> analyzes the data packet and data.

0220At Step <b>S24,</b> the receiver <b>2</b> determines whether or not data is present in a buffer. In the case where data is not present, the process returns to Step <b>S21</b>.

0221At Step <b>S25</b>, the receiver <b>2</b> decompresses one block of stream data present in the buffer by the number of kinds of required stream data.

0222At Step <b>S26</b>, the receiver <b>2</b> determines whether or not one block of data has been decompressed at Step <b>S25</b>. In the case where one block of data has been decompressed, the receiver <b>2</b> transmits the decompressed data to a reproduction unit. That is, the process proceeds to Step <b>27</b>. In the case where one block of data has not been decompressed, the process proceeds to Step <b>S28</b>.

0223At Step <b>S27</b>, the receiver <b>2</b> transmits the decompressed stream data to a reproduction unit. At Step <b>S27</b>, the receiver <b>2</b> determines whether or not a final flag has been generated by a user. In the case where no final flag has been generated, the process returns to Step <b>S21</b>. In the case where a final flag has been generated, the receiver <b>2</b> terminates its operation.

0224In Embodiment 2, the transmitter may include an artificial satellite, and the receiver may be a digital TV.

0225In Embodiment 2, the transmitter may include an artificial satellite, and the receiver may be a set top box.

0226The artificial satellite receives the multi-dimensional stream data from a ground station.

0227In Embodiment 2, the transmitter may include a ground wave broadcasting station, and the receiver may be a digital TV.

0228In Embodiment 2, the transmitter may include a ground wave broadcasting station, and the receiver may be a set top box.

0229In Embodiment 2, the transmitter may include a wire broadcasting station, and the receiver may be a digital TV.

0230In Embodiment 2, the transmitter may include a wire broadcasting station, and the receiver may be a set top box.

0231A program for allowing a computer to execute the above-mentioned multi-dimensional stream data transmitting method and/or receiving method may be recorded in a recording medium capable of being read by a computer.

0232According to the present invention, motion data, voice data, and the like of a three-dimensional CG character are separately compressed depending upon the kind, and the compressed data is transferred as a stream in real time, while information required for reproducing each data in synchronization is simultaneously transferred. Thus, a smooth motion of the three-dimensional CG character and sound information in synchronization with the motion can be interactively transmitted and received in accordance with a user's request in a transmitting/receiving system based on a network, and the amount of data to be transferred can be substantially reduced.

0233Various other modifications will be apparent to and can be readily made by those skilled in the art without departing from the scope and spirit of this invention. Accordingly, it is not intended that the scope of the claims appended hereto be limited to the description as set forth herein, but rather that the claims be broadly construed.