Apparatus and method for transmitting packet for multimedia streaming service

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the priority of Korean Patent Application No. 2002-14847, filed Mar. 19, 2002 in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

[0002] 1. Field of the Invention

[0003] The present invention relates to an apparatus and a method to transmit a packet for a multimedia streaming service, and more particularly, to an apparatus and a method for transmitting a packet for a multimedia streaming service which applies a modification of a coding format to a packetizing process according to a state of a network.

[0004] 2. Description of the Related Art

[0005] Encoded contents are necessary to provide a multimedia data streaming service. Such contents or data are encoded in advance by using authoring tools. The encoded data is stored in a storing device and connected to a database and transmitted to a plurality of terminals upon a service request of a streaming terminal.

[0006] Here, a state of a channel where a packet loss occurs may not be identical in all the terminals. Even in the case of terminals having channels of identical state, variations may be made with time. Accordingly, problem of packet loss cannot be properly dealt with if the encoded contents are coded without considering the packet loss, and the streaming service is not normally provided until the channel returns to a normal state.

[0007] In general, video contents are coded in frame units in consideration of a storage efficiency, and key frames are maintained in certain units among the coded frames units in consideration of the packet loss.

[0008] Such key frames occupy large storage spaces, thereby reducing storage efficiency. Therefore, predictive frames, which can be predicted by the key frames, are coded between the key frames to improve the storage efficiency. The coded predictive frames and the key frames are packetized and transmitted. Here, the coded predictive frames and the key frames are divided into a plurality of packets and then transmitted to the network.

[0009] However, when the packet loss occurs during the transmission of the coded predictive frames and the key frames, the loss of one packet is equivalent to the loss of one frame which includes the plurality of packets. As a result, the data can be restructured after reception of the succeeding key frame, and the video contents are stopped on the screen.

SUMMARY OF THE INVENTION

[0010] Various aspects and advantages of the invention will be set forth in part in the description that follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

[0011] According to an aspect of the present invention, there is provided an apparatus and a method to transmit a packet for a multimedia streaming service which can improve storage efficiency and transmission efficiency by modifying a coding format of a packetizing process according to a state of a network, when a server provides the multimedia streaming service access to the Internet or wireless Internet and contents having same image quality are transmitted to a plurality of terminals.

[0012] According to an aspect of the present invention, there is provided an apparatus to transmit a packet to provide a multimedia streaming service to one or more terminals connected through a network, including: a contents producing unit encoding and compressing multimedia data into bit streams; a contents storing unit storing the bit streams compressed by the contents producing unit; and a packetizing unit packetizing the bit streams stored in the contents storing unit in a predetermined coding format, and packetizing the bit streams in a different coding format when a state of the network changes.

[0013] More specifically, when the network has an abnormal state, the packetizing unit packetizes the bit streams in a packet resilient coding format to be resilient from a packet loss. And, the state of the network is notified from the one or more terminals.

[0014] According to another aspect of the present invention, there is provided an apparatus to transmit a packet to provide a multimedia streaming service to one or more terminals connected through a network, including: a contents producing unit encoding and compressing multimedia data into bit streams; a contents storing unit storing the bit streams compressed by the contents producing unit; a network monitoring unit notifying changes of a state of the network connected to the one or more terminals; and a packetizing unit packetizing the bit streams stored in the contents storing unit in a predetermined coding format, and packetizing the bit streams in a different coding format corresponding to the state of the network.

[0015] More specifically, when the network has an abnormal state, the packetizing unit packetizes the bit streams in a packet resilient coding format to be resilient from a packet loss. And, when the packetizing unit modifies the coding format into a different coding format, the packetizing unit confirms whether the coding format enables the bit streams to be decoded in the one or more terminals, and when the coding format enables data to be decoded in the one or more terminals, the packetizing unit generates and transmits packets in the modified coding format

[0016] According to another aspect of the present invention, there is provided method of transmitting a packet to provide a multimedia streaming service to one or more terminals connected through a network, including: informing the one or more terminals of contents information comprising coding formats and playback time of contents; receiving a coding request from the one or more terminals to perform a coding process in one of the coding formats according to a state of the network; and packetizing and transmitting bit streams in the requested coding format to the one or more terminals.

[0017] More specifically, when the network has an abnormal state in the receiving of the coding request, the coding format is modified into a packet resilient coding format to be resilient from a packet loss,

[0018] According to another aspect of the present invention, there is provided a method of transmitting a packet to provide a multimedia streaming service to one or more terminals connected through a network, including: packetizing bit streams in a different coding format when a state of the network connected to the one or more terminals changes, and confirming whether the different coding format is a coding format to enable the bit streams to be decoded in the one or more terminals; and generating and transmitting packets in the different coding format when the different coding format enables the data to be decoded in the one or more terminals

[0019] More particularly, the network has an abnormal state, the different coding format is a packet resilient coding format to be resilient from a packet loss

[0020] According to an aspect of the present invention, there is provided a packet transmitting method for providing a multimedia streaming service to one or more terminals connected through a network, including: packetizing a packet to be transmitted having a descriptor field that describes a coding format of an inner payload, generating the packet according to another coding format; and transmitting the generated packet of another coding format to the one or more terminals

[0021] According to an aspect of the present invention, there is provided a packet receiving method for providing a multimedia streaming service to one or more terminals connected through a network, including: receiving a packet having a descriptor field indicating a coding format of an inner payload, wherein the received packet is packetized in the another coding format; and decoding, where the packet in the another coding format is de-packetized.

[0022] More specifically, the another coding format is a coding format without a data partitioning to deal with packet loss, or a coding format applied with the data partitioning to deal with the packet loss. And, when the another coding format is of data division type, the field describes as to whether the packet contains one of header region and text region.

[0023] According to an aspect of the present invention, there is provided an apparatus to transmit a packet for a multimedia streaming service to improve storage efficiency and transmission efficiency of one or more terminals connected to a server through a network, including: a packetizing unit modifying a coding format of a packetizing process according to a state of the network; and a server providing the multimedia streaming service access to the Internet or wireless Internet and transmitting contents having same image quality to the one or more terminals.

[0024] More specifically, the server transmits bit streams to the one or more terminals through the network using a real time streaming protocol (RTSP).

[0025] And the server comprises: a contents producing unit encoding multimedia data using a standardized compression method for the multimedia streaming service, a contents storing unit storing bit streams compressed by the contents producing unit, and a network interface unit transmitting the bit streams to network media or receiving the bit streams from the one or more terminals in the network. Wherein the packetizing unit packetizes bit streams in a predetermined coding format.

[0026] According to an aspect of the present invention, there is provided a method of transmitting a packet to provide a multimedia streaming service to one or more terminals connected to a server through a network, including: transmitting a describe command from the one or more terminals to the server to obtain contents information; transmitting the contents information from the server to the one or more terminals; packetizing and transmitting bit streams in a coding format from the server to the one or more terminals; decoding the bit streams in a format corresponding to the coding format; modifying the coding format into a packet resilient coding format to be resilient from a packet loss; packetizing the bit streams in the modified coding format; and transmitting the packetized bit streams to the one or more terminals.

[0027] Additional advantages, aspects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The aspects and advantages of the invention may be realized and attained as particularly pointed out in the appended claims.

[0028] More specifically, determining an abnormal state of the network when RTP packets are not normally transmitted from the server to the one or more terminals and the packet loss increases, and determining a normal state of the network when the RTP packets are normally transmitted.

BRIEF DESCRIPTION OF THE DRAWINGS

[0029] These and/or other aspects and advantages of the invention will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

[0030]FIG. 1 is a structure view illustrating a network system using an apparatus to transmit a packet for a multimedia streaming service, in accordance with a first aspect of the present invention;

[0031]FIG. 2 is a flowchart showing a method to transmit the packet for the multimedia streaming service, in accordance with the first aspect of the present invention;

[0032]FIG. 3 is a structure view illustrating a network system using the apparatus to transmit the packet for the multimedia streaming service, in accordance with a second aspect of the present invention; and

[0033]FIG. 4 is a flowchart showing a method to transmit the packet for the multimedia streaming service, in accordance with the second aspect of the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0034] Reference will now be made in detail to the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout. The embodiments are described below in order to explain the present invention by referring to the figures.

[0035]FIG. 1 is a structure view illustrating a network system using an apparatus to transmit a packet for a multimedia streaming service, in accordance with a first aspect of the present invention.

[0036] Referring to FIG. 1, a server <b>10</b> transmits data streams to terminals <b>20</b> through a network <b>30</b> by using a real time streaming protocol (RTSP). The server <b>10</b> includes a contents producing unit <b>11</b>, a contents storing unit <b>12</b>, a packetizing unit <b>13</b>, and a network interface unit <b>14</b>.

[0037] The contents producing unit <b>11</b> encodes multimedia data by using an encoder (not shown) for the multimedia streaming service. The multimedia data can be encoded in a standardized compression method such as MPEG-4.

[0038] The contents storing unit <b>12</b> stores bit streams compressed by the contents producing unit <b>11</b>. The network interface unit <b>14</b> transmits the data to network media or receives the data from the terminals <b>20</b> in the network <b>30</b>.

[0039] The packetizing unit <b>13</b> packetizes the bit streams in a predetermined coding format. In the case of MPEG-4, the coding formats are divided into a coding format to code one general frame into a whole and a coding format using a data partitioning method.

[0040] In the first aspect according to the present invention, when the server <b>10</b> is connected to the terminal <b>20</b>, if the terminal <b>20</b> transmits a describe command to the server <b>10</b>, the server <b>10</b> transmits contents information, such as the coding formats and a playback time of the contents to the terminal <b>20</b>. Accordingly, when a state of the network <b>30</b> is changed, the terminal <b>20</b> adaptively selects the coding format according to the state of the network <b>30</b> and requests the selected coding format to the server <b>10</b>. The packetizing unit <b>13</b> packetizes the bit steams in the coding format requested by the terminal <b>20</b>.

[0041]FIG. 2 is a flowchart showing a method to transmit the packet for the multimedia streaming service, in accordance with the first aspect of the present invention.

[0042] At operation S<b>102</b>, when the terminal <b>20</b> is connected to the server <b>10</b>, the terminal <b>20</b> transmits the describe command to the server <b>10</b> to obtain the contents information. At operation S<b>104</b>, the server <b>10</b> transmits the contents information such as the coding formats and the playback time of the contents to the terminal <b>20</b>. Here, the server <b>10</b> provides or informs of at least two types of coding formats and the terminal <b>20</b> recognizes that the corresponding contents can be coded in at least two coding formats.

[0043] At operation S<b>105</b>, the server <b>10</b> packetizes and transmits the bit streams in a general coding format to the terminal <b>20</b>. At operation S<b>106</b>, the terminal <b>20</b> decodes the transmitted data in a decoding format suitable for the coding format and monitors the state of the network <b>30</b>. Here, when RTP packets are not normally transmitted from the server <b>10</b> to the terminal <b>20</b> and a packet transmission loss is increased, the network <b>30</b> is deemed to have an abnormal state, and when the RTP packets are normally transmitted, the network <b>30</b> is deemed to have a normal state.

[0044] At operation S<b>108</b>, when monitoring the abnormal state of the network <b>30</b>, at operation S<b>110</b>, the terminal <b>20</b> requests the server <b>10</b> to modify the coding format into a packet resilient coding format to be resilient from the packet loss. At operation S<b>112</b>, the server <b>10</b> modifies the coding format into the packet resilient coding format, packetizes the bit streams in the modified format, and transmits the packetized bit streams i.e., the multimedia streams, to the corresponding terminals <b>20</b>.

[0045] On the other hand, the method goes to operation S<b>106</b> to decode the transmitted data in a decoding format suitable for the coding format of transmitted data and monitors the state of the network <b>30</b>. At operation S<b>108</b>, when monitoring the normal state of the network <b>30</b>, at operation S<b>110</b>, the terminal <b>20</b> requests the server <b>10</b> to modify the coding format into the previous coding format, and, at operation S<b>112</b>, the server <b>10</b> packetizes and transmits the bit streams in the previous coding format.

[0046]FIG. 3 is a structure view illustrating a network system using the apparatus to transmit the packet for the multimedia streaming service, in accordance with a second aspect of the present invention.

[0047] In FIG. 3, same drawing reference numerals are used for the same elements as FIG. 1, and, thus, explanations thereof are omitted.

[0048] As illustrated in FIG. 3, a server <b>40</b> includes the contents producing unit <b>11</b>, the contents storing unit <b>12</b>, a packetizing unit <b>43</b>, the network interface unit <b>14</b> and a network monitoring unit <b>45</b>.

[0049] The packetizing unit <b>43</b> packetizes the bit streams in a predetermined coding format. According to an aspect of the present invention, the coding format can be modified according to the state of the network <b>30</b>. In the second aspect of the present invention, when feedback is delayed between terminal <b>50</b> and the server <b>40</b>, the network monitoring unit <b>45</b> monitors the abnormal state of the network <b>30</b> due to an increased packet loss and informs or flags the packetizing unit <b>43</b> of the abnormal state of the network <b>30</b>.

[0050] When recognizing the abnormal state of the network <b>30</b> by the network monitoring unit <b>45</b>, the packetizing unit <b>43</b> modifies the coding format and packetizes the bit streams in the modified coding format. Here, the packetizing unit <b>43</b> informs the terminal <b>50</b> of the coding format to confirm whether the coding format enables the data to be decoded in the terminal <b>50</b>.

[0051] When normally receiving a response signal for the coding format from the terminal <b>50</b>, the packetizing unit <b>43</b> recognizes that the coding format enables the data to be decoded in the terminal <b>50</b>, and packetizes and transmits the bit streams in a new coding format, for example, a packet resilient coding format. When the network <b>30</b> has the normal state, the packetizing unit <b>43</b> generates and transmits the packets in the previous coding format.

[0052]FIG. 4 is a flowchart showing a method to transmit the packet for the multimedia streaming service, in accordance with the second aspect of the present invention.

[0053] At operation S<b>201</b>, the server <b>40</b> packetizes and transmits the bit streams in the general coding format to the terminal <b>50</b>. In addition, at operation S<b>202</b>, the server <b>40</b> monitors the state of the network <b>30</b> connected to the terminal <b>50</b>. At operation S<b>204</b>, when the network <b>30</b> is deemed to have the abnormal state, at operation S<b>206</b>, the server <b>40</b> notifies the terminal <b>50</b> of a new coding format for modification, that is, to packetize the bit streams in the new coding format. At operation S<b>207</b>, the terminal <b>50</b> transmits a feedback signal to the notified signal of the server <b>40</b>.

[0054] At operation S<b>208</b>, when normally receiving the response signal from the terminal <b>50</b>, at operation S<b>210</b>, the server <b>40</b> recognizes that the corresponding coding format enables the data to be decoded in the terminal <b>50</b>, and packetizes and transmits the bit streams in the notified coding format. Thereafter, at operation S<b>212</b>, the terminal <b>50</b> decodes the transmitted data in a suitable decoding format.

[0055] In addition, at operation S<b>202</b>, the server <b>40</b> monitors the state of the network <b>30</b>. When monitoring the abnormal state of the network <b>30</b>, at operation S<b>204</b>, the server <b>40</b> goes to operation S<b>201</b> to packetize the bit streams in the previous coding format. On the other hand, at operation S<b>208</b>, when the server <b>40</b> does not receive, or does receive but abnormally receive the response signal from the terminal <b>50</b>, the server <b>40</b> goes to operation S<b>206</b> to re-notify the coding format for modification to the terminal <b>50</b>, or goes to operation S<b>201</b> to packetize the bit streams in the previous coding format.

[0056] Modifications of the coding format in the first and second aspects of the invention will now be explained in detail by exemplifying video data.

[0057] In the case where MPEG-4, or H.263 is used as a video data compression format, there are mainly two coding formats that are used. One format is a bit stream coding which transforms the respective macro blocks of one frame into a series of bit streams in accordance with a predetermined coding syntax, and the other format is a data partitioning coding which reorders the macro blocks according to header information, a motion vector, or text information by using a data partitioning method.

[0058] Because the two coding formats use the same tables, coding can be efficiently performed through the transformation of the packetized payload into bit stream data. As for the motion vector, however, computations for re-definition of values may be required when the coding format changes.

[0059] In the case of changing the bit stream coding method that codes one frame into a series of bit streams, to a data partitioning coding that codes by a predetermined size of the packet, it is the first motion vector of the inner payload of the packet that is required to be re-defined. The first motion vector is required to be re-defined because the first motion vector is a difference of the estimated motion vector and an actual motion vector that is used in the bit stream for coding.

[0060] By the same token, it is the first motion vector of the payload that is applied with a different estimation value when the coding format is changed from the bit stream coding format to the data partitioning coding format. In order to deal with such situation that accompanies the change of coding formats, an actual value that can be identified without requiring estimation thereof, is positioned in a beginning of the inner payload of the packet together with a start macro block number for future use, i.e., for use in the change of coding format. Accordingly, decoding can be efficiently performed.

[0061] In addition, when a function of inserting a macro block refresh into a packet of a predetermined length is added to refresh the macro block during the encoding process, it is possible to efficiently handle the packet loss and it is more efficient in the modification of the coding format of the bit streams.

[0062] When the coding format changes according to the situation of the network <b>30</b>, the bit streams coded by the above two coding formats are packetized. In other words, packetization is performed such that the packet to be transmitted includes a field that indicates by which coding format that an associated inner payload has been coded. As for the coding formats such indicated, as described above, there is the bit stream coding format which generates bit streams of the sequential macro blocks without using data partitioning, and the data partitioning coding format which generates bit streams re-ordered by the data partitioning. Also, in order to effectively deal with the packet loss problem of the data partitioning coding, whether the header region is a region that was partitioned by the data partitioning, or a text region that includes a coefficient value may be described.

[0063] In other words, as the server monitors the state of the network <b>30</b>, the coding format is changed in accordance with the state of the network <b>30</b>. Accordingly, changed coding format is applied in the packetizing process, and the packets generated by such changed coding format are transmitted to the terminal. A receiving terminal may recognize the changed coding format of the received packets either from notification by the server, or by decoding the descriptor field that describes the coding format of the received packets. Accordingly, the receiving terminal can decode the received packets in accordance with the changed coding format. Through such depacketizing process and the decoding of the bit stream in accordance with the coding format, the receiving terminal can recover the image.

[0064] By the coding and packet transmitting apparatus and method thereof for multimedia streaming service that applies a change of coding format into a packetizing process according to an aspect of the present invention, when identical contents stored in a server are transmitted to a plurality of terminals, the data packets, which are coded in different coding formats according to a state of a network connected to the respective terminals, are transmitted. As a result, the current content can be generated as if pluralities of contents are stored according to the coding formats. In addition, storage efficiency is considerably improved.

[0065] While the invention has been shown and described with reference to certain preferred aspects thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

[0066] The foregoing aspects and advantages are merely exemplary and are not to be construed as limiting the present invention. The present teaching can be readily applied to other types of apparatuses. The description of the present invention is intended to be illustrative, and not to limit the scope of the claims. Many alternatives, modifications, and variations will be apparent to those skilled in the art. In the claims, means-plus-function clauses are intended to cover the structures described herein as performing the recited function and not only structural equivalents but also equivalent structures.