Apparatus and method for transmitting multimedia data in hybrid network

CA 02852002 2014-04-11

WO 2013/055176 PCT/KR2012/008377

Description

Title of Invention: APPARATUS AND METHOD FOR TRANSMITTING MULTIMEDIA DATA IN HYBRID NETWORK Technical Field

[1] The present invention relates to an apparatus and a method for transmitting multimedia data in a hybrid network. More particularly, the present invention relates to an apparatus and a method for transmitting multimedia data by applying a Motion Pictures Experts Group (MPEG) Media Transport (MMT).

Background Art

[2] As multimedia services have increased, Motion Pictures Experts Group (MPEG) Media Transport (MMT) technology has been developed for providing a hybrid network that simultaneously connects a broadcasting network and a communication network and hybrid contents in which multimedia data, an application, and files are simultaneously provided. The MMT technology is a multimedia transmission technology based on MPEG technology, may be used for broadcasting data for viewing by devices including and similar to a multifunctional smart TeleVision (TV), a multi-view TV and a multimedia service, and may provide an efficient MPEG transmission technology in a multimedia service environment that changes according to an Internet Protocol (IP) address.

Disclosure of Invention

Technical Problem

[3] However, when multimedia data is transmitted in the current hybrid network, it is difficult to provide a service suitable for the hybrid network environment.

[4] Therefore, a need exists for a system and method for an apparatus and a method for transmitting multimedia data by applying MMT.

Solution to Problem

[5] Aspects of the present invention are to address at least the above-mentioned problems and/or disadvantages and to provide at least the advantages described below. Accordingly, an aspect of the present invention is to provide an apparatus and a method for transmitting multimedia data in a hybrid network.

[6] Another aspect of the present invention is to provide an apparatus and a method for generating Motion Pictures Experts Group (MPEG) Media Transport (MMT) Transport Packet (TP) data and transmitting multimedia data in a hybrid network.

[7] Another aspect of the present invention is to provide an apparatus and a method for defining MMT TP data header information and payload data information and transmitting multimedia data.

WO 2013/055176 PCT/KR2012/008377

[8] Another aspect of the present invention is to provide an apparatus and a method for defining information required for transmitting an MMT TP and transmitting multimedia data.

[9] Another aspect of the present invention is to provide an apparatus and a method for guaranteeing Quality of Experience (QoE) for an MMT TP and transmitting multimedia data.

[10] In accordance with an aspect of the present invention, an apparatus for transmitting data in a hybrid network that simultaneously provides broadcasting and Internet communication is provided. The apparatus includes a payload part including at least one of first data synchronized according to a time to be reproduced and second data other than the first data, a first header part including information on the payload part, a packet part for generating a packet, the packet part including a second header part including information on the packet, and a transceiver for generating and transmitting a signal including the payload part, the first header part, and the packet part to the hybrid network.

[11] In accordance with another aspect of the present invention, a method of transmitting data in a hybrid network that simultaneously provides broadcasting and Internet communication is provided. The method includes generating a payload part including at least one of first data synchronized according to a time to be reproduced and second data other than the first data, generating a first header part including information on the payload part, generating a packet, the packet including a second header part including information on the packet, and transmitting the packet to the hybrid network.

[12] Exemplary embodiments of the present invention may provide header information of an MMT transport packet and payload information for transmitting hybrid multimedia data in a hybrid network, and thus provide a service suitable for a hybrid network environment through application of an MMT technology.

[13] Other aspects, advantages, and salient features of the invention will become apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the annexed drawings, discloses exemplary embodiments of the invention.

Brief Description of Drawings

[14] The above and other aspects, features, and advantages of certain exemplary embodiments of the present invention will be more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[15] FIG. 1 illustrates an apparatus for transmitting hybrid multimedia contents according to an exemplary embodiment of the present invention; and

[16] FIG. 2 illustrates a method of transmitting hybrid multimedia contents according to

CA 02852002 2014-04-11

WO 2013/055176 PCT/KR2012/008377 an exemplary embodiment of the present invention

[17] Throughout the drawings, like reference numerals will be understood to refer to like parts, components, and structures.

Mode for the Invention

[18] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of exemplary embodiments of the invention as defined by the claims and their equivalents. It includes various specific details to assist in that understanding but these are to be regarded as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. In addition, descriptions of well-known functions and constructions may be omitted for clarity and conciseness.

[19] The terms and words used in the following description and claims are not limited to the bibliographical meanings, but, are merely used by the inventor to enable a clear and consistent understanding of the invention. Accordingly, it should be apparent to those skilled in the art that the following description of exemplary embodiments of the present invention is provided for illustration purpose only and not for the purpose of limiting the invention as defined by the appended claims and their equivalents.

[20] It is to be understood that the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to “a component surface” includes reference to one or more of such surfaces.

[21] According to exemplary embodiments of the present invention, a hybrid network environment may be an environment having a network connection between a server and a client where a dedicated network for broadcasting, hereinafter, referred to as a “broadcast” network, and a network for Internet communication, hereinafter, referred to as a “broadband” network. The hybrid network environment may be simultaneously provided to the same client and the client may receive a service through a plurality of networks. Further, a plurality of network environments may include a plurality of connection environments including a broadcast network, a broadcast environment, a broadband network, and a broadband environment. The plurality of network environments may be configured by hybridly connecting two environments, and may include a wired network and/or wireless network.

[22] Furthermore, hybrid multimedia contents, according to the present exemplary embodiments, may refer to contents which may be hybridly used because the hybrid multimedia contents contain both multimedia contents reproduced with temporal, i.e., real-time, characteristics, such as an audio signal, a video, a synchronized caption, and an image and data with non-real time characteristics such as an audio book, a

CA 02852002 2014-04-11

WO 2013/055176

PCT/KR2012/008377

CA 02852002 2014-04-11 multimedia widget, a file and/or other similar contents and data.

[23] FIG. 1 illustrates an apparatus for transmitting hybrid multimedia contents according to an exemplary embodiment of the present invention.

[24] An apparatus 10 for transmitting the hybrid multimedia contents according to an exemplary embodiment of the present invention may transmit the hybrid multimedia contents by using an Motion Pictures Experts Group (MPEG) Media Transport (MMT) technology. A transport layer of the MMT may configure a transport structure according to data contained in the hybrid multimedia contents and may serve a function for efficiently transmitting the data. To this end, the MMT transport layer may include a structure where common transport information and distinguishing transport information are discriminated.

[25] Referring to FIG. 1, an MMT payload of the transmission apparatus 10 for transmitting multimedia contents may include a streaming payload 150 for transmitting real time data, for example, real time broadcasting, and a file transfer pay load 160 for transmitting non-real time data. The streaming payload 150 may be configured using at least one of an MMT Media-unit 110, hereinafter, referred to as an “M-unif ’ 110, and an MMT Media Fragment Unit 100, hereinafter, referred to as an “MFU” 100, of the hybrid multimedia contents as a basic unit transmitted for real time data transmission. In a video which may be independently decoded, the MFU 100 may refer to a frame of an encoding or one slice when the frame includes a plurality of slices. Further, in a video which may be independently decoded, the M-unit 110 refers to a frame of an encoding or a Group of Picture (GOP) unit corresponding to an encoding handling unit which may be independently decoded. The MFU 100 or the M-unit 110 may determine its structure by considering an attribute and a service of the hybrid media. The file transfer pay load 160 may be used for non-real time data transmission, and all of, or a particular part unit of, an MMT asset 120, hereinafter, referred to as an “asset” 120, or an MMT package 130, hereinafter, referred to as a “package” 130, of the hybrid multimedia contents may be configured as a basic unit for transmission.

[26] The asset 120 is consecutive streams of the M-units 110, all of which may be configured in the same media type. Here, the M-units 110 configured in the same media type may include an audio M-unit, a video M-unit, an audio/video M-unit, and any other similar media type M-units. Furthermore, the package 130 may include additional information for a service in a single asset 120 or a plurality of assets 120. At this time, the additional information may include transmission environment information considering a hybrid network environment related to screen configuration information of the hybrid multimedia, for example, temporal and spatial arrangement orders, a transmission requirement bandwidth, information on whether loss is possible, information on whether a delay time is accepted and any other similar and/or suitable

WO 2013/055176

PCT/KR2012/008377 information.

[27] The transmission apparatus 10 may configure the payload in units of MFUs 100 or M-units 110 and may transmit multimedia contents in a service using real time transmission, for example, real time broadcasting, and may configure the payload in units of assets 120 or packages 130. The transmission apparatus 10 may also transmit multimedia contents in a service using non-real time transmission, for example, a Video on Demand (VoD) service using a broadcasting network. The streaming payload 150 for real time data transmission and the file transfer payload 160 for non-real time data transmission may be included in an MMT transport packet 140 operating in a Transmission Control Protocol (TCP) / User Datagram Protocol (UDP) 170 environment corresponding to an Internet Protocol (IP) based address or a payload of an Real-time Transport Protocol (RTP) packet 180 corresponding to an IP address, and then transmit such to a terminal.

[28] Meanwhile, the MMT payload, e.g., the streaming payload 150, and the file transfer pay load 160 may be configured by a header region including configuration information used according to a transmission environment and a data region that includes transmission data. The transmission data may be transmitted by a single payload or may be multiplexed and then transmitted by a plurality of payloads. The payload for multiplexing may be divided into two or more slices by a predetermined unit.

[29] Furthermore, the payload may provide an interleaving function by a predetermined unit in consideration of a particular transmission function, and may be considerably efficiently and simply defined by a general available unit structure of the transport layer. A header region of the universal MMT payload for hybrid multimedia contents transmission in the hybrid network environment may be configured as indicated in Table 1.

[30] Table 1

[Table 1]

<td> Pay load Type</td><td> Fragmentation Indicator</td><td> Sequence Number</td>

<td colspan="2"> Sequence Number</td><td> Length</td>

<td colspan="2"> Length</td><td> Timestamp</td>

<td colspan="3"> Timestamp</td>

<td colspan="2"> Timestamp</td><td> Multiplex Info.</td>

<td> Aggregation Info.</td><td colspan="2"> Payload Configuration</td>

[31] i) Pay load Type

[32] A payload type may be a data configuration type of the corresponding payload, and more specifically, may be a type of transmitted data and configuration information. At

CA 02852002 2014-04-11

WO 2013/055176 PCT/KR2012/008377 this time, the type of data and the configuration information according to the type of data may indicate a type of pay load data, that is, information on whether the pay load data is a complete M-unit 110, a simple fragment of the M-unit 110, a complete MFU 100, or an aggregation of a plurality of MFUs 100. Furthermore, the type of data and the configuration information may include a factor identifying whether the type of data transmitted by the corresponding payload is transmission data or control data, and may also include information on a discriminator for discriminating types of transmission data.

[33] For example, the type of data and the configuration information may include information on a media type of transmitted data, for example, a video type, an audio type, an image type, a text type, or any other similar type, and an associated codec type. Furthermore, an additional data type may include information on whether data transmitted in the payload is a data type of single time stamp or a data type of multiple time stamp, and may include information on an identifier indicating a non-time stamp.

[34] ii) Fragmentation Indicator

[35] A fragment indicator may be information indicating at least one of a start, a center, and an end of the M-unit 110 when a transmitted M-unit 110 is divided into fragments. For example, when data of a corresponding pay load is the MTU 100 of the media, the client may handle the data in the bundle unit of processing the data by using the information and then transmit the data. Furthermore, when receiving a last fragment of the M-unit 110, the terminal configures the complete M-unit 110 and performs a decoding processing.

[36] In a case of transmitting a live stream or a VoD, a payload for a transmission may have a size of the single MFU 100 or the M-unit 110, and during transmission of a file, the payload may have a total size of the file. In addition, chained payloads may be indicated through such information, and a multiplexing function in units of a plurality of MFUs 100 or M-units 110 may be provided through a corresponding information configuration. A plurality of M-units 110 or MFUs 100 may be transmitted in one payload.

[37] iii) Sequence Number

[38] A sequence number may be increased, e.g., iterated, and transmitted by one unit with respect to each M-unit 110, and may be used by a receiver in order to detect a pay load loss. The MMT performs no action for MMT packet loss, and operations concerning MMT packet loss remains for an application to perform. The MMT transmission may not guarantee transmission, but existence of the sequence number makes it possible to detect a lost M-unit 110.

[39] iv) Length

[40] A length field indicates a length of the payload, including a header and data.

CA 02852002 2014-04-11

WO 2013/055176 PCT/KR2012/008377

[41] v) Time stamp

[42] A time stamp filed may be used for identifying a reproduction time of the payload, and may be shown in the payload for real time transmission.

[43] vi) Multiplexing Information

[44] Multiplexing information may indicate that a transmission unit included in the corresponding payload is hybridly multiplexed. For example, the multiplexing information may indicate a case where a plurality of M-units 110 may be included in the pay load. Furthermore, corresponding information may include a number of multiplexed configuration units, a size of each configuration unit, an offset of the configuration unit, and any other similar and/or suitable information.

[45] vii) Aggregation

[46] An aggregation field may indicate setting information on the M-unit 110, and information regarding this field may be dependent on a payload type. In further detail, the aggregation field may indicate a number of remaining fragments after fragments included in the corresponding payload when the M-unit 110 is in a fragmented state, and may indicate a number of M-units 110 or MFUs 100 included in the corresponding payload when the M-unit 110 is in an aggregated state.

[47] viii) Pay load Configuration

[48] The payload may be configured according to various transmission environments and services, and may be configured in parts with respect to information applicable to all payloads and information applicable to only a particular payload. Furthermore, a very small configuration may be generally required for a particular payload, but the present exemplary embodiment may use particular configuration modes for particular purposes in order to allow a simple payload configuration except for a dedicated receiver.

[49] For example, an MMT streams mode, a file delivery mode, and a generic mode may be used for transmitting MMT contents. Furthermore, an auxiliary field may be configured to transmit any data in order to predict requirements for transmission of information related to an additional system in the future. Specifically, the auxiliary field may include a function providing identifier which may provide a function for a loss control, loss reconstruction encoding information, information for requesting an automatic error loss reconstruction, a minimum memory buffer size provided by a terminal, a temporary memory buffer size used in transmission data, critical transmission data, a maximum loss acceptance rate, a maximum acceptance packet delay time, an average packet delay time, synchronization code information for a network synchronization, and any other similar and/or suitable information.

[50] Next, a case where data is transmitted using the MMT payload configured as described above will be described. First, in hybrid contents data transmission for real time broadcasting, the terminal may be connected with at least one server in a

CA 02852002 2014-04-11

WO 2013/055176 PCT/KR2012/008377 broadcast network and broadband network environments, and the terminal may be used to provide a real time soccer game broadcasting service and a user additional service related to the soccer game, for example, a caption of another language and user audio explanation information to a client having a wide screen display device, in real time. In this case, the M-unit 110 or the MFU 100, which may be handled as a basic transmission unit among data units configuring real time broadcasting contents, may be contained in a single payload or a plurality of payloads and then transmitted. When the M-unit 110 is the basic transmission unit, one Group of Picture (GOP) is handled as the M-unit 110 and is transmitted using the single pay load in video data. When data of the one GOP is a maximum video unit which may be independently decoded and is encoded into an H.264 video, the maximum video unit which may be independently decoded includes a slice, and the slice may be used as a minimum unit of the MFU 100.

[51] When the one GOP that is encoded into the H.264 video is configured by the pay load for transmission by using the M-unit 110, the configuration may be made according to Table 2, and the one GOP may be configured by several pay loads since a size of the one GOP is larger than a size of the payload. Furthermore, the payload, as shown in Table 2, may have the same structure when unit data of the payload is the MFU 100 as well as when the unit data of the pay load is the M-unit 110.

[52] Table 2 [Table 2]

- payload type : H.264 Video GOP- fragmentation indicator : on- sequence number : increase from 0- length : 1024 (byte) indicate a length of the pay load- timestamp : a representative value of a reproduction time of a corresponding GOP is expressed as an NTP timestamp.- multiplexing information : off- aggregation : 5 end payload of the corresponding GOP is completed after 5- pay load configuration : ALFEC indicator, DRM indicator- payload

[53] Next, a case where progressive download contents are transmitted by non-real time broadcasting through a service transmitted from a plurality of servers will be described. A transmission data basic unit structure of the MMT payload for non-real time data transmission may be divided into two cases, that being a case having the asset 120 and a case having the package 130, and a pay load type may be determined according to a type of each transmission data.

[54] Furthermore, since transmission characteristics of corresponding data are according to a non-real time case, attribute information related to a time may be excluded from corresponding payload data, which may provide an efficiency in high-capacity data transmission. In addition, according to characteristics of transmission data sensitive to

CA 02852002 2014-04-11

WO 2013/055176 PCT/KR2012/008377 loss, additional information used for a loss reconstruction may be described through the auxiliary field, and information on the auxiliary field may provide data structure information related to the loss reconstruction rather than real time transmission.

[55] In hybrid multimedia contents transmission for real time broadcasting and non-real time hybrid multimedia contents transmission, the terminal may be connected with at least one server in the broadcast network and the broadband network environments, and the terminal may be used to provide a real time soccer game broadcasting service and a user additional service related to the soccer game, for example, a caption of another language and user audio explanation information to a client having a wide screen display device in real time.

[56] It may be considered that basic unit data of the transmission structure hybridly transmits the MFU f00 and the M-unit 110 corresponding to real time transmission data and the asset 120 and the package 130 corresponding to non-loss transmission data, and transmits additional information for describing the corresponding transmission structure through multiplexing information. The additional information may provide identification and discrimination unit information of corresponding data information through a multiplex type corresponding to multiplex structure information, an offset of each transmission data unit, a length, and an IDentification (ID) of the transmission data unit.

[57] Meanwhile, the MMT payload, according to an exemplary embodiment of the present invention, may be transmitted by the MMT transport packet 140 or the RTP packet 180 corresponding to an IP address. The MMT transport packet 140 may have a data packet structure for MMT payload transmission and may perform a function of packetizing data by the unit of payloads and may transmit the data to the network. Furthermore, the MMT transport packet 140 includes additional information related to transmission in the header region, and the additional information may include a packet ID, a packet number, a flow ID for providing Quality of Service (QoS), a timestamp applicable as a transmission time and control information, and any other similar and/or suitable information. The above described additional information will be described below in detail.

[58] i) Packet ID

[59] A packet ID may be an identifier of the MMT transport packet 140, and may be an identification value allocated by a transmission side when fragments of a datagram of a lower protocol are assembled. Furthermore, the packet ID may be used as information indicating a media type of data included in the MMT payload, and may include information capable of distinguishing respective media types when data corresponding to a plurality of same media types is included in the MMT payload.

[60] ii) Sequence Number

CA 02852002 2014-04-11

WO 2013/055176

PCT/KR2012/008377

[61]

[62]

[63]

[64]

[65]

[66]

[67]

[68]

[69]

[70]

A sequence number may be an identification number of the transmitted packet. A reception side may determine whether a packet corresponding to a sequence number is lost through the sequence number and may perform a packet arrangement function according to the sequence number. Furthermore, sequence number information may sequentially increase, and may be set for each M-unit 110 or each asset 120.

iii) Flow Label

A flow label may identify a particular QoS used for each flow in each data transmission. For example, the flow label may include a type of packet, a delay, a throughput, a synchronization parameter and any other similar and/or suitable information, and may be fragmented and indicated according to a data transmission type as shown in Table 3 to Table 6.

- Type of Packet

A Type of Packet field may indicate type information of the packet for the MMT transport packet 140, and may be a 3 bit value.

Table 3

[Table 3]

<td> Bits 0-2</td><td> Bit rate</td>

<td> 111</td><td> Constant Bit Rate(CBR)</td>

<td> 110</td><td> Real-Time Variable Bit Rate(rt-VBR)</td>

<td> 101</td><td> Non-Real-Time Variable Bit Rate(nrt-VBR)</td>

<td> 100</td><td> Available Bit Rate(ABR)</td>

<td> Oil</td><td> Unspecified Bit Rate(UBR)</td>

<td> 010</td><td> Reserved</td>

<td> 001</td><td> Reserved</td>

<td> 000</td><td> Reserved</td>

- Delay

A delay field may indicate a delay degree, and may be a 1 bit value, as shown in

Table 4.

Table 4

[Table 4]

<td> Bit 3</td><td> Delay</td>

<td> 0</td><td> Normal delay</td>

<td> 1</td><td> Low delay</td>

- Throughput

CA 02852002 2014-04-11

WO 2013/055176

PCT/KR2012/008377

[71] A throughput field may indicate a throughput degree, and may be a 1 bit value, as shown in Table 5.

[72] Table 5

[Table 5]

<td> Bit 4</td><td> Throughput</td>

<td> 0</td><td> Normal throughput</td>

<td> 1</td><td> High throughput</td>

[73] A synchronization field may indicate synchronization information, and may be a 3 bit value, as shown in Table 6.

[74] Table 6

[Table 6]

<td> Bit 5</td><td> Synchronization</td>

<td> 0</td><td> no relationship</td>

<td> 1</td><td> relationship with another packet</td>

<td> Bits 6-7</td><td> Reserved for future use</td>

[75] iv) Packet Configuration Information

[76] Packet configuration information may provide configuration information, i.e., parameters, for approaching a flexible and efficient mechanism of a corresponding service class in transmission of packetized data. Such parameters are used for measuring a QoS of a connection in a network layer and an end-to-end network capability. The network may guarantee a negotiated QoS by satisfying particular values of the parameters, and corresponding parameter information may be used as a reference of the parameter of an IP version 4 (IPv4) type of service (tos) field and an IP version 6 (IPv6) QoS field. Table 7 and Table 8 show detailed parameter structures.

[77] - Delay (Range)

[78] A delay field, which may be referred to as a range field, may indicate a maximum end-to-end acceptance delay time related to transmission. The reception side may easily determine whether a delay time is exceeded based on corresponding delay information and may reduce a corresponding time for efforts on a delay time reduction.

[79] Table 7

CA 02852002 2014-04-11

WO 2013/055176

PCT/KR2012/008377

[Table 7]

<td> Bit 0-2</td><td> Delay</td>

<td> 111</td><td> 500ms</td>

<td> 110</td><td> lsec</td>

<td> 101</td><td> 5sec</td>

<td> 100</td><td> 10sec</td>

<td> Oil</td><td> unlimited</td>

<td> 010</td><td></td>

<td> 001</td><td></td>

<td> 000</td><td></td>

[80] - AU Loss Rate (%)

[81] An Access Unit (AU) loss rate may indicate an acceptable loss ratio of the transmitted packet. A percentage of the AU loss rate may indicate a loss allowance of the packet per second. The reception side may more easily determine a method of handling a reconstruction function for the packet loss and whether the packet loss is reconstructed according to corresponding information.

[82] Table 8

[Table 8]

<td> Bit 3-4</td><td> AULR</td>

<td> 111</td><td> 0%</td>

<td> 110</td><td> 10%</td>

<td> 101</td><td> 20%</td>

<td> 100</td><td> 30%</td>

<td> Oil</td><td> 50%</td>

<td> 010</td><td> not specified</td>

<td> 001</td><td></td>

<td> 000</td><td></td>

[83] EIG. 2 illustrates a method of transmitting hybrid multimedia contents according to an exemplary embodiment of the present invention.

[84] Referring to EIG. 2, the hybrid multimedia contents may be transmitted by the MMT transport packet 140, and the MMT transport packet 140 includes at least one of data synchronized and reproduced according to a time, for example, real time broadcasting,

CA 02852002 2014-04-11

WO 2013/055176 PCT/KR2012/008377 and non-real time data in order to generate a pay load part in step 200. Next, in step 210, a pay load header part, including information on the pay load part, is generated. The packet header part, including information on the MMT transport packet 140, is then generated in step 220, the MMT transport packet 140, including the pay load part, the payload header part, and then a packet header part, is generated in step 230, and the hybrid multimedia contents are transmitted using at least one of a transmitter, a transceiver, or any other similar and/or suitable transmission device. The transmitter may include at least one of an antenna and a wire line for transmitting a signal including the hybrid multimedia contents to the hybrid network.

[85] While the invention has been shown and described with reference to certain exemplary embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by appended claims as their equivalents.

CA 02852002 2014-04-11