Data reproduction method, data receiving terminal and data receiving method

Description: TECHNICAL FIELD [Detailed description of the invention]

【0001】

[Technical field to which the invention belongs]

The present invention provides a method for reproducing AV (audio / video) data that can be adapted to changes in usable bandwidth, transmission errors, failures, etc., and a data receiving terminal for receiving and reproducing AV data transmitted from a server. It relates to a data receiving method.

【0002】

[Conventional technology]

Recently, with the development of video coding technology such as MPEG (Moving Picture Experts Group) -4, AV distribution services for mobile wireless terminals such as mobile phones have started, and such AV distribution services will become widespread in the future. It is thought that it will continue.

【0003】

Currently, there are two main types of data transmission methods for AV distribution: HTTP / TCP / IP (Hyper Text Transfer Protocol / Transmission Control Protocol / Internet Protocol) download playback and RTP / UDP /. There is streaming playback by IP (Realtime Transpot Protocol / User Datagram Protocol / Internet Protocol). In the control system of streaming playback, for example, a protocol called RTSP (Real Time Streaming Protocol) is used (H. Schulzrinne et al., "Real Time Streaming Protocol", RFC 2326, Internet Engineering Taskforce, Apr. 1998).

【0004】

Download playback by HTTP / TCP / IP uses the same data transmission protocol as when browsing the Internet homepage, so it is easy to realize, but data is retransmitted when a transmission error occurs. It is not very suitable for real-time playback of AV data that requires real-time performance. On the other hand, in streaming playback by RTP / UDP / IP, data is not retransmitted even in the event of a transmission error, so there is no guarantee that data will be transmitted reliably, but real-time AV data that requires real-time performance is not guaranteed. Suitable for reproduction.

【0005】

Further, in the download playback by HTTP / TCP / IP, since the normally received data is stored in the memory on the terminal even after the playback, it is not possible to deliver the content for a very long time due to the memory limitation. On the other hand, in streaming playback by RTP / UDP / IP, since the normally received data is discarded after the playback, there are few time restrictions on the content to be distributed, and the content can be received and played for a long time.

【0006】

[Problems to be Solved by the Invention]

In the conventional streaming playback by RTP / UDP / IP, AV data after recovery, such as when the transmission line is disconnected, the power of the terminal is suddenly turned off, or the playback is interrupted by an interrupt of another application. It was not easy to resume the playback of. Specifically, if the connection with the server is disconnected during data playback of a certain content, the time when the connection is disconnected cannot be specified, and if the connection is reconnected to the same content, the connection is renewed. It was necessary to receive and play back the data from the beginning. Therefore, due to the convenience of the user, it is not possible to view specific content in several times at different times and places, which is inconvenient especially when viewing the content for a long time. It was.

【0007】

In addition, mobile wireless terminals such as mobile phones are characterized in that they can be used while the user is moving, but if they move to an area where radio waves do not reach (unreceivable area) such as a tunnel, it will take a long time. There is a risk that you will not be able to receive data. In such a case, in streaming playback by RTP / UDP / IP, the data transmitted while in the non-receivable area is not retransmitted, so the data when in the non-receivable area is completely lost and the data is completely lost in the non-receivable area. It had a problem that it was not possible to view the data while it was there.

【0008】

The present invention has been made in view of the above problems, and an object of the present invention is to provide a data reproduction method, a data receiving terminal, and a data receiving method capable of restarting reproduction from the middle of a data stream in streaming reproduction.

【0009】

[Means for solving problems]

In order to solve the above-mentioned problems, in the present invention, the time information at the time when the reproduction of the data stream is interrupted is retained, and the reproduction is restarted from the middle of the data stream based on the time information.

【0010】

According to the present invention, the current playback time is held by the data receiving terminal or the transmitting terminal (server), the receiving buffer of the data receiving terminal becomes empty, or the transmission line is disconnected due to a failure or the like. When the power of the terminal is turned off, AV playback is temporarily stopped when an interrupt occurs from another application (for example, an internet phone). Then, when the transmission line is restarted, when the power of the terminal is turned on again, or when the interrupt from another application is canceled, the reception buffer becomes empty, or the transmission line is disconnected. , Data buffering is automatically started from the playback time when the power of the terminal is turned off or when an interrupt from another application occurs, and playback is performed to realize fault-resistant AV transmission. It becomes possible. Specifically, the playback time t1 is stored in the data receiving terminal (application name, playback content name, user name, etc. are also stored as necessary), and even if the transmission line is temporarily disconnected, the transmission line is restarted. If so, a request is issued from the data receiving terminal to the transmitting terminal (server) so that the data can be played back from the playback time t1.

【0011】

For example, in a data receiving terminal, the time stamp of the reproduced data is stored, and even if the connection with the server is disconnected due to the user's convenience or deterioration of the reception situation, the data is reproduced by the previous connection. It enables playback from the continuation of the data that has been created.

【0012】

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, the data receiving terminal according to the embodiment of the present invention will be described with reference to the drawings.

【0013】

(First Embodiment) FIG. 1 is a block diagram showing an example of a configuration of a data receiving terminal according to the first embodiment of the present invention. In FIG. 1, the data receiving terminal 12 according to the present invention receives and reproduces an RTP packet transmitted by an RTP / UDP / IP transmission method from a transmitting device (server) 11, and the data receiving terminal 12 Consists of a data reception unit 101, a reception buffer 102, a decoding unit 103, a display unit 104, a memory 105, a control unit 106, a user input unit 110, and a message transmission / reception unit 111. The control unit 106 includes a connection destination address detection unit 107, a memory management unit 108, and a connection request creation unit 109.

【0014】

FIG. 2 is a diagram for explaining an RTP packet transmitted from the transmitting device 11 by the transmission method by RTP / UDP / IP. As shown in FIG. 2, the RTP packet is composed of a header part and a data part. The header portion includes time information such as a playback time. An MPEG4 video stream is stored in the data section. For example, one frame of the video stream is stored in the data section of the RTP packet. In MPEG-4, this unit (1 frame) is called VOP (Video Object Plane). Here, it is assumed that one VOP data is transmitted for each RTP packet.

【0015】

FIG. 3 is a diagram showing an example of the memory structure of the receive buffer 102. The data receiving unit 101 receives the RTP packet transmitted from the server, calculates the VOP size, and receives the calculated VOP size, a time stamp indicating the reproduction order of the received VOP data, and VOP data in the receiving buffer 102. Store in. As shown in FIG. 3, the reception buffer 102 stores the time stamp 201, the VOP size 202, and the VOP data 203 as a set. The decoding unit 103 decodes each VOP data stored in the receive buffer 102 according to the time stamp attached to each, outputs the data to the display unit 104, and the decoded data is I-VOP (intra-encoded VOP). If, the time stamp corresponding to the I-VOP is output to the memory management unit 108. The display unit 104 displays the input data on the display screen.

【0016】

FIG. 4 is a diagram showing an example of the stored contents of the memory 105. As shown in FIG. 4, the memory 105 is indicated by an active flag indicating the content being received, a playback completion flag indicating the content played to the end, a connection destination address indicating the connection destination address, and a connection destination address. A plurality of entries are stored in which a time stamp indicating the position where the content playback has ended and an access date and time indicating the date and time of access are combined. In the present embodiment, the active flag sets the content being received to 1 and the others to 0. The playback completion flag sets 1 for the content that has been played to the end and 0 for the others.

【0017】

The control unit 106 controls the entire data receiving terminal 12, manages the memory 105, and requests a connection to the server. Specifically, the connection destination address detection unit 107 detects whether or not the entry of the connection destination address output from the user input unit 110 is registered in the memory 105, and the entry of the connection destination address is registered. If the connection request creation unit 109 requests to detect the time stamp of the entry, the time stamp of the entry is detected. The memory management unit 108 manages the memory 105, and by creating a new entry and outputting a time stamp from the decryption unit 103, the time stamp for the currently receiving connection destination in the memory 105 can be obtained. Replace with the time stamp output from the decryption unit 103. The connection request creation unit 109 creates a connection request to the server based on the user's instruction input to the user input unit 110 and the information stored in the memory 105.

【0018】

The user input unit 110 is, for example, an operation button or a touch panel, analyzes external inputs such as user selection and pressing, and outputs the analysis result to the control unit 106. The message transmission / reception unit 111 transmits the connection request created by the connection request creation unit 109 to the server, and processes the response from the server.

【0019】

Next, the operation of the data receiving terminal 12 of the present invention will be specifically described. For the user input unit 110, for example, an area in which the user inputs a connection destination address of desired content or a link to video data or the like is provided from the scene of the content displayed by the display unit 104. By selecting, the connection destination address is output from the user input unit 110 to the connection destination address detection unit 107. When the connection destination address is output from the user input unit 110, the connection destination address detection unit 107 searches for whether or not the entry of the connection destination address output from the user input unit 110 is registered in the memory 105, and searches for whether or not the entry of the connection destination address output from the user input unit 110 is registered. The detection result is output to the memory management unit 108 and the connection request creation unit 109.

【0020】

FIG. 5 is a flowchart showing an example of the operation of the memory management unit 108 that receives the detection result from the connection destination address detection unit 107, and the memory management unit 108 receives the detection result from the connection destination address detection unit 107. The operation described below is performed. First, the memory management unit 108 receives the detection result from the connection destination address detection unit 107 in step S101, and if an entry for the connection destination address is registered, goes to step S102 and the entry for the connection destination address is registered. If not, go to step S103. When the entry of the connection destination address is registered, the memory management unit 108 sets the active flag of the entry registered in the memory 105 to 1 (step S102). On the other hand, if the connection destination address is not registered, the memory management unit 108 searches for free space in memory 105 in step S103, and if there is free space in memory, goes to step S104 and memory. If there is no free space in, go to step S105. If there is free space in the memory, a new entry is created in the free space (step S104). To create a new entry by the memory management unit 108, set the connection destination address specified by the user input unit 110 to the connection destination address, set the active flag to "1", and set the playback completion flag to "0". , By setting the time stamp to "0". On the other hand, when there is no free area in the memory, the memory management unit 108 searches in step S105 whether or not there is an entry whose reproduction completion flag of the entry stored in the memory 105 is "1". If there is an entry whose playback completion flag is "1", the process goes to step S106, and if there is no entry whose playback completion flag is "1", the process goes to step S107. If there is an entry whose playback completion flag is "1", a new entry is overwritten on the entry whose playback completion flag is "1". (Step S106). If there are multiple entries with the playback completion flag set to "1", the one with the oldest access date and time is selected from those entries, and a new entry is overwritten on the selected entry. On the other hand, if there is no entry whose playback completion flag is "1", the one with the oldest access date and time is selected from the registered entries, and a new entry is overwritten with the selected entry (step S107). ).

【0021】

FIG. 6 is a flowchart showing an example of the operation of the connection request creation unit 109 that has received the detection result from the connection destination address detection unit 107, and the connection request creation unit 109 displays the detection result from the connection destination address detection unit 107. Receive and perform the operation described below. First, the connection request creation unit 109 receives the detection result from the connection destination address detection unit 107 in step S201, and if an entry for the connection destination address is registered, goes to step S202 and registers the entry for the connection destination address. If not, go to step S205. When the entry of the connection destination address is registered, the connection request creation unit 109 refers to the reproduction completion flag of the entry and detects whether or not the reproduction completion flag is "0" (step S202). .. If the playback completion flag is "0", the process goes to step S203, and if the playback completion flag is not "0", the process goes to step S205. If the playback completion flag is "0", the previous playback has ended halfway, so the connection request creation unit 109 asks the display unit 104 to determine whether or not to receive from the continuation of the previous time. A signal indicating that a message is to be displayed is output to the display unit 104 and displayed (step S203). Then, when a selection signal for selecting to receive data from the previous continuation is output from the user input unit 110, the connection request creation unit 109 enters the entry via the connection destination address detection unit 107. In order to detect the time stamp of and request the reception of data from the previous continuation of the content indicated by the connection destination address, a connection request indicating the reception start position indicated by the time stamp is created, and the message transmission / reception unit 111 is notified. Output (step S204). On the other hand, when the selection signal for selecting not to receive the data from the previous continuation is output from the user input unit 110, or when the entry of the connection destination address is not registered, the connection destination address which is further registered If the entry's playback completion flag is not "0"

【0022】

Now, the message transmission / reception unit 111 that receives the connection request from the connection request creation unit 109 transmits the connection request created by the connection request creation unit 109 as the SETUP message of RTSP to the server. Then, when it receives an OK response from the server, it sends a PLAY message to the server. Further, when the message transmission / reception unit 111 receives an OK response from the server in response to the PLAY message, the message transmission / reception unit 111 releases the data reception unit 101 so that data can be received. On the other hand, when the message transmission / reception unit 111 cannot receive the OK response from the server, it determines that the connection is not possible, and notifies the user by displaying the connection failure through the display unit 104. The data reception unit 101 released by the message transmission / reception unit 111 receives the RTP packet data string transmitted from the server, and stores the time stamp, data size, and VOP data for each VOP in the reception buffer 102.

【0023】

The decoding unit 103 sequentially reads data from the reception buffer 102 according to the time stamp of each VOP stored in the reception buffer 102, decodes the data, and outputs the decoded data to the display unit 104. At the same time, the decoding unit 103 determines whether or not the data being decoded is I-VOP at the time of decoding the data, and if the decoded data is I-VOP, it corresponds to the I-VOP. Output the time stamp to be performed to the memory management unit 108. On the other hand, if the decoded data is not I-VOP, the time stamp corresponding to the I-VOP is not output to the memory management unit 108. When the memory management unit 108 receives the output of the time stamp from the decryption unit 103, the memory management unit 108 detects an entry having an active flag of "1" indicating the connection destination address currently being received from the memory 105, and decrypts the time stamp of the entry. Replace with the time stamp output from 103. Further, the data decoded by the decoding unit 103 is displayed by the display unit 104 and provided to the user.

【0024】

When the playback of the content being received is completed to the end, the decoding unit 103 outputs a signal to the memory management unit 108 that the playback of the content is completed, and the memory management unit 108 has an active flag of "1" in the memory 105. Detects an entry that is, sets the playback completion flag of the entry to "1", and resets the active flag to "0".

【0025】

Next, the operation of the data receiving terminal 12 when the user performs an operation for disconnecting the connection with the server to the user input unit 110 during data reproduction will be described. For example, when a button for interrupting data reception / playback is pressed on the user input unit 110, a signal indicating that data reception / playback is interrupted is output from the user input unit 110 to the control unit 106. To. Upon receiving the signal from the user input unit 110 to interrupt the transmission / reception of data, the control unit 106 performs the processing of canceling the reception / reproduction of the data. That is, the control unit 106 closes the data reception unit 101, resets the reception buffer 102 and the decoding unit 103, and resets all the active flags of the entries in the memory 105 to "0". At this time, in the memory 105, the time stamp of the I-VOP decoded / reproduced at the end of the previously reproduced data is recorded by the memory management unit 108. Further, the control unit 106 instructs the message transmission / reception unit 111 to stop the transmission of data, and the message transmission / reception unit 111 transmits the RTSP TEARDOWN message to the server. As a result, transmission from the server is stopped.

【0026】

As described above, in the data receiving terminal 12 according to the present embodiment, the memory management unit 108 stores the time stamp of the last decoded / reproduced data at the time of data reproduction in the memory 105, so that the user ends the reception in the middle. Even if this is the case, when the content is played again, data can be received from the continuation of the previous playback, and the content can be played for a long time at different times and places for the convenience of the user several times. It is possible to watch separately.

【0027】

Further, among the entries stored in the memory 105, the entry indicating the currently connected content is represented by using the active flag, so that the memory management unit 108 sees the active flag when updating the time stamp. As a result, the entry for updating the time stamp can be detected, so that the time stamp update processing speed can be improved and the power consumption can be reduced. Further, among the entries stored in the memory 105, the entry indicating the content played to the end is represented by using the playback completion flag. Therefore, the connection request creation unit 109 uses the playback completion flag when the connection request is created. By looking at it, it is possible to detect the entry for which playback has been completed to the end, so it is possible to improve the speed of creating connection requests and reduce power consumption.

【0028】

Further, when the decoding unit 103 decodes the intra-encoded data (I-VOP), the time stamp of the data is output to the memory management unit 108. Therefore, the number of times the time stamp is updated by the memory management unit 108. This can lead to a reduction in power consumption, and when data is received from the continuation of the previous reproduction, reception can be resumed from the intra-encoded data.

【0029】

The connection request creation unit 109 of the data receiving terminal 12 according to the present embodiment displays a message as to whether or not to perform playback from the continuation of the previous time, and allows the user to determine whether or not to perform playback from the middle. However, for example, if the entry of the connection destination address is registered and the playback completion flag is "0", a connection request is automatically made to receive the content data from the previous continuation. It may be the one to be created.

【0030】

(Second Embodiment) FIG. 7 is a block diagram showing an example of the configuration of a data receiving terminal according to the second embodiment of the present invention. In FIG. 7, the data receiving terminal 22 according to the present invention receives and reproduces an RTP packet transmitted by a transmission method using RTP / UDP / IP from a transmitting device (server) 21. The transmission device 21 includes a reception status analysis unit 201. Further, the data receiving terminal 22 includes a data receiving unit 101, a receiving buffer 102, a decoding unit 103, a display unit 104, a memory 105, a control unit 106, a user input unit 110, a message transmitting / receiving unit 111, and the like. It consists of a reception status notification unit 202. The control unit 106 includes a connection destination address detection unit 107, a memory management unit 108, and a connection request creation unit 109.

【0031】

The data receiving terminal 22 according to the second embodiment is the first in that the data receiving terminal 12 described in the first embodiment is further provided with a reception status notification unit 202 for notifying the connection status with the transmission device 21. It differs from the data receiving terminal 12 according to the embodiment. The same components as those of the data receiving terminal 12 according to the first embodiment are designated by the same reference numerals, and the description thereof will be omitted.

【0032】

The reception status notification unit 202 of the data reception terminal 22 operates while connected to the transmission device 21, and transmits a reception report indicating that data has been received to and from the reception status analysis unit 201 of the transmission device 21. , The data receiving terminal 22 constantly receives a transmission report indicating that the transmitting device 21 has transmitted data, and if the transmitting report transmitted from the transmitting device 21 is not received for a predetermined time. It is determined that the user has entered an area (unreceivable area) where radio waves cannot reach, such as a tunnel, and a message to that effect is output to the control unit 106. In response to this, the control unit 106 closes the data receiving unit 101, resets the receiving buffer 102 and the decoding unit 103, resets all the active flags of the entries in the memory 105 to "0", and displays the display unit 104. Indicates that the connection has been disconnected via. At this time, in the memory 105, the time stamp of the I-VOP decoded / reproduced at the end of the previously reproduced data is recorded by the memory management unit 108.

【0033】

On the other hand, the reception status analysis unit 201 of the transmission device 21 also operates during the connection with the data reception terminal 22, and receives and transmits the reception report transmitted from the reception status notification unit 202 of the data reception terminal 22. If the report is always transmitted and the reception report transmitted from the data receiving terminal 22 is not received for a predetermined time, the data receiving terminal 22 is in an area where radio waves such as a tunnel do not reach (reception is not possible). It is determined that the area) has been entered, and the session is disconnected, that is, the transmission of RTP data from the transmission device 21 is stopped.

【0034】

In this way, the transmitting device (server) 21 and the data receiving terminal 22 according to the present embodiment transmit and receive a communication report as to whether or not the connection is always maintained between the transmitting device 21 and the data receiving terminal 22. , Data receiving terminals 22 can detect each other that they have entered an unreceivable area such as a tunnel, and subsequent sessions can be disconnected, so communication costs can be reduced without transmitting unnecessary data. Bandwidth can be used effectively. In addition, since the time stamp of the last displayed I-VOP is recorded in the memory 105, playback can be resumed from the place where the connection was disconnected, and the content whose connection was disconnected can be restarted from the beginning. There is no need to review the content, and the content can be viewed efficiently.

【0035】

In the present embodiment, when the reception status notification unit 202 does not receive the transmission report transmitted from the transmission device 21 for a predetermined time, it is determined that the data receiving terminal 22 has entered the unreceivable area. For example, when the reception status notification unit 202 monitors the reception status of the RTP packet of the data reception unit 101 and the data reception unit 101 does not receive the RTP packet for a certain period of time, the data reception terminal 22 May be determined to have entered the unreceivable area.

【0036】

Further, when the reception status analysis unit 201 of the transmission device 21 determines that the data reception terminal 22 has entered the unreceivable area, the time stamp of the I-VOP at that time is recorded in the transmission device 21. May be good. Even in this way, playback can be resumed from the place where the connection is disconnected. However, since it is not possible to estimate to what point in time the data receiving terminal 22 can actually receive the data, the present embodiment is superior in terms of certainty.

【0037】

In the data receiving terminals 12 and 22 according to the above embodiments, when the decoding unit 103 decodes the I-VOP data, the time stamp of the content being received stored in the memory 105 is updated. However, regardless of the type of data, the decryption unit 103 updates the time stamp of the content being received stored in the memory 105 every time the data is decrypted, or updates the time stamp every time a certain period of time elapses. It may be something to do. Further, not only the time stamp that affects the decoding order, that is, DTS (Decoding Time Stamp), but also so-called CTS (Composition Time Stamp) or PTS (Presentation Time Stamp) may be used.

【0038】

[Effect of the invention]

As described above, according to the present invention, since the time information at the time when the playback of the data stream is interrupted in the streaming playback is retained, the playback is restarted from the middle of the data stream based on the time information. can do. For example, when the transmission line is disconnected, the power of the terminal is suddenly turned off, or an application interrupt occurs, there is an effect that the playback of AV data can be automatically restarted after the connection is restored.

[Simple explanation of drawings]

[Figure 1]

It is a block diagram which shows an example of the structure of the data receiving terminal by 1st Embodiment of this invention.

[Figure 2]

It is a figure for demonstrating the RTP packet transmitted from the transmission device in FIG. 1 by the transmission method by RTP / UDP / IP.

[Fig. 3]

It is a figure which shows an example of the memory structure of the receive buffer in FIG.

[Fig. 4]

It is a figure which shows an example of the storage contents of the memory in FIG.

[Fig. 5]

It is a flowchart which shows an example of the operation of the memory management unit in FIG.

[Fig. 6]

It is a flowchart which shows an example of the operation of the connection request creation unit in FIG.

[Fig. 7]

It is a block diagram which shows an example of the structure of the data receiving terminal by 2nd Embodiment of this invention.

[Explanation of symbols]

11,21 Transmitter 12,22 Data receiving terminal 101 data receiving unit 102 Receive buffer 103 Decryption unit 104 Display unit 105 memory 106 Control unit 107 Connection destination address detection unit 108 Memory management unit 109 Connection request creation unit 110 User input unit 111 Message Send / Receive Unit 201 Reception status analysis unit 202 Reception status notification unit