Method of distributed editing of video clips over a network

CROSS-REFERENCE TO RELATED APPLICATIONS

Not applicable

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

This invention was developed under a federally sponsored research project, and the United States Government has certain rights as specified in the United States Department of Commerce Contract Number 70NANB5H1176.

BACKGROUND OF THE INVENTION

The present invention relates to the distribution of packetized data over networks, and more particularly to an audio/video media server for distributed editing over networks using a low-resolution format or a high-resolution format, depending upon the available network bandwidth.

Existing media servers stream video and audio packets over a network under control of a server. A client consumes video at whatever rate the server sends the packets -- the client consumes a contiguous movie. In other words the current media servers "push" media from the server to the client using a "stream" from the server. These servers are often called "video on demand" servers, however they refer to the delivery of contiguous movies, not in-progress material suitable for an edit session.

In an editing environment a client is looking to put together a "movie" from a plurality of video clips that may be distributed over a plurality of servers. With the "video on demand" servers the timing of the video to the client is determined by the servers. To provide concatenated clips, especially when a first clip is from one server and a second clip is from another server, the timing issue is "when does the second server start to 'push' the second clip?" This presents a very complex timing problem.

What is desired is an audio/video media server for distributed editing over a network that allows clients to access media from many distributed servers or even different media files on a server to enable the playing of "edited" movies with the timing controlled by the client.

BRIEF SUMMARY OF THE INVENTION

Accordingly the present invention provides an audio/video media server for distributed editing over a network by responding in an asynchronous manner to media frame requests from one or more clients. Each request contains a clip identifier, a delivery destination and the frames desired from the clip. The media server asynchronously accesses a file system to retrieve the media frames from a storage medium. The media frames are then asynchronously input to a FIFO buffer and a clock rate is adjusted based upon the "fullness" of the FIFO buffer. For each interrupt generated by the clock a media frame is transferred from the buffer, packetized and sent over the network to the requesting client. In this way the client controls the timing of the media frames it receives to assure a continuous stream of media frames during editing.

The objects, advantages and other novel features of the present invention are apparent from the following detailed description when read in conjunction with the appended claims and attached drawing.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

Fig. 1 is a general block diagram view of a distributed network for media editing according to the present invention.

Fig. 2 is a block diagram view of a media server for distributed editing over a network according to the present invention.

Fig. 3 is a message flow chart for the media server according to the present invention.

Fig. 4 is a message sequence chart for the operation of the media server according to the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Referring now to Fig. 1 a distributed editing system <b>10</b> has a plurality of servers <b>12</b>, such as Silicon Graphics, Inc. workstation, coupled over a network <b>14</b>, such as Ethernet, to a plurality of clients <b>16</b>, such as a personal computer having a Windows NT operating system with a Sigma Designs MPEG decoder. One or more of the clients <b>16</b> may access any one of the servers <b>12</b> at any time, and even access the same server simultaneously.

One of the clients <b>16</b> requests media frames from one of the servers <b>12</b> via the network <b>14</b>. The client requests are in the form of command packets. As shown in Fig. 2 a network interface <b>18</b> in the server <b>12</b> receives the command packets and routes them to a command dispatcher <b>20</b>. The command dispatcher <b>20</b> routes the command packets to an appropriate processing module depending upon what command is included within the packet. Commands may include "Lookup" commands, "Register" commands or "Request" commands.

A "Lookup" command from the client <b>16</b> is a search for a particular file or video clip by the client. The command dispatch module <b>20</b> in the server <b>12</b> forwards the command to a Lookup command module <b>22</b>. The Lookup command module <b>22</b> returns an "id" to the client <b>16</b> for later use in accessing the clip via a file descriptor table <b>24</b>. If the clip is already "open" as a result of a prior Lookup command, the "id" is simply returned to the client <b>16</b>. This provides information to the client <b>16</b> about where clips are that it might want to access in performing an edit.

A "Register" command from the client <b>16</b> is forwarded by the Command Dispatch module <b>20</b> to a Register Client command module <b>26</b>. The Register Client command module <b>26</b> accesses a Client Delivery Handle table <b>28</b> and returns a destination identifier "did" to the client <b>16</b>. In this manner the server <b>12</b> may keep track of what clients <b>16</b> have accessed it and the clients may use the "did" in subsequent requests to notify the server where to deliver media frames.

Once the client <b>16</b> has located the desired clips via the Lookup command and registered with the server <b>12</b> with the Register command, the client now makes requests for media frames using the Request command. The Request command includes the clip "id", the "did" and a "framelist" that includes the frames desired in response to the request. The Command Dispatch module <b>20</b> routes the Request command to a Frame Request module <b>30</b>. The Frame Request module <b>30</b> converts the "id" and "framelist" together with the file descriptor information from the File Descriptor table <b>24</b> into a file address, an offset from the start of the file, a count corresponding to the number of frames requested and the delivery identification. Each media frame in the framelist is treated individually at this point.

A Read module <b>32</b> receives the information from the Frame Request module <b>30</b> and asynchronously accesses a file system <b>34</b> that retrieves the requested media frames one at a time from a storage unit <b>36</b>, such as disk. A Handler module <b>38</b> then asynchronously receives the media frames one at a time from the file system <b>34</b> and provides the retrieved frame to a FIFO Input module <b>40</b>. The FIFO Input module <b>40</b> stores each frame as received into a media frame first-in/first-out (FIFO) buffer <b>42</b> and also, either before or after storing each frame in the FIFO, checks the "fullness" of the FIFO. Based upon the "fullness" the FIFO Input module <b>40</b> provides an adjusted clock rate signal to a FIFO output clock <b>46</b>. In this manner the data flow from the media server <b>12</b> is smoothed out. The FIFO output clock provides a periodic interrupt to a FIFO Output module <b>44</b> which extracts a media frame from the FIFO <b>42</b> for transmission to the requesting client <b>16</b>. A Send module <b>48</b> receives the media frame from the FIFO Output module <b>44</b> and a destination address from the Client Delivery Handle table <b>28</b> and produces a media packet which is sent via the network interface <b>18</b> over the network <b>14</b> to the requesting client <b>16</b>.

Referring now to Figs. 3 and 4 the message flow is in the form of commands (REGISTER, LOOKUP and REQUEST) from the client <b>16</b> to a request server <b>50</b> within the media server <b>12</b>. The request server <b>50</b> processes the REGISTER, LOOKUP and REQUEST commands, as indicated above, to provide a delivery identification for the client in the Handle table <b>28</b> and establish a clip identification and to provide frame read requests for the file system <b>34</b>. In response to the asynchronous read requests the file system <b>34</b> retrieves files from the disk <b>36</b>, and in response to handler requests the retrieved files on a frame by frame basis are transferred to a FIFO Manager <b>52</b>. The FIFO Manager <b>52</b> provides rate adjustment for the clock <b>46</b>, stores the frames as retrieved from the file system <b>34</b>, and outputs the frame packets to the client <b>16</b> in response to the clock interrupts.

A typical message sequence would start with a REGISTER command from the client <b>16</b>, which binds a network address and path to the client. The request server <b>52</b> returns to the client <b>16</b> the destination identifier ("did"). The client <b>16</b> then provides a LOOKUP command for a particular clip. The request server <b>52</b> asks the file system <b>34</b> to open the clip, if not already open, and the file system returns to the request server a file descriptor identifier. The request server <b>52</b> then returns the clip identifier ("id") to the client <b>16</b>. The client <b>16</b> now has the information it needs to make media frame requests in the form of REQUEST commands to the request server <b>50</b> that include the clip identifier and delivery identifier as a hybrid identifier ("hid") and the number of frames from the clip desired-- generally from three to ten frames per REQUEST command. The request server <b>50</b> sends a read request to the file system <b>34</b> and acknowledges the client's request. Upon receipt of the acknowledgement the client <b>16</b> may send another REQUEST command to the request server <b>50</b> for the next group of frames from the clip. Again the request server <b>50</b> forwards a read request to the file system <b>34</b> and acknowledges the request from the client.

The Handler module <b>38</b> returns the requested frame to the request server <b>50</b> which then forwards the frame information to the FIFO Manager <b>52</b>. The FIFO Manager <b>52</b> provides a rate command to the clock <b>46</b> which returns interrupts to the FIFO Manager. The FIFO Manager <b>52</b> in response to the interrupts outputs to the client <b>16</b> frame packets for each requested frame.

For multiple clients <b>16</b> simultaneously accessing a single media server <b>12</b> the media frames from the FIFO <b>42</b> may be read out by separate FIFO output modules <b>44</b> for each client. To avoid "starving" any one client <b>16</b> the FIFO output modules <b>44</b> are accessed in a rotational manner so that packets for each client are sent to the network <b>14</b> in an intermingled fashion, thus preventing the server <b>12</b> from being "captured" by a single client when multiple clients are accessing it.

Thus the present invention provides a media server that leaves timing to requesting clients by processing request commands from clients on a demand basis, and asynchronously accessing the requested media files and returning them to the client at a sufficient rate so that the client has a steady stream of video images to process, regardless of where the clips are coming from.