System and method for communicating media signals

Related application This application is a non-provisional application such as Reynold, entitled "System and Method for Communicating Media Signals", application number 60 / 325,483, agent reference number ID-PAT-001PR1 filed on September 26, 2001. It is a provisional application Field This disclosure relates to a system and a method for communicating a media signal between a source device and a receiving device. More specifically, it relates to a system and a method for compressing and decompressing streaming media signals and static media signals and efficiently communicating the signals between a source device and a receiving device using an artificial intelligence mechanism.

The ability to efficiently communicate streaming and static media between devices located remotely to each other is a significant demand that has emerged exponentially with the advent of networked communications such as the Internet. This demand has been met in recent years by substantial development resources on a global scale.

Intended herein by the term "media" is to mean information that can be communicated from a source device to an arriving device in the form of a signal for use in that arriving device and is used herein. If so, it is generally intended that the media will contain either a streaming media signal or a static media signal. For the purposes of this disclosure, the term "use" applied as the operation of a receiver with respect to a media signal is intended for reproduction (eg, sound, image, video), processing (eg, telemetry data), or media signal intent. It is intended to include any other use or operation that is the intended purpose.

The term "streaming media" is intended herein to mean a media signal containing information intended to be communicated to or used by an arriving device in a transient, flowing state. Has been done. The term "streaming" as applied to a streaming media signal is a signal that can be communicated and processed in a continuous state over time, a signal that is communicated and processed in a continuous state over time, or is interconnected and processed. It is intended herein to include a series of individual packets, pieces, or signals that can be communicated in blocks that are subsequently used in a continuously interconnected state by the arriving device. .. For the purposes of this disclosure, examples of streaming signals therefore include, without limitation, media of the type such as video, audio, audio combined with video, data strings such as temporary telemetry. The term "streaming media" is typically used in most cases by referring to the digitized form of data that represents the media of interest.

The term "static media" is intended to generally mean media that is not "streaming" as defined above. Static media signals are of the type that can generally be communicated and are intended to be used as a single packet, block, or piece. Therefore, static media, for example, without limitation, individual images, individual and relatively temporary short video clips, sounds or sound bytes, or piece information or blocks such as telemetry information. May contain information. However, it is conceivable that such a "single piece" of static media will be, for example, multiple regions or pixels of the overall image, individual separate frames, groups that together form a video clip. Can be as large as possible, consisting of multiple sounds, including sounds, sound bytes, or multiple smaller pieces or subparts, such as information bits containing larger information blocks as a group. That is.

Streaming media generally contains data files that are significantly larger than static files and often represents more variables over the temporary communication of such files than is experienced with most media files. .. Therefore, the ability to efficiently compress streaming media for proper communication to the arriving device for use is often considerably more complex and difficult to reach the goal. Therefore, much of this disclosure is provided specifically with reference to streaming media, and the present invention has been considered to provide significant benefits for such communications. However, it is contemplated that static media will also be appropriate according to one of ordinary skill in the art if streaming media is specifically referred to herein for this background technique and for the many benefits of the invention disclosed herein. Has been done.

A number of different "type-specific" media systems transmit or transmit specific types of streaming and static media signals (eg, video, audio, images, voice, etc.) between sources and remote destinations. Has been used for quite some time. Typical examples of such type-specific media systems include television transmission systems, telephone line systems, and radio (radio) transmission systems, where televisions, telephones, and radios are therefore media receivers. Therefore, the demand for efficient communication of streaming and static media is about to become a very diverse telecommunications industry, such as telephone, television, film, music, and more recently two-way (interactive). Includes various industries for games.

Furthermore, many media communication systems, including the various long-lasting type-specific systems described above, are also "format-specific" in which the target media signal is communicated in a particular format to provide the source, transmit channel, and arrival device. The work within that format must be particularly obedient. Examples of format-specific media systems include, for example, coded cable television systems that work only on certain types of media and are only delivered from cable carriers in a particular coded format. Therefore, these systems are dedicated in hardware and software only to the types and formats of media that should generally be provided by content providers.

The needs of society have outweighed the capabilities of these dedicated content-specific and format-specific systems. Dedicated systems, in particular, are not configured to accommodate the ever-growing real-time client demands for specific streaming media. Moreover, technology development in today's interconnected world has challenged society's intellectual taste for the ability to pull, receive, push, and transmit multiple types of media in multiple formats using a single device. In addition, content providers must be able to deliver many different media signals to their client-side offices, living rooms, and many different types of equipment at hand. Individuals and companies also want to communicate with each other using a variety of different formats and a variety of different devices.

Therefore, a fairly large industry has emerged for delivering streaming and static media over centralized networks on the Internet. Content distributors today distribute a wide range of streaming media, from live horse racing and entertainment to medical telemetry and education, over the Internet and in video and audio formats. According to one public report from DFC Intelligence, it grew 215 percent in 2000 to reach a total access stream of over 900 million. This included broadband streams, which created a total access of approximately 29%. The same report also estimates that around 15 percent of valid stream inventory is now being promoted by advertising one after another. Jupiter Media as an internet researcher Another report published by Metrix reports that businesses along streaming video technology are moving towards electronic dialogue in communication with employees, consumers, and other businesses, just as companies are heading for electronic dialogue with employees, consumers, and other businesses. It will grow from 140 million US dollars (US dollar 140M) a year to about 3 billion US dollars (US dollar 3B) by 2005.

Moreover, the rapid increase in population and the increasing number of people transmitting on these systems have violently collided with the effective bandwidth for effective information. Therefore, it is of great importance to society where the ability to efficiently stream media with limited bandwidth resources and limited effective transmission speeds has been increased.

<u style="single">Compression / decompression algorithm ("CODEC")</u> In view of the exponential requirements for communicating different types of media, various compression / decompression systems (CODECs) have been developed over the years and have become the subject of particularly significant research and development. It's coming. A system that manages a specific type of CODEC and the operation of the CODEC regarding communication of streaming media signals and static media signals is for a specific type of media including still frame images such as graphics and video and streaming media, for example. It has been developed.

Image CODEC Various different types of static media codecs have been developed, and a wide variety of these CODECs are widely known and used. One particular type of static media that has become a subject of particular interest is streaming with more complex variables (such as frame-to-frame size and temporary relationships that significantly impact proper compression / decompression requests). Includes various images (via a long sequence of correlated image frames) that are commonly treated in video context as media. Examples of static media codec processing are illustrated by reference to certain specific types of conventional image CODEC technology and methods.

The two most common file formats for graphic images on the Worldwide Web are known as "GIF" and "JPEG" formats, commonly considered as standards for drawings (eg line art) and video. It is further described below along with other image compression formats for the purpose of further understanding.

"JPEG" is an acronym for "Joint Photographic Experts Group" and is a graphic image file that conforms to the ISO (International Organization for Standardization) standard 10918. JPEG files commonly used for video compression / decompression can be selected from a range of compression qualities, or, as mentioned earlier, selected from one of a set of multiple compression algorithms. Created by doing. The desired image quality must be specified in order to produce a JPEG file or to convert an image from another format to JPEG. In general, the highest quality is the largest file, so a compromise can be made between image quality and image size as selected by the user. Although JPEG embodying means may not use all of them, the JPEG mode of compression generally involves 29 individual coding processes. JPEG images are typically given the name subscript ".jpg".

"GIF" is an acronym for "Graphics Interchange Format" and is generally considered to be the standard format for drawing image compression / decompression for Internet communication. GIF formatting uses a compression algorithm known as the LZW algorithm, which is Abraham Lempel, Jacob Ziv, Terry Developed by Welch and commercially available from Unisys Corporation (generally such algorithms were publicly available without the need for a toll license). More specifically, the "LZW" compression algorithm takes each input sequence of multiple bits of a given length (eg 12 bits) and is often "dictionary" or "dictionary" or "" for that particular bit pattern. Create an entry in a table called a "code book". The entry consists of the pattern itself and shorter code. When the input is read, any previously read pattern replaces the shorter code, effectively compressing the total amount of input to a smaller one. Earlier approaches known as LZ77 and LZ78 did not include the lookup table as part of the compressed file. However, the more recent LZW algorithm format includes a table in the file, and a decoding program that decompresses the file for viewing can build the table using the algorithm as it processes the encoded input. The GIF format is binary encoded using the 2D raster data type.

Two versions of the GIF format are in GIF87a, a more recent GIF89a that allows you to create "video GIF" files, or in a single GIF file that is played in a sequence to represent behavior or changes in an image. A short sequence of multiple images (either in an endless loop or through progression to reach the end). GIF89A is also acceptable with "interlaced GIFs", which are incrementally in seven continuous waves consisting of a bitstream that arrives at the recipient and fills the missing lines by the recipient until full resolution is achieved. Displayed as a fuzzy outline of the replacement image. Interlaced GIFs are intended to monitor shorter wait times before certain information in a subject image can be processed, for example to make a decision (such as clicking on an image to perform an operation such as a link). Viewers are enabled using 14.4Kbps and 28.8Kbps modems.

By displaying a wave of resolution that meets the image sequence, an interlaced GIF describes an image created using a JPEG set consisting of a compression program that will "fade in" in a continuous wave. It is the same as "Progressive JPEG". Progressive JPEG is often noted as a more attractive way to deliver images at modem connection speeds, but users with faster connections will probably not notice the difference.

"PNG" or "Protable Network" The "Graphics" format was recently developed for image compression and was eventually published to replace GIF for Internet use (the JPEG format generally does not tolerate size / quality compromises). This format was developed for public consumption and development. Like GIF, PNG is a "lossless" compression format, and therefore all image information is restored when the compressed file is decompressed during viewing. However, PNG-formatted files are generally intended to be compressed by 10 to 30 percent more than GIF format. A further aspect of the PNG file format is that (i) color slides cannot be limited to just one color, transparency can be controlled (opaque), and (ii) interlacing of images Improved over standard GIFs, (iii) "gamma correction" is possible, allowing the "tuning" of images, called color brightness, required by certain display manufacturers, (iv). ) Images are saved using genuine colors, palettes, and grayscale formats similar to GIFs, (v) "animations or videos" are generally not supported, but PNGs are generally It can be extensible and therefore layered to provide scriptable image animation.

"TIFF" is an acronym for "Tag Image File Format" and is a general term for exchanging raster graphic (or "bitmap") images between application programs, such as graphics used for scanner images. Format. TIFF files are usually given the subscript ".tif" or ".tiff" and were generally developed in the mid-1980s with the support of Adobe Software, Microsoft, and Hewlett-Packard. .. TIFF files can be in any of several categories, including grayscale, color palettes, or RGB full color, the description and differences being further expanded elsewhere in this disclosure. TIFF files also include files with JPEG, LZW, or CCITT Group 4 standard run-length image compression, which are further described elsewhere here. As one of the most common graphic image formats, TIFF files are typically used for desktop publishing, faxing, 3-D applications, and medical image formation applications.

Video CODEC Video compression is a strongly developed subject for a variety of applications, including pre-recorded video (such as "Video on Demand"), teleconferencing, and live video (such as broadcast). hand. "Desktop" computers, wireless devices, traditional televisions, and high-definition televisions are examples of the various types of receivers that efficient video compression systems must support.

In general, the video CODEC algorithm is based on individual frame units or / or to "temporary compression" where each frame is the most common video compression of conventional use and is based on some mathematical principles. It works on either or both of the based, including the individual cosine transform (DCT), the wavelet (wavelet) transform, and the pure fractal.

The "individual cosine transform" or "DCT" is the most popular transform used in image compression applications. In general, DCT is a technique for representing waveform data as a total weighted cosine. The DCT transforms a signal or image from the spatial domain to the frequency domain, similar to the individual Fourier transform. The DCT helps separate the image into different parts (or spectral subbands) of different importance (in terms of the visual quality of the image). The reason for its popularity is not only the good performance of energy compression for typical images, but also the effectiveness of some fast algorithms. DCT is used in two international image / video compression standards, JPEG and MPEG.

A "wavelet transform" is generally a mathematical algorithm that transforms signal data into a set of mathematical representations, many of which can be decoded by an arrival receiver, such as the Fourier transform. .. Wavelets have been noted for enhancing the recovery of weak signals from noise, and therefore images processed in this way can be improved without significant blurring or ambiguity in detail. For this reason, wavelet signal processing has been specifically applied to medical applications, especially X-rays and magnetic resonance imaging. In Internet communications, wavelets have been used to compress images to a greater extent than is generally possible with other conventional methods. In some cases, wavelet-compressed images can be reduced to about 25 percent of the size of similar quality images using the more common JPEG format, which will be discussed in more detail elsewhere in this disclosure. .. So, for example, a video that requires 200Kb in JPEG format and takes a little time to download can only require 50Kb in wavelet compression format and only 15 seconds to download. .. Wavelet compressed image files are often given the subscript ".wif" and the receiver (eg, an internet browser on a computer receiver) must support those format files or plug in. -The in-program will be needed to read such files.

Fractal image compression is a modern technology for lossy image coding, offering some improvements over existing Fourier series compression schemes. When modeled as a step function, the edge depiction is improved because the edges require a large number of Fourier series terms for proper depiction. Other advantages of fractals include fast decoding time and scale independence. Fractal compression is based on the Mandelbrot set, which utilizes the natural self-similar scaling dependent statistical function (Mandelbrot 1983). Fractal compression and decompression involves a clustering approach that finds regions that exhibit the same properties as sample regions that are independent of rotation and scale. Fractal images compress images as inductive equations and instructions on how to reproduce them. These equations describe the image in terms of the relationships between its components. The reduction in memory need is due to the fact that fractal compression saves equations and instructions instead of pixel representation of the image.

"MPEG" is an acronym for "Moving Picture Experts Group" and has come to be used synonymously with certain advanced video compression standards and the audio compression standards that have spread from them. In general, personal computers require sufficient processor speed, internal memory, and hard disk space to handle and play typically large MPEG files in order to use MPEG video files, usually with the subscript ". "mpg" is given. The identified MPEG viewer or client software that plays the MPEG file must be valid on the client system, and generally the software or version of a commercial MPEG player from various sites on the web Can be downloaded. Modes of operation for MPEG formatted media are described herein by reference to these sequentially expanded standards below.

More specifically, the MPEG-1 standard was generally designed for coding progressive video with transmission rates of about 1.5 Mbps. It was generally designed for specific applications on video-D and CD-I media. MPEG-1 Audio Layer-3 ("MP3") also evolved from early MPEG research. "MPEG-2" is a standard generally designed for coding interlaced images with transmission speeds above 4 Mbps, and it is generally noted that many MPEG-2 players can also handle MPEG-1 data. And the opposite is not generally noticed to be true, and MPEG-2 coded video is generally incompatible with MPEG-1 players. Yet another progressive standard, "MPEG-3," has also been proposed for high-definition television ("HDTV"), but MPEG-3 is generally believed to meet HDTV requirements with MPEG-2. It was merged. Finally, the "MPEG-4" standard was also most recently developed and is intended to provide a considerably more ambitious standard for dealing with speech and video composition, fractal geometry, and computer visualization. Further disclosed to incorporate artificial intelligence to reconstruct.

The MPEG-1 and -2 standards specify techniques for compressing digital video by factors that vary from 25: 1 to 50: 1. This compression is five different compression techniques: (i) individual cosine transform (DCT), frequency-based transform, (ii) "quantization", a technique that loses selective information. For example, the fact that some images are predicted from the images immediately before and after the images is generally achieved according to these standards.

More detailed examples of commercially available compression technologies include Microsoft Media Player (trademark) (available from Microsoft Corporation), RealPlayer (trademark), and RealSystem G2. (RealSystem G2) (trademark) (commercially available from RealNetwork (trademark)), Apple's QuickTime (trademark) (commercially available from Sorenson (trademark)), and Includes "VDO". Microsoft Media Player is generally believed to apply to the CODEC's MPEG standard for compression / decompression, but others have been argued to use CODEC's proprietary types. Standard compression algorithms such as MPEG4 have put them in the hands of developers building enterprise streaming, security, and embedded systems for the like.

An example of a more recent study providing streaming video solutions across wireless and IP networks was published by a company called Emblaze Systems (LSE: BLZ). The company has disclosed specific technologies intended for coding and playback of live and on-demand video messages and content on any platform such as PCs, PDAs, video mobile phones, and interactive television. .. Embrese Systems is formerly believed to be the GEO Interactive Media Group. The following international patent applications as published Patent Documents 1 and 2 appear to be associated with Embrese Systems to the extent that GEO Interactive Media Group has been nominated as an "assignee." The disclosures of these references are hereby incorporated throughout them by reference to them.<patcit num="1"><text>Carmel et al WO9731445</text></patcit><patcit num="2"><text>Carmel's WO9910836</text></patcit>

Another company that has published CODEC technology intended to improve streaming media communication for wireless applications is Packetvideo Corporation, more specifically to bring streaming video to mobile phones. Intended to communicate. In addition, they believe in encouraging CODEC technology, which is intended to track temporary scalability and signal error resistance, protecting video and audio streams from the hazards of wireless environments. Patent Document 3 below discloses a further example of a particular streaming media compression / decompression technology that appears to be associated with PacketVideo as the designated "assignee" in the patent document. The disclosure of this patent document is incorporated herein by reference in its entirety.<patcit num="3"><text>Lengweha satit U.S. Pat. No. 6,167,092</text></patcit>

Another leading reference provides CODEC technology intended to provide a cost-effective and continuously adaptable digital video system and method for compressing color video data for moving images. It is disclosed. This method captures analog video frames and has unique lossy (lossy) and lossless (lossless) subband coding, wavelet transform, motion detection, run length coding, and Includes digitizing an image into a suitable source input format for compression using a combination of digital compression technologies that includes variable length / coding.

Another leading reference provides CODEC technology intended to provide a cost-effective and continuously adaptable digital video system and method for compressing color video data for moving images. It is disclosed. This method captures analog video frames and digitally includes subband coding, wavelet transform, motion detection, run length coding, and variable length / coding, with and without the inherent loss. It involves digitizing an image into a suitable source input format for compression using a combination of compression technologies. This system is an encoder and decoder generally disclosed to use a "Huffman encoder" for compressing and decompressing visual images that provides high compression intended to provide good to excellent video quality. Includes (CODEC) indicator. Compressed video data provides basic video and an additional layer of compressed digital audio and multiplexed video, including an internet or wide area wireless network. Provides a data stream that can be packetized for distribution across the intranet. The disclosed CODEC system is intended to continuously adjust the compression of digital images on a frame-by-frame basis as the effective bandwidth on the data channel is compared to the effective bandwidth on the channel for the preceding frame. Provides an output data stream that corresponds to the effective bandwidth of the network transmission channel and the receiver resource capacity of the client user. The compression can be further adjusted by adjusting the frame speed of the output data stream.

Further detailed examples of CODEC systems intended to be used in at least a portion of streaming video communication are disclosed in Patent Documents 4-10 below. This allows the disclosure of references to be incorporated here throughout them by reference to them.<patcit num="4"><text>Adolph et al U.S. Pat. No. 6,081,295</text></patcit><patcit num="5"><text>Guetz et al, U.S. Pat. No. 6,091,777</text></patcit><patcit num="6"><text>Lei U.S. Pat. No. 6,130,911</text></patcit><patcit num="7"><text>US Pat. No. 6,173,069B1 of Daly et al</text></patcit><patcit num="8"><text>Gardos et al U.S. Pat. No. 6,263,020B1</text></patcit><patcit num="9"><text>Murakami et al, U.S. Pat. No. 6,272,177</text></patcit><patcit num="10"><text>Lei U.S. Pat. No. 6,272,180B1</text></patcit>

Most, if not all, advanced streaming video compression methods focus on the extremely complex mathematical tools within such CODECs and the subtle changes to them: "one size fits all. ) Video is transmitted across all types of public and private networks from ultra-low bandwidth networks such as those found in wireless networks to ultra-fast fiber optics equipment via satellite communications. Among the various traditional methods of compression, user-specified parameters that include compromises between image size, frame size, color depth, contrast, brightness, sensed frame quality, buffer length, etc. Is generally. In addition, there are many quality and weighting calculations within the algorithm itself that are not user-specified. It is up to the developer to set these once for one "general" concern and then package and ship the product.

However, while the video streaming market continues to grow rapidly, the world also has one standard for compression, as there is no ideal algorithm for every video source, destination, or mode of transmission. I haven't selected it. The first CODEC may be best for one type of signal or the first part of the signal (eg, a frame or scene containing a series of frames), while another second CODEC is the signal. May be best for another type of or yet another second part of the signal. Furthermore, one CODEC may best fit the compression / decompression of a particular streaming signal between transmission, reception, and transmission devices in a communication network, and another second CODEC is the communication device. It may fit better than the first, which fits for the same streaming media signal, rather than for another set of parameters. For example, some video streams deliver color to handheld devices, while others can take advantage of pixel loss in black-and-white transmission to mobile phones to increase frame speed. All of the required sound quality, frame speed, clarity, and buffering tolerances categorically shock the selected compression algorithm for optimized video and audio delivery across multiple platforms.

In practice, certain communication device parameters may not last long enough during streaming media transmission, so an initially appropriate CODEC for an initial set of parameters would be these parameters during signal transmission of the same stream. It can be made less efficient than another CODEC due to changes in. Examples of such short-lived parameters are, without limitation, the effective bandwidth in the data transmission channel, the effective memory or processing power in either the transmitter or receiver, and the dedicated display resolution / window in the receiver (eg, the receiver). Minimize windows on the screen). These issues allow one CODEC to be distinguished from the other so that it is most efficient for compressing and decompressing certain streaming media signals and delivering those signals along a particular communication equipment system. It is exponentially mixed by a large number of repetitions of various combinations of multiple factors.

Since the CODEC system is "format specific", the source and arriving devices must be "preconfigured" to communicate media signals between each other according to a common specific compression / decompression mode and are used by the transcoder. It must be. However, even if a conventional transcoder is used, constraints in the communication system (eg, source, transmission channel, arrival device) are generally not considered and the communication can be significantly incomplete. For further explanatory purposes, FIGS. 1A and 1B show two different schematics of conventional methods of communicating media between sources 110-120 and arrival devices 130-140. These illustrations specifically illustrate streaming video communication, but other media forms may be displayed by the system as well.

It has been noted that the CODEC algorithm can be modified for a particular application and can work over more restricted examples than a similar unmodified set. However, in general this must be done for a series of frames, or ideally for each individual frame. Some DCT-based algorithms have a large number of mathematical operations, about 2 billion, that occur for each frame, with higher resolution and lower perceived quality. In general, this is too much mathematical work to run 30 to 60 times per second, even for the average machine or even a commercial server. This is the reason for the advent of dedicated compression boards or ASICs.

Audio codec In addition to recent social concerns about improving video compression, audio compression also includes audio broadcasts, music, transmission synchronized with video, and various live performances including live two-way audio (eg telephone). Alternatively, it has become a topic of remarkable research and development for pre-recorded application examples. Any or all of these audio compression applications are compatible with a wide range of client-side receivers / players, such as on a large number of handheld or desktop devices with widely variable capabilities and operating parameters. There must be.

Traditional audio codecs generally include several different types, a few of which are briefly summarized here for explanatory purposes.

"Code Excited Linear Prediction" or "CELP" uses "Analysis-by-Synthesis" or "AbS" within the excitation filter framework for the waveform code of the target signal. This is one type of speech compression method using the waveform CODEC. CODEC, which is based on CELP, has recently gradually evolved as the predominant technology for high-quality speech compression, and has been announced to transmit paid-quality compressed speech at data speeds as low as about 6 kbps. However, at least one publication discloses that the sound quality of CELP coded speech is significantly reduced for bit rates below 4 kbps.

"Vocoders" are speech codecs that are not based on waveform coding schemes, but integrate reconstructed output speeches using quantized parameter descriptions of target input speeches. Vocoders have been disclosed to deliver better speech quality at low bitrates such as about 4kbps and have been developed for such applications. Low bitrate vocoders use the periodic characteristics of spoken speech and the "noise-like" characteristics of static non-voiced speech for speech decomposition, coding, and integration. Some early versions of Vocoders (eg, Federal Standard 1015 LPC-10 use time domain decomposition and integration methods, however, of more recent versions that at least one publication labels as "wave coder". Most utilize a tuned spectrum model for the voiced speech segment.

Despite the prior description of certain speech compression techniques, a plethora of speech codecs and standards have been developed in industry and managed by industry and nonprofits. Examples of such organizations include, without any restrictions, and those standards are often used as CODEC reference types. That is, the European Telecommunications Standards Institute (ETSI), the Institute of Electrical and Electronics Engineers (IEEE), and the International Telecommunication Union Standardization Division (ITU-T), formerly CCITT.

One more recently disclosed method and device for hybrid coding of speech identified at 4 Kbps encodes the speech for communication to the decoder for speech reproduction, and the speech signal is (i) stable. There are three types of state speech or "tuning", (ii) static devoicing, and (iii) "temporary" or "transition" speech. A particular type of coding scheme is used for each classification. Harmonic coding is used for stable state speech, "noise-like" coding is used for static non-speech speech, special coding mode is used for transition speech, and local time events that characterize the transition part of the speech. Designed to capture the location, structure, and strength of the. The compression scheme is intended to be applied to speech signals or LP residual signals.

Other recently disclosed methods and devices that add new speech encoding methods to existing telecommunications systems are also summarized below. CODECs are introduced in speech transmission transceivers in digital telecommunications systems to use "new" and "old" CODECs in parallel within the system. The CODEC was chosen by embodying the speech encoding method within all transceivers and the hand-shaking procedure between transceivers as previously used within the relevant telecommunications systems. To. Handshaking is used at the beginning of each connection. After the beginning of the phone call or the story, this method investigates whether both use the new speech encoding. Hand-shaking messages are selected to minimize their impact on the sound quality of the speech and maximize the likelihood of identifying the message.

Yet another relatively recent reference discloses a harmonious perceptual weighting filter for tandem CODECs intended for speech compression. Certain filter parameters are harmonized to provide improved performance in terms of vertical context. More specifically, the parameter used is the 10th LPC Prediction Factor. This system has been specified to use a "Low-Delay Excited Linear Predictive" CODEC or "LD-CELP".

Further detailed examples of the streaming audio communication system using the CODEC according to the examples described above are provided in the US patents of Patent Documents 11 to 13. The disclosure of these references is incorporated herein by reference throughout them.<patcit num="11"><text>Chen et al, U.S. Pat. No. 6,144,935</text></patcit><patcit num="12"><text>Haavisto et al, U.S. Pat. No. 6,161,085</text></patcit><patcit num="13"><text>Gersho U.S. Pat. No. 6,233,550</text></patcit>

Neural network with artificial intelligence ("AI") and CODEC Recently, various systems and methods have been disclosed and are intended to integrate neural networks with compression and decompression of artificial intelligence (AI) or streaming media signals.

The term "artificial intelligence" is intended here to mean the simulation of a person's intelligent process by a computer system, learning (acquisition of information and rules for using that information), reasoning (using rules, using rules, To reach an approximate or clear conclusion), as well as self-correction. A particular application of AI includes an "expert system", which is a computer program that mimics the judgment and behavior of a person or an organization with expertise and experience in a particular field. Typically, an expert system may include a knowledge base for each particular situation described in the program and may be enhanced by adding to the knowledge base or set of rules.

The term "neural network" refers to the behavior of the human brain, which usually contains a large number of processors operating in parallel, each with its own small knowledge sphere to access data in its local memory. It is intended herein to mean a system of programs and data structures that approximate. Typically, neural networks are initially trained or fed with large amounts of data and data-related rules, after which the program sends the network in response to external stimuli (eg, input information). Can teach you how to behave. When making decisions, neural networks use several principles, including unrestricted gradient-based training and fuzzy logic. Neural networks are further described in terms of knowledge layers and generally include more complex networks with deeper layers. In a "feedforward" neural network system, relationships about learned data are "feedforwarded" to a higher layer of knowledge. Neural networks can also learn transient concepts and have been widely used in signal processing and time series analysis. Other published applications of neural networks include petroleum exploration data analysis, weather forecasting, and interpretation of nucleotide series at the Institute of Biology.

The term "fuzzy logic" refers to an approach to calculations based on "truth" rather than just "binary logic" operating within a "binary" area such as "1 or 0". Intended here. Fuzzy logic was first proposed by Dr. Lotfi Zadeh of the University of California in Berkeley in the 1960s in connection with his work on the problem of computer comprehension of natural languages that are not easily translated into absolute binary logic terms. Fuzzy logic often includes the extreme cases of truth, 0 and 1, but also includes various intermediate truth states (eg, the existence state is some threshold, such as 0.98). The decision can assist the decision to assign 1 to the tolerable low occurrence of error in the calculation).

An example of a previously disclosed streaming media compression / decompression system intended for use over neural networks with artificial intelligence is the Radon conversion to compress data such as video data. Is used. Some previously disclosed systems of AI and / or neural networks use AI and / or neural networks for the purpose of error correction during the use of certain specified lossless compression CODECs. Is intended. For example, a learning system is used to determine the difference between what is received by the receiver after compression and transmission and what is expected to be received at the transmission end. This difference is treated as learning and changes the CODEC's tuning for additional transmissions.

Another example of the disclosed method and device is intended to extrapolate past signal-history data for insertion into a lost data segment to conceal digital speech frame errors. This extrapolation method uses the past signal history stored in the buffer. This method utilizes an artificial neural network trained by finite impulse response (FIR), multi-layer, feedforward, and backpropagation of speech compression algorithm (SCA) parameters for one-step extrapolation. It is embodied by the disclosed device. Once the speech connection is set up, the speech compression algorithm device begins sending coded speech frames. When speech frames are received, they are decoded and converted back to the speech signal voltage. Since it is regular, the required SCA parameters are preprocessed during the coding process, and the result is stored in the past history buffer. If a speech frame is detected as lost or inaccurate, the extrapolation module is executed and an alternative SCA parameter is generated and sent as the parameter required by the SCA. In this way, the transfer of information to the SCA is intended to be transparent and the SCA process continues normally. This disclosure alleges that the speech frame has been lost due to a smooth transition between what was received at the end of the speech frame, what was lost, and what was next received. Is usually unnoticed.

A more detailed example of a system intended to use artificial intelligence and / or neural networks within the system for media compression and / or decompression is generally a media type specific CODEC method (eg, eg). It is related to speeches and videos) and is variously disclosed in the US patents of Patent Documents 14 to 23. Further examples are provided in Patent Documents 24 and 25, which are published international patent applications. The disclosures of all those references in this paragraph are incorporated here throughout them by reference to them.<patcit num="14"><text>U.S. Pat. No. 5,005,206 of Naillon et al</text></patcit><patcit num="15"><text>Yoshida et al, U.S. Pat. No. 5,041,916</text></patcit><patcit num="16"><text>Gerdes U.S. Pat. No. 5,184,218</text></patcit><patcit num="17"><text>Burel et al, U.S. Pat. No. 5,369,503</text></patcit><patcit num="18"><text>Fang et al U.S. Pat. No. 5,598,354</text></patcit><patcit num="19"><text>Kurdziel U.S. Pat. No. 5,692,098</text></patcit><patcit num="20"><text>Fang et al U.S. Pat. No. 5,812,700</text></patcit><patcit num="21"><text>Imade et al U.S. Pat. No. 5,872,864</text></patcit><patcit num="22"><text>Prieto, Jr. U.S. Pat. No. 5,907,822</text></patcit><patcit num="23"><text>Mitchell U.S. Pat. No. 6,216,267</text></patcit><patcit num="24"><text>Rising WO 01/54285</text></patcit><patcit num="25"><text>Naillon et al's EPO 0372608 A1</text></patcit>

Other disclosures of CODEC systems with feedback, or other systems that manipulate CODECs for processing various streaming media signals, are likely to use the label "AI" or "neural network" in particular. Although not, it is disclosed in the US patents of Patent Documents 26-29. The disclosures of these references are incorporated herein by reference to them throughout.<patcit num="26"><text>Betts et al, U.S. Pat. No. 6,072,825</text></patcit><patcit num="27"><text>Malvar U.S. Pat. No. 6,182,034 B1</text></patcit><patcit num="28"><text>Malvar U.S. Pat. No. 6,253,165 B1</text></patcit><patcit num="29"><text>Malvar U.S. Pat. No. 6,256,608 B1</text></patcit>

Bandwidth that fluctuates despite significant advances in the CODEC algorithm itself, and despite the previously intended use of AI and other feedback systems that operate the CODEC to improve compression efficiency with respect to communication. There is a significant need for further improvements in the ability to efficiently deliver a wide variety of streaming media signals to a wide variety of arrival receiver devices across a wide variety of transmission channels, depending on the width and communication protocol.

There is a further need to incorporate AI and / or neural networks and apply them to the appropriate CODECs for communication of streaming media signals based on various parameters, with these various parameters being unrestricted: Includes one or more of them. (a) Automatic selection of properly optimized CODECs from a library of valid CODECs of different types and operations, especially with artificial intelligence and / or standards compared to other CODEC operations of the selected CODEC operation. Including those based on artificial intelligence, (b) knowledge of pre-learning and / or iterative learning of specific CODEC operations within a given set of operational parameters representing existing situations, and ( c) Tuning to the appropriate CODEC reference parameters based on the intellectual knowledge of the operation regarding the existing and / or test status.

In particular, there is a further need for intellectual codecs that make the applied CODEC based on the existing situation specified by one or more of the following parameters, which are the streaming media signal itself. Parameters, transmission channel capacity and constraint parameters, and receiver device capability and constraint parameters are included.

In addition, there is a further need for intelligent codec systems that operate with intellectual knowledge of their manipulation and context parameters in order to optimize proper compression, transmission, decompression, and reproduction of thematic streaming media signals. is there.

<u style="single">Traditional transcoder for streaming media</u> Also, one of the recent concerns in the field of streaming media communication is to provide intercommunication between a wide array of "format-specific" encoding systems in current use. The existing disciplines of various different format-specific systems and pre-encoded content create a broadly divided ability to process encoded content, resulting in significant compatibility issues between content providers and client users. It is creating a predicament. If a client wants to see or hear streaming content from a particular source, and that content needs to pass through a CODEC for compression, a compatible CODEC will decompress to enjoy the signal. Must be used on the client side for. Unfortunately, source content is often combined with only a few and often only one specific CODEC scheme. Therefore, if a client requests such coded content (or if the source wants the coded content to be delivered to a particular client), then one of the two criteria must be met. That is, (1) the client must download or retain a format-specific CODEC (decoder), or (2) the source media must pass through a "transcoder" and the source media must be passed. Decode from the first format to a second format compatible with the client's device / system. The term "transcoder" is intended herein to mean a system that translates a media signal from one coding (ie, compressed) format to another.

Various techniques for transcoding one media format to another have been previously disclosed. Figure 1C shows an example that helps explain one of the general processes that is unique to many known transcoding techniques. More specifically, request 159 is first received from a particular type of device or player for content that initially exists in an incompatible format. Request 159 from Microsoft Media is received for Real video content according to the specific example shown in Figure 1C. Since the content is clearly limited and requested, the content is decoded from its initial format (eg, real-encoded format) and then suitable for the requesting player (eg, Microsoft Media format). The recoded media is then "recoded" into a format that is useful for the requesting client as it is within the rest of the player's system.

This traditional system has significant scalability limitations and simultaneous supply to multiple channels for multiple clients must be supported by an equivalent number of transcoders. For example, Figure 1D shows a schematic embodiment of the traditional transcoding technology described earlier to manage four simultaneous stream requests from four Microsoft media players, where the requested content is real ( Initially coded in a trademark) format. This system architecture, which needs to support four encoders 151-154 and four decoders 155-158, requires significant computational resources. For example, it is convinced that each encoder 151-154 provided in this example has 128 Mbytes of effective RAM and has 600 MHz (eg, Pentium® III), or 256 Mbytes of effective RAM. It is necessary to have a computer having a dual 400MHz processor (eg, Pentium® II) equipped with. Further convinced is that each decoder 155-158 requires a 233 MHz machine with 64 Mbytes of effective RAM (eg, Pentium® II). So, four such streams are equivalent to the Quad 900 Xeon (available from Compaq, Hewlett-Packard, Dell, and IBM, estimated to be around $ 9,000 at retail price at the time of this filing disclosure). Needs. This is for four simultaneous streams, and society is currently demanding thousands of simultaneous streams.

There is still a need for a transcoder system that can cost-effectively convert a large number of format-specific coded streaming media signals to a large number of other formats using minimal computational resources.

<u style="single">Parameter-influenced media communication</u> For the purpose of further explaining the many variables that can shock the selection of the appropriate CODEC to communicate a particular streaming media signal to the desired target, the following are various different types of streaming video formats. Provides a brief summary with the processing system. It is convinced that each of these different systems requires a different type of compression mode (eg, CODEC), communication and streaming media signals, taking into account effective transmission speeds and bandwidths. Optimize with receiver processing parameters.

Specific types of communication formats and systems are described in more detail here, but Table 1 below shows a variety of different communication systems and systems that are currently valid and disclosed in consideration of effective speed or bandwidth. It provides a summary of important cross-sections of transmission carriers.

<tables num="1-1"><img file="JP4852228B2_D0001.tif" /></tables>

<tables num="1-2"><img file="JP4852228B2_D0002.tif" /></tables>

<tables num="1-3"><img file="JP4852228B2_D0003.tif" /></tables>

<tables num="1-4"><img file="JP4852228B2_D0004.tif" /></tables>(i) The term "Kbps" is an abbreviation for "thousands of bits per second". In international English outside the United States, the use of equivalents is "k bits s"^-1Or "k bits / s". (ii) Technicians use data rates rather than speeds, but speeds (such as "Why my web pages aren't faster here?") Are more meaningful for less technical trends. It looks like there is. (iii) A related term, bandwidth or "capacity" for data communication means how wide the pipe is and how quickly bits can be transmitted to the channels in the pipe. That is, data on multiple signal channels within a carrier is typically distributed by channel for different applications or between different users. Keys: (i) "T" = T-carrier system in the United States, Canada, and Japan ... (ii) "DS" = digital signal (moving on T-carrier or E-carrier) ... (iii) "E" = equivalent of "T" using a total of 8 bits per channel; used in the United States, Canada, and countries other than Japan ... (iv) "OC" = optical carrier (synchronous light) Network) "STM" = Synchronous Transport Module (see Synchronous Digital Hierarchy). (v) Only the most common technologies are shown. (vi) "Physical medium" is generally described and does not specify the classification or number of twisted pair pairs or whether the optical fiber is in single or multiple mode. (vii) No effective distance for technology has been shown. (viii) There are published standards for many of these technologies. Kabul Modem Note: The 52Mbps limit on cable is for ISPs, not for personal computers at this time. Most PCs today are limited to internal designs that can accommodate less than 10 Mbps (the PCI bus itself carries data faster). The 52Mbps cable channel is split between individual and separate users. Obviously, the faster the channels, the fewer channels are needed by the ISP and the lower the cost of supporting individual users.

Internet carrier system Streaming video communication over the Internet can take place across a variety of modes of communication, such as digital subscriber lines (DSL), T1 lines, cable modems, and traditional telephone services (POTS). -Includes dial-up modems as well as wireless carriers. Although a number of different wireless communication style descriptions are dealt with individually here, various summaries of these other communication modes are immediately provided here for the purpose of further explanation, such as: To.

The terms "POTS", "traditional telephone service", or "dial-up" as applied to communication transmission channels are interchangeably used herein. These terms generally refer to "narrowband" communication that connects end users in a home or small business to a telco office via a copper wire that is wound around each other or in a "twisted pair". Intended to do. Traditional telephone services exchange audio information with other telephone users via analog signals that represent acoustic analog signals converted to electrical equivalents in terms of volume (signal amplitude) and pitch (frequency change). It was created to make you do it. Since the telephone company's signal transmission has already been built up for this analog wave communication, it is easy to use it as a way to move information back and forth between each telephone and the telephone company. Therefore, dial-up modems are used to demodulate analog signals and convert their values into strings of 0 and 1 values called digital information. Since analog transmission uses only a small portion of the effective amount of information that can be transmitted over copper wire, the maximum amount of data that can be received using a regular modem is about 56 Kbps. The ability of your computer to receive information is modified by the telephone company to filter the information that arrives as digital data, convert it to analog form for your telephone line, and convert it back to digital on your modem. It is constrained by the fact that it requires that it be done. In other words, analog transmission between your home or company and your telco is a bandwidth bottleneck.

Data rates coming in at up to about 128 Kbps can be achieved for some end-user clients by "ISDN" or "Internet subscriber digital networks", which are somewhat considered DSL-restricted precursors.

A "DSL" or "digital subscriber line" is commonly defined as a "broadband" transmission carrier that communicates high bandwidth communications over conventional copper telephone lines. Many different types of DSL services have been disclosed, but generally have variable data rates and intended uses. Further discussion is provided here for specifics of these DSL types, but Table 2 below provides a summary of information for specifics of these DSLs for the purpose of further developing the overall understanding. doing.

<tables num="2-1"><img file="JP4852228B2_D0005.tif" /></tables>

<tables num="2-2"><img file="JP4852228B2_D0006.tif" /></tables>

Typical published data rates for DSL services that can vary depending on the distance from the provider's central station include rates up to 6.1 Mbps (theoretically published at 8.448 Mbps) and that value. Is believed to enable continuous transmission of motion video, audio, and 3-D effects. A more typical individual connection provides a downlink from 512Kbps to 1.544Mbps and an uplink of about 128Kbps. The DSL line can carry both data and audio signals, and the data part of the line is continuously connected. DSLs have been predicted in several publications to replace ISDN in many areas and compete with cable modems for home and business multimedia communications. DSLs operate purely within the digital domain and do not require a change to analog form and vice versa. The digital data is transmitted directly to the arriving computer as digital data, which allows the telco to use a much wider bandwidth for forward transmission. On the other hand, if the client user chooses, the signal can be separated and some of its bandwidth can be used to carry the analog signal and the telephone and computer can be used on the same line and at the same time. It will be.

Most DSL technologies require a signal splitter to be installed in a home or store, which requires the cost of a telephone company visit and installation. However, it is possible to manage the split or split remotely from the central station. It is known as splitless DSL, "DSL Lite", G. Lite, or Universal ADSL (more defined below) and has recently become the norm. Several modulation technologies are used by various types of DSLs, which are standardized by the International Telecommunication Union (ITU). Different DSL modem manufacturers use either DMT (multicarrier modulation) or CAP (carrierless amplitude phase modulation). A third technology, known as Multiple Leased Line (MVL), is another possibility.

The various parameters of DSL operation are variables that affect the achievable and effective data rate. DSL modems generally follow data rate multiples set by North American and European standards. Generally, the maximum effective range for repeaterless DSLs is 5.5km (18,000ft). As the distance to the telephone office decreases, the data rate increases. Another factor is the gauge of copper wire. The heavier 24-gauge wire can carry the same data rate much farther than the 26-gauge wire. For arrivals that exceed the 5.5-kilometer range, DSLs can still be provided, even if each telco provider simply extends the local loop with fiber optic cables.

To interconnect multiple DSL users with a high-speed network as the "backbone," telcos use digital subscriber line access multiplexers ("DSLAMs"). Typically, this DSLAM connects to an asynchronous transfer mode (ATM) network that can sum up data transmission at gigabit data rates. At the other end of each transmission, the DSLAM decomposes or demultiplexes the signals and sends them to the appropriate individual and separate DSL connections.

"ADSL" or "asymmetric digital subscriber line" is a form of DSL and is most familiar to home and small business users. ADSL is called "asymmetric" because its two-way or "duplex" bandwidth is devoted to the downstream direction to send data to the user. Only a small portion of the bandwidth is useful for upstream or user bidirectional messages. However, most Internet data, especially graphics-media or multimedia-intensive web data, requires a lot of downlink bandwidth, but user requests and responses are small and requires very little uplink bandwidth. Using ADSL, data up to 6.1 megabits per second can be transmitted downlink and uplink up to 640 Kbps is possible. High downstream bandwidth means that the telephone line can carry operating video, audio, and 3-D images to the arriving computer or television player. In addition, a small portion of the downstream bandwidth can be devoted to voice rather than data, and telephone conversations can be carried without the need for a separate line. Unlike similar services over "cable" television lines, ADSL does not compete for bandwidth with nearby areas within a given area. In many cases, your existing telephone line will run with ADSL. Upgrades may be required in some areas.

"CDSL" or "Consumer DSL" is a trademark version of DSL, made available by Rockwell Corporation, which is somewhat slower than ADSL (1Mbps downlink, generally lower uplink). It has the advantage that there is no need to install a "splitter" at the user end (which is expected to be). The hardware may need to carry the CDS to the home or store by the local telephone company. CDSL uses its own carrier technology rather than DMT or CAP ADSL technology.

Various companies have worked with telcos in developing standards and easier installed versions of ADSL called "G. Lite," which is believed to be in development at the time of this disclosure. "G. Lite" or "DSL Lite" (also known as "splitterless ADSL" and "universal ADSL") is believed to be a slower ADSL in nature, which splits the line at the user end. Not required, but managed by the telco to remotely split for users and is believed to be of lower cost. G.Lite, officially the ITU-T standard G-992.2, has been announced to provide data rates for downlinks from 1.544Mbps to 6Mbps and uplinks from about 128Kbps to about 384Kbps. At least one publication predicted that G. Lite would be the most widely installed form of DSL.

"HDSL" or "High bite-rate DSL" appears to be an earlier version of DSL and is widely used for wideband digital transmission within corporate sites and among telcos and customers. The main characteristic of HDSL is symmetry, and equivalent amounts of bandwidth are valid in both directions. For this reason, the maximum data rate is generally lower than for ADSL. HDSL can be carried on T1 in North America, or in large quantities on a single twisted pair wire, as can be carried on E1 lines in Europe (up to about 2.32 Mbps).

"IDSL" or "ISDN DSL" is a somewhat incorrect name that is actually closer to the ISDN data rate and serves at about 128 Kbps compared to the considerably higher rates commonly associated with ADSL. ..

"RADSL" or "Rate-Adaptive DSL" is an ADSL technology available from Westell Company, whose software determines the rate at which a signal can be transmitted over a given customer telephone line. You can therefore adjust the delivery rate. Westel's FlexCap2 version system can use RADSL to deliver downlinks from about 640Kbps to about 2.2Mbps and uplinks from about 272Kbps to about 1.088Mbps over existing lines.

An "SDSL" or "symmetric DSL" is similar to a single HDSL with a single twisted pair line and carries about 1.544 Mbps (US and Canada) or about 2.048 Mbps (Europe) in each direction on the duplex line. Since the data rate is the same in both directions, it is called a symmetric type.

"UDSL" or "unidirectional DSL" has been proposed by European companies and is generally believed to provide a unidirectional version of HDSL.

"VDSL" or "Very high data rate DSL" is a significantly higher data rate over a relatively short distance, for example between about 51 Mbps and about 55 Mbps, over a line about 1,000 feet or about 300 meters long. It seems to be a technology under development with a promise. At least one publication predicted that VDSL could appear some time after ADSL was widely deployed and coexist with it. Communication technology (CAP, DMT, or others) and its effectiveness in some environments have not yet been determined. Many standards bodies are working on it.

The "x2 / DSL" is a 3Com modem that supports 56Kbps modem communication, but can be upgraded via the installation of new software on ADSL when the user's area becomes available. At least one publication cites 3Com as stating that the technology is "the latest modem you need."

The "T1" line is generally considered a "broadband" carrier and is defined as a type of "T-carrier" system, the United States in the 1960s as the first successful system to support digitized voice transmission. It is believed to have been first introduced by the Bell System. This T-system is entirely digital with pulse code modulation and time division multiplexing. The audio signal is typically sampled approximately 8,000 times per second, with each sample digitized into an 8-bit word. Thus, the simultaneously digitized 24 channels transmit 192 bit frames, each representing an 8-bit word, approximately 8,000 times per second. Each frame is separated from the following by a unit bit into a 193 bit block. The data rate published by the 1.544 Mbps T-1 represents 192 bit frames and 1 bit signaling bits multiplied by 8,000.

A T-1 system typically uses four wires to provide duplex capability, with two wires dedicated to reception and two wires dedicated to transmission at the same time. The T-1 digital stream contains 24,64Kbps channels to be multiplexed, and standard 64Kbps channels are based on the bandwidth required for voice conversation. The four wires were originally a pair of twisted pair copper wires, but more recent systems offer coaxial cables, fiber optics, digital microwaves, and other carrier technologies.

The original transmission rate (1.544 Mbps) for the T-1 line is commonly used today in Internet service provider (ISP) connections to the Internet. Another level, the T-3 line, has been announced to offer 44.736 Mbps, which is also commonly used by Internet service providers. Another commonly used service is the "fractional T-1", which is some rental of 24 channels on the T-1 line, with other unused channels.

Display capabilities / constraints and related standards Various different types of receiver display capabilities have also had a significant impact on the proper CODEC for efficiently communicating specific streaming media signals for display by the receiver. A brief summary of certain examples explaining such various display parameters (eg, resolution, clarity, color, depth, size, type / format identification, etc.) is provided below for a better understanding. ..

One parameter that is a large variable between different types and manufacturers of streaming media receiver devices, and therefore one parameter that can have a significant impact on the appropriate CODEC to be used is the display device or Is the range of colors that can be represented by the "palette". A standard "browser surf" palette that can be adapted by most software for Internet-based streaming media display can contain, for example, about 216 colors, but for web-based streaming media computer display capabilities are browser-based. Must be understood along with software capabilities.

For computer display technology, colors are set for each individual pixel or addressable lighting element for the screen. Each pixel has red, green, and blue (RGB) components. Individual colors are given to the pixels by specifying the amount of density or intensity for each of these components. "True color (true) "color)" displays generally use 24-bit values to specify the color of two pixels on the display screen, allowing up to 16,777,216 possible colors. The number of bits used to define the color shading of a pixel is called the "bit depth". Color is often referred to as "24-bit color," but many modern color display systems offer 32-bit color modes. The extra bytes, referred to as the "alpha channel", are typically used for control and special effect information. The "grayscale" (consisting of individual shades of gray) display setting is generally defined as having a depth of N bits, where N represents the saturation of black within the pixel. If N = 1, the image is not called grescale, but instead is monochromatic or black and white, because the bit can be just on or off and cannot contain any shading information. Is.

Typical computer resolutions are, for example, and without limitation. (i) VGA or video graphic array capability to display 640x480 pixels in 16 colors or 320x240 pixels in 256 colors with a pixel ratio of 4: 3. (ii) 800x600x6 bits / pixel (16 colors) or 650x480x8 bits / pixel (256 colors) SVGA or Super Video Graphic Array capability, which was created by the Video Electronics Association (VESA). (iii) XGA (v1-4) or Extended Graphic Array capability of 32,768 colors and 1024x768 pixels, Including matters of.

Additional standards such as SXGA have been added to specify pixel sizes greater than 1960x1440 and color depths of 32 bits / pixel or greater.

When a larger range of colors (or palettes) is used by a particular display or media signal that the browser can handle, most browsers are typically adapted to "dither" to color, so here. What is intended is to mean that the browser finds multiple colors in the palette that can replace any color outside the palette. Systems using Windows® (commercially available from Microsoft Corporation) and Macintosh® (commercially available from Apple Corporation) to further illustrate the wide range of various system display capabilities. Operating systems based on have no equivalent palette, and within a typical 256-color palette, 216 colors are common to both types of browsers, but 40 colors are different, and therefore the image signal is of those systems. Communication to the other system type in the format specified by one requires dithering by a browser running on the other system.

There are also many different technologies regarding how visual display is enabled from electronic information. The term "VDT" or "video display terminal" is commonly used within the computer industry and is intended here to be paraphrased by a simple reference to "display". For computer terminal use, VDT includes a computer surface and a projection mechanism that presents text and graphic images to computer users. VDTs can use a variety of specific display technologies, such as cathode ray tubes (CRT), liquid crystal displays (LCD), light emitting diodes (LED), gas plasma, or other image projections. Includes technology. A display is usually considered to include a screen or projection plane and a device that creates information on the screen. In some computers, the display will be packaged in a separate unit or "monitor", or the display will be fully integrated into a single unit with the computer processor.

Especially with respect to LCDs, this technology generally requires minimal volume and physical depth compared to other VDTs and is therefore typically used in laptop computers and mobile phones / PDAs. LCDs consume significantly less power than LEDs and gas-display VDTs, because they generally operate on the principle of blocking light rather than emitting light. The LCD can be either a "passive matrix" or an "active matrix", also known as a "thin film transistor" or "TFT" display. The passive matrix LCD has a grid of conductors, with pixels located at each intersection in the grid. Current is sent across two conductors in the grid to control the light for any pixel. The active matrix has transistors located at each pixel intersection and requires less current to control the brightness of the pixels. For this reason, the current in the active matrix display can be switched on and off more often to improve screen refresh time and therefore the effect on higher speeds of streaming media (action video). It is improving. Some passive matrix LCDs do dual scanning, which scans the grid twice with current in the same amount of time as one scan in earlier versions, but the active matrix is still the better technology of the two. Is generally considered to be. Reflective color display technology, a display structure that integrates color filters into a passive matrix, is an alternative to low power and low cost active matrix technology. Reflective LCDs perform particularly well during daytime outdoor use because they reflect ambient light. Various different display technologies and therefore transmission formats have also been developed specifically for television viewing. Therefore, several different standards have been developed for television transmission. These differences can have a significant impact on the nature and degree of compression (and hence the choice of a particular CODEC) desired for communicating streaming media in a television environment. These standards include standard definition television (SDTV) and high definition television (HDTV) without particular and limitation.

"SDTV" or "standard sharpness television" and "HDTV" or "high definition television" are two categories of display formats for digital television ("DTV") transmission and have become standard. There is. These formats provide video quality similar to digital universal discs (DVD) and are summarized below for their similarities and differences.

HDTV provides a higher quality display with a vertical resolution display from about 720p to at least about 1080i and an approximately 16: 9 aspect ratio (screen width to height) for a movie-like viewing experience. Equipped. The new television receivers will be either HDTV or SDTV capable with receivers capable of converting signals into their unique display formats. Co and HDTV through SDTV, using MPEG-2 file compression method, generally to reduce the digital signal from about 166Mbps to about 3Mbps. This allows broadcasters to transmit digital signals using existing cables, satellites, and terrestrial systems. MPEG-2 uses a lossy compression method, which compresses the digital signal sent to the television and loses some data, which is about how the human eye sees the image. Affects or cannot. Both ATSC and DVB chose MPEG-2 for video compression and transfer. MPEG-2 compression standards are described in more detail elsewhere here.

Because compressed SDTV digital signals are smaller than compressed HDTV signals, broadcasters can simultaneously transmit up to five SDTV programs instead of just one HDTV program, also known as "multicasting." Multicasting is an attractive feature because TV stations can earn additional revenue from the additional advertising offered by those surplus programs. With today's analog television systems, only one program can be transmitted at a time. It should be noted that this use of the term "multicasting" is distinguished from its use in streaming video when it is involved using special addressing techniques.

When the United States decided to move from analog television to DTV, the US Federal Communications Commission decided to let broadcasters decide whether to broadcast SDTV or HDTV programs. Most broadcasters have decided to broadcast SDTV programs during the day and HDTV programs during the Golden Hour. These SDTVs and HDTVs are supported by standard definition receivers for Digital Video Broadcasting (DTV) and the Advanced Television Systems Commission (ATSC).

HDTV as a television display technology offers video quality similar to that of 35mm movies, as well as sound quality similar to that of today's compact discs (and when it comes to audio quality, HDTV offers Dolby Digital 5.1. Receive, play, and output). Some television stations have begun to transmit HDTV broadcasts to users on a limited number of channels. HDTV generally uses digital rather than analog signal transmission. However, in Japan, the first analog HDTV program aired on June 3, 1989. The first images that appeared were the Statue of Liberty and New York Harbor. It required a 20MHz channel because analog HDTV broadcasting is rarely established in most countries.

HDTV offers a higher quality display than SDTV, with vertical resolution display from 720p to 1080i. This p stands for progressive scanning, where each scan contains each line for one complete image, and i stands for interlaced scanning, every other half of the image. Includes lines. These rates convert to frame rates up to 60 frames per second, which is twice that of heavy television. One of the most striking features of HDTV is its wide aspect ratio of 16: 9 (screen width vs. height), which leads to research that the viewer's experience is enhanced by a wider screen. It is a development based on the belief based on. The number of HDTV pixels ranges from 1 to 2 million compared to the range of SDTV from 300,000 to 1 million. The new TV receiver will be either HDTV capable or SDTV capable, and the receiver will be able to convert the signal to their unique display format.

In the United States, the FCC (Federal Communications Commission) has designated broadcast channels for DTV transmission. In the SDTV format, DTV allows the specified channel to be used for multiple signals at the current quality level instead of a single signal at the HDTV level, which allows more programs with the same bandwidth usage. Will be done. Commercial and public broadcasters are now meticulously deciding how they embody their use of HDTV.

Simulcast is the simultaneous transmission of the same television program in both analog and digital versions using two different channels or frequencies. At the end of the DTV transition period, it is believed that analog transmission will effectively replace and the current analog channels will be used exclusively for DTV. The surplus channels used for digital broadcasting may be used for other services such as more television channels or data broadcasting, for example at auction. Simulcast is for the transmission of simultaneous television and Internet services, the transmission of analog and digital radio broadcasts, and the transmission of television programs in various screen formats such as traditional and wide screen formats. Is also used. Simulcast broadcasts are used all over the world.

The move to DTV is not an easy or cheap move. For television stations that transmit DTV programs, DTV facilities must be built, and some stations must make money to build these facilities. Simulcast allows TV stations to continue to make money from traditional analog programming and also to make surplus revenue from surplus digital programming. Another obstacle to the transition to DTV is the lack of interest among consumers. The need for special equipment prevents viewers from seeing the differences between digital and analog programming, which also slows public interest in DTV.

The equipment required to operate a DTV depends on whether terrestrial services, cable services, or satellite services are used as transmission channels / carriers. In any case, according to known or foreseen systems, consumers must, at a minimum, purchase converters to watch DTV transmissions on their old television receivers. In addition, consumers who use terrestrial services or antennas to receive television signals require antennas equipped for digital signals. Consumers located in mountainous terrain in ATSC compliant countries may not be able to receive terrestrial digital signals due to multipath effects. This is common even in today's analog television systems. In DVB compliant countries, the ground does not affect the reception of digital signals. Satellite users are already enjoying DTV broadcasts, but larger satellite antennas will be needed to watch HDTV programs.

A "set-top" box is here as a device that allows a television receiver to be a user interface to the Internet while allowing an analog television receiver to receive and decode DTV broadcasts. Defined. DTV sets-Top boxes are often referred to as receivers. It is estimated that 35 million homes will use digital set-top boxes by the end of 2006, which is the end of the transition to DTV.

A typical digital set-top box typically includes one or more microprocessors to run the Linux or Windows® CE and to parse the MPEG transport stream. The set-top box also contains RAM, an MPEG decoder chip, and more chips for audio decoding and processing. The contents of the set-top box depend on the DTV standard used. A DVB-compliant set-top box contains parts for decoding COFDM transmissions, while an ATSC-compliant set-top box contains parts for decoding VSB transmissions. Higher-performance set-top boxes include storage for recorded television broadcasts, storage for downloaded software, and hard disks for other applications provided by DTV service providers. Digital set-top boxes can be used for satellite and terrestrial DTV, but are mostly used for cable TV. Set-top box prices range from $ 100 for basic configurations to over $ 1,000 for higher performance boxes.

In the Internet space, the set-top box often acts as a specialized computer that can, in practice, "talk" to the Internet, ie it is a web browser (actually a Hypertext Transfer Protocol client (Hypertext)). Includes Transfer Protocol client)) and TCP / IP, the main Internet program. The service that the set-top box binds to can be provided through telephone rotation such as web TV, or through a cable TV company such as TCI.

In order to use Dolby Digital 5.1 channels for satellite broadcasting, a satellite receiver that provides Dolby Digital output is required. For cable users, all digital set-top boxes are equipped with a Dolby Digital 2-channel decoder. To use 5.1-channel sound, you need a 5.1-channel compliant set-top box or an external 5.1-channel decoder unit.

The most dramatic demonstration of digital television benefits is through high-performance HDTV, because of its larger screen, wider aspect ratio, and better resolution. However, like most new technologies, HDTV is expensive. Nonetheless, cheaper digital TVs offer a significantly improved viewing experience over traditional TVs, adding a set-top converter for those who choose to keep their old receivers. Will deliver recognizable and improved video and sound.

The Federal Communications Commission's (FCC) schedule for the transition to DTV states that everyone in the United States will have access to DTV by 2002, and that switching to digital transmission will be by 2006, or for households in specific areas. Eighty-five percent of them must be completed when purchasing a digital television receiver or set-top converter.

In the early 1990s, European broadcasters, consumer equipment manufacturers, and coordinating bodies were the European Launching Group: ELG) was formed and the "DVB" or "Digital Video Broadcasting" program was launched to introduce DTV throughout Europe. DVB is intended to provide an open system that opposes a closed system. Closed systems are content provider specific, not extensible, and optimized only for their evolved systems. Open systems such as DVB allow subscribers to select different content providers and integrate multiple PCs and televisions. The intent of the DVB system is to be optimized for television, but also to support home shopping, banking, private network broadcasting (or leased line broadcasting), and interactive viewing. Is. The intent of DVB is to open up the possibility of delivering very clear television programming to bus, car, and train televisions, or even to mobile televisions. DVBs are beneficial and facilitated by content providers because they can provide their own services wherever DVBs are supported, regardless of their geographic location. They can also easily and inexpensively extend their own services, ensuring restricted access to subscribers and reducing revenue losses from unauthorized viewing. Today, the DVB program consists of 220 organizations in more than 29 countries around the world, and DVB broadcasting services are effective in Europe, Africa, Asia, Australia, and parts of the Americas.

Format specific media The various different formats themselves for streaming media signals are also summarized here by non-limiting examples to provide a better understanding of how CODECs can vary for a particular case.

"DVD" is "digital versatile" An acronym for "disc," it is commonly used as a relatively recent optical disc technology that holds up to about 4.7 gigabytes of information on one of its two sides, or enough information for an average of about 133 minutes of cinema. Is defined. Two layers on each of its two sides hold up to 17 gigabytes of video, audio, or other information compared to current CD-ROMs of approximately the same physical size, which hold about 600 megabytes. Get (DVD holds about 28 times more information than CD-ROM). DVD players are needed to play DVDs, but they will also play regular CD-ROM discs. DVDs are three general optimized for (i) video (eg serial movies), (ii) audio (eg long playing music), and (iii) mixed (eg interactive multimedia presentations). It can be recorded in any format. DVD drives have a slightly faster transfer rate than 8-speed CD-ROM players. The DVD format typically uses MPEG-2 files and compression standards, which have about four times the resolution of MPEG-1 images and at about 60 skip fields / second, where two fields make up one image. Can be delivered (MPEG-1 delivers at about 30 non-jump frames / second). The MPEG-2 and -1 standards are defined in more detail elsewhere here. Audio quality on DVD is comparable to current audio compact discs.

"DVD-Video" is the name typically given to a DVD format designed for non-abbreviated movies and is a box that works with a television receiver. "DVD-ROM" is the name given to players who are believed by someone to be a future alternative to CD-ROMs in computers, and newer drives are usually CD-ROM discs along with DVD-OM discs. This is because it is intended to regenerate both. "DVD-RAM" is the name given to the writable version of DVD. "DVD-Audio" is the name typically given to players designed to replace compact discs / players.

"VHS" is an acronym for "Video Home System", commonly defined as a magnetic videotape cartridge format, typically half an inch wide for recording and playing analog video and audio signals. Developed for home use with the ability to do. VHS has become a popular format and has become the de facto standard for home movie distribution and playback, primarily due to its widespread presence and recording performance. VHS uses technology similar to that of audio cassettes to store signals in analog format on magnetic tape. These tapes are played and recorded using a VHS videocassette recorder (VHS VCR). VHS tapes typically store videos for up to about 2 hours, but some VCRs can record them at a slower rate. , You can record up to 6 hours or even 8 hours per tape.

The VHS format outputs a little over 200 horizontal resolutions. This is comparable to a DVD that outputs over 500 horizontal resolutions. Technically and sensuously, VHS is a format that outperforms other formats, including, for example, DVD, S-VHS, Hi-8, and the like. However, VHS continues to be a popular means of watching video, and VHS tapes are still easily found in the country and around the world, from movie rental stores to grocery stores.

"CD" is an acronym for "compact disc" for electronically recording, storing, and / or playing audio, video, text, and other information in digital form. Generally defined as a small, portable, round medium. Initially, CDs were read-only, but newer versions are also recordable (ie, "CD-RW").

"Super audio disc" or "SACD" is a high-definition audio CD format, along with DVD-Audio ("DVD-A"), two competing formats to replace standard audio CDs (most industries). (With a general exception that seems to be Philips and Sony), boosting DVD-A). SACDs like DVD-A provide 5.1-channel Saland sound in addition to 2-channel stereo. Both formats improve the complexity of the sound by increasing the bit rate and sample rate, and can be played on existing CD players, but generally at just the same quality level as that of traditional CDs. SACD is announced to be owned by Sony Direct Stream Using Digital or Direct Stream Digital (DSD) recording, instead of pulse code modulation (PCM) and the filtering used on standard CDs, analog waveforms are converted to 1-bit signals for direct recording. Convert. DSD uses lossless compression and a sampling rate of about 2.8MHz to improve sound complexity and realism. The SACD may include text, graphics, as well as additional information such as video clips.

For the purpose of further understanding, Internet-based communications also have specific protocols for communications that must be adapted by streaming media communications using the Internet "super highway". These protocols are briefly summarized below for the purpose of providing a more detailed understanding, especially with respect to streaming media communications.

For Internet notifications, streaming media signals meet data packets and are generally communicated in digital format. The term "packet" is meant herein to mean a unit of data routed between sources and destinations via the Internet or any other packet-switched network. More specifically, when a file is transmitted, the protocol layer of the communication system (eg, the TCP layer of a system based on TCP / IP) divides the file into large chunks of efficient size for routing. Each of these packets is individually numbered and contains the Internet address of the destination. Individual packets for a given file can travel different routes through the Internet. When they all arrive, they are reassembled into the original file by TCP, for example at the receiving end. The packet switching method is an efficient method for handling transmission on a network without a connection such as the Internet. Alternative methods, such as circuit exchange methods, are used for networks assigned to approximately voice connections. In circuit switching, lines in a network are shared by packet switching among a plurality of users, and each connection generally requires the exclusive use of a specific route during the period of the connection.

Wireless communication and WAP gateway Equally important to the Internet age, the age of wireless communication has significantly expanded its social capabilities to interact outside the fixed territory of the home or office, thereby allowing our telecommunications. It made it possible to be released from the electric cable wire and the cable. For example, in 2000, the number of mobile (or mobile) subscribers grew to nearly 50%.

However, wireless communication systems, protocols, and enabling technologies have evolved globally in a significantly fragmented "format-specific" market. This is especially true when comparing systems for widespread use in the United States when compared to the rest of the world except the United States. Therefore, considerable effort has been extended in overcoming competing issues between multiple format-specific systems and related wireless devices operating on different platforms. For the purpose of further understanding wireless communication as later related to the present invention, the following is a brief overview of the significant technologies, systems, and protocols used in the wireless communication industry.

In general, the development of wireless communication systems for cell telephones is commonly given by the terms "1G", "2G", "2.5G", and "3G", which represent the first generation, the second generation, etc., respectively. There is. Early systems were pure analog, known as 1G telephones and systems. However, with rapid growth, the available bandwidth for mobile phones has been sharply reduced, paving the way for digital signal processing in 2G, which is available for complex signal processing for advanced telecommunications. Bandwidth and capacity have been significantly expanded. However, with the increased demand for wireless Internet access, technology advances have moved from 2G phones (generally not enabled on the Internet) to 2.5G and 3G (progressively more enabled). As further deployments immediately below, systems, protocols, and enabling technologies have thus evolved towards a focused focus on bringing 2.5G and 3G modes to industry and consumers.

In general, there are four major digital wireless networks based on 2G technology: Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), and Global Systems for Mobile Communications (GSM). ), And cellular digital packet data (CDPD). These are briefly described here as follows.

Time Division Multiple Access (TDMA) is a technology used in digital cell phone communications that divides each cell channel into three time slots to increase the amount of data that can be carried. TDMA has been adopted by Digital-American Mobile Phone Services (D-AMPS), Global Systems for Mobile Communications ("GSM"), and Personal Digital Cellular ("PDC"). An alternative multiplexing method for TDMA and FDMA is code division multiple access (CDMA).

Code division multiple access (CDMA) refers to any of the several protocols used in 2G and 3G wireless communications. As the term implies, CDMA allows a number of signals to occupy a single transmission channel, optimizing the use of available bandwidth. This technology is used in ultra high frequency (UHF) mobile telephone systems in the 800MHz to 1.9GHz band. CDMA uses analog-to-digital converters (ADCs) in combination with diffusion spectrum technology. The audio input is first digitized into binary elements. The frequency of the transmitted signal is made to fluctuate according to a defined pattern (code) and its frequency response can only be disturbed by receivers programmed with the same code and tuned exactly to the transmitter frequency. To do. There are trillions of potential frequency sequencing codes that improve privacy and make cloning or copying difficult. The CDMA channel is nominally 1.23 MHz wide. CDMA networks use a technique called "soft handoff" to minimize signal splitting as the phone moves from one cell to the other. The combination of digital mode and diffuse spectrum mode supports several times as a large number of signals per unit bandwidth, as in analog mode. CDMA is compatible with other cellular technologies, which allows for national roaming.

Originally known as CDMA One, CDMA was standardized in 1993 and is considered to be the 2G technology that is still prevalent in mobile (cellular) telephones in the United States. One version of cdmaOne, IS-95A, is a protocol that utilizes the 1.25MHz carrier and operates in either the 800MHz or 1.9GHz RF band, which supports data rates up to 14.4Kbps. Another version IS-95B can support speeds up to 115Kbps by bundling up to 8 channels.

More recent CDMA variants, CDMA2000, as well as wideband CDMA, have provided many faster data rates. CDMA2000, also known as IMT-CDMA Multi-Carrier or IS-136, is a CDMA version of the IMT-2000 standard developed by the International Telecommunication Union (ITU). The CDMA200 standard is 3G technology and is intended to support data communications at speeds ranging from 144Kbps to 2Mbps. Companies that have developed this standard version include Ericsson corporation and Qualcomm corporation. Broadband CDMA or "WCDMA" is an ITU standard derived from CDMA, IMT-2000 direct Also known as spread. WCDMA is a 3G technology that supports data rates up to 2Mbps for local area access and data rates up to 384Kbps for wide area access, and is mobile at these speeds / It is intended to support portable voice, image, data, and video communications. WCDMA digitizes the input signal and transmits its digitized output in coded diffusion spectrum mode over a 5 MHz wide carrier, which is significantly wider than the 200 KHz wide narrowband CDMA.

The Global System for Mobile Communications (GSM) is a digital mobile phone system that is widely used in Europe and elsewhere in the world, and this system is one of the TDMA (introduced shortly below). Using a variant, it is the most widely used of the three digital wireless telephone technologies (TDMA, GSM, and CDMA). GSM digitizes and compresses the data, each in its own time slot, and sends it on one channel along with the other two streams of user data. It operates in either the 900MHz or 1800MHz frequency band. At the time of this disclosure, GSM was generally considered the European wireless phone standard and was announced to be available in 120 countries with more than 120 million users worldwide. At least one company in the United States, American Personal Communications (Sprint Subsidy), uses GSM as a technology for broadband personal communications services ("PCS"). PCS is a telecommunications service that throws voice communications, numerical and text messages, and voice mail into a single device, service contacts, and invoices. PCS is most often communicated via digital cellular links. The service is planned to have more than 400 base stations for various small mobile phones manufactured by manufacturers such as Ericsson corporation, Motora corporation, and Nokia corporation. , Telephones, pagers, and answering machines in general. GSM is a wireless mobile system that includes High Speed Circuit Exchange Data (HCSD), General Packet Radio System (GPRS), Enhanced Data GSM Environment (EDGE), and Universal Mobile Telecommunications Services (UMTS). It is part of the evolution of Telecommunications.

Cellular Digital Packet Data (CDPD) is a wireless standard that provides bidirectional 19.2 Kbps packet data communication across existing cellular phone (or mobile phone) channels.

Several different protocols have been incorporated for use in communications across various wireless networks.

"X.25" is a packet-based protocol that was used in principle at the time of this disclosure in Europe and was applied as a standard by the International Telegraph and Telephone Advisory Committee (CCITT). X.25 is a commonly used network that allows computers on different public networks (eg Compuserve, Symnet, or TCP / IP networks) to communicate at the network layer through intermediate computers. The X.25 protocol closely corresponds to the data-link and physical layer protocols specified by the Open Systems Interconnection (OSI).

"OSI" is a model of network architecture, a set of protocols (protocol stack) to embody it, and in 1978 as a framework for international standards in foreign computer network architecture. Developed by ISO. The OSI architecture is divided into seven layers, from lowest to highest: (1) physical layer, (2) data link layer, (3) network layer, (4) transport layer, (5) session layer, (6). The display layer and (7) adaptive layer. Each layer uses the layer directly below it to provide services to the layers above. In some embodiment, one layer may itself consist of multiple secondary layers.

General Line Radio Service (GPRS) is a packet-based wireless communication service that promises data rates from 56 to 114 Kbps and continuous connectivity to the Internet for mobile phone and computer users. Higher data rates allow users to participate in video conferencing and use mobile handheld devices with laptops to interact (conversate) with multimedia websites. GPRS is based on the Global System for Mobile Communications (GSM) and supplements existing services such as circuit exchange cellular phone connections and short message services (SMS). SMS is a messaging service provided by the GSM Digital Cellular Telephone System. Using SMS, a short alphanumeric message (160 alphanumeric characters) can be sent to and displayed on a mobile phone, much like an alphanumeric calling system. Messages are buffered in the GSM network until the phone is active.

GPRS packet-based services are less costly to users than circuit exchange services, but are used for shared use according to packet needs rather than dedicating a communication channel to just one user at a time. Because. The intent is also to make the application available to mobile users, which is a middleware that has faster data rates but now needs to adapt the application to the slower speeds of wireless systems. This is because haze means that nothing is needed. GPRS is widely enabled, allowing mobile users of virtual private networks or virtual private lines (VPNs) to access their private networks more continuously than through dial-up connections. Further intended in GPRS is to supplement "Bluetooth", a standard that replaces wired connections between devices with wireless wireless connections. In addition to the Internet Protocol (IP), GPRS supports the X.25 protocol. GPRS is also believed to be one stage of development towards enhanced data GSM environments (EDGE) and universal mobile communication systems (UMTS).

UMTS is intended for 3G, broadband, text packet-based transmission, digitized voice, video, and multimedia at data rates up to 2 Mbps. Further intended at UMTS is to provide mobile computer and telephone users with a consistent set of services, regardless of where they are located in the world. The service is based on GSM communications standards and has been approved by major standards bodies and manufacturers, and is a planned standard for mobile users worldwide by 2002. Once UMTS is fully embodied, computer and telephone users will be able to constantly connect to the Internet as they move.

Enhanced Digital GSM Enterprise (EDGE) services are faster versions of global systems for mobile (GSM) wireless services, delivering data at rates up to 384Kbps and multimedia and other broadband applications. It is designed to enable distribution for mobile phone and computer users. The EDGE standard is built on top of the existing GSM standard using the same time division multiple access (TDMA) frame structure and existing cell sequences. EDGE is expected to be commercially available in 2001. This is seen as an evolutionary standard on the way to Universal Mobile Telecommunications Services (UMTS).

Wireless Application Protocol (WAP) includes wireless devices such as cellular phones and wireless transceivers, including email, worldwide web, newsgroups, and Internet Relay Chat (IRC). A specification for a set of communication protocols to standardize the methods available for Internet access. While Internet access was possible prior to WAP, various manufacturers used "format identification" technology.

Recently, considerable effort has been spent merging the fields of wireless communications and the Internet to bridge the cord, wire, and cable gaps that previously separated the "information highway" from reaching people on wireless devices. Was done. Such technology mergers have evolved within, for example, the home and office network settings themselves, where wireless infrared and radio frequency communication systems are being developed to interface devices in "wireless" offices or homes. Another substantive effort is underway to communicate and share information with more remote wireless devices such as cell phones and personal digital assistants (PDAs).

A PDA is typically a small mobile device that can be "handheld" and typically has a limited processor and display for managing, storing, and displaying phone books, calendars, computers, and the like. May include a display screen. Recently available PDAs have a wireless modem embedded within the PDA itself, or are "wireless enabled" by being combined with a wireless modem "plug-in" such as a cell phone. ". Wireless-enabled PDAs are also generally "Internet-enabled" with limited "browser" capabilities, allowing the PDA to communicate with server devices over the Internet. Examples of commercially available wireless "enabled" PDAs are Palm VII (Palm, Inc.) and iPAQ (Compaq, Inc.). Inc.) is included. These PDAs include the Windows® CE operating system, providing screen displays for limited browser capabilities and content. These phones have processing power from about 33MHz to about 220MHz and various screen display powers such as 320x240 pixel screen display.

Similarly, the cell phone itself is "Internet-enabled" and has limited browser capabilities and a screen to display content. Examples of "Internet-type" cell phones include Sanyo SCP-4000 , Motorola i1000plus , among other widespread examples, a wide range of fields claiming hundreds of different processing and display capabilities. There is.

Compatibility with the Internet Protocol of communication must be achieved for either "Internet-enabled" PDAs or cell phones. Generally, wireless communication is done via the Wireless Application Protocol ("WAP"), but communication across the Internet follows one of several different protocols, the most common being the Transmission Control Protocol / Internet. -Protocol ("TCP / IP"). Therefore, as shown in Figure 1E, WAP gateways are basically bridges between the world of the Internet (or other IP packet networks) and wireless telephone / data networks, each with different technologies. Is forming. This gateway essentially makes an interpretation between these two separate entries and allows consumers to use their cell phones or handheld computers (eg PDAs) to access the Internet without wires. It is possible.

However, streaming media formatted for transmission to higher performance computing devices such as desktop computers with outstanding display capabilities are received and viewed on these devices with strictly limited processing and display capabilities. Generally incompatible for. Certain "format-specific" compression schemes have been developed specifically for use with these devices only, and only certain media content may be transmitted to these devices in their format.

<p> A wide variety of streaming media signals in the proper format for playback by wireless devices such as cell phones and PDAs, which have unique constraints such as limited and variable processing, memory, and display capabilities. There is still a demand for transmission-compatible streaming media communication systems.</p>

<p> The present invention addresses and overcomes various limitations, inefficiencies, resource limitations, and incompatibilities with previously known methods for streaming media communication, as described below. Provided in useful modes, aspects, examples, as well as variants.</p><p> The present invention according to one embodiment is a streaming media communication system, which uses a computer-embodied intelligent system such as artificial intelligence in a network system such as a neural network to be used between a transmission device and at least one arrival device. Communicate streaming media signals.</p><p> According to another embodiment, the present invention is a system for communicating a streaming media signal between a transmission device and a plurality of arrival devices, each of which has a different media signal processing capability.</p><p> According to another embodiment, the present invention, from a single transmission device to at least one arrival device, uses streaming media via a plurality of different transmission channels, each of which has different transmission capabilities or restrictions on the streaming media signal. A streaming media communication system suitable for communicating signals.</p><p> According to another embodiment, the present invention compresses a streaming media signal at a source into a compressed representation of the streaming media signal, transmits the compressed representation to an arrival device over a transmission channel, and compresses the streaming media signal. Artificial intelligence adapted to be trained in a conforming learning process with respect to the capabilities of the streaming media compression system to decompress the representation and make it into a decompressed representation of the streaming media signal adapted to be played by the arrival device. It is a neural network that incorporates means.</p><p> According to another embodiment, the present invention is a system that compresses a streaming media signal according to a CODEC based on at least one parameter that affects the communication of the streaming media signal, at least in part. According to one aspect of this mode, the CODEC is pre-learned for the pre-learned behavior of the CODEC for another reference signal, and for the pre-learned attempt for compression and decompression of the same streaming media signal. Behavior, reference algorithm for streaming media signals Comparison of CODEC behavior against streaming media signals against compression, learned constraints on transmission channels, and at least one of the various parameters of learned constraints on the arriving device. Used according to one. In one informative embodiment the CODEC is used based on two or more of these parameters, and in a further informative variant the CODEC is used based on all of those parameters.</p><p> The present invention, according to another embodiment, uses a CODEC library adapted to store a large number of CODECs of various types and behaviors, and to be investigated and accessed by network systems such as neural networks. A system that compresses streaming media signals by providing the appropriate codecs from the CODEC library and using them to compress the input streaming media signals into a compressed representation for transmission to the arriving device.</p><p> The present invention, according to another embodiment, is a CODEC operating system adapted to interface with a neural network together with a CODEC library, and uses the neural network in a process such as an artificial intelligence process to obtain an appropriate CODEC from the CODEC library. And use the selected CODEC to compress the streaming media signal into a compressed representation of the streaming media signal for transmission to the arriving device.</p><p> According to one aspect, the CODEC library is adapted to receive and store new CODECs, and the new CODECs can be interfaced with neural networks to compress and apply the streaming media signals provided. Will be done.</p><p> The present invention, according to another embodiment, is an arrival medium adapted to memory by an arrival device for use in decompressing a compressed representation of a streaming media signal. This arrival medium is adapted to communicate with a remotely located compressed streaming media transmission system, which receives and reproduces streaming media signals. In a particularly useful aspect, the software mediator adapts to deliver information about the arriving device to a compressed streaming media transmission system while receiving certain encoded streaming media signals from that compressed streaming media transmission system. It is also suitable for decoding.</p><p> According to another embodiment, the present invention decompresses a compressed representation of a streaming media signal into a decompressed representation that can be reproduced by the arriving device. It is a system that communicates signals.</p><p> According to one aspect of this embodiment, the arrival medium has a diagnostic medium and a thaw medium. Diagnostic mediators are adapted to determine values for at least one parameter of the arrival device associated with processing, memory, or regenerative capacity. The decompression medium is adapted to apply a CODEC decompressor that decompresses the compressed representation of the streaming media signal to the decompressed representation, using the CODEC at least in part based on the value of at least one of its parameters.</p><p> According to another aspect, the arrival medium includes a software medium. In one variant, the software medium is embedded within the arrival device. In another variant, the software mediator is adapted to load a compressed representation of the streaming media signal into the arriving device, at least in part, by a remotely located source that is adapted to deliver the compressed representation to the arriving device.</p><p> According to another embodiment, the present invention is a transcoder for transcoding a streaming media signal between at least one initial format and at least one truss coded format.</p><p> According to another embodiment, the present invention is a video-on-demand streaming media system that incorporates the embodiments shown or otherwise described in the accompanying drawings.</p><p> The present invention, according to another embodiment, is a mobile telephone communication system that incorporates the embodiments set forth herein or otherwise shown in the accompanying drawings.</p><p> The present invention, according to another embodiment, is an interactive gaming system that incorporates the embodiments set forth herein or otherwise set forth in the accompanying drawings.</p><p> The present invention according to another embodiment processes (eg, compression) various modes, examples, aspects, features, variations, etc. disclosed above or elsewhere in this specification. ) Later incorporated into static media along with locally stored and untransmitted media.</p>

The present invention, variously illustrated through the following examples (and by reference to the drawings), provides a media communication system, including a compression system, a delivery system or a distribution system, and a decompression system. And in another aspect, it involves a transcoder system. In general, these individual subsystem combinations, in addition to customizing the compression, delivery, and decompression of randomly selected streaming media signals based on a larger array of system parameters as variables. Provides the ability to efficiently transcode media between multiple encoding formats. These variables include, for example, without limitation, parameters related to the source video signal, the source transmission device, the transmission mode, and the arrival device. Media signal compression, distribution, and decompression are thus customized to be optimally efficient for a given and changing usage environment. As a result, a wide range of complex streaming media signals can be communicated at an efficient level and with significantly improved device compatibility over other known systems.

Despite the benefits of all the streaming media communication systems described herein, each subsystem described above also individually provides useful and useful results for streaming media communications. It is also intended that the various iterations of the various subsystems themselves and the combinations of these subsystems, which are at least in part apparent to those of skill in the art based on this disclosure, are within the scope of the present invention. In addition, various aspects of the overall communication system, along with each of the subsystems described, are also intended to be useful, especially for applications other than those for streaming media communications. Therefore, as will be apparent to those of skill in the art, such additional applications are further intended to be within the scope of the invention, despite the particularly useful modes applied to improved streaming media communications. Has been done.

<u style="single">Transcoder</u> The video / audio transcoder 200 is provided in accordance with the present invention, where one incoming video source 210 can be used in numerous formats 215 (eg MPEG4, Real Video , as well as QuickTime ) without human intervention. )) Enables flow from one device across. The Transcoder 200 according to this embodiment offers substantially greater functionality at a fraction of the price of other commercially available transcoder systems. Moreover, since the system runs "without rest", the pre-compression of the video source 210 is significantly reduced.

More specifically, the transcoder 200 system and method according to the present invention allows the digitized media resulting from any compressed or uncompressed format to be in any other compressed format, eg, real-time on request. ) To be transcoded. The system 200 and method can also enable efficient and simultaneous processing of multiple streams 215 of different data from many different compressed or uncompressed formats to many different compressed formats.

The transcoder 200 of the present invention is illustrated by reference to FIG. 3 and is described herein in an overall system. As shown, the first player first makes a connection with the server 300 that houses the transcoder 200. The player format (eg Microsoft Media ), connection speed (eg 32Kbps), and protocol (HTTP) are identified. Server 300 pulls raw or pre-encoded video into a "live buffer" or "cache" 310 and encodes it as largely uncompressed but digitized data (eg AVI or MPEG2). ). Server 300 then loads the appropriate CODEC thread (eg Microsoft Media ) at a connection speed (eg 32Kbps). Server 300 then loads an HTTP / MS player that serves the first client. So the second stream is requested by the client with MMS using a 100Kbps M / S player. The server is the right MS at the right 100Kbps rate Load the CODEC thread. Server 300 then loads an MMS / MS player thread that serves the second client. So the third stream is requested by the client with RTSP using a 40Kbps realplayer. Server 300 loads the appropriate Real CODEC at the appropriate 40Kbps rate. Server 300 then loads the RTSP / Real player thread that serves the third client. Again, this illustration is exemplary and other specific CODECs appropriately substitute with other bit rates.

To provide a better understanding of this transcoder embodiment, FIG. 3 shows the transcoder 200 with a further example applied to carry a number of different video streams to different clients.

In short, this illustrated and described transcoder 200 is an "IPC" or "Inter Processor" that follows a number of traditional transcoding techniques. Use "threaded" communication instead of "Communications". For the purposes described in this Transcoder 200, the term "thread" is intended herein to mean the inclusion of a control flow within a program. Single-threaded programs only execute one path "temporarily" through their code. A multithreaded program can have several threads running "simultaneously" through different code paths. In a typical process with a large number of threads, zero or more threads can actually run at any time. This depends on the number of CPUs in the computer on which the process is running, as well as how the thread system is embodied. While a machine or system with n CPUs can be adapted to run less than n threads in parallel, the threading behavior according to this thread invention is n by sharing the CPU among multiple threads. It can give the appearance of running less than a few threads.

The Transcoder 200 provides an abstract API and therefore the CODEC is accessed without the (significantly larger) original encoder overhead. Buffering 310 is created as a client traction function for different video streams. Furthermore, the transcoder 200 of the present invention utilizes a network architecture, that is, a single thread for each different connection, and if the client is in a buffered segment of the same content, it will be combined into the same thread. The use of the Thread Transcoder 200 as shown and described here is considered to be very beneficial because the context switch between two threads in a single process is the context switch between the two processes. This is because it seems to be considerably cheaper (processing / memory / IO) than using a switch. In addition, the fact that all data except stacks and registers is shared among multiple threads can be broken down into multiple subtasks that can be run collaboratively to the natural institutions that perform multiple tasks. Make a thread.

While various special architectures can be constructed around the Transcoder 200 embodiments described above to achieve specifically desired results on a case-by-case basis. However, for further illustration purposes, the following example is a more detailed system using the transcoder 200 described above. The Transcoder 200 is adapted and provided to support a large number of simultaneous customer streams, each in a different format. In particular, such systems can support more than 5000 simultaneous streams, and in some situations can support more than 7000 simultaneous customer streams, each with a different video format. Furthermore, the Transcoder 200 is uniquely suitable or required for any of a wide range of video sources for a large number of different individual clients, each with different needs. It can be embodied to be converted into the format described. In one particular example, the transcoder 200 described herein is MPEG1, MPEG2, MPEG4, Motion JPEG, AVI, H.261, H.263, H.263 +, RealVideo , G-8, QuickTime. It can be embodied to simultaneously support such high demands for , Shockwave Flash , Indeo Cinepak , and any of the various formats of ASF.

Further intended, the transcoder 200 may be adapted to be obedient to all existing, directly foreseen and fixed mobile terminals and devices in the overall communication system. Furthermore, the transcoder 200 can be embodied to adapt to the output stream format variables and dynamically adapt to the state of each client's channel and platform. In addition, systems incorporating transcoders are adapted to support load balancing services and routers for multi-transcoder installations. Therefore, the transcoder 200 of the present invention delivers significantly greater functionality due to a significantly lower cost than other preceding transcoding techniques and systems.

As mentioned above, a variety of different system architectures can incorporate the Transcoder 200 of the present invention without departing from the scope of the present invention. However, more details of the special architecture that appear to adequately provide the beneficial levels of support described above are (i) dual P3-933 processors, (ii) any variant of the Unix® OS. , (Iii) 512MB RAM, Redundant Firewire or Gigabit Ethernet®, as well as Redundant Power Supplies aspects. Such a system may be provided in a rack-mounted cell phone or may be provided according to a particular requirement.

The following various aspects of the transcoder 200 according to the present invention will be widely beneficial, at least in part, either alone or in various combinations, as will be apparent to those skilled in the art based on this disclosure.

To take advantage of asynchronous software thread communication in both the user and kernel space to perform efficient transcoding on multiprocessors and / or distributed computing platforms (such as clustering). Systems and methods are provided. This method is expected to be more efficient than using the traditional IPC method to embody the transcoder. A shared library of CODEC algorithms was created and used to access various CODEC algorithms, thereby providing lower processing overhead and traditional combination encoder features such as those used in the majority of commercial encoders. Inviting with lower memory usage than required by. Of particular benefit is that common threads can be used for multiple connections, and in practice even a single thread can be used for each individual connection using the transcoder of the present invention.

Systems and methods that combine as many clients as possible (for efficiency) to be served by the same thread, no matter when the same content is requested and when the dynamic buffer (cache) can fit all of the requested data points. Is also provided.

Media compression and distribution system<u style="single">The present invention also provides a data compression and distribution system 400 for real-time dynamic data signal processing that optimally reproduces an approximation of the original media data under a given set of constraints, and methods thereof.</u>The system 400 and method are schematically illustrated in block flow diagrams in FIGS. 4A and 5.<u style="single">Below, other descriptions of the various beneficial features and behaviors of this system are presented by examples that largely incorporate the descriptions shown in FIGS. 4A-5.</u>

FIG. 4A is a block diagram of an embodiment of the data compression and distribution system 400 according to the present invention. As shown in Figure 4A, the data compression and distribution system 400 includes media module 405, dynamic player module 407, image processor 410, baseline snapshot module 415, classifier 417, standard quality (QoS) module 420. , Network layer input module 425, and network layer output module 430. The system includes a neural network processing module 440, a timer 435, a CODEC library module 445, a dynamic client request module 450, an ICMP module 455, an instrument and network parameter measurement module 460, and a delivery and transmission module 465. Further included.

In one embodiment, system 400 resides on a server node (s) and inputs.<u style="single">Be done</u>Uncompressed<u style="single">of</u>Alternatively, it processes the previously compressed data. System 400<u style="single">、</u>artificial intelligence<u style="single">To</u>Input using the accompanying neural network 440<u style="single">Be done</u>Monitor the data to determine multiple key characters for each data segment.<u style="single">System 400 includes a library of pre-developed, self-referenced, empirically learned pattern rules for scenes within a series of frames of input signals (eg, video signals), as well as externally imposed constraints. , And select a suitable commercially available compression / decompression algorithm (ie, CODEC) for each data segment, associating with the characteristics of the input data.</u>Then system 400<u style="single">、</u>Choice<u style="single">Shi</u>To optimize the algorithm<u style="single">、</u>Assemble a long array of usage controls, parameters, and variables. Algorithm selection and parameter and variable setup<u style="single">Developmental</u>Optimization process<u style="single">It</u>With itself<u style="single">Similarly</u>Depending on the characteristics of the data, on each segment of the input data<u style="single">Therefore</u>It fluctuates dynamically.<u style="single">Can take</u>There are many sets of algorithms, limited only by their effectiveness and other commercial considerations. Each segment of data is encoded and compressed as described above before being served on the communication channel.<u style="single">Re</u>To.

<u style="single">here</u>Description<u style="single">To do</u>Compression system 400<u style="single">、</u>Especially useful as a streaming media compression engine, based on information from valid CODECs and streaming media distribution systems, using another artificial intelligence neural network 440, input video<u style="single">To</u>For each frame<u style="single">Analysis</u>To do. System 400 then selects the most appropriate compression format and<u style="single">For example</u>In one embodiment<u style="single">Is the lower layer</u>Peak signal / noise ratio from the system environment<u style="single">Extraction</u>Based on the best quality as measured by<u style="single">、</u>Build or configure compression parameters for optimal video compression. The result is<u style="single">For this device and the current state</u>"Optimal"<u style="single">Na</u>Video and audio<u style="single">Become a service</u>。

Applies to streaming media signals<u style="single">Ru,</u>A clearer description of the aspects of the artificial intelligence / neural network 440 of this system<u style="single">,</u>less than<u style="single">State</u>.. First, individual<u style="single">different</u>A library consisting of CODECs<u style="single">, Search</u>Possible<u style="single">Na</u>As CODEC library 445<u style="single">this</u>Added to the system. Relation<u style="single">Reference to</u>A library of information<u style="single">add to</u>Being done<u style="single">is this,</u>Includes Network Transport Standard (NTS) Library 443 and Quality of Service (QoS) Library 447. Then<u style="single">、</u>Video (media source)<u style="single">、</u>Via image processor 410<u style="single">、</u>Digital or non-digital format (AD converter)<u style="single">Using</u>)<u style="single">Will be taken in</u>.. Image processor 410<u style="single">(If necessary)</u>Unzip the source and<u style="single">(Single or plural)</u>Various standards used to "purify" source images<u style="single">of</u>Use an image processing algorithm. The resulting source media is a baseline snapshot<u style="single">Storage location</u>Passed to 415, where it will be used as the "perfect gold standard" for later comparison. At the same time this result<u style="single">can get</u>Source media is also supplied to classifier 417.

Classifier 417 analyzes the source media for temporary, spatial, and logical features, as well as for temporary, spatial, and logical features.<u style="single">of</u>Create a source media subsegment that shows the combination<u style="single">For</u>Is. This "similar"<u style="single">When</u>Is<u style="single">Means a continuous, continuous subsegment of source media that contains common temporary, spatial, and logical features.</u>They themselves<u style="single">(As found in CODEC library 445)</u>It will be lent to a specific encoding / compression algorithm. This source media subsegment (or, in one embodiment, a group of continuous video and audio frames)<u style="single">To</u>With "scene"<u style="single">Name</u>。

The neural network process 440 then operates on this scene by utilizing the CODEC from the CODEC library 445 to compress the scene. The internal configuration of each CODEC is manipulated / modified according to the inputs obtained from NTS library 443, Qos library 447, timer process 435, network input layer 425, ICMP mediator 455, and equipment and network parameter measurement mediator 460. To.<u style="single">Then</u>The compressed scene is decompressed<u style="single">hand</u>, Using quality measurement by quality standard process 420<u style="single">Ta,</u>Against baseline snapshot 415<u style="single">To do</u>A comparison is made. In one embodiment of the invention, the quality standard process 420<u style="single">、</u>Using the peak signal-to-noise ratio (PSNR) algorithm<u style="single">hand</u>, Source media for unzipped scenes<u style="single">To</u>For baseline snapshots<u style="single">for</u>Compare. This comparison process<u style="single">For various CODECs from the CODEC library 445</u>From NTS library 443, QoS library 447, timer process 435, network input layer 425, ICMP mediator 455, and instrument and network parameter measurement mediator 460.<u style="single">Receive</u>Input constraints<u style="single">Within range</u>And the neural network process 440 is the result<u style="single">can get</u>The quality of the compressed scene<u style="single">Satisfied</u>Until<u style="single">Repeated</u>To. Finally as a result<u style="single">can get</u>The compressed scene is transmitted to network layer output 430, which transports the compressed scene to the client using the appropriate network transport protocol and QoS algorithm.

The previous process is until all source media has been transmitted to the client, or the client.<u style="single">From</u>Cancellation request, network transportation<u style="single">of</u>Failure, client hardware<u style="single">of</u>Various things that can include failures, etc.<u style="single">Conceivable</u>It repeats until the condition aborts this process.

NTS library 443<u style="single">It is a storage place for network transportation services, and the network transportation services are</u>Transport compressed source media to clients<u style="single">Ru</u>Selected by network layer output 430<u style="single">,further</u>, Receive information from the client<u style="single">Ru</u>Selected by network layer input 425. This selection is received from network layer input 425, ICMP mediator 445, and equipment and network parameter measurement mediator 460.<u style="single">Su</u>Qualitative<u style="single">And quantitative</u>Based on input.

QoS library 447<u style="single">A storage location for service algorithms, which are</u>Transport compressed source media to clients<u style="single">Ru</u>Selected by network layer output 430. This choice is<u style="single">、</u>Received from network layer input 425, ICMP mediator 445, and instrument and network parameter measurement mediator 460<u style="single">Su</u>Qualitative<u style="single">And quantitative</u>Based on input.

ICMP mediator 455<u style="single">、</u>Input to neural network process 440<u style="single">Generate</u>And to the neural network process 440<u style="single">、</u>Processor<u style="single">When</u>client<u style="single">When</u>Used between<u style="single">Su</u>Of transportation<u style="single">Quantitative</u>And qualitative characteristics dynamically<u style="single">Offer</u>To do. In one embodiment of the invention, for this purpose<u style="single">ICMP protocol</u>use<u style="single">Su</u>To.

Device and network parameter measurement mediator 460<u style="single">、</u>Input to neural network process 440<u style="single">Generate</u>And to the neural network process 440<u style="single">、</u>Of the client environment<u style="single">Quantitative</u>And qualitative characteristics dynamically<u style="single">Offer</u>To do. In one embodiment of the invention, these client environment characteristics are central processing unit (CPU) capacity, network interface characteristics, storage capacity, and media interpreter.<u style="single">function</u>including.

Also Figure 4A<u style="single">Shown in</u>So the network layer input 425<u style="single">、</u>Up (derived from client)<u style="single">To do.</u>) Provide network transportation services. Network layer output 430 is downlink (derived from processor<u style="single">To do.</u>) Provide network transportation services. Timer process 435<u style="single">、</u>The present invention<u style="single">of</u>A user<u style="single">Against</u>, Maximum time a neural network process 440 will spend processing a given source media<u style="single">value</u>To limit<u style="single">Provide a method</u>。

FIG. 4B is a block diagram of the CODEC selection method of the neural network processing module 440 according to the embodiment of the present invention. The neural network processing module 440 shown in FIG. 4B includes a video frame selection module 475, a CODEC parameter module 480, an input layer module 485, a hidden layer 486-487, and an output module 488. In one embodiment of the present invention, the neural network processing module 440<u style="single">To</u>Enter<u style="single">Power</u>Used as a reference baseline for signals<u style="single">To do</u>To<u style="single">Suitable</u>CODEC display signal<u style="single">,</u>This neural network processing module 440<u style="single">But</u>Generate<u style="single">Su</u>To. In one embodiment, the classifier 417<u style="single">, Input from the viewpoint of valid parameters of the lower layer CODEC</u>Which scene in the segment of the video signal represents the best scene<u style="single">Judgment</u>To do. Neural network processing module 440<u style="single">Is a list of multiple standards (standard list)</u>use<u style="single">Shi</u>Which scene in the signal represents the best scene<u style="single">Judgment</u>To do. In one embodiment, the neural network process 440<u style="single">、</u>Video identification<u style="single">of</u>Sampling a large number of pixels in a frame for predetermined parameters in the video signal<u style="single">、</u>That identification<u style="single">of</u>Number of pixels in frame<u style="single">of</u>Change<u style="single">Judgment</u>To do. In another embodiment, identification in a video signal<u style="single">of</u>Notable changes in behavior in the scene<u style="single">、</u>Continue to enter<u style="single">Be empowered</u>For video<u style="single">、</u>Can be used as a baseline reference scene (best scene).

In one embodiment of the invention, the neural network processing module 440 takes a segment of video from classifier 417 as input and then samples this input to extract sufficient information to characterize the video signal. For example, illustrated in Figure 4B<u style="single">Su</u>In this method, the neural network process 440<u style="single">, Inspection</u>Take a window snapshot (for example, a 176x144 pixel window) to do so. Generate enough information about video signals for neural networks<u style="single">But look at the center of the sample window</u>thing<u style="single">Is beneficial</u>Is. In one embodiment of the invention, the neural network process 440<u style="single">、</u>Minimal<u style="single">of</u>With 8 frames<u style="single">、</u>Regarding video signals<u style="single">To do</u>necessary<u style="single">Na</u>Generate information. Information from the sample window<u style="single">、</u>From parameter module 480 to input layer 485<u style="single">、</u>specific<u style="single">of</u>Passed with CODEC parameters.

Input layer 485<u style="single">、</u>It is connected to multiple hidden layers 486-487 via multiple neurons, each connection from one neuron to the other.<u style="single">、</u>It forms a strong or weak link. In one embodiment, each CODEC supported by the neural network processing module 440<u style="single">、</u>As such<u style="single">To</u>Equipped with a neural network, specific CODEC<u style="single">Entered with</u>CODEC<u style="single">To</u>specific<u style="single">To do</u>Handle parameters. Neural network process 440 is called "baked"<u style="single">Like a round robin</u>Video sampling through the process<u style="single">of</u>From multiple CODECs processed during the capture period<u style="single">、</u>"best"<u style="single">of</u>Generate a video signal. Enter<u style="single">Power</u>Best from the signal<u style="single">Na</u>For each CODEC in video display processing<u style="single">To do</u>Each of the corresponding neural networks<u style="single">、</u>Best from hidden layers 486-487<u style="single">of</u>A display sample is generated and the signal is supplied to the output module 488. In one embodiment of the invention, the best from each class of CODEC processed by neural network process 440.<u style="single">of</u>The CODEC output data set is<u style="single">、</u>It has two possibilities.<u style="single">The first is a neural network process 440 that gives the best results for each CODEC to module 488, which outputs the "best" sample to the "burned" neural network for each of the multiple CODECs. Process 440 then produces the best and best CODECs out of multiple best CODECs.</u>Burn-up neural networks are smaller than neural networks that handle CODEC processing<u style="single">、</u>and<u style="single">Is fast</u>。

In the second processing method, the neural network 440<u style="single">、</u>Best generated by multiple CODECs<u style="single">of</u>CODEC genetic algorithm processing<u style="single">May have</u>.. The genetic algorithm is<u style="single">、</u>Marble game<u style="single">When</u>Follow the same statistical selection approach. Therefore, from various neural networks<u style="single">, The best output CODEC</u>"Baked" neural network<u style="single">To</u>Instead of supplying<u style="single">Feeding the output module 488 from various neural networks into a bucket, and also performing genetic algorithm processing to select the best CODEC display from a collection of scenes in end source media such as movies. Can be applied.</u>In one embodiment of the invention, the neural network process 440<u style="single">, To process the CODEC</u>Using a combination of algorithms that propagate back and forth<u style="single">Ru</u>。

Return to Figure 4A<u style="single">Ri</u>, One application for a better understanding of this artificial intelligence process<u style="single">To below</u>Offer<u style="single">Su</u>To. For you to understand, this<u style="single">An exemplary</u>The system features and behavior provided by the application examples<u style="single">、</u>In the present invention<u style="single">Take</u>Data compression and distribution<u style="single">For</u>Neural network 440<u style="single">Of the range</u>It can be considered as a broad description. Other application examples<u style="single">Also</u>Can be done<u style="single">、</u>It can fall within the scope of the present invention.

Video content providers<u style="single">、</u>Install the system of the invention on that server. Sample video on the system<u style="single">Captured</u>And mentioned earlier<u style="single">Shi</u>Ta<u style="single">the first</u>Run the AI process.<u style="single">To be taken out later</u>For example, each bit rate, video pattern, etc.<u style="single">like</u>CODEC<u style="single">Characteristic</u>Complex matrix<u style="single">But</u>Be created. Next, the client end user<u style="single">、</u>Connect with your content provider and watch Video M. The communication system of the present invention resident on the server<u style="single">、</u>software<u style="single">Agent</u>To the client's device,<u style="single">Therefore, in order for the client to distribute the device specific information.</u>Connect with that communication system<u style="single">And also receive the appropriate compressed signal with the decompressed CODEC for playback</u>Enable<u style="single">To.</u>Next, the AI system<u style="single">, To select the appropriate CODEC for each frame and to properly compress each frame for transmission,</u>Start loading the video M into the buffer as a streaming signal<u style="single">To.</u>The duration of the buffer depends on a number of variables, but mainly on the processing power of the system, pre-recorded but uncompressed.<u style="single">of</u>For systems with appropriate capabilities for video media<u style="single">、</u>Generally it can be approximately 15 minutes. In the buffer, each frame<u style="single">、</u>Matrix shown on the drawing<u style="single">In</u>According to the "type" of the pretested series<u style="single">、</u>Compared to each CODEC.

The system 400 then sets end-user parameters such as screen resolution, available memory, etc.<u style="single">、</u>Software on the client's device<u style="single">Agent</u>Received from<u style="single">Su</u>View through information.<u style="single">Therefore,</u>The most appropriate CODEC is selected and a specific variable within that CODEC is set to a fixed quantity (eg, a comparison of the source video's past patterns, transfer channel capabilities or constraints, and arrival device capabilities or constraints. By)<u style="single">、</u>Optimal<u style="single">Na</u>Configured / adjusted for performance. The process just described is generally done frame by frame by classifier 417, but the CODEC<u style="single">、</u>Compare with temporary compression efficiency<u style="single">Shi</u>hand,<u style="single">Considering other leading and delayed frames</u>, Process for each frame<u style="single">I do</u>.. Once the appropriate CODEC has been selected and tuned for each frame (or for blocks consisting of multiple frames if determined automatically and appropriately by the system), the delivery system is the client.<u style="single">Agent</u>Report to, unzip and play the adjusted CODEC<u style="single">Su</u>Deliver to the corresponding frame (or multiple frames).

As you can see, the neural network 440 of this system 400<u style="single">、</u>Continuously learn the performance and operation of CODEC in CODEC library 445<u style="single">Memories</u>And that learning<u style="single">Contents</u>Is continuously used to improve the compression efficiency of the input media signal.<u style="single">The process of running signal frames through the library, changing CODEC operating parameters, comparing reference standard compression to compression performance with comparison logic 525 (Figure 5), and restarting the loop with further changes is generally a process. Continue iteration 550 (Figure 5) to improve compression efficiency.</u>In fact, one in library 445 or<u style="single">Multiple</u>CODEC compression<u style="single">、</u>Reached a better level of improvement than the reference compression algorithm (s)<u style="single">can do</u>。

Nevertheless, time<u style="single">of</u>If constraint 435 (Figure 4A) is present (<u style="single">For streaming media content</u>Real-time push-pull request, etc.),<u style="single">So that a particular frame or sequence of frames being processed can be compressed (575) and delivered (580) to their destination without the unacceptable delay of timer 435.</u>This process must someday stop at some point<u style="single">Absent.</u>Then the next frame or a series of frames<u style="single">,</u>Operated by neural network 440 in CODEC operating system<u style="single">can do</u>.. these<u style="single">of</u>The final destination is<u style="single">、</u>Predetermined desire<u style="single">of</u>result<u style="single">Reaching</u>Specified by<u style="single">Can be</u>For example<u style="single">Not limited to this</u>, (I)<u style="single">For example</u>Standard standard<u style="single">etc</u>A given percentage compared to<u style="single">of</u>Compression efficiency<u style="single">To</u>A predetermined or imposed time limit set on the process according to the time associated with the arrival, (ii) eg buffer time (eg 15 seconds).<u style="single">To</u>Reach or<u style="single">, (Iii)</u>It can be defined by the earlier occurrence of (i) or (ii). Either way<u style="single">hand</u>Also select the appropriate CODEC to compress 575 and deliver 580<u style="single">operation</u>Reach the final destination to perform, but this is<u style="single">、</u>Neural network 440<u style="single">Continues</u>Table of final destinations for training<u style="single">Not something</u>.. The information gathered through each loop in the process<u style="single">、</u>550<u style="single">Remember</u>Will be done. Similar to continue<u style="single">of</u>Frame or on<u style="single">Power</u>The system constraint parameter in the frame<u style="single">, Later encountered 545</u>If<u style="single">To</u>、<u style="single">Neural network 440</u>Stored information<u style="single">To</u>memories<u style="single">Shi</u>re-search<u style="single">Shi</u>Improves compression 575 and delivery 580 efficiency.

While many different communication protocols are intended, one example that seems to be beneficial is<u style="single">, Enables two-way communication between server and client devices</u>Using the "full duplex network stack" protocol<u style="single">Ru</u>。<u style="single">Again in this case</u>, Other protocols<u style="single">This</u>While suitable for a particular application, this full duplex system is more preferred.

<u style="single">here</u>The described system 400 utilizes a streaming media delivery architecture to overcome latency issues and by utilizing an embedded neural network 440 to overcome speed issues.<u style="single">well-known</u>Encountered in CODEC system<u style="single">Su</u>Deal with the difficulties. Then system 400<u style="single">、</u>The algorithm used for compression within the neural network 440 can be reconstructed,<u style="single">Purpose</u>Is<u style="single">、</u>Optimal at any time across any network configuration<u style="single">Na</u>To achieve the result.

A wide variety of CODECs can be used within the CODEC library 445 according to the overall compression system and method described above, but any beneficial use of any particular CODEC according to the present invention may be used alone or elsewhere. Intended for such CODECs in combination with CODECs. For example, a suitable CODEC library 445 can contain one or more of the following types of CODECs: (i) block CODECs (eg, Micorosoft Medeia or QuickTime ). Etc.), (ii) fractal CODEC, and (iii) wavelet CODEC (eg, Real ), etc., which may include one or more. According to another aspect, the appropriate CODEC library 445 may contain one or more of the following types of CODECs, i.e. (i) motion forecast CODECs and (ii) stationary CODECs. Etc. may include one or more of them. Furthermore, the CODEC library 445 may include one or more of (i) lossy CODECs, (ii) lossless CODECs, and the like.

In one embodiment of the invention, all of these different types of CODECs may be represented by the CODEC library 445 according to the invention, and two or more specific CODECs of a given type may be included in that library. Otherwise, different combinations of those different types can be provided to achieve the desired capabilities and stream media over a wide range of real-time variables within the signal itself, transmission channel constraints, or arrival device constraints. Optimize communication compression. Furthermore, an additional highly beneficial aspect of the invention allows the new CODEC to be loaded into library 445 and immediately enabled for use in the neural network 440 compression / delivery system 400. .. Nonetheless, 1 of the CODEC library 445, which appears to be beneficial for its use in optimally communicating a wide range of predicted streaming media signals and especially for image signals. One particular example includes the following specific CODECs, i.e., MPEG versions 1, 2, 4 (eg, Micorosoft). Medeia and QuickTime ), DUCK TruMotion , ON2, Real Media , MJPEG: H.261, H.263, H.263 +, GIF, JPEG, JPEG2000, BMP, WBMP, Including DIVX.

Below, at least in part, as will be apparent to those skilled in the art based on this disclosure, various aspects of the compression systems and methods described above that should be considered widely beneficial, both alone and in combination. Is a further example of. Further examples of such a wide range of aspects are provided in the "Summary of Invention" and "Claims".

Accompanied by artificial intelligence to achieve the various CODEC actions described<u style="single">U</u>Neural network 440<u style="single">To</u>use<u style="single">To do</u>Is broadly and uniquely beneficial. In particular<u style="single">、</u>Signal quality, data content, and data format<u style="single">Of the learned response to</u>Determined by application<u style="single">、</u>Systems and methods for preprocessing 410 source data are provided. Select and apply a suitable CODEC (from a set of multiple available CODECs in the CODEC library 445), depending on the observed properties of the source data and the previously learned response to similar compressed data. By doing so, a system and method for processing each unit of source data (eg, a frame or a block consisting of multiple frames) is provided. Systems and methods for processing each unit of source data are provided by setting a number of compression characteristics within the compression algorithm chosen to optimize the integrity capture and retention of the original data. There is. Furthermore, the signal processing described above<u style="single">Step</u>Each or all of are applied to each of the unique and sequential units of signal data, such as signal clips, video frames, or, where appropriate, individual packets.

It is further intended that the CODEC management system 400 according to the present invention normalizes the original source data / image and resizes the original data to meet the specifications of the neural network processing module 440. Is to provide a system and method for image processing capable of classifying and sampling. The ability to provide any transmission or recording channel on a single system or any source data stream is also provided. Furthermore, the various systems and methods described herein are individually and beneficially in combination and include, but are not limited to, TCP, UDP, WTP / WDP, HTTP, etc. It is compatible with the connection protocol or non-connection protocol of.

Illustrated here<u style="single">Shi</u>The described invention allows for a very useful use of accelerating the learning speed of a neural network 440, while embodying that network.<u style="single">、</u>Minimizes data storage requirements. Different classifications of data streams<u style="single">Has unique characteristics</u>、<u style="single">They are,</u>It requires substantially larger processing by the neural network 440. Video data streams, for example, depend on the predominance and degree of movement, color contrast, and fine pattern and visibility. Larger processing<u style="single">、</u>Optimal<u style="single">Na</u>function<u style="single">sex</u>To reach<u style="single">To</u>Needs longer time. Also, larger processing is larger<u style="single">Measurement</u>Library<u style="single">of</u>Needs memory and often<u style="single">、</u>It grows to an unlimited size. For real-time neural network processing<u style="single">To</u>、<u style="single">By providing a pre-developed prediction library of data stream classification characteristics to greatly increase functionality.</u>Processing time and memory<u style="single">To</u>Minimize<u style="single">can do</u>。

Therefore,<u style="single">To</u>, Alone or in combination (including combinations with other examples illustrated and described elsewhere).<u style="single">、</u>Prior to the present invention<u style="single">To</u>Training<u style="single">Shi</u>Example of neural network 440 aspects<u style="single">Show</u>.. Creating and using artificial intelligence in neural network 440<u style="single">In addition, it provides systems and methods to pre-train the intelligent networks used to solve the problem, and the problem is:</u>necessarily<u style="single">to this</u>Not limited<u style="single">、</u>Certain beneficial of the present invention<u style="single">Aspect</u>To<u style="single">Take</u>It can be streaming media compression. Also<u style="single">、</u>Solution<u style="single">Su</u>Area of the problem to be<u style="single">Systems and methods are provided that can be classified into useful categories and processed according to the learning history of the intelligent network.</u>

<u style="single">The present invention</u>Intelligent streaming / media distribution system and method<u style="single">And</u>,end user<u style="single">of</u>Based on ability, content transmission, for example<u style="single">Not limited to this</u>Manage transmission speed or bandwidth, as well as transmission channel constraints such as Internet congestion.<u style="single">Also provides systems and methods for</u>.. Data compression and distribution system 400 is connected (<u style="single">Not limited to this,</u>(Including different bit rates, latencies, transmission characteristics, and equipment restrictions)<u style="single">Aspect</u>Analyze and change the compression method<u style="single">Do</u>With<u style="single">、</u>Quality of service ("Qos") 420 to manage issues<u style="single">、</u>Artificial intelligence processes based on neural networks, etc.<u style="single">like,</u>Computer<u style="single">Run on</u>Take advantage of intelligent processes. compression<u style="single">Was done</u>digital<u style="single">Of, restore</u>Possible<u style="single">Na</u>And / or defrost<u style="single">Possible</u>The data stream is<u style="single">Therefore</u>For many different local and / or remote devices<u style="single">、</u>different<u style="single">function</u>Characterized by<u style="single">、</u>It is distributed via a large number of transmission media. In addition, decompression system<u style="single">With</u>Play the decompressed data on the terminal device.

In one informative embodiment, the terminal device<u style="single">、</u>Set up a link to the system that resides on the server node. Except for the software normally required to set up communication, the terminal device is initially inside it.<u style="single">, According to the present invention</u>May not have resident software. In linking the terminal device with the server node, the system transmits a software medium to the terminal device, which works with other server-side software modules to work together with the overall distribution system To form. software<u style="single">Agent</u>Is<u style="single">、</u>To that system<u style="single">、</u>Notify the terminal device configuration and the processing capacity for decompressing and displaying data. That software<u style="single">Agent</u>Is<u style="single">、</u>Give the system certain relevant information about the characteristics of the communication channels between the terminal device and the server.<u style="single">Also</u>Report. Such information is<u style="single">Not limited to this,</u>Includes latency, bandwidth, and signal path integrity. Based on the terminal equipment configuration and real-time updates of channel characteristics and capabilities, the system can change parameters such as buffer length, transmitted bit rate, and error correction.<u style="single">、</u>Transmission of compressed data<u style="single">Active</u>Manage to. Also<u style="single">、</u>the system<u style="single">、</u>Encoding and compression settings to provide operating conditions to the compression system and optimize data delivery<u style="single">dynamic</u>Change to. Distribution software that resides on the terminal device<u style="single">Agent</u>Is a compression / decompression algorithm<u style="single">、</u>And the changes for each segment in the setting<u style="single">、</u>Unzip the data stream. Various instructions depending on the terminal device configuration<u style="single">,</u>For each segment of decompression algorithm and ending setting combination<u style="single">, Especially for very dull clients</u>refresh<u style="single">can do</u>.. Decompression instructions also remain, if appropriate for the terminal device<u style="single">Can keep you</u>。

Transmission to arrival device<u style="single">,and</u>Software described for operation by the arrival device<u style="single">Agent</u>Therefore, of the described compression / delivery systems and methods<u style="single">,very</u>beneficial<u style="single">Aspect</u>You might also say that. software<u style="single">Agent</u>By delivering from the source to the device, a widespread arrival device<u style="single">To</u>, One or more algorithms<u style="single">Variable use, or</u>Other behavior at the transmission source<u style="single">To</u>In ways that can be included<u style="single">Take</u>For communication<u style="single">、</u>use<u style="single">can do</u>.. In other words, the arrival device is "format-specific" as required by many traditional streaming and static media communication systems.<u style="single">of</u>Being a player<u style="single">To</u>request<u style="single">You don't have to</u>.. It also has a diagnostic function.<u style="single">Destination agent</u>Appropriate CODEC for a given set of different situations by providing<u style="single">processing</u>To<u style="single">Do</u>Diagnostic information in neural network processes for<u style="single">To</u>Collect in the arrival device<u style="single">Can be</u>、<u style="single">further,</u>Return to the source in a format that conforms to the source usage<u style="single">Su</u>As transmitted<u style="single">can do</u>。

In real time<u style="single">、</u>Client side<u style="single">of</u>Device data<u style="single">、</u>And communication channel status<u style="single">、</u>Quality of service information<u style="single">To provide</u>Client side<u style="single">Agent</u>use<u style="single">To do</u>Also,<u style="single">Therefore</u>, Specific use or provided here<u style="single">Su</u>Others of the present invention<u style="single">Aspect</u>Beyond the combination of<u style="single">、</u>Seems to be widely beneficial. In addition, the processing of each unit of transmission-ready data compressed to meet client-side equipment and real-time communication channel conditions also has widespread benefits.<u style="single">To be interpreted</u>.. Furthermore, the client side<u style="single">Agent</u>Instruct<u style="single">Sequentially uniquely compressed data units</u>Systems and methods that enable each decompression<u style="single">To</u>Description<u style="single">Shi</u>ing. Therefore, another broad benefit of the present invention is<u style="single">、</u>Decompress the compressed representation of the original media signal on the arriving device to decompress the original media signal<u style="single">Aspect</u>Related to at least one of the transmission channel constraints, as well as the arrival device constraints<u style="single">To do</u>Based on variable parameters<u style="single">、</u>Defrost<u style="single">did</u>Provides a CODEC that can be expressed (described here for a particular embodiment)<u style="single">Shi</u>From a transmission source, etc.). Another widespread<u style="single">Aspect</u>In, the arrival device<u style="single">、</u>Of the original media signal<u style="single">Aspect</u>Related to<u style="single">To do</u>A CODEC selected from the CODEC library based on the parameters can be used.

The systems and methods described herein are also considered applicable to the signal processing of each unique and sequential unit of signal data, such as signal clips, video frames, or, where appropriate, individual packets. Be done. In addition, the system and its various subsystems must also be loaded on the appropriate device, respectively, or on the server side, etc., and in certain situations on the client side (eg, various arrival mediators). Phase) It can be pure software that can be embedded in a component or chip of the host hardware or otherwise stored in flash memory or the like.

Various aspects of the media systems and methods just described are described, for example, in various communication devices, communication / transmission channel formats and standards, as well as other than those described herein (eg, in the "Background Technology" section above). It is believed to be beneficial for use in compliance with a wide range of known and directly foreseen media communication requirements, including compliance with media types and formats.

However, for the purpose of further understanding, FIG. 6 shows a schematic representation of the overall streaming media communication system 600 as specifically applied to the "video on demand" aspect according to an embodiment of the present invention. , Many different end users 610-620 in many different locations may request and receive pre-recorded video from remote sources in real time (eg, with virtually no delay). In addition to the information provided in Figure 6, at least one specific embodiment is to deliver the following types of video at the following bit rates (observed by the arrival device by the eyes of a typical human observer): Note the compressed representation of the original signal that can be converted to a decompressed representation with no or substantial loss as such), ie a low VHS-format video of about 250 Kbs, a DVD of about 400 Kbs. -Format video, HDTV at about 900Kbps-Format video. According to these aspects, video on demand could be provided by telephone carriers across resident transmission line channels, for example over existing DSL lines 630-640.

However, greater efficiency will be achieved to deliver compressed representations of these types of video signals even at lower bit rates, as available bandwidth and mass communication continue to present problems. Again, as described elsewhere, the compression efficiencies of the present invention continue to be trained and trained on the processing powers available to the neural network 440 and the modified types of media. It is closely related to the function with the neural network 440 and is improved as the function. These resources make it possible to achieve more significant compression efficiencies without modification to the basic features of the invention.

Therefore, the following is a further example of the transmission rate for a particular compressed video signal that is desired and is believed to be achievable according to an embodiment of the invention, ie, about 200 Kbps, more preferably about 150 Kbps, And even more preferably as low as about 100 Kbps VHS-format video, about 350 Kbps, more preferably about 300 Kbps, and even more preferably as low as about 250 Kbps DVD-format video, and about 800 Kbps, and even more preferably about. HDTV-format video as low as 700Kbps.

Furthermore, at least one embodiment of the media communication system 400 according to an embodiment of the present invention delivers 20-24 frames / sec color video at a transmission rate of 7 Kbps. This will enable substantial advancement in the communication of streaming media signals to wireless arrival devices via WAP gates, as will be further deployed elsewhere in the future.

It should also be understood that while video communication was emphasized in this disclosure, other types of streaming or static media are also intended. For example, at least one embodiment of the compression and delivery embodiment is observed and there is no substantial loss as observed by the arrival device by the ears of a typical human observer. It provides (via a compressed representation of the original signal) that can be converted to a decompressed representation (or without substantial loss) at a bit rate of approximately 24Kbps. At these rates, audiophile-quality sound can be delivered for playback across dial-up modems. However, with respect to more available resource obligations and the degree of neural network training, it is further intended that the present invention produce CD quality sound at speeds as low as about 20 Kbps and even lower as about 15 Kbps or 10 Kbps. It can be delivered.

<u style="single">Wireless audio communication system</u> Further intended, the streaming media communication systems of the present invention have particularly useful applications within wireless audio communication networks, especially within cellular communication networks. Therefore, FIGS. 7 and 8 illustrate streaming media communication systems 700 and 800 specifically applied to wireless audio communication systems according to various specific embodiments of the present invention, with an increasing amount of detail. It is shown in. While the particular device, system parameters, or array of communication devices shown may be useful for the overall use of the present invention, they should not be considered limiting and this. It can be appropriately replaced by other alternatives according to those skilled in the art based on the disclosure. Thus, the standards and protocols of the various wireless communication systems 700 and 800 referenced elsewhere in this disclosure include various aspects of compression, distribution, decompression, and transcoding according to an embodiment of the present invention. Incorporated in this column for the purpose of integration.

The combination of the communication system 400 according to an embodiment of the present invention with other components of the cellular communication network allows for improved compression, distribution, and decompression according to the present invention, for wireless audio communication. It is manifested in the increased quality of service. Improvements in cellular communications in accordance with the present invention include, without limitation, examples of increased available bandwidth, extended coverage, and low signal quality or low reception levels while providing appropriate demotion. Maintain a bond or connection during the period.

More specifically, cellular communication signals are a relatively high degree of variable due to, for example, the roaming position of the client and the limited cell range, atmospheric conditions, and the significantly limited and variable available bandwidth over the daily usage cycle. Characterized by. Therefore, the self-optimized CODEC management system according to the present invention is particularly well suited for adjusting proper communication and compression modes for changing environments. At the very least, the increased compression efficiency and the resulting reduction in bandwidth used for a given signal is a valuable feat when wireless channel traffic continues to be congested.

In one particular point, the increased compression efficiency according to the present invention is fully applied to improve the bandwidth problem during "soft handoff" between cells, as illustrated in FIG. Whenever a transmitter or receiver moves within a cell coverage area during cellular telephone communication, communication bandwidth requirements and the resulting costs are increased by system requirements to "successfully" perform active communication between cells. To. The act of successfully communicating creates a "backward" channel from the preceding active cellular transmitter to the central office and transfers it to the newly activated cellular transmitter. This reverse channel represents a major use of bandwidth. Savings result from increased compression. As shown in the figure, such "backward transmission" doubles the bandwidth used to communicate a particular signal (media reversely transmitted from cell 1 for communication leaves to cell 2). It can include (retransmitted) or even quadrupled (partially duplicated communication from both the first and second cells).

The media communication system 400 of the present invention can recognize when backhaul occurs according to the transmission channel diagnosis provided in the software medium, and can respond by adjusting the degree of compression to be compensated.

<u style="single">WAP video gateway</u> Due to a special study of the rapid growth observed and predicted on the wireless or mobile Internet, the embodiments of the present invention are intended for use in intelligent compression / decompression embodiments in combination with the WAP gateway function.

Therefore, systems and methods for encoding, compressing, and transmitting complex digital media (eg, videographs) over bandwidth-constrained wireless communication systems that utilize the Wireless Application Protocol (WAP) are also in accordance with the present invention. It is provided. In one embodiment, utilizing a neural network with artificial intelligence, the data is processed by a system resident at a server node (s). Sample segments of data are taken from the input stream and processed to meet the unique requirements of client classification. As detailed above, the system correlates the characteristics of a continuously changing digital data stream with a library of pre-developed, empirically learned rules, and ultimately the client. It correlates with the coherence, continuity, and externally imposed constraints to optimally choreograph or configure the details as received, decoded, and presented at the interface.

The gateway with the added functionality of the streaming media communication system described here is outlined in Figure 8. According to the WAP gateway system 830, client mediation is provided, it can run on various platforms and does not require any specialized hardware to decode the video stream. According to the use of the streaming media distribution system of the present invention described elsewhere, the viewer of the WAP apparatus maintains constant communication with the upstream system server, and the user client 825 streams to the encoding platform. Streaming by being able to provide relevant information for media communication, i.e., without limitation, including effective screen size, processing power, client operating system and browser version, connection speed, and latency. The media distribution system allows the stream to be customized and made for each individual client that "speaks". Therefore, an AI-driven server 830 that incorporates AI compression as described here can be combined with the WAP gateway 830 to allow the required WAP to TCP / IP protocol (or other protocol, ie dual server stack). ) The translation is combined with a compression system, a distribution system, and a video and audio server 835 that utilizes the decompression system and the methods described herein. The WAP gateway 830 may further include, for example, a transcoder system and a video transcoder incorporating the methods described herein. A suitable host architecture (not shown) according to this system generally includes a rack-mounted system running the Linux OS either by the modified WAP Gateway 830 or as a plug-in to an existing server.

The WAP Gateway 830 is also offered in a master / slave relationship as another benefit aspect of the overall streaming media delivery architecture, especially applicable to other delivery systems other than wireless. Various content distribution networks, such as those available through Akamai and Inktomi, have the concept of improving data delivery across the Internet by using "smart caching" on servers residing on the boundaries of the Internet. I used it. Such a master / slave relationship is maintained by the system, the master server resides at the source of the content to be delivered, and the slave server resides on the boundary. These servers communicate "intelligently" to optimize content delivery across the Internet, reducing latency, bandwidth, and storage requirements to improve the overall quality of video / audio streams for end users. While improving, it is reducing the cost of media distribution to content providers.

The WAP gateway 830 of the present invention supports continued growth in mobile communications, as large telecommunications operators move to multi-service broadband networks and the number of mobile Internet subscribers increases. This is the case when it continues to expand rapidly. Mobile communications in particular are a broad classification of systems and protocols, each with its own constraints and the need for bidirectional devices to communicate streaming media. A particularly beneficial aspect, the Gateway 830 will be accompanied by various "2.5G" and "3G" network technology improvements (numerical advances in the system generally represent advances in Internet-enablement capabilities). Can support "2G" systems.

Table 3 below provides examples of known mobile communication standards and the specifics used by the AI system according to the invention to optimize streaming media communication between the fields of mobile arrival devices as a media player. Provides related information.

<tables num="3"><img file="JP4852228B2_D0007.tif" /></tables>

In addition, the present invention is particularly beneficial to the ability to send a wide variety of media signals to a variety of different types of wireless communication devices. Examples of wireless communication devices suitable for use with the streaming media communication system and the methods of the present invention are provided in Table 4 below, with examples where these systems and methods are interchangeably supported.

<tables num="4"><img file="JP4852228B2_D0008.tif" /></tables>

Various specific examples are described below and provide observations of actual wireless internet applications of the present invention as described herein. Such examples include, at a minimum (without limitation), the use of CODEC libraries according to incoming wireless communication devices, transmission channels, communication protocols, and various parameters associated with each of their own streaming media signals. The various unique features of the systems and methods used according to these examples appear to further define the independently beneficial aspects of the invention.

<u style="single">Shared interactive environment</u> Provided are systems and methods according to the invention for enabling real-time remote client interaction with high-definition, multidimensional, multi-participation simulated environments without the need for significant client-side processing power. To. More specifically, FIG. 10 shows an overall streaming media communication system as applied to a shared interactive game according to the present invention.

The system includes (i) a proxy server, (ii) a client software medium that supports the distribution system of the invention, and (iii) server-to-client streaming. The intent is that for a large number of clients that typically represent an interactive game shared by design, a large number of components, such as those just described, are provided to support each client. is there.

The interactive game embodiment is intended to embody data compression and a distribution embodiment comprising a device that is also an arrival device for a compressed signal from another similarly remotely located device system. This configuration has a wide range of benefits, for example in further interactive media realizations such as video conferencing. Therefore, each remote system is both a source and an arrival device, and it sends and receives mediators between it and other remote systems.

<u style="single">Arrival device</u> The communication system of the present invention is capable of communicating for a wide variety of arrival devices of streaming media signals, and a further intended feature of the present invention should be accommodated as an arrival device / plier by the client user. To provide a remote receiver. This set-top box is Video on Demand (VOD); Music on Demand (MOD); Interactive Games on Demand (IGOD); Voice Over Internet Protocol ("VoIP") Yes, any technology that provides voice telephony services over IP connectivity; TV web access; digital video recording, pause, and playback live to record, pause, and play live TV. Television; E-mail; Chat; DVD player; and, among other uses that are obvious to those skilled in the art, preferably many, and possibly all, may be adapted to serve at least one. .. All of this can be delivered to existing televisions in the comfort of the user's own home. Furthermore, a client utilizing this box or other system interfaced with the communication system of the present invention may receive DVD quality video and surround sound over a cable and DSL connection.

<u style="single">Example</u> For the purpose of further explaining the highly beneficial results that can be achieved in the present invention, the following are examples of specific examples used for different types of streaming media communication, including observations with relevant controversy. .. These examples are the communication of the same pre-recorded video over different transmission channels, illustrating communication for different arriving devices, where the pre-recorded video has a resolution of 720 lines, 32 bits of color information, and a occurrence file. It has various occurrence characteristics of about 1.4 gigabytes in size.

Example 1 An "iPAQ" model 3650 handheld PDA (commercially available from Compaq Inc. for approximately $ 500 at the time of this disclosure) was offered. The PDA is a 14.4Kbps (maximum) wireless CDPD modem (a commercially available "AirCard 300" wireless external modem from Sierra Wireless for approximately $ 200 at the time of this disclosure) and an expansion assembly (Compaq Inc.). Interfaceed using the iPAQ PCMCIA Expansion Sleeve) with a PCMCIA card slot that couples with this wireless modem. The iPAQ used is generally 206MHz processor, 32Mb memory, 12b / pixel color, 240x320 screen dimensions, Microsoft. It is characterized to have Corp. PocketPC operating system version 3.0, as well as stereo sound processing parameters. iPAQ was connected to the Internet in San Francisco, California via an interfaced CDPD modem across the AT &amp; T Cellular Wireless Carrier System with a connection bandwidth of approximately 13.3 Kbit / s. A server located in San Jose, California (about 5 miles away) was contacted by a PDA using the http and rtsp protocols, and the PDA had a resolution of 720 lines, 32-bit color information, and a occurrence file size of 1.4 gigabytes. Used to initiate the demand for pre-recorded video with characteristics. Within about 7 seconds, a compressed approximation of the pre-recorded video was received, decompressed, and displayed by the PDA on the PDA screen. All video was viewed at full operating 240x320x12bpp resolution with no observable delay or defects.

Example 2 A "Jornada " model 548 handheld PDA (commercially available from HP Inc. for approximately $ 300 at the time of this disclosure) is available. The PDA includes a 9.6Kbps (maximum) wireless CDMA phone (a "Motorola i85s" wireless external digital cellular phone commercially available from a Motorola approved vendor for about $ 200 at the time of this disclosure) and this They were interfaced using adapter cables (Motorola and HP RS-232 standard interface cables from Motorola and HP) that combined the telephone and PDA together to form a wireless modem. The Jordnada model PDA device used is a 133MHz processor, 32Mb memory, 12b / pixel color, 240x320 screen dimensions, Microsoft. It is characterized to have Corp. PocketPC operating system version 3.0, as well as stereo sound processing parameters. Jordana was connected to the Internet in Newark, NJ via an interfaced CDMA phone / modem over the Nextel Digital Cellular Wireless Carrier System with a connection bandwidth of approximately 8 Kbit / s. A server located in San Jose, California (approximately 2900 miles away) was contacted by a PDA using the http and WDP protocols, which resulted in 720 resolutions, 32-bit color information, and a occurrence file size of 1.4 gigabytes. Used to initiate the demand for pre-recorded video with properties. Within about 7 seconds, a compressed approximation of the pre-recorded video was received, decompressed, and displayed by the PDA on the PDA screen. All video was viewed in full color 176x120x8bpp resolution with no observable delay or defects.

Example 3 A "set-top box" model st850 book-type PDA (commercially available from MSI Inc. for approximately $ 300 at the time of this disclosure) is available. This set-top box uses a 10Mbps (maximum) Ethernet® / 802.11 connection and a CAT Ethernet® cable (Generic) that combines this set-top box with a broadband connection (DS3). Was interfaced. Set-top boxes used include 400MHz processor, 64Mb memory, 32b / pixel color, 720 screen resolutions, Microsoft Corp. CE operating system version 2.11 and AC3 digital 6-channel surround sound. It is characterized to have various processing parameters. This set-top box has an Alter.Net Internet with a connection bandwidth of approximately 376 Kbit / s. Across the backbone, it was connected to the Internet in Newark, NJ via an interfaced shared DS3 connection. A server located in San Jose, California (approximately 2900 miles away) is contacted by this set-top box using the http and rtsp protocols, and the set-top box has 720 resolutions and 32-bit color information. Used to initiate the demand for pre-recorded video with occurrence characteristics of occurrence file size 1.4 gigabytes. Within about 9 seconds, a compressed approximation of the pre-recorded video was received, decompressed, and displayed by the set-top box on a reference monitor (Sony) screen available for commercial play. All video was seen at 720 x 32bpp in full operation with no observable delay or defects.

Various specific examples have been shown and described herein in significant detail for the purpose of describing the invention, but it should be understood that further changes and improvements deviate from the intended scope of the invention. Nothing can be done by one of ordinary skill in the art based on this disclosure. For example, various possible combinations of various embodiments not described in detail may be made, but they remain within the intended scope of the invention. According to another example, obvious improvements or changes can be made in various embodiments and still fall within the intended scope of the invention.

<figref num="1A">FIG. 1A shows a schematic block diagram showing two various modifications of the media communication system according to the prior art using the conventional CODEC system.</figref><figref num="1B">FIG. 1B shows a schematic block diagram showing two various modifications of the media communication system according to the prior art using the conventional CODEC system.</figref><figref num="1C">Figure 1C shows a schematic block diagram showing two variants of the media transcoder system according to the prior art.</figref><figref num="1D">Figure 1D shows a schematic block diagram showing two variants of the media transcoder system according to the prior art.</figref><figref num="1E">Figure 1E shows a schematic block flow diagram of various interrelated components in the prior art WAP gateway communication system.</figref><figref num="2">2 to 3 show a schematic block diagram of a transcoder system according to an embodiment of the present invention in two various modes of use.</figref><figref num="3">2 to 3 show a schematic block diagram of a transcoder system according to an embodiment of the present invention in two various modes of use.</figref><figref num="4A">4A-5 show schematic block flow diagrams of various correlation components of a media communication system according to an embodiment of the present invention.</figref><figref num="4B">4A-5 show schematic block flow diagrams of various correlation components of a media communication system according to an embodiment of the present invention.</figref><figref num="5">4A-5 show schematic block flow diagrams of various correlation components of a media communication system according to an embodiment of the present invention.</figref><figref num="6">FIG. 6 shows a schematic block flow diagram of various correlation components of a video on demand streaming video communication system according to an embodiment of the present invention.</figref><figref num="7">FIG. 7 shows a schematic block flow diagram of various correlation components of a wireless streaming video communication system according to an embodiment of the present invention.</figref><figref num="8">FIG. 8 shows a schematic block flow diagram of various correlation components of a WAP gateway media communication system according to an embodiment of the present invention.</figref><figref num="9">FIG. 9 shows a schematic block flow diagram of various correlation components of a media communication system in reverse transmission according to one specific use mode of the media communication system according to an embodiment of the present invention.</figref><figref num="10">FIG. 10 shows a schematic block flow diagram of various correlation components of an interactive gaming communication system and set-top TV browsing according to an embodiment of the present invention.</figref>

Code description

100 source media 130,135,140 Arrival device 145 CODEC 200 Video / Audio Transcoder 210 video source 300 servers 400 Data compression and distribution system 405 Media Module 407 Dynamic Player Module 410 image processor 415 Baseline snapshot 417 Classifier 420 Standard Quality (QoS) Module 425 Network Layer Input Module 430 Network Layer Output Module 435 timer 440 Neutral Network Processing Module 445 CODEC library module 450 Dynamic Client Request Module 455 ICMP module 460 Equipment and network parameter measurement module 465 Delivery and transmission module