Processing and providing an image in which a plurality of symbols are encoded.

PROCESS AND PROVIDE AN IMAGE IN WHICH ONE IS CODED

PLURALITY OF SYMBOLS

BACKGROUND OF THE INVENTION

FIELD OF THE INVENTION

[0001} The embodiments described herein are generally related to a method, a non-transient computer readable storage medium, and a receiving apparatus for processing an image in which a plurality of symbols are encoded; and a method, a non-transient computer readable storage medium, and an apparatus that provides information to provide the image.

BACKGROUND S

[0002] Implementing effective methods for the distribution of digital data within digital television systems is a significant consideration for designers and manufacturers of contemporary electronic entertainment systems. However, effectively implementing such systems can create substantial challenges for system designers. For example, the increased demand for increased system performance and functionality may require more capabilities and requires additional hardware and software resources. Impediments to the effective delivery of digital data in advanced systems can result in a corresponding detrimental economic impact due to operational inefficiencies, lost revenue opportunities, and reduced functionality.

[0003] Furthermore, the enhanced system ability to perform various advanced operations may offer additional benefits to the end user, but may also place increased demands on the control and handling of various system components. For example, an improved electronic system that effectively supports the functionality of synchronized television graphics elements may benefit from methods that provide flexible transport of the data stream that supports this functionality.

[0004] Due to the increasing demands of system resources and the magnitudes of data increasing substantially, it is apparent that new techniques to implement and use data distribution through digital television systems and other video distribution systems are to be developed, is a matter of concern for related electronic technologies. Therefore, for all of the foregoing reasons, developing effective systems to implement and utilize data distribution across digital television systems, and other video distribution systems, remains a significant consideration for designers, manufacturers, and users. of contemporary electronic entertainment systems.

COMPENDIUM OF THE INVENTION

[0005] In accordance with an embodiment of the present disclosure, a receiving apparatus is provided, including circuitry that is configured to receive or retrieve an image in which a plurality of symbols are encoded. The circuitry determines a set of luminance values ​​used to encode the symbols, based on luminance values ​​of a plurality of pixels included in the image. The circuitry determines a higher luminance value used to encode the symbols in the image, based on the determined set of luminance values. In addition, the circuitry derives data values ​​from the encoded symbols in the image, based on the set of luminance values ​​and using the highest determined luminance value.

[0006] Furthermore, in accordance with an embodiment of the present description, a method is provided for processing an image in which a plurality of symbols are encoded. The method includes receiving or retrieving the image in which the plurality of symbols is encoded, via circuitry from a receiving apparatus. A set of

<img file="MX370229B_D0001.tif" />

luminance values ​​used to encode symbols based on luminance values ​​of a plurality of pixels included in the image. A higher luminance value, used to encode the symbols in the image, is determined by the circuitry based on. the determined set of luminance values. In addition, data values ​​of the symbols encoded in the image are derived, using circuitry, based on the set of luminance values ​​and using the highest determined luminance value.

[0007] In addition, in accordance with one embodiment of the present disclosure, a non-transient computer readable medium is provided that stores instructions, which when executed by a computer cause the computer to perform a method of processing an image in which encodes a plurality of symbols. The method includes receiving or retrieving the image in which the plurality of symbols is encoded. A set of luminance values ​​used to encode the symbols is determined based on luminance values ​​of a plurality of pixels included in the image. A higher luminance value used to encode the symbols in the image is determined based on the determined set of luminance values. Furthermore, data values ​​of the symbols encoded in the image are derived based on the set of luminance values ​​and using the highest determined luminance value.

[0008] Further, in accordance with an embodiment of the present disclosure, a receiving apparatus is provided including circuitry configured to receive or retrieve an image in which a plurality of symbols are encoded. The circuitry determines a first set of luminance values ​​used to encode a first subset of the symbols, based on luminance values ​​of a first plurality of pixels included in the image. The circuitry derives data values ​​from the first subset of the symbols encoded in the image, based on a first predetermined cutoff point. The circuitry determines a luminance value used to encode a second subset of the symbols in the image, based on the data values ​​derived from the first subset of the symbols. The circuitry determines, a second set of luminance values ​​used to encode the second subset of the symbols, based on luminance values ​​of a second plurality of pixels included in the image. In addition, the circuitry derives data values ​​from the second subset of the encoded symbols in the image, based on the second set of luminance values ​​and using the determined luminance value.

[0009] Furthermore, according to an embodiment of the present description, a method is provided for processing a

<img file="MX370229B_D0002.tif" />

image in which a plurality of symbols is encoded. The method includes receiving or retrieving, via circuitry from a receiving apparatus, an image in which a plurality of symbols are encoded. A first set of luminance values ​​used to encode a first subset of the symbols is determined based on luminance values ​​of a first plurality of pixels included in the image. Data values ​​are derived from the first subset of the symbols encoded in the image based on a first predetermined cutoff point. A luminance value used to encode a second subset of the symbols in the image is determined by the circuitry based on the data values ​​derived from the first subset of the symbols. A second set of luminance values ​​used to encode the second subset of the symbols is determined, based on luminance values ​​of a second plurality of pixels included in the image. Data values ​​of the second subset of the symbols encoded in the image are determined by the circuitry based on the second set of luminance values ​​and using the determined luminance value.

[0010] In addition, in accordance with an embodiment of the present disclosure, a non-transient computer readable medium is provided that stores instructions, which when executed by a computer cause the computer to perform a method of processing an image in which encodes a plurality of symbols. The method includes receiving or retrieving an image in which a plurality of symbols are encoded. A first set of luminance values ​​is determined. used to encode a first subset of the symbols, based on luminance values ​​of a first plurality of pixels included in the image. Data values ​​are derived from the first subset of the symbols encoded in the 1st image based on a first predetermined cutoff point. A luminance value used to encode a second subset of the symbols in the image is determined based on the data values ​​derived from the first subset of the symbols. A second set of luminance values ​​used to encode the second subset of the symbols is determined based on luminance values ​​of a second plurality of pixels included in 1.a. image. Data values ​​of the second subset of the symbols encoded in the image are determined based on the second set of luminance values ​​and using the determined luminance value.

[0011] In addition, in accordance with an embodiment of the present disclosure, an information-providing apparatus is provided, including circuitry configured to receive or retrieve an image in which a plurality of symbols will be encoded. The circuitry encodes the plurality of symbols in the image. The plurality of symbols is encoded in the image using luminance values ​​of a plurality of pixels included in the image. Furthermore, the circuitry provides the image in which the plurality of symbols is encoded to a receiving apparatus. An luminance value used to encode at least one of the plurality of symbols is set by an operator.

[0012] In addition, in accordance with an embodiment of the present disclosure, a method is provided for providing an image in which a plurality of symbols are encoded. The method includes receiving or retrieving an image in which the plurality of symbols will be encoded. The plurality of symbols is encoded in the image by circuitry from an information-providing apparatus. The plurality of symbols is encoded in the image using luminance values ​​of a plurality of pixels included in the image. Furthermore, the image in which the plurality of symbols is encoded is provided by the circuitry to a receiving apparatus. An luminance value used to encode at least one of the plurality of symbols is set by an operator.

[0013] In addition, in accordance with one embodiment of the present disclosure, a non-transient, computer-readable medium is provided that stores instructions that, when executed by a computer, cause the computer to perform a method of providing an image in the which encodes a plurality of symbols. The method includes receiving or retrieving an image in which the plurality of symbols will be encoded. The plurality of symbols is encoded in the image. The plurality of symbols is encoded in the image using luminance values ​​of a plurality of pixels included in the image. Furthermore, the image in which the plurality of symbols is encoded is provided to a receiving apparatus. An luminance value used to encode at least one of the plurality of symbols is set by an operator.

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] A more complete appreciation of the present disclosure and many of the advantages associated therewith will be readily obtained, as it is better understood in reference to the following detailed description when considered in connection with the accompanying drawings, in where:

[0015] FIGURE IA is a block diagram of an electronic system, in accordance with an embodiment of the present description;

[0016] FIGURE IB is a diagram of a television display screen of FIGURE IA, according to one embodiment of the present disclosure;

[0017] FIGURE 2A is a block diagram for one embodiment of the content source of FIGURE IA, according to the present disclosure;

[0018] FIGURE 2B illustrates an apparatus that provides exemplary information that is implemented in the content source, in accordance with one embodiment of the present disclosure.

[0019] FIGURE 3 is a block diagram for a source memory embodiment of the content source, in accordance with the present disclosure;

[0020] FIGURE 4 is a block diagram for one embodiment of the metadata stored in the source memory, in accordance with the present description;

[0021] FIGURES 5A-5C are exemplary block diagrams of the receiving apparatus, in accordance with certain embodiments of the present disclosure;

[0022] FIGURE 6 is a block diagram for an embodiment of the TV memory of FIGURE 5A, in accordance with the present disclosure;

FIGURES 7A and 7B are digital data diagrams embedded in video data, in accordance with two different embodiments of the present disclosure; and

[0024] FIGURES 8A-8C are a flow chart of the steps of the method for distributing digital data, in accordance with an embodiment of the present description.

[0025] FIGURE 9 is a flow chart of the steps of the method of processing an image in which a plurality of symbols are encoded, in accordance with one embodiment of the present disclosure.

[0025] FIGURE .10 is a flow chart of the steps of the method of processing an image in which a plurality of symbols are encoded, in accordance with one embodiment of the present disclosure.

[0027] FIGURE 11 is a flow chart of the steps of the method for determining whether to encode a plurality of symbols in an image, in accordance with an embodiment of the present disclosure.

[0028] FIGURES 12A and 12B illustrate examples of I-bit and 2-bit encoding by symbol luminanoia;

[0029] FIGURE 13 illustrates an exemplary analysis of watermark symbols, in accordance with one embodiment of the present disclosure;

[0030] FIGURES 14A-14C illustrate exemplary JavaScript code for implementing a watermark retrieval algorithm, in accordance with one embodiment of the present disclosure; and [0031] FIGURE 15 is an exemplary computer.

DETAILED DESCRIPTION

[0032]

Although the present description is susceptible to

<img file="MX370229B_D0003.tif" />

represented in many different ways, shown in the drawings and specific modalities will be described in detail herein, with the understanding that the present description of such modalities will be considered as an example of the principles and is not intended to limit the present description to the specific modalities shown and described. In the. In the following description, like reference numerals are used to describe the same, like, or corresponding parts in the various views of the drawings. The specification includes the. following description and attached appendices.

[0033] The terms one or one, as used herein, are defined as one or more than one. The term plurality, as used herein, is defined as two or more than two. The term other, as used herein, is defined as at least one second or more. The terms it includes and / or has, as used herein, are defined as comprising (i.e., open language). The term coupled, as used herein, is defined as connected, although necessarily directly, and not necessarily mechanically. The term program similar terms, as defined as a sequence of execution in a system or computer program or used herein, is instructions designed for your computer. A computer program or program may include a subroutine, a program module, a set of instructions, a function, a procedure, an object method, an object implementation, in an executable application, an applet, a servlet, a source code, an object code, a shared library / dynamic load library and / or another sequence of instructions designed for execution on a computer system.

[0034] The term program, as used herein, can also be used in a second context (the definition above is for the first context). In the second context, the term is used in the sense of a television program. In this context, the term is used to mean any coherent sequence of audio / video content such as those that would be interpreted as and reported in an Electronic Program Guide (EPG) as a single television program, regardless of whether the content is a movie, sports event, multi-part series segment, 'news broadcast, etc. The term can also be interpreted to encompass commercial spaces and other program-like content that may not be reported as a program on an EPG. [0035] Reference throughout this document to a modality, certain modalities, an modality, an implementation, an example or similar terms means that a particular trait, structure or characteristic described in connection with the modality is included in at least one modality of the present description. Thus, the occurrence of such phrases or in various places throughout this specification does not all necessarily refer to the same modality. Furthermore, the particular features, structures or characteristics can be combined in any suitable way in one or more modalities without limitation.

[003S] The term or as used herein is to be construed as an inclusive or to mean any or any combination. Therefore, A, B, or C means any of the following: A; B; C; A and B; A and C; B and C; A, B, and C. An exception to this definition will occur only when a combination of elements, functions, steps, or actions are somehow inherent and mutually exclusive.

[0037] The embodiments of the present disclosure relate to the delivery of digital data (eg, metadata, application data, program guide data, subtitle data, etc.) into video data. Digital data is integrated as a watermark in video data. Using the first or two lines of a video to deliver digital data works well for cases where display screen devices operate in an overscan mode (for example, they only display half -95% of active video lines). If the display screen is set to a full pixel display screen mode, any watermarks in the first two lines will be visible. Therefore, it may be advantageous to use a lower visibility watermark to avoid it. possibility of the watermark becoming visible on the display screen.

[0038] Ξ1 method of modulating luminance values ​​depends on being able to retrieve the digital data at the receiver. Like any pulse amplitude modulation (PAM) system, the receiver must set a cutoff point to use to determine whether a particular symbol is a one or zero (or, for a 2-bit per symbol system, value 00, 01, 10, or 11). The modalities of the present description avoid the need for a standard to specify exactly the nominal luminance values ​​that are used for the different symbol values. Therefore, the broadcaster is provided with the flexibility to adjust the values ​​to compensate between watermark visibility and robustness. Higher luminance values ​​lead to greater visibility and higher solidity and / or bits per symbol. Lower values ​​lead to less visibility at the cost of less solidity and / or bits per symbol.

[0039] Certain modalities of the present description involve a system through which data is delivered

<img file="MX370229B_D0004.tif" />

digital within a video signal, by modulating the luminance level in the first or second video line. Digital data can be considered a video watermark. The design issue relates to how the receiver determines the best method to retrieve the watermark. For a fixed definition system, the standard would specify a modulation method, and the receivers would then implement a retrieval algorithm that matched the standard.

[0040] It would be advantageous if the diffuser could have some flexibility in the way the luminance signal is modulated. In this case, a standard could specify that the lowest luminance value used to encode symbols will be 0 (where the luminance value 16 is considered black), and the highest luminance value will be in a proportionate range of values ​​(for example , from 30 to 235). For a 2-bit-per-symbol encoding, the two average luminance values ​​could be calculated. If N is the highest luminance value used for encoding, the four codes in use would be 0, N / 3, 2N / 3, N. For 1-bit-per-symbol encoding, the values ​​would be 0 and N. you can use more bit-by-symbol encoding in other modes.

[0041] For example, some broadcasters may choose to use low values ​​(visibly darker) because they are concerned about visibility (some consumer devices, under some display screen settings, may make the first two lines of video visible to the user ). Others may not be concerned with visibility, and may instead want to optimize robustness, knowing that a broader range of luminance values ​​outlives transcoding and heavy video compression, rather than a smaller range. A wider range of luminance values ​​could also allow diffusers to include more bits per symbol.

[0042]. Using certain modalities, described herein, instead of or in addition to having a standard that specifies the required levels of luminance, the receiver determines the range of values ​​in use by analyzing and adjusting the watermark recovery algorithm accordingly . In other embodiments, the watermark itself or any encoded portion indicates the appropriate parameters to use for optimal recovery. In this case, the protocol delivers data that can be recovered and used to establish the optimal recovery. The modalities described herein allow the broadcaster to set the level of robustness and / or bits per symbol in use, as desired, as a tradeoff between robustness and / or bits per symbol, and visibility of the watermark.

[0043] Although the present disclosure is primarily described using a watermark embedded in line 1 of a video frame, the watermark may be embedded in other lines or other predetermined portions of a video frame. In addition, in certain modalities the watermark can be integrated into other types of images.

[0044] The present disclosure is described herein as a system and method for distributing digital data (eg, represented by a plurality of symbols) embedded in video data, and includes a content source that integrates digital data into video data. The content source then encodes the video data along with the digital data to create a distribution multiplex that includes compressed video data. A decoder receives and decompresses the distribution multiplex to reproduce the video data with the integrated digital data. A television or other display device then detects and extracts the digital data from the video data.

[0045] The television or other device processes the digital data to receive information, for example, which enables the display device to identify a channel currently being watched and recognize a channel change; identify the content being viewed, which includes short content such as transition ads; to discover a location to access additional information about the content (for example, a URL from a remote server ·; identify the temporary location within the content being played, ideally at a precision level per sample or per access unit; and / or to receive a real-time, time sensitive event trigger.

[0046] Referring now to FIGURE IA, a block diagram of an electronic system 110 is shown, in accordance with one embodiment of the. present description. In the FIGURE IA embodiment, the electronic system 11.0 may include, but is not limited to, a content source 114, a set top box 118, an interface 126, a television 122, an optional network 134, and a server 130 optional. In alternate embodiments, the electronic system 110 may be implemented using components and configurations in addition to, or instead of, certain of those components and configurations discussed in conjunction with the embodiment of FIGURE 1A. For example, any number of televisions 122 can be similarly displayed on the electronic system 110. Additionally, network 134 and server 130 may not be included in all modes of the present disclosure.

[0047]

In the FIGURE mode. IA, the content source 114 may be implemented as one or more electronic devices or other entities that prepare and distribute content data, including video data and audio data, for playback on television 122. In the embodiment of FIGURE 1A, content source 114 may be implemented as any appropriate entity. For example, content source 114 may include a television broadcast facility, cable television distribution facility, satellite television distribution facility, or an Internet server entity. Further details regarding the implementation and use of content source 114 are discussed further below in conjunction with FIGURES 2-4.

[0048] In the embodiment of FIGURE IA, content source 114 creates a coded distribution multiplex containing the content data in a compressed format, and then distributes the distribution multiplex over a distribution network via the route 116 (eg, a terrestrial television broadcast channel, cable TV network, satellite broadcast channel, etc.) to a receiving apparatus for decoding. In certain embodiments, additional devices or entities may come between the content source 114 and the set-top box 118. Examples of such entities may include, but are not limited to, a zl broadcast network affiliate and a service provider (such such as a satellite or cable headend).

[0049] In the embodiment of FIGURE IA, the converter-decoder box 118 decodes the encoded distribution multiplex to generate uncompressed A / V data (video data and audio data) that is provided to television 122 by a appropriate 126 interface. In the embodiment of FIGURE IA, interface 126 can be implemented in any effective way. For example, interface 126 can be implemented in accordance with a standard High Definition Multimedia Interface (HDMI) that provides a parallel high-speed interface to deliver uncompressed video and audio data, and / or control / synchronize signals to the television 122. Then television 122 can sensitively receive and reproduce the video and audio data for use by a system user. Further details regarding the implementation and use of television 122 are discussed further below in conjunction with FIGURES 5-6. [0050] In the embodiment of FIGURE 1Ά, the electronic system 110 supports additional services that relate to the main content data. Additional services include Declarative Objects (DOs), also called applications, to provide the interactive user experience. DOs and other additional services are described in ATSC Candidate Standard: Interactive Services Standard A / 105: 2014 (S13 ~ 2 ~ 389r7, Rev. 7 to April 24, 2014), which is incorporated herein for reference in its entirety.

[0051] Television 122 can obtain metadata from any appropriate source including, but not limited to, content source 114 or server 130. In the embodiment of FIGURE IA, television 122 can communicate with server 130 through any effective network 134 that includes, but is not limited to, the Internet. Additional details regarding the creation, distribution, and use of metadata are discussed further below.

[0052] The present disclosure generally involves integrating digital data (eg, metadata) into a video signal so that digital data can be retrieved quickly and easily by receiving devices such as television 122. In certain embodiments, the content source 114 inserts the digital data into a distributed video signal so that the digitals travel through the distribution chain, enter a consumer's home via a compressed interface (of a cable, satellite, or IPTV service provider), are unzipped into the converter-decoder box 118, and then moved to television 122 in an uncompressed format, where television 122 retrieves and uses the embedded metadata to support additional services, such as synchronized DOs. The foregoing techniques can prevent service providers or other entities from voluntarily or unintentionally blocking consumer access to the metadata that is required to provide enhanced functionality to television 122.

[0053] Certain cable, satellite, and IPTV entities typically provide system users with converter-box decoders that interface to digital televisions through uncompressed HDMI video interfaces or other appropriate means. Whether a content owner wants to include metadata (such as a URL, applet, etc.) with the content data, and whether that metadata is scrolled with the content data as a separate digital stream (or as metadata within the stream of bits compressed), the metadata will be locked in the converter-decoder box 118.

[0054] Typically, the converter-decoder box 114 does not pass auxiliary data streams into the distribution multiplex, because the converter-decoder box decodes only audio and video data, and then passes only the video data and uncompressed audio data through television. Therefore auxiliary data streams are available for television. Furthermore, if the service providers (those that offer the converter-decoder boxes) do not perceive that it is competitive to their business model to provide access to any auxiliary data, they may not be willing to help the consumer electronics industry by provide such access.

[0055] By integrating digital data into video data, digital data survives compression / decompression and is capable of reaching television intact 122. In addition, in embodiments of the present disclosure, digital data is integrated as a trademark. of water in a way that is suitable for visibility. In other words, the present disclosure advantageously integrates digital data within the video signal (encoded within the video image, not as a separate auxiliary data stream) in a manner that decreases its visibility to a viewer. Therefore the present description does not successfully overcome only the architectural barriers discussed in the foregoing, but also limits the visibility of the integrated watermark to avoid possible distraction from the viewer. The implementation and use of the electronic system 110 illustrated in FIGURE 1A is further discussed below.

[0056] Referring now to FIGURE IB, a diagram of a television 138 display screen 138 of FIGURE IA is shown, in accordance with one embodiment of the present disclosure. The embodiment of FIGURE IB is presented for illustration purposes, and in alternate embodiments, display screen 138 may be implemented using components and configurations in addition to, or instead of, certain of those components and configurations discussed in conjunction with the embodiment of FIG. FIGURE IB.

[0057] In the embodiment of FIGURE IB, display screen 138 includes a main screen region that typically displays video data provided by content source 114 (FIGURE IA). In the embodiment of FIGURE IB, display screen 138 also includes an DO 144 that resides in a discrete area shown on display screen 138 to provide any desired additional information. In various different modalities, DO 144 can be implemented in any desired shape or size, and can be displayed at any appropriate location. In addition, any desired number of different DOs are also contemplated, including the possibility of multiple DOs on the display screen at any given time.

[0058] In the mode of FIGURE IB, display screen 138 supports synchronized DOs that operate to

Cm provide information that is related (synchronized) to the video data currently displayed on the display screen 138. For example, DO 144 can be used to display financial information of specific relevance to the viewer (for example, your investment portfolio) during a television program regarding economic news or investment topics. In another example, DO 144 can be used during a televised car race to display relevant information or statistics regarding drivers of specific race cars, race cars, or car races in general.

[0059] Referring now to FIGURE 2A, a block diagram is shown for the content source 114 of an embodiment of FIGURE IA, in accordance with the present disclosure. In the embodiment of FIGURE 2A, content source 114 may include, but is not limited to, a central processing unit (CPU) 212, source memory 220, and input / output interfaces 224 (I / O interfaces). O). In alternate embodiments, the content source 114 may be implemented using components and configurations in addition to, or in place of, those components and configurations discussed in conjunction with the embodiment of FIGURE 2A. Additionally, content source 114 may alternatively be implemented as any other type of electronic device or desired entity.

[0060] In the embodiment of FIGURE 2A, CPU 212 may be implemented to include any appropriate and compatible microprocessor device that preferably executes software instructions to accordingly control and manage the operation of content source 114. In the embodiment of FIGURE 2A, source memory 220 can be implemented to include any desired combination of storage devices, including, but not limited to, read-only memory (ROM), random access memory (RAM), and various types of nonvolatile memory, such as floppy disks or hard drives. The content and functionality of the source memory 220 is discussed further below in conjunction with FIGURES 3 and 4.

[0061] In the embodiment of FIGURE 2A, I / O interfaces 224 may include one or more input and / or output interfaces to receive and / or transmit any types of information required by content source 114. For example, in the FIGURE 2A mode, content source 114 may use I / O interfaces 224 to communicate with other entities in electronic system 110 (FIGURE IA). Furthermore, a system user can use I / O interfaces 224 to communicate with content source 114 using any appropriate and effective technique. Further details regarding content source 114 are further discussed below together

<img file="MX370229B_D0005.tif" />

with FIGURES 2B, 3-4.

[00S2] FIGURE 2B illustrates an apparatus 250 that provides exemplary information, which is for example used by content source 1.14 to provide an image in which a plurality of symbols are encoded, including one or a combination of a pattern of start, one or more parameters used to retrieve values ​​of the plurality of symbols, and other digital data (eg, metadata, application data, program guide data, subtitle data, etc.). Generally speaking, a single content provider can provide multiple programs (eg Programs A and B) on one or more transport streams. For example, the audio, video, and subtitle data for Program A is provided to a 256A encoder while the audio, video, and subtitle data for Program B is provided to a 256B encoder. A transport stream multiplexer 258 receives the outputs from the encoders 256A, 256B and provides an output that can be distributed via a physical channel medium such as terrestrial, cable, or satellite broadcast. A communication interface 260 (eg, a terrestrial broadcast transmitter) distributes the output of the transport stream multiplexer 258 via the physical channel medium.

[OOS3J The information providing apparatus 2 or 0 further includes a digital data generator 252 and a digital data inserter 254. Digital data generator 1102 generates digital data to be integrated into the video portions of Program A.

[0064] The digital data inserter 254 integrates the digital data generated in the Program A video portions. In certain embodiments, the digital data inserter 254 also integrates a predetermined starting pattern which indicates that the digital data has been integrated. in the video portion. In addition, the digital data inserter 254 optionally integrates parameters to be used to retrieve the integrated digital data (eg, one or more luminance values ​​used to encode symbols, one or more cut points, etc.) for a receiving apparatus 550 (as illustrated in FIGURE 5B, for example).

[0065] In certain embodiments, the digital data inserter 254 encodes the generated digital data within luminance values ​​on one or more lines of active video (eg, line 1 and optionally line 2). Digital data inserter 252 encodes each of the digital data in a different frame, or each of one or more lines, of the video. As described above, digital data can be repeated for a predetermined number of frames.

<img file="MX370229B_D0006.tif" />

[0066] The digital data inserter 254 for better robustness optionally repeats the encoding of the metadata generated on line 2, due to errors that may be introduced in encoding or re-encoding. Due to the nature of video encoding, it has been found that the integrity of .metadata in the. line 1 if the same data is repeated on line 2. Also, other lines such as the last lines 1 or 2 can be used to repeat the same data.

[0067] Referring now to FIGURE 3, a block diagram for a source memory 220 of the embodiment of FIGURE 2Ά is shown, in accordance with the present disclosure. In the FIGURE 3 embodiment, source memory 220 includes, but is not limited to, one or more 15 source applications 312, video data 316, audio data 318, an encoder 320, metadata 322, a manager 324 metadata, and varied 328 information. In alternate modes, the source memory 220 may include components in addition to, or instead of, those components discussed in conjunction with the mode of FIGURE 3.

[0068] In the embodiment of FIGURE 3, font application 312 may include program instructions that are preferably executed by CPU 212 (FIGURE 2A) to perform various functions and operations for content source 114 25, including for example control and / or implementation of the apparatus 250 that provides information. The particular nature and functionality of the preferred font application 312 vary depending on factors such as the specific type and particular functionality of the corresponding content source 114. In. In the embodiment of FIGURE 3, the video data 316 may include any appropriate information or data to be displayed on, or to be processed within, television 122 (FIGURE IA). Similarly, audio data 318 can include any information or data appropriate for playback on television 122 (FIGURE IA).

[0069] In the mode of FIGURE 3, encoder 320 is configured to convert video data 316 and audio data 318 into a compressed distribution multiplex for distribution to television 122. In the mode of FIGURE 3 , he. Metadata manager 324 coordinates and manages various functions to create the metadata 322 (or other digital data), and integrate the metadata 322 as an integral part of the video data 316, in accordance with the present disclosure. The varied information 328 may include any additional information for use by the content source 114.

[0070] In the embodiment of FIGURE 3, the present disclosure is described and discussed as if it were primarily implemented as software. However, in alternate embodiments, some or all of the functions of the present disclosure may be performed by appropriate electronic hardware circuits that are configured to perform various functions that are equivalent to those functions of the software modules discussed herein. Further details regarding the functionality of the metadata manager 324 and metadata 322 are discussed further below.

[0071] Referring now to FIGURE 4, a block diagram of the metadata 322 of FIGURE 3 is shown, in accordance with one embodiment of the present disclosure. In the FIGURE 4 embodiment, metadata 322 may include, but is not limited to, trigger data 4'12, DO content 416, sync (sync) data 418, content identification (ID) data 420, indicator data 422, and miscellaneous information 424. In alternate modalities, metadata 322 can be implemented using various components and functionalities in addition to, or instead of, those components and functionalities discussed in conjunction with the modality of FIGURE 4.

[0072] In the embodiment of FIGURE 4, trigger data 412 may include any type of information for DO 144 related control processes (FIGURE IB). For example, trigger data 412 may include, but is not limited to, data defining DO 14 4 regarding visual appearance and behavior, information presented by a DO (such as read values), graphic DO states (such as colors, levels, or settings), and optimal placement of DO, shape, size, and display times. In certain embodiments, trigger data 412 contains one or more triggers that perform various signaling functions related to synchronization in support of interactive services, as defined in ATSC Candidate Standard A / 105: 2014, as referenced in previous.

[0073] In a modality, the trigger can be considered to include three parts, two that are required and the third that is optional: &lt;domain ñame part&gt; / &lt;directory path&gt; [? &lt;paratneters&gt;]. The &lt;domain yam part&gt; refers to a registered Internet domain name. The cdirectory path&gt; is an arbitrary character string that identifies a directory path under the control and management of the entity that owns the rights to the identified domain name. In the TDO model, the combination of &lt;domain ñame part&gt; and &lt;directory path&gt; should uniquely identify a TPT that can be processed by a receiver to add interactivity to the associated content. In the direct execution model, the combination of &lt;domain ñame part&gt; and &lt;directory path&gt; should uniquely identify the DO to be submitted. The &lt;parameters&gt; portion of the Activator is optional. When present, it can carry one or more parameters associated with the trigger. An exemplary trigger is xbc. tv / el2.

[0074] The trigger is a data object, which is optionally linked to a particular content item or segment (eg, a TV show) that references a specific TDO instance, by using a file name or identifier of an object that has already been or will be downloaded. Certain TDOs will only make sense along with certain content. An example is a TDO that collects response data from the viewer, such as a vote on a game show or contest.

[0075] In the embodiment of FIGURE 4, DO content 416 may include any content data to be displayed in DO 144. In certain embodiments, DO content 416 may alternatively be obtained from sources or entities other than metadata 322 DOs may include discrete areas shown on Television 122 to provide any desired information. Further details regarding DO are also provided above in conjunction with FIGURE IB. In the embodiment of FIGURE 1Ά, the electronic system 110 advantageously supports synchronized DOs that provide information that is related (synchronized) to the main content data currently displayed on television 122. The electronic system 110 also provides certain types of metadata (eg triggers, Parameter Table

Cm from TDO, etc.) to television 122, in order to successfully support synchronized DOs (for example, activated declarative objects (TDO)).

[0076] A TDO is a downloadable software object created by a content provider, content creator, or other type of service providers, including declarative content (eg, text, graphics, descriptive margin, instruction sets, and / or audio) whose function is tied in some way to the accompanying content. A modality of the TDO is described in the ATSC Candidate Standard A / 105: 2014. However, the TDO is not limited to the structure described in the ATSC Candidate Standard since many attributes defined therein as part of a TDO may be placed in a trigger or vice versa or not presented at all depending on the function and trigger of a particular TDO.

[0077] TDO is generally considered as declarative content to distinguish it from executable content such as a Java applet or an application running on an operating system platform. Although TDO is usually considered a declarative object, a TDO player (for example, the DO Engine) supports a scripting language that is an object-oriented programming language (for example, JavaScript). In the examples shown herein, TDOs are received from a content or service provider, for example server 130, before they are executed so that the TOO is available when needed. On the other hand, an explicit trigger signal may not be required and a TDO may self-trigger or trigger by some different action received from a trigger signal. ' Various standard bodies can define behaviors, appearances, trigger actions, and transport methods associated with content and metadata for a TDO. In addition, requirements regarding the timing accuracy of audio / video related TDO behaviors can be defined by standard bodies.

[0078] The TPT contains metadata about a TDO of a content segment and defines one or more events for the TDO. TDO events can be triggered based on a stream sync of the content being played or by referring to one or more events contained in one or more triggers. For example, one or more parameters associated with a trigger may be provided to the receiving apparatus 550 in the TPT.

[0079] Although a trigger indicates that the time is right for the TDO to perform a certain action, a series of synchronized actions can be carried out without a trigger, for example when using the TPT. The TPT, or optionally a separate Activation Message Table (AMT), optionally provides the timing information for various interactive media timing events. Each interactive content item has a timeline for completion; an instant of 5 time on this schedule is called media time. For example, a 30-minute program may have an interactive event in ten minutes, 41 seconds, and 2 frames of media time, from the beginning of the program, or media time of -10: 41 + 02. The TPT may include an entry 10 indicating the details of the event that will occur at the time 10: 41 + 02. Once the receiving apparatus 550 determines the current timing relative to the start of the program, it can use the TPT, and optionally the ΆΜΤ, to carry out all subsequent events.

[0080] In the embodiment of FIGURE 4, the sync (4) data 418 may include any appropriate means to allow television 122 to detect metadata 322 while it is integrated into video data 316. In certain embodiments, the sync data 418 may include a predefined identification pattern (eg, a default startup pattern) indicating the presence, and in one embodiment, a specific location, of the metadata 322 within the data 316 Of video. Metadata can be encoded within 25 video using modulation of the value of

<img file="MX370229B_D0007.tif" />

video luminance (brightness). For example, on a video line, each set of a predetermined number of pixels (for example, 8) may correspond to a symbol, where the luminance value of each pixel is set to one of 5 four values. In this case, each symbol carries two bits of information. In other modes, each set of pixels can be set to one of two levels. In that case, each set would carry one bit of information.

[0081] In one embodiment, a predetermined number of symbols (eg, 8 or 16) is used to define a predetermined starting pattern that is used to indicate whether a video frame is marked. For example, the first eight symbols can be set to a fixed pattern, [3, 3, 0 ,. 0, 2, 1, 3, 0], to allow a detector to quickly identify whether the video includes a watermark or not. When each symbol corresponds to 1-bit, the fixed pattern can be [1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0] and, in certain modalities, the number of symbols is increased (for example, to 16). Additionally, different boot patterns can be used for different protocol versions so that reverse compatibility can be achieved by ensuring that version 1.0 implementations discard any data that does not include the version 1.0 boot pattern. [0082] In the embodiment of FIGURE 4, the content ID data 420 may include any appropriate information

<img file="MX370229B_D0008.tif" />

to identify the specific content of a given corresponding program. In one embodiment, the content ID data 420 may include a number, International Standard Audio-Visual Number (ISAN) as an identifier. In another embodiment, the content ID data 420 may include an Entertainment Industry Data Registration (EIDR) code and / or a media time. For example, content ID data 420 may include a content ID message that is designed to carry a 12-byte EIDR code and a 2-byres media time. The exemplary bitstream syntax of the Content ID message is as follows:

<td>Syntax</td><td>No. d @ Bits</td><td>Foojato</td>

<td>Content id messageO {</td><td></td><td></td>

<td>tatol®__id</td><td> 8</td><td>0x01</td>

<td>Tatole length</td><td> 8</td><td>uimsbf</td>

<td>niDR</td><td> 96</td><td>uimsbf</td>

<td></td><td> 16</td><td>uimsbf</td>

<td>CRC__32</td><td> 32</td><td>uimsbf</td>

<td> }</td><td></td><td></td>

[0083] tablejid - Set to value 0x01. Identify the data to follow as a content_id_message ().

[0084] table__length - Indicates the number of bytes to follow at the end of the CRC. In this case the value is 18.

[0085] EIDR - A 96-bit value intended to carry the. Value of the Entertainment Industry Data Record (EIDR) code for this content item.

0

[0086] E. @ dia__tixn @ A 16-bit number representing the media time within content in seconds, where the value zero indicates the first second of the content item.

[0087] CRC__32 - A 32-bit CRC checksum over the entire message, up to but not including the CRC 32 field itself. An exemplary generation polynomial is 1 + x + x² + x⁴ + x⁵ + x⁷ + x ° + κ^±0 + x¹¹ + x¹² + xⁱ⁶ + x^z2 + x²³ + x²⁶.

[0088] In one embodiment, the content ID message may further, or alternatively, include an Ad-ID field for commercial material. The AD-Id field is a 96-bit field that represents the Ad-ID code associated with the content.

[0088] In the embodiment of FIGURE 4, indicator data 422 may include any type of information required that television 122 uses to locate and obtain additional information (such as DO content or trigger data) for use in production of the synchronized DOs 144. For example, indicator data 422 may include, but is not limited to, a URL identifying an Internet location where further information pertaining to the currently displayed video data 316 may be found. The URL could represent a website on the 13Q server (FIGURE 1A) or somewhere else that provides more information about an advertised product, a URL from a

<img file="MX370229B_D0009.tif" />

home page of an episode or series, a website where a viewer could register for a service or vote on a show, etc.

[0090] In the embodiment of FIGURE 4, the miscellaneous information 424 may include any additional information for use by television 122. For example, in certain embodiments, the miscellaneous information 424 may include one or more data streams or executable programs. . In one embodiment, the miscellaneous information 424 includes a frame count message. The purpose of the frame count message is to provide finer granularity to the synchronization provided in the media_time field of the Content ID message, and to indicate the original frame rate of the content (at the time the watermark was applied). The exemplary bitstream syntax of the frame count message is as follows:

<td>Syntax</td><td>No. of Bits</td><td>Format</td>

<td>Phrase count message () {</td><td></td><td></td>

<td>table__id</td><td> 8</td><td>0x02</td>

<td>fcable__length</td><td>g</td><td>Uimsbf</td>

<td>criginalfraserate</td><td>b</td><td>Uimsbf</td>

<td>Frarne</td><td> 8</td><td>uimsbf</td>

<td>CRC_32</td><td> 32</td><td>uimsbf</td>

<td> }</td><td></td><td></td>

[0091] tablejid - Set to value 0x02. Identify the data to follow as a frame_count_message ().

[0092] bable length - Indicates the number of bytes to follow. In this case the value was set to 6.

[0093] original rat frams - An 8-bit unsigned integer indicating the frame rate, in frames per second, of the original content at the time the watermark was applied. The value is set to 24 for animated content and 30 for other content types.

[0094] phrasing - An 8-bit unsigned integer indicating the frame number within the period of one second identified by media_time. The account is zero based.

[0095] CRCJ32 - A 32-bit CRC checksum on the entire message, up to but not including the CRC_32 field itself. An exemplary generation polynomial is 1 + X + X² + X⁴ + χ³ + χ⁷ + x⁸ + X¹⁰ 4x¹¹ + X¹²· + X¹⁶ + X²² 4- x²³ + X²⁶·

[0096] Further details regarding the creation, distribution, and use of 322 metadata are further discussed below.

[0097] Referring now to FIGURE 5A, a block diagram for a television 122 (TV) in the embodiment of FIGURE 1A is shown in accordance with the present disclosure. In the embodiment of FIGURE 5A, TV 122 may include, but is not limited to. a central processing unit 512 (CPU), a display screen 138, a TV memory 520, and input / output interfaces 524 (I / O interfaces). In alternate embodiments, TV 122 may be implemented using components and configurations in addition to, or in place of, those components and configurations discussed in conjunction with the embodiment of FIGURE 5A. Additionally, TV 122 may alternatively be implemented as any other desired type of electronic device or entity.

[0098] In the embodiment of FIGURE 5A, CPU 512 may be implemented to include any appropriate and compatible microprocessor device that preferably executes software instructions to thereby control and manage the operation of TV 122. The display screen 138 of FIGURE 5 can include any type of effective display screen technology including an organic light emitting diode display screen or a liquid crystal display screen with an appropriate display to display various information to a device user. In the embodiment of FIGURE 5A, TV memory 520 can be implemented to include any desired combination of storage devices, including, but not limited to, read-only memory (ROM), random access memory (RAM), and various types of nonvolatile memory, such as floppy disks or hard drives. The content and functionality of TV memory 520 is discussed further below in conjunction with FIGURE 6.

In the embodiment of FIGURE 5A, interfaces 524

[0099] I / O may include one or more input and / or output interfaces to receive and / or transmit any types of information required for TV 122. For example, in the embodiment of FIGURE 5A, TV 122 may using the I / O interfaces 524 to communicate with other entities in the electronic system 110 (FIGURE IA). Furthermore, a system user can use the I / O interfaces 524 to communicate with the TV 122 using any appropriate and effective techniques. Further details regarding TV 122 are discussed further below. [00100] FIGURE 5B is a block diagram of a receiving apparatus 550 that is incorporated into TV 122 in accordance with one embodiment. Receiving apparatus 550 is a digital television receiving device that can be incorporated into TV 122 or converter-decoder box 118. The receiving apparatus 550 includes a tuner / demodulator 552, which receives content from one or more content providers such as a terrestrial broadcast or a cable television transmission. The receiving apparatus 550 may also, or alternatively, receive content from a satellite broadcast. Tuner / demodulator 552 receives a packet stream (PS) such as a transport stream (TS) or IP packet stream, which is demultiplexed by demultiplexer 554 into audio and video (A / V) streams. Exemplary IP packet streams are described in the ATSC Mobile DVT standard, ATSC-M / H (A / 153) and the Enhanced Multicast Multimedia Broadcast (eMBMS) standard, which are incorporated herein by reference in their entirety. . The audio is decoded by an audio decoder 556 and the video is decoded by a video decoder 558. Also, uncompressed A / V data can be received through an uncompressed A / V interface (for example, an HDMI interface) that can be used selectively. The uncompressed A / V data can be received from a set-top box, digital video recorder, DVD player, or any other consumer electronic device connected to the receiving apparatus 550 via the uncompressed A / V interface.

[00101] The TS may include ancillary information such as one or more of subtitle data, TDO, triggers, TPT, content identifiers, and other metadata. One or more of the A / V content and / or auxiliary information may also be received via the communication network 134 (eg. Internet) and a network interface 560. In certain embodiments, ancillary information is integrated such as one or a combination of triggers, content identifiers, subtitle data, or other metadata, or is otherwise inserted into the video portion of the content of

Α / \ symbols.

A CPU 562 extracts auxiliary information from the video portion of the A / V content and. performs one or more processes based on the extracted auxiliary information.

[00102] A storage unit 564 is provided to store NRT, recorded content, or content delivered to the Internet such as Internet Protocol Television (IPTV). Recorded or stored content can be played back by demultiplexing the content stored in the storage unit 564 by demultiplexer 554 in a similar way to other content sources. Storage unit 564 can also store one or more TDOs, triggers, and TPTs acquired by receiving apparatus 550.

[00103] The receiving apparatus 550 generally operates under the control of at least one processor, such as CPU 562, which is coupled to a working memory 566, program memory 568, and a graphics subsystem 574 by one or more buses (for example, bus 572). CPU 562 can receive closed caption data from demultiplexer 554 as well as any other information such as TDO and EPG prompts used to render graphics, and pass the information to graphics subsystem 574. The graphics produced by the graphics subsystem 574 are combined with video images by the composer and the video interface 568 to produce an output suitable for display on a video display screen. In one mode, the CPU

562 retrieves digital data such as the auxiliary information encoded in the video portion of the A / V content by receiving an image from the video decoder 558.

[00104] In addition, CPU 562 operates to perform functions of receiving apparatus 550 including processing related triggers, TDO, TPT, browser operations, metadata, digital data extraction integrated into the image, etc. Browser operations include accessing a service specified by a URL provided by the TDO or trigger. The CPU 550 further operates to execute objects of a script (control objects) contained in the TDO, its trigger (yes, etc.), using for example DO Engine 584 illustrated in FIGURE 5C.

[00105] Although not illustrated in FIGURE 5B, the CPU 562 may be coupled to any or a combination of the resources of the receiving apparatus 550 to centralize control of one or more functions. In one embodiment, CPU 562 also operates to monitor control of receiving apparatus 550, which includes tuner / demodulator 552 and other television resources.

[00106] A more central view of the processor of the receiving apparatus 550 is illustrated in FIGURE 5C. Memory and storage 564, 566, 568 are collectively represented as memory 582. In addition, a 580 processor includes one or more

<img file="MX370229B_D0010.tif" />

processing units such as CPU 562. Similarly, the various demodulators, decoders, etc., that initially process digital television signals are collectively represented as television receiver / tuner 590. The receiving apparatus 550 further includes a remote controller 598 that communicates with a remote controller receiver interface 596. Furthermore, the display screen 594 is connected to a display screen interface 592, including for example the uncompressed A / V interface and / or compositor 570, and is either an integral display screen to the receiving apparatus 550 as in a television set or a connected display screen device as in the case where the receiving apparatus 550 is integrated into the set-top box 118.

[00107] Memory 582 contains various modules and functional program data. Memory 582 stores the data used by the receiving apparatus 550. The memory 582 within the receiving apparatus 550 can be implemented 20 using one form of disk storage as well as other forms of storage such as non-transient storage devices including, for example, network memory devices, magnetic storage elements, elements magneto-optical storage, 25 flash memory, central memory and / or other technologies

<img file="MX370229B_D0011.tif" />

non-volatile storage. The term non-transient is a limitation of the medium itself (i.e. tangible, not a signal) as opposed to a limitation on persistence of data storage (eg RAM vs. ROM).

[00108] When a TDO 586 is received, the TDO 586 is stored in memory 582. The execution of). TDO is performed using a DO 584 Engine. When the TDO is run by the DO 584 Engine, it presents auxiliary content based on one or more triggers associated with the TDO. Memory 582 also stores a TPT 5 68, which in one mode defines one or more parameters for each trigger associated with the TDO.

[00109] Referring now to FIGURE 6, a block diagram for a TV memory 520 of the embodiment of FIGURE 5A is shown in accordance with the present disclosure. In the FIGURE 6 embodiment, TV memory 520 includes, but is not limited to, one or more TV applications 612, video data 316, audio data 318, a detection module 620, and extraction module 622 , a metadata module 624, metadata 322, and miscellaneous information 628. In alternate modes, TV memory 520 may include components in addition to, or instead of, those components discussed in conjunction with the mode of FIGURE 6.

[00110] In the embodiment of FIGURE 6, the TV application 312 may include program instructions that

<img file="MX370229B_D0012.tif" />

Preference is executed by CPU 512 (FIGURE 5A) to perform various functions and operations for TV 122. The particular nature and functionality of the preferred TV application 612 varies depending on factors such as the specific type and particular functionality of the TV. 122 corresponding. In the embodiment of FIGURE 6, the video data 316 may include any information or data appropriate for display on television 122 (FIGURE 1A). Similarly, audio data 318 can include any information or data appropriate for playback on television 122 (FIGURE 1A).

[00111] In the embodiment of FIGURE 6, the detection module 620 can be used by the TV 122 to detect and locate the metadata 322 that has been integrated into the video data 316, as discussed above. In the embodiment of FIGURE 6, the extraction module 620 can be used by the TV 122 to remove the detected metadata 322 from the video data 316. In the embodiment of FIGURE 3, the metadata module 624 coordinates and manages various functions to process the extracted metadata 322 to effectively support synchronized DG 144 (FIGURE IB) or other TV applications, in accordance with the present disclosure. The varied information 628 may include any additional information for use by TV 122.

[00112] In the embodiment of FIGURE 6, the 'present disclosure is described and discussed' as if it were primarily implemented as software. However, in alternate embodiments, some or all of the functions of the present disclosure may be performed by appropriate electronic hardware circuits that are configured to perform various functions that are equivalent to those functions of the software modules discussed herein. Further details regarding the functionality of the metadata module 324 and the 10 metadata 322 are discussed further in the following, together with FIGURES 7 and 8.

[00113] The modalities of the present description integrate the metadata 322, and / or other digital data, such as a watermark using luminance values ​​of a video frame. Luminance values ​​are bound within the 15th decimal range [16, 235]. The luminance value 16 corresponds to black and the luminance value 235 corresponds to white, as defined in Recommendation ITU-R BT.709, which is incorporated herein by reference in its entirety. A watermark data symbol is encoded in pixels M 20 (where M is typically 6, 8, or 16). Each symbol encodes one or more bits of data. When one-bit-per-symbol encoding is used, the symbol values ​​can be either zero or 100%, and a threshold value of 50% luminance is used to distinguish '1' bits from '0' bits. When using two-bit encoding per symbol, the symbol values ​​may be zero,

33.33%, 66.67%, or 100% luminance., And threshold values ​​of 16.67% can be used

50%, and 83.33%. Alternatively, lower luminance values ​​can be used to reduce visibility. A balance can be made between robustness against heavy compression or video transcoding against visibility, by selecting margins and luminance values.

[00114] Examples of one bit and two bit encoding per symbol are illustrated in FIGURES 12A and 12B. In FIGURE 12A, the symbols representing the watermark data use two luminance values: 0 and 60. In FIGURE 12B, the symbols representing the watermark data use four different equally spaced luminance values: 16 , 89, 162, and 235 (decimal). The luminance values ​​are shifted down 16 units so that the first luminance value is 0 in certain modes.

Threshold values ​​for decoding are shown in a 2r

FIGURE 12A and 12B. For example, in FIGURE 12A, a symbol value of 0 is detected if the luminance is less than or equal to 30. The value 1 is detected if the luminance is greater than 30. In FIGURE 12B, a symbol value of 0 it detects if the luminance is less than or equal to 42, the value 1 is detected if the luminance is in the range of 43 to 127, the value 2 is detected if the luminance is in the range of 128 2, and the value 3 detects if the luminance is

<img file="MX370229B_D0013.tif" />

at 213 or above. Although in most cases televisions do not display the few top or bottom video lines, a problem that arises from using such a luminance coding scheme is that the integrated 5-watermark may be visible to a viewer if the portion of the video frame occupied by the watermark is displayed on TV 122.

[00115] To reduce the visibility of the watermark, embodiments of the present disclosure use one or a combination of different methods including (1) decreasing the data capacity of the watermark; (2) use a luminance value below black; and (3) decrease the rate that the watermark changes (for example, once per second instead of per frame).

[00116] In certain modalities, the metadata is integrated in line 1 of the video data. The video on line 1 consists of N-encoded pixels (for HD or UHD content, usually 1280, 1920, or 3840). As noted above, a watermark data symbol is encoded in pixels M (where M is typically 6, 8, or 16). Furthermore, in one modality, the same metadata is also integrated in line 2 for better robustness due to errors that may be entered in the encoding or re-encoding. Due to the nature of video encoding, it has been found that the integrity of Ips metadata on line 1 is improved if the same data is repeated on line 2.

[00117] To reduce the visibility of embedded metadata, in one modality, the data capacity of the watermark can be decreased. For example, 60 bytes of data can be encoded per line, when the number of horizontal pixels per line is 1920, the number of pixels per symbol is 8, and the number of bits encoded per symbol is 2. However, a margin should be used of longer luminance values ​​in order to encode 2 bits per symbol. To decrease the visibility of the watermark, the data capacity of the watermark can be reduced so that the maximum value of luminance required to identify a symbol value, for example the value 235, is decreased. For example, they can be used Luminance values ​​16 and 89, rather than 16, 89, 162, and 235, to encode the watermark when the number of bits encoded per symbol is reduced to 2, resulting in 30 bytes of data encoding per line.

[00118] In one embodiment, using a luminance value below black decreases the visibility of the watermark. Video standards specify that luminance values ​​range from 16 (black) to 235 (white) when encoded at 8 bits. A luminance value of 0 (or any other value below 16) can survive transcoding. Using a minimum luminance value of instead of 16 allows a reduction in the maximum luminance value needed to encode the watermark and have greater robustness. For example, for 2-bit per symbol encoding, the range from 16 to 235 can be reduced from 0 to 219, and for 1-bit per symbol encoding, the range from 16 to 89 can be reduced from 0 to 73 without loss. of solidity. In one embodiment, the luminance range is set from 0 to 42 for 1 bit per symbol encoding. The luminance value 42 is a level of dark gray that is almost imperceptible. However, any luminance value range can be set in which the range starts at a value below 16 in certain modes. In certain embodiments, luminance values ​​above 235 can be used to increase the range of luminance values ​​used to encode or shift the range of luminance values. at higher values.

[00119] In one embodiment, the rate the watermark changes is decreased from frame to frame to reduce the visibility of the watermark embedded in the video data 316. For example, the same watermark can be embedded in a predetermined number of frames, or for a predetermined amount of time (for example, 1 second) before it is changed, rather than once per frame. Although this reduces the rate at which data is transmitted, decreasing the rate of change reduces the

<img file="MX370229B_D0014.tif" />

Possible distraction to a viewer that may result from frequently changing pixel luminance values ​​when the watermark is within a visible area of ​​the display screen.

[00120] The number of horizontal pixels representing a symbol varies depending on the horizontal resolution. In one embodiment, 16 pixels per symbol are used for the 3840 horizontal resolution to allow the video watermark to be preserved during downward 4K to 2K resolution.

[00121] Referring now to FIGURES 7A and 7B, digital data diagrams (eg, metadata 322) embedded in video data 316 are shown in accordance with two different embodiments of the present disclosure. FIGURES 7A and 7B present a front view of a TV 122 display screen 138 (FIGURE IA). The modalities of FIGURES 7A and 7b are presented for illustration purposes, and in alternate modalities, metadata 322 can be integrated using techniques and configurations in addition to, or instead of, certain of those techniques and configurations discussed in conjunction with the modalities of FIGURE 7A and 7B. For example, in some embodiments, digital data may be placed at the bottom of the display screen.

[00122] In the embodiment of FIGURE 7A, display screen 138 includes a main screen region that typically displays video data 316 provided by content source 114 (FIGURE 1A). In the modality of the

FIGURE 7A, the video data 316 displayed on the display screen '5 138 also includes the digital data that is located in a discrete area of ​​the display screen 138. In various different embodiments, the digital data can be implemented in any desired shape or size, and can be displayed at any appropriate location 10 on the display screen 138. For illustration purposes, the metadata 322 in FIGURE 7A is represented as a thin, framed line. However, any configuration or appearance effective for implementing 322 metadata is also contemplated.

[00123] In the embodiment of FIGURE 7A, the digital data may be encoded to represent the required information in any effective way (see FIGURE 4).

For example, in certain embodiments, digital data may be formatted as one or more horizontal lines of digital video information, positioned at or near the region of the vertical blanking interval of the video signal (VBI). Because digital television is frequently encoded at 1280 to 1920 horizontal pixels per scan line, the VBI setting for metadata 322 in FIGURE 7A can provide a substantial amount of digital information to TV 122.

Thus, the present description supports a method of camouflaging digital data in video data 316, so that a portion of active video (potentially visible to the viewer) is used to transport the digital data. Additionally, the present description includes standardizing a format for encoding digital data so that they survive video compression and decompression. The present disclosure further supports integrating digital data into the video image so that the digital data can be retrieved (detected, extracted, and processed by TV 122) in a standardized manner, without excessive CPU overhead. The implementation and use of digital data is further discussed in the following along with FIGURES 8A ~ 8C and 9-11.

[00125] In the embodiment of FIGURE 7B, display screen 138 includes a main display region that typically displays video data 316 provided by content source 114 (FIGURE IA). In the embodiment of FIGURE 7B, the video data 316 displayed on the display screen 138 also includes the embedded digital data (eg, metadata 322) which is preferably located in a discrete area of ​​the display screen 138. For example, digital data can be located in any region of the display screen that is outside the video display area (for example, a movie that is displayed at a different aspect ratio). In various, different embodiments, digital data can be implemented in any desired shape or size, and can be displayed at any appropriate location on display screen 138. For purposes of illustration, the digital data in FIGURE 7B is represented as a small woven rectangle. However, any configuration or appearance effective for implementing the embedded digital data is also contemplated.

[00126] In the embodiment of FIGURE 7B, the digital data may be encoded to represent any required information in any effective way (see FIGURE 4). For example, in certain embodiments, digital data may be formatted using conventional or improved barcode technologies. In other words, digital data can be effectively formatted as a two-dimensional video barcode that integrates into a corner or edge of the displayed video data 316. Additionally, the barcode or other formats of the digital data may be displayed as a part of a small graphic logo icon known as a glitch. Furthermore, in various other modalities, digital data can be encoded or displayed using any other effective technique. Such encoding of the metadata 322 could represent a substantial amount of information, and could be quite small and dense, as the metadata 322 was read by the TV 122 as it processed the video data 316 in the video memory. Where printed barcodes are optimized for reading by laser scanners, the type of video barcode used for 322 metadata is integrated into a digital video signal, which is processed directly by TV 122 (such as pixel luminance or chrominance samples).

[00127] In certain modalities, quantization errors in video compression could possibly erase a video barcode (so a barcode that is presented within a fast moving, difficult to compress video sequence may not survive). However, that concern is mitigated if the barcode is left on the screen for any amount of time (a few seconds). The resulting barcode image may not need to be displayed in high contrast (black lines on a white background) as the TV 122 will be able to extract the information using a filtering mechanism. In this way, the barcode could be coded with various shades of gray (as long as there is enough contrast for reliable extraction). By

That example. The barcode can be displayed using a luminance value below 16, using 1-bit encoding per symbol, and / or reduced rates of change, as described above.

[00128] As discussed in 1c · above, digital data may be displayed in conjunction with a small graphic logo (glitch) icon, such as a caption or margin, or may be placed on one or more of the end edges of the image ( since these are usually cut before showing, and are less troublesome in any case). The bits of digital data can be spatially spread over the area of ​​the video frame if the pattern of its location was known by TV 122 beforehand. Even a small amount of digital data such as content ID data 420 or indicator data 422 in FIGURE 4 can go a long way in improving the user experience as this information can be expanded through interaction with a web server 130 (see FIGURE 1A) for additional required information, including, but not limited to, metadata 322 or content data.

[00129] With reference. now to FIGURES 9A-8C, a flow chart of the method steps for distributing the metadata 322, and / or other digital data, integrated into the video data 316, is shown in accordance with one embodiment of the present disclosure. The example of FIGURE is presented for purposes of illustration, and in alternate embodiments, the present disclosure may use different steps and sequences of certain of those steps and sequences discussed in conjunction with the embodiment of FIGURE 8. [00130] In the embodiment of FIGURE 8A, at step 812, content source 114 or other appropriate entity initially produces A / V content data that typically includes video data 316 and audio data 318. In step 814, the content source 114 or other appropriate entity then creates the metadata 322 to support various advanced interactive features on the television device 122, such as displaying one or more synchronized DOs 14 4. At step 816, the content source 114 or other appropriate entity inserts the metadata 322 into the video data 316.

[00131] In step 818, the content source 114 or other appropriate entity compresses the audio data 318 and the video data 316 (including the embedded metadata 322) to create a compressed distribution multiplex. The process of FIGURE 8A then proceeds to step 822 of FIGURE 8B by connecting the letter A.

[00132] In step 822 of FIGURE 8B, a receiving apparatus 550 receives and demultiplexes the distribution multiplex distributed by the content source 114 to produce compressed audio and video data. In the

<img file="MX370229B_D0015.tif" />

Step 824, a decoding device of the receiving apparatus 550 then decompresses the compressed audio and video data to produce uncompressed audio data 318 and uncompressed video data 316 (including the embedded metadata 322). At step 826, the receiving apparatus 550 formats the audio data 318 and the video data 316 to deliver to the television display screen 138 122. The process of FIGURE 8B then proceeds to step 828 of FIGURE 8C through the letter B connection.

[00133] In step 828 of FIGURE 8C, the television 122 or other appropriate entity receives or further processes the uncompressed audio data 318 and uncompressed video data 316 (including the integrated metadata 322). At step 830, the television detection module 620 scans the video data 316 to detect the integrated metadata 322 using any effective technique (eg, detecting the presence of the start pattern).

[00134] At step 832, the television extraction module 622 122 extracts the located metadata 322 from the video data 316. In one embodiment, television 122 determines the watermark symbol values ​​representing the embedded metadata based on the luminance values ​​in pixels of a first portion (eg, first line) of a video frame of the content. For example, when eight pixels make up a symbol, television 122 averages the luminance values ​​across the eight pixels making up the symbol and determines if the symbol is a 1 or 0 based on luminance threshold encoding values. For example, for 1-bit encoding per symbol, television 122 determines that the symbol is 0 when the detected average luminance is less than or equal to a predetermined percentage (eg 50%) of a coding range, and the symbol is 1 when the detected average luminance is greater than the predetermined percentage of the coding range.

[00135] Finally, in step 834, the metadata module 624 processes the extracted metadata 322 to successfully support appropriate advanced features, such as displaying one or more synchronized DOs 144 on the television display screen 138. The processes of FIGURE 8C may then end.

[00136] Regarding advanced features. In one embodiment, television 122 could recognize a channel change (or content change) either by detecting that the content is no longer Marked (for example, the watermark is no longer detected), or by detecting a Content Frame marking (for example, a watermark) at which the EIDR value changed. In one embodiment, the ID content is directly identified by the EIDR value in the Content ID Message. In another embodiment, a URL from a remote server or any other content information is provided as embedded metadata, such as a watermark. In another embodiment, two data elements are included in the embedded metadata to identify the media time, the media time in full seconds is specified in the content ID message while the media time in frames is specified in the message frame count, so the timing accuracy is at the frame level. Additionally, the built-in metadata can be used to provide event triggers, which are time sensitive, in real time.

[00137] In certain alternate modalities, metadata 322 can be similarly created and inserted into video data 316 by any other appropriate entity at any point along the distribution path. In certain of these alternate modes, the metadata 322 can be inserted without fully decompressing the video data 316. For example, the individual compressed video macro blocks 316 (without 322 metadata) can be replaced by the corresponding compressed macro blocks containing the already integrated 322 metadata. For all of the foregoing reasons, the present disclosure thus provides an improved system and method for distributing embedded metadata into video data.

Cm.

[00138] FIGURES 9 and 10 illustrate flow diagrams of exemplary steps of the method for retrieving the digital data embedded in the video data 316. FIGURE 10 illustrates an embodiment in which a luminance value used to encode a plurality of symbols in an image (eg, a video frame) is calculated by the receiving apparatus 550. FIGURE 10 illustrates an embodiment in. which the luminance value used to encode the plurality of symbols in the image, or any other parameter (s) (for example, additional luminance values ​​used to encode, one or more cutpoints, etc.) is encoded in the image.

[00139] Since the receiving apparatus 550 cannot know in advance exactly how to cut the luminance data to extract the digital data, FIGURE 9 illustrates an embodiment in which an algorithm is used to optimally establish one or more cut points. The algorithm is analogous to data recovery circuits in general use with analogous communication systems. In one embodiment, the optimal cutoff point for 2-level PAM encoding is determined by finding the location of a peak on a frequency curve of occurrence of different luminance values ​​and dividing that number by 2.

[00140] FIGURE 13 illustrates an exemplary analysis of watermark symbols (eg, a plurality of

<img file="MX370229B_D0016.tif" />

symbols encoded in an image). The graph was made by analyzing the watermarks that were degraded by a video encoding / decoding process. In total, 36,966 video frames were analyzed. The two encoding levels used to encode symbols were luminance values ​​0 and 42 (in an 8-bit luminance sample. On the Y axis, the graph shows the percentage of symbols found to have (on average) a particular luminance value provided on the X axis. For example, approximately 7% of all symbols had a luminance value of 42 ( the nominal, peak). Near zero they had a value of 21 (exactly between the two originally coded luminance values).

[00141] As illustrated in FIGURE 9, the receiving apparatus 550 receives or retrieves an image in which a plurality of symbols can be encoded in step 902. The image can be a video frame embedded in a digital television signal received, recorded audio / video content, or any other video or image source.

[00142] In step 904, the receiving apparatus 550 checks whether the image includes a plurality of symbols encoded therein. If it is determined that the image does not contain the plurality of symbols, the process ends. The receiving apparatus 550 determines whether the image includes a predetermined starting pattern before proceeding to step 906. The starting pattern consists of a fixed number of symbols containing predetermined values. An exemplary process for making this determination in greater detail is described below with respect to FIGURE 11. [00143] In step 906, the receiving apparatus 550 determines a set of luminance values ​​used to encode the symbols based on luminance values ​​of a plurality of pixels included in the image. For example, when a symbol is encoded using a plurality of pixels (eg, 8), the luminance values ​​of all or a subset of the plurality of pixels used to encode the symbol may be averaged to generate an average luminance value corresponding to that symbol. In this case, the first set of luminance values ​​can include the average luminance values ​​corresponding to the symbols.

[00144] In step 908, the receiving apparatus 550 determines a higher luminance value used to encode the symbols in the image based on the determined set of luminance values. In one embodiment, the receiving apparatus 550 determines the largest luminance value used to encode the symbols, based on a number of instances of each luminance value within a predetermined range of luminance values. The largest luminance value is the luminance value within the default luminance value range that has the highest number of instances. For example, referring to FIGURE 13, the receiving apparatus 550 identifies a peak within the instances of the different luminance values. Exemplary JavaScript for determining the highest luminance value is illustrated in FIGURES 14A-14C. In one embodiment, when more than one bit per symbol is encoded, the receiving apparatus 550 identifies a plurality of different peaks to identify which luminance values ​​are used to encode the different symbol values. In another embodiment, the luminance values ​​used to encode the different symbol values ​​are calculated, based on the highest determined luminance value.

[00145] In step 910, the receiving apparatus 550 derives the data values ​​of the symbols encoded in the image, based on it. set of luminance values ​​and the highest determined luminance value. Determinado1 determined higher luminance value is used to determine a cut point. The luminance values ​​in the luminance value set can be compared to the cut point in order to derive the values ​​of the encoded symbols in the image.

[00146] In one embodiment, when 1 bit per symbol is encoded and the lowest luminance value used to encode a symbol is 0, the determined highest luminance value is divided by a predetermined number (eg 2) to determine a cut point. However, when the lowest luminance value used to encode a symbol is not 0, the cut-off point can be calculated using the following formula: lowest luminance value + (highest luminance value - lowest luminance value) / 2 .

In one mode, when the determined breakpoint is not an integer, it is rounded to the nearest integer. Subsequently, each of the luminance values ​​in the luminance value set is compared to the cutoff point to derive the symbol cytate values. For example, when one bit per symbol is encoded and the luminance value in the set of luminance values ​​for a symbol is less than or equal to the cutoff point, the value for that symbol is determined to be 0. Also, when the luminance value in the first set of luminance values ​​is greater than the cut-off point, the value for that symbol is determined to be 1.

[00147] In another embodiment, when 2 bits per symbol are encoded, the determined highest luminance value is used to determine three cut points, which are compared to the first set of luminance values ​​to derive the symbol values. For example, when the largest luminance value determined is N and the four luminance values ​​used to encode the symbols are 0, N / 3, 2N / 3, and N, a first cutoff point can be determined by N / 6, a second cut point can be determined by N / 2, and a third cut point can be determined by 5N / 6. In a modality, any cut point that is not an integer is rounded to the nearest integer. Subsequently, each of the luminance values ​​in the luminance value set is compared to at least one of the first, second, or third cutoff points to derive the symbol data values.

[00148] In one embodiment, when the luminance value in the set of luminance values ​​corresponding to a symbol is less than or equal to the first cut-off point, the value for that symbol is determined to be. the first of one of 0, 1, 2, or 3 (for example, 0). When the luminance value in the set of luminance values ​​for a symbol is greater than the first cut point but less than or equal to the second cut point, the value for that symbol is determined to be one second of 0, 1, 2.0 3 (for example, 1). When the luminance value in the set of luminance values ​​for a symbol is greater than the second cut point but less than or equal to the third cut point, the value for that symbol is determined to be a third of 0, 1, 2, or 3 (for example, 2). When the luminance value in the luminance value set is greater than the third cutoff point, the value for that symbol is determined to be a quarter of 0, 1, 2, or 3 (for example, 3).

[00149] FIGURE 10 illustrates an embodiment in which a luminance value used to encode symbols in an image, and / or another parameter (eg, one or more cutpoints, a plurality of luminance used to encode symbols, etc.). In step 1002, the receiving apparatus 550 receives or retrieves an image in which a plurality of symbols can be encoded. The image may be a video frame embedded in a received digital television signal, recorded audio / video content, or any other video or image source.

[00150] In step 1004, the receiving apparatus 550 checks whether the image includes a plurality of symbols encoded therein. If it is determined that the image does not contain the plurality of symbols, the process ends. In one embodiment, the receiving apparatus 550 determines whether the image includes a predetermined starting pattern before proceeding to step 1006. As described above, the boot pattern consists of a fixed number of symbols that contains default values. An exemplary process for making this determination is described in more detail below with respect to FIGURE 11.

[00151] laη step 1006, the receiving apparatus 550 determines a first set of luminance values ​​used to encode a first subset of the symbols, based on luminance values ​​of a first plurality of pixels included in the image. The first plurality of pixels is located in a predetermined region of the image. In one embodiment, the first plurality of pixels corresponds to predetermined pixels on lines 1 and / or 2 of a video frame. For example, when the symbols 10 encoded in the image include the default start pattern, the default start pattern corresponds to a plurality of initial pixels on lines 1 and / or 2, and the first plurality of pixels corresponds to the pixels that they immediately follow the initial plurality of pixels 15 on lines 1 and / or 2.

[00152] When a symbol is encoded using a plurality of pixels (eg 8), the luminance values ​​of all or a subset of the plurality of pixels used to encode the symbol can be averaged, 20 to generate an average luminance value corresponding to that symbol. In this case, the first set of luminance values ​​includes the average luminance values ​​corresponding to the first subset of the symbols.

[00153] In step 1008, the receiving apparatus 550 derives data values ​​from the first subset of the símbolosν encoded symbols in the image, based on a first predetermined cut-off point (eg, luminance value 15). For example, the receiving apparatus 550 compares the. first set of luminance values ​​with the first predetermined cutoff point. When a luminance value in the. first set of luminance values ​​is less than or equal to the first predetermined cut-off point, a symbol value corresponding to that luminance value is determined to be 0. When the luminance value in the first set of luminance values ​​is greater than the First predetermined cut-off point, the symbol value corresponding to that luminance value is determined to be 1.

[00154] In step 1010, the receiving apparatus 550 determines a luminance value used to encode a second subset of symbols in the image, based on the data values ​​derived from the first subset of the symbols. In other embodiments, the receiving apparatus 550 determines one or more cut points, a plurality of luminance values ​​used to encode symbols, again based on the values ​​derived from the first subset of the symbols, etc. for example, data values ​​derived from the first subset of symbols define the highest luminance value used when encoding 1 bit per symbol, in a case where the lowest luminance value is predetermined (for example, as specified by a standard ). In another embodiment, the data values ​​derived from the first subset of the symbols define the single or more cutpoints, and / or the plurality of luminance values ​​used to encode symbols, to be used to derive values ​​from a second subset of the symbols. . For example, the cutoff point used to determine symbol data values ​​can be identified instead of the upper luminance value. In another embodiment, when 2 bits per symbol are encoded, three luminance values ​​used to encode symbol data values ​​or corresponding cutoff points can be identified by the data values ​​derived from the first subset of the symbols.

[00155] In step S1012, the receiving apparatus 550 determines a second set of luminance values ​​used to encode the second subset of the symbols based on luminance values ​​of a second plurality of pixels included in the image. The second plurality of pixels is located in a predetermined region of the image. In one embodiment, the second plurality of pixels is adjacent to (eg, immediately follows) the first plurality of pixels. When a symbol is encoded using a plurality of pixels (eg 8), the luminance values ​​of all or a subset of the plurality of pixels used to encode the symbol can be averaged to generate an average luminance value for that symbol . In this case, the second set of luminance values ​​includes the average luminance values ​​corresponding to the symbols.

[00156] In step S1014, apparatus 550 of. Receive derives data values ​​from the second subset of the symbols encoded in the image, based on the second set of luminance values ​​and using the largest luminance value determined in step 1010. For example, as described above, the values ​​of the second subset of symbols can be derived in a similar way as in step 1008. In another embodiment, the data values ​​of the second subset of the symbols are derived by comparing the second set of luminance values ​​with at least one of one or more cutoff points defined by the first subset of the symbols. Note that one or more of the breakpoints used to derive the data values ​​of the second subset of the symbols may or may not include the first default breakpoint. In addition, note that the number of bits encoded per symbol may or may not be the same between the first and second subsets of the symbols depending on the mode.

[00157] FIGURE 11 illustrates a method of determining, for example, whether an image such as that received or retrieved in step 902 or 1002 includes a plurality of symbols (eg, digital data) encoded therein. The receiving apparatus 550 determines whether a predetermined starting pattern (eg, a fixed pattern of predetermined symbol data values) is encoded in the image according to certain modalities. The presence of the predetermined start pattern in certain modes indicates that the image includes the plurality of symbols encoded therein.

[00158] As illustrated in FIGURE 11, at step 1102, the reception apparatus 550 processes the image and determines a set of luminance values ​​that is used to encode a plurality of symbols based on a plurality of pixels included in a image.

[00159] In. Step 1102, the receiving apparatus 550 determines, a set of luminance values ​​of a predetermined plurality of pixels included in the image, in which the start pattern could typically be encoded. The predetermined plurality of pixels is located in a predetermined region of the image, such as a predetermined first number of pixels on lines 1. and / or 2 of a video frame. When a symbol is encoded using a plurality of pixels (eg 8), the luminance values ​​of all or a subset of the plurality of pixels used to encode the symbol can be averaged to generate an average luminance value for that symbol . In this case, the first set of luminance values ​​includes the average luminance values ​​corresponding to the symbols. Other algorithms can be used to determine the luminance value of the symbol that was originally encoded. For example, the highest and lowest luminance values ​​among the plurality of pixels can be discarded before averaging. These methods are designed to account for the luminance errors introduced by the video compression process.

[00160] In step 1104, the receiving apparatus 550 derives data values ​​of the plurality of symbols encoded in the image based on a predetermined cut-off point (eg, luminance value 15). The default cut point may or may not be the same as the first default cut point used in. Step 1000. For example, for two-level encoding (ie, 1 bit per symbol), the receiving apparatus 550 compares the set of luminance values ​​with the predetermined cut-off point. When the luminance value in the luminance value set is less than or equal to the predetermined cutoff point, a symbol data value corresponding to that luminance value is determined to be 1. When the luminance value in the luminance value set is greater than the default cutoff point,

<img file="MX370229B_D0017.tif" />

determines that the symbol data value corresponding to that luminance value is 1.

[00161] In another example, for four-level encoding (ie, 2 bits per symbol), the receiving apparatus 550 compares the set of luminance values ​​with three different cutoff points. In one embodiment, when the luminance value in the set of luminance values ​​for a symbol is less than or equal to the first cutoff point, the value for that symbol is determined to be a first of 0, 1, 2, or 3 (for example, 0). When the luminance value in the set of luminance values ​​for a symbol is greater than the first cut point but less than or equal to the second cut point, the value for that symbol is determined to be one second of 0, 1, 2, or 3 (for example, 1). When the luminance value in the set of luminance values ​​for a symbol is greater than the second cut point but less than or equal to the third cut point, the value for that symbol is determined to be a third of 0, 1, 2, or 3 (for example, 2). When the luminance value in the luminance value set is greater than the third cutoff point, the value for that symbol is determined to be a quarter of 0, 1, 2, or 3 (for example, 3).

[00162] In step 1106, the receiving apparatus 550 determines whether the data values ​​derived from the plurality

Ib

<img file="MX370229B_D0018.tif" />

symbol matches the default boot pattern (for example, a fixed pattern of default symbol data values). When the receiving apparatus 550 determines that the values ​​derived from the plurality of symbols do not match the predetermined starting pattern, the receiving apparatus 550 waits for the next image in step S1110. For example, the process returns to step 902 or 1002.

[00163] In another embodiment, the receiving apparatus 550 compares the values ​​derived from the plurality of symbols with a plurality of different predetermined starting patterns in the event that the receiving apparatus 550 may attempt to perform the method illustrated in FIGURE 9 or 10 based on the matching default boot pattern. For example, it can be known whether or not the modulation level is encoded in the data just after the boot pattern depending on the boot pattern it is in.

[00164] When the receiving apparatus 550 determines that the values ​​derived from the plurality of symbols match the predetermined starting pattern, the receiving apparatus 550 proceeds to 1.a. step 1108, at the same time the process proceeds to step 906 or 1006. In one embodiment, a determination is made whether to proceed to step 906 or 1006 based on which of a plurality of predetermined starting patterns.

match.

[00165] In certain modalities of the. On a one-bit-per-symbol basis, the luminance values ​​to be used for the value of 1 can be restricted to a predetermined set of values, such as multiples of 10 in a range (eg, range from 40 to 100, even). Thus, if the luminance value to be used for the value of 0 is 0, the receiving apparatus 550 would use cutoff points of either 20, 25, 30, 35, 40, 45, or 50, as appropriate. If a luminance value other than 0 is used for the value of 0, the cut points would be adjusted accordingly. For example, if the default luminance value to be used for the value of 0 is 4, the receiver would use cutoff points of either 22, 27, 32, 37, 42, 47, or 52, as appropriate. In certain embodiments the determination in the receiving apparatus 550 of which of these six options is currently in use can be determined using the method illustrated in FIGURE 9 or 10. In other embodiments the luminance values ​​may be restricted to other multiples. In one embodiment the use of predetermined multiples can also facilitate the determination of the highest luminance value in step 908. For example, the receiving apparatus 550 selects the closest luminance value among the possible predetermined numbers.

[00166] Note that one or more cutoff points used to derive the data values ​​of the plurality of symbols in FIGURE 11 may or may not include the same cutoff point as in FIGURE 9 or 10. In addition, note that the number Bit rate per symbol may or may not be the same between the symbols in FIGURE 11 and FIGURES 9 and 10, depending on the mode.

[00167] As described above with respect to FIGURES 9 and 10, luminance values ​​of all or a subset of the plurality of pixels used to encode a symbol may be averaged to generate an average luminance value corresponding to that symbol. . In one embodiment, instead of averaging all pixels, the first and last pixels are skipped, and only half the pixels (for example, the 6 half. Pixels) are averaged. In another embodiment, pixels that have the smallest and / or largest values ​​can be discarded before averaging.

[00168] Although modalities of the present description describe that the data value of a symbol is set to 0 when a corresponding luminance value is less than or equal to a cut-off point and is set to 1 when the luminance value corresponding is greater than the cut-off point, it should be noted that the present description is not so limited. In other modalities, the. setting of 0 and 1 can be reversed. In addition, in other embodiments comparisons less than or equal to and greater than, with the predetermined cut-off point can be replaced with less than a and greater than or equal to the comparison. Similarly, various other methods can be used to set a symbol data value when 2 or more bits are encoded by 5 symbol.

[00169] FIGURES 14A-14C illustrate exemplary JavaScript codes that implement 1 an algorithm that can be used to process a plurality of encoded symbols in an image according to certain modalities. The algorithm can be used to estimate the peak value (42 in the example illustrated in FIGURE 13), which corresponds to the luminance value originally encoded before video compression. The algorithm involves the following stages: in a period or 1 or more frames: (1) for each frame, derive the 15 symbol values ​​by averaging each set of 8 pixels in line 1 and rounding to the nearest integer; (2) count the number of symbol values ​​that are presented for each luminance level in the range of 20 to 100; and (3) determine the luminance value that has the largest number of observed values.

[00170] The JavaScript code illustrated in FIGURES 14A14C uses the following variables:

[00171] rawLumaf] ~ an array of 1920 elements containing the raw luminance values ​​(8 25 bit values)

[00172] mySymbolsí] - an array of 240 elements containing the symbol values

[00173] bins [] - a 256-element array containing the cumulative number of instances where a symbol value matched the index value of the array. For example: bins [30] contains the total number of times a symbol value of 30 was found within the processed number of frames of the marked video.

[00174] FIGURE 14A illustrates exemplary JavaScript codes that can be used to derive symbol values ​​given the raw luminance values. FIGURES 14B and 14C illustrate exemplary JavaScript codes that can be used to derive the symbol luminance peak value in a range in which the higher encoded value may appear. In FIGURE 14B, after the symbols are collected in mySymbols, the accumulation in deposits can be performed for each frame. In FIGURE 14C, the peak can be determined after one or more frames have been processed (deposited data collected). For a two-level encoding, in case the luminance value used to encode the value of 0 is known to be M, the appropriate cutoff point would be M + (peak - 14) / 2. Tests have shown that even if only one frame is processed in this way, a good first approximation is provided. Processing multiple frames can further refine the value.

[00175] FIGURE 15 is a block diagram showing an example of a hardware configuration of a 1500 computer configured to perform one or a combination of the functions described above, such as one or more of the functions · Content resource 114, converter-decoder box 118, television 122, server 130, apparatus 250 that provides information, and / or reception apparatus 550.

[00176] As illustrated in Figure 15, computer 1500 includes a central processing unit (CPU) 1502, read-only memory (ROM) 1504, and random access memory (RAM) 1506 interconnected by one or the other. more buses 1508. The one or more buses 1508 further connect to an input-output interface 1510. The input-output interface 1510 connects to an input portion 1512 consisting of a keyboard, mouse, microphone, remote controller, etc. The input-output interface 1510 is also connected to an output portion 1514 consisting of an audio interface, video interface, display screen, speaker, etc .; a registry portion 1516 consisting of a hard disk, nonvolatile memory, etc .; a communication portion 1518 consisting of a network interface, modem, USB interface, FireWire interface, etc .; and a unit 1520 for controlling removable media 1522, such as magnetic disk, optical disk, magneto-optical disk, a semiconductor memory, etc.

[00177] In accordance with one embodiment, CPU 1502 loads a program stored in register portion 1516 into RAM 1506 via input-output interface 1510 and bus 1508, and then executes a program configured to provide the functionality of one or combination of content resource 114, set top box 118, television 122, server 130, apparatus 250 providing information, and / or receiving apparatus 550.

[00178] As described above, the receiving apparatus 550 in accordance with embodiments of the present disclosure can optimally retrieve integrated digital data, regardless of the amplitude, (luminance values) used in the encoding. Flexibility is desired, and the broadcaster can choose the appropriate tradeoff between visibility and robustness against errors caused by video compression or video transcoding.

[00179] The present description has been explained above with reference to certain embodiments. Other modalities will be apparent to those of skill in the art in light of this description. For example, the present description can be easily implemented using configurations and techniques different from those

<img file="MX370229B_D0019.tif" />

described in the previous modalities. Furthermore, the present disclosure can be effectively used in conjunction with systems other than those described above. Therefore, these and other variations on the discussed modalities are intended to be covered by the present disclosure, which is limited only by the appended claims.

[00180] The various processes discussed in the above need not be processed chronologically in the sequence represented as flow charts; Stages can also include those processed parallel or individually (for example, parallel or object oriented).

[00181] Furthermore, the programs can be processed by a single computer or by a plurality of computers on a distributed basis. Programs can also be transferred to a remote computer or computers for execution.

[00182] Furthermore, in this specification, the term system means an aggregate of a plurality of component elements (apparatus, modules (parts), etc.). All component elements may or may not be housed in a single cabinet. Therefore, a plurality of appliances each housed in a separate cabinet and connected via a network is considered a network, and a single appliance made up of a plurality of modules housed in a single cabinet can also be considered as a system.

[00183] Furthermore, it should be understood that when this technology is represented, it is not limited to the modalities described in the foregoing and that various modifications, variations and alternatives of this technology can be made to the extent that it is within the spirit and scope of it.

[00184] For example, this technology can be structured for cloud computing whereby a single function is shared and processed collaboratively between a plurality of devices over a network.

[00185] Also, each one of the stages explained in reference, a. The flow charts described above can be executed not only by a single appliance but also by a plurality of appliances in a shared manner.

[00186] Furthermore, if a stage includes a plurality of processes, these processes included in the stage can be performed not only by a single apparatus but also by a plurality of apparatus in a shared manner.

[00187] Numerous modifications and variations of the embodiments of the present disclosure are possible in view of the above teachings. It is therefore understood that within the scope of the appended claims, the embodiments may be practiced in a manner other than as specifically described herein.

<img file="MX370229B_D0020.tif" />

[00188] The above description also encompasses the modalities noted below.

[00189] (1) A receiving apparatus, including circuitry, configured to receive or retrieve an image in which a plurality of symbols are encoded, determining a set of luminance values ​​used to encode the symbols based on luminance values ​​of a plurality of pixels included in the image, determining a higher luminance value used to encode the symbols in the image based on the determined set of luminance values, and deriving data values ​​of the symbols encoded in the image based on the set of luminance values ​​and using the determined highest luminance value.

[00190] (2) The receiving apparatus according to characteristic (1), in which the circuitry is configured to determine the set of luminance values ​​at. average different subsets of the luminance values ​​of the plurality of pixels included in the image.

[00191] (3) The receiving apparatus according to feature (1) or (2), in which the circuitry is configured to determine a cutoff point using the following equation: cut-off point = Μ + (Ν - M) / 2, where M equals a lower luminance value used to • IXSSSSS · encode the symbols in the image and N equals the determined higher luminance value, and derive the symbol data values ​​when comparing the set of luminance values ​​to the cut point.

[00192] (4) The receiving apparatus according to any of the characteristics (1) to (3), in which the circuitry is configured to determine the highest luminance value used to encode the symbols based on a number of instances For each luminance value within a predetermined range of luminance values, the largest luminance value is the luminance value within the predetermined range of luminance values ​​that has the highest number of instances.

[00193] (5) The receiving apparatus according to any one of features (1) to (4), in which the highest luminance value used to encode the symbols is restricted to a predetermined set of values.

[00194] [6) The receiving apparatus according to any of the characteristics (1) to (5), in which the image is a video frame, and the symbols are encoded in the first line of the video frame .

[00195] 7) The receiving apparatus according to any of the characteristics (1) to (6), in which the circuitry is configured to: determine if a fixed pattern

<img file="MX370229B_D0021.tif" />

of default symbol data values ​​is encoded in the image, and only when the fixed pattern of default symbol data values ​​is determined to be encoded in the image, determine the set of luminance values, determine the largest luminance value, and derive the data values ​​of the symbols.

[00196] (8) A method of processing an image in which a plurality of symbols is encoded, the method including: receiving or retrieving, by means of circuitry from a receiving apparatus, the image in which the plurality of symbols; determining a set of luminance values ​​used to encode the symbols based on luminance values ​​of a plurality of pixels included in the image; determining, by circuitry, a higher luminance value used to encode the symbols in the image based on the determined set of luminance values; and deriving, by circuitry, data values ​​of the symbols encoded in the image based on the set of luminance values ​​and using the highest determined luminance value.

[00197] (9) The method according to feature (8), in which the step of determining the set of luminance values ​​comprises: determining the set of luminance values ​​by averaging different subsets of the luminance values ​​of the plurality of pixels included

<img file="MX370229B_D0022.tif" />

in the image.

[00198] (10) The method according to characteristic (8) or (9), further includes: determining a cut point using the following equation: cut point = Μ + (NM) /^z2, where M is equal to a lower luminance value used to encode the symbols in the image and N is equal to the higher determined luminance value, in which the step of deriving the symbols includes deriving the data values ​​of the symbols when comparing the set of luminance values ​​to the cut point.

[00199] (11) The method according to any of the characteristics (8) to (10), in which the step of determining the. Higher luminance value includes determining the largest luminance value used to encode symbols based on a number of instances of each luminance value within a predetermined range of luminance values, the largest luminance value is the luminance value within the range The default luminance value that has the highest number of instances.

[00200] (12) The method according to any one of features (8) to (11), in which the highest luminance value used to encode the symbols is restricted to a predetermined set of values.

[00201] (13) The method according to any of the features (8) to (12), in which, the image is a video frame, and the symbols are encoded in the first line of the video frame.

[00202] (14) The method according to any of the characteristics (8) to (13), also includes: determining whether a fixed 5 pattern of predetermined symbol data values ​​is encoded in the image, in which the steps for determining the set of luminance values, determining the largest luminance value, and deriving the symbol data values ​​are performed only when determining the predetermined fixed pattern 10 of symbol data values ​​to be encoded in the image.

[00203] (15) A non-transient computer readable medium that stores instructions which, when executed by a computer, causes computer 15 to perform a method of processing an image in which a plurality of symbols are encoded, the method including: receiving or retrieving the image in which the plurality of symbols is encoded; determining a set of luminance values ​​used to encode the symbols based on 20 luminance values ​​from a plurality of pixels included in the image; determining a larger luminance value used to encode the symbols in the image based on the determined set of luminance values; and deriving data values ​​of the symbols encoded in image 25 based on the set of luminance values ​​and using

4 the highest luminance value determined.

[00204] (16) A receiving apparatus, including:

circuitry configured to receive or retrieve an image in which a plurality of symbols is encoded, determining a first set of luminance values ​​used to encode a first subset of the symbols, based on luminance values ​​of a first plurality of pixels included in the image, deriving data values ​​from the first subset of the symbols encoded in the image based on a first predetermined cut-off point, determine a luminance value used to encode a second subset of the symbols in the image based on the data values ​​derived from the first subset of the symbols, determine a second set of luminance values ​​used to encode the second subset of the symbols based on luminance values ​​of a second plurality of pixels included in the image, and deriving data values ​​from the second subset of the symbols encoded in the image based on the second set of luminance values ​​and using the determined luminance value.

[00205] (17) The receiving apparatus according to feature (16), in which the circuitry is configured to: determining a third set of luminance values ​​used to encode a third plurality of symbols, based on luminance values ​​of a third plurality of pixels included in the image, deriving data values ​​of the third plurality of symbols encoded in the image based on a second default cut point, determining whether the data values ​​derived from the third plurality of symbols match a fixed pattern of predetermined symbol data values, and only when the data values ​​derived from the third plurality of symbols match the fixed pattern of data values ​​of default symbol, determine the first set of luminance values, derive data values ​​from the first subset of symbols, determine the luminance value, determine the second set of luminance values, and derive the data values ​​of the second subset of the symbols.

[00206] (18) A method of processing an image in which a plurality of symbols is encoded, the method including: receiving or retrieving, by a circuitry of a receiving apparatus, an image in which a plurality of symbols; determining a first set of luminance values ​​used to encode a first subset of the symbols based on luminance values ​​of a first plurality of pixels included in the image; derive data values ​​of the first subset of the symbols encoded in the image based on a first point

6 default cut; determine, using circuitry, a luminance value. used to encode a second subset of the symbols in the image based on the data values ​​derived from the first subset of the symbols; determining a second set of luminance values ​​used to encode the second subset of the symbols based on a second plurality of pixels included in the image; and deriving, through circuitry, data values ​​from the second subset of the symbols encoded in the image based on the second set of luminance values ​​and using the determined luminance value.

[00207] (19) The receiving apparatus according to feature (18) further includes: determining a third set of luminance values ​​used to encode a third plurality of symbols based on luminance values ​​of a third plurality of pixels included in the image; deriving data values ​​from the third plurality of symbols encoded in the. image based on a second default cut point; and determining whether the data values ​​derived from the third plurality of symbols match a fixed pattern of predetermined symbol data values, in which steps to determine the first set of luminance values, derive the data values ​​from the first subset of the symbols, determine

<img file="MX370229B_D0023.tif" />

the luminance value, determining the second set of luminance values, and deriving the data values ​​of the second subset of the symbols are performed only when the data values ​​derived from the third plurality of symbols match the fixed pattern of data values default symbol.

[00208] (20) A non-transient computer-readable medium that stores instructions that when executed by a computer cause the computer to perform a method of processing an image in which a plurality of symbols are encoded, the method including: receiving or retrieve an image in which a plurality of symbols is encoded; determining a first set of luminance values ​​used to encode a first subset of the symbols based on luminance values ​​of a first plurality of pixels included in the image; deriving values, from data of the first subset of the symbols encoded in the image based on a first predetermined cutoff point; determining a luminance value used to encode a second subset of the symbols in the image based on the data values ​​derived from the first subset of the symbols; determining a second set of luminance values ​​used to encode the second "subset of the symbols based on luminance values ​​of a second plurality of pixels included in the image;

<img file="MX370229B_D0024.tif" />

and deriving data values ​​from the second subset of the symbols encoded in the image based on the second set of luminance values ​​and using the determined luminance value.

[00209] (21) An apparatus that provides information, including: circuitry configured to receive or retrieve an image in which a plurality of symbols will be encoded, encode the plurality of symbols in the image, the plurality of symbols is encoded in the image using luminance values ​​of a plurality of pixels included in the image, and provide the image in which the plurality of symbols are encoded to a receiving apparatus, in which a luminance value used to encode at least one of the plurality of symbols is set by an operator.

[00210] (22) The apparatus that provides information according to feature (21), in which the data values ​​of a subset of the plurality of symbols identify the luminance value set by the operator.

[00211] (23) The apparatus that provides information according to feature (21) or (22), in which the operator is from a terrestrial television broadcaster.

[00212] (24). A method of providing an image in which a plurality of symbols is encoded, the method including: receive or retrieve an image in which the plurality of symbols will be encoded, encode, by means of an information-providing apparatus circuitry, the plurality of symbols in the image, the plurality of symbols is encoded in the image using luminance values ​​of a plurality of pixels included in the image, and providing, through the circuitry, the image in which the plurality of symbols is encoded to a receiving apparatus, in which a luminance value used to encode at least one of the plurality of symbols is set by an operator.

[00213] (25) • The method according to feature (24), in which the data values ​​of a subset of the plurality of symbols identify the luminance value set by the operator.

[00214] (23) The method according to feature (24) or (25), in which the operator is from a terrestrial television broadcaster.

[00215] (20) A non-transient computer-readable medium that stores instructions that, when executed by a computer, cause the computer to perform a method of providing an image in which a plurality of symbols are encoded, the method including : receive or retrieve an image in which the plurality of symbols will be encoded, encode the plurality of symbols in the image, the plurality of symbols will be encoded in the image, using luminance values. of a plurality of pixels included in the image, and providing the image in which the plurality of symbols is encoded to a receiving apparatus, in which a luminance value used to encode at least one of the plurality of symbols is set by an operator.