Resource scalable decoding
Abstract
Decoder system and method with scalable complexity. The system includes a decoder with multiple functional blocks for decoding compressed video data. The decoder also includes multiple hierarchical functions, which can selectively reduce the complexity of at least one functional block. The local resource controller is used to generate a grading rule for the decoder, where a grading rule is selected in response to a complexity request from the system resource manager. Furthermore, the grading rule is a plurality of predetermined rules available from the local resource controller To choose from. Look up the pre-determined classification rules in the lookup table, and the lookup table is designed in an offline state.
Term
Term ended
Projected expiry passed 28 June 2022, 4.2 years ago.
- Priority
- Filed
- Published
- Projected expiry
- Today
22 claims: 3 independent, 19 dependent
- 1一种复杂性可分级解码器系统[12],包括:一个有多个功能块的解码器[24],用于对压缩视频数据[58]解码,其中,解码器[24]还包括多个可分级功能[40],用于有选择地降低至少一个功能块的复杂性;一个本地资源控制器[14],用于为解码器[24]产生分级规则[50],其中,分级规则[50]响应来自系统资源管理器[18]的复杂性的要求[56]进行选择,其中,分级规则[50]是在本地资源控制器[14]可用的多个预定的规则中进行选择的。
- 2根据权利要求1的复杂性可分级解码器系统[12],其中,多个预定的分级规则列于查询表[46]。
- 3根据权利要求1的复杂性可分级解码器系统[12],其中,多个预定的分级规则是脱机设计的[54]。
- 4根据权利要求1中的复杂性可分级解码器系统[12],其中,多个预定的分级规则中的每一个规则,指示令至少一个分级功能如何减少解码器[24]的复杂性。
- 5根据权利要求4的复杂性可分级解码器系统[12],其中,解码器复杂性是从包括下述内容组中选择的:CPU处理能力要求,带宽要求,功耗和存储容量要求。
- 6根据权利要求4的复杂性可分级解码器系统[12],其中,多个预定的分级规则中的至少一个含有处理提交到资源管理器[18]的PIP请求的规则。
- 7根据权利要求6的复杂性可分级解码器系统[12],其中,处理PIP请求的分级规则包含一个PIP分级功能。
- 8根据权利要求1的复杂性可分级解码器系统[12],其中,分级功能是从包括下述内容组中选择的:自适应B帧分级,IDCT分级,运动补偿分级和嵌入式存储容量重置。
- 9根据权利要求1的复杂性可分级解码器系统[12],其中,本地资源控制器[14]还包括一个数据相关规则系统[48],它考察解码器所处理的数据[52],当输出的复杂性等级低于预先确定的门限时,就改变选定的分级规则。
- 10根据权利要求9的复杂性可分级解码器系统[12],其中,数据相关规则系统[48]包含一个系统,用于确定输出质量是否低于预先确定的门限。
- 11根据权利要求9的复杂性可分级解码器系统[12],其中,数据相关规则系统[48]包含一个系统,用于确定解码器所处理的非零DCT数据量是否抵于预定的门限。
- 12根据在有多个处理功能[20,22,24,26,28]和一个资源管理器[18]的媒体处理器核中提供复杂性可分级解码器系统[12]的方法,此方法包含以下步骤:从资源管理器[18]复杂性请求发送一个复杂性请求到与分级解码器[24]相连的本地资源控制器[14],响应复杂性请求[56],从查询表[46]选择一个分级规则[50],将分级规则[50]传送到分级解码器,并且基于所传送的分级规则[50],降低分级解码器[24]的复杂性。
- 13根据权利要求12的方法,还包含如下的步骤:考察解码器所处理的数据[52];并且若考察的数据[52]突破预先确定的门限,则改变分级规则[50]。
- 14根据权利要求13的方法,其中考察数据[52]包括输出质量水平。
- 15根据权利要求13的方法,其中考察数据[52]包括统计非零数据量。
- 16根据权利要求12的方法,包含下述预备步骤:在脱机方式下,对多个减少复杂性请求中的每一个,确定一个优选的分级规则,基于优选的规则,生成查询表[46]。
- 17根据权利要求12的方法,其中所选择的分级规则至少包含一个分级功能,它能完成从包括下述内容组中选择的减少复杂性的功能:减少功耗,减少存储容量,减少带宽以及减少处理能力。
- 18一种存储在可记录介质上的程序产品,当运行时,对媒体管理器核[16]中的可分级解码器[24]进行本地资源控制,该程序产品包括:程序代码,用于接收来自资源管理器[18]的复杂性请求[56],程序代码,基于收到的复杂性请求[56],从多个预先确定的分级规则中选择一个分级规则[50];以及程序代码,用于把选择的分级规则[50]传送到分级解码器[24]。
- 19根据权利要求18的程序产品,还包括:程序代码,用于分析从解码器[24]收到的数据[52];以及程序代码,用于在所分析的数据[52]突破预先确定的门限时,改变选定的分级规则。
- 20根据权利要求18的程序产品,还包括一个查询表[46],它含有多个预先确定的分级规则。
- 21根据权利要求20中的程序产品,其中,在查询表[46]中,多个预先确定的分级规则中的每一个规则,包含一条相关的复杂性请求。
- 22根据权利要求18中的程序产品,其中,多个预先确定的分级规则包括从由下述内容组成的组中选择的分级功能:自适应B帧分级,IDCT分级,嵌入式存储容量重置,运动补偿分级,以及PIP处理。
Independent claims22
32 paragraphs, as filed
Resource hierarchical decoding
1. Technical Field The present invention relates to the processing of compressed video signals, and in particular to a system and method, which is used to implement hierarchical control of the main stream decoding process by a local resource controller in a media processing core (core).
2. Background technology involved. Multimedia processing systems are becoming a rapidly growing part of the consumer electronics market. Standards based on video processing, such as MPEG2, MPEG4, H.263, etc., have become a key factor for the success of this industry. Among all video processing functions, video decoding is usually the process that consumes the most resources among various functions. The video decoding process is carried out according to one of various standards, which guarantees an appropriate level of output signal quality. However, such standards usually assume that the decoder works in an environment with unlimited resources.
The heart of the multimedia processing system is a media processor core (MPC), which performs all processing functions. However, it is often found that the requirements for simultaneous multimedia processing and decoding functions exceed the capabilities of MPC (ie, computing power). When this happens, the resources allocated to each function (such as CPU cycles, storage capacity, memory bandwidth, power consumption, etc.) appear limited. For systems constrained by resources, one solution is "grading", that is, reducing the complexity of certain processing functions or algorithms. However, the price of hierarchical operation is reduced output signal quality.
For systems with limited resources, the computer resources available for specific processing functions will change over time according to the load of the system. In order to ensure that the system can work in a timely manner, the complexity of the algorithm must always be dynamically adapted to the available resources by exchanging the quality of the output signal. Therefore, there is a need for a hierarchical decoder that can dynamically adapt to resource constraints. Furthermore, a low-price, high-efficiency, resource-constrained, and scalable-complexity video decoder is crucial for a new generation of multi-functional and multi-purpose multimedia video equipment. Summary of the invention The present invention overcomes The above and other problems provide a decoder system with scalable complexity that can dynamically adapt to resource constraints. First, the present invention provides a scalable decoder system with multiple functional blocks. The decoder is used to decode compressed video data, where the decoder also includes multiple grading functions, which can selectively reduce the complexity of at least one functional block; there is also a local resource controller for generating decoder grading rules, Among them, the selection of the grading rule is to respond to the complexity request of the system resource manager, and the grading rule is selected from a plurality of predetermined grading rules available to the local resource controller.
Secondly, the present invention provides a method to provide a decoder system with scalable complexity in a media processor core with multiple processing functions and a resource manager. The method has the following steps: send a complexity request from the resource manager to the local resource controller connected to the hierarchical decoder, select a hierarchical rule from the query table in response to the complexity request; transmit the hierarchical rule to the hierarchical decoder ; Reduce the complexity of the hierarchical decoder based on the incoming hierarchical rules.
Third, the present invention provides a program product, which is stored on a recordable medium, and when it runs, performs local resource control on the hierarchical decoder in the media processor core. The program product includes: program code for receiving a complexity request from a resource manager; program code for selecting a rule from a plurality of predetermined grading rules based on the received complexity request; and transmitting the selected grading rule to The program code of the hierarchical decoder.
Brief Description of the Drawings Hereinafter, preferred embodiments of the present invention will be described in conjunction with the accompanying drawings. In the figures, similar symbols are used to denote corresponding units, and Fig. 1 depicts a block diagram of a media processor core according to the present invention.
Figure 2 depicts a lookup table according to the invention.
Figure 3 depicts a resource scalable decoding system according to a preferred embodiment of the present invention.
Figure 4 depicts a typical flow chart of a scalable decoding system.
Detailed description of the present invention 1. Overview For ease of description, the present invention is described with an embodiment of an MPEG2 video decoder. However, it should be understood that the present invention can be used in any decoding system with a similar structure. The standard MPGE2 video decoder is well known in the art. It uses four functional blocks and an adder to decode the MPEG2 video bitstream. These functional blocks are Variable Length Decoder (VLD), Inverse Scanning and Inverse Vectorization (IQ), Inverse Discrete Cosine Transform (IDCT) and Motion Compensation (MC). The MPEG2 decoder uses two frame buffers as reference frame storage. Generally speaking, the higher the frame resolution, the more decoding resources (storage capacity, memory bandwidth, and computational complexity) are required. Various schemes of MPEG2 hierarchical decoders have been proposed, some of which will be discussed in detail below.
Referring to FIG. 1, the system block diagram 10 shown contains different multimedia processing functional units 20, 22, 24, 26, 28. Each unit has its own local resource controller (LRC1-LRC5). The resource manager 18 of the media processor core (MPC) (shown here as "RM-MPC" or "resource manager") controls each functional unit through the local resource controller of each unit. For example, RM-MPC18 can contain a high-quality service manager and a rule manager. RM-MPC 18 monitors the resource utilization of all functional units in the system 10 and dynamically allocates these resources to each functional unit. In this way, when a functional unit requests more resources, RM-MPC18 will weight the priority of the request and reallocate all resources; then, MPC16 will power off a certain functional unit, (such as removing some Complex post-processing to produce lower-quality audio output, or reduce the computational load of the main stream MPEG2 decoding) to support other units.
2. Resource scalable decoding The feature of the present invention is to provide a resource scalable decoding system. In this exemplary embodiment, the resource scalable decoding system is implemented using LRC314 and the main stream MPEG2 decoding 24. In the conventional operating mode, resources are allocated to each functional unit according to the default design, which usually does not cause resource conflicts with the main stream MPEG2 decoding process (ie, decoder) 24. For example, when the main stream decoding 24 is performed, the PIP (Picture in Picture) encoding/decoding 22 is not always activated. However, when other functional units other than the main code stream decoding unit 24 request considerable resources, the LRC3 activates the resource scalable MPEG2 decoding, thereby reducing the resources occupied by the main code stream decoding unit 24. For example, when a viewer wants to record PIP in the memory in a high-quality data format (such as MPEG2 or MPEG4), the PIP encoder 22 has a higher resource priority, and at the same time, the main stream decoding 24 has to be graded In order to release the resources it occupies.
The LRC314 connected to the main stream decoder 24 receives the predetermined dynamic resources from the RM-MPC 18, and reallocates the received resources to each functional block in the decoder. In this way, LRC314 uses one or more possible classification rules to locally decide how to allocate resources for the decoding process. LRC3 selects and controls classification rules through a lookup table. The lookup table will be described in detail below with reference to Figures 2 and 3.
3. The query table derives the classification rules through offline experiments, and lists these rules in the gradable query table. In order to generate this lookup table, a number of applicable hierarchical functions or algorithms should be provided to the designer. Each hierarchical function reduces the complexity of a certain functional block in the decoder at the cost of reducing the quality of the output signal. Examples of such classification functions are listed in the following references, including: (1) Adaptive B-frame classification, which was submitted on March 29, 2001, serial number 09/821140, titled "Can The hierarchical MPEG-2 Video Decoder is disclosed in the US patent application. (2) The IDCT classification is disclosed in the co-pending US patent application filed on January 11, 2001, serial number 09/759042, titled "Scalable MPEG2 Decoder". (3) Motion compensation classification, in the co-pending US patent application filed on January 9, 2001, serial number 09/709260, titled "Scalable MPEG-2 Video Decoder with Selective Motion Compensation" Disclosure in; embedded storage capacity reset (resizingscaling), in the co-pending US patent filed on May 30, 2001, serial number 09/867970, titled "Vertical Grading of Interlaced Video in Frequency Domain" Disclosure in the application. Other classification rules, such as dedicated PIP processing, which optimizes the processing of picture-in-picture, can also be used. Therefore, it should be understood that the grading functions listed above are just a few examples, and other grading functions can also be implemented.
The purpose of establishing a query table with grading rules is to determine a grading rule for each possible complexity request, and each complexity request contains restrictions on the complexity of one or more resources. For example, a complexity request can be stated as bandwidth consumption must drop below 80%, and CPU processing power must drop below 90%.
The designer knows in advance what quality level each decoder functional block can be graded to under what circumstances. For example, in the case of activating PIP, a special PIP process can be used to reduce the complexity by a known percentage. For each complexity request, the designer tests various possible combinations of different hierarchical functions to obtain the best results. From all the rules that satisfy the complexity request, the designer chooses the rule that has the best subjective quality level for a particular degree of complexity. This process is repeated for each complexity request until the complexity query table is completed.
An exemplary lookup table 46 is shown in Figure 2. On the left side of Figure 2 are 14 different complexity requests. For example, complexity request "1" enumerates a memory capacity with an upper limit of 90% (that is, a reduction of at least 10%); complexity request "7" enumerates a bandwidth reduction with an upper limit of 85%; complexity request "14 ", enumerates the memory capacity reduction with the upper limit of 80%, the CPU reduction below 80%, and the activation of picture-in-picture (PIP). For each complexity request, there is a corresponding classification rule (listed on the right side of Figure 2). In this example, five different combinations of classification functions are used, namely, A=adaptive B-frame classification, I=IDCT classification, MC=motion compensation classification, E=embedded storage capacity reset; and P=PIP processing. For example, embedded storage capacity reset E is used for complexity request "1-4". In another case, a combination of adaptive B frame (A), IDCT classification (I), embedded storage capacity reset (E), and PIP processing (P) is used for complexity request "14".
It should be recognized that the table in Figure 2 is for example purposes only. Therefore, the number of entries, the range of complexity, the type of complexity (memory, bandwidth, CPU, function), the combination of various complexity, hierarchical function The combination of, the types of hierarchical functions, etc. are all for illustration, and should not be a limitation to the present invention. For example, the power consumption of the battery can also be included in the table. Furthermore, various variations of each grading function can also be adopted. For example, inverse discrete cosine transform IDCT classification should have more than one variant. Furthermore, the lookup table itself can be implemented in any known format, including databases, program codes, data objects, and so on.
4. Data-related rules The above-mentioned query table is composed of rules that have nothing to do with the data, which means that the required complexity level can be obtained according to the corresponding rules, regardless of the characteristics of the data. The present invention may also include a data-related rule system, which can dynamically select or change the classification rule based on the characteristics of the data processed by the decoder. The data-related rules can be combined into the above-mentioned query table, or processed by a separate rule body system and/or query table. Two examples of data-related rules used to reduce complexity are described below.
The first example involves the degradation of the output signal quality. In particular, if the output quality is poorer than the audience can accept, then another rule must be selected to improve the quality. Objective quality measurements (such as test data) can be used to determine acceptable output quality. If the result of the quality measurement is lower than the threshold, then the local resource controller can dynamically change the grading rule. For example, if the IDCT classification algorithm is used in the decoding loop to obtain a complexity level of 80%, the output quality is unacceptable; then, the system can reduce the size of the B picture by half, and at the same time, change the IDCT classification algorithm to further reduce The complexity of IDCT dynamically selects the rules in order to obtain the required complexity level of 80%. After the decoding loop has a much smaller complexity, the final picture size can be restored.
The second example involves further simplifying the selected classification rules according to the characteristics of the data. For example, if a sequence of DCT data is identified as "sparse" (that is, there is only a small amount of non-zero data), the IDCT classification rules can be instructed to provide a very low complexity. In this way, the originally required 80% complexity can be further reduced without any degradation in output quality.
It should be understood that the above two examples are only examples, and other data-related rules can also be implemented.
5. Resource Scalable Decoding System Referring to FIG. 3, a typical complexity scalable decoding system 12 is shown in the figure, which is composed of a decoder 24, a local resource controller (LRC) 14 and an offline design system 54. The decoder 24 includes a functional block 42, which constitutes a typical decoder (such as VLD, IQ, IDCT, and MC). Correspondingly, the decoder 24 receives the bit stream 58 and produces an output 60. The bit stream 58 may be discrete cosine transform (DCT) data, and the output is based on the pixels of the video image. The decoder 24 also includes a hierarchical function 40, which can be used to reduce the complexity of the functional block 42. Some examples of the classification function 40, including adaptive B frame classification, IDCT classification, embedded storage capacity reset, motion compensation classification, and PIP processing have been described above. It should be recognized that other classification functions can also be used. Therefore, they It also belongs to the scope of the present invention.
Rule 50 determines what hierarchical function is called to implement a scalable decoder. LRC14 determines rule 50 as follows. First, the LRC rule manager 44 receives a complexity request 56 from the resource manager. Then, the LRC rule manager 44 examines the lookup table 46 and selects the appropriate rule for the requested request 56. Referring to Figure 2, the example of the lookup table is as described above. Once the appropriate rule 50 is selected, it is submitted to the decoder 24 for implementation.
The LRC 14 also includes a data-related rule system 48. Based on the data 52 processed by the functional block 42 of the decoder 24, the data-related rule system 48 can replace or refine the selected rule 50. Examples of data-related rules have been described above, including: (1) comparing the output quality with a threshold, (2) determining whether the amount of non-zero data falls below the threshold.
As described above, the look-up table 46 and the data-related rules 48 are designed using the offline design system 54 in an offline state. The rules can be obtained through the process of trial and error handling. In this process, the designer subjectively examines the output quality, and/or through automated applications, such as the joint serial number 09/817981 submitted on March 27, 2001 As described in the pending application (with references).
It should be realized that the lookup table 46 has a broad interpretation and does not necessarily constitute a real table. The look-up table 46 may contain any system that can select classification rules based on the complexity request received. For example, table 46 may contain "if-then-else" or "case" program code statements.
6. For an example of the rules, see Figure 4, which shows a typical scalable decoding operation 80. First, the resource manager 18 receives resource information from the system, including PIP requests, memory status, and computing capability status. Then, the resource manager submits a complexity request related to resource constraints to the MPEG2 decoder LRC14. Next, LRC14 determines whether the complexity of the decoder needs to be reduced by 62. If it needs to be reduced, the complexity reduction rate of 64 is calculated, and the complexity classification function 66 in the decoder is called at the same time. If no complexity reduction is requested, LRC 14 asks if there is a PIP request 68. If there is a PIP request, the LRC14 will enter the PIP position 70 and at the same time call the hierarchical decoding function with B frame shrinkage 72. If no PIP request is made, LRC 14 will determine whether there is a request 74 to reduce storage capacity. If there is a request for memory capacity reduction, the embedded memory capacity reset grading function in the decoder 76 is called.
It should be understood that the systems and methods described herein can be implemented by hardware, software, or a combination of hardware and software. They can be implemented as any type of computer system, or other devices suitable for implementing the method. A typical combination of software and hardware is a general-purpose computer with a computer program. When the program is downloaded and run, it controls the computer system to implement the method described here. In addition, a dedicated computer may be used, which contains dedicated hardware for implementing one or more functional tasks of the present invention. The present invention can also be embedded in a computer program product, which has all the characteristics that can realize the methods and functions, and when downloaded to a computer system, it can realize these methods and functions. The terms computer program, software program, program, program product, or software in this article mean any expression of a set of instructions, using any computer language, code, or logo, with the intention of enabling the system to have information processing capabilities that can directly complete specific Function, or after completing the following two things or one of them, a specific function can be realized: (a) Transform into another language, code or logo, (b) Copy in a different medium.
The preferred embodiment of the present invention described above is only used for description and explanation, this embodiment is not exhaustive, and it is not used to limit the present invention to the specific form described. According to the above-mentioned instruction, the realization method can appear many different modifications and variations. It is obvious to those of ordinary skill in the art that these modifications and variations, as defined in the claims, also belong to the scope of the present invention. For example, although the description herein generally relates to MPEG-2 decoders, it is self-explanatory that the present invention can be applied to similar systems using MPEG-1, MPEG-4, H.26L, H.261 and H.263 standards.
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US8908763B2 | Cited by | United States of America | Applicant |
| CN102595193A | Cited by | China | Search report |
| US8948822B2 | Cited by | United States of America | Applicant |
| US8948270B2 | Cited by | United States of America | Applicant |
| CN110070878A | Cited by | China | Search report |
| CN103577269A | Cited by | China | Search report |
| CN102124724A | Cited by | China | Search report |
| US9462326B2 | Cited by | United States of America | Applicant |
| CN102017647A | Cited by | China | Search report |
| US8964828B2 | Cited by | United States of America | Applicant |
| US9565467B2 | Cited by | United States of America | Applicant |
8 members in 6 offices
Priority claims5
| Document | Office | Kind | Date |
|---|---|---|---|
| 09899880 | United States of America | – | |
| 89988001 | United States of America | A | |
| 89988001 | United States of America | A | |
| 09899880 | – | – | – |
| US20010899880 | – | – | – |
Members8
| Document | Office | Kind | |
|---|---|---|---|
| US2003007566A1 | United States of America | A1 | |
| WO03005729A1 | World Intellectual Property Organization (WIPO) | A1 | |
| KR20030061798A | Republic of Korea | A | |
| US6704362B2 | United States of America | B2 | |
| EP1407614A1 | European Patent Office (EPO) | A1 | |
| CN1522541AThis record | China | A | |
| JP2004534485A | Japan | A | |
| CN1247027C | China | C |
8 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Termination of patent right or utility modelEXPY | EXPY | |
| Succession or assignment of patent rightASS | ASS | |
| Transfer of patent application or patent right or utility modelC41 | C41 | |
| Succession or assignment of patent rightASS | ASS | |
| Transfer of patent application or patent right or utility modelC41 | C41 | |
| Grant of patent or utility modelGrantedC14 | C14 | |
| Entry into substantive examinationC10 | C10 | |
| PublicationC06 | C06 |
Numbers
- Publication
- 1522541
- Publication, DOCDB
- 1522541
- Publication, EPODOC
- CN1522541
- Application
- 28134699
- Application, DOCDB
- 02813469
- Application, EPODOC
- CN2002813469
Titles2
- Chinese
- 资源可分级解码
- English
- Resource hierarchical decoding
Classification
- CPC, 12
- H04N21/454
- H04N19/156
- H04N21/25808
- H04N21/2662
- H04N21/4402
- H04N21/4424
- H04N21/4516
- H04N19/61
- H04N19/127
- H04N19/42
- H04N19/423
- H04N19/90
- IPC, 20
- H04N19 102
- G06T9 00
- H04N19 127
- H04N19 136
- H04N19 146
- H04N19 156
- H04N19 176
- H04N19 196
- H04N19 423
- H04N19 426
- H04N19 44
- H04N19 59
- H04N19 625
- H04N19 91
- H04N21 258
- H04N21 2662
- H04N21 4402
- H04N21 442
- H04N21 45
- H04N21 454