Data processing device, data processing method, recording medium and program
Abstract
The purpose of the present invention is to provide images of various image qualities. The solution is that the enhancement information generating unit 11 generates most types of enhancement information used for image quality enhancement of the image (broadcasting image data) of the program broadcast. The integration unit 12 integrates by embedding one or more of the many types of promotion information generated by the promotion information generation unit 11 into broadcast video data, etc., and outputs an integrated signal. This integrated signal is transmitted via the transmitting unit 13.

Term
No projected expiry on record.
- Priority
- Filed
- Granted
- Today
77 claims: 77 independent, 0 dependent
- 1A8 B8 C8 D8 經濟部智慧財產局員工消費合作社印製 577201 j iii,, b. 23 六、申請專利範圍 第90104580號專利申請案 中文申請專利範圍修正本 民國92年5月23‘日修正 1 · 一種資料處理裝置,係具備: 提升資訊生成手段,用於生成提升資訊俾提升資料之 品質;及 埋入手段,用於將上述提升資訊埋入上述資料。 2 ·如申請專利範圍第1項之資料處理裝置,其中 上述提升資訊生成手段,係以上述資料品質提升用之 品質提升資料之預測値之預測使用之預測係數,作爲上述 提升資訊予以生成。 3 ·如申請專利範圍第2項之資料處理裝置,其中 上述提升資訊生成手段,係依每一特定等級生成上述 預測係數。 4 ·如申請專利範圍第3項之資料處理裝置,其中 上述提升資訊生成手段,係具備: 令成爲教師之教師資料中之注目之注目教師資料之等 級計算使用之等級抽頭,使用成爲學生之學生資料來予以 構成的等級抽頭構成手段; 依上述等級抽頭,算出上述注目教師資料之等級,進 行等級分類的等級分類手段; 爲預測上述注目教師資料,令和上述預測係數同時使 用之預測抽頭,使用上述學生資料來構成的預測抽頭構成 手段;及 本紙張尺度適用中國國家標準(CNS ) A4規格(210X297公釐) 77 ~~ (請先聞讀背面之注意事項再填寫本頁) 577201 k 5. 2S :摘 A8 B8 C8 D8 六、申請專利範圍 使用上述教師資料及預測抽頭,來算出上述每一等級 之上述預測係數的預測係數運算手段。 (請先閲脅背面之注意事項再填寫本頁) 5 ·如申請專利範圍第4項之資料處理裝置,其中 上述提升資訊生成手段,係生成多數種類之上述提升 資訊。 6 ·如申請專利範圍第5項之資料處理裝置,其中 上述提升資訊生成手段,係以相對於不同等級數之上 述預測係數,作爲上述多數種類之提升資訊予以生成。 7 ·如申請專利範圍第5項之資料處理裝置,其中 上述提升資訊生成手段,係以使用不同品質之上述學 生資料或教師資料所算出之多數種類之預測係數,作爲上 述多數種類之提升資訊予以生成。 8 .如申請專利範圍第5項之資料處理裝置,其中 上述提升資訊生成手段,係至少以上述預測係數,及 進行線形插補之資訊,作爲上述多數種類之提升資訊予以 生成。 經濟部智慧財產局員工消費合作社印製 9 ·如申請專利範圍第5項之資料處理裝置,其中· 上述提升資訊生成手段,係以使用不同構成之上述等 級抽頭或預測抽頭所算出之多數種類之預測係數,作爲上 述多數種類之提升資訊予以生成。· _ 1 〇 .如申請專利範圍第5項之資料處理裝置,其中 上述提升資訊生成手段,係對上述等級分類,以使用 不同方法算出之多數種類之預測係數,作爲上述多數種類 之提升資訊予以生成。 本紙張尺度適用中國國家標準(CNS ) A4規格(210X297公釐) 577201 气5.月2¾修正丨 _ 補充丨 六、申請專利範圍 1 1 ·如申請專利範圍第5項之資料處理裝置,其中 (請先閲讀背面之注意事項再填寫本頁) 上述提升資訊生·成手段,係以提升上述資料品質之品 質提升資料之預測値之預測使用之,表示資料等級的等級 碼,作爲上述提升資訊予以生成。 1 2 ·如申請專利範圍第1 1項之資料處理裝置,其 中 上述提升資訊生成手段,係具備: 令成爲學習教師之教師資料中之注目之注目教師資料 之預測使用之預測抽頭,使用成爲學習學生之學生資料予 以構成的預測抽頭構成手段; 將進行學習所算出之,上述各等級碼之預測係數予以 記憶的預測係數記憶手段; 使用上述預測抽頭及預測係數,算出上述注目教師資 料之預測値的預測運算手段;及 檢測上述注目教師資料之預測値爲最小之上述預測係 數之等級碼的等級碼檢測手段; 經濟部智慧財產局員工消費合作社印製 以上述等級碼檢測手段檢測之上述等級碼,作爲上述 提升資訊予以輸出。 1 3 ·如申請專利範圍第1 1項之資料處理裝置,其 中 . 上述提升資訊生成手段,係具備: 令成爲學習教師之教師資料中之注目之注目教師畜料 之等級計算使用之等級抽頭,使用上述教師資料予以構成 的等級抽頭構成手段;及 本紙張尺度適用中國國家標準(CNS ) A4規格(210X297公釐) 577201 A8 B8 C8 D8 六、申請專利範圍 依上述等級抽頭,計算上述注目教師資料之等級進行 等圾分類的等級分類手段; (請先閱讀背面之注意事項再填寫本頁) 以上述等級分類手段算出之等級對應之等級碼,作爲 上述提升資訊予以輸出。 1 4 ·如申請專利範圍第1項之資料處理裝置,其中 上述埋入手段,係使用上述資料具有之能量偏移,使 上述資料及提升資訊可回復至原來地,於上述資料埋入上 述提升資訊。 1 5 ·如申請專利範圍第1項之資料處理裝置,其中 上述埋入手段,係進行頻譜擴散,於上述資料埋入上 述提升資訊。 1 6 ·如申請專利範圍第1項之資料處理裝置,其中 上述埋入手段,係進行將上述資料之1位元以上,變 更爲上述提升資訊,俾於上述資料埋入上述提升資訊。 1 7 ·如申請專利範圍第1項之資料處理裝置,其中 上述資料,係影像資料, 上述提升資訊,係提升上述影像資料之畫質的資訊。· 經濟部智慧財產局員工消费合作社印製 1 8 · —種資料處理方法,係具備: 提升資訊生成步驟,用於生成提升資訊俾提升資料之 品質;及 . 埋入步驟,用於將上述提升資訊埋入上述資料。 1 9 · 一種記錄媒體’係電腦執行之程式被記錄之記 錄媒體,其特徵爲: · •具備用於生成提升資訊俾提升資料之品質的提升資訊 本紙張尺度適用中國國家標準(CNS gt;A4規格(210X297公釐) 577201 A8 B8 C8 D8 六、申請專利範圍 生成步驟;及用於將上述提升資訊埋入上述資料的埋入步 驟,之程式被記錄之記錄媒體。 (請先閱背背面之注意事項再填寫本頁) 2 0 · —種資料處理裝置,係針對資料,處理埋入有 提升該資料品質用之提升資訊的埋入資料之資料處理裝置 ;其特徵爲具備: 由上述埋入資料,抽出上述提升資訊的抽出手段;及 針對上述資料之品質,使用上述提升資訊予以提升的 提升手段。 2 1 ·如申請專利範圍第2 0項之資料處理裝置,其 中 上述提升資訊,係上述資料品質提升用之品質提升資 料之預測値之預測使用之預測係數; 上述提升手段,係使用上述資料及預測係數,來算出 上述品質提升資料之預測値。 2 2 .如申請專利範圍第2 1項之資料處理裝置,其 中 上述提升資訊,係依各特定等級算出之上述預測係數 經濟部智慧財產局員工消費合作社印製 » 上述提升手段,係使用上述資料及各等級之預測係數 ,來算出上述品質提升資料之預測値。 2 3 ·如申請專利範圍第2 2項之資料處理裝置,其 中 上述提升手段,係具備: 令被注目之上述品質提升資料之注目品質提升資料之 本^張尺度適用_國國家標準(€奶)八4規格(21(^297公釐) 77^ : 577201 A8 B8 C8 D8 六、申請專利範圍 等級計算使用之等級抽頭,使用上述資料來予以構成的等 級抽頭構成手段; 依上述等級抽頭,算出上述注目品質提升資料之等級 ,進行等級分類的等級分類手段; 爲預測上述注目品質提升資料,令和上述預測係數同 時使用之預測抽頭,使用上述資料來予以構成的預測抽頭 構成手段;及 使用上述注目品質提升資料之等級之上述預測係數’ 及上述預測抽頭,來算出上述注目品質提升資料之預測値 的預測手段。 2 4 ·如申請專利範圍第2 0項之資料處理裝置,其 中 上述提升資訊,係上述資料品質提升用之品質提升資 料之預測値之預測使用之特定之各等級之預測係數,之表 示該等級的等級碼, 上述提升手段,係使用上述資料及上述等級碼對應之 預測係數,來算出上述品質提升資料之預測値。 經濟部智慧財產局員工消費合作社印製 (請先閱讀背面之注意事項再填寫本頁) 2 5 ·如申請專利範圍第2 4項之資料處理裝置,其 中 上述提升手段,係具備: . 爲預測被注目之上述品質提升資料之注目品質提升資 料,令和上述預測係數同時被使用之預測抽頭,使用上述 資料予以構成的預測抽頭構成手段;及 · 令上述注目品質提升資料之預測値,使用作爲上述提 本紙張尺度適用中國國家標準(CNS ) A4規格(210X297公釐) TrZ : ~ 577201 A8 B8 C8 D8 經濟部智慧財產局員工消費合作社印製 六、申請專利範圍 升資訊之等級碼對應之預測係數,及上述預測抽頭,來算 出的預測手段。 2 6 ·如申請專利範圍第2 3項之資料處理裝置,其 中 於上述埋入資料,埋入有多數種類之上述提升資訊。 2 7 ·如申請專利範圍第2 6項之資料處理裝置,其 中 於上述埋入資料,係以不同等級數之上述預測係數, 作爲上述多數種類之提升資訊被埋入。 2 8 .如申請專利範圍第2 6項之資料處理裝置,其 中 上述預測係數,係使用成爲學生之學生資料,及成爲 教師之教師資料予以生成, 於上述埋入資料,係使用不同品質之上述學生資料或 教師資料算出之多數種類之預測係數,作爲上述多數種類 之提升資訊被埋入。 2 9 ·如申請專利範圍第2 6項之資料處理裝置,其 中 於上述埋入資料,至少上述預測係數,及進行線形插 補之資訊,作爲上述多數種類之提升資訊被埋入。 3 0 ·如申請專利範圍第2 6項之資料處理裝置,其 中 於上述埋入資料,係以使用不同構成之上述等級抽頭 或預測抽頭所算出之多數種類之預測係數,作爲上述多數 本紙張尺度適用中國國家標準(CNS ) A4規格(210X297公釐) : '^ (請先閲讀背面之注意事項再填寫本頁) C 訂 577201 A8 B8 C8 D8 六、申請專利範圍 種類之提升資訊被埋入。 (請先閣讀背面之注意事項再填寫本頁) 3 1 ·如申請專利範圍第2 6項之資料處理裝置,其 中 於上述埋入資料,係令上述等級分類,以不同方法算 出之多數種類之預測係數,作爲上述多數種類之提升資訊 被埋入。 3 2 .如申請專利範圍第2 6項之資料處理裝置,其 中 另具備:由上述多數種類之提升資訊,選擇上述資料 品質提升使用者的提升資訊選擇手段。 3 3 ·如申請專利範圍第2 0項之資料處理裝置,其 中 上述抽出手段,係利用上述資料具有之能量偏移,由 上述埋入資料抽出上述提升資訊。 3 4 ·如申請專利範圍第2 0項之資料處理裝置,其 中 經濟部智慧財產局員工消費合作社印製 上述抽出手段,係進行逆頻譜擴散,由上述埋入資料. 抽出上述提升資訊。 3 5 .如申請專利範圍第2 0項之資料處理裝置,其 中 上述抽出手段,係以上述埋入資料之1位元以上,作 爲上述提升資訊予以抽出。 3 6 ·如申請專利範圍第2 0項之資料處理裝置,其 中 本紙張尺度適用中國國家標準(CNS ) A4規格(210X297公釐) _ 8 — 577201 Α8 Β8 C8 D8 ϋ修正 補充 •、申請專利範圍 上述資料,係影像資料, 上述提升資訊,係提升上述影像資料之畫質的資訊。 (請先閲讀背面之注意事項再填寫本頁) 3 7 · —種資料處理方法,係針對資料,處理埋入有 提升該資料品質用之提升資訊的埋入資料之資料處理方法 ;其特徵爲具備: 由上述埋入資料,抽出上述提升資訊的抽出步驟;及 針對上述資料之品質,使用上述提升資訊予以提升的 提升步驟。 3 8 · —種記錄媒體,係針對資料,處理埋入有提升 該資料品質用之提升資訊的埋入資料之,電腦執行之程式 被記錄的記錄媒體;其特徵爲具備: 由上述埋入資料,抽出上述提升資訊的抽出步驟;及 針對上述資料之品質,使用上述提升資訊予以提升的 提升步驟,的程式被記錄。 3 9 · —種資料處理裝置,係具備: 資料品質提升用之多數種類之提升資訊之生成用的提 升資訊生成手段;及 經濟部智慧財產局員工消費合作社印製 送信上述資料,及1種類以上之上述提升資訊的送信 手段。 4 0 ·如申請專利範圍第3 9項之資料處理裝置,其 中 另具備:由上述多數種類之提升資訊之中,選擇與上 述資料伺時送信者的提升資訊選擇手段。 · 4 1 ·如申請專利範圍第4 0項之資料處理裝置,其 本紙張尺度適用中國國家標準(CNS ) Α4規格(2丨0 gt;lt;297公釐) -9 577201 8 8 8 8 ABCD 經濟部智慧財產局員工消費合作社印製 六、申請專利範圍 中 上述提升資訊選擇手段,係依受信上述資料之受信裝 置之要求,來選擇上述提升資訊。 4 2 .如申請專利範圍第4 1項之資料處理裝置,其 中 另具備:依上述提升資訊選擇手段選擇之上述提升資 訊,進行計費處理的計費手段。 4 3 ·如申請專利範圍第3 9項之資料處理裝置,其 中 上述提升資訊生成手段,係至少以上述資料品質提升 用之品質提升資料之預測値之預測使用之預測係數,作爲 上述提升資訊予以生成。 4 4 ·如申請專利範圍第4 3項之資料處理裝置,其 中 上述提升資訊生成手段,係依每一特定等級生成上述 預測係數。 4 5 ·如申請專利範圍第4 4項之資料處理裝置,其 中 上述提升資訊生成手段,係具備: 令成爲教師之教師資料中之注目之注目教師資料之等 級r卜算使用之等級抽頭,使用成爲學生之學生資料來予以 構成的等級抽頭構成手段; 依上述等級抽頭,算出上述注目教師資料之等級,進 行等級分類的等級分類手段; 本矣氏張尺度適用中國國家標準(CNS ) A4規格(210X297公釐) ΤΤτΓ : ' -- 裝 ---訂------ (請先閲脅背面之注意事項再填寫本頁) 577201 德 A8 B8 C8 D8 經濟部智慧財產局員工消費合作社印製 六、申請專利範圍 爲預測上述注目教師資料,令和上述預測係數同時使 用之預測抽頭,使用.上述學生資料來構成的預測抽頭構成 手段;及 使用上述教師資料及預測抽頭,來算出上述每一等級 之上述預測係數的預測係數運算手段。 4 6 ·如申請專利範圍第4 5項之資料處理裝置,其 中 上述提升資訊生成手段,係以相對於不同等級數之上 述預測係數,作爲上述多數種類之提升資訊予以生成。 4 7 .如申請專利範圍第4 5項之資料處理裝置,其 中 上述提升資訊生成手段,係以使用不同品質之上述學 生資料或教師資料所算出之多數種類之預測係數,作爲上 述多數種類之提升資訊予以生成。 4 8 ·如申請專利範圍第4 5項之資料處理裝置,其 中 上述提升資訊生成手段,係至少以上述預測係數,及 進行線形插補之資訊,作爲上述多數種類之提升資訊予以 生成。 4 9 ·如申請專利範圍第4 5項之資料處理裝置,其 中 上述提升資訊生成手段,係以使用不同構成之上述等 級抽頭或預測抽頭所算出之多數種類之預測係數,作爲上 述多數種類之提升資訊予以生成。 本紙張尺度逋用中國國家標準(CNS ) A4規格(2iOX 297公釐) _ H (請先閱讀背面之注意事項再填寫本頁) 裝· 、1T 577201 92· 5· 23 年月日 補亦t A8 B8 C8 D8 夂、申請專利範圍 5 0 ·如申請專利範圍第4 5項之資料處理裝置,其 中 (請先閲背背面之注意事項再填寫本頁) 上述提升資訊生成手段,係對上述等級分類,以使用 不同方法算出之多數種類之預測係數,作爲上述多數種類 之提升資訊予以生成。 5 1 ·如申請專利範圍第3 9項之資料處理裝置,其 中 上述送信手段,係使用上述資料具有之能量偏移,使 上述資料及提升資訊可回復至原來地,於上述資料埋入上 述提升資訊,並送信上述資料及1種類以上之提升資訊。 5 2 ·如申請專利範圍第3 9項之資料處理裝置,其 中 i述送信手段,係進行頻譜擴散,於上述資料埋入上 述提升資訊’並送信上述資料及1種類以上之提升資訊。 5 3 ·如申請專利範圍第3 9項之資料處理裝置,其 中 經濟部智慧財產局員工消費合作社印製 上述送信手段,係進行將上述資料之1位元以上,變 更:爲述提升資訊,俾於上述資料埋入上述提升資訊,並 送信上述資料及1種類以上之提升資訊。 5 4 ·如申請專利範圍第3 9項之資料處理裝置,其 中 上述送信手段,係送信上述資料及上述多數種類之提 升資訊之全部。 · 5 5 .如申請專利範圍第3 9項之資料處理裝置,其 -12- 本紙張用t國國家樣準(CNS ) A4· ( 210χ297公董) A8 B8 C8 D8 六、申請專利範圍 中 上述資料,係影像資料, 上述提升資訊,係提升上述影像資料之畫質的資訊。 5 6 · —種資料處理方法,係具備: 提升資訊生成步驟,用於生成多數種類之提升資訊俾 提升資料之品質;及 送信步驟,用於送信上述資料及1種類以上之提升資 訊。 5 7 . —種記錄媒體,係電腦執行之程式被記錄之記 錄媒體,其特徵爲: 具備:用於生成多數種類之提升資訊俾提升資料之品 質的提升資訊生成步驟;及用於送信上述資料、與1種類 以上之提升資訊的送信步驟,之程式被記錄之記錄媒體。 5 8 . —種資料處理裝置,係受信資料、及該資料之 品質提升用之1種類以上之提升資訊予以處理的資料處理 裝置,其特徵爲具備: 受信上述資料及1種類以上之提升資訊的受信手段;_ . 令上述資料之品質,使用上述1種類以上之提升資訊 中之任一予以提升的提升手段;及 依上述資料品質提升使用之上述提升資訊,進行計瞢 處理的計費手段。 5 9 ·如申請專利範圍第5 8項之資料處理裝置,其 中 上述受信手段,係受信多數種類之提升資訊, 本紙張尺度適用中國國家標準(CNS ) A4規格(210X297公釐) _ 13 · (請先閱讀背面之注意事項再填寫本頁) -裝. 訂 經濟部智慧財產局員工消費合作社印製 577201 Α8 Β8 C8 D8 六、申ϋ利範圍 另具備:由上述多數種類之提升資訊之中,選擇上述 資料品質提升使用者的提升資訊選擇手段。 6 0 ·如申請專利範圍第5 9項之資料處理裝置,其 中 上述提升資訊選擇手段,係依使用者之要求,來選擇 上述提升資訊。 6 1 ·如申請專利範圍第5 8項之資料處理裝置,其 中 另具備:對送信上述資料及1種類以上之提升資訊的 送信裝置要求,上述資料品質提升使用之上述提升資訊, 的要求手段, 上述受信手段’係受信上述送信裝置依要求手段之要 求而送信之上述提升資訊。 6 2 ·如申請專利範圍第5 8項之資料處理裝置,其 中 上述提升資訊,係上述資料品質提升用之品質提升資 料之預測値之預測使用之預測係數, 經濟部智慧財產局員工消費合作社印製 (請先閲部背面之注意事項再填寫本頁) 上述提升手段,係使用上述資料及預測係數,來算出 上述品質提升資料之預測値。 6 3 ·如申請專利範圍第6 2項之資料處理裝置,其 中 上述提升資訊,係依各特定等級算出之上述預測係數 y 上述提升手段,係使用上述資料及各等級之預測係數 本紙張尺度適用中國國家標準(CNS ) Α4規格(210Χ297公釐) _ 14_ : 577201 A8 B8 C8 D8 々、申請專利範圍 ’來算出上述品質提升資料之預測値。 6 4 ·如申請專.利範圍第6 3項之資料處理裝置,其 中 上述提升手段,係具備: 令被注目之上述品質提升資料之注目品質提升資料之 等級計算使用之等級抽頭.,使用上述資料來予以構成的等 級抽頭構成手段; 依上述等級抽頭,算出上述注目品質提升資料之等級 ,進行等級分類的等級分類手段; 爲預測上述注目品質提升資料,令和上述預測係數同 時使用之預測抽頭,使用上述資料來予以構成的預測抽頭 構成手段;及 使用上述注目品質提升資料之等級之上述預測係數, 及上述預測抽頭,來算出上述注目品質提升資料之預測値 的預測手段。 6 5 ·如申請專利範圍第6 4項之資料處理裝置,其 中 上述受信手段,係受信多數種類之上述提升資訊。 6 6 ·如申請專利範圍第6 5項之資料處理裝置,其 Φ . 上述受信手段,係以不同等級數之上述預測係數,作 爲上述多數種類之提升資訊予以受信。 6 7 ·如申請專利範圍第6 5項之資料處理裝置,其 中 本紙張尺度適用中國國家標準(CNS ) A4規格(210X297公瘦) 镛裝-- (請先閲讀背面之注意事項再填寫本頁) 、1T 經濟部智慧財產局員工消費合作社印製 577201 A8 B8 C8 D8 六、申請專利範圍 上述預測係數,係使用成爲學生之學生資料,及成爲 教師之教師資料予以生成, 上述受信手段,係以使用不同品質之上述學生資料或 教師資料算出之多數種類之預測係數,作爲上述多數種類 之提升資訊予以受信。 6 8 ·如申請專利範圍第6 5項之資料處理裝置,其 中 上述受信手段,係至少以上述預測係數,及進行線形 插補之資訊,作爲上述多數種類之提升資訊予以受信。 6 9 ·如申請專利範圍第6 5項之資料處理裝置,其 中 上述受信手段,係以使用不同構成之上述等級抽頭或 預測抽頭所算出之多數種類之預測係數,作爲上述多數種 類之提升資訊予以受信。 7 〇 ·如申請專利範圍第6 5項之資料處理裝置,其 中 經濟部智慧財產局員工消費合作社印製 (請先閲脅背面之注意事項再填寫本頁) 上述受信手段,係對上述等級分類,以使用不同方法. 算出之多數種類之預測係數,作爲上述多數種類之提升資 訊予以受信。 7 1 ·如申請專利範圍第5 8項之資料處理裝置,其 中 上述受信手段,係受信,在上述資料埋入有上述1種 類以上之提升資訊的埋入資料, · 另具備:由上述埋入資料抽出上述提升資訊的抽出手 本紙張尺度適用中國國家標準(CNS ) A4規格(210X297公釐) · 16 · 57珊一 jj:: A8 B8 C8 D8 補充 穴、申請專利乾圍 段。 7 2 .如申請專.利範圍第7 1項之資料處理裝置,其 (請先閲讀背面之注意事項再填寫本頁) 中 上述抽出手段,係使用上述資料具有之能量偏移,由 上述埋入資料抽出上述提升資訊。 7 3 ·如申請專利範圍第7 1項之資料處理裝置,其 中 上述抽出手段,係進行逆頻譜擴散,由上述埋入資料 抽出上述提升資訊。 7 4 ·如申請專利範圍第7 1項之資料處理裝置,其 中 上述抽出手段,係以上述埋入資料之1位元以上,作 爲上述提升資訊予以抽出。 7 5 ·如申請專利範圍第5 8項之資料處理裝置,其 中 上述資料,係影像資料, 上述提升資訊,係提升上述影像資料之畫質的資訊。. 經濟部智慧財產局員工消費合作社印製 7 6 _ —種資料處理方法,係受信資料、及該資料之 品質提升用之1種類以上之提升資訊予以處理的資料處理 方法,其特徵爲具備: . 受信上述資料及1種類以上之提升資訊的受信步驟; .令上述資料之品質,使用上述1種類以上之提升資訊 中之任一予以提升的提升步驟;及 ’ 依上述資料品質提升使用之上述提升資訊,進行計費 本紙張尺度適用中國國家標準(CNS ) A4規格(210X 297公釐) _ ] 7 _ A8 B8 C8 D8 577201 抵修正丨 補充 六、申請專利範圍 處理的計費步驟。 7 7 · —種記錄·媒體,係受信資料、及該資料之品質 提升用之1種類以上之提升資訊予以處理的,電腦執行之 程式被記錄的記錄媒體,其特徵爲具備: 受信上述資料及1種類以上之提升資訊的受信步驟; 令上述資料之品質,使用上述1種類以上之提升資訊 中之任一予以提升的提升步驟;及 依上述資料品質提升使用之上述提升資訊,進行計費 處理的計費步驟,之程式被記錄的記錄媒體。 ^裝-- (請先聞讀背面之注意事項再填寫本頁) *11 經濟部智慧財產局員工消費合作社印製 本紙張尺度適用中國國家標準(CNS ) A4規格(210X297公釐)
345 paragraphs, as filed
Data processing device, data processing method, recording medium and program
The present invention relates to a data processing device, a data processing method, a recording medium and a program, and particularly to a data processing device, a data processing method, and a recording medium and a program that can provide images of various image qualities.
In recent years, the domestic provision of paid program broadcasting services such as cable television broadcasting or digital satellite broadcasting has gradually taken shape.
<p>However, paid program broadcasting services are generally charged based on audiovisual programs or actual audiovisual programs. If the video quality of the program can be changed according to the audiovisual fee paid by the user, more sophisticated services can be provided.</p><p>In view of this, the purpose of the present invention is to provide images of various image qualities.</p>
<p>The first data processing device of the present invention is characterized by having: an enhanced information generation means for generating enhanced information to improve the quality of the data; and an embedding means for embedding the above-mentioned enhanced information into the above-mentioned data.</p><p>The above-mentioned improvement information generating means can generate the above-mentioned improvement information as the above-mentioned improvement information using the prediction coefficient used for the prediction of the predicted value of the quality improvement data for the above-mentioned data quality improvement.</p><p>The above-mentioned promotion information generating means can generate the above-mentioned prediction coefficient according to each specific level.</p><p>The above-mentioned means of generating information for promotion can be set up: the grade tap used for the grade calculation of the noticeable teacher data in the teacher data of the teacher, and the grade tap structure method constructed by using the student data of the student; according to the above grade tap, Calculate the level of the above noted teacher data and classify the level classification method; in order to predict the above noted teacher data, order the prediction taps used together with the above prediction coefficients to use the above student data to construct the prediction tap composition method; and use the above teachers The data and prediction taps are used to calculate the prediction coefficient calculation method of the prediction coefficient of each level.</p><p>The above-mentioned promotion information generating means can generate most types of the above-mentioned promotion information.</p><p>The above-mentioned promotion information generating means can be generated as the above-mentioned most types of promotion information with respect to the above-mentioned prediction coefficients of different levels.</p><p>The above-mentioned promotion information generation means can use the most types of prediction coefficients calculated from the above-mentioned student data or teacher data of different qualities as the above-mentioned most types of promotion information to be generated.</p><p>The above-mentioned promotion information generating means is to generate at least the above-mentioned prediction coefficients and linear interpolation information as the above-mentioned most types of promotion information.</p><p>The above-mentioned promotion information generating means can use the various types of prediction coefficients calculated by the above-mentioned level taps or prediction taps of different compositions to generate the above-mentioned many types of promotion information.</p><p>The above-mentioned promotion information generating means is to classify the above-mentioned grades, and generate prediction coefficients of most types calculated using different methods as the above-mentioned most types of promotion information.</p><p>The above-mentioned improvement information generating means is used to predict the predicted value of the quality improvement data to improve the quality of the above-mentioned data, and a grade code indicating the level of the data is generated as the above-mentioned improvement information.</p><p>The above-mentioned means of enhancing information generation can be set up: the forecasting tap used for predicting the noticeable teacher data in the teacher data of the learning teacher, and the forecasting tap composition method composed of the student data of the learning student; the calculation of the learning is calculated The predictive coefficient memory means for memorizing the predictive coefficients of the aforementioned grade codes; the predictive calculation means for calculating the predictive value of the noted teacher data using the predictive taps and predictive coefficients; and detecting the minimum predictive value of the noted teacher data The level code detection means of the level code of the prediction coefficient; the level code detected by the level code detection means is output as the above-mentioned promotion information.</p><p>The above-mentioned enhancement information generation means can be set: the grade tap used in the grade calculation of the noticeable teacher data in the teacher data of the learning teacher, the grade tap composition method constructed by the above-mentioned teacher data; and the calculation based on the above grade tap The grade classification method of the above mentioned teacher materials is classified as rubbish; the grade code corresponding to the grade calculated by the above grade classification method is output as the above-mentioned promotion information.</p><p>The above-mentioned embedding means uses the energy offset of the above-mentioned data, so that the above-mentioned data and the promotion information can be restored to the original place, and the above-mentioned promotion information is integrated into the above-mentioned data.</p><p>The above-mentioned embedding means can carry out spectrum diffusion, and embed the above-mentioned enhancement information in the above-mentioned data.</p><p>The above-mentioned embedding means can change more than one bit of the above-mentioned data to the above-mentioned promotion information, so as to embed the above-mentioned promotion information in the above-mentioned data.</p><p>The above-mentioned data is image data, and the above-mentioned enhancement information is information that improves the image quality of the above-mentioned image data.</p><p>The first data processing method of the present invention is characterized by comprising: an enhanced information generation step for generating enhanced information to improve the quality of the data; and an embedding step for embedding the above-mentioned enhanced information into the above-mentioned data.</p><p>The first recording medium of the present invention is characterized in that it has an enhancement information generating step for generating enhancement information to enhance the quality of the data; and an embedding step for embedding the enhancement information into the data, and the program is recorded.</p><p>The first program of the present invention is characterized by having: an enhancement information generating step for generating enhancement information to enhance the quality of the data; and an embedding step for embedding the enhancement information into the data.</p><p>The second data processing device of the present invention is characterized by having: an extraction means for extracting the above-mentioned improvement information from the above-mentioned embedded data; and an improvement means for improving the quality of the above-mentioned data by using the above-mentioned improvement information.</p><p>The above improvement information may be the prediction coefficient used for the prediction of the predicted value of the quality improvement data for the above data quality improvement; in this case, the above improvement means may use the data and the prediction coefficient to calculate the prediction value of the above quality improvement data.</p><p>The above-mentioned improvement information may be the above-mentioned prediction coefficient calculated according to each specific level, and the above-mentioned improvement means may use the above-mentioned data and the prediction coefficient of each level to calculate the predicted value of the above-mentioned quality improvement data.</p><p>The above improvement means can be set: the level tap used for the level calculation of the noticed quality improvement data of the above mentioned quality improvement data, the level tap composition means constructed by using the above data; the above mentioned quality improvement is calculated based on the above level taps The grade of the data is a grade classification method for grade classification; in order to predict the above-mentioned attention-quality improvement data, the prediction taps used together with the above-mentioned prediction coefficients are used to construct the prediction tap composition method of the above-mentioned data; and the above-mentioned attention quality improvement data is used The prediction coefficients of the above-mentioned levels and the above-mentioned prediction taps are used to calculate the prediction means of the predicted value of the above-mentioned attention-quality improvement data.</p><p>The above improvement information can be the specific prediction coefficients of each level used in the prediction of the prediction value of the quality improvement data for the above data quality improvement, which represents the level code of the level. The above improvement means can be set to use the above data and the above The prediction coefficient corresponding to the grade code is used to calculate the predicted value of the above-mentioned quality improvement data.</p><p>The above-mentioned enhancement means can be set up: the notable quality-improving data for predicting the above-mentioned quality-improving data to be noticed, so that the prediction taps that are used simultaneously with the above prediction coefficients are formed by using the above-mentioned data to construct the forecasting tap composition means; The prediction value of the promotion data is calculated by using the prediction coefficient corresponding to the level code of the promotion information and the prediction tap above.</p><p>In the above-mentioned embedded data, there are many types of above-mentioned promotion information embedded.</p><p>In the above-mentioned embedded data, the above-mentioned prediction coefficients of different levels can be embedded as the above-mentioned most types of promotion information.</p><p>The above prediction coefficients can be generated by using the data of students who become students and teachers who become teachers. In this case, the above embedded data are most types of prediction coefficients calculated using the above student data or teacher data of different qualities as Most of the above types of promotion information are buried.</p><p>The above-mentioned embedded data is at least the above-mentioned prediction coefficient and the information for linear interpolation, which is embedded as the above-mentioned most types of promotion information.</p><p>The above-mentioned embedded data is embedded as the above-mentioned most types of promotion information using the prediction coefficients of most types calculated using the above-mentioned grade taps or prediction taps of different compositions.</p><p>The above-mentioned embedded data is based on the above-mentioned level classification, and the prediction coefficients of most types calculated by different methods are embedded as the promotion information of the above-mentioned most types.</p><p>The second data processing device of the present invention is further provided with: from the above-mentioned most types of improvement information, the user selects the above-mentioned improvement information selection means for improving the quality of the data.</p><p>The aforementioned extraction means can use the energy offset of the aforementioned data to extract the aforementioned boost information from the aforementioned embedded data.</p><p>The above-mentioned extraction means can perform inverse spectrum diffusion, and extract the above-mentioned enhancement information from the above-mentioned embedded data.</p><p>The aforementioned extraction means can extract more than one bit of the aforementioned embedded data as the aforementioned enhancement information.</p><p>The above-mentioned data is image data, and the above-mentioned improvement information can be set as information for improving the image quality of the above-mentioned image data.</p><p>The second data processing method of the present invention is characterized by comprising: an extraction step of extracting the above-mentioned improvement information from the above-mentioned embedded data; and an improvement step of using the above-mentioned improvement information to improve the quality of the above-mentioned data.</p><p>The second recording medium of the present invention is characterized by having: an extraction step of extracting the above-mentioned improvement information from the above-mentioned embedded data; and an improvement step of using the above-mentioned improvement information for the quality of the above-mentioned data, and a program is recorded.</p><p>The second program of the present invention is characterized by having: an extraction step of extracting the above-mentioned improvement information from the above-mentioned embedded data; and an improvement step of using the above-mentioned improvement information to improve the quality of the above-mentioned data.</p><p>The third data processing device of the present invention is characterized by having: promotion information generating means for generating most types of promotion information for data quality improvement; and means for transmitting the above-mentioned data and one or more types of the above-mentioned promotion information.</p><p>The third data processing device of the present invention can be configured to select the promotion information selection means of the person sending the data at the same time from among the above-mentioned types of promotion information.</p><p>The above-mentioned promotion information selection means can select the above-mentioned promotion information according to the requirements of the trusted device that trusts the above-mentioned data.</p><p>The third data processing device of the present invention may be additionally provided with: a charging means for performing charging processing based on the above-mentioned promotion information selected by the above-mentioned promotion information selection means.</p><p>The above-mentioned improvement information generating means can be generated as the above-mentioned improvement information by at least the prediction coefficient used for the prediction of the predicted value of the quality improvement data for the above-mentioned data quality improvement.</p><p>The above-mentioned promotion information generating means generates the above-mentioned prediction coefficient according to each specific level.</p><p>In the above-mentioned means of enhancing information generation, it is possible to set: the grade tap used for the grade calculation of the noticeable teacher data in the teacher data of the teacher, and the grade tap composition method using the student data of the student to construct; according to the above grade tap , Calculate the level of the noted teacher data, and perform the level classification method; in order to predict the noted teacher data, order the prediction taps used together with the above prediction coefficients to use the above student data to form the prediction tap composition method; and use the above Teacher data and prediction taps are used to calculate the prediction coefficient calculation method of the above prediction coefficient for each level.</p><p>The above-mentioned promotion information generating means is generated by using the above-mentioned prediction coefficients relative to the number of different levels as the above-mentioned most types of promotion information.</p><p>The above-mentioned promotion information generation means can use the most types of prediction coefficients calculated from the above-mentioned student data or teacher data of different qualities as the above-mentioned most types of promotion information to be generated.</p><p>The above-mentioned promotion information generating means can at least use the above-mentioned prediction coefficients and linear interpolation information as the above-mentioned most types of promotion information to be generated.</p><p>The above-mentioned promotion information generation method is to generate the above-mentioned many kinds of promotion information by using the prediction coefficients of the majority types calculated by using the above-mentioned level taps or prediction taps of different compositions.</p><p>The above-mentioned promotion information generation means can classify the above-mentioned grades, and generate prediction coefficients of most types calculated using different methods as the above-mentioned most types of promotion information.</p><p>The above-mentioned sending means can use the energy offset of the above-mentioned data, so that the above-mentioned data and promotion information can be restored to the original place, embed the above-mentioned promotion information in the above-mentioned data, and send the above-mentioned data and more than one type of promotion information.</p><p>The above-mentioned sending means can carry out spectrum diffusion, embed the above-mentioned promotion information in the above-mentioned data, and send the above-mentioned data and more than one type of promotion information.</p><p>The above-mentioned sending means can change one bit or more of the above-mentioned data to the above-mentioned promotion information, so as to embed the above-mentioned promotion information in the above-mentioned data, and send the above-mentioned data and promotion information of L types or more.</p><p>The above-mentioned means of sending can send all of the above-mentioned data and most of the above-mentioned types of promotion information.</p><p>The above-mentioned data is image data, and the above-mentioned improvement information may be information for improving the image quality of the above-mentioned image data.</p><p>The third data processing method of the present invention is characterized by having: an enhanced information generating step for generating most types of enhanced information to improve the quality of the data; and a sending step for sending the above-mentioned data and more than one type of enhanced information.</p><p>The third recording medium of the present invention is characterized by having: an enhancement information generating step for generating multiple types of enhancement information to improve the quality of the data; and a sending step for transmitting the above-mentioned data and one or more types of enhancement information, which The program is recorded.</p><p>The third program of the present invention is characterized by having: an enhancement information generating step for generating multiple types of enhancement information to improve the quality of the data; and a sending step for transmitting the above data and more than one type of enhancement information.</p><p>The fourth data processing device of the present invention is characterized by having: a trusted means for trusting the above-mentioned data and more than one type of promotion information; and an promotion means for enhancing the quality of the above-mentioned data by using any one of the above-mentioned promotion information of more than one type ; And a billing method for billing processing based on the above-mentioned improvement information used for the above-mentioned data quality improvement.</p><p>In the above-mentioned trusted means, most types of promotion information can be trusted. In this case, the fourth data processing device of the present invention may additionally have: among the above-mentioned most types of promotion information, select the above-mentioned data quality promotion user's promotion information selection means .</p><p>The above-mentioned promotion information selection means can be selected according to the user's request.</p><p>The fourth data processing device of the present invention may additionally include: a request means for a sending device that sends the above data and more than one type of upgrade information, the above upgrade information used for the above data quality improvement, and the above-mentioned receiving means is the above-mentioned sending device. The above-mentioned promotion information sent in accordance with the requirements of the required means.</p><p>The above improvement information can be set as the prediction coefficient used for the prediction of the predicted value of the quality improvement data for the above data quality improvement. The above improvement means uses the above data and the prediction coefficient to calculate the prediction value of the above quality improvement data.</p><p>The above-mentioned improvement information may be the above-mentioned prediction coefficient calculated according to each specific level; the above-mentioned improvement means may use the above-mentioned data and the prediction coefficient of each level to calculate the predicted value of the above-mentioned quality improvement data.</p><p>In the above improvement means, it is possible to set: the grade tap used for the grade calculation of the noticed quality improvement data of the noticed quality improvement data, the grade tap composition means constructed using the above data; the above notice quality is calculated according to the above grade taps To increase the level of the data and classify the classification method; in order to predict the above-mentioned attention-quality improvement data, the prediction taps used together with the above-mentioned prediction coefficients are used to construct the prediction-tap composition method using the above-mentioned data; and use the above-mentioned attention quality improvement The above prediction coefficient of the data level and the above prediction tap are used to calculate the prediction method of the prediction value of the above noted quality improvement data.</p><p>The above-mentioned trusted means can be the above-mentioned promotion information of most types of trusted.</p><p>The above-mentioned trusted means are trusted with the above-mentioned predictive coefficients of different grades as the above-mentioned most types of promotion information.</p><p>The above prediction coefficients can be generated using the data of students who become students and the data of teachers who become teachers. The above-mentioned trusted means is based on the most types of prediction coefficients calculated using the above-mentioned student data or teacher data of different qualities as the most of the above types Improve the information to be trusted.</p><p>The above-mentioned trusted means can be trusted with at least the above-mentioned prediction coefficients and linear interpolation information as the above-mentioned most types of promotion information.</p><p>The above-mentioned trusted means can use the various types of prediction coefficients calculated by the above-mentioned level taps or prediction taps of different compositions, and be trusted as the above-mentioned majority types of promotion information.</p><p>The above-mentioned trusted means is to classify the above-mentioned grades, and use different methods to calculate the prediction coefficients of most types, and be trusted as the promotion information of the above-mentioned most types.</p><p>The above-mentioned trusted means can be trusted. If the above-mentioned data is embedded with embedded data of more than one type of the above-mentioned promotion information, in this case, the fourth data processing device may additionally be equipped with: extraction means for extracting the above-mentioned promotion information from the above-mentioned embedded data .</p><p>The aforementioned extraction means can use the energy offset of the aforementioned data to extract the aforementioned boost information from the aforementioned embedded data.</p><p>The above-mentioned extraction means can perform inverse spectrum diffusion, and extract the above-mentioned enhancement information from the above-mentioned embedded data.</p><p>The above-mentioned extraction means extracts one or more bits of the above-mentioned embedded data as the above-mentioned enhancement information.</p><p>The above-mentioned data is image data, and the above-mentioned improvement information may be information for improving the image quality of the above-mentioned image data.</p><p>The fourth data processing method of the present invention is characterized by having: a trusting step of trusting the above data and more than one type of promotion information; and an promotion step of using any one of the above one or more types of promotion information to improve the quality of the above data ; And according to the above-mentioned promotion information used for the above-mentioned data quality improvement, the charging step of charging processing is carried out.</p><p>The fourth recording medium of the present invention is characterized by having: a trusting step of trusting the above-mentioned data and more than one type of promotion information; and an upgrading step of using any one of the above-mentioned promotion information of more than one type to improve the quality of the above-mentioned data; And according to the above-mentioned promotion information used for the above-mentioned data quality improvement, the procedure of the charging process of the charging process is recorded.</p><p>The fourth program of the present invention is characterized by having: a trusting step of trusting the above-mentioned data and more than one type of promotion information; and an promotion step of using any one of the above-mentioned promotion information of more than one type to improve the quality of the above-mentioned data; and According to the above-mentioned promotion information used for the above-mentioned data quality improvement, the charging step of the charging processing is carried out.</p><p>In the first data processing device and data processing method, recording medium and program of the present invention, enhancement information for data quality enhancement is generated, and the enhancement information is embedded in the data.</p><p>In the second data processing device and data processing method, recording medium and program of the present invention, enhancement information is extracted from embedded data, and the enhancement information is used to improve data quality.</p><p>In the third data processing device and data processing method, recording medium and program of the present invention, most types of improvement information for data quality improvement are generated, and data and more than one type of improvement information are sent.</p><p>In the fourth data processing device and data processing method of the present invention, in the recording medium and the program, the data and the more than one type of promotion information are sent, and the quality of the data is improved using any one of the more than one type of promotion information. In addition, Perform billing processing based on the promotion information used for the data quality improvement.</p>
Fig. 1 is a configuration example of one embodiment of a digital satellite broadcasting system (system means a logical collection of multiple devices, regardless of whether the constituent devices are located in the same frame) to which the present invention is applied.
In the transmitting device 1, a satellite broadcast wave, which is a radio wave corresponding to the program broadcast, is transmitted from the antenna (dish antenna) 1A to the satellite 2. The satellite (communication satellite or broadcasting satellite) 2 receives the satellite broadcast wave of the transmission device 1, and performs other necessary processing such as amplifying the satellite broadcast wave, and sends it out.
The satellite broadcast wave sent by the satellite 2 is received and displayed by the antenna (dish antenna) 3A of the receiving device 3.
The transmitting device 1 and the receiving device 3 communicate via a network 4 capable of bi-phase communication such as telephone lines, the Internet, a CATV (cable television) network, and a wireless communication network, between the transmitting device 1 and the receiving device 3 , Perform charging processing such as charging information processing through the network 4.
In the embodiment of FIG. 1, for the sake of simplicity of explanation, only one receiving device 3 is shown, but it is possible to transmit multiple receiving devices of the same configuration as the receiving device 3.
Fig. 2 is a configuration example of the transmission device 1 of Fig. 1.
In the promotion information generating unit 11, it is input as the image data of the program broadcast (hereinafter referred to as the image data for broadcasting), or the image data of the same content and higher image quality as the image data for the broadcasting (high-quality image data). The enhancement information generating unit 11 is connected to the receiving device 3 and generates enhancement information for image quality enhancement of broadcast image data.
That is, the enhancement information generating unit 11 is also provided with a method selection signal for the selection of the enhancement method for the image quality enhancement of the broadcast image data. The promotion information generating unit 11 generates more than one type of promotion information according to the supplied method selection signal. The promotion information generated by the promotion information generating unit 11 is supplied to the integration unit 12.
In the integration unit 12, in addition to the promotion information provided by the promotion information generation unit 11, video data for broadcasting is also provided. The integration unit 12 systematically integrates the image data for broadcasting and upgrade information, and generates an integrated signal for the transmission unit 13.
In addition to time-sharing multiplexing or frequency multiplexing, the method of integrating broadcast image data and enhanced information can be embedded coding as described later. In addition, the broadcast video data and promotion information are not integrated, but can be sent as a separate program.
The transmitting unit 13 modulates, amplifies, and other necessary processing on the integrated signal output by the integrated unit 12, and supplies it to the antenna 1A.
The billing processing unit 14 communicates with the trusted device 3 via the communication interface 15 and the network 4 to perform billing processing for the program provided to the trusted device 3.
The communication interface 15 is used for communication control via the network 4.
The program transmission processing performed by the transmission device 1 of FIG. 2 will be described with reference to the flowchart of FIG. 3.
In step S1, the enhancement information generating unit 11 generates one or more types of enhancement information for image quality enhancement of broadcast video data according to the supplied method selection signal, and supplies it to the integration unit 12. The unit of broadcast image data used for the promotion of information generation (hereinafter referred to as the unit of promotion of information generation) is, for example, 1 frame unit or 1 program unit.
The integration unit 12, when the promotion information is received by the promotion information generation unit 11, in step S2, the broadcast image data and the promotion information are integrated, and an integrated signal is generated and supplied to the transmission unit 13. The transmitting unit 13 performs modulation, amplification, and other necessary processing on the integrated signal output by the integrating unit 12 in step S3, and supplies it to the antenna 1A. According to this, the integrated signal is sent from the antenna 1A as a satellite broadcast wave.
After that, return to step S1, and repeat the same processing below.
The satellite broadcast wave broadcast by the satellite 2 is received from the antenna 3A, and the received signal is supplied to the receiving unit 21. The receiving unit 21 applies modulation, amplification, and other necessary processing to the received signal from the antenna 3A, and the integrated signal is supplied to the extraction unit 22.
The extracting unit 22 extracts the integrated signal supplied by the receiving unit 21, extracts broadcast video data and one or more types of enhancement information, supplies the broadcast video data to the quality enhancement unit 24, and supplies more than one type of enhancement information to Selection part 23.
The selection unit 23 selects the type corresponding to the image quality level signal of the billing processing unit 27 from more than one type of improvement information from the extraction unit 22, and at the same time the improvement information and the selected image quality improvement based on the improvement information The method selection signal of the improvement method is provided to the quality improvement part 24.
The quality improvement section 24 uses the improvement information provided by the selection section 23 for the broadcast image data supplied by the extraction section 22 to perform a process of selecting a signal representation method. According to this, the quality improvement unit 24 can obtain the image data with improved image quality, and supply the image data to the display unit 25. The display unit 25 is composed of, for example, a CRT (Cathode Ray Tube), or a liquid crystal panel, a DMD (Dynamic Mirror Device), etc., and displays images corresponding to the image data provided by the quality improvement unit 24.
The operation unit 26 is operated by the user when the image quality displayed on the display unit 25 is selected, and the operation signal corresponding to the operation is supplied to the charging processing unit 27.
The billing processing unit 27 performs billing processing corresponding to the image quality selected by the user in accordance with the operation signal of the operating unit 26. That is, the billing processing unit 27 recognizes the image quality requested by the user based on the operation signal of the operating unit 26, and supplies the image quality level signal indicating the level of the image quality to the selection unit 23. In the selection part 23, the image quality improvement information suitable for the user's requirements is selected. In addition, the billing processing unit 27 transmits the image quality level signal to the transmission device 1 via the communication interface 28 and the network 4.
The image quality level signal transmitted by the billing processing unit 27 to the transmitting device 1 is received by the billing processing unit 14 via the communication interface 15 of the transmitting device 1 (FIG. 2 ). In the billing processing unit 14, the user of the trusted device 3 is billed according to the image quality level signal. That is, the billing processing unit 14, for example, calculates the audio-visual fee of each user, and transfers the billing signal including at least the passbook account number of the transmitting device 1 side, the user's passbook account number, and the accrued audio-visual expense through the communication interface 15 and network 4 send the letter to the billing center (not shown). The billing center, when receiving the billing signal, deducts the amount corresponding to the audiovisual fee from the user's passbook account number, and at the same time performs the checkout process to add the passbook on the side of the sending device 1.
The communication interface 28 is used for communication control via the network 4.
The flowchart of FIG. 5 illustrates the program receiving processing performed by the receiving device 3 of FIG. 4.
The antenna 3A receives the received signal output from the satellite broadcast wave and is supplied to the receiving unit 21. In the receiving unit 21, in step S11, the received signal is received and converted into an integrated signal. This integrated signal is supplied to the extraction unit 22.
In the extracting unit 22, in step S12, the integrated signal of the receiving unit 21 extracts broadcast video data and one or more types of promotion information. The image data for broadcasting is supplied to the quality improvement section 24, and the improvement information of more than one type is provided to the selection section 23.
In the selection unit 23, in step S13, one or more types of promotion information from the extraction unit 22 are selected to select the type corresponding to the image quality level signal from the charging processing unit 27, and the promotion information is used to indicate the The method selection signal of the improvement method of the image quality improvement indicated by the improvement information is supplied to the quality improvement section 24.
In step S14, the quality improvement unit 24 uses the improvement information provided by the selection unit 23 on the broadcast image data supplied by the extraction unit 22 to perform processing in the manner indicated by the mode selection signal. According to this, the quality improvement part 24 can obtain the image data with the improved image quality, and provide the image data to the display part 25. After that, return to step S11, and repeat the same processing below.
In the above receiving processing, the image quality level signal output by the billing processing unit 27 corresponds to the level image quality required by the user operating the operating unit 26. Therefore, the image of the image quality requested by the user is displayed on the communication unit 25.
Hereinafter, the image quality improvement method of the image data can be, for example, the level classification adaptive processing previously proposed by the present inventor.
The grade classification adaptive processing is composed of grade classification processing and adaptive processing. Through the grade classification processing, the data is classified according to its nature, and the adaptive processing is applied to each grade. The adaptation process is as follows.
That is, in adaptive processing, for example, the SD image is calculated by combining the pixels (hereinafter referred to as SD pixels) that constitute a standard-resolution or low-resolution SD (Standard Definition) image with the linear shape of a specific prediction coefficient The predicted value of the pixel of the HD (High Definition) image used for the resolution enhancement can obtain the image whose resolution of the SD image has been improved
Specifically, for example, while a certain SD image is used as the teacher data, the number of pixels of the HD image is reduced, that is, the SD image with degraded resolution is used as the student data. Consider the predicted value E[y] of the pixel value y of the pixels constituting the HD image (hereinafter referred to as HD pixels), by the pixel value X of several SD pixels (the pixels constituting the SD image)<sub>1</sub>,X<sub>2</sub>, The set of ......, and the specific prediction coefficient W<sub>1</sub>,W<sub>2</sub>, ...... is calculated by the linear combination model defined by the linear combination. In this case, the predicted value E[y] can be expressed by the following formula,
E[y]=W<sub>1</sub>X<sub>1</sub>+W<sub>2</sub>X<sub>2</sub>+‥‥‥(1)
In order to generalize the formula (1), let the prediction coefficient W<sub>j</sub>The collection of ranks W, student information X<sub>I</sub><sub>J</sub>The rows and columns of the set X, and the predicted value E[y<sub>j</sub>] The ranks of the set Y, with
[Equation 1] Definition,
[Formula 1]
<maths><img file="TW577201B_D0001.tif" /></maths>
Then the following observation equation holds.
XW=Y
Here, the component X of the rank X<sub>I</sub><sub>J</sub>, Refers to the collection of the i-th student data (the i-th teacher data y<sub>i</sub>The set of student data used for prediction) No. j student data, the component W of row W<sub>j</sub>, Represents the data prediction coefficient calculated by the product of the jth student data in the set of student data. Again, Y<sub>i</sub>The i-th teacher data in the table, so E[y<sub>j</sub>] The predicted value of the i-th teacher's data in the table. Also, the y on the left side of formula (1) omits the component y of the row Y<sub>i</sub>If the subscript i is, again, the X on the right side of formula (1)<sub>1</sub>, X<sub>2</sub>, ......, omit the component X of the rank X<sub>i</sub><sub>j</sub>Those who are under the bid ij.
The observation equation of equation (2) applies the least self-multiplication method to calculate the predicted value E[y] close to the pixel value y of the HD pixel. In this case, suppose that it becomes the rank Y of the set of true pixel values y of the HD pixels of the teacher data, and the rank of the set of residual e of the predicted value E[y] relative to the pixel value y of the HD pixels E, when defined by Equation 2,
<maths><img file="TW577201B_D0002.tif" /></maths>
From Equation 2, we can see that the following residual equation holds,
XW=Y+E.
In this case, the prediction coefficient W of the predicted value E[y] close to the pixel value y of the HD pixel<sub>j</sub>, Can be multiplied by the error
<maths><img file="TW577201B_D0003.tif" /></maths>
Set it to the smallest value and calculate it.
Therefore, the above self-multiplied error is used to predict the coefficient W<sub>j</sub>The differential is 0, that is, the prediction coefficient W that satisfies the following formula<sub>j</sub>, Can be said to be the optimal value for the calculation of the predicted value E[y] close to the pixel value y of the HD pixel.
<maths><img file="TW577201B_D0004.tif" /></maths>
First, use equation (3) to predict coefficient W<sub>j</sub>Differential can be established as the following formula,
<maths><img file="TW577201B_D0005.tif" /></maths>
From equation (4) and equation (5), equation (6) can be obtained
<maths><img file="TW577201B_D0006.tif" /></maths>
Also, in the residual equation of equation (3), consider student data X<sub>I</sub><sub>J</sub>, Prediction coefficient W<sub>j</sub>, Teacher profile y<sub>i</sub>, And residual e<sub>i</sub>For the relationship, the following normal equation can be obtained from Equation 6.
Formula 7
<maths><img file="TW577201B_D0007.tif" /></maths>
Form 7 of the formal equations, the department will be the student's data X<sub>I</sub><sub>J</sub>And teacher information<sub>i</sub>, Prepare a certain number of sets, you can create and calculate the predicted coefficient W<sub>j</sub>The number J is the same number. Therefore, formula (7) is solved (however, when formula (7) is solved, in formula (7), the prediction coefficient W<sub>j</sub>The relevant coefficient composition must be regular), then the most appropriate prediction coefficient W can be calculated<sub>j</sub>. In addition, when solving equation (7), for example, the Gauss-Jordan elimination method can be used.
As mentioned above, calculate the most appropriate prediction coefficient W<sub>j</sub>, And then use the prediction coefficient W<sub>j</sub>, The predicted value E[y] of the pixel value y of the HD pixel calculated from the formula (1) is the adaptive location.
In addition, although adaptive processing is not included in SD video, the point of reproduction of components included in HD video is different from simple interpolation processing, for example. That is, the adaptive processing, when looking at equation (1) alone, is the same as the interpolation processing using the so-called interpolation filter, but the prediction coefficient W, which is equivalent to the tap coefficient of the interpolation filter, uses the teacher data y, the so-called Calculated by learning, so the components contained in the HD video can be reproduced. It can be seen from this that adaptive processing can also be called processing with the function of image creation (resolution imagination).
In addition, the above is an example of increasing the resolution to illustrate the adaptive processing, but the adaptive processing can also be used for other purposes. For example, even if the number of pixels before and after the adaptive processing is unchanged, the calculation of the predicted value of the image to remove noise or black points from the image is calculated. Wait. In this case, noise removal or black point improvement can be achieved by adaptive processing to improve image quality.
Fig. 6 is an example of the configuration of the promotion information generating unit 11 of Fig. 2 when the prediction coefficient is calculated as promotion information by the rank classification adaptive processing.
In the embodiment of FIG. 6, high-quality video data with the same content as the broadcast video data exists, and this is provided to the promotion information generating unit 11 as teacher data for learning prediction coefficients.
The high-quality image data as the teacher data is provided to the frame memory 31 in units of frames, for example, and the frame memory 31 sequentially stores the supplied teacher data.
The down-converter 32 reads the teacher data stored in the frame memory 31, for example, in frame units, applies LPF (Low Pass Filter) processing, and performs thinning, etc., basically with the same image quality as the broadcast image data That is, here is low-quality image data, which is generated as student data for predictive coefficient learning, and is provided to the frame memory 33.
The frame memory 33 stores the low-quality image data output by the down-converter 32 as student data, for example, sequentially memorizes it in frame units.
The predictive tap formation circuit 34 makes the pixels (hereinafter referred to as the teacher pixels) constituting the image of the teacher data stored in the frame memory 31 (hereinafter referred to as the teacher image) as the notable pixels in order, so that one of the notable pixels The position corresponding to the image of the student data (hereinafter referred to as the student image) is the position of several student data pixels (hereinafter referred to as the student pixels) that are close in space or time, according to the control signal of the control circuit 40 , Is read from the frame memory 33 and constructed as a prediction tap for multiplication calculation with a prediction coefficient.
That is, for example, as shown in FIG. 7, assuming that the student image is obtained by dividing the teacher image by 1/4, the predictive tap forming circuit 34 is based on a certain control signal of the control circuit 40, so that, for example, the eye-catching picture The 4 student pixels a, b, c, and d at the positions close to the position of the student image corresponding to the position of the element are used as prediction taps. In addition, the predictive tap formation circuit 34 is based on another control signal of the control circuit 40, so that, for example, the position of the student image corresponding to the position of the attention pixel is spatially close to 9 student pixels a, b, c, d, e, f, g, h, and i are used as prediction taps.
Also, basically predicting the tap, using 9 student pixels compared to 4 student pixels a, b, c, d, the teachers pixel prediction accuracy is higher, and higher quality images can be obtained (the obtained Prediction coefficient).
Prediction taps, (the same applies to the level taps described later), as shown in Figure 7, in addition to rectangular pixels, it can also be cross-shaped or diamond-shaped<sup>,</sup>Or other arbitrary shapes of pixel composition. In addition, the prediction tap may not be composed of adjacent pixels, but may be composed of every other pixel.
Returning to FIG. 6, the prediction taps formed by the prediction tap forming circuit 34 are supplied to the normal equation adding circuit 37.
The grade tap configuration circuit 35 causes the student pixel for the grade classification for any one of several grades to be read out from the frame memory 33 as the notable pixel. That is, the level tap formation circuit 35 makes the position of the student image corresponding to the position of the attention pixel start from several student pixels that are spatially or temporally close to each other. According to the control signal of the control circuit 40, the frame memory 33 It is read out, used as a grade tap for grade classification, and supplied to the grade classification circuit 36.
In addition, the prediction tap, that is, the grade tap, can be composed of the same student pixel or different student pixels.
The grade classification circuit 36 constitutes the grade taps of the circuit 35 according to the grade taps. According to the method of the control signal of the control circuit 40, it classifies the attention pixels and provides the grade codes corresponding to the grades of the attention pixels obtained as a result. Normal equation addition circuit 37.
Here, the method of class classification can be, for example, the method of using the threshold value, or the method of using ADRC (Adaptive Dynamic Range Coding).
Use the threshold method. For example, it is binarized according to whether the pixel value of the student pixels constituting the level tap is greater than a specific threshold value, and the level of the attention pixel is determined according to the result of the binarization.
Using the ADRC method, the student pixels that constitute the grade tap are treated with ADRC, and the grade of the attention pixel is determined according to the resultant ADRC code.
In addition, in the K-bit ADRC, for example, the maximum value MAX and the minimum value MIN of the pixel values of the student pixels constituting the grade tap are detected, and DR=MAX-MIN is used as the local dynamic range of the grade tap. According to the dynamic range DR, the student pixels constituting the grade tap are requantized into K bits. That is, the minimum value MIN is subtracted from the pixel value of the pixels constituting the level tap, and the result is expressed as DR/2<sup>k</sup>Divide (quantify). Therefore, when the grade tap is processed with, for example, 1-bit ADRC, the pixel value of each student pixel constituting the grade tap is 1 bit. In this case, the 1-bit pixel value of each pixel constituting the level tap obtained above is juxtaposed in a specific order and output as an ADRC code.
Therefore, according to ADRC, when the grade tap is composed of N student pixels, and the K-bit ADRC processing result of the grade tap is set as the grade code, the attention pixel is classified as (2<sup>k</sup>) Any of the levels.
The regular equation addition circuit 37 reads out the teacher pixels as the attention pixels from the frame memory 31, and performs addition operations with prediction taps (constituted student pixels) and attention pixels (teacher pixels) as objects.
That is, the normal equation addition circuit 37 uses the predictive taps (student pixels) for each grade corresponding to the grade code supplied by the grade classification circuit 36, and performs the multiplier of the predictive coefficient on the left side of the normal equation of formula (7). Student pixel multiplication operation (X<sub>i</sub><sub>n</sub>XX<sub>i</sub><sub>m</sub>), and an operation equivalent to the sum (Σ).
The regular equation addition circuit 37 uses predictive taps (student pixels) and attention pixels (teacher pixels) for each level corresponding to the level code supplied by the level classification circuit 36 to perform the student drawing on the right side of the regular equation Multiplication of pixel and attention pixel (teacher pixel) (X<sub>i</sub><sub>n</sub>X<sub>y</sub><sub>i</sub>), and an operation equivalent to the sum (Σ).
The normal equation addition circuit 37 performs the above-mentioned addition operation on all the teacher pixels stored in the frame memory 31 as the attention pixels, and generates the normal equation shown in equation (7) for each level accordingly. According to the regular equation, the promotion information is generated for each specific teacher's pixel number.
The predictive coefficient determining circuit 38 solves the normal equation generated by the normal equation addition circuit 37 according to each level, calculates the predictive coefficient of each level, and supplies it to the address corresponding to each level of the memory 39. The memory 39 uses the prediction coefficients provided by the prediction coefficient determination circuit 38 as a memory for improving information, and is supplied to the integration unit 12 when necessary (FIG. 2).
In the normal equation addition circuit 37, there are levels of normal equations for which the necessary number cannot be obtained when the prediction coefficients are calculated. The prediction coefficient determination circuit 38 outputs, for example, a preset prediction coefficient (for example, a prediction coefficient calculated in advance using more teacher images, etc.) for this level.
The control circuit 40 is supplied with a method selection signal for the selection of the improvement method of the image quality improvement of the broadcast image data (FIG. 2 ). The control circuit 40 selects the boosting method indicated by the signal in this way, and controls the predictive tap forming circuit 34, the level tap forming circuit 35 and the level classification circuit 36 to generate necessary boosting information to improve the image quality of the broadcast image data.
In this embodiment, the billing amount (audiovisual expense) of the billing processing unit 14 is different depending on the promotion method used by the trusted device 3 (use promotion information).
The billing amount and the way of raising can be set according to, for example, whether the use level classification is processed or not. For example, as shown in Figure 8(A), the lifting method is based on the use of linear interpolation, only adaptive processing, or level classification adaptive processing, and the billing amounts are different.
Using only adaptive processing means not to perform hierarchical classification, but to perform hierarchical classification only. Therefore, it is equivalent to only one level (black and white) in the level classification adaptation process.
In addition, when linear interpolation is used in the lifting method, no prediction coefficient is required, and no special processing is performed in the lifting information generating unit 11, for example, the subject of instructing linear interpolation is output as lifting information.
The billing amount can also be set according to the level classification used as an upgrade method to adapt to the number of levels in the processing. That is, as shown in Fig. 8(B), there can be 3 cases of using linear interpolation according to the promotion method, using a small number of grades adaptive processing, and using a large number of grades adaptive processing , Set different billing amounts.
In addition, the billing amount can be adapted to the image quality setting of the student image or teacher image used in the prediction coefficient generation in the processing according to the level classification used in the upgrading method, that is, for example, when the teacher image is good, the image data for broadcasting can be greatly improved. In terms of quality, the so-called high-performance prediction coefficients. Conversely, when the image quality of the teacher's video is not good, the so-called low-performance prediction coefficients can only slightly improve the image quality of the broadcast image data. As shown in Figure 8(C), there can be three different settings according to the three conditions of using linear interpolation, using low-performance prediction coefficients for class classification adaptation processing, and using high-performance prediction coefficients for class classification adaptation processing. The billing amount.
In addition, the billing amount can be set according to the grade tap or prediction tap constructed in the grade classification adaptation process used as an upgrade method. That is, depending on the composition method of the grade tap or prediction tap (tap shape, number of pixels that make up the tap, taps composed of one or two pixels in the space direction or time direction, etc.), the image quality obtained as above is mutually consistent. Therefore, according to this composition method, different billing amounts can be set.
In addition, the billing amount can be set according to the level classification method of the level classification adaptive processing used as the promotion method. That is, as shown in Fig. 8(D), there can be three situations of using linear interpolation of the promotion method, the adaptive processing of the level classification using the above threshold value, and the adaptive processing of the level classification using ADRC processing. Different billing amounts.
The lifting mode and mode selection signal, for example, can correspond as shown in Fig. 8(A) and drawing 8(D). The control circuit 40 is capable of obtaining the lifting information used by the mode selection signal corresponding to the supplied mode selection signal. The control signal for instruction is output to the prediction tap configuration circuit 34, the grade tap configuration circuit 35, and the grade classification circuit 36. In addition, the lifting method can also use most of the above-mentioned combinations.
Hereinafter, referring to the flowchart of FIG. 9, the promotion information generation process of the promotion information generation performed by the promotion information generation part 11 of FIG. 6 will be described.
In step S21, the teacher images equivalent to the promotion information generating unit are stored in the frame memory 31. Moving to step S22, the control circuit 40 enables it to obtain the boost information used in the boost mode corresponding to the supplied mode selection signal, and outputs the control signal for instruction to the prediction tap configuration circuit 34, the level tap configuration circuit 35, and Level classification circuit 36. According to this, the prediction tap forming circuit 34, the level tap forming circuit 35, and the level classification circuit 36 are set to be capable of processing to obtain the boosting information used as the boosting method instructed by the control signal of the prediction coefficient.
In addition, the mode selection signal supplied to the control circuit 40 includes information indicating a plurality of lifting modes, and the control circuit 40 outputs the control signals corresponding to the plurality of lifting modes according to the processing of step S22.
When the control signal output by the control circuit 40 indicates linear interpolation, the memory 39 indicates that the subject of the linear interpolation is memorized as the lifting information. After that, the processing from step S23 to step S28 is skipped, and the process moves to step S29.
After the processing of step S22 moves to step S23, in the down-converter 32, for the teacher image stored in the frame memory 31, LPF processing or thinning processing is applied if necessary, and the image of the same image quality as the broadcast image data is used as The student image is generated and stored in the frame memory 33.
In addition, the student image can be set as an image with a different image quality from the broadcast image data. In this case, the control signal of the subject is sent to the down-converter 32 by the control circuit 40, and the down-converter 32 is controlled according to the control. The control signal of the circuit 40 generates student images.
After that, it moves to step S24. Among the teacher pixels stored in the frame memory 31, only those pixels that have not been set as attention pixels are set as attention pixels. The predictive tap formation circuit 34 responds to the control signal of the control circuit 40 The prediction taps of the attention pixels of the composition are constructed using the student pixels stored in the frame memory 33. Furthermore, in step S24, in the grade tap formation circuit 35, the grade taps of the notable pixels according to the composition of the control signal of the control circuit 40 are constructed using the student pixels stored in the frame memory 33. After that, the prediction tap is supplied to the normal equation addition circuit 37, and the grade tap is supplied to the grade classification circuit 36. The grade classification circuit 36, in step S25, constructs the grade taps of the circuit 35 according to the grade taps, uses the control signal instruction method of the control circuit 40 to classify the notable pixels, and provides the grade code corresponding to the resulting grade to the standard The equation addition circuit 37 moves to step S26.
In step S26, in the regular equation addition circuit 37, the teacher pixels that become the attention pixels are read from the frame memory 31, and the prediction taps (constitute the student pixels) and the attention pixels (teacher pixels) are used as objects to perform the above Addition operation.
After that, it moves to step S27, and the control circuit 40 determines that all the teacher pixels of the promotion information generating unit stored in the frame memory 31 are the attention pixels, and if the addition operation is performed, it is determined that all the teacher pixels are not the attention pixels When performing addition operation, return to step S24. In this case, one of the teacher pixels that is not set as the attention pixel is used as the new attention pixel, and the processing from step S24 to step S27 is repeated.
In step S27, the control circuit 40 determines that all the teacher pixels in the promotion information generating unit are the attention pixels, and performs the addition operation, that is, when the normal equation addition circuit 37 obtains the normal equations of each level, it moves to step S28 to predict The coefficient determination circuit 38 respectively solves the normal equations generated by each level, calculates the prediction coefficients of each level, and supplies them to the addresses corresponding to each level of the memory 39. The memory 39 uses the prediction coefficient provided by the prediction coefficient determination circuit 38 as the memory for improving information.
The memory 39 has a large number of blocks, so that it can simultaneously store a large number of types of promotion information.
After that, it moves to step S29, and the control circuit 40 judges whether the boost information is obtained for all of the most boosting modes included in the supplied mode selection signal.
In step S29, it is judged that among the majority of the lifting information used by the majority of the lifting methods included in the mode selection signal, if there is an unobtained one, move to step S22, and the control circuit 40 outputs the control signal corresponding to the lifting method for which the lifting information is not obtained. Repeat the same processing as above.
In addition, in step S29, it is determined that all of the majority of the lifting methods included in the mode selection signal have obtained the lifting information, that is, when the majority of types of lifting information used by the majority of the lifting methods included in the mode selection signal are stored in the memory 39, shift Go to step S30, read the most type promotion information from the memory 39, and supply it to the integration unit 12 (FIG. 2) to end the process.
In addition, the promotion information generation process of FIG. 9 is repeated for each of the teacher images of the promotion information generation unit supplied to the frame memory 31.
The embodiment in Figure 6 is based on the premise that high-definition video data with the same content as the broadcast video data exists, but the high-definition video data does not exist (for example, when the original video is used as the broadcast video data ). In this case, the teacher image does not exist, so the promotion information generating unit 11 in FIG. 6 cannot generate the prediction coefficient as the promotion information.
FIG. 10 is another example of the promotion information generating unit 11 that can generate the prediction coefficient of the promotion information when there is no high-quality image data that becomes the teacher's image when the transmission device 1 sends an image of the same size as the original image. In the figure, the parts corresponding to those in FIG. 6 are given the same symbols, and their descriptions are omitted. The step-up information generating unit 11 of FIG. 10 does not have the down-converter 32, and newly installs a frame memory 41, a feature quantity estimation circuit 42, a virtual teacher data generation circuit 43, and a virtual student data generation circuit 44. The others are as shown in FIG. 6 same.
In Fig. 10, the promotion information generation unit 11, because there is no real teacher data, it is generated from the broadcast image data, and the relationship between the real teacher data and the broadcast image data as the student data has the same relationship as the virtual one. A teacher image and a virtual student image (hereinafter referred to as a virtual teacher image and a virtual student image, respectively), use the virtual teacher image and virtual student image to generate prediction coefficients as enhancement information.
That is, the frame memory 41 is supplied with broadcast video data, and the frame memory 41 stores the supplied broadcast video data to improve the memory of the information generating unit.
The feature quantity estimation circuit 42 calculates the feature quantity of the broadcast image data stored in the frame memory 41, and supplies it to the virtual teacher data generation circuit 43 and the virtual student data generation circuit 44.
The feature data of broadcast image data can be, for example, the autocorrelation coefficient in the horizontal or vertical direction, the histogram of pixel values, the histogram of the difference between adjacent pixels (active histogram), etc.
The virtual teacher data generation circuit 43 is based on the feature quantity of the broadcast image data output by the feature quantity estimation circuit 42 to estimate the feature quantity of the original teacher image (real teacher image) relative to the broadcast image data (hereinafter referred to as the inferred teacher feature) the amount). In addition, the virtual teacher data generation circuit 43 applies LPF to the broadcast image data stored in the frame memory 41 to perform a thinning process to generate an image with the same feature quantity as the inferred teacher feature quantity, and use the image as the virtual teacher image Supplied to the frame memory 31 and the virtual student data generating circuit 44.
The virtual student data generation circuit 44 applies LPF to the virtual teacher image supplied by the virtual teacher data generation circuit 43, and generates an image with the same feature amount as the feature quantity of the original student image supplied by the feature quantity estimation circuit 42 for broadcast image data , And use the image as a virtual student image for the frame memory 33.
Hereinafter, the promotion information generation process of the promotion information generation performed by the promotion information generation part 11 of FIG. 10 will be described with reference to the flowchart of FIG. 11.
In the frame memory 41, when the broadcast video data is supplied to memory, first in step S41, the feature quantity estimation circuit 42 extracts the feature quantity of the broadcast video data stored in the frame memory 41 and supplies it to the virtual teacher data generating circuit 43 And the virtual student data generating circuit 44.
When the virtual teacher information generating circuit 43 receives the feature amount of the broadcast image data from the feature amount estimation circuit 42, in step S42, the feature amount of the original teacher image relative to the broadcast image data is estimated based on the feature amount (inferred teacher featurethe amount), move to step S43. In step S43, the virtual teacher data generation circuit 43 sets the filter characteristics and the thinning width (thinning rate) of the LPF based on the estimated teacher feature quantity, so as to obtain an image with the same feature quantity as the inferred teacher feature quantity from the broadcast image data , And then move to step S44.
In step S44, the virtual teacher data generating circuit 43 applies a thinning process to the broadcast image data stored in the frame memory 41 by setting the gap between settings, and applies a setting to the image data after the thinning process. LPF processing with filtering characteristics generates virtual teacher images.
In step S44, the reason why the image data for broadcasting is thinned out is because the high-quality image data has a sharper self-correlation shape and a higher spatial frequency compared with a lower-quality image of the same size. The self-related images with sharper shapes are used as virtual teacher images.
After step S44 is processed, it moves to step S45, the virtual teacher information generating circuit 43 calculates the feature amount of the virtual teacher image generated in step S44, and judges whether the feature amount approximates the estimated teacher feature amount. When it is judged in step S45 that the feature value of the virtual teacher image is not similar to the estimated teacher feature value, the process moves to step S46, and the virtual teacher data generating circuit 43 changes the setting value of the LPF filtering characteristic or the thinning width applied to the broadcast image data. Go back to step S44. In this way, the generation of virtual teacher images can be corrected.
When it is judged in step S45 that the feature amount of the virtual teacher image approximates the estimated teacher feature amount, the virtual teacher image is supplied to the frame memory 31 for storage, and is supplied to the virtual student data generating circuit 44, and the process moves to step S47.
In step S47, the virtual student data generating circuit 44 sets the filter characteristic of applying LPF to the virtual teacher image supplied by the virtual teacher data generating circuit 43, and moves to step S48.
In step S48, the virtual student data generating circuit 44 applies the set filter characteristic LPF to the virtual teacher image to generate the virtual student image.
Moving to step S49, the virtual student data generating circuit 44 calculates the feature value of the virtual student image generated in step S48, and determines whether the feature value approximates the feature value of the broadcast image data supplied by the feature value estimation circuit 42. When it is judged in step S49 that the feature value of the virtual student image is not similar to the feature value of the broadcast image data, it moves to step S50, and the virtual student data generating circuit 44 changes the setting value of the filter characteristic of applying LPF to the virtual teacher image, and returns Go to step S48. In this way, the generation of virtual student images can be corrected.
In addition, when it is determined in step S49 that the feature amount of the virtual student image is similar to the feature amount of the broadcast image data, the virtual student image is supplied to the frame memory 33 for storage, and the process moves to step S51.
In step S51 to step S58, while taking the virtual teacher image stored in the frame memory 31 as the original teacher image, and using the pseudo student image stored in the frame memory 33 as the original student image, proceed to step S22 in Fig. 9 , Step S24 to step S30 are the same process, according to this, most types of promotion information can be generated and stored in the memory 39. After that, the most types of promotion information are read from the memory 39 and supplied to the integration unit 12 (FIG. 2 ), and the processing ends. In the implementation form of Map 11, the teacher image and the student image are of the same size. Although the grade taps that constitute the prediction taps are different from those in Fig. 7, they are composed of most of the student images located around the position of the eye-catching pixel in the teacher image. The level tap, which is the point of the prediction tap, is the same as the embodiment in FIG. 9.
In addition, the promotion information generation process of FIG. 11 is also repeated as in FIG. 9 when the frame memory 41 is supplied with the broadcast image data for the promotion information generation unit.
FIG. 12 is an example of the configuration of the promotion information generating unit 11 of the transmitting device 1 (FIG. 2 ), which constitutes the quality improvement unit 24 of the receiving device 3 (FIG. 4) as shown in FIG. 6 or 9.
The frame memory 51 is supplied with the broadcast video data output by the extraction unit 22 (FIG. 4 ). The frame memory 51 memorizes the broadcast image data as an information generating unit.
The predictive tap forming circuit 52 performs the same processing as the predictive tap forming circuit 34 of FIG. 6 according to the control signal of the control circuit 57. According to this, the broadcast image data stored in the frame memory 51 can be used to construct the predictive taps, which are used for prediction Arithmetic circuit 56.
The gradation tap forming circuit 53 performs the same processing as the grading tap forming circuit 35 of FIG. 6 according to the control signal of the control circuit 57. According to this, the broadcasting image data stored in the frame memory 51 can be used to form the gradation taps, which are provided to the ranks. Classification circuit 54.
The grade classification circuit 54 performs the same processing as the grade classification circuit 36 of FIG. 6 according to the control signal of the control circuit 57. Based on this, the grade code of the grade classification result performed by the grade tap of the grade tap configuration circuit 53 can be used as The address is supplied to the memory 55.
The memory 55 stores the prediction coefficients provided by the selection unit 23 (FIG. 4) as promotion information. In addition, the memory 55 reads out the prediction coefficients of the address memory corresponding to the class code from the class classification circuit 54 and supplies it to the prediction operation circuit 56.
The prediction calculation circuit 56 uses the prediction taps supplied by the prediction tap formation circuit 52, that is, the prediction coefficients supplied by the memory 55, to perform the linear prediction operation (accumulation operation) of formula (1), and the pixel value obtained from the result is used for broadcasting The predicted value of the high-quality image (teacher image) used to improve the image quality of the image data is output.
The control circuit 57 is supplied with a mode selection signal output by the selection unit 23 (FIG. 4 ). The control circuit 57 selects the signal in this way, and outputs the same control signal as in the case of the control circuit 40 of FIG. 6 to the prediction tap formation circuit 52, the grade tap formation circuit 53, and the grade classification circuit 54.
The mode selection signal supplied from the selection unit 23 to the control circuit 57 is the information for indicating most of the boost modes included in the mode selection signal supplied by the control circuit 40 of FIG. 6, and only includes the billing processing unit 27 (FIG. 4) according to One piece of information corresponding to the image quality level signal requested by the user. Therefore, the control circuit 57 controls the predictive tap configuration circuit 52, the level tap configuration circuit 53, and the level classification circuit 54 as if the image quality required by the user can be obtained.
Referring to the flowchart of FIG. 13, the quality improvement process of the image quality improvement of the broadcast video data performed by the quality improvement unit 24 of FIG. 12 will be described.
In the receiving device 3 (FIG. 4), when the extraction unit 22 supplies the quality enhancement unit 24 with the broadcast image data of the enhancement information generating unit, the selection unit 23 provides the quality enhancement unit 24 with the majority of the enhancement information according to the image quality One type of enhancement information for level signal selection, and the method selection signal supply for the display of the enhancement method that uses the enhancement information to enhance the picture quality.
In step S61, the broadcast video data supplied by the extracting unit 22 is stored in the frame memory 51 in the form of an enhanced information generating unit. In step S61, the boost information supplied by the selection unit 23 is stored in the memory 55. Furthermore, in step S61, in the control circuit 57, the mode selection signal supplied by the selection unit 23 is received, and the enhancement information corresponding to the mode selection signal is received, and the control signal instructing to enhance the image quality of the broadcast video data is supplied to the predictive tap configuration The circuit 52 and the grade taps constitute the circuit 53 and the grade classification circuit 54. According to this, the prediction tap forming circuit 52, the grade tap forming circuit 53, and the grade classification circuit 54 are set as if they are processed according to the boost mode indicated by the control signal of the control circuit 57.
Furthermore, in this embodiment, the mode selection signal supplied to the control circuit 57 indicates linear interpolation, and the boost information stored in the memory 55 is the prediction coefficient.
Furthermore, when the mode selection signal supplied to the control circuit 57 indicates linear interpolation, the control circuit 57 supplies a control signal to the predictive calculation circuit 56 to perform linear interpolation on the broadcast video data stored in the frame memory 51. In this case, the predictive calculation circuit 56 reads out the broadcast video data stored in the frame memory 51 via the predictive tap formation circuit 52, and performs linear interpolation and output. In this case, the processing after step S62 is not performed.
After step S61 is processed, it moves to step S62. Among the pixels of the high-quality image composition of the broadcast image data stored in the frame memory 51, it is set as one of the pixels that has not been set as a noticeable pixel. For the attention pixels, the prediction taps of the attention pixels formed by the predictive tap formation circuit 52 based on the control signal of the control circuit 57 are constructed using the broadcast video data stored in the frame memory 51. Furthermore, in step S62, in the gradation tap formation circuit 53, the gradation taps of the notable pixels based on the composition of the control signal of the control circuit 57 are constructed using the broadcast video data stored in the frame memory 51. After that, the prediction tap is supplied to the prediction calculation circuit 56, and the grade tap is supplied to the grade classification circuit 54.
In step S63, the grade classification circuit 54 uses the grade tap from the grade tap forming circuit 53 to classify the notable pixels by the control signal of the control circuit 57, and uses the grade code corresponding to the obtained grade as the address It is supplied to the memory 55 and moves to step S64.
In step S64, among the prediction coefficients stored in the memory 55 and stored in step S61 as the boosting information, the address indicated by the level code output by the level classification circuit 54 is read out and supplied to the prediction operation circuit 56.
The predictive calculation circuit 56 uses the predictive taps supplied by the predictive tap forming circuit 52 and the predictive coefficients supplied from the memory 55 in step S65 to perform the linear predictive calculation represented by the formula (1), and the obtained pixel value is used as a notable plot The predicted value of Su is temporarily memorized.
After that, it moves to step S66, and the control circuit 57 executes that all of the constituent pixels of the high-quality image frame corresponding to the frame of the broadcast image data stored in the frame memory 51 are the notable pixels, and the predicted value is calculated. The judgment. In step S66, it is determined that all the constituent pixels of the frame of the high-definition image are the attention pixels, and if there is a predicted value that has not been calculated, return to step S62, among the constituent pixels of the frame of the high-definition image , The ones that have not been set as attention pixels are set as new attention pixels, and the same processing will be repeated below.
In step S66, it is judged that all the constituent pixels of the frame of the high-quality image are notable pixels, and when the predicted value has been calculated, it moves to step S67, and the prediction arithmetic circuit 56 calculates the high-quality image of the calculated predicted value They are sequentially output to the display unit 25 (FIG. 4 ), and the processing ends.
In addition, the quality improvement process of FIG. 13 is repeated when the frame memory 51 is supplied with the broadcast image data of the improvement information generating unit.
/As mentioned above, the sending device 1 sends most types of enhancement information, and on the receiving device 3, the most types of enhancement information select the image quality counterpart requested by the user, and the selected enhancement information is used to enhance the image quality. It can provide the image quality requested by the user, and can perform subdivided billing based on the image quality provided by the user.
Furthermore, in the above case, the sending device 1 sends most types of promotion information, and the receiving device 3 selects the image quality corresponding to the user's request from the most types of promotion information. For other example, in the sending device 1, the image quality corresponding to the user's request is selected. The trusted device 3 accepts the user's request, and only transmits the enhancement information corresponding to the requested image quality to the trusted device 3. In this case, as shown by the dotted line in FIG. 2, under the control of the charging processing unit 14, in the integration unit 12, the improvement information corresponding to the image quality requested by the user is included in the integration signal.
Furthermore, in the above-mentioned case, most types of promotion information are also sent to indicate most types of prediction coefficients or the subject of linear interpolation, but other, for example, most types of prediction coefficients, are not sent by the transmitting device 1. The pre-calculated prediction coefficients are stored in the memory 55 of the receiving device 3, and information related to any one of the most types of prediction coefficients stored in the memory 55 may be used, and it may be sent as most types of promotion information.
Furthermore, in this case, the level classification adaptive processing and linear interpolation are used as the improvement method, but other processing can also be used as the improvement method. The signal generation unit 11 and the quality improvement unit 24 do not need to perform the level classification.
FIG. 14 is an example of the configuration of the promotion information generating unit 11 calculated by using the level code of the appropriate prediction coefficient used in the attention pixel prediction as the promotion information. In the figure, parts corresponding to those in FIG. 6 are given the same reference numerals, and their description is omitted.
In the memory 101, the prediction coefficient of the grade code calculated by learning in the learning device (FIG. 16) described later is memorized. The memory 101 is controlled by the control circuit 40 to sequentially read out the prediction coefficients of each level and supply them to the prediction calculation circuit 102.
In the prediction calculation circuit 102, in addition to the prediction coefficients provided by the memory 101, the prediction taps are also provided by the prediction tap formation circuit 34. The prediction operation circuit 102, like the prediction operation circuit 56 in FIG. 12, uses the prediction taps supplied by the prediction tap configuration circuit 34 and the prediction coefficients supplied by the memory 101 to perform the linear prediction operation (accumulation operation) of formula (1). However, the prediction calculation circuit 102 performs a linear prediction calculation between the prediction coefficients of each level sequentially supplied from the memory 101 for a certain prediction tap, and calculates the predicted value of the teacher pixel. Therefore, in the predictive calculation circuit 102, for each teacher pixel, the predictive value of the same number as the total number of levels is calculated.
The prediction value calculated by the prediction calculation circuit 102 is supplied to the comparison circuit 103, and the comparison circuit 103 is also supplied with the teacher image from the frame memory 31. The comparison circuit 103 compares the predicted values of the teacher pixels constituting the teacher image provided by the frame memory 31 with the predicted values of the predicted coefficients of each level of the teacher pixels provided by the prediction calculation circuit 102, and calculates the prediction error, and supplies it to Detection circuit 104.
In addition, in the above case, in the promotion information generating unit 11, the prediction coefficient is generated as the promotion information, and in the quality improvement unit 24, the prediction coefficient is used to perform the level classification adaptive processing to improve the image quality. But for other example, in the promotion information generating unit 11, the grade code of the appropriate prediction coefficient for the attention pixel prediction is used as promotion information, and in the quality promotion unit 24, the prediction coefficient of the grade code is used for adaptive processing to improve the image picture. Quality can also be.
That is, in the enhancement information generating unit 11 and the quality enhancement unit 24, the prediction coefficients of each level obtained by learning in advance are memorized. After that, in the promotion information generating unit 11, the prediction coefficients of each level stored in advance are used for adaptive processing to calculate the predicted value of the high-quality image, and the predicted value of the predicted value that can be the closest to the true value is calculated for each pixel. The grade code is used as promotion information. In addition, the quality improvement unit 24 uses the predicted coefficients stored in advance as the level code corresponding to the improvement information to calculate the predicted value of the high-quality image to obtain the image with the improved image quality. In this case, the receiving device 3 can obtain an image of the same image quality as the one obtained by the transmitting device 1 (enhancement information generating unit 11).
Also, in this case, the promotion information generation unit 11 and the quality improvement unit 24 do not perform level classification. That is, in the promotion information generating unit 11, the grade code of the appropriate prediction coefficient for the prediction value calculation is calculated using, for example, the prediction coefficients of all grades for adaptive processing (prediction calculation), and the quality improvement unit 24 uses the The grade code is the promotion information, and the prediction coefficient of the grade code is used for adaptive processing to improve the image quality.
The detection circuit 104 detects the predicted value of the teacher pixel with the smallest prediction error provided by the comparison circuit 103. In addition, the detection circuit 104 detects the level code used to indicate the level of the predictive coefficient when the predictive value is available, and outputs it as the boosting information.
The following describes the promotion information generation process of the promotion information generation performed by the promotion information generation part 11 of FIG. 14 with reference to the flowchart of FIG. 15.
In step S111, teacher images equivalent to the promotion information generating unit are stored in the frame memory 31. After that, moving to step S112, the control circuit 40 supplies the control signal to the predictive tap forming circuit 34 to instruct to obtain the boosting information used by the boosting mode corresponding to the supplied mode selection signal. According to this, the predictive tap forming circuit 34 is set to perform processing to obtain the level code to be used as the boosting information indicated by the control signal.
In addition, the mode selection signal supplied to the control circuit 40 includes information indicating a plurality of lifting modes, and the control circuit 40 sequentially outputs the control signals corresponding to the plurality of lifting modes when the processing in step S112 is performed.
When the control signal output by the control circuit 40 indicates linear interpolation, the memory 39 stores the purpose of indicating linear interpolation as the boosting information. After that, the processing from step S113 to step S122 is skipped, and the process moves to step S123.
After the processing in step S112, move to step S113, in the down-converter 32, apply necessary LPF or thinning processing to the teacher image stored in the frame memory 31 to generate an image with the same image quality as the broadcast image data as a student image , For storage in the frame memory 33.
The embodiment of the map 15 is the same as the embodiment of Fig. 9. The student image can be an image with a different image quality from the broadcasting image data. In this case, the control signal of the subject is supplied to the down-converter 32 by the control circuit 40. The down-converter 32 generates a student image of the image quality indicated by the control signal of the control circuit 40.
After that, it moves to step S114. Among the teacher pixels stored in the frame memory 31 that are not set as the attention pixels, the attention pixels are set in the predictive tap configuration circuit 34, and the attention of the structure indicated by the control signal of the control circuit 40 The prediction tap of the pixel is composed of the student pixels stored in the frame memory 33. This prediction tap is supplied to the prediction operation circuit 102.
After that, it moves to step S115, the control circuit 40 sets the variable i of the level count to 0 of the initial value, and moves to step S116. In step S116, the control circuit 40 supplies the variable i as an address to the memory 101. In step S116, the prediction coefficient corresponding to the level code #i is read from the memory 101 and supplied to the prediction operation circuit 102.
The prediction calculation circuit 102, in step S117, uses the prediction taps provided by the prediction tap formation circuit 34 and the prediction coefficients provided by the memory 101 to perform the linear prediction calculation of formula (1), and use the obtained pixel value as the prediction of the notable pixel The value is supplied to the comparison circuit 103.
The comparison circuit 103, in step S118, reads the pixel value of the notable pixel from the frame memory 31, compares it with the prediction value from the prediction operation circuit 102, and calculates the prediction error of the prediction value. Furthermore, in step S118, the comparison circuit 103 supplies the prediction error to the detection circuit 104, and moves to step S119.
In step S119, the control circuit 40 increments the variable i by 1, and moves to step S120. In step S120, the control circuit 40 determines whether the variable i is less than N of the total number of levels, and when it determines that the variable i is less than N, it returns to step S116, and the same processing is repeated below.
In addition, in step S120, it is determined that the variable i is not less than N, that is, for the pixel of interest, the prediction coefficients corresponding to all levels have been used to calculate the prediction error of the predicted value, then it moves to step S121, and the detection circuit 104, that is, for the pixel of interest, The level of the prediction coefficient with the smallest prediction error is detected, and the level code corresponding to the level is memorized as the promotion information.
After that, it moves to step S123, and the control circuit 40 judges whether or not the boost information is obtained for all of the most boosting modes included in the supplied mode selection signal.
In step S123, it is determined in step S123 that among the majority of the lifting information used by the majority of the lifting methods included in the mode selection signal, if there is one that has not been obtained, move to step S112, and the control circuit 40 outputs the control signal corresponding to the lifting method for which the lifting information is not obtained, and the following repeats The same treatment as the above situation.
In step S123, when it is determined that all of the majority of the lifting methods included in the mode selection signal have obtained the lifting information, that is, when the majority of types of lifting information used by the majority of the lifting methods included in the mode selection signal are memorized in the detection circuit 104, go to step S124, the detection circuit 104 reads out the most types of promotion information, and supplies it to the integration unit 12 (FIG. 2), and the processing ends.
In addition, the promotion information generation process of FIG. 15 is the same as the embodiment of FIG. 9, when the frame memory 31 is supplied with a teacher image (for example, a teacher image per frame) for the promotion information generation unit.
FIG. 16 is a configuration example of an embodiment of a learning device for calculating prediction coefficients of each level stored in the memory 101 of FIG. 14. FIG.
In the embodiment of FIG. 16, the frame memory 111, the down-converter 112, the frame memory 113, the prediction tap formation circuit 114, the grade tap formation circuit 115, the grade classification circuit 116, the normal equation addition circuit 117, and the prediction coefficient determination circuit 118. The structure of the memory 119 is the same as that of the frame memory 31, the down-converter 32, the frame memory 33, the predictive tap forming circuit 34, the grade tap forming circuit 35, and the grade classification circuit of the boost information generating unit 11 in FIG. 6 36. The normal equation addition circuit 37, the prediction coefficient determination circuit 38 and the memory 39 are the same.
Therefore, the learning device of FIG. 16 basically performs the same processing as the promotion information generating unit 11 of FIG. 6 to calculate the prediction coefficients of each level. In the memory 101 of FIG. 14, the prediction coefficients of each level calculated in advance by learning in the learning device of FIG. 16 are stored.
Furthermore, in the embodiment of FIG. 16, by controlling the down-converter 112, the prediction tap configuration circuit 114, the level tap configuration circuit 115, and the level classification circuit 116, the tap configuration or the level classification method can be changed to calculate most types of prediction coefficients. .
Fig. 17 is an example of the configuration of the quality enhancement unit 24 of the receiving device 3 (Fig. 4) when the configuration of the enhancement information generating unit 11 is as shown in Fig. 14. In the figure, the same reference numerals are attached to the corresponding parts in FIG. 12, and the description thereof will be omitted.
The grade code storage unit 121 stores the grade code of the promotion information. That is, in the present situation, the transmitting device 1 transmits each pixel of the high-quality image for the image quality improvement of the broadcast image data using the level code as the improvement information. As the level code of the promotion information, it is supplied from the selection unit 23 of the trusted device 3 (FIG. 4) to the quality improvement unit 24 (FIG. 17). The grade code storage unit 121 stores the grade code of the promotion information. The level code storage unit 121 supplies the memorized level code as an address to the memory 122 under the control of the control circuit 57.
The memory 122 stores the prediction coefficients of each level calculated by the learning device in FIG. 16, and the level code storage unit 121 reads out the prediction coefficients corresponding to the level codes supplied as addresses, and supplies them to the prediction calculation circuit 56.
The quality improvement process for the image quality improvement of broadcast video data performed by the quality improvement unit 24 of FIG. 17 will be described with reference to the flowchart of FIG. 18.
In the receiving device 3 (FIG. 4), when the extracting unit 22 supplies the quality enhancement unit 24 with the broadcast image data of the enhancement information generating unit, among the most enhancement information, one type of enhancement information is selected according to the image quality level signal. And the method selection signal used to indicate the improvement method that uses the improvement information to improve the image quality supply is supplied from the selection section 23 to the quality improvement section 24.
After that, in step S131, in the frame memory 51, the broadcast image data supplied by the extracting unit 22 is memorized in the promotion information generating unit. In addition, in step S131, the promotion information supplied by the selection unit 23 is stored in the level code storage unit 121. In step S131, the control circuit 57 receives the mode selection signal supplied from the selection unit 23, and uses the mode selection signal corresponding to the boost mode to provide the control signal for instructing the image quality enhancement of the broadcast video data to the prediction tap configuration circuit 52. Accordingly, the predictive tap forming circuit 52 is set to perform processing in accordance with the boost mode indicated by the control signal of the control circuit 57.
In addition, in this embodiment, the mode selection signal supplied to the control circuit 57 indicates the linear interpolation, and the promotion information stored in the level code storage unit 121 is the level code.
When the mode selection signal supplied to the control circuit 57 indicates linear interpolation, the control circuit 57 supplies the predictive calculation circuit 56 with a control signal instructing to perform linear interpolation on the broadcast video data stored in the frame memory 51. In this case, the predictive calculation circuit 56 reads out the broadcast video data recorded in the frame memory 51 via the predictive tap forming circuit 52, and performs linear interpolation and output. Also, in this case, the processing after step S132 is not performed.
After the processing in step S131, it moves to step S132, and among the high-quality image constituent pixels for the image quality improvement of the broadcast image data stored in the frame memory 51, one of the pixels that has not been set as the notable pixel is set For the attention pixel, in the prediction tap configuration circuit 52, the prediction tap of the attention pixel of the configuration indicated by the control signal of the control circuit 57, the frame memory 51 is used to store the pixel configuration of the image data for broadcasting. This prediction tap is supplied to the prediction operation circuit 56.
After that, the process moves to step S133, and the control circuit 57 controls the level code storage unit 121 to enable reading of the level code as the promotion information of the relative attention pixel. Accordingly, the level code of the promotion information of the relatively noticeable pixel is read out and supplied to the memory 122.
In the memory 122, in step S133, the prediction coefficient stored in the address indicated by the class code of the class code storage unit 121 is read out and supplied to the prediction operation circuit 56.
The predictive calculation circuit 56 uses the predictive taps supplied by the predictive tap forming circuit 52 and the predictive coefficients supplied from the memory 55 in step S135 to perform the linear predictive calculation represented by the formula (1), and the obtained pixel value is used as a notable plot The predicted value of Su is temporarily memorized.
After that, it moves to step S136, and the control circuit 57 executes that all the constituent pixels of the frame of the high-quality image corresponding to the frame of the broadcast image data stored in the frame memory 51 are the notable pixels, and the predicted value is calculated. The judgment. In step S136, it is determined that all the constituent pixels of the frame of the high-definition image are notable pixels, and if there is a predicted value that has not been calculated, return to step S132, among the constituent pixels of the frame of the high-definition image , The ones that have not been set as attention pixels are set as new attention pixels, and the same processing will be repeated below.
In step S136, it is determined that all the constituent pixels of the frame of the high-quality image are notable pixels, and when the predicted value has been calculated, it moves to step S137, and the prediction arithmetic circuit 56 calculates the high-quality image of the calculated predicted value They are sequentially output to the display unit 25 (FIG. 4 ), and the processing ends.
In addition, the quality improvement processing of FIG. 18 is the same as the embodiment of FIG. 13 and is repeated when the frame-by-frame memory 51 is supplied with the image data for broadcasting of the improvement information generating unit.
In the embodiments of Figs. 14 and 17, the prediction coefficients commonly stored in the memory 101 of Fig. 14 and the memory 122 of Fig. 17 are obtained in the learning device of Fig. 16, and the level taps are classified using SD image composition. However, the prediction coefficients commonly stored in the memory 101 and the memory 122 do not use SD images, but may be obtained by using HD images to form a level tap for classification. In this case, in the promotion information generating unit 11, as the level code of the promotion information, the prediction coefficients of each level can be separately obtained as described above without calculating the predicted value of the attention pixel. That is, in this case, in each pixel constituting the HD image in the promotion information generating unit 11, the HD image composition level tap is used for classification, and the obtained level code can be used as the promotion information.
FIG. 19 is an example of the structure of the promotion information generating unit 11 which is a grade code of the promotion information by which the HD video (teacher video) constitutes a grade tap to classify the grades. In the figure, the same reference numerals are attached to the corresponding parts in FIG. 6, and the description thereof will be omitted. That is, the boost information generating unit 11 of FIG. 19 is the same as that of FIG. 6 except that the down-converter 32, the frame memory 33, the prediction tap formation circuit 34, the normal equation addition circuit 37, and the prediction coefficient determination circuit 38 are not provided. The composition is the same.
Hereinafter, the promotion information generation process for the promotion information generation performed by the promotion information generation unit 11 of FIG. 19 will be described with reference to the flowchart of FIG. 20.
In step S141, teacher images equivalent to the promotion information generating unit are stored in the frame memory 31. After that, it moves to step S142, the control circuit 40 enables it to obtain the boost information for the boost mode corresponding to the supplied mode selection signal, and outputs the instruction control signal to the level tap configuration circuit 35 and the level classification circuit 36. . According to this, the level tap forming circuit 35 and the level classification circuit 36 are set to be processed as a level code of the promotion information used in the promotion mode indicated by the control signal.
In addition, the mode selection signal supplied to the control circuit 40, like the above, contains information indicating a plurality of lifting modes. The control circuit 40 is a control signal corresponding to the plurality of lifting modes and outputs the control signals in accordance with the processing of step S142. .
When the control signal output by the control circuit 40 indicates linear interpolation, the memory 39 indicates the subject of the linear interpolation as the boosting information. After that, the processing from step S143 to step S145 is skipped, and the process moves to step S146.
After the processing of step S142, it moves to step S143. Among the teacher pixels stored in the frame memory 31, those that are not set as attention pixels are set as attention pixels. The level taps constitute the circuit 35 and the control circuit 40 The level classification of the attention pixels of the composition indicated by the control signal is constructed using the teacher pixels stored in the frame memory 33. The grade tap is supplied to the grade classification circuit 36.
The grade classification circuit 36, in step S144, constructs the grade taps of the circuit 35 according to the grade taps, uses the control signal instruction method of the control circuit 40 to classify the attention pixels, and supplies the grade codes corresponding to the resulting grades to After the memory 39 is memorized, it moves to step S145.
In step S145, it is determined by the control circuit 40 that all the teacher pixels of the promotion information generating unit stored in the frame memory 31 are regarded as the attention pixels, and whether to perform the classification classification, it is judged that all the teacher pixels are not used as the attention pixels for the classification. When sorting, go back to step S143. In this case, one of the teacher pixels that is not set as the attention pixel is regarded as the new attention pixel, and the processing from step S144 to step S145 is repeated.
In step S145, the control circuit 40 determines that all the teacher pixels of the boosting information generating unit are the attention pixels. When performing the addition operation, it moves to step S146. The control circuit 40 selects one of the most boosting methods included in the supplied method selection signal. All, determine whether to obtain promotion information.
In step S146, it is determined that among the majority of the lifting information used by the majority of the lifting methods included in the mode selection signal, if there is an unobtained one, move to step S142, and the control circuit 40 outputs the control signal corresponding to the lifting method for which the lifting information is not obtained. Repeat the same processing as above.
In addition, in step S146, it is determined that all of the most promotion methods included in the method selection signal have been obtained as the level code of the promotion information, that is, the most promotion methods included in the method selection signal are used separately as the level of the most types of promotion information When the code is stored in the memory 39, the process moves to step S147, and the plurality of types of promotion information are read out from the memory 39, and then sent to the integration unit 12 (FIG. 2) to end the process.
In addition, the promotion information generation process of FIG. 20 is repeated for each of the teacher images of the promotion information generation unit supplied to the frame memory 31.
Fig. 21 is an example of the configuration of the quality enhancement unit 24 of the receiving device 3 (Fig. 4) when the configuration of the enhancement information generating unit 11 is as shown in Fig. 19. In the figure, the same reference numerals are attached to the parts corresponding to those in FIG. 17, and the description thereof will be omitted. That is, the quality improvement part 24 of FIG. 21 replaces the memory 122 and has the same configuration as that of FIG. 17 except that the memory 131 is modified.
In the embodiment of Fig. 17, in the memory 122, the learning device of Fig. 16 uses a grade tap composed of student pixels to classify the prediction coefficients of each level obtained by learning. However, in the embodiment of Fig. 21, In the memory 131, the prediction coefficients of each level obtained by the learning of the level classification using the level taps composed of teacher pixels are memorized.
That is, FIG. 22 is an example of the configuration of an embodiment of a learning device that uses level taps composed of teacher pixels to perform level classification learning. In addition, in the figure, the same reference numerals are assigned to the corresponding parts in FIG. That is, the learning device of FIG. 22 basically has the same structure as that of FIG. 16.
However, in the learning device of FIG. 22, the level tap constitutes the circuit 115, and the student image stored in the frame memory 113 is used, but the teacher image stored in the frame memory 111 constitutes the level tap. In addition, the gradation tap configuration circuit 115 has the same gradation tap configuration as the gradation tap configuration circuit 35 that constitutes the boost information generating unit 11 in FIG. 19.
As mentioned above, even if the grade code is used as the promotion information, the picture quality can also be improved according to the user's request. Therefore, in this case, the image quality can be provided according to the user's request, and a subdivided billing can be applied to the image quality provided by the user.
In addition, as described above, when using the level code as the promotion information, even if the high-quality video data of the same content as the broadcast video data does not exist, it can be processed as effectively as using the prediction coefficient as the promotion information.
Also, as described above, broadcast video data and enhancement information can be used to form an integrated signal by, for example, time-sharing multiplexing or frequency multiplexing, but for broadcast video data, enhancement information can be embedded to form an integrated signal.
Hereinafter, the method of arranging the promotion information into the broadcast image data and the decoding method of decoding the promotion information incorporated in the method will be explained.
It is generally said that the informant has an energy (average amount of information) deviation (universality), and this deviation is identified as information (valuable information). That is, for example, an image obtained by capturing a landscape, the reason why people recognize such landscape ten images is because the image (the pixel value of each pixel that constitutes the image, etc.) has an energy shift corresponding to the landscape, and an image without energy shift , It's just noise, etc., of no use value.
Therefore, in the case of performing arbitrary operations on valuable information to destroy the original energy offset of the information, if the energy offset of the damage is restored to the original, the information subjected to arbitrary operations can also restore the meta-information. That is, the operation result data obtained from the operation information can be restored to the original valuable information by using the original energy offset of the information.
The representation of the energy shift that the information has has, for example, relevance.
The relevance of information refers to the correlation (such as self-correlation or the distance between a certain component and other components, etc.) between the various constituent elements of the information (for example, an image, which is the pixels or lines that constitute the image, etc.) . For example, there is a correlation between the lines of the image that represents the correlation of the image, and the correlation value of the correlation is, for example, the sum of the squares of the difference of the pixel values corresponding to the two lines. (In this case, the correlation value is smaller, Indicates that the correlation between the lines is large, and the larger the correlation value indicates that the correlation between the lines is small).
That is, regarding the image, for example, the line in the first row from the top (the first row) is related to other rows. Generally speaking, the closer the distance to the first row, the larger the row, and the greater the distance from the first row. The trip away becomes smaller. Therefore, the closer to the first row, the greater the correlation with the first row, and the smaller the correlation with the farther.
Below, perform the operation of replacing the pixel values of the Mth row closer to the first row and the Nth row farther from the first row (1<M<N). For the changed image, Calculate the value of the correlation between the first row and other rows, then the correlation between the M row close to the first row (the Nth row before replacement) becomes smaller, and the correlation between the Nth row farther from the first row (before the replacement) The correlation between line M) becomes larger.
Therefore, the closer the image after replacement is to the first row, the correlation becomes larger, and the further away, the smaller the correlation offset is destroyed. But for images, generally, the closer to the first row, the greater the correlation, and the farther away from the correlation offset, which reduces the correlation, to restore the corrupted correlation offset. That is, in the replaced image, the correlation with the Mth row close to the first row becomes smaller, and the correlation with the Nth row farther from the first row becomes larger. This is due to the original correlation bias of the image. If it is obviously unnatural (strange), the M and N rows should be replaced. Among the replaced images, by replacing the M line and the N line, the image with the original correlation shift can be restored, that is, the original image.
In this case, for example, which row of the moving image is determined based on the lifting information, or changing certain rows, etc., the movement or replacement of such rows is to embed the lifting information in the image. In addition, if the above-mentioned image with enhanced information is embedded, that is, the image in which the row has been replaced, the correlation can be used to replace the row back to its original position and restore to the original image. This is the decoding of the image and the enhanced information. That is, when restoring the original image, the detection of moving the first few lines or replacing certain lines is the decoding enhancement information.
As mentioned above, the energy offset of the image can be used for decoding. When the boosting information is embedded in the image, the embedded image can use the energy offset of the original image without increasing the decoding burden. Next, it can be restored to (decoded into) the original image and enhanced information.
In addition, for images, images obtained by embedding enhanced information (hereinafter referred to as embedded images) are different images from the original images, and are not images that can be recognized as valuable information. Therefore, the original images are , Can realize the coding without increasing the burden.
FIG. 23 is a configuration example of the integration unit 12 of FIG. 2 in which the enhancement information is embedded in the broadcast video data to generate an integrated signal as described above.
The frame memory 61 stores video data for broadcasting in units of, for example, one frame. In addition, the frame memory 61 is composed of a plurality of blocks. By the switching of the blocks, the memory of the supplied broadcast image data can be replaced at the same time as described below, and the data can be read from the frame memory 61 .
The exchange information generation unit 62 receives the promotion information from the promotion information generation unit 11 (FIG. 2), and generates rows of images (broadcasting image data) representing one frame unit stored in the frame memory 61 based on the promotion information Exchange information on what kind of replacement of the position, that is, when a frame of image stored in the frame memory 61 is composed of M rows and N columns of pixels, the nth column (nth column from the left) of the image is replaced with the nth<img file="TW577201B_D0008.tif" />When rowing, in the exchange information generating unit 62, n and n are generated<img file="TW577201B_D0009.tif" />Is assigned the corresponding exchange information (n, n'are integers from 1 to N).
When the number of rows in a frame of image is N, when all the rows are replaced, the replacement method is only N! (! Department means factorial). Therefore, in this case, a maximum of log2(N!) bits of boost information can be embedded in one frame.
The exchange information generated by the exchange information generating unit 62 is supplied to the exchange unit 63. The exchange unit 63 replaces the positions of each row of a frame of image stored in the frame memory 61 based on the exchange information supplied by the exchange information generating unit 62. Accordingly, the enhancement information can be embedded in the broadcast image data stored in the frame memory 61.
The embedding process performed by the integration unit 12 of FIG. 23 will be described below with reference to the flowchart of FIG. 24.
The frame memory 61 is supplied with broadcast video data, and the frame memory 61 sequentially stores the broadcast video data.
In addition, in the exchange information generating unit 62, in step S71, the enhancement information of the amount of data that can be embedded in one frame of the image (broadcasting image data) from the enhancement information generating unit 11 is received. That is, for example, the number of rows of video data for broadcasting in the above-mentioned general one frame is N rows, and when all rows are used as replacement objects, a maximum of log2(N!) bits of promotion information can be embedded in one frame, and the number of bits ( The promotion information below) is trusted.
After that, the exchange information generating unit 62 moves to step S72, and generates exchange information according to the trusted promotion information in step S71. That is, the exchange information generating unit 62 is based on the promotion information to generate a display for embedding the memory of the frame memory 61 in the first to Nth rows of the processing target frame (hereinafter referred to as the processing target frame), for example Except for the first row, the second to Nth rows are replaced with the exchange information of the first row. The exchange information is supplied to the exchange section 63.
The exchange unit 63, when the exchange information is received by the exchange information generating unit 62, moves to step S73, and replaces the position of each row of the processing target frame stored in the frame memory 61 according to the exchange information. In this way, different from the processing target frame, the enhancement information is embedded, that is, the broadcast image data (embedded image) in which the enhancement information is embedded is read out from the frame memory 61 and supplied to the transmitting unit 13 as an integrated signal (Fig. 2).
In addition, the position of each row of the frame can be changed by changing the memory position of the image data (composed pixels) of the frame memory 61, but other configurations are controlled by the address when the frame memory 61 is read out. As a result, the frame in which the column position change is performed can be read from the frame memory 61.
Furthermore, in this embodiment, as described above, the exchange information includes information indicating that the second row to the Nth row are replaced with the row, but does not include the information indicating the replacement of the first row with the row. Therefore, the exchange unit 37 performs the replacement of the second row to the Nth row, but does not perform the replacement of the first row. The reason will be described later.
After all the replacements of the second row to the Nth row of the processing target frame are completed, move to step S74 to determine whether the frame memory 61 stores a frame of the broadcast image data that is not set as the processing target frame, and return to when the judgment is stored In step S71, a frame that has not been set as a processing target frame is a new processing target frame, and the same processing is repeated below.
In step S74, when it is determined that the frame that is not stored in the frame memory 61 but is not set as the processing target frame, the embedding process is ended.
According to the above-mentioned embedding process, a certain frame of image (here, broadcast image data) can be formed into the integrated signal of the embedded image below.
That is, the promotion information, for example, as shown in FIG. 25, is changed so that the second row of the processing target frame of N rows (FIG. 25(A)) becomes the sixth row (FIG. 25(B)) and the third row. It becomes the 9th column (Figure 25(C)), the 4th column becomes the 7th column (Figure 25(D)), the 5th column becomes the 3rd column (Figure 25(E)), and the 6th column becomes the 8th column ( Figure 25(F)), the seventh column becomes the fourth column (Figure 25(G)), the eighth column becomes the fifth column (Figure 25(H)), and the ninth column becomes the second column (Figure 25(I)) ), ......, and the Nth column becomes the Nth column, which means that the exchange information for this kind of replacement is generated in the exchange information generating unit 62. After that, in the exchange section 63, for example, the frame shown in FIG. 25(J) is replaced by the above exchange information so that the second row becomes the sixth row, the third row becomes the ninth row, and the fourth row becomes the seventh row. The 5th column becomes the 3rd column, the 6th column becomes the 8th column, the 7th column becomes the 4th column, the 8th column becomes the 5th column, the 9th column becomes the 2nd column, ......, the Nth column becomes the Nth Column. As a result, the image in FIG. 25(J) becomes the buried image as shown in FIG. 25(K).
As mentioned above, the position of each row of pixels in the set of more than one pixel constituting the image stored in the frame memory 61 can be replaced corresponding to the elevation information. When the elevation information is embedded in each row, the reverse replacement can be performed. Get the original image. Moreover, which kind of replacement is performed becomes the promotion information. Therefore, it is possible to eliminate the deterioration of image quality as much as possible, and to embed improved information in the image without increasing the amount of data.
That is, each row of the image whose position is replaced by the row of the image with the enhanced information is embedded, and the correlation provided by the image can be used, that is, the correlation between the rows in the same correct position as the original image can be used. In the case of extra burden, it can be replaced to the original position, and with this replacement method, the information can be decoded and improved. Therefore, the resultant decoded image (reproduced image) basically does not cause deterioration of image quality due to increased information immersion.
In addition, when the embedded image does not have a row in the correct position, as described above, it is difficult to decode the image and enhance the information by using the correlation of the image. Therefore, in the embedding process of FIG. 24, the first row of each frame is not performed. replace.
However, it is also possible to embed all the rows including the first row as the replacement object. In this case, the original position of at least 1 or more of the row after the replacement is used as a supplement, and it is included in the integrated signal as the embedded image. Decoding of images and enhanced information is easy.
In addition, to improve the information, the rows can be replaced in order to be embedded in the image, or all rows can be replaced in the image at a time. That is, according to the promotion information, one row is replaced, one after the other, the next row is replaced, and so on, the promotion information can be embedded in the image, or the replacement pattern of all the rows can be determined according to the promotion information, and this is done again. This kind of replacement will embed the promotion information into the image.
Fig. 26 is an example of the configuration of the extracting unit 22 of the receiving device 3 (Fig. 4) when the integrating unit 12 of the transmitting device 1 (Fig. 2) is configured as shown in Fig. 23.
The frame memory 71 has the same structure as the frame memory 61 of the image selection unit 23, and the embedded image output from the receiving unit 21 (FIG. 4) as an integrated signal is sequentially memorized in, for example, frame units.
The exchange part 72 is the latest row in the embedded image stored in the calculation frame memory 71 (the processing target frame) that has been replaced with the original position, and the other rows (the row that has not returned to the original position) According to the correlation, the position in the processing target frame that has not been restored to the original position is replaced according to the correlation, and the original position (decoding example position) is restored accordingly. In addition, the exchange unit 72 supplies exchange information indicating how each row of the frame is replaced to the exchange information conversion unit 73.
The exchange information conversion unit 73 decodes the enhancement information of the embedded image based on the exchange information from the exchange unit 72, that is, the correspondence between the position before the replacement of each row of the processing target frame and the position after the replacement. .
Hereinafter, with reference to the flowchart of FIG. 27, the decoding process of the embedded image decoding performed by the extracting unit 22 of FIG. 26 to extract the original broadcast image data and the enhancement information will be explained.
In the frame memory 71, the supplied embedded images (encoded data) are sequentially stored, for example, in units of 1 frame.
72, switching to the step portion in step S81, counting the number of columns of a frame of the variable n is set to an initial value 1 for example, moves to step S82, the variable n is determined whether the number of columns N of the frame N-1 of subtracting 1 the following.
In step S82, when it is determined that the variable n is N-1 or less, the process moves to step S83, and the exchange unit 72 reads the nth column of pixels (pixel column) from the processing target frame stored in the frame memory 71, and uses the nth Column frame pixel (the pixel value) generates a parallel vector (hereinafter referred to as column vector) V<sub>n</sub>. In this embodiment, as the above-mentioned frame is composed of M rows of pixels, the column vector V<sub>n</sub>(The column vector V described later<sub>k</sub>The same is true) is an M-dimensional vector.
After that, in step S84, the initial value of the counting variable n in the right column of the Nth column is set to n+1, and the process moves to step S85. The exchange unit 72 reads out the pixels in the kth column and generates Is a list of elements vector V<sub>k</sub>, Move to step S86.
In step S86, the exchange unit 72 uses the column vector Vn and the column vector V<sub>k</sub>, Calculate the correlation between the nth column and the kth column.
That is, in the exchange part 72, the column vector Vn and the column vector V<sub>k</sub>The distance d(n,k) can be calculated according to formula (8),
<maths><img file="TW577201B_D0010.tif" /></maths>
However, in formula (8), Σ represents the sum of m from 1 to M. In addition, A(i,j) represents the pixel (pixel value) in the i-th row and j-th column of the processing target frame.
After that, in the exchange unit 72, the column vector Vn and the column vector V<sub>k</sub>The reciprocal 1/d(n,k) of the distance between d(n,k) is calculated as the correlation value between the nth column and the kth column.
After the correlation between the nth column and the kth column is calculated, it moves to step S87 to determine whether the variable k is N-1 or less of the number of columns N minus 1 of the frame. When it is judged in step S87 that the variable k is less than N-1, in step S88, the k-th column is incremented by 1, and the process returns to step S85. After that, in step S87, repeat step S85 before judging that the variable k is not less than N-1 Go to the processing of step S88. That is, according to this, the individual correlation between the nth row and the row of embedded images on the right is calculated.
After that, in step S87, when it is determined that the variable k is not less than N-1, the process moves to step S88, and the exchange unit 72 calculates the k with the largest correlation with the nth column. After that, the correlation with the nth column is the largest k, for example, when represented by K, the exchange unit 72, in step S90, exchanges the n+1th column and the Kth column of the processing target frame stored in the frame memory 71 , That is, the Kth column and the n+1th column to the right of the nth column are replaced.
After that, in step S91, the variable n is decremented by 1, and the process returns to step S82. Following step S82, the process from step S82 to step S91 is repeated until it is determined that the variable n is not less than N-1.
In this embodiment, the first column of the embedded image is the first column of the original image, so when the variable n is the initial value 1, the column of the embedded image with the highest correlation with the first column of the class is replaced with The first column is the second column to the right. The column with the highest correlation with the first column is basically the second column of the original image in terms of image correlation. Therefore, in this case, in the embedding process, it is replaced with the original position of the embedded image The second column of the image is restored to its original position (decoding).
When the variable n is 2, the column of the embedded image with the highest correlation with the second row that was replaced with the original position as described above is replaced with the third row next to the second row. The column with the highest correlation with the second column is basically the third column of the original image in terms of image correlation. Therefore, in this case, in the embedding coding process, it is replaced with the position of the embedded image. The third column of the original image is restored to its original position
In the same manner below, the embedded image stored in the frame memory 71 is decoded into the original image (video data for broadcasting).
In step S82, when it is determined that the variable n is not N-1, that is, the correlations of all used images from the second row to the Nth row constituting the embedded image are restored to the original position, and the memory 71 is embedded according to this frame When the image is decoded into the original image (video data for broadcasting), it moves to step S92, and the decoded image is read out from the frame memory 7L. In step S92, when the embedded image is decoded into the original image by the exchange unit 72, the exchange information representing the individual replacement method of the second row to the Nth row of the embedded image is output to the exchange information conversion unit 73. After that, in the exchange information conversion part 73, according to the exchange information of the exchange part 72, the enhancement information embedded in the image is decoded.
After that, it moves to step S93 to determine whether there is a frame that has not been set as a processing target in the frame memory 71 and is memorized. If it is determined that sometimes, it returns to step S81, and the embedded image frame that is not set as the processing target is a new one. For processing objects, repeat the same processing below.
In addition, in step S93, when it is determined that the frame that has not been set as the processing target is no longer stored in the frame memory 71, the decoding process is terminated.
As mentioned above, using the correlation of the image, the embedded image embedded with the enhancement information is decoded into the original image and the enhancement information. Therefore, even if there is no auxiliary information for decoding, the embedded image can be decoded into the original image. Images and enhanced information. Therefore, the decoded image basically does not degrade the image quality due to the embedding of the enhancement information.
In addition, in the decoding process structure of Fig. 27, the latest decoded column (for example, when n=1, the first column that was not replaced when embedded) is calculated, and the correlation between the column that has not been decoded is calculated according to The correlation detects the latest decoded column to the right of the column to be replaced, but others, such as the correlation between the decoded majority column and the undecoded column, can be used to detect the decoded latest column to the right. Farming is also possible.
Moreover, in the above case, it is the replacement of borrowed rows, and the enhancement information is embedded in the broadcast image data, but the embedded replacement can also be borrowed, or a certain pixel at the same position of a specific number of frames juxtaposed in the time direction The position of the column is changed, or by the replacement of the column and the row, etc.
In addition, embedding is not an operation of replacing rows, etc., but making pixel values operate according to the elevation information, and making horizontal rows and the like rotate according to the elevation information. In either case, the energy offset can be used to decode the original information.
In addition, as described above, the method of embedding as the original information can be decoded by using the energy offset, the details of which are as previously proposed by the present inventors, such as Japanese Patent Application No. 10-200093, Japanese Patent Application No. 10-222951, and Japanese Patent Application No. 10-222951. No. 10-333700, Special Yuanping No. 11-129919, Special Yuanping No. 11-160529, Special Yuanping No. 11-160530, Special Yuanping No. 11-284198 (domestic priority based on Special Yuanping No. 10-285310 Patent Claims), Special Yuanping No. 11-284199 (domestic priority claims based on Special Yuanping No. 10-285309), Special Yuanping No. 11-284200 (based on Special Yuanping No. 10-285308) The record of the domestic priority claim case), the record method can be used in the integration section 12 and the extraction section 22.
In the following, the embedding processing method for embedding and enhancing information in broadcast image data can also use spectrum diffusion.
Fig. 28 is an example of the configuration of the integration unit 12 of the transmission device (Fig. 2) that uses the spectrum diffusion to embed the enhancement information in the broadcast image data.
The boosting information output by the boosting information generating unit 11 (Figure 2) is supplied to the spectrum diffusion signal generating circuit 81. The spectrum diffusion signal generating circuit 81 generates PN (Pseudo random noise) of the M series in order, for example, according to a specific sequence. Code column. After that, the spectrum diffusion signal generating circuit 81 applies spectrum diffusion to the boosted information according to the PN code train to obtain a spectrum diffusion signal, which is supplied to the addition circuit 82.
In addition circuit 82, in addition to the spectrum diffusion signal supplied by the spectrum diffusion signal generating circuit 81, it is also supplied to broadcast video data. The addition circuit 82 is used for broadcasting video data to superimpose the spectrum diffusion signal to obtain broadcast video data. The integrated signal with the enhanced information is embedded in it, and it is output to the transmitting unit 13 (Figure 2).
In addition, broadcast video data and spectrum diffusion signals may be supplied to the addition circuit 82 after D/A (Digital Analog) conversion.
Fig. 29 is an example of the configuration of the extracting unit 22 of the receiving device 3 (Fig. 4) when the integration unit 12 of the transmitting device 1 (Fig. 2) is configured as shown in Fig. 28.
The integrated signal output from the receiving unit 21 (FIG. 4) is supplied to the inverse spectrum diffusion circuit 91 and the decoding circuit 92.
The inverse spectrum diffusion circuit 91 generates the same PN code sequence as that generated by the spectrum diffusion signal generating circuit 81 of FIG. 28, and applies inverse spectrum diffusion to the integrated signal according to the PN code sequence, and thereby can decode and enhance the information. The decoded promotion information is supplied to the selection part 23 (FIG. 4).
In addition, the inverse spectrum diffusion circuit 91 supplies the generated PN code string to the decoding circuit 92.
The decoding circuit 92 is based on the PN code string supplied by the inverse spectrum diffusion circuit 91 to remove the spectrum diffusion signal superimposed on the integrated signal, and thereby can decode the broadcast image data. The decoded broadcast video data is supplied to the quality improvement unit 24 (Figure 4).
As shown in FIG. 29, the extraction unit 22 may not include the decoding circuit 92. In this case, the broadcast video data on which the spectrum diffusion signal is superimposed is supplied to the quality improvement unit 24.
The above description is based on the method of embedding as the original information can be decoded by using energy shift and the method of embedding by using spectrum diffusion. However, for the embedding of enhanced information in broadcast image data, other electronic transmission methods such as conventional ones can also be used. law.
That is, if the lower 1 bit or 2 bits that constitute the broadcast image data are replaced (replaced) with enhancement information, the enhancement information can be embedded in the broadcast image data.
Secondly, the above series of processing can also be carried out by hardware or software. When a series of processing is carried out by software, the programs constituting the software can be installed on general-purpose computers, etc.
Fig. 30 is a configuration example of an embodiment of a computer for program installation for executing the series of processes described above.
The program can be pre-stored in the hard disk 205 or ROM 203 of the recording medium in the computer.
Alternatively, the program can be temporarily or permanently stored in removable recording media such as floppy disk, CD-ROM (Compact Disc Read Only Memory), MO (Magneto optical) disc, DVD (Digital Versatile Disc), magnetic disk, semiconductor memory, etc. 211. The erasable recording medium 211 can be provided as a so-called packaged software.
In addition, the program can be installed on the computer by the above-mentioned removable recording medium 211, and can also be transmitted to the computer wirelessly from the download side, via the artificial satellite for digital satellite broadcasting, or via LAN (Local Area Network), The Internet and other networks are transmitted to the computer in a wired manner, and on the computer side, the program received by the communication unit 208 is installed on the built-in hard disk 205.
The computer contains a CPU (Central Processing Unit) 202. The CPU 202 is connected to the I/O interface 210 via the bus 201. The CPU 202 is connected to the I/O interface 210. When the user operates the keyboard or mouse, etc., the input unit 207 is used for input When instructed, execute the program stored in ROM203 according to the instruction. Alternatively, the CPU 202 transmits the program stored in the hard disk 205 via satellite or network, and receives the program installed in the hard disk 205 from the communication unit 208, or reads and installs the program from the removable recording medium 211 installed in the drive 209 The program on the hard disk 205 is loaded into RAM (Random Access Memory) 204 and executed. According to this, the CPU 202 can perform the processing of the above-mentioned flowchart or the processing of the above-mentioned block diagram configuration. When necessary, the CPU 202 can output the processing result via the input/output interface 210, an output unit 206 composed of LCD (Liquid Cuystal Display) or a speaker, etc., or can be transmitted by the communication unit 208, or recorded on the hard disk 205.
In addition, in this manual, the processing steps used by the computer to perform various processing programs are not necessarily time-series processing in the order described in the flowchart, and include processing performed in parallel or individually (such as parallel processing or object processing, etc.).
In addition, the program can be processed by one computer or distributed by multiple computers. The program can also be sent and executed by a remote computer.
In addition, in this embodiment, video data is used as the object, but the present invention can also be used with other audio data, for example.
In addition, in this embodiment, the embedded image is provided through the satellite circuit as an explanation, but the embedded image can also be provided through other transmission media such as ground waves, the Internet, CATV, etc., which are stored in Various recording media such as optical discs, optical magnetic discs, magnetic tapes, and semiconductor memories are also possible.
(Effects of Invention)
According to the first data processing device and data processing method, as well as the recording medium and the program of the present invention, the improvement information for data quality improvement is generated, and the improvement information is embedded in the data. Therefore, for example, it is possible to embed data with the state of improving information, data with the state of improving the information being extracted, and data whose quality is improved by improving the quality of the information.
According to the second data processing device and data processing method, as well as the recording medium and the program of the present invention, the enhanced information can be extracted from the embedded data, and the quality of the data can be improved by using the enhanced information. Therefore, the provision of high-quality information is acceptable.
According to the third data processing device and data processing method, as well as the recording medium and program of the present invention, it is possible to generate most types of improvement information for data quality improvement, and data and more than one type of improvement information are transmitted. Therefore, most quality data can be provided.
According to the fourth data processing device and data processing method of the present invention, as well as the recording medium and program, data and more than one type of promotion information are trusted, and the data quality can be improved by any one of more than one type of promotion information. In addition, it can be The data quality improvement uses the promotion information for billing processing. Therefore, the corresponding quality can be accepted according to the payment amount.
<p>1. . . Transmission device 1A. . . Antenna 2. . . Satellite 3. . . Trusted device 3A. . . Antenna 4. . . Network 11. . . Improve the information generation department 12. . . Integration Department 13. . . Communication Department 14. . . Billing processing section 15. . . Communication interface 21. . . Department of Information 22. . . Extraction part 23. . . Selection Department 24. . . Quality Improvement Department 25. . . Department of Communications 26. . . Operation Department 27. . . Billing processing section 28. . . Communication interface 31. . . Frame memory 32. . . Down-frequency converter 33. . . Frame memory 34. . . Predictive tap formation circuit 35. . . Hierarchical taps constitute the circuit 36. . . Level classification circuit 37. . . Normal equation addition circuit 38. . . Prediction coefficient determination circuit 39. . . Memory 40. . . Control circuit 41. . . Frame memory 42. . . Feature quantity estimation circuit 43. . . Virtual teacher information generating circuit 44. . . Virtual student data generation circuit 51. . . Frame memory 52. . . Predictive tap formation circuit 53. . . Hierarchical taps constitute the circuit 54. . . Level classification circuit 55. . . Memory 56. . . Predictive calculation circuit 57. . . Control circuit 61. . . Frame memory 62. . . Exchange information generating unit 63. . . Exchange Department 71. . . Frame memory 72. . . Exchange Department 73. . . Exchange Information Conversion Section 81. . . Spectrum diffusion signal generating circuit 82. . . Adding circuit 91. . . Inverse spectrum diffusion circuit 92. . . Decoding circuit 101. . . Memory 102. . . Predictive operation circuit 103. . . Comparison circuit 104. . . Detection circuit 111. . . Frame memory 112. . . Frequency reduction inverter 113. . . Frame memory 114. . . Predictive tap formation circuit 115. . . Level taps constitute the circuit 116. . . Hierarchical classification circuit 117. . . Normal equation addition circuit 118. . . Prediction coefficient determination circuit 119. . . Memory 121. . . Level code memory 122, 131. . . Memory 201. . . Busbar 202. . . CPU203. . . ROM204. . . RAM205. . . Hard Disk 206. . . Interface 207. . . Input unit 208. . . Department of Communications 209. . . Drive 210. . . I/O interface 211. . . Erasable recording media</p>
Fig. 1: An example of the configuration of an embodiment of a broadcasting system to which the present invention is applied.
Fig. 2: A block diagram of a configuration example of the transmission device 1.
Fig. 3: A flowchart illustrating the processing of the transmitting device 1.
Figure 4: A block diagram of an example of the structure of the trusted device 3.
Fig. 5: A flowchart of the processing description of the trusted device 3.
Fig. 6: A block diagram of the first configuration example of the promotion information generating unit 11.
Figure 7: The composition diagram of the prediction tap (level tap).
Figure 8: Correspondence diagram of mode selection signal and lifting mode.
Fig. 9: The processing flowchart of the promotion information generating unit 11 in Fig. 6.
Fig. 10: A block diagram of the second configuration example of the promotion information generating unit 11.
Fig. 11: The processing flow chart of the promotion information generating unit 11 in Fig. 10.
Fig. 12: A block diagram of the first configuration example of the quality improvement unit 24.
Fig. 13: The processing flow chart of the quality improvement unit 24 in Fig. 12.
Fig. 14: A block diagram of the third configuration example of the promotion information generating unit 11.
Fig. 15: The processing flowchart of the promotion information generating unit 11 in Fig. 14.
Figure 16: Block diagram of the first example of the configuration of the learning device for calculating prediction coefficients.
Fig. 17: A block diagram of the second configuration example of the quality improvement unit 24.
Fig. 18: The processing flowchart of the quality improvement unit 24 of Fig. 17.
Fig. 19: A block diagram of the fourth configuration example of the promotion information generating unit 11.
Fig. 20: The processing flow chart of the promotion information generating unit 11 in Fig. 19.
Fig. 21: A block diagram of the third configuration example of the quality improvement unit 24.
Figure 22: A block diagram of the second example of the configuration of the learning device for calculating prediction coefficients.
Fig. 23: A block diagram of a configuration example of the integration unit 12.
Fig. 24: The processing flow chart of the integration unit 12 in Fig. 24.
Figure 25: An explanatory diagram of the replacement of the image column.
Fig. 26: A block diagram of an example of the configuration of the extraction unit 22.
Fig. 27: The processing flowchart of the extraction unit 22 of Fig. 26.
Fig. 28: A block diagram of another configuration example of the integration unit 12.
Fig. 29: A block diagram of another example of the configuration of the extraction unit 22.
Fig. 30: A block diagram of a configuration example of an embodiment of a computer to which the present invention is applied.
18 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18
17 members in 8 offices
Priority claims7
| Document | Office | Kind | Date |
|---|---|---|---|
| 2000053098 | Japan | – | |
| 2000053098 | Japan | A | |
| 2001045251 | Japan | A | |
| 20000053098 | – | – | – |
| 20010045251 | – | – | – |
| JP20000053098 | – | – | – |
| JP20010045251 | – | – | – |
Members17
| Document | Office | Kind | |
|---|---|---|---|
| WO0165847A1 | World Intellectual Property Organization (WIPO) | A1 | |
| JP2001320682A | Japan | A | |
| KR20010113047A | Republic of Korea | A | |
| EP1176824A1 | European Patent Office (EPO) | A1 | |
| CN1366769A | China | A | |
| US2003103668A1 | United States of America | A1 | |
| TW577201BThis record | Taiwan Province of China | B | |
| EP1176824A4 | European Patent Office (EPO) | A4 | |
| EP1755344A2 | European Patent Office (EPO) | A2 | |
| US2008240599A1 | United States of America | A1 | |
| CN101370131A | China | A | |
| CN100477779C | China | C | |
| US7679678B2 | United States of America | B2 | |
| EP1176824B1 | European Patent Office (EPO) | B1 | |
| DE60141734D1 | Germany | D1 | |
| EP1755344A3 | European Patent Office (EPO) | A3 | |
| CN101370131B | China | B |
1 legal event, as the office reported them to INPADOC
Events
| Event | Code | |
|---|---|---|
| Annulment or lapse of patent due to non-payment of feesLapsedMM4A | MM4A |
Numbers
- Publication
- 577201
- Publication, DOCDB
- 577201
- Publication, EPODOC
- TW577201B
- Application
- 90104580
- Application, DOCDB
- 90104580
- Application, EPODOC
- TW20010104580
Titles4
- English
- Data processing device, data processing method, recording medium and program
- Chinese
- 資料處理裝置及資料處理方法以及記錄媒體及程式
- Unlabeled
- 資料處理裝置及資料處理方法以及記錄媒體及程式
- Unlabeled
- Data processing device, data processing method, recording medium and program
Classification
- IPC, 7
- H04N7 08
- H04H1 00
- H04L29 00
- H04N7 081
- H04N7 16
- H04N21 238
- H04N21 4402