Image processing apparatus, and method and program product for detecting image updates
11 claims: 4 independent, 7 dependent
- 1ネットワークを介して接続された端末装置に画面を表示させるための画像情報を生成して送信する画像処理装置であって、 前記端末装置に送信する前記画像情報を記憶可能な第1画像記憶部と、 前記画面に含まれる予め定められた大きさの矩形領域に対応して前記第1画像記憶部の記憶領域に設定されるページ ごとに 、前記第1画像記憶部の仮想アドレスを物理アドレスに変換するための エントリを有する 変換テーブルを記憶するテーブル記憶部と、 前記矩形領域に含まれる画素に対する仮想アドレスを算出するアドレス算出部と、 前記 変換テーブル を参照して 、書込みが要求された前記画像情報の各画素に対して算出された前記仮想アドレスに対応する前記物理アドレスを取得する取得部と、 取得された前記物理アドレスに、書込みが要求された前記画像情報を書込む書込み部と、 前記取得部が前記物理アドレスを取得する際に参照した前記エントリに対応するページを、 書込み前後で前記画像情報が一致しない部分を表す前記画像情報の更新部分を 含むページとして 検出する検出部と、 検出された 前記ページに含まれる 前記更新部分の前記画像情報を圧縮する圧縮部と、 を備えたことを特徴とする画像処理装置。
- 2前記ページ単位で前記画像情報を記憶可能な第2画像記憶部と、 書込みが要求された前記画像情報の前記仮想アドレスを含む前記ページに対して、前記ページの書込みが要求された場合に、ページ例外の割込みを発生する例外発生部と、 前記ページ例外が発生した前記ページに記憶された前記画像情報であって、前記書込み部によって書込む前の前記画像情報を前記第2画像記憶部に保存する保存部と、をさらに備え、 前記検出部は、さらに、前記画像情報が前記第2画像記憶部に保存された前記ページを、前記画像情報が更新された前記ページとして検出すること、 を特徴とする請求項1に記載の画像処理装置。
- 3前記検出部は、更新された前記ページそれぞれについて、前記第2画像記憶部の前記画像情報と、前記書込み部によって書込まれた後の前記第1画像記憶部の前記画像情報とを比較して前記更新部分を検出すること、 を特徴とする請求項2に記載の画像処理装置。
- 4前記テーブル記憶部は、前記ページごとに、前記ページに対する書込みの有無を表す第1判定情報をさらに記憶し、 書込みが要求された前記画像情報の前記仮想アドレスを含む前記ページの前記第1判定情報を書込み有りに変更する変更部をさらに備え、 前記検出部は、さらに、前記第1判定情報が書込み有りである前記ページを、前記画像情報が更新された前記ページとして検出すること、 を特徴とする請求項1に記載の画像処理装置。
- 5前記書込み部によって書込む前の前記画像情報を記憶する第2画像記憶部と、 前記書込み部によって前記画像情報が書込まれた後に、書込み後の前記画像情報を前記第2画像記憶部に保存する保存部と、をさらに備え、 前記検出部は、更新された前記ページそれぞれについて、前記第2画像記憶部の前記画像情報と、前記書込み部によって書込まれた後の前記第1画像記憶部の前記画像情報とを比較して前記更新部分を検出すること、 を特徴とする請求項4に記載の画像処理装置。
- 6前記テーブル記憶部は、予め定められた個数の前記ページのグループごとに、前記グループに含まれる前記ページに対する参照の有無を表す第2判定情報をさらに記憶し、 前記変更部は、さらに、書込みが要求された前記画像情報の前記仮想アドレスを含む前記ページを含む前記グループの前記第2判定情報を参照有りに変更し、 前記検出部は、前記第2判定情報が参照有りである前記グループを検出し、検出した前記グループに含まれる前記ページを対象として、前記第1判定情報が書込み有りである前記ページを、前記画像情報が更新された前記ページとして検出すること、 を特徴とする請求項4に記載の画像処理装置。
- 7前記テーブル記憶部は、前記画面を分割した分割領域に含まれる前記矩形領域それぞれに対応する前記ページの前記グループごとに、前記第2判定情報を記憶すること、 を特徴とする請求項6に記載の画像処理装置。
- 8前記アドレス算出部は、前記矩形領域内の水平または垂直の方向である特定方向に隣接する各画素を含むラインに対し、前記ライン内の画素間で前記特定方向に連続し、かつ、隣接するライン間で連続する前記仮想アドレスを算出すること、 を特徴とする請求項1に記載の画像処理装置。
- 9前記アドレス算出部は、同一の前記画像情報に対して、書込みが要求された場合に第1仮想アドレスを算出し、読出しが要求された場合に第2仮想アドレスを算出し、 前記テーブル記憶部は、前記第1仮想アドレスおよび前記第2仮想アドレスをそれぞれ同一の前記物理アドレスに変換するための前記変換テーブルを記憶し、 前記ページ単位で前記画像情報を記憶可能な第2画像記憶部と、 前記第1仮想アドレスを含む前記ページに対して書込みが要求された場合に、ページ例外の割込みを発生する例外発生部と、 前記ページ例外が発生した前記ページに記憶された前記画像情報であって、前記書込み部によって書込む前の前記画像情報を前記第2画像記憶部に保存する保存部と、をさらに備え、 前記検出部は、さらに、前記画像情報が前記第2画像記憶部に保存された前記ページを、前記画像情報が更新された前記ページとして検出すること、 を特徴とする請求項1に記載の画像処理装置。
- 10ネットワークを介して接続された端末装置に画面を表示させるための画像情報を生成して送信する画像処理装置における画像情報の更新検出方法であって、 前記画像処理装置は、 前記端末装置に送信する前記画像情報を記憶可能な第1画像記憶部と、 前記画面に含まれる予め定められた大きさの矩形領域に対応して前記第1画像記憶部の記憶領域に設定されるページ ごとに 、前記第1画像記憶部の仮想アドレスを物理アドレスに変換するための エントリを有する 変換テーブルを記憶するテーブル記憶部と、を備え、 アドレス算出部 が 、前記矩形領域に含まれる画素に対する仮想アドレスを算出するアドレス算出ステップと、 取得部 が、前記 変換テーブル を参照して 、書込みが要求された前記画像情報の各画素に対して算出された前記仮想アドレスに対応する前記物理アドレスを取得する取得ステップと、 書込み部 が 、取得された前記物理アドレスに、書込みが要求された前記画像情報を書込む書込みステップと、 検出部 が、前記取得部が前記物理アドレスを取得する際に参照した前記エントリに対応するページを、 書込み前後で前記画像情報が一致しない部分を表す前記画像情報の更新部分を 含むページとして 検出する検出ステップと、 を備えたことを特徴とする更新検出方法。
- 11ネットワークを介して接続された端末装置に画面を表示させるための画像情報を生成して送信する画像処理装置における画像情報の更新検出プログラムであって、 前記画像処理装置は、 前記端末装置に送信する前記画像情報を記憶可能な第1画像記憶部と、 前記画面に含まれる予め定められた大きさの矩形領域に対応して前記第1画像記憶部の記憶領域に設定されるページ ごとに 、前記第1画像記憶部の仮想アドレスを物理アドレスに変換するための エントリを有する 変換テーブルを記憶するテーブル記憶部と、を備え、 前記矩形領域に含まれる画素に対する仮想アドレスを算出するアドレス算出手順と、 前記 変換テーブル を参照して 、書込みが要求された前記画像情報の各画素に対して算出された前記仮想アドレスに対応する前記物理アドレスを取得する取得手順と、 取得された前記物理アドレスに、書込みが要求された前記画像情報を書込む書込み手順と、 前記取得手順で前記物理アドレスを取得する際に参照した前記エントリに対応するページを、 書込み前後で前記画像情報が一致しない部分を表す前記画像情報の更新部分を 含むページとして 検出する検出手順と、 をコンピュータに実行させる更新検出プログラム。
Independent claims11
172 paragraphs, as filed
The present invention relates to a device for generating image information to be transferred to a terminal device connected via a network, a method for detecting image updates, and a program.
Generally, a computer having a function of displaying an image such as a PC (Personal Computer) has a frame buffer such as VRAM (Video Random Access Memory) that holds graphic data (image information) to be displayed on the display. The frame buffer is a memory area corresponding to XY coordinates, and has addresses linearly arranged in an order according to a raster scan method. The LCD (Liquid Crystal Display) controller and CRT (Cathode Ray Tube) controller acquire image information from the frame buffer and display it on the monitor in accordance with the raster scan.
In this way, the frame buffer of the graphic device of the conventional computer is configured to arrange the screen data linearly in the raster direction of the screen display so as to be advantageous when displaying on the monitor connected to the computer. ..
On the other hand, a system for displaying a display screen of an application or the like running on a virtual computer server on a terminal device, a system for remotely controlling another terminal device, and the like have been developed. In such a system, it is necessary to transfer the image information acquired from the frame buffer to an external device.
For example, Patent Document 1 proposes a technique relating to a remote control system for remotely controlling a remote terminal from a local terminal. Specifically, in Patent Document 1, for the purpose of improving security and power saving, the screen of the remote terminal to be operated during remote operation is set to the power saving mode, and the passphrase on the operation source terminal side is set. It discloses a method of unlocking the remote terminal by input.
Generally, in the conventional technology including Patent Document 1, the data to be written in the frame buffer and the data to be transferred to the remote control terminal are determined by updating the desktop screen, and only the image information of the updated part is transferred. .. In this way, the increase in communication load in a computer having a high-spec graphic function is dealt with. Further, a method of performing image compression at the time of transfer in order to reduce the communication load is also known.
As a method of detecting the updated portion of the image information stored in the frame buffer, there is a method in which software that writes to the frame buffer declares the coordinate information of the updated portion. However, there are drawbacks such as the need to change the software procedure to declare the coordinate information of the updated part and the increase in the overhead for declaration.
Further, in the case of a processor having a virtual memory management function, a method of detecting an updated part of image information by using a memory management unit (MMU) can be considered. The MMU usually has a translation table for finding the physical address from the virtual address. Then, a bit value (dirty bit) for determining whether or not to write is set for each page is set in the conversion table. Therefore, in the method using the MMU, the page updated by the value of the dirty bit is detected, and the updated part of the image information is detected for the detected page.
<patcit num="1"><text>Japanese Unexamined Patent Publication No. 2003-85135</text></patcit>
<p> However, the method using the dirty bit of the conversion table has a problem that the detection efficiency of the updated part may deteriorate. For example, in the case of a frame buffer structure in which the amount of data per pixel is 4 bytes and one page is 4096 bytes on a screen of 1024 x 768 pixels, one line in the raster direction corresponds to one page. To do.</p><p> Therefore, for example, in a desktop application that handles a relatively large number of rectangular figures, if the rectangular area is updated, the same number of page updates as the lines corresponding to the height of the rectangle will be detected. .. Then, it becomes necessary to detect the rectangular update portion using each of the detected update pages. That is, since the ratio of the updated portion to the updated page becomes small, it is necessary to execute the update portion detection process for more updated pages.</p><p> The present invention has been made in view of the above, and is a device capable of reducing the processing load related to detection of an updated portion of an image when managing a frame buffer by a virtual memory method, and detects an image update. The purpose is to provide methods and programs.</p>
<p> In order to solve the above-mentioned problems and achieve the object, the present invention is an image processing device that generates and transmits image information for displaying a screen on a terminal device connected via a network. The first image storage unit that can store the image information to be transmitted to the terminal device and the storage area of the first image storage unit corresponding to a rectangular area of a predetermined size included in the screen are set. page<u style="single">Every time</u>, For converting the virtual address of the first image storage unit into a physical address<u style="single">Have an entry</u>A table storage unit that stores a conversion table, an address calculation unit that calculates a virtual address for pixels included in the rectangular area, and an address calculation unit.<u style="single">Said</u>Conversion table<u style="single">See</u>, The acquisition unit that acquires the physical address corresponding to the virtual address calculated for each pixel of the image information for which writing is requested, and the image information for which writing is requested for the acquired physical address. And the writing part to write<u style="single">The page corresponding to the entry referred to when the acquisition unit acquires the physical address,</u>The updated part of the image information representing the part where the image information does not match before and after writing<u style="single">As a page to include</u>The detector to detect and the detected<u style="single">Included on the page</u>It is characterized by including a compression unit for compressing the image information of the update portion.</p><p> Further, the present invention is a method for detecting an update of image information in an image processing device that generates and transmits image information for displaying a screen on a terminal device connected via a network. It is set in the storage area of the first image storage unit that can store the image information to be transmitted to the terminal device and the storage area of the first image storage unit corresponding to a rectangular area of a predetermined size included in the screen. Page<u style="single">Every time</u>, For converting the virtual address of the first image storage unit into a physical address<u style="single">Have an entry</u>It has a table storage unit that stores the conversion table, and an address calculation unit.<u style="single">But</u>, An address calculation step for calculating a virtual address for a pixel included in the rectangular area, and an acquisition unit.<u style="single">But the above</u>Conversion table<u style="single">See</u>, An acquisition step of acquiring the physical address corresponding to the virtual address calculated for each pixel of the image information requested to be written, and a writing unit.<u style="single">But</u>, A writing step of writing the image information requested to be written to the acquired physical address, and a detection unit.<u style="single">However, the page corresponding to the entry referred to when the acquisition unit acquires the physical address is displayed.</u>The updated part of the image information representing the part where the image information does not match before and after writing<u style="single">As a page to include</u>It is characterized by having a detection step for detecting.</p><p> Further, the present invention is an update detection program for image information in an image processing device that generates and transmits image information for displaying a screen on a terminal device connected via a network. It is set in the storage area of the first image storage unit that can store the image information to be transmitted to the terminal device and the storage area of the first image storage unit corresponding to a rectangular area of a predetermined size included in the screen. Page<u style="single">Every time</u>, For converting the virtual address of the first image storage unit into a physical address<u style="single">Have an entry</u>An address calculation procedure that includes a table storage unit that stores a conversion table and calculates a virtual address for pixels included in the rectangular area.<u style="single">Said</u>Conversion table<u style="single">See</u>, The acquisition procedure for acquiring the physical address corresponding to the virtual address calculated for each pixel of the image information for which writing was requested, and the image information for which writing was requested for the acquired physical address. Writing procedure to write and<u style="single">The page corresponding to the entry referred to when acquiring the physical address in the acquisition procedure is displayed.</u>The updated part of the image information representing the part where the image information does not match before and after writing<u style="single">As a page to include</u>It is an update detection program that causes the computer to execute the detection procedure to be detected.</p>
<p> According to the present invention, when the frame buffer is managed by the virtual memory method, it is possible to reduce the processing load related to the detection of the updated portion of the image.</p>
With reference to the accompanying drawings, the image processing apparatus according to the present invention, the method for detecting image updates, and the best embodiment of the program will be described in detail below.
(First Embodiment) The image processing apparatus according to the first embodiment utilizes the fact that the address of the frame buffer is configured so that the rectangular area of the screen corresponds to one page, and a page exception occurs when the page is updated by the virtual memory method. The update page is detected, and the updated part of the image information is efficiently detected.
In the following, a virtual computer environment is created to provide various processes to the terminal device, and a virtual computer server that provides the terminal device with image information for displaying the result of the process executed in the computer environment. An example of realizing an image processing device will be described. The applicable device is not limited to such a virtual computer server, and can be applied to any device as long as it detects an updated part from the image information of the screen stored in the frame buffer.
FIG. 1 is a block diagram showing an outline of a network configuration including the image processing apparatus 100 according to the first embodiment. As shown in the figure, the image processing device 100 is connected to a terminal device 200 operated by a user via a network 300 such as the Internet or a LAN (Local Area Network).
The terminal device 200 has a function of displaying the image information transferred from the image processing device 100 on a display device such as a display in the terminal device 200. Further, it is premised that the terminal device 200 has a function of receiving image information of only the updated portion of the screen transferred from the image processing device 100 and displaying only the updated portion based on the received image information.
The image processing device 100 includes a host OS 110, an application 120, a guest OS 130, a virtual display unit 140, and a back-end virtual display unit 150, which are the bases for operating the image processing device 100.
The guest OS 130 is an OS that operates a virtual computer environment corresponding to the terminal device 200, and operates on the host OS 110. Application 120 is a program that provides various processes that run on guest OS 130.
The virtual display unit 140 is a virtual display device for outputting image information generated by the guest OS 130, and includes a frame buffer 142 for storing image information as described later. The back-end virtual display unit 150 executes common processing related to image display and operates on the host OS 110.
Next, detailed functions of each component of the image processing apparatus 100 will be described. FIG. 2 is a block diagram showing a configuration of the image processing apparatus 100 according to the first embodiment.
The guest OS 130 includes a graphic library 131 as a function related to image display. In addition to the functions related to image display, the guest OS 130 has all the functions necessary to operate a virtual computer environment.
The graphic library 131 includes a renderer 132 and an address calculation unit 133.
The renderer 132 performs various image processing according to a drawing command specified by the application 120 or the like, and outputs the resulting image information. For example, when the renderer 132 receives a drawing command for enlarging / reducing a certain area, the renderer 132 identifies the area to be enlarged / reduced from the coordinate information included in the drawing command, and enlarges / reduces the area. Outputs image information including coordinate information of the area. The coordinate information refers to information represented by a coordinate system such that, for example, in the case of a screen of 1024 × 768 pixels, the upper left of the screen is (0, 0) and the lower right of the screen is (1023, 767). As described above, in the first embodiment, the coordinate information of the pixels on the screen is represented by the XY coordinates with the right direction of the screen as the X coordinate direction and the lower direction of the screen as the Y coordinate direction.
The address calculation unit 133 inputs the image information output by the renderer 132, and performs a process of converting the coordinate information of the image information into the address of the frame buffer 142 (described later). In the first embodiment, the address calculation unit 133 divides the display screen displayed on the terminal device 200 into a rectangular area of 32 × 32 pixels, and each pixel in the divided rectangular area has one page (4096 bytes). The virtual address of the frame buffer 142 is calculated so that the virtual address of the frame buffer 142 corresponds to.
Specifically, the address calculation unit 133 converts the coordinate information (X, Y) into the address addr of the frame buffer 142 according to the following formula (2) instead of the following formula (1) which is the conventional calculation formula. To do. Note that OFFSET represents an offset value up to the start address at which the storage of image information is started. addr = OFFSET + (X << 2) + (Y << 12) (1) addr = OFFSET + ((X & ~ 0x1f) << 10) × 3 + ((X & 0x1f) << 2) + (Y << 7) (2)
Further, the method of optimizing the actual calculation formula differs depending on the display mode and the computing power of the address calculation unit 133.
The virtual display unit 140 includes a memory management unit 141 and a frame buffer 142. The memory management unit 141 manages access to the frame buffer 142 by a virtual memory method, and includes a table storage unit 141a, an acquisition unit 141b, a change unit 141c, a write unit 141d, and an exception generation unit 141e. I have.
The table storage unit 141a stores a conversion table for converting a virtual address into a physical address. Here, the details of the structure of the translation table and the flow of address translation using the translation table by the memory management unit 141 will be described with reference to FIG. FIG. 3 is an explanatory diagram showing a flow of conversion from a virtual address to a physical address by the memory management unit 141.
In the case of the virtual memory method, the software accesses the logically linear memory space on the virtual address space by the virtual address. In this embodiment, a 32-bit length virtual address is used. A 32-bit length virtual address is separated by 10 bits, 10 bits, and 12 bits, and each bit has the following meaning.
The upper 10 bits of the virtual address are used to identify the entry in PageDirectory (hereinafter referred to as pd), which is the upper table of the translation table represented by the two-step hierarchical structure. The identified entry contains a pointer to identify one of a plurality of PageTables that are subordinate tables. Figure 3 shows an example where the PageTable represented by pt2 is identified.
The middle 10 bits of the virtual address are used to identify the entry in the PageTable identified by this pointer. The identified entry contains a pointer to identify a page block (hereinafter simply referred to as a page) on the physical address space. A page is the smallest unit of physical memory that can be mapped to a virtual address, and continuous physical memory is allocated to the page.
The lower 12 bits of the virtual address represent the offset within the page identified by this pointer. In this way, a 32-bit length virtual address can be translated into a physical address.
The details of PageDirectory and PageTable entries are described below.
The PageDirectory entry contains a pointer to the PageTable, an Enable bit that indicates whether the entry is valid / invalid, and an Accessed bit that is set to 1 when the entry is referenced during the conversion process from a virtual address to a physical address. ..
PageTable entries include a pointer to a page in physical memory, an Enable bit that indicates whether the entry is valid / invalid, a Writable bit that indicates whether the page can be written / disabled, and data written to the page. Includes the Dirty bit, which is set to 1 if, and the Accessed bit, which is set to 1 when the entry is referenced in the process of converting a virtual address to a physical address.
Note that the bit name may differ depending on the type of processor, or it may differ depending on whether it is positive logic or negative logic, but usually there is an entry containing the above information.
Returning to FIG. 2, the acquisition unit 141b acquires the physical address corresponding to the virtual address of the image information requested to be written to the frame buffer 142 by using the conversion table configured in this way.
The change unit 141c changes the information of each entry in the conversion table in response to an access request to the frame buffer 142. For example, the change unit 141c changes the Accessed bit and the Dirty bit of the PageTable for the page containing the virtual address of the image information requested to be written to 1. At the same time, the change part 141c changes the Accessed bit of PageDirectory to 1.
The writing unit 141d writes the image information requested to be written to the physical address acquired by the acquisition unit 141b.
The exception generation unit 141e generates a page exception for the page for which writing is requested. Specifically, the exception generation unit 141e generates a page exception when a write is requested for a page for which the Writable bit is not possible. All Writable bits are set to disabled in advance. This makes it possible to raise a page exception whenever a write is requested.
In addition, the exception generation unit 141e also generates other exceptions handled by the general virtual memory method. For example, when access to the virtual address space is requested and the translation table is searched by acquisition unit 141b, exception generation unit 141e is when the searched PageDirectory entry or the Enable bit of the PageTable entry is invalid. , Raises a page exception for the processor (not shown).
When a page exception occurs, the processor can detect which virtual address was accessed. In addition, when a page exception occurs, the processor performs necessary processing such as page mapping. In addition, the processor updates the conversion table as necessary to recover from the exception.
The frame buffer 142 is a storage unit that stores image information for one screen displayed on the terminal device 200. Image information is stored in the frame buffer 142 according to the physical address acquired by the acquisition unit 141b.
Here, the details of the address arrangement of the frame buffer 142 will be described. First, the address arrangement of the frame buffer generally used in the conventional computer will be described with reference to FIGS. 4 to 6. FIG. 4 is an explanatory diagram showing an example of the address arrangement of the frame buffer used in the conventional computer.
The figure shows an example of address placement when the amount of data per pixel is 4 bytes on a screen of 1024 x 768 pixels. As shown in the figure, the conventional frame buffer has an address arrangement that is continuous in the horizontal direction with the upper left corner of the screen as a base point.
FIG. 5 is a schematic diagram showing the order of the addresses of the conventional frame buffer. As shown in FIG. 5, in the conventional frame buffer, the addresses are arranged linearly in the raster scan direction in consideration of outputting to a display unit such as a monitor.
FIG. 6 is a schematic diagram showing the correspondence between the screen area and the page in the conventional frame buffer. If one page is 4096 bytes, it matches the amount of data of one raster, which is 4096 bytes (= 1024 x 4 bytes), so one raster corresponds to one page. Therefore, 768 pages are arranged vertically to form one screen.
Next, the address arrangement of the frame buffer 142 used in the image processing apparatus 100 in the first embodiment will be described with reference to FIG. 7. FIG. 7 is an explanatory diagram showing an example of the address arrangement of the frame buffer 142 used in the image processing apparatus 100 according to the first embodiment.
As shown at the top of the figure, in the first embodiment, the addresses are arranged linearly within a rectangular area of 32 × 32 pixels. Then, as shown in the lower part of the figure, one screen is constructed by laying out 32 × 24 rectangular areas.
The rectangular area is not limited to 32 × 32 pixels, and any rectangular area may be used as long as it can be associated with the address of each page. Further, the address arrangement in the rectangular area is not limited to the method of continuously arranging the addresses in the X coordinate direction. For example, addresses are assigned in the Y-coordinate direction, and when the number of pixels reaches a predetermined number, the addresses are arranged in an order in which the pixels in the adjacent column are further given consecutive addresses in the Y-coordinate direction. You may.
Returning to FIG. 2, the back-end virtual display unit 150 includes a backup storage unit 151, a storage unit 152, a detection unit 153, a compression unit 154, and a transfer unit 155.
The backup storage unit 151 is a storage unit that can store backup data of image information for each updated page.
When a page exception occurs due to a request for writing to a page for which the Writable bit is not possible by the exception generation unit 141e, the storage unit 152 stores the image information of the page in which the page exception occurred in the backup storage unit 151. Is what you do.
The detection unit 153 detects an updated portion of image information in page units managed by the memory management unit 141. Specifically, the detection unit 153 first detects the page on which the image information is backed up as the page on which the image information is updated. Then, the detection unit 153 compares the image information of the backup storage unit 151 with the latest image information after being written to the frame buffer 142 by the writing unit 141d for each updated page, and includes differences. Detect the rectangle as the update part.
The compression unit 154 performs image compression processing by a JPEG method or the like on the updated portion detected by the detection unit 153. The compression method by the compression unit 154 is not limited to the JPEG method.
The transfer unit 155 transfers the image information (compressed image information) compressed by the compression unit 154 to the terminal device 200.
The frame buffer 142 and the backup storage unit 151 can be configured by any commonly used storage medium such as a RAM (Random Access Memory), an HDD (Hard Disk Drive), an optical disk, and a memory card. Further, the backup storage unit 151 may be provided in the virtual display unit 140 other than the back-end virtual display unit 150, for example.
Next, the image processing by the image processing apparatus 100 according to the first embodiment configured in this way will be described. The image processing is an image writing process for writing image information to the frame buffer 142 and the backup storage unit 151, and an update part detection process for detecting an updated part of the image information by comparing the backed up image information with the latest image information. Divided.
First, the outline of the image writing process and the updated partial detection process will be described with reference to FIG. FIG. 8 is an explanatory diagram showing an outline of the image writing process and the updated partial detection process according to the first embodiment.
In the image writing process, first, the Writable bits of all PageTables related to the memory area of the frame buffer 142 are set to disabled in advance. As a result, when the graphic library 131 subsequently writes to any area of the framebuffer 142, a page exception regarding the corresponding page occurs.
From this state, the application 120 sends a drawing command to the frame buffer 142 via the graphic library 131 for screen display (1). The memory management unit 141 refers to the conversion table and converts the virtual address specified by the drawing instruction from the graphic library 131 into a physical address (2).
Since the Writable bit of PageTable was set to disabled in advance, exception occurrence part 141e raises a page exception when the conversion table is referenced (3). Due to the page exception that occurred, the storage unit 152 detects the corresponding page in the frame buffer 142 that the graphic library 131 requested to update, and leaves a backup of the contents of the page (4).
On the other hand, the change part 141c changes the Writable bit of the PageTable of the page to be possible (5). After this, image writing by the graphic library 131 is continued. After that, the image is written to the page without causing a page exception. For writing an image to another page, a page exception occurs (3), the contents of the page are backed up (4), and the Writable bit of PageTable is changed (5) in sequence.
In this way, a page that fits the rectangular area on the screen is configured, and for only the page for which image writing is requested, a page exception is generated only at the first writing to the page, and image writing occurs. It is possible to keep a backup of the previous image information. That is, it is possible to execute efficient processing without waste by targeting only the minimum necessary pages.
Next, the detection unit 153 executes the update portion detection process of the image information, for example, by a periodic interrupt from the timer. The update partial detection process is a process of comparing the backed up image information with the image information of the current page for the page for which the backup is left, and cutting out a rectangle in which the difference exists in the page. The timer is set so that, for example, when the screen of the terminal device 200 is updated 60 times per second, an interrupt is generated at 1/60 second intervals.
First, the detection unit 153 starts the update part detection process by a trigger from the timer (6). Next, the detection unit 153 detects the page in which the backup is saved in the above (4) as an update page (7). Next, the detection unit 153 compares the backed up image information with the latest image information written in the frame buffer 142, and detects the rectangle of the updated portion (8). As shown in the figure, the detection unit 153 can detect the smallest rectangular area including the updated area as the updated part from a plurality of adjacent pages.
The image information of the detected updated portion is compressed by the compression unit 154. Further, the compressed image information after compression is transmitted to the terminal device 200 by the transfer unit 155 (9).
Although omitted in the figure, after the compressed image information of the updated portion is transferred to the terminal device 200, all the image information backed up in the backup storage unit 151 is discarded. Also, the Writable bits of all PageTables related to the memory area of framebuffer 142 are changed to disabled. Also, in the figure, page exception (3), backup (4), and change of Writable bit (5) represent four pages at once, but in reality, they are written from the graphic library 131. It is executed 4 times each.
By configuring the page that fits the rectangular area on the screen in this way, it is possible to detect the updated part on a page-by-page basis by using the page exception that can occur for each page.
Next, the detailed flow of the image writing process and the updated portion detection process will be described with reference to FIGS. 9 and 10. FIG. 9 is a flowchart showing the overall flow of the image writing process according to the first embodiment.
First, the application 120 outputs a drawing command requesting drawing of the display screen when the screen information related to the executed process is updated (step S901). Next, the renderer 132 performs image processing in response to the drawing command and generates image information as a result (step S902). The image information includes information on the start coordinate and the end coordinate corresponding to the area to be updated.
Next, the address calculation unit 133 calculates the virtual address by address-converting the coordinate information of the pixel to be updated according to the frame buffer 142 of the addressing configured so that the rectangular area of the screen corresponds to the page ( Step S903). Specifically, the address calculation unit 133 calculates the virtual address by the above equation (2).
Next, the graphic library 131 specifies the calculated virtual address and requests the writing of image information to the frame buffer 142 (step S904).
Next, the acquisition unit 141b refers to the conversion table stored in the table storage unit 141a and acquires the physical address corresponding to the specified virtual address (step S905). When the conversion table is referenced, the exception raiser 141e raises a page exception for the page for which writing was requested (step S906).
The storage unit 152 that has detected the page exception stores the image information corresponding to the page in which the page exception has occurred in the backup storage unit 151 (step S907).
On the other hand, the writing unit 141d writes the requested image information to the physical address acquired by the acquisition unit 141b in step S905 (step S908), and ends the image writing process.
Next, the detailed flow of the update partial detection process will be described. FIG. 10 is a flowchart showing the entire flow of the update partial detection process according to the first embodiment. The update partial detection process is started by an interrupt from the timer or the like as described above.
First, the detection unit 153 detects the page (update page) in which the backup is saved in the backup storage unit 151 (step S1001). Next, the detection unit 153 compares the latest image information written in step S908 of FIG. 9 with the image information saved in the backup storage unit 151 with respect to the detected update page, and updates the difference. Detect as (step S1002).
Next, the compression unit 154 compresses the update portion detected by the detection unit 153 (step S1003). Then, the transfer unit 155 packetizes the compressed image information and transfers it to the terminal device 200 (step S1004) to end the update partial detection process.
In this way, in the image processing apparatus according to the first embodiment, the address of the frame buffer is configured so that the rectangular area of the screen corresponds to one page, and the address is updated by the page exception that occurs when the page is updated by the virtual memory method. The page can be detected. As a result, it is possible to reduce the processing load related to the detection of the updated portion of the image when the updated portion of the image information is efficiently detected and the frame buffer is managed by the virtual memory method.
(Second embodiment) In the first embodiment, the updated page is detected by using the page exception. On the other hand, the image processing apparatus according to the second embodiment detects the updated page by referring to the bit value of the conversion table that is changed when the page is updated.
FIG. 11 is a block diagram showing a configuration of the image processing apparatus 1100 according to the second embodiment. As shown in FIG. 11, the image processing apparatus 1100 includes a host OS 110, an application 120, a guest OS 130, a virtual display unit 1140, and a back-end virtual display unit 1150.
In the second embodiment, the functions of the virtual display unit 1140 and the back-end virtual display unit 1150 are different from those in the first embodiment. Other configurations and functions are the same as those in FIG. 2, which is a block diagram showing the configuration of the image processing apparatus 100 according to the first embodiment. Therefore, the same reference numerals are given, and the description thereof will be omitted here.
The virtual display unit 1140 includes a memory management unit 1141 and a frame buffer 142. Since the configuration of the frame buffer 142 is the same as that of the first embodiment, the same reference numerals are given, and the description thereof will be omitted here.
The memory management unit 1141 includes a table storage unit 141a, an acquisition unit 141b, a change unit 141c, a writing unit 141d, and an exception generation unit 1141e. The configurations and functions other than the exception generating unit 1141e are the same as those in FIG. 2, which is a block diagram showing the configuration of the image processing apparatus 100 according to the first embodiment. Omit.
The exception generation unit 1141e is different from the exception generation unit 141e of the first embodiment in that the function of generating a page exception for the page for which writing is requested has been deleted.
The back-end virtual display unit 1150 includes a backup storage unit 1151, a storage unit 1152, a detection unit 1153, a compression unit 154, and a transfer unit 155.
In the second embodiment, the storage method in the backup storage unit 1151 and the functions of the storage unit 1152 and the detection unit 1153 are different from those in the first embodiment. Other configurations and functions are the same as those in FIG. 2, which is a block diagram showing the configuration of the image processing apparatus 100 according to the first embodiment. Therefore, the same reference numerals are given, and the description thereof will be omitted here.
In the first embodiment, the backup data of the image information is stored for each page, but the backup storage unit 1151 of the second embodiment stores the backup of the image information of the entire frame buffer 142. is there.
The storage unit 1152 stores the image information of the entire frame buffer 142 in the backup storage unit 1151 in the initial state of the image processing device 1100 or after the transfer of the updated portion is completed.
The detection unit 1153 differs from the detection unit 153 of the first embodiment in that it detects an updated page by referring to the Accessed bit of PageDirectory, the Accessed bit of PageTable, and the Dirty bit of PageTable.
Next, the image processing by the image processing apparatus 1100 according to the second embodiment configured in this way will be described. First, the outline of the image writing process and the updated partial detection process will be described with reference to FIG. FIG. 12 is an explanatory diagram showing an outline of the image writing process and the updated partial detection process according to the second embodiment.
In the image writing process, first, the entire backup of the frame buffer 142 is saved in the backup storage unit 1151 in advance. If it is possible to determine whether or not the initial state is based on the pixel value of the frame buffer 142, the backup holding at this point may be omitted. For example, if all the pixels are black in the initial state, it can be determined that the backup does not exist if all the pixels are black, so that the backup holding can be omitted.
From this state, the application 120 sends a drawing command to the frame buffer 142 via the graphic library 131 for screen display (1). The memory management unit 1141 refers to the conversion table and converts the virtual address specified by the drawing instruction from the graphic library 131 into a physical address.
With reference to the conversion table by the memory management unit 1141, the change unit 141c changes the Accessed bit of the PageDirectory entry corresponding to the upper 10 bits of the virtual address to 1. Further, the change unit 141c changes the Accessed bit and the Dirty bit of the entry corresponding to the middle 10 bits of the virtual address with respect to the PageTable indicated by the entry to 1 (2). After this, image writing by the graphic library 131 is continued.
Next, the detection unit 1153 executes the update portion detection process of the image information, for example, by a periodic interrupt from the timer. First, the detection unit 1153 starts the update part detection process by a trigger from the timer (3). The detector 1153 first refers to the Accessed bit and the Dirty bit in the conversion table, and searches for a page in which each bit is set to 1 (4). Detects updated pages by doing (5). That is, the detection unit 1153 first searches for the entry related to the frame buffer 142 of the Page Directory, and identifies the Page Table whose Accessed bit is 1. Then, the detection unit 1153 detects the page where the image has been updated by searching the entry related to the frame buffer 142 of the PageTable and finding the page whose Dirty bit is 1.
Next, the detection unit 1153 compares the backed up image information with the latest image information written in the frame buffer 142 with respect to the detected update page, and detects the rectangle of the update portion (6).
The image information of the detected updated portion is compressed by the compression unit 154. Further, the compressed image information after compression is transmitted to the terminal device 200 by the transfer unit 155 (7).
Although omitted in the figure, after the compressed image information of the updated portion is transferred to the terminal device 200, the backup of the image information of the page is updated with the current contents. Also, the Accessed and Dirty bits of the conversion table are changed to 1.
Next, the detailed flow of the image writing process and the updated portion detection process will be described with reference to FIGS. 13 and 14. FIG. 13 is a flowchart showing the overall flow of the image writing process according to the second embodiment.
The drawing command processing, the address calculation processing, and the physical address acquisition processing in steps S1301 to S1305 are the same processes as in steps S901 to S905 in the image processing apparatus 100 according to the first embodiment. Is omitted.
After the physical address is acquired by the acquisition unit 141b, the change unit 141c changes the Accessed bit and the Dirty bit of the conversion table to 1 (step S1306). Although the description of the Accessed bit and Dirty bit change process is omitted in FIG. 9 showing the image writing process of the first embodiment, the same change process as in step S1306 is actually executed.
Next, the writing unit 141d writes the requested image information to the physical address acquired by the acquisition unit 141b in step S1305 (step S1307), and ends the image writing process.
As described above, the image writing process of the second embodiment is different from the first embodiment in that the process of generating a page exception and saving the backup of the page on which the page exception has occurred is deleted. There is.
Next, the detailed flow of the update partial detection process will be described. FIG. 14 is a flowchart showing the entire flow of the update partial detection process according to the second embodiment.
First, the detection unit 1153 detects a page (update page) in which the Accessed bit and the Dirty bit are 1 (step S1401). Next, the detection unit 1153 acquires the image information corresponding to the detected update page from the backup storage unit 1151, compares the acquired image information with the latest image information, and detects the difference as the update part ( Step S1402).
Since the image compression process and the image transfer process from step S1403 to step S1404 are the same processes as steps S1003 to S1004 in the image processing apparatus 100 according to the first embodiment, the description thereof will be omitted.
After transferring the compressed image information, the storage unit 1152 saves a backup of the latest image information written in the frame buffer 142 in the backup storage unit 1151 (step S1405), and ends the update portion detection process.
As described above, in the image processing apparatus according to the second embodiment, the address of the frame buffer is configured so that the rectangular area of the screen corresponds to one page, and the bit value of the conversion table changed at the time of page update is referred to. By doing so, the update page can be detected. This makes it possible to efficiently detect the updated portion of the image information and reduce the processing load related to the detection of the updated portion of the image when the frame buffer is managed by the virtual memory method.
(Modification example) The frame buffer 142 having a structure as shown in FIG. 7 was assumed to be 32 × 24 pages, that is, a continuous area of 3 Mbytes (= 32 × 24 × 4096 bytes) on the virtual address.
On the other hand, in the case of virtual addresses separated by 10 bits, 10 bits, and 12 bits as shown in FIG. 3, PageDirectory has 1024 entries (corresponding to 10 bits). Similarly, the PageTable has 1024 entries. Since one page is composed of 4096 bytes, each entry in PageDirectory manages an area of 4 Mbytes (= 4096 × 1024).
Therefore, the frame buffer 142, which is a continuous area of 3 Mbytes, is often managed by one entry of PageDirectory unless it is arranged in the virtual address space so as to straddle the boundary of the entries of PageDirectory. Also, out of the 1024 entries in the PageTable indicated by the PageDirectory of the one entry, 768 entries show the correspondence to all pages.
FIG. 15 is a schematic diagram showing the correspondence between such a conversion table and the frame buffer 142. As shown in FIG. 15, in the method of arranging the frame buffer 142 in the continuous area of the virtual address, the Accessed bit of the entry corresponding to the frame buffer 142 of PageDirectory indicates whether or not a memory access exists in any of the frame buffer 142. It becomes a flag indicating. Further, this flag is updated not only by the memory access by writing but also by the memory access by reading.
Therefore, every time some kind of memory access occurs, 768 entries of the corresponding PageTable are searched every time, and there is a problem that the processing load for detecting the updated part increases.
In this way, the structure of the conversion table (PageDirectory and PageTable) related to the framebuffer 142 changes depending on which area of the virtual address space the framebuffer 142 is distributed and arranged, which affects the time and effort of the update partial search. ..
Therefore, in this modification, the screen is divided into a predetermined number of areas, and each divided area is arranged on the virtual address space of the frame buffer 142 so as to correspond to one entry of PageDirectory. In addition, in order to arrange in this way, the addresses of the entire frame buffer 142 may be discontinuous in the virtual address space.
FIG. 16 is a schematic diagram showing the correspondence between the conversion table and the frame buffer 142 in this modified example. FIG. 16 shows an example in which the screen is divided into eight areas, each of which is arranged so as to correspond to one entry in PageDirectory.
By arranging in this way, for example, if the screen is updated only in the leftmost area of the screen (the area corresponding to pt1 of the PageTable), only the Accessed bit of the corresponding one of the eight entries in the PageDirectory will be displayed. Changed to 1. You can then find the updated page by searching only the 96 entries in the PageTable that correspond to this entry.
The screen division method is not limited to the above, but the screen update pattern of the assumed application such as the method of dividing the screen at the top and bottom or left and right, the method of dividing into rectangles such as 4x4, the method of dividing at the center and others Any method considered can be applied. Also, regarding the unit of division, the method of rough division and the method of fine division can be changed according to the design.
In addition, until now, the target was a conversion table composed of two stages, PageDirectory and PageTable, but it can also be applied to a conversion table composed of two or more stages. For example, in the case of three stages, the screen can be divided into left and right to correspond to the upper table, and each of the divided areas can be further divided into 2 × 2 areas to correspond to the middle table. .. In this way, it is possible to adopt an efficient method of dividing the frame buffer 142 according to the structure of the conversion table to be used.
(Third embodiment) In the first embodiment, a Writable bit indicating whether or not the page can be written is used to generate a page exception of the page for which writing is requested. However, depending on the conversion table configuration method, the exception generation method using the Writable bit may not be adopted.
For example, a processor that implements a virtualization support function called SVM (Secure Virtual Machine) from Advanced Micro Devices, which supports virtualization at the hardware level, uses two conversion tables, GuestPageTable and NestedPageTable. And, in this processor, access to NestedPageTable is always treated as a write. For this reason, the method of using the Writable bit, which is set to be disabled in advance, has a problem that it is determined that the access is not possible even if the access is read, and the access cannot be performed.
Therefore, in the third embodiment, instead of using the Writable bit, a different virtual address is calculated at the time of writing and at the time of reading, and the Enable bit is used to display the page for writing for which access is requested. It raises a page exception.
In the third embodiment, an image processing device that uses two conversion tables as described above will be described as an example. That is, in the third embodiment, the theoretical conversion of the memory address by the memory management unit is not one step from the virtual address to the physical address as in the first embodiment, but from the guest virtual address to the guest physical address. It is assumed that the method is executed in two steps of converting to the host physical address via.
FIG. 17 is a block diagram showing a configuration of the image processing apparatus 1700 according to the third embodiment. As shown in FIG. 17, the image processing apparatus 1700 includes a host OS 110, an application 120, a guest OS 1730, a virtual display unit 1740, and a back-end virtual display unit 1750.
In the third embodiment, the functions of the guest OS 1730, the virtual display unit 1740, and the back-end virtual display unit 1750 are different from those in the first embodiment. Other configurations and functions are the same as those in FIG. 2, which is a block diagram showing the configuration of the image processing apparatus 100 according to the first embodiment. Therefore, the same reference numerals are given, and the description thereof will be omitted here.
The guest OS 1730 is different from the guest OS 130 of the first embodiment in the function of the graphic library 1731 which is a function related to image display.
The graphic library 1731 includes a renderer 132 and an address calculation unit 1733. Since the function of the renderer 132 is the same as that of the first embodiment, the same reference numerals are given and the description thereof will be omitted here.
The address calculation unit 133 of the first embodiment calculates the same address when writing to the frame buffer 142 and when reading from the frame buffer 142. On the other hand, the address calculation unit 1733 of the third embodiment calculates different addresses when writing to the frame buffer 142 and when reading from the frame buffer 142.
For example, the frame buffer 142 having the structure shown in FIG. 7 is 32 × 24 pages, that is, an area of 3 Mbytes (= 32 × 24 × 4096 bytes) on the virtual address. Therefore, the address calculation unit 1733 calculates addresses that differ by 3 Mbytes, for example, when writing to the frame buffer 142 and when reading from the frame buffer 142.
Further, the address calculation unit 1733 divides the display screen displayed on the terminal device 200 into a rectangular area of 32 × 32 pixels, and each of the divided rectangular areas is divided into the address calculation unit 133 of the first embodiment. The virtual address of the frame buffer 142 is calculated so that the virtual address of one page (4096 bytes) corresponds to the pixel.
Specifically, for example, in the case of reading, the address calculation unit 1733 inputs the coordinate information (X, Y) to the address of the frame buffer 142 according to the above-mentioned equation (2) which is the calculation equation of the first embodiment. Convert to addr. On the other hand, in the case of writing, the address calculation unit 1733 converts the coordinate information (X, Y) into the address addr of the frame buffer 142 according to the following equation (3). addr = OFFSET + 3M bytes + ((X & ~ 0x1f) << 10) × 3 + ((X & 0x1f) << 2) + (Y << 7) (3)
The value to be added does not have to be 3 Mbytes as long as different addresses can be calculated for writing and reading. In addition, the method of optimizing the actual calculation formula differs depending on the display mode and the computing power of the address calculation unit 1733.
The virtual display unit 1740 includes a memory management unit 1741 and a frame buffer 142. Since the configuration of the frame buffer 142 is the same as that of the first embodiment, the same reference numerals are given, and the description thereof will be omitted here.
The memory management unit 1741 includes a table storage unit 1741a, an acquisition unit 141b, a change unit 141c, a writing unit 141d, and an exception generation unit 1741e. The configurations and functions other than the table storage unit 1741a and the exception generation unit 1741e are the same as those in FIG. 2, which is a block diagram showing the configuration of the image processing apparatus 100 according to the first embodiment. The description here will be omitted.
The table storage unit 141a of the first embodiment stores only one conversion table for converting a virtual address into a physical address. On the other hand, the table storage unit 1741a of the third embodiment stores two conversion tables for converting a virtual address into a physical address.
Here, the flow of two-step conversion using two conversion tables from the guest virtual address to the guest physical address and from the guest physical address to the host physical address will be described with reference to FIG. FIG. 18 is a diagram showing an outline of the flow of two-step translation of an address using two translation tables.
The figure shows an example in which the guest virtual address 1811 in the guest virtual address space 1801 is converted to the guest physical address 1812 in the guest physical address space 1802, and further converted to the host physical address 1813 in the host physical address space 1803. .. In the first-stage conversion, the first conversion table 1821 specified in the gCR3 register is referenced. The first conversion table 1821 is a conversion table for each guest OS 1730. In reality, it exists on the host physical address space 1803, and the first translation table 1821 and the first translation table 1823 are the same. In the second stage conversion, the second conversion table 1822 specified by the nCR3 register is referred to.
There may be a plurality of first conversion tables 1823 depending on the number of guest OS 1730s. The GuestPageTable and NestedPageTable described above correspond to the first conversion table and the second conversion table, respectively.
In this way, the table storage unit 1741a stores two conversion tables, the first conversion table and the second conversion table. The details of the entries of PageDirectory and PageTable that compose each conversion table and the method of specifying the entries to be used for conversion are the same as those of the first embodiment, and thus the description thereof will be omitted here.
Generally, when the conversion table is accessed, it is possible to distinguish between a write request and a read request based on the information contained in the access request and the like. However, as described above, there are cases where access to the second conversion table is always treated as a write.
In the third embodiment, there are two PageTable entries for the same page in framebuffer 142, one for writing and one for reading, so that writing and reading can be distinguished even in such a case. The address is calculated as follows.
FIG. 19 is a diagram showing an example of a PageTable for the frame buffer 142 in the second conversion table. As shown in FIG. 19, two virtual addresses are associated with each area of the frame buffer 142, which is physically one.
For example, Page Directory entries 1901 and 1902 in the figure correspond to virtual addresses calculated for reading and writing, respectively. And entries 1901 and 1902 contain pointers to PageTable 1911 for reading and PageTable 1912 for writing, respectively. In the following, the PageTable for writing to the frame buffer 142 in the second conversion table is simply referred to as the PageTable for writing.
In the figure, for example, the first entries in PageTable 1911 and 1912 are shown to correspond to the same host physical address 1921 on the host physical address space 1803, respectively.
As described above, since the address calculation unit 1733 calculates different addresses when writing to the frame buffer 142 and when reading from the frame buffer 142, even when accessing the same host physical address, writing and reading can be performed. You can access different PageTables depending on your needs. As a result, even if the access to the conversion table is always treated as a write, whether the actual access is a write or a read is determined by the accessing PageTable, and the page update is detected appropriately. It becomes possible to do.
Returning to FIG. 17, the exception occurrence unit 1741e will be described. The exception-raising unit 141e of the first embodiment generates a page exception when a write is requested for a page for which the Writable bit is not possible. On the other hand, the exception generation unit 1741e of the third embodiment generates a page exception when a write is requested for an entry of the write PageTable in which the Enable bit is invalid. All Enable bits are set to invalid in advance. This makes it possible to raise a page exception whenever a write is requested.
The back-end virtual display unit 1750 includes a backup storage unit 151, a storage unit 1752, a detection unit 153, a compression unit 154, and a transfer unit 155. Since the configurations and functions other than the storage unit 1752 are the same as those in FIG. 2, which is a block diagram showing the configuration of the image processing apparatus 100 according to the first embodiment, the same reference numerals are given and the description thereof is omitted here. To do.
The storage unit 152 of the first embodiment stores the image information of the page in which the page exception occurred when a page exception occurs due to a write request to the page in which the Writable bit is not possible. .. On the other hand, the storage unit 1752 of the third embodiment generates the page exception when a page exception is generated due to a write request to the page on which the Enable bit is invalid by the exception generation unit 1741e. The image information of the created page is saved in the backup storage unit 151.
Next, the image processing by the image processing apparatus 1700 according to the third embodiment configured in this way will be described. First, the outline of the image writing process and the updated partial detection process will be described with reference to FIG. FIG. 20 is an explanatory diagram showing an outline of the image writing process and the updated partial detection process according to the third embodiment.
In the image writing process, first, the Enable bits of all the writing PageTables for the frame buffer 142 in the second conversion table are set to invalid in advance. As a result, if the graphic library 1731 subsequently writes to any area of the framebuffer 142, a page exception for that page will occur.
From this state, the application 120 sends a drawing instruction to the frame buffer 142 via the graphic library 1731 for screen display (1). The memory management unit 1741 refers to the two conversion tables and converts the virtual address specified by the drawing instruction from the graphic library 1731 into a physical address (2).
Since the Enable bit of the page table for writing was set to invalid in advance, the exception occurrence section 1741e raises a page exception when the second conversion table is referenced (3). Due to the page exception that occurred, the save unit 1752 detects the corresponding page in the frame buffer 142 that the graphic library 1731 requested to update, and leaves a backup of the contents of the page (4).
On the other hand, the change unit 141c effectively changes the Enable bit of the PageTable of the page (5). After this, image writing by the graphic library 1731 is continued. After that, the image is written to the page without causing a page exception. For writing an image to another page, a page exception occurs (3), the contents of the page are backed up (4), and the Enable bit of PageTable is changed (5) in sequence.
Next, the detection unit 153 starts the update part detection process by the trigger from the timer (6). Next, the detection unit 153 detects the page in which the backup is saved in the above (4) as an update page (7). Next, the detection unit 153 compares the backed up image information with the latest image information written in the frame buffer 142, and detects the rectangle of the updated portion (8). As shown in the figure, the detection unit 153 can detect the smallest rectangular area including the updated area as the updated part from a plurality of adjacent pages.
The image information of the detected updated portion is compressed by the compression unit 154. Further, the compressed image information after compression is transmitted to the terminal device 200 by the transfer unit 155 (9).
Although omitted in the figure, after the compressed image information of the updated portion is transferred to the terminal device 200, all the image information backed up in the backup storage unit 151 is discarded. Also, the Enable bits of all write PageTables for the framebuffer 142 in the second conversion table are changed to invalid. Also, in the figure, page exception (3), backup (4), and change of Enable bit (5) represent four pages at once, but in reality, they are written from the graphic library 1731. It is executed 4 times each.
Next, the flow of the image writing process will be described with reference to FIG. FIG. 21 is a flowchart showing the overall flow of the image writing process according to the third embodiment. Compared with FIG. 9, which shows the overall flow of the image writing process of the first embodiment, the processing content in step S2103 is only different from that of step S903 in FIG. Omit.
In step S2103, the address calculation unit 1733 calculates the virtual address by the above equation (2) or (3) depending on whether the access request is read or write. The address calculation unit 1733 can determine whether the access request is read or write by a drawing command.
Since the flow of the update partial detection process is the same as that shown in FIG. 10 showing the update partial detection process of the first embodiment, the description thereof will be omitted here.
As described above, in the image processing apparatus according to the third embodiment, different virtual addresses are calculated at the time of writing and at the time of reading, and the Enable bit is used to page the page for writing for which access is requested. You can raise an exception and detect the updated page with this page exception. As a result, even when the exception occurrence by the Writable bit cannot be used, the processing load related to the detection of the updated part of the image when the updated part of the image information is efficiently detected and the frame buffer is managed by the virtual memory method. Can be reduced.
Next, the hardware configuration of the image processing apparatus according to the first to third embodiments will be described with reference to FIG. 22. FIG. 22 is an explanatory diagram showing a hardware configuration of the image processing apparatus according to the first to third embodiments.
The image processing device according to the first to third embodiments communicates with a control device such as a CPU (Central Processing Unit) 51 and a storage device such as a ROM (Read Only Memory) 52 or RAM 53 by connecting to a network. Connect each part to a communication I / F 54, an external storage device such as an HDD (Hard Disk Drive) or CD (Compact Disc) drive device, a display device such as a display device, and an input device such as a keyboard or mouse. It is equipped with a bus 61 and has a hardware configuration using a normal computer.
The image processing program executed by the image processing apparatus according to the first to third embodiments is a file in an installable format or an executable format, and is a CD-ROM (Compact Disk Read Only Memory) or a flexible disk (FD). ), CD-R (Compact Disk Recordable), DVD (Digital Versatile Disk), etc., which are recorded on a computer-readable recording medium and provided.
Further, the image processing program executed by the image processing apparatus according to the first to third embodiments is stored on a computer connected to a network such as the Internet and provided by downloading via the network. It may be configured. Further, the image processing program executed by the image processing apparatus according to the first to third embodiments may be provided or distributed via a network such as the Internet.
Further, the image processing programs of the first to third embodiments may be configured to be provided by being incorporated in ROM or the like in advance.
The image processing program executed by the image processing apparatus according to the first to third embodiments has a module configuration including the above-mentioned parts (address calculation unit, acquisition unit, writing unit, detection unit, compression unit). As actual hardware, when the CPU 51 (processor) reads an image processing program from the storage medium and executes it, each of the above parts is loaded on the main storage device, and each of the above parts is generated on the main storage device. It has become like.
As described above, the device for processing the image, the method for detecting the update of the image, and the program according to the present invention are suitable for the virtual computer server that generates the image information of the screen to be displayed on the terminal device and transfers it to the terminal device. There is.
<figref num="1">It is a block diagram which shows the outline of the network configuration including the image processing apparatus which concerns on 1st Embodiment.</figref><figref num="2">It is a block diagram which shows the structure of the image processing apparatus which concerns on 1st Embodiment.</figref><figref num="3">It is explanatory drawing which shows the flow of conversion from a virtual address to a physical address.</figref><figref num="4">It is explanatory drawing which shows an example of the address arrangement of the frame buffer used in the conventional computer.</figref><figref num="5">It is a schematic diagram which showed the order of the address of the conventional frame buffer.</figref><figref num="6">It is a schematic diagram which showed the correspondence between a screen area and a page in a conventional frame buffer.</figref><figref num="7">It is explanatory drawing which shows an example of the address arrangement of the frame buffer used in the image processing apparatus in 1st Embodiment.</figref><figref num="8">It is explanatory drawing which shows the outline of the image writing process and the update part detection process in 1st Embodiment.</figref><figref num="9">It is a flowchart which shows the whole flow of the image writing process in 1st Embodiment.</figref><figref num="10">It is a flowchart which shows the whole flow of the update part detection process in 1st Embodiment.</figref><figref num="11">It is a block diagram which shows the structure of the image processing apparatus which concerns on 2nd Embodiment.</figref><figref num="12">It is explanatory drawing which shows the outline of the image writing process and the update part detection process in 2nd Embodiment.</figref><figref num="13">It is a flowchart which shows the whole flow of the image writing process in 2nd Embodiment.</figref><figref num="14">It is a flowchart which shows the whole flow of the update part detection process in 2nd Embodiment.</figref><figref num="15">It is a schematic diagram which shows the correspondence between a conversion table and a frame buffer.</figref><figref num="16">It is a schematic diagram which shows the correspondence between a conversion table and a frame buffer in a modification.</figref><figref num="17">It is a block diagram which shows the structure of the image processing apparatus which concerns on 3rd Embodiment.</figref><figref num="18">It is a figure which shows the outline of the flow of the two-step conversion of an address.</figref><figref num="19">It is a figure which shows an example of PageTable in 3rd Embodiment.</figref><figref num="20">It is explanatory drawing which shows the outline of the image writing process and the update part detection process in 3rd Embodiment.</figref><figref num="21">It is a flowchart which shows the whole flow of the image writing process in 3rd Embodiment.</figref><figref num="22">It is explanatory drawing which shows the hardware configuration of the image processing apparatus which concerns on 1st to 3rd Embodiment.</figref>
Code description
51 CPU 52 ROM 53 RAM 54 Communication I / F 61 bus 100 image processing equipment 110 Host OS 120 applications 130 guest OS 131 graphics library 132 Renderer 133 Address calculation unit 140 Virtual display 141 Memory Management Department 141a Table storage 141b Acquisition Department 141c Change 141d Writing section 141e Exception occurrence part 142 framebuffer 150 Backend virtual display 151 Backup storage 152 Preservation section 153 Detector 154 Compressor 155 Transfer section 200 terminal device 300 networks 1100 image processing equipment 1140 Virtual display 1141 Memory Management Department 1141e Exception generator 1150 Backend virtual display 1151 Backup storage 1152 Preservation section 1153 Detector 1700 Image processing device 1730 guest operating system 1731 graphics library 1733 Address calculation unit 1740 Virtual display 1741 Memory Management Department 1741a Table storage 1741e Exception generator 1750 backend virtual display 1752 Preservation section 1801 Guest virtual address space 1802 guest physical address space 1803 Host physical address space 1811 Guest virtual address 1812 Guest physical address 1813 Host physical address 1821, 1823 1st conversion table 1822 Second conversion table 1901, 1902 entries 1911, 1912 PageTable 1921 Host physical address
22 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22
Every citation, both ways
| Document | Relation | Office |
|---|---|---|
| JP2003085135A | Cites | Japan |
| JP10105367A | Cites | Japan |
| JP11331610A | Cites | Japan |
| JP2004086550A | Cites | Japan |
| JP2007025073A | Cites | Japan |
| JP05197795A | Cites | Japan |
6 members in 3 offices
Priority claims7
| Document | Office | Kind | Date |
|---|---|---|---|
| 2007179673 | Japan | A | |
| 2007179673 | Japan | A | |
| 2007179673 | Japan | – | |
| 2007228052 | Japan | A | |
| 20072007179673 | – | – | – |
| JP20070179673 | – | – | – |
| JP20070228052 | – | – | – |
Members6
| Document | Office | Kind | |
|---|---|---|---|
| US2009016566A1 | United States of America | A1 | |
| JP2009037581A | Japan | A | |
| CN101383040A | China | A | |
| US8045828B2 | United States of America | B2 | |
| CN101383040B | China | B | |
| JP4852012B2This record | Japan | B2 |
11 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Cancellation because of no payment of annual feesLAPS | LAPS | |
| Renewal fee payment (event date is renewal date of database)FPAY | FPAY | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Written amendmentJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Report on accelerated examinationJAPANESE INTERMEDIATE CODE: A971005A975 | A975 | |
| Explanation of circumstances concerning accelerated examinationJAPANESE INTERMEDIATE CODE: A871A871 | A871 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 |
Numbers
- Publication
- 4852012
- Publication, DOCDB
- 4852012
- Publication, EPODOC
- JP4852012B
- Application
- 228052
- Application, DOCDB
- 2007228052
- Application, EPODOC
- JP20070228052
Titles2
- Japanese
- 画像を処理する装置、画像の更新を検出する方法およびプログラム
- English
- Devices that process images, methods and programs that detect image updates
Classification
- IPC, 3
- G06F3 153
- H04N1 21
- G06T1 60
