Client and image display system

The present invention relates to a client and an image display system, and more particularly to a technique for appropriately dealing with a decrease in operability due to a change in display resolution in the client.

Among the documents known before the filing of the present invention, for example, Patent Document 1 is a document in which a technique related to the present invention is described.

In Patent Document 1, a screen transmission application and an application program for a mobile phone terminal are downloaded from a management server to each of a personal computer and a mobile phone terminal, and the full screen of the personal computer is displayed on the display device of the mobile phone terminal. Is described (eg, paragraph 0033, etc.). Here, since the size of the display screen of the personal computer and the display screen of the mobile phone terminal are often different, in the technology disclosed in Patent Document 1, the screen of the mobile phone terminal is used as a center server for relaying screen information. The aspect ratio and size of the transmitted image are compressed according to the size.

<p><patcit num="1"><text>JP-A-2007-251630</text></patcit></p>

<p> Recently, a computer remote control technology called "remote desktop" is known. This is an image output by a server, especially a desktop image, using a server and a client that are communicably connected via a network, such as a personal computer (hereinafter, "PC") at home and a mobile PC. It is sent to the client and displayed on the display device of the client. At the same time, the client uploads the operation performed on the displayed desktop image (for example, pointing by a pointing device, character input, etc.) to the server, and reflects the operation on the server.</p><p> In addition to mobile PCs, it is becoming more realistic to use mobile phone terminals as clients for remote desktops due to the improvement in performance and miniaturization of processing devices. However, the size of the display screen is usually different between the home PC and the mobile phone terminal, and the aspect ratio may also be different. Then, on the home PC on the desktop screen sending side, image processing such as resizing the desktop screen (mainly reducing the size) or cutting out a part is performed, and the cut out desktop image is sent to the mobile phone terminal. Is required.</p><p> On the other hand, some mobile phone terminals are equipped with a gravity sensor, an acceleration sensor, etc., and have a function of detecting the tilted state of the mobile phone terminal and changing the orientation of the screen to be displayed on the display device according to the detected tilted state. There is something to prepare for. That is, it is a function of detecting that the user is browsing the screen vertically or horizontally and switching the orientation of the screen. With this function, the user can display a portrait screen when browsing the Web, and a landscape screen when browsing a lot of landscape content such as video and photo browsing.</p><p> Consider adopting such a mobile phone terminal as the above-mentioned remote desktop client and using the function of switching the screen orientation when displaying the above-mentioned desktop screen. Then, the resolution is changed by switching the screen orientation, but this resolution change changes the resolution of the screen displayed on the server and sends the changed screen image to the client and displays it on the client. The problem is that it takes time. After changing the resolution, it takes time to display, which reduces operability.</p><p> Here, in consideration of a mobile phone terminal having a function of detecting the tilted state of the terminal and switching the screen orientation, it takes time to display on the client when the screen resolution is changed. He pointed out the problems of the prior art, such as the deterioration of operability. However, the trigger for changing the screen resolution is not necessarily limited to switching the screen orientation (screen rotation), and for example, manual resolution change can be considered. Further, for example, even when a notebook PC, a PDA, or the like is adopted as a client in addition to a mobile phone terminal, the above-mentioned problems still exist.</p><p> The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a client and an image display system in which a decrease in operability due to a change in display resolution is improved.</p>

<p> In order to achieve the above object, the present invention is, as a first aspect, a display means for displaying a desktop screen sent from the server, which is a client connected to the server via a network, and a display means for displaying the desktop screen. When the resolution of the image to be changed is changed, the remote operation control means for controlling the display in the display means to be displayed in the resolution before the change until the screen with the changed resolution is sent. It provides a client characterized by having.</p><p> Further, as a second aspect, the present invention is an image display system in which a server and a client are communicably connected via a network, and the client is a display means for displaying a desktop screen sent from the server. , When the resolution of the image to be displayed on the display means is changed, a resolution change notification including the changed resolution information is transmitted to the server, and until the screen with the changed resolution is sent. The server has a remote operation control means for controlling the display in the display means to be displayed at the resolution before the change, and when the server receives the resolution change notification, the size and the size suitable for the resolution information after the change are obtained. It provides an image display system characterized by having a desktop screen generation means for generating an orientation desktop screen and sending it to the client.</p>

<p> According to the present invention, it is possible to provide a client and an image display system in which a decrease in operability due to a change in display resolution is improved.</p>

<figref num="1">It is a figure which shows the whole structure of one Embodiment which carried out the present invention preferably.</figref><figref num="2">It is a conceptual diagram for demonstrating the change of the resolution in this embodiment.</figref><figref num="3">It is a figure which shows the transition example of the screen displayed on the display part 102 of FIG.</figref>

FIG. 1 shows the overall configuration of an embodiment in which the present invention is preferably implemented. In this embodiment, using a server and a client that are communicably connected via a network, an image output by the server, particularly a desktop image, is sent to the client and displayed on the display device of the client, and at the same time, the client. , An embodiment of a so-called "remote desktop" technique in which an operation performed on the displayed desktop image (for example, pointing by a pointing device, character input, etc.) is uploaded to the server and the operation is reflected on the server. Is. Although not limited to this, in the present embodiment, the home PC2 is used as the server device and the mobile phone terminal 1 is used as the client device. In addition, an electronic device such as a mobile PC or PDA may be used as the client device.

First, the configuration of the home PC2 will be described. The home PC2 has an operation system 201 installed, which is basic software that realizes predetermined information processing such as calculation by using the hardware resources of the home PC2. In addition, the hardware and software collaborate in the same manner. A functional block called the desktop screen generator 202 is configured by the function.

The operation system 201 includes a GUI (graphical user interface), and provides an interface for the user to control the operation system 201 or the home PC2 by the GUI.

This GUI generates a screen that would be displayed in the entire display area of the display if a display such as a liquid crystal display is connected to the home PC2, as shown by the dotted line in Fig. 2. There is. Note that FIG. 2 is a conceptual diagram for explaining the resolution change in the present embodiment.

On the other hand, the desktop screen generation unit 202 causes the display unit 102 of the mobile phone terminal 1 to display from the screen generated by the GUI based on the resolution information and the like included in the resolution change notification received from the mobile phone terminal 1. Generate a screen (this is called a "desktop screen"). This desktop screen is illustrated by the part surrounded by the solid line in the part surrounded by the dotted line in Fig. 2. The desktop screen generation unit 202 transmits the generated desktop screen to the mobile phone terminal 1. The transmission timing may be determined by the frame rate on the display unit 102 (for example, 30 fps is transmitted 30 times per second).

Next, the configuration of the mobile phone terminal 1 will be described. The mobile phone terminal 1 has a remote control control unit 101, a display unit 102, an inclination state detection unit 103, an operation unit 104, and an operation recording unit 105.

The remote control control unit 101 is a functional block that executes various processes in order to realize remote control and resolution change according to the present embodiment. The remote control unit 101 receives the desktop screen transmitted from the home PC 2 and displays it on the display unit 102. At this time, if there is an area that cannot be displayed beyond the display unit 102 and / or an area that cannot be displayed due to lack of screen data, the former is ignored and the image is not displayed. Image processing such as filling in black or image processing such as reducing and displaying the sent desktop screen is performed.

Further, the remote control control unit 101 recognizes that the resolution has been changed when the tilt state detection unit 103 detects the tilt of the mobile phone terminal 1, and transmits a resolution change notification to the home PC 2. However, this shall be performed when a tilt (change in tilt as shown in (a) (b) in FIG. 2) that is greater than or equal to the extent to which the orientation of the screen provided by the display unit 102 is switched is detected. The resolution change notification includes the changed resolution information. Further, the resolution change notification is input to the desktop screen generation unit 202.

The resolution is automatically changed by the tilt detection unit 103, the manual change by the user using the operation unit 104, and the automatic change by the application installed on the mobile phone terminal 1 (not shown). Changes etc. are possible. It is not limited to the one by detecting the tilt.

Further, the remote control unit 101 receives the operation by the user received by the operation unit 104 as the operation information from the operation unit 104, and transmits the operation information to the home PC 2. The operation information is input to the operation system 201. The operation system 201 that has received the operation information executes a predetermined process based on the operation information (file operation, character input, etc.).

The remote control unit 101 records the operation information received within a certain period of time due to the above-mentioned resolution change in the operation recording unit 105 for a while without transmitting it to the home PC 2 as it is. Perform processing. This is because, shortly after the resolution is changed, the desktop screen generation unit 202 does not generate the desktop screen, cannot execute the process based on the transmitted operation information, and becomes invalid.

The display unit 102 is a functional block that displays the desktop screen received by the remote control control unit 101, and can be sufficiently implemented by conventionally known techniques such as a liquid crystal display device and an organic EL display device.

The tilt state detection unit 103 detects the tilt state of the mobile phone terminal 1. Since the mobile phone terminal 1 is usually held by the user by hand, the tilted state is a concept that includes the state of orientation of the terminal depending on how it is held. The tilt state detection unit 103 can be sufficiently implemented by conventionally known techniques such as a gravity sensor and an acceleration sensor.

The operation unit 104 is a functional block that accepts operations by the user. It may be carried out with a numeric keypad or the like. Although the present invention is not limited, the touch panel may be used and the touch panel may function as the display unit 102 and the operation unit 104. The operation unit 104 generates operation information including character input, movement of the pointing cursor, and the like by the operation by the user, and passes the operation information to the remote operation control unit 101.

The operation recording unit 105 can be executed, for example, in a memory or the like, and temporarily records the operation information received from the operation unit 104 by the remote operation control unit 101 within a certain period of time due to a change in resolution. It is a functional block to put. The operation information recorded in the operation recording unit 105 is appropriately read out by the remote operation control unit 101.

The change in resolution in the present embodiment will be described once again with reference to FIG. In the present embodiment, the tilted state (in other words, the holding mode) of the mobile phone terminal 1 is detected, and the resolution is adjusted to each of the vertical state (FIG. 2 (a)) and the horizontal state (FIG. 2 (b)). It is a system that generates a desktop screen of a home PC2, sends it to a mobile phone terminal 1, and remotely controls it. The tilted state detection unit 103 detects an inclined state such as a vertical state or a horizontal state.

Further, the change of the resolution in the present embodiment will be described with reference to FIG. FIG. 3 is a diagram showing a transition example of the screen displayed on the display unit 102 of FIG. As shown in the figure, when a change in resolution is detected during display at the first resolution, the remote control unit 101 transmits a resolution change notification to the home PC 2. However, the time required for these processes and the time required for the remote control unit 101 to receive the resolution change notification on the home PC2 side, generate a new desktop screen, send the generated desktop screen, and receive it on the remote control unit 101. There is a time lag due to the need to reconnect.

By this new resolution, that is, the second resolution, until the newly generated desktop image is displayed, in the present embodiment, first, the resolution before the resolution change (first resolution) is used. Continue to display. Next, the operation recording unit 105 records the operation information input from the operation unit 104. Then, when the desktop screen generated by the new resolution is displayed, the operation information recorded by the operation recording unit 105 is reflected on the desktop screen.

That is, in the present embodiment, even after switching the screen orientation, there is no deterioration in operability such as display stop until the desktop screen generated by the new resolution is displayed, and the previous resolution remains. The desktop screen continues to be displayed with. Then, in the display device of the mobile phone terminal, an area that protrudes and cannot be displayed and / or an area that cannot be displayed due to lack of screen data is generated. On the other hand, in the present embodiment, image processing is performed such that the former is ignored and not displayed, and the latter is painted in black. However, such measures cannot make full use of the size of the screen of the display device, and there is room for further improvement in the deterioration of operability. Therefore, in the present embodiment, the operation information input to the desktop screen that continues to be displayed even after the resolution is changed is recorded and retained, and the desktop screen at the new resolution is reflected after being received. Therefore, the user can continue the operation without interruption, and the operability is further improved.

According to the present embodiment described above, in the "remote desktop", even if the client having the display means changes the resolution, the display means is used until the desktop screen with the changed resolution is sent. Since the display continues to be displayed at the resolution before the change, the desktop screen does not disappear. Therefore, the decrease in operability due to the change in resolution is improved.

Further, according to the above embodiment, the operation performed on the desktop screen that continues to be displayed at the resolution before the change is temporarily held as an operation record and reflected on the desktop screen after the resolution change. The user can continue the operation without interruption.

1 Mobile phone terminal (client) 101 Remote control unit 102 Display 103 Tilt state detector 104 Operation unit 105 Operation recording unit 2 Home PC (server) 201 Operating system 202 Desktop screen generator