Method for processing a digital image

^ls5?

(12) Patent Application SE SE 533 185 02 ω

<img file="SE533185C2_D0001.tif" />

Sweden (21) Patent application number: 0700446-8 (45) Patent granted: 2010-07-13 (41) Application generally available: 2008-08-17 (22) Patent application submitted: 2007-02-16 (24) Maturity date: 2007- 02-16 (83) Deposit of microorganism: - (30) Priority information: - (51) International class:

G06T3 / 20 (2006.01)

G06T3 / 40 (2006.01)

<td>(73) Patent holders:</td><td>Scalado AB, Scheelevägen 17, Beta 3, 223 70 Lund SE</td>

<td>(72) Inventor:</td><td>Sami Niemi, Skanör SE John Stén, Malmö SE</td>

<td>(74) Agents:</td><td>AWAPATENT AB, Box 5117, 200 71 Malmö SE</td>

<td>(54) Name:</td><td>Method for processing a digital image and image representation format</td>

<td>(56) Quoted publications:</td><td>US 6281874 Bl · WO 0109836 Al · US 20040175047 Al · US 20050008236 Al • WO 2005032119 Al · US 20060023953 Al · US 6381371 Bl</td>

(47) Summary:

Method of changing a view of a digital image sent to a monitor. The method includes: retrieving image data corresponding to a first area of ​​the digital image from a reduced size, with respect to the number of pixels, representation of the digital image, transmitting said retrieved image data corresponding to the first area to the screen, receiving a user input signal requesting that a second area should be sent to the monitor, retrieving image data corresponding to the second region of the digital image from the reduced size representation of the digital image and transmitting said received image data corresponding to the second region to the display screen.

<img file="SE533185C2_D0002.tif" />

533 185

SUMMARY

Method of changing a view of a digital image sent to a monitor. The method includes: retrieving image data corresponding to a first area of ​​the digital image from a reduced size, with respect to the number of pixels, representation of the digital image, transmitting said retrieved image data corresponding to the first area to the screen, receiving a user input signal requesting that a second area should be sent to the monitor, retrieving image data corresponding to the second region of the digital image from the reduced size representation of the digital image and transmitting said received image data corresponding to the second region to the display screen.

533 185

Technical area

The present invention relates to a method of changing a display view of a digital image sent to a monitor and a data structure for storing a digital image on a portable device.

Background of the invention

Today, it is common for images, such as photographs, still images, graphics, etc., to be viewed by any portable device having a display screen. However, it is not enough to allow images to be viewed on a monitor. The users of electronic devices that display images on a monitor are mostly interested in changing the display view of the image. Some common operations desired by the user are zooming into an image for viewing details of the image, panning the zoomed image for tracking an object or just for providing an overview of the details, rotating images to facilitate viewing on the screen, etc. In addition, the images handled by users of such electronic devices have increasing resolution, ie the number of pixels that define an image becomes larger and larger.

Said electronic devices may be, for example, mobile phones, small laptops (PDAs), PDAs, or other devices that have limited processing capacity with respect to the images to be handled. For example, many of said electronic devices having a display screen for displaying images do not have sufficient processing capacity to perform operations such as zooming, panning, etc. without exhibiting frequent and annoying delays between consecutive views. This can result in continuous zooming in or out of an image such as presenting a plurality of images with a long delay between the images. Thus, no experience of continuous zooming is obtained which can be annoying to the user. This can also lead to incorrect handling or input by the user.

533 185

A common way to address this problem is to either increase the processing capacity of the device or to avoid operations where the user expects an experience of continuous flow.

Summary of the Invention

An object of the present invention is to improve operations on images and to improve the experience of the user of continuous operations on images.

This object is achieved by a method of changing a display view of a digital image to be sent to a monitor according to claim 1 and a data structure for storing a digital image on a handheld device according to claim 10. Further embodiments of the invention is described in the dependent claims.

In particular, in accordance with a first aspect of the invention, a method of changing a view of a digital image to be sent to a display comprises retrieving image data corresponding to a first area of ​​the digital image from a reduced size, with respect to the number of pixels, representation of the digital image, transmission of the retrieved image data corresponding to the first area to the display, receiving a user-input signal requesting a second region to be sent to the display, retrieving image data corresponding to the second region of the digital image from the size-reduced representation of the digital image, and transmitting said retrieved image data corresponding to the second the area of ​​the monitor.

By manipulating the image view to be presented on a screen in the size-reduced representation of the digital image, the manipulation requires less processing time and can therefore be performed in a shorter time. One reason for this is that there is no need, in this embodiment, to decode the full size image to present the new view.

533 185

In another embodiment, the method further comprises receiving a signal input from the user requesting a third area to be sent to the display, and retrieving image data corresponding to the third area of ​​the digital image from a full size representation of the digital image if the selected third area indicates an area of ​​the digital image which, in the size of the reduced image, comprises less number of pixels than the size of the screen.

In yet another embodiment, the method comprises the step of analyzing said image data of full size representation of the digital image at a time after the digital image has been selected to be displayed and before said image data to be sent to the display is retrieved from a full size representation of the digital image.

In another embodiment, the method comprises storing the information obtained by the analysis.

In another embodiment, the method comprises using the information obtained by analyzing the image data from the full size representation of the digital image in the event that it is determined that the image data to be sent to the screen is retrieved from a full size representation of the digital image.

By using the analysis information during manipulation of the image at a time when the resolution of the reduced size image is not sufficient to provide acceptable quality, the manipulation rate of the image may still be high even though the image data must be retrieved from the fully resolved image.

According to another aspect of the invention, a method of changing a view of a digital image to be sent to a monitor comprises: retrieving image data corresponding to a first area of ​​the digital image from a reduced size, with respect to the number of pixels, representation of the digital image, transmitting said retrieved image data corresponding to the first area to the display, receiving a user input signal requesting that a second area should be sent to the monitor,

533 185 retrieving image data corresponding to the second region of the digital image from a full size representation of the digital image using information from an analysis of the full size image, and transmitting said retrieved image data corresponding to the second region to the display.

In one embodiment, the method further comprises analyzing said image data of the full-size representation of the digital image at a time before the digital image has been selected for reference.

In another embodiment of the method, said first region corresponds to a subset of the total size reduced image.

In yet another embodiment of the method, said second region corresponds to an enlargement of objects into said first region.

In another embodiment of the method, said second region specifies a region in a different position relative to the first region and within the digital image.

In another embodiment of the method, the reduced size representation of said digital image is a factor k greater than the size of the display view of the monitor, the factor k&gt; 1.

In another embodiment of the method, the full size representation of the digital image is stored as a compressed image, said analysis of said image data of the full size representation of the digital image comprising creating at least one pointer to a data block within the full size representation of the digital image, and wherein said created at least one pointer is used to access the image data in the selected area when it is determined that the image data to be sent to the monitor is retrieved from a full-size representation of the digital image.

In another embodiment of the method, the pointer points to a data block of the first data unit of a smallest coded unit, the MCU.

In another embodiment of the method, said information from the analysis of said image data comprises an absolute DC coefficient corresponding to the first data blocks of each color component of the MCU.

In another embodiment of the method, there is at least one pointer to a data block in each MCU sweep line.

185

0 ο

Woe

In another embodiment of the method, a data block is a data unit in a JPEG encoded image.

In another embodiment of the method, the method is performed in a server located at a distance from the handheld device and in which the transmission of said retrieved image data represents an area to be displayed sent to the monitor via a network.

In another embodiment, the method further comprises creating and displaying an animated transition between the displayed first area and the display of the second area.

In yet another aspect of the invention, a data structure for storing a digital image on an electronic device comprises:

image data corresponding to a full size representation of said digital image stored on the electronic device, image data corresponding to a reduced size representation of said digital image stored on the electronic device and having a size k a factor k larger than the size of the display device of the handheld device. , the factor k&gt; 1, the size of the images and the monitor are measured in pixels, and the full-size image is linked to the size-reduced image.

In one embodiment of the data structure, the image data corresponding to the size-reduced image is stored in a data area intended for information relating to the digital image.

In another embodiment of the data structure, said data area dedicated to information related to the digital image is a file header of an image file comprising the digital image.

In yet another embodiment of the data structure, said image data representing the reduced size image is arranged in a data area of ​​a database record which refers to the image data of the full size image.

In another embodiment of the data structure, said image data representing the reduced size image comprises non-differential DC coefficients valid for compressed image data representing the full size image.

533 185

In another embodiment of the data structure, said image data representing the full size image is in a compressed format represented as a sequence representing sequential image blocks, each block comprising one or more data units and each data unit represented as a sequence of Variable Length Coded coefficients .

In another embodiment of the data structure, the data area additionally includes indicators indicating the position of the image blocks in the sequence.

In another embodiment of the data structure, the data area additionally includes indicators indicating the position in the sequence for at least every sixty-four image blocks.

In another embodiment of the data structure, said data area further comprises indicators for at least one DC coefficient of each image block in the sequence.

According to a further aspect of the invention, a method of preparing a digital image for manipulation comprises:

retrieving information that speeds up the analysis of a compressed full-size image, analyzing the full-size image using the information that speeds up the analysis of a compressed full-size image, and compressed image data of the full-size image, and storing features, derived from the analysis, facilitates the full-size image, which facilitates the full-size image.

In one embodiment of the method, said compressed full size image is in a compressed format represented by a sequence represented sequential image blocks, each block comprises one or more data units and each data unit is represented by a sequence of variable length encoded coefficients of base functions and wherein said information accelerates an analysis. compressed full size image includes image block indicators.

In another embodiment of the method, indicators are included at least every sixty-fourth image block of the compressed

533 185 the full-size image in the information that speeds up the analysis of a compressed full-size image.

In yet another embodiment of the method, said information which speeds up the analysis of a compressed full-size image further comprises DC coefficients.

Further uses of the present invention are illustrated in the detailed description below. However, it will be appreciated that the detailed description and the examples given, which show preferred embodiments of the invention, are given as explanatory examples since various changes and modifications within the scope of the invention will be apparent to those skilled in the art from this detailed description.

Brief description of the drawings

The invention will now be described in more detail by way of example with reference to the accompanying drawings.

Figure 1 shows schematically an outer casing of a mobile phone.

Figure 2 shows a schematic functional block of a mobile phone.

Figure 3 is a flow chart of a method for generating an image file. Figure 4 shows a structure of an image file according to one embodiment. Figures 5a-c are flow diagrams of a method for performing manipulations of an image view in accordance with an embodiment of the invention.

Figures 6a-e are schematic views of the relationship between a reduced size image, a full size image, and a screen view.

Detailed description of a preferred embodiment

In the following description, the invention is described as used on a mobile phone. However, the invention can be used for presenting and manipulating images displayed by other handheld electronic devices such as small laptops (PDAs), palmtops, etc. or displayed on personal computers, terminals, etc. connected to servers that perform operations for manipulating the display of images and wherein the operations are performed on a server that has limited processing capacity with respect to the load on the server. The manipulation of the display of

533 The 185 image may include zoom in and out within the image, pan within the image, and rotation.

With reference to Figure 1, according to one embodiment of the invention, a mobile telephone 10 realizing the invention may comprise a display screen displaying a display screen 12 and a user input means 14.

A display screen 12 of a monitor is arranged to have a resolution rx in an x-direction and a resolution ry in a y-direction, i.e. the display is arranged to present a matrix of pixels, each row having a length of rx pixels and each column has a length of row pixels. Where reference is made to presentation of an image or image data on a monitor or monitor view of a monitor in the following description, the monitor or monitor view of the monitor shall be interpreted as the display surface provided for presenting the image or image data.

The user input means 14 may be any known means used for controlling the telephone. For example, it may be the buttons on the telephone, a touch screen, ie, input via the screen, a pressure-sensitive plate, an external mouse or joystick, etc. With respect to the present invention, the input means are used to control the manipulation of the image display.

Further, the mobile telephone may comprise an optical input 16 such as a lens arranged for receiving light for generating still or video images.

Referring to Figure 2, according to one embodiment of the invention, the mobile telephone comprises a processor 18, e.g. a microprocessor, a CPU, etc., a memory 20, a display driver 22, a communication means 24, and a camera circuit 26. The processor 18 is arranged to execute software that embodies at least parts of the invention, of course, the method realized by the software can be implemented by others. means, e.g. via electrical circuit systems.

The memory 20 may be a volatile memory, e.g. a RAM memory, a non-volatile memory, e.g. a ROM memory, a flash memory, etc., or the memory may be a volatile memory and a non-volatile memory. In view of the present invention, the memory is arranged to store the software which realizes at least parts of the invention and to store image data for processing.

533 185

The display driver circuit 22 is arranged to control the pixels of the display 12 in response to display instructions from the processor. The display driver circuit 22 may be any drive circuit compatible with the specific display screen 12.

The communication means 24 is arranged to enable communication. The communication means may be realized partly as software and partly as hardware.

The camera circuit 26 includes an image sensor as well as the hardware and software needed to generate images from the image sensor.

A mobile phone includes additional organs and devices for its operation. However, in order to facilitate the understanding of the invention, the means and units which do not contribute to the understanding of the invention are not described.

An image to be displayed on the mobile phone 10 can be stored as an image file in the memory 20 of the mobile phone 10. The image represented by the image file can be captured by the optical input 16 and the camera circuit 26. The image is stored in the image file as a full-size image, which represents significantly more pixels than the display 12 of the mobile phone 10 can show. Figure 3 schematically describes an example of a method for generating an image file.

From the beginning, a full-size image is obtained, step 100. From the full-size image, a size-reduced image is generated having a width of w pixels and a height of h pixels, step 102. Each of the width w and the height h of the size-reduced image is a factor k greater than the size of the corresponding width rx and height ry of the display 12. The factor k, according to one embodiment of the invention, may be in the range 1-2. According to one embodiment, the factor k is chosen such that the height and width of the size-reduced image is of a size corresponding to n raised in two smaller than the full-size images, where n can be any integer that results in a size-reduced image having a respective width and height. is equal to or greater than the width and height of the display area of ​​the monitor, respectively.

If the full size image is not already coded or compressed, as it usually is, then the full size image, step 104, is encoded or compressed into a predetermined format, e.g. jpeg, tiff, gif, bitmap format, protected format etc. It

533 The 185 reduced size image is also encoded and / or compressed, step 106, to a predetermined format, e.g. jpeg, tiff, gif, bitmap format, protected format, etc. The full size image is then stored as an image file, step 108, and the reduced size image is stored as file information relating to the image file, step

110th The reduced size image can be stored in a file header to the image file or in a database. If the resized image is stored in a database, information relating to the resized image to the image file may be stored in the database, information relating the image file to the resized image in the database may be stored in a file header to the image file, or information relating to the image file may be stored. the image file and the resized image to each other are stored both in the database and in a file header to the image file. Yet another way is to link a size-reduced image in a database to the full-size image via knowledge of the file of the full-size image. For example, this can be accomplished by using a hash code calculated from the file name of the full size image and or by using the date of capture. Then, the size-reduced image can be stored in a file system with the hash code as part of its file name.

The structure of an image file 120 according to an embodiment of the invention is shown in Figure 4. The full-size image is stored as encoded or compressed image information 122. In addition, the image file 120 comprises a file head 124. The file head 124 comprises image and / or file information in a section 126. Image information and file information may be information relating to the time and date of the image being produced, settings of the camera when the image was taken, the camera's model and type, the time and date when the file was last modified, the size of the image file, etc. The file header may further include image information such as allows you to speed up viewing, zooming and panning of the full-size image or include image information that speeds up the analysis of the full-size image. In section 128 of file head 124, the size-reduced image can be stored as image data. The reduced size image may be coded or compressed.

Furthermore, the reduced size image may be generated in any known manner. For example, if the size-reduced image is to be generated from an uncompressed or decompressed image, a method such as

533 185 nearest neighbor, car linear or bicubic re-sampling is used. However, if the image is compressed using JPEG, the reduced image size can be generated by decoding only DC coefficients and / or a limited number of AC coefficients, where decoding to the desired resolution / scale is performed using only the frequencies needed for the desired resolution / scale.

By storing a reduced image that has a size k times the size of the monitor, as previously described, initial manipulation of the image, such as zooming and panning, can be performed without the user having to wait for the processor to decode or decompress the image. Thus, the response to a manipulation instruction by the user is essentially immediate and substantially no waiting time is experienced.

The image information enabling accelerated viewing, zooming, panning and rotation, also called manipulation, of the full-size image may include, in the case where the image is compressed by JPEG or similar compression methods, indicators for direct access to one or a plurality of individual MCUs along with absolute DC coefficients for each first data unit of MCU's color components or channels. According to an alternative embodiment, the image information which makes it possible to accelerate the manipulation of the full-size image may also include indicators for each of the other data units of the MCU.

An indicator shall, if not specified, be understood as an absolute or relative address of the indicated bit in a data stream or a marker in a bit stream.

A method for carrying out the manipulations according to one embodiment of the invention is described in Figures 5a-c. Initially, the user selects an image to be presented on the monitor by manipulating the mobile phone's input means. Then the mobile phone receives an instruction to display a specified image, step 200. In response to this instruction, the mobile phone retrieves the size-reduced image relating to the specified image, step 202, and scales it down to fit the display, step 204, before presenting the selected image on the mobile phone's display, step 206.

533 185

Now the selected image has become visible on the screen and the user is interested in looking at a specific detail of the image. Accordingly, the user indicates that he wants to zoom into the image. The zoom-in instruction is received by the mobile phone, step 208, and the processor crops and scales the size-reduced image in accordance with the zoom-in instruction, step 210 to fit the screen view. It is possible to zoom in using the size-reduced image and, depending on the embodiment, it is done either by using a size-reduced image that is larger than the size of the screen or by allowing magnification of size-reduced images more than a single image. to a scaling ratio, ie for some applications an enlargement up to 25% -30% is acceptable. Consequently, the zoom operation becomes faster because the number of pixels to take into consideration is few and the size of the reduced image does not need to be decompressed. Then, the image resulting from the peeling is presented in steps

210 on the screen, step 212. Now that the image has been zoomed once, it will also be possible to pan within the image. The user can now zoom in further, zoom out or pan. Let's assume the user chooses to zoom in further. The mobile phone receives the instruction, step 214, and identifies the instruction as a zoom-in instruction, step 216. Thereafter, the resulting zoom factor z, step 218, is checked. If the zoom factor z is greater than p, the expected view requires such magnification that the size-reduced image is not possible to use to provide acceptable or necessary quality and therefore the process proceeds to step 219 to begin using the full-size image instead. If the zoom factor z is less than or equal to p, then the size-reduced image is still used and the process proceeds to step 220. The zoom factor z can be a value that indicates how much scaling of the size-reduced image is needed to display the image that the user has requested. For example, if the size of the reduced size image, ie w and hi pixels, is twice the size of the screen view, ie rx and ry in pixels, then the zoom factor z is from the beginning, when the whole image is to be displayed, Vz. Accordingly, in such an embodiment, the zoom factor corresponds to a scaling factor to be applied to the size-reduced image to present a

533 185 image zoomed to such a degree as the user requested. In other words, in the example where the size reduced image is twice the size of the screen and no zoom is required then w / 2 = rx and h / 2 = ry, ie wz = rx, hz = ry and z =¹A In addition, when the requested zoom is of such a value that a pixel of the size-reduced image corresponds to a pixel of the screen, then z = 1. The value of p, which defines the limit for using the size-reduced image, can be 1, ie p = 1. For some applications, however, it is even possible to enlarge the size-reduced image over the zoom factor z = 1 without noticeable effects and in such cases p can be set to 1.25, ie the image presented on the monitor may be a 25% magnification of the size reduced image. Magnifications of the reduced size image may be acceptable up to a height and width of approximately 1.5 times larger than the unscaled size.

Let us now return to Figure 5b and assume that the zoom factor z is less than or equal to p, in which case the resized image is scaled and scaled again in accordance with the zoom instruction, step 220, and then the resulting image is presented on the screen, step 222. The process then returns to step 214 ready to receive a new instruction. If the zoom factor z is larger than p, the process switches to the full size image, which is described below in connection with Figure 5c.

If the received instructions relate to a panning manipulation of the image view, step 224, a new position of the image view, i.e., a viewing area corresponding to a portion of the entire image displayed, is arranged within the image associated with the instruction and the image data representing the viewing area is retrieved from the reduced size. the image, step 226. The image is then cropped and scaled according to the current zoom selection, step 228, and displayed on the screen, step 230. The process then returns to step 214 ready to receive a new instruction.

Figure 5c shows the process of manipulating the image view when the image has been zoomed past the boundary for switching from generating the displayed image view from the reduced size image to generating the displayed image view from the full size image. When the zoom into the image has passed the limit given in

533 Accordingly, Figure 5b, step 218, is retrieved, cropped, and scaled the full size image according to the zoom instructions, step 232. Next, the scaled full size image is displayed on the screen, step 234. The process is then ready to receive additional instructions for manipulating the image view, step 238. If the subsequent received instruction is a zoom instruction, step 238, the full size image is trimmed and scaled according to the zoom instructions, step 240 and the scaled full size image is displayed, step 242. The mobile phone is then ready to receive the next instruction, step 236.

If the instruction received in step 235 is a panning instruction, step 244, then a new position of the image view is arranged, i.e. a viewing area corresponding to a portion of the entire image displayed, within the image associated with the instruction and the image data representing the viewing area is retrieved from the full size image, step 246. The image is then cropped and scaled according to the current selection of zoom, step 248, and displayed on the screen, step 250. The process then returns to step 236 ready to receive a new instruction.

Figures 6a-e show schematically an overview of the relationship between a reduced size image, a full size image and the image view of the screen. The overview also relates to the method of carrying out the manipulations in accordance with one embodiment of the invention as described in Figures 5a-c. Figure 6a depicts the relationship between the image view 260 presented on the screen and the size-reduced image 262. The image view 260 is rx pixels wide and ry pixels high and the size-reduced image 262 is w pixels wide and h pixels high. As previously mentioned, the width w of the size-reduced image 262 is a factor k times the width rx of the image view and the height h of the size-reduced image 262 is a factor k times the height ry of the image view. Accordingly, an initial image view 260 intended to display the entire image is generated by downscaling to fit the display of the size-reduced image 262, i.e., downscaling to the size of the image view.

Figure 6b depicts the situation when the image is zoomed in and the image data to be displayed in the image view 260 is still retrieved from the size-reduced image 262. As shown in Figure 6, a zoom in

533 185 from one or a plurality of zooming instructions a region 264 comprising fewer pixels than the whole size of the reduced image 262. Thus, the generation of the display view 260 of the image includes scaling the image in the region 264 for adaptation to the display.

Figure 6c depicts a situation when panning the zoomed image, ie

the area 264 intended for the screen is moved within the image area.

Figure 6d depicts a situation where zooming has resulted in an image area 264 of the same size as the screen, with respect to the height and width of the number of pixels. In this situation, no scaling of the area is intended to be performed.

Figure 6e depicts a situation where the zooming has resulted in an image area within the size-reduced image smaller than the view view of the image and therefore the image area 264 of the size-reduced image needs to be scaled up to the size of image view 260. This results in a presentation of the display that does not fully utilize the display resolution.

However, this may be acceptable for situations when the loss of perceived quality is small.

Figure 6f depicts a situation when the image information to be displayed is retrieved from the full-size image 266 instead of from the size-reduced image because the zoom manipulations have reached such a level that the size-reduced image can no longer provide acceptable quality. As a result of switching to full size image 266, the image area now needs to be scaled down again to fit the screen.

By using the reduced size image, the manipulation of the image view related to a specific image becomes faster and the response time from the moment a user indicates a manipulation until the manipulation is performed is shorter compared to if a compressed full size image would have been used. One reason for this is that in today's systems, the entire set of image data representing the image needs to be decoded each time a manipulation of the image view is to be performed if a compressed full-size image is used. This is very time consuming. If said image data is decompressed once and all manipulation is performed on said decompressed image data, the time consuming will not be so large

533 185 problems longer, however, a decompressed image would occupy a large memory area, about 6-20 times larger memory area than that occupied by the compressed image. In addition, the initial decoding of a full-size image would take much longer than the total time required for analysis and decoding of a few selected areas.

However, as described in the above process, it is not satisfactory to process and present image data from the resized image when the image view to be displayed is zoomed in to such an extent that it is no longer possible to use the resized image's image data to generate a picture view of the desired quality. Once this has gone so far, it is suggested that the full-size image data begin to be processed instead. Accordingly, the problem of the slow processing of the compressed image will be present when the image will be zoomed deep into the details of the image.

To eliminate the problem of delays, ie that compressed full size images are time consuming to process, and storage capacity, i.e. uncompressed images occupy a large memory area, the full size image is analyzed and features that facilitate quick handling of an image are stored in the image file, as is the description of Figure 4. temporarily stored in memory for the current manipulation, or stored as a record in a database, said record in a database can refer to the image file. A variety of methods for analyzing, extracting and storing such properties with respect to an image are described in patent application WO 2005/050567 by Scalado AB.

In order to be able to extract said features to facilitate the rapid manipulation of an image, said features can either be extracted during the compression of the image or they can be extracted during an analysis of a compressed image made after the compression. If the full-size image is compressed using JPEG compression, or a similar compression method, the features that facilitate rapid manipulation of a retrieved and stored image may be any or all of the MCU indicators, where an MCU is a small image block of the image, indicators of one or more data units, wherein a data unit is a data block representing a channel or color component of said MCU, one or

533 185 a plurality of absolute or relative DC coefficients from one or several of the color components of the retrieved MCUs and / or of the retrieved data units, or the number of bits between data units, or between specific coefficients of the data units. How to use such features to achieve fast processing of an image is described in the above application, ie WO 2005/050567 by Scalado AB.

In one embodiment, as mentioned earlier, information that speeds up the analysis of a full-size image may be associated with the image, e.g. by storing the information in the file or by referring to a location where said information regarding the particular image is stored.

An example of information that speeds up the analysis is data units from an image compressed using JPEG compression, or a similar compression method. By storing indicators for at least some data units of the compressed full size image, the variable length coded portions of the data units do not necessarily need to be decoded during analysis of the compressed full size image. Parts that are variable length encoded can be coded or decoded according to any variable length coding, e.g. Huffman code, arithmetic code, etc. Consequently, the analysis process is speeded up. In a particular embodiment, an indicator for each data unit is stored by the compressed full size image. As a result, the variable length coded portions do not need to be decoded to locate the data units.

By knowing the location of the data units, it is possible to decode the differential DC coefficients quickly, since the DC coefficients are the first coefficients in a data unit, thereby enabling the creation of a table or list of the absolute DC values. This allows direct access to any of the data units in the image and allows skipping of time-consuming variable length codes during the decoding of an image for less detailed viewing.

Another example of information that accelerates the analysis is a combination of storing indicators to data units as described above and storing at least some of the DC coefficients, preferably DC coefficients related to data units represented by a stored

533 185 indicator. By storing these features, it becomes possible to directly generate analysis features from the information that speeds up the analysis. In an embodiment where all the data units and DC coefficients of the full-size image are stored as information that speeds up the analysis, a table can be generated without any decoding or with minimal effort.

In a further example, DC coefficients of the full-size image are stored as image data or part of the image data in the size-reduced image. Thus, when the DC coefficients are needed, they are retrieved from the information representing the reduced size image.

The indicators of the data units may be absolute, ie the absolute position of the file is specified, or relative, ie the position of the data unit within the image file is specified relative to the previous data unit, or some other indicator indicating the position. An advantage of absolute indicators is that each data unit can be accessed independently and directly. An advantage of relative pointers is that each pointer can be described using less memory space.

As discussed above, said features that facilitate rapid manipulation of the full-size image can be extracted for a period of time starting after the image has been selected for display and linked to the image file selected for display, i.e., the image specified in step 200 of Figure 5a, The compressed image is analyzed during the image manipulation process during which process said size-reduced image is used. Decompression and / or analysis of the compressed full-size image, step 252 of Figure 5a, for retrieval of said features can be started as soon as the image to be displayed has been selected or as soon as a first zoom instruction regarding the image has been received. Waiting for the first zoom instruction can be advantageous because no processing of the full-size image is started before the user expresses an interest in manipulating the image and thus reduces the risk of wasting processing capacity on an image displayed while browsing the images. By carrying out the analysis in parallel with the manipulation, it may be possible to achieve non-delayed transition between manipulation of the reduced size image and the full size image, if the analysis is completed before the full size image needs to be used for

533 185 manipulation, and then proceed to rapid manipulation of the image using the features obtained from the analysis. In one embodiment, in addition, an animation of the manipulation can be displayed when the analysis is performed or at intervals when the analysis is performed. For example, an animated zoom effect may be displayed in conjunction with a zoom manipulation, an animated rotation effect may be displayed in conjunction with a rotation manipulation, an animated gliding effect may be displayed in conjunction with a pan manipulation. In this way, the risk of the user experiencing a delay is minimal.

In another embodiment, indicators for one or more MCUs and at least one absolute DC coefficient related to each color component of said MCUs may be included in the image information which speeds up the analysis of the full size image. By including these features, an initial analysis can be performed only on the zoomed image area. Consequently, this results in rapid analysis as it is possible to analyze only part of the image. At 200% zoom, this results in about four times faster analysis than if the entire image had to be analyzed. If the image view is paned, only a portion of the image information of the new view is not analyzed and thus the analysis of the rest of the image information for the new view can be analyzed very quickly.

According to another embodiment of the invention, the image information which speeds up the analysis of the entire image is indicators of data units of the compressed full size image.

According to yet another embodiment of the invention, the image information which speeds up the analysis of the compressed full size image is indicators to data units of the compressed full size image and absolute DC coefficients related to the specified data units.

The number of indicators for different MCUs, which indicators are stored as image information that speeds up the analysis, can vary from each of said MCUs, every second, every fourth, every eighth, etc. In one embodiment, indicators are stored for at least one MCU on each row of MCUs in the image as image information that speed up the analysis of the image.

533 185

In an embodiment where one of the methods of storing image information which speeds up the analysis of the image mentioned above is realized, the size of the reduced size image may be set to the same size as the screen.

According to another embodiment, an image file as described above need not necessarily be retrieved from the memory of the device displaying the image but may be downloaded via communication means, e.g. communication means 24 of the mobile telephone in Figure 2. Furthermore, the image file can be stored on an external server and referred to in the portable device by a reference stored in the portable device. Information that speeds up the handling of the image can also be stored on the external server.

According to yet another embodiment, the method of manipulating an image can be performed entirely on an external server and the resulting view can be transmitted to the portable device. In this way, it becomes possible for external servers to process a greater number of image manipulations over a specific period of time, since only the requested areas need to be processed. The resulting view can be transmitted via any network or combination of networks, e.g. Internet, LAN, WAN, mobile phone networks, WIFI, land-based telephone networks etc.

533 185