Digital image processing method for mobile phone, involves performing Huffman decoding of preset number of coefficients of data unit, so that rest of coefficient is skipped by jumping to next zeroth/first order coefficient in bitstream

(54) Title: Method for processing a digital image and image representation format (56) Publications cited: - (47) Abstract:

An image representation format for representing a digital image includes: image information stored as a bit stream representing sequential image views, each block comprising one or more components, each comprising one or more data units and each data unit represented as a Huffman-encoded stream of coefficients of base functions, and a zero the coefficient of the order is represented as a difference to the previous zero order coefficient of the corresponding component, and a block information table which includes: indicators for the first coefficient of a specified order of each image block in said bitstream, information indicating the number of bits in the bitstream between adjacent coefficients of said specified order of the image block, and the zero order coefficient of at least the first data unit of each component, said zero the coefficient of the order is represented in a non-differential form.

3i 32 33 rp

Ί1 J 1

528 172

SUMMARY

An image representation format for representing a digital image comprises: image information stored as a bit stream representing sequential image blocks, each block comprising one or more components, each comprising one or more data units and each data unit represented as a

Huffman-coded stream of coefficients of basic functions, and wherein a zero order coefficient is represented as a difference to the previous zero order coefficient of the corresponding component, and a block information table which includes: indicators for the first coefficient of a specified order of each image block in said bitstream, information indicating the number of bits in the bitstream between adjacent coefficients of said specified order of the image block, and the zero order coefficient of at least the first data unit of each component, said zero the coefficient of the order is represented in a non-differential form.

2-J06-06-29 10:56 V: \ _HoOrganisatif, n \ SCfiXAK3 W \ PATEMT \ _NcFainily \ SS \ 2102073ö \ 21Q20736 ApplicaLioritezLToInstrucccr C, AS 2005-08-11 i.do ·; ·

528 172

Technical field of the invention

The present invention relates to a method of processing a digital image and to an image representation format for representing a digital image.

The present invention further relates to a method of encoding raw image data into a compressed digital image representation and a method of analyzing a JPEG compressed digital image.

Background of the invention

In today's society, huge amounts of information are created every day. Much information is presented in the form of pictures. In addition, large amounts of the information are stored and presented electronically, for example on the Internet. Meanwhile, there is an increased use of wireless connections to the Internet, in which the data transmission rate is relatively low. Thus, there is a need to present information in a very compact form. This is especially important for images, since a normal digital image, stored as a pixel array, is represented as a fairly large amount of data.

In addition, mobile phones containing a camera are becoming increasingly popular. Thus, the mobile phone must be able to handle digital images. Mobile phones or other hand-held devices that handle images have a limited memory space and limited data processing power. Therefore, if image processing is to be performed on such devices, the digital images need to be efficiently and intelligently stored so that low demands are placed on memory space and data processing power.

For these reasons, there is a great interest in image compression. A popular method to compress

2OÖ5-C6 ~ z9 1C: 56 V: \ JJOOxganisatiOfiXSCÄIADO AB \ P &? BH? \ „SnFawlly \ SE \ 21020736 \ 2t020738 Appli cst.lontextTolnstructor CÄS 2005-08-11 l.doc

528 172 images are the JPEG (Joint Photographic Experts Group) standard. The JPEG standard is defined in CCITT Rec. T.

81.

However, for the sake of clarity, a brief description of the image file format is presented below

JPEG standard.

The JPEG standard defines a destructive baseline coding system, which is based on the DCT transformation, and an extended coding system for presenting the transformed image in smaller amounts of data. When converting a digital image to the JPEG file format, a DCT transformation and quantization of the image is made, each component of a color space model for the image being separately DCT transformed. All color components are represented as blocks, which are processed sequentially. The DCT transformed blocks are thresholded and quantized so that information from basic functions that have a low impact on how the image is perceived is rejected. The zero order coefficient (DC coefficient) of each component of each block is stored as the difference to the previous DC coefficient by Huffman coding. The higher order coefficients (AC coefficients) are arranged sequentially, the sequence being obtained by a zigzag order from the array. The AC coefficients are zero run length coded and additionally coded with Huffman coding.

The JPEG file format is designed to create a standard compression that significantly reduces the storage size of a digital image. Thus, the JPEG30 file format is not suitable for image manipulation. If there is a desire to process a digital image, it is most practical to transfer the digital image back into a representation in the room domain. However, when processing images on a device that has a small storage space, such as a mobile phone, it is not possible to safely handle the large storage requirement of the digital image represented in the room domain.

2006-06-2? 10:56 V: \ So0xgar.isfttion \ SCftIA30 ftB \ PÄTSHT \ KoFe »ily \ 3E \ 2i020736 \ 21020730 AprUicationtexfrcUnstxuct;!»! CAS 2005-0 (3-11 i .doc

528 172

EP 1,037,165 describes a method for manipulating digital images stored in JPEG format. The bit stream of the JPEG image is scanned to identify image area locations in the bit stream. Some designated locations are stored in a pre-scan table for easy access, whereby selected areas of the image can be accessed without the need to decode the entire bit stream when manipulating a portion of the image. However, there is still a need to further increase the speed of image processing while maintaining low memory requirements.

Summary of the Invention

It is an object of the invention to provide compressed images that can be easily analyzed and / or manipulated. It is a further object of the invention to provide an opportunity to easily merge digital images into a compressed image representation format.

These and other objects of the invention are achieved, according to a first aspect of the invention, by a method of processing a digital image. The method includes: providing the digital image in a compressed format, in which the digital image is represented as a bitstream representing sequential image blocks, each block comprising one or more components, each comprising one or more data units and each data unit represented as a Huffman -encoded stream of coefficients of basic functions, and wherein a zero order coefficient is represented as a difference to the preceding zero order coefficient of the corresponding component, and a block information table which comprises: indicators for a zero-order or first-order coefficient of each image block in the bitstream, information indicating the number of bits in the bitstream between zero or first-order coefficients in adjacent data units of the image block, and the zero-order coefficient for at least one

2005-06-29 10: 56 V: \ _8o0tga »isation \ SCALAD0 NoFa» ily \ SB \ 2102073S \ 21C20736

ApplicationtextToInstructor CAS 2Ö05-Q8-11 l.doc

528 172 data unit of each component, wherein said zero order coefficient is represented in a non-differential form. The method further comprises for each data unit of at least one image block: retrieving the zero order coefficient of the data unit and Huffman decoding none or a predetermined number of coefficients of the data unit, skipping the rest of the coefficients by jumping to the next zero or first order coefficient in the bit stream using the information in the block information table for the number of bits in adjacent data units in the bit stream, whereby a reduced set of Huffman coded coefficients is decoded.

In the context of this application, the term image blocks should be interpreted to represent a portion of the space of an image, said blocks may have information from various color space components. Each image block can be represented as one or more sets of coefficients for each color space component.

According to a second aspect of the invention, the objects are achieved with an image representation format for representing a digital image. The image representation format includes: image information stored as a bit stream representing sequential image blocks, each block comprising one or more components, each comprising one or more data units and each data unit represented as a Huffman coded stream of coefficients of base functions, and wherein a zero order coefficient is represented as a difference to the previous zero order coefficient of the corresponding component; and a block information table, comprising: indicators of a zero-order or first-order coefficient of each image block in said bitstream; information indicating the number of bits in the bitstream between zero or first order coefficients in adjacent data units of the image block; and the zero order coefficient of at least one

2006-06-29 10:56 V: \ JfoOi ^ MiisationXSCAIÅPQ ΑΒ \ Ε & 7Ε8Τ \ NoFaHilly \ SE \ 2102073 »\ 21Q2C736 ApplicatioTrtrjxtToInstru.'t · '···., CAS 2005-08-11 i.doc

528 172 data unit of each component, wherein said zero order coefficient is represented in a non-differential form.

According to a third aspect of the invention, a method for encoding raw image data is provided to a compressed digital image representation. The method comprises: randomly reading image blocks of a specified size of said raw image data and for each image block: transforming the image block into one or more data units for one or more components, said transformation creating a representation of each data unit as coefficients of base functions; calculate a quantized approximation of said coefficients; represent at least some of the quantized coefficients as a stream of coefficients of sequential image blocks; Huffman encodes said stream of coefficients, whereby a zero order coefficient is represented as. a difference to the previous zero order coefficient of the corresponding component; storing said Huffman-encoded stream of coefficients in a bit stream; storing in a block information table indicators to a zero-order or first-order coefficient for each image block in the bitstream; storing in the block information table information indicating the number of bits in the bit stream between zero or first order coefficients in adjacent data units of the image block; and storing in the block information table the coefficient of zero order for at least one data unit of each component, said coefficient of zero order being represented in a non-differential form.

According to a fourth aspect of the invention, there is provided a method of analyzing a JPEG compressed digital image, wherein the JPEG compressed digital image is represented as a bit stream, said bitstream representing sequential image blocks each comprising one or more components which were and one comprises one or more data units and each

2006-06-29 10:56 V: \ _ HoOxganisation \ SCAIAPO AB \ PATBKT \ _ »oPa« iiy \ SS \ 21020736 \ 21020736 ApplicationtextToInstructor CÄS 2005-06-11 l.doc

528 172 data unit is represented as a Huffman-encoded stream of coefficients of basic functions, whereby a zero order coefficient is represented as a difference to the previous zero order coefficient of the corresponding component. The method comprises: sequentially stepping through the bitstream and during the step through the bitstream: storing an indicator in a block information table to a zero order or first order coefficient of each image block; decoding the zero order coefficients and storing in the block information table the zero order coefficient for at least one data unit of each component, said zero order coefficient being represented in a non-differential form; and storing in the block information table information indicating the number of bits in the bitstream between zero or first order coefficients in adjacent data units of the image block. The step through non-zero order coefficients of a data unit in the bitstream, which non-zero order coefficients are represented by a sequence of bitstream entries, includes: looking at a bit sequence of a predetermined number of the following bits in the bitstream; making a table lookup for determining the category and zero sequence length of at least the first bitstream record in the bit sequence and for determining the bit length of the first bitstream record; skipping the number of bits in the bit stream corresponding to the determined bit length; summing the number of skipped bits to collect information regarding the number of bits in the bitstream between zero or first order coefficients in adjacent data units; and to sum the number of coefficients that have been stepped through until all the coefficients of the data unit have been stepped through or a block end symbol is encountered.

According to a fifth aspect of the invention, there is provided a method of joining two digital images. The method comprises: determining a ratio in the space between the two digital images; to assign image blocks; C0 € -ö6-29 10: 5 <5 V: \ _ NoOxganisalioxi'.SCMAPO AEAeATBNT \ _NoFanily \ SE \ 21020736 \ 21020736 Application textToInstructor CAS 0001-08-11 i.doc

528 172 of digital image information in the two digital images index according to the ratio in the space between the two digital images; forming a bitstream representing sequential image blocks in accordance with the assigned indices, each block comprising one or more components, each comprising one or more data units and each data unit represented as a Huffman coded stream of coefficients of base functions; storing image block information for each image block in a block information table in accordance with the position of the image block, said picture block information comprising: indicators of a zero order or first order coefficient of each image block in said bitstream, information indicating the number of bits in the bitstream between zero or first order coefficients in adjacent data units of the image block, and the zero order coefficient of at least one data unit of each component, said coefficient of the zero order is represented in a non-differential form.

Due to at least some aspects of the invention, a digital image is represented in an image representation format that requires little storage space, while the digital image in the compressed representation format can still be easily processed. This is especially useful for applications with small storage space and low data processing power, such as mobile phones. The invention enables a digital image to be stored in a compressed format, but can still be processed and manipulated in real time while presented on a screen. Thanks to the image representation format and the method of processing an image according to the invention, the image can very quickly be presented on a reduced scale or with a reduced resolution. The storage of the zero order coefficients in the image blocks means that the zero order coefficient need not be calculated using the information of the previous zero order

2006- 06-29 10:56 pm: \ _NoOxganisationXSCAtaDO ÄB \ PATEaT \ _HoFaBlily \ SB \ 2102Ci736 \ 21Q20736 appllcaciontcxtrolnstruetoi · CÄS 2005-08-11 l.doc

528 172 coefficients. Furthermore, any desired number of non-zero order coefficients can be decoded. The rest of the non-zero order coefficients in the bit stream can be quickly skipped as the block information table provides information about the bit length between coefficients of adjacent data units, allowing quick access to the next data unit. This means that the image can be decoded quickly on a reduced scale, since there is no need to decode the non-zero order coefficients in the bit stream to find the start of the next data unit in the image block or the start of the next image block. Thus, the image representation format allows quick access to a digital image.

The storage of indicators of a zero order or first order coefficient of each image block provides fast access to particular portions of the images, without the need to decode the Huffman-encoded stream of coefficients from the start of the stream. Instead, the image block can be accessed directly using the indicator. Furthermore, the zero order coefficient is represented by at least one data unit of each component in a non-differential form in the block information table. Thus, the need to calculate the value of the Huffman-encoded stream of coefficients is avoided. This enables presentation and manipulation of portions of the digital image while in the compressed image representation format, as portions of the image can be accessed and analyzed at random.

In accordance with the image representation format of the invention, it is somewhat facilitated by the aim of compressing a digital image to a minimal size. Thus, the size of the digital image is not optimally compressed, but instead some additional information about the Huffman-encoded stream of coefficients is stored separately to allow for quick retrieval of certain portions of the image. In particular, it is possible to decode the image or parts of an image very quickly on another scale through

2006-06-2? '10: 56 V: \ _ NoOxqanisatlon \ SCAtAno AB \ PÄl'Ei-JT \ _KoFamily \ SE \ 21020736 \ 2102073o ApplicationtextTolnatructor CAS 2005-08-11 l.doc

528 172

Huffman decoding only a fraction of the non-zero order coefficients and using an inverse discrete cosine transform adapted to a smaller block, such as 4x4, to calculate a smaller size block. The image representation format can easily be transferred to a JPEG image, as the image representation format is very similar to the JPEG image format. Thus, it is possible to effect the transfer to a JPEG image and remove the indicators and stored values ​​of the coefficients when the image needs no further manipulation.

The Huffman-coded stream of coefficients need not include the zero-order coefficients that are stored separately in the block information table. However, the Huffman15 coded stream of coefficients may in any case include all coefficients. This may be appropriate if the image representation format is to be transmitted in another image format, since the bitstream itself can then be used directly in the other image format.

The term block information table should not be strictly interpreted as a table, but merely as the fact that information stored is stored in a controlled manner where the position of the stored information in the table is related to the position in the space in the image, which position the information represents. Thus, for example, the block information table can be divided into several lists or tables. Furthermore, Huffman-encoded stream of coefficients does not necessarily mean that the entire stream is Huffman-encoded. The current may include

Huffman codes mixed with raw data for the coefficients. For example, in a JPEG compressed file, a zero sequence length and category of coefficients Huffman is encoded and the value of a coefficient within a category is described by uncompressed bits.

By storing indicators and coefficients in a block information table, a distinct structure is obtained to store the image information.

2006-06-29 10:56 V: \ _ MoOrgani £ ation \ SCALSW AB \ EÄTEHT \ _2loFa »llv \ SE \ 21020736 \ 21020736 Appllcat.i.-ntRZtTolnstruetor CAS 2005-03-11 l.doc

528 172

For example, an image block may comprise three color space components, one luminance component and two chrominance components. Representation of the digital image in a luminance component and two color components means that the color components can be represented in a lower resolution without actually losing any information that an eye can perceive. Thus, an initial compression of the information in the digital image can be achieved relative to a representation of the digital image as three color components. For example, when using such compression, each image block may comprise four data units for the luminance component and one data unit for each color component. The data units are sequentially arranged in the Huffman-encoded stream of coefficients.

According to a sixth aspect of the invention, there is provided an image representation format for representing a digital image. The image representation format includes: image information stored as a bit stream representing sequential image blocks, each of which comprises one or more components, each comprising one or more data units and each data unit is represented as a Huffman coded stream of coefficients of base functions, the coefficient of the order is represented as a difference to the previous zero order coefficient of the corresponding component, and bitstream information, which is stored adjacent to the bitstream, which bitstream information comprises information indicating the number of bits of each data unit in the image blocks.

This image representation format requires a relatively small storage space. The image representation format contains only bitstream information which enables the rapid creation of a block information table in accordance with the image representation format of the second aspect of the invention. Therefore, this image representation format according to the sixth aspect of the invention is suitable for long-term storage of an image. When the image is downloaded can

2006-06-29 10:56 V: \ _HoOrganis.atioa \ SCAXAI »AB \ PATSNT \ JSIoFaBlily \ SB \ 21C20736 \ 21020736

ApplicationteztToInetruetox CAS 2005-08-11 l.doc

528 The 172 bitstream information is used for rapid bitstream analysis and creation of a block information table. The information indicating the number of bits of each data unit in the image blocks can be used for quick access of the data units. When the data units are retrieved, the indicators can be created, the information indicating the number of bits between zero or first order coefficients of adjacent data units can be collected, and the zero order coefficients can be decoded to store at least one zero order coefficient of each component in a non-differential form.

The bitstream information may be compressed. This means that the image representation format according to the sixth aspect can be even more effectively stored.

The indicators in the block information table may indicate the bit shift from a static position to the coefficient of the specified order. For example, the static position may be the start of the bit stream. Thus, the indicators are realized as pointers to a specific bit position in each image block. This means that each image block can be retrieved quickly, whereby access to specific portions of the Huffman-encoded stream of coefficients can be accelerated.

Alternatively, the indicators in the block information table can indicate the bit shift to the coefficient from a benchmark in the bit stream. The image representation format may then also include a list which provides information about in which image block each benchmark in the bit stream is located. In this way, the bit offset of the indicator can be stored with a smaller number of bits, since the bit offset from a benchmark in the bitstream is almost always less than the offset from a static position. This means that the indicators can be stored using less memory. Instead, a list is needed that provides information on which image block each benchmark in the bit stream is located on. Thus, when a specific image block is accessed, this is done

2006-06-29 10:56 V: \ JicOrganis-ationXSCMADO AB \ PÄTi (lT \ _H <jFa »ily \ SK \ 2102073S \ 21020736

ApplicationxextTolftstrucCor CÄS 2005-08-11 l.doc

528 172 first a check in the list so that the last benchmark in the bitstream before the specific image block is found. Then, the information about the relevant benchmark in the bitstream and the bit offset from the benchmark is used to find the desired image block in the bitstream.

The indicator can point to a zero order or first order coefficient of an image block. When the indicator points to a zero order coefficient, the start of a data unit can be easily accessed. Then the zero-order coefficient needs to be decoded to access the non-zero-order coefficients, even though the zero-order coefficient may already be stored in a non-differential format in the block information table. When the indicator points to a first-order coefficient, the non-zero-order coefficients can be directly accessed.

Further, the information indicating the number of bits in the bitstream between coefficients of adjacent data units may indicate the number of bits between any combination of zero order and first order coefficients. This information can be used for fast jump in the bit stream from a data unit to an adjacent data unit. As described above, the information of a data unit can be easily accessed either by accessing the zero order coefficient or, when the zero order coefficient is known from the block information table, by directly accessing the first order coefficient. Therefore, the information indicating the number of bits can either indicate the number of bits to the zero order coefficient or to the first order coefficient. Likewise, the image representation format is structured in such a way that it is convenient to either indicate the number of bits from the zero order coefficient or the first order coefficient.

The indicator can also point to a coefficient in any data unit in the image block. Then can

2005-06-29 10:56 V: \ _ NoOiganisationlSCMAPO _aoFaMly \ SB \ 21020? 3rd \ 2102ö? 3S toplicationtextTclnatruetc ,! cas 2005 οο n i.doc

528 The information about the number of bits between adjacent data units is used to access any data unit in the image block. Furthermore, the zero order coefficient of the data unit to which the indicator points is preferably stored in the block information table. The zero order coefficients of the other data units of the component can be calculated using the information about the difference between the zero order coefficients stored in the bit stream. It is convenient that the indicator points to the zero or first order coefficient in the first data unit of the image block. Thus, direct access to the start of the image block is given by the indicator.

The block information table may include the zero order coefficient represented in a non-differential form for each zero order coefficient represented in the bitstream as a difference to a zero order coefficient of a previous image block. This means that each image block can be accessed independently, since the block information table provides all information presented in the bitstream as dependent on the previous image block. According to a specific embodiment, the block information table includes indicators indicating the bit offset of the coefficient from a benchmark in the bitstream and only the zero-order coefficients in non-differential form for coefficients represented in the bitstream as a difference to a zero-order coefficient of a previous image block. This embodiment provides a small size block information table, which is advantageous when storage capacity is limited.

Alternatively, the block information table includes each zero order coefficient represented in a non-differential form. This means that there is no need to compute the zero order coefficient for data units whose zero order coefficient

2006-06-29 10:56 AM: \ _ HoOigaT = isation \ SCAIÄIX) AB \ PME8T \ _Ho? Ainili '\ SS \ 21020736 \ 2102ö73S

Application textToIiwlruvtoi: CA3 2005-08-11 l.doc

528 172 is represented in the bitstream as a difference to the zero order coefficient of a data unit within the image block. Thus, the information of the image block can be accessed more quickly. However, more information is required in the block information table.

Furthermore, the bit stream may represent the digital image in the JPEG format. Thus, a standard JPEG compression or an already compressed JPEG image may be associated with additional information for quick access, manipulation, and / or analysis of specific portions of the image.

According to an embodiment of the method according to the first aspect of the invention, the method further comprises presenting the decoded image blocks to a data handling or data presentation unit, whereby the image or part of the image is presented on a reduced scale. The data management or data presentation unit may be, for example, a screen, a printer or a hardware unit for performing image processing. The image or parts of the image can be quickly presented on, for example, a screen, since the method decodes the relevant parts of the digital image for display on the screen very quickly. This means that the image can be presented to a user without the user experiencing annoying waiting times.

The method may further comprise performing image processing calculations on the decoded image blocks. In this way, fewer calculations are needed because the image is decoded to a reduced set of Huffman-coded coefficients. Therefore, image processing can be performed relatively quickly.

The method may further comprise presenting results of the performed calculations on a screen when the calculations are performed. The method provides presentation of an image stored in a compressed format in real time when the image is selected and presentation of manipulations of the image in real time, since the method provides a very fast way to access relevant parts of the image and

2006-06-29 10:56 Vs \ _Noöigar.isatiOB \ SCALäDO AB \ PATBOT \ JHoFae> ily \ SB \ 21020? 36 \ 21C20736 Applicator «t« ztToInstructor CÄS 2005-08-11 l.doc

528 172 decode them into a small amount of data so that manipulation can be quickly performed and presented. The method further permits the definition and execution of manipulations of images on a device having a small memory space, such as a mobile phone.

According to one embodiment of the method according to the first aspect of the invention, the number of coefficients of the data unit that Huffman decodes is used to approximate a decoded image block corresponding to a larger number of coefficients. The approximation allows the Huffman decoded coefficients to represent more coefficients, whereby a lower quality image can be created or a smaller image having fewer pixels per image block can be created. Although the approximation loses the finest details of the image, the quality of the image may still be satisfactory, since the rough features of the image are represented by the first coefficients in each data unit.

The predetermined number of coefficients decoded by Huffman20 can be, for example, four, nine, thirteen, eighteen or twenty-four. When four and twenty-four coefficients are decoded, the decoded coefficients represent a scaling of each image block from 8x8 pixels to 2x2 pixels and 4x4 pixels, respectively. When nine, thirteen or eighteen Huffman decoders are decoded, the decoded coefficients can be used to approximate a representation of a 4 x 4 pixel image block. These numbers of decoded coefficients are particularly suitable, since the coefficients immediately following the ninth, thirteenth, and eighteenth coefficients cannot be used as information in a 4x4 pixel image block.

According to one embodiment of the image representation format of the second aspect of the invention, the indicators, the information indicating the number of bits between coefficients of adjacent data units and the zero order coefficients represented in a non-differential form in a stream of block information are stored at 10: 56 V: \ _ HoOx Organization \ SCALAM AB \ eATKHT \ _Mofaaily \ SE \ 21020736 \ 21020736 ApplicationtextToInstructor CAS 1005-06-11 i.doc

528 172 table. In an alternative embodiment, the indicators, coefficient values ​​and image information are stored in separate memory areas.

According to one embodiment of the method according to the fourth aspect of the invention, each bit sequence looked at comprises sixteen bits. This is especially suitable for the format of bitstream entries. A bit stream entry encoding the non-zero coefficients consists of two parts. The first part is Huffman-coded and encodes the zero sequence length and category for the value of the current coefficient. The second part is data representing the value of the current coefficient. The first part of the bitstream record thus contains information on how many coefficients are encoded by the record (zero sequence length + 1 coefficient) and how many bits the bitstream record consists of. Since the first part of a bit stream record is a maximum of sixteen bits long, it is advisable to look at sixteen bits. Thus, when looking at sixteen bits at a time, each bit sequence will always comprise at least the first part of a bit stream entry and the sixteen bits will therefore contain information about the number of coefficients encoded by the bit stream entry and the length of the bit stream entry.

The lookup in table may include doing a first table lookup for the first eight bits of the bit sequence. If a table lookup is done for the sixteen pieces at once, a table of 65,536 items is needed, which consumes quite a lot of memory. Furthermore, the first part of the most common bitstream entries is eight bits or shorter. Therefore, in most cases, a table lookup of the first eight bits will provide the information needed about the bitstream record.

The first table lookup can return information about the bit length of the first bit stream record and the number of coefficients passed through, or return information to a second table lookup. In this way, the first table lookup will either

2006-06-29 10:56 V: \ _ HoOrganisat: ioa \ SCAIADO AP. \ BA? EHT \ _NoF £ »rnily \ SE \ 2102ö736 \ 2102073i5 Applicatioat« äxtToInstructoi CÄS 2005-08-11 l.doc

528 172 π

return the information needed for the bitstream record or return information for analysis of the first portion of the bitstream record using the last eight bits of the bit sequence.

The lookup in the table may further comprise doing a second table lookup for the last eight bits of the bit sequence for determining the bit length of the first bit stream record and the number of coefficients being traversed. When a second table lookup is needed, the first table lookup can return a pointer to a table to be used in the second lookup depending on the first eight bits. There is only a need for a few different tables for the second table lookup, as there are only a few combinations for the first eight bits, where the first part of the bitstream record is longer than eight bits. Thus, when a table lookup is done in two steps, there is no need for as many table entries to analyze the first bit stream record of each bit sequence.

According to one embodiment of the method according to the fifth aspect of the invention, certain image blocks of one of the digital images are manipulated with information from the other digital image. The manipulation can be a mixture of the effect on the image block from the two digital images, if the content of the image block is depicted in both the digital images. This means that the two digital images can be more seamlessly joined together.

In a further embodiment, the image blocks of a portion of a first digital image are first handled, while the image blocks of the remainder of the image are temporarily stored in an uncompressed format. Then these latter image blocks can be used to calculate the manipulation of the second digital image, before the image blocks of the second digital image are handled.

The merged image can further be transferred to another image compression format, such as the JPEG format, by sequentially retrieving the image blocks of the

20Ö6-C6-29 10:56 V: \ JtoQigardsationlSCALACO AB \ PÄTBKT \ _HoFaaiily \ SE \ 23 02ö736 \ 21020736 ÄppiicationteztTomstxuctor CAS 2005-M-il l.doc

528 172, the image merged using the indicators of the image representation format and storing the stream of Huffman coded coefficients for sequential image blocks with zero order coefficients represented as a difference to the corresponding preceding zero order coefficient. Thus, the digital images can be merged and manipulated while requiring little memory capacity and when the merging is complete, the image can be moved to a representation format that is even better compressed.

Brief description of the drawings

The invention will now be described in more detail by way of example with reference to the accompanying drawings, in which:

Fig. 1 is a flow chart of a method for compressing a digital image according to an embodiment of the invention.

Figures 2a-c are schematic views of the image representation format according to embodiments of the invention.

Fig. 3 is a flow chart for a method of analyzing a data unit in the bit stream.

Fig. 4 is a flow chart of a method for reading a specific portion of a digital image stored in a compressed image file format.

Fig. 5 is a flow chart of a method of decoding a data unit for manipulating a digital image stored in a compressed image file format according to an embodiment of the invention.

Fig. 6 is a flow chart of a method of joining two digital images into a compressed image file format according to one embodiment of the invention.

Fig. 7 is a schematic overview of the joining of two digital images.

Figs. 8-9 are screenshots of a device that captures digital images and joins the digital images into a compressed image file format.

2906-06-29 10-.S6 V: \ JSoOiganisari.on \ SCV.A! X5 AB \ PÄTE8T \ _ilo? A »lly \ 3B \ 21020736 \ 2102Ö? 36 ApplicaeiontesitToInstruetoi: CÄS 20C5-08-11 l.doc

528 172

Detailed description of a preferred embodiment

In the following description, image compression will be described with reference to JPEG compression, although other compressions using other transformers may be conceivable. It should be noted that the scope of the present invention is in no way limited to JPEG compression.

Referring now to Fig. 1, a method for compressing a digital image will now be described. First, the digital image is represented in the YUV color model, step 10, with each pixel in the image having three components: luminance, Y, and two color components U and V. The two color components 0 and V represent color features whose finest details are difficult to detect. human eye.

Thus, these components may be represented at a lower resolution than the luminance component. A 16 x 16 pixel block of the image, which is an example of an image block as defined herein, can be represented by four 8 x 8 pixel Y data units, an 8 x 8 pixel U data unit and an 8 x 8 V data unit 8 pixels. This corresponds to a 50% compression of the amount of data compared to an RGB representation of the digital image. However, four CJ data units and four V data units can also be used, whereby no compression is obtained compared to the RGB25 representation.

The image is treated as discrete 16 x 16 image blocks in any order. Each data unit of each component is transformed by the discrete cosine transform (DCT), step 12. Since each data unit of each component comprises 64 pixels, the DCT transform will generate 64 coefficients for the base functions of the DCT transform. These coefficients include a zero order coefficient (DC coefficient) and 63 higher order coefficients (AC coefficients).

Then a thresholded and quantized approximation of the coefficients of each data unit is created, step 14.

2006-06-29 10:56 V: \ JJoOrga »isation \ SCÄIAPO AB \ PÄTEhT \ HoFaasily \ SB \ 21020736 \ 21020736

ApplicationtextTalnstructoi · CSS 2005-08-11 in. Doc

528 172

The thresholded and quantized approximation is achieved by scaling and cutting each coefficient by dividing it by a value according to a normalization matrix. This means that the coefficient for basic functions that have been determined to have low perceptual weight is given low weight and many coefficients are given the value 0 (zero).

Then the coefficients are reorganized into a stream of coefficients by means of a zigzag arrangement. According to standard JPEG compression, the DC coefficients are represented as the difference to the previous DC coefficient of the previous data unit by the same color component and the difference is Huffman coded. The zero coefficient AC coefficients are further coded and coded with Huffman coding and stored directly after the Huffman coded DC coefficient. When there are only ÄC coefficients that have a zero value left in the data unit, a block end code is introduced into the stream of coefficients. Thus, a bit stream of Huffman-encoded coefficients for sequential data units and image blocks is obtained. According to one embodiment of the invention, the Huffman-encoded coefficients are calculated in a similar way to standard JPEG compression, step 16. The bit stream is a compressed representation of the digital image.

However, since a zero sequence length coding and Huffman coding are used, the length of each data unit is unknown. Therefore, the start of a data unit is unknown until the bit stream has been decoded from the start of the bit stream to the start of the data unit. Furthermore, since the representation of the DC30 coefficient is dependent on the previous DC coefficient, the DC coefficient of a data unit is not known unless the previous DC coefficient has been determined by decoding the Huffman-encoded stream of previous coefficients.

To enable rapid retrieval of specific blocks of the image and thereby manipulation and / or analysis of the specific block, a block information is created2806-06-29 10:56 AM V: \ _ HoargarisationXSCALACO AP. \ S-AT2HT \ _aoFax6ily \ 3E \ 21020736 \ 21020736

ApplicationtextToInstructor CAS 2005-06-11 l.doc

528 172, Table 18, which includes an indicator for each image block, information indicating the number of bits in the bitstream between adjacent coefficients of a specified order and a DC coefficient for each color component of each image block, the DC coefficient being represented in a non-differential form.

According to other embodiments of the invention, the bit stream of Huffman encoded coefficients may have different contents or be stored in other ways. For example, since the DC10 coefficients that depend on a previous image block are already stored in the block information table, they do not need to be present in the bitstream.

Furthermore, all information about an image block is known from the block information table, which includes the DC coefficients that depend on other image blocks and an indicator for the image block in the bitstream. Thus, the bit stream need not be represented as a specific sequence of image blocks or even be stored in a stream.

Thus, according to one embodiment of the invention, the information is stored in the block information table in a manner in which the position of a block information in the table specifies what portion of the image it represents. The block information may also be stored in a manner in which the position it represents is stored together with the other block information. The Huffman-encoded stream of image block coefficients can even be stored adjacent to the block information. The stored block information may also have a mechanism for determining whether the block has been encoded, which makes it possible to treat the non-coded blocks as, for example, black blocks.

Thus, a package of the Huffman-encoded stream of coefficients and the block information table represents a digital image, which requires little storage capacity while allowing analysis and manipulation of specific portions of the image without the need to decode the entire image. This also allows for compri- 2006-06-29 10:56 V: \ _ NaOxganisatiOB \ SCALSOO AB \ PATENT! KoFan) lly \ SE \ 21020736 \ 2102073S

Application) · ».zi-ToInstructor (1A3 0005-06-11 l.doc

528 172 merging the image in a non-linear way, where the order in which the blocks are compressed is not essential.

Reference is now made to Fig. 2a, in which the structure of the compressed image representation format is presented. The block information table may be stored in the RAM of a device, while random access to read or write image blocks is needed by the device. As shown in Fig. 2b, the block information table 30 and the bitstream of Huffman-encoded coefficients 36 can be stored in separate memory compartments. The block information table 30 includes the indicator 31, information 32 indicating the number of bits in the bit stream between zero or first order coefficients in adjacent data units and the DC coefficients stored in a non-differential form 33. The indicator 31 of the block information table 30 then includes an indication of the first coefficient of a specified order of the image block in the bitstream 36.

As an alternative, shown in Figure 2c, the block information table 30 'may be stored as a header or as a special marker for the bit stream 36', which may constitute a standard JPEG format.

It will also be appreciated that when information is stored in the bitstream header, the header need not include all the information needed in the block information table.

Instead, the header may include information to allow the block information table to be created very quickly. Thus, the header of bitstream 36 'need only hold information about the lengths of each data unit. When the image is downloaded, the block information table can then be quickly created and loaded into RAM. Using the lengths of the data units, the information needed in the block information table can be accessed quickly. The data units are incremented while indicators are created to the start of each image block, the DC35 coefficients are decoded and stored and the information indicating the number of bits in the bitstream between the coefficient2006-06-29 10:56 AM V: \ _ HoOxgar, in »atioa \ SCAIA5« AB \ PATS8T \ _NoFa »Lly \ SB \ 21020736 \ 21020736 ApplicationtextTolnstruclca CAS 2005-08-11 l.doe

528 172 ut of zero or first order in adjacent data units are updated.

This head can even be compressed, such as Huffman-encoded, to save storage space. The length information has values ​​that vary only slightly. Therefore, in a manner corresponding to the coding of DC coefficients, Huffman encoding the lengths would substantially compress the information. The essential compression of the file means that the file can be stored in a space efficient format while allowing the block information table to be created quickly. The head is then decoded when the image is downloaded.

However, it should be noted that the invention is in no way limited to these presented ways of representing the image.

The indicator 31 may point to the coefficient of the first zero order of the image block. However, since the zero order coefficient is provided in the block information table, there is no need to access the zero order coefficient. Thus, the indicator 31 may alternatively point to the coefficient of the first first order of the image block. The indicator can be implemented as a bit shift from a static position in the bit stream to the coefficient of a specified order in the image block. Preferably, the indicator provides information about the bit offset to the coefficient from the start of the bitstream.

Alternatively, the indicator 31 may indicate the bit shift to the coefficient from a benchmark in the bit stream. If indicator 31 indicates the bit offset from the start of the bitstream, a bitstream of a size greater than 2 Mbytes would require indicators represented by 4 bytes. The benchmarks in the bitstream can be placed in the bitstream so that an indicator 31 can be represented by 2 bytes. When the offset becomes greater than, for example, 65,536 bits (the largest number represented by 2 bytes), a benchmark is registered in the bit stream.

2306-06-29 10:56 V: \ _ RoOxganisation \ SCALAPO AB \ PÄTEHT \ HcFattlly \ SB \ 21020736 \ 21020736 ApplicationtextToInstructor CÄS 2005-08-11 l.dcc

528 172

The benchmarks in the bit stream are sequentially numbered. A list of benchmarks in the bitstream is created to provide information about which image block each benchmark in the bitstream is located in. When an image block is accessed using the indicator, two steps are accessed. First, a comparison is made with the list of benchmarks in the bitstream so that the number of the benchmark in the bitstream located closest to the bitstream before the image block to be accessed is found. Then, from the start of the bit stream, the bit offset of the image block can be calculated as (the number of the benchmark in the bitstream) * 65,536 + the bit offset provided by the indicator. Of course, any number of bits could be used between the benchmarks in the bit stream.

The information indicating the number of bits between adjacent coefficients of the specified order could be used for quick access of specific data units within an image block. There is no need to decode the previous data units to know where a data unit starts. This could advantageously be used for fast decoding of image blocks into a reduced set of decoded coefficients, as described in more detail below.

The block information table may include the DC coefficient represented in a non-differential form for each data unit. This means that there is no need to calculate the DC coefficient for any data unit. However, the block information table may alternatively comprise only the DC coefficients represented in the bitstream as a difference to a DC coefficient of a preceding image block. Since an image block comprises multiple data units for each component, there is only a need to store the DC coefficient of the first data unit of each component. This means that the block information table requires less storage space.

The structure of the compressed image representation format can also be obtained by starting from one

2006-06-29 10:56 Vi \ _SoOrganisation \ SCALADO AB \ PATEOT \ _SoFa »ily \ SB \ 2102C

ApplicationTolstructor CAS 2005-03-11 l.doc

528 172 already compressed JPEG file. The JPEG file is then decoded to determine the indicators for each image block, information indicating the number of bits between adjacent coefficients of specified order and a DC5 coefficient in a non-differential form for each component of each image block. Thus, the indexes and DC coefficients are stored in the block information table, while the Huffman-encoded stream of coefficients is kept intact. Alternatively, the AC coefficients of such a JPEG image may be copied to a new memory space, whereby all information of the JPEG image will be accessible without the need to retain the original JPEG image stored in the device.

The image representation format created in this way opens opportunities to perform image processing directly on a JPEG file.

A specific method of analyzing a JPEG image to create the image representation format will now be described. The method involves sequentially stepping through the bit stream to collect the information needed. During the throughput of the bitstream, the indicators for each image block are stored, the information indicating the number of bits in the bitstream between adjacent coefficients of specified order, and the DC25 coefficients of at least the first data unit of each component of the block information table.

During the throughput of the bit stream, the DC coefficients of the components are decoded and the last coefficient is temporarily stored to allow determination of the next DC coefficient.

Referring to Fig. 3, step through AC coefficients of a data unit in the bit stream will be described. The analysis only wants to determine where the next data unit starts and the number of bits between the DC coefficients or first AC coefficients of adjacent data units. Therefore, it is not necessary to decode the AC coefficients.

2006-06-29 10:56 V: \ _ HoOxganisatioa \ SCALÄEO AB \ 9 & TEOT \ _HoFamiiy \ SE \ 21O2O730 \ 21O20736

ApplicationToIustructor CAS 2095 * 38-11 l.doc

528 172

A bit stream entry of the AC coefficients consists of two parts. The first part is a Huffman code that codes how many zero coefficients precede the current coefficient and in which category the value of the current coefficient is. The category determines the number of bits for the second part of the bitstream record, which encodes the current value of the coefficient. Thus, by analyzing the first part of the bitstream record, the number of coefficients encoded by the bitstream record and the number of bits used by the bitstream record can be determined.

The step through the AC coefficients involves looking at a bit sequence of a predetermined number of bits in the bit stream, step 20. Preferably, the bit sequence 16 is the number of bits long, corresponding to the longest first portion of any bit stream entry. Thus, by looking at 16 bits at a time, the required information can always be collected for at least one bit stream entry. The information is collected by doing a table lookup, step 22. The table lookup thus returns the bit length of at least the first bit stream record contained in the bit sequence and the number of coefficients encoded by the Huffman code. A number of bits corresponding to the determined bit length is then skipped, step 24. The accumulated number of skipped bits is summed, step 26, and the accumulated number of skipped coefficients is summed, step 28. If a block end symbol has not been found and the maximum number of coefficients of a data unit has not been skipped, the process is returned to step 20 and a new bit sequence is looked at.

The table lookup in step 22 can be performed in two steps. First, a table lookup is done for the first eight bits of the bit sequence. This lookup is sufficient for determining the first part of the Huffman code for most lookups for normal JPEG images. If the first table lookup is sufficient to determine the first part of the first Huffman code, the last eight bits are not further analyzed at this stage. If

2006-06-29 10:56 V: \ _ 8oOrganisation \ SCALAPC AB \ PÄTBHT \ JJsFamlly \ S2 \ 21Q20736 \ 21C20736 ÄpplxcationteztToInstxucCcz C&S 2005-08-11 l.doc

528 172 the first part of the Huffman code is longer than eight bits a further table lookup is needed. The first table lookup then returns a pointer to a new table, where a lookup of the last eight bits is to be done. The pointer is dependent on the first eight bits of the bit sequence. The second table lookup will then determine the number of bits for the Huffman code and the number of coefficients to be skipped. The second table lookup can be done in different tables or in different parts of the same table. There are only a few variants of the first eight bits of Huffman codes that are longer than eight bits. Therefore, the number of entries in the second table is limited and much less than 65,536 (ie, the total number of possible bit sequences of sixteen bits).

Alternatively, the table lookup in step 22 is performed in one step. Then a table lookup is performed for a 16 bit code. This table lookup may in some cases return information about two or more Huffman codes. Thus, where the bit sequence comprises the first portion of several Huffman20 codes, these Huffman codes can be skipped simultaneously. It is also possible to skip multiple Huffman codes simultaneously when a lookup is done for eight bits. However, only a few Huffman codes are short enough to provide the required information in eight bits.

Referring now to Fig. 4, a method for reading a specific portion of a digital image stored in the image representation format will now be described.

First, the position or area of ​​interest in the image is defined, step 40. Then the correct image block or blocks corresponding to the defined position or area is determined by simply moving the right or down from the upper left corner to the sequence of the image blocks, step 42 .

Then, the position of the correct image block or blocks in the bitstream is determined by looking up the block information table, step 44. Further, DC2006-06-29 Ϊ 0: 56 V: \ _NoOrgarjisation \ SCAIADO AB \ PÄTBNT \ _NoPamily \ SB \ 21020736 \

Application textToIiistructor CAS 2025-08-11 l.doc

528 172 the coefficient of the correct image block or blocks from the block information table, step 46. Then access to the position in the bit stream, step 48, and the image block is decoded at this position, step 50, using the retrieved DC coefficient.

With reference to Fig. 5, a method for processing and manipulating a digital image stored in the image representation format will now be described. The processing and manipulation of the digital image is so fast that it can be performed and displayed on a mobile phone in real time to a user. Thus, the user can define the manipulations to be performed and see them performed within a few seconds.

The user can first define an image or part of an image to be displayed. To allow the image to be decoded quickly for display on a screen, only a reduced amount of Huffman-coded coefficients for each data unit is decoded. The reduced amount of coefficients can be used to approximate the image with decreased resolution or to display an image with fewer pixels. The number of AC coefficients decoded may conveniently be zero, reducing an image block of 8 x 8 pixels to 1 pixel, four, reducing the image block to 2x2 pixels or twenty-four, reducing the image block to

4x4 pixels. When the image block is reduced to 4x4 pixels, a smaller number of AC coefficients can be decoded to approximate the 4x4 pixel image block. Thus, for example, nine, thirteen or eighteen coefficients can be decoded. Since the main information of each image block is placed in the first coefficients, the information lost in the non-decoded coefficients is not very important.

Thanks to the image representation format, the reduced amount of Huffman-encoded coefficients can be retrieved and decoded very quickly to display the image or portion of the image. First, the image blocks to be decoded are determined, step 60. Access to each image block is determined by

2006-05-29 tC: 56 V: \ _ HaOiganisatioa \ SCAIATO ABXEATBtm .Ho? Aw.tly \ SB \ 21fi20736 \ 21C2ö73ö AppllcntiOTitextTolnstructox CAS 2035, -08-11 l.ctoc

528 172 using the indicators in the block information table, step 62. The DC coefficient of the first data unit of each component is also provided by the block information table, step 64. Then, the desired number is decoded

AC coefficients, step 66. Then the next data unit of the image block is quickly retrieved by skipping the rest of the AC coefficients using the information indicating the number of bits between, for example, the first AC coefficients in adjacent data units, step 68. Now the next data unit can be decoded . In this way, the image is decoded very quickly for presentation on a screen, which reduces annoying wait times for a user. This is particularly useful when a device with low data processing power and small storage space is used, such as a mobile phone.

The user can then define a manipulation to be performed on the image presented on the screen. The calculations needed for the manipulation can now be performed on the reduced amount of Huffman decoded coefficients. Thus, the calculations can be performed faster and the results can be displayed on a real-time display, without the user experiencing long waiting times.

Referring now to Figures 6-9 , a method for joining two digital images into a compressed image file format will now be described. Figure 6 shows a flow chart showing an overview of the method. Fig. 7 shows a schematic overview of the joining of images, while Figs. 8-9 show various steps in the method when implemented on a device for taking pictures, such as a mobile phone with a built-in camera.

First, a first digital image 200 is recorded, step 100.

The contents of the digital image are displayed in the viewfinder of a camera in Figure 8. The recorded image is then transformed to take into account lens corrections and a projection of the corrected image onto a cylinder is created.

Thus, the recorded image is adapted to be joined with another image to create a panoramic image. A section B of the digital image 200 is compressed

2006-06-29 10:56 V: \ _ No0iganisatiOR \ SCAlAP0 AB \ PATENT \ _MoFai »ily \ SB \ 21020736 \ 21020736 MpplicatlontextToInstructoi CAS 2005-08-11 l.doc

528 172 then to the compressed image representation format, as described with reference to Fig. 1, step 102, whereby the image blocks are assigned indexes for positions in the image according to their current position in the space of the image. The second section A of the image is stored, step 104, with the intention that a mixing operation is to be performed later. If the user wishes to create a panoramic image, the direction of panning can be defined in the image capture device. A section of the recorded first digital image 200 closest to the panning direction is then stored as the section stored with the intention of mixing.

This section A of the first image can be presented in the viewfinder of the camera, step 106, when a second image is to be recorded. The section of the first digital image can be presented in a first layer of the viewfinder, with each other pixel being transparent so that the object space viewed by the camera and presented in a second layer of the viewfinder can be perceived behind the section of the first digital image, as shown in Figure 9. . If a panorama is made in a right-hand direction, a section on the far right of the first digital image will be presented in it. part that is farthest to the left of the viewfinder. If the camera itself performs the lens correction and the cylindrical projection, as described above, in real time, the stored section A of the first image can simply be presented in the viewfinder. However, if no real-time correction is performed by the camera, section A is inversely transformed as it would be positioned in the far left position of the viewfinder to better fit the object space displayed in the viewfinder. Thus, the user is directed to occupy a second image 202, step 108, which positions corresponding objects in the first and second digital images in an overlapping area of ​​the viewfinder. The recorded image is then transformed to take into account lens corrections and a projection of the corrected image onto a cylinder is made.

2006-06-29 10:56 V: \ BoOrger.isationXSCALADO AB \ PÄTEHT \ _So? A »ily \ 3B \ 21020736 \ 21C2ö? 3S ApplicationteztToInstruct & i · OAS 2005-06-11 l.doc

528 172

The second digital image now comprises a section C which substantially corresponds to section A of the first digital image. Thus, the camera can easily correlate the two digital images to each other, step 110, so that the two images can be properly merged with each other. Of course, the correlation between the two digital images can be obtained in any other way. For example, a computer unit can calculate and find the correlation between the two images or the correlation can be defined by a user, for example when there are no overlapping areas. The correlation determines the displacement of the images in relation to each other.

Then, the section A stored for mixing into the second image is mixed according to the correlation, step 112. Then a section D of the second image is compressed with correct indexes to the image blocks, according to the determined offset, into the compressed image representation format as described with reference to Fig. 1, step 114. Further, another section E of the second image is stored in an uncompressed format, step 116, with the intention of being used in a blending operation to be performed if additional images are to be taken to the panorama. As shown in Fig. 7, section E may comprise portions containing no image information, due to the displacement of the two recorded images relative to each other. The storage of image section E thus has a mechanism for determining whether a pixel stored represents image data or unknown information due to the offset. This information is later used by the mixture and can also be used in the correlation operation.

Then, if desired, steps 106-116 are repeated to merge additional images to the already merged images, step 118. The last inserted image is compressed in its entirety to the image representation format as described with reference to Fig. 1.

2006-06-29 10:56 V: \ _ NoOrganisarion \ SCALÄDC Ä3 \ PÄTBH? \ _ NoF8eily \ SE \ 2l020736 \ 21020736 AppilcafciATitextTalnstructOl · 'CAS 2005-08-11 l.doc

528 172

Then, the largest rectangular image that is possible to form from the two or more digital images is determined and the first image block that completely fits into the rectangle in the upper left corner is determined. Then, the image representation format is converted to a JPEG image file of the merged images, step 120, by introducing the Huffman-encoded stream of coefficients of the sequential blocks, starting from the determined block and moving from left to right and from top to bottom through the determined largest rectangular image. In this way, a large, merged image will be represented in a JPEG image file format. The merged digital images can now be displayed on a screen of the mobile phone which includes the camera.

Although the method of joining has been described as a sequence in which a first digital image is first compressed into a compressed image file format and the second digital image is later added to this compressed image representation format, it is conceivable that the compressed image representation format can be created by directly incorporating the two digital images into the compressed image representation format or alternatively, the two digital images can each be represented in a compressed image file format and combined into a large image via a representation in the compressed image representation format .

In contrast to merging images, a method for cropping a JPEG baseline coded image will now be described. First, the JPEG image is analyzed by decoding Huffman-encoded data. During this decoding, indicators are stored for each image block and information indicating the number of bits between data units in the block information table. The DC coefficient of the first data unit of each color component of each image block is further calculated and stored in the block information table.

Now the cropping of a JPEG image is accomplished into a new JPEG baseline coded representation of a portion of the image

2006-06-29 10:56 am_s\ NoOiganisationXSCAXADC AB \ PATENT! JWoFattilv \ 8B \ 21ö2G736 \ 2102073ο ApplicationeztT & InstructOi CAS 2005-08-11 i.doc

528 172 by first determining an area to be retained from the pruning. For the image block at the far left of each row of the region, a new difference for the DC coefficients needs to be calculated using the information about the DC coefficient in the block information table. Then, the representation of the rest of the line can simply be copied bit by bit from the Huffman-encoded data of the original JPEG image to the new JPEG image using the indexes in the block information table to determine the bit length to be copied.

Thus, a new, cropped JPEG image can be created very easily.

If the cropping of the JPEG image is to be performed to an uncompressed representation of the image, instead, the image blocks in the area are determined and decoded to the correct position in the uncompressed representation of the image using the indexes in the block information table. The decoding is performed using the DC coefficients in the block information table, whereby all Huffman-encoded data does not need to be decoded at this time.

Furthermore, the JPEG image can also be manipulated by processing the image, while the image information is represented as the DCT coefficients. By converting the JPEG image to the image representation format presented herein, manipulations can be performed on the image blocks while being represented as DCT coefficients. Thus, by means of matrix operations, the image can be rotated or scaled, for example.

It should be emphasized that the preferred embodiments described herein are in no way limiting and that many alternative embodiments are possible within the scope of protection defined by the appended claims.

20Ö6-06-2O 10:56 V: \ Organizing organization SCIA & O AB \ eOT2 »TVr ?? a®ily \ SB \ 2102 <J? 36V2102Q736 Applicati-MextTolr-structor C&S 2005-08-11 l.doc

528 172