EP0786909A2

Locating a frame position in an MPEG data stream

Abstract

A method for locating a frame position in an MPEG data stream within a computer system is disclosed. MPEG standard is a set of defined algorithms and techniques for the compression and decompression of moving pictures (video) and sound (audio), and the formation of a multiplexed data stream that includes the compressed video and audio data plus any associated ancillary service data. Although the MPEG standard is extremely flexible, there is a fundamental deficiency associated with the packet-oriented nature of the MPEG format, and that is there being no information about the position of each video frame encoded in the data stream. Even though such information can be deduced from the byte-rate, but because the calculation of a frame position depends on a constant byte-rate, a problem may still arise when the byte-rate is non-existent, incorrectly encoded, or constantly changed due to the presence of several packs with varying rates. This invention presents an improved method for locating a frame position in the MPEG data streams regardless of the byte-rate condition.

EP0786909A2, drawing sheet 1 — Sheet 1 of 9

Term

Term ended

Projected expiry passed 14 January 2017, 9.7 years ago.

12 claims: 3 independent, 9 dependent

1
A method for efficiently locating a frame in an MPEG data stream within a computer system, said method comprising the steps of:determining (80) a frame number for a desired frame;determining (82) a byte-rate of said MPEG data stream;determining (84) an estimate byte position for said desired frame from said byte-rate of said MPEG data stream;identifying (86) a presentation time stamp which is nearest to said estimate byte position for said desired frame;determining (88) a time position of said desired frame;determining (90) whether said presentation time stamp is within a specified proximity to said time position of said desired frame;in response to a determination that said presentation time stamp is not within said specified proximity to said time position of said desired frame, returning to said determining (82) a byte-rate step;and in response to a determination that said presentation time stamp is within said specified proximity of said time position of said desired frame, searching (92) for said desired frame within said specified proximity.
2
A method as claimed in Claim 1, wherein said determining (82) a byte-rate step further comprises the step of:dividing the product of a clock frequency and the difference between a byte position of a last system clock reference from a current frame and a byte position of a next system clock reference from said current frame by the product of a frame rate and the difference between a value of said last system clock reference from said current frame and a value of said next system clock reference from said current frame.
3
A method as claimed in Claim 1, wherein said determining (84) an estimate byte position step further comprises the step of:adding a byte position of a current frame to the product of said byte-rate and the difference between a frame number for said desired frame and a frame number for said current frame.
4
A method as claimed Claim 1, wherein said determining a time position step (88) further comprises the step of:dividing a frame number for said desired frame by a frame rate.
5
A method as claimed in Claim 1, wherein said determining (90) whether said presentation time stamp is within a specified proximity step further comprises the step of:determining whether said presentation time stamp is within two second of said time position of said desired frame.
6
A method as claimed in Claim 1, said method further comprises the steps of:searching (94) back for an I-frame;and decoding (96) all frames subsequent to said I-frame until said desired frame is reached.
7
A computer system for efficiently locating a frame position in an MPEG data stream, said computer system comprising:means for determining a frame number for a desired frame;means for determining a byte-rate of said MPEG data stream;means for determining an estimate byte position for said desired frame from said byte-rate of said MPEG data stream;means for identifying a presentation time stamp which is nearest to said estimate byte position for said desired frame;means for determining a time position of said desired frame;means for determining whether said presentation time stamp is within a specified proximity to said time position of said desired frame;means for determining another byte-rate, in response to a determination that said presentation time stamp is not within said specified proximity to said time position of said desired frame;and means for searching said desired frame within said specified proximity, in response to a determination that said presentation time stamp is within said specified proximity of said time position of said desired frame.
8
A system as claimed in Claim 7, wherein said means for determining a byte-rate further comprises:means for dividing the product of a clock frequency and the difference between a byte position of a last system clock reference from a current frame and a byte position of a next system clock reference from said current frame by the product of a frame rate and the difference between a value of said last system clock reference from said current frame and a value of said next system clock reference from said current frame.
9
A system as claimed in Claim 7, wherein said means for determining an estimate byte position further comprises:means for adding a byte position of a current frame to the product of said byte-rate and the difference between a frame number for said desired frame and a frame number for said current frame.
10
A system as claimed in Claim 7, wherein said means for determining a time position further comprises:means for dividing a frame number for said desired frame by a frame rate.
11
A system as claimed in Claim 7, wherein said means for determining whether said presentation time stamp is within a specified proximity further comprises:means for determining whether said presentation time stamp is within two second of said time position of said desired frame.
12
A system as claimed in Claim 7, wherein said system further comprises:means for searching back for an I-frame;and means decoding all frames subsequent to said I-frame until said desired frame is reached.

Independent claims12

25 paragraphs in 3 sections, as filed

Technical Field

0001The present invention relates to a method and system for data processing in general, and in particular to a method for processing MPEG data streams within a computer system. Still more particularly, the present invention relates to a method for locating a frame position in an MPEG data stream within a computer system.

Description of the Prior Art

0002In 1988, the Moving Picture Experts Group (MPEG) was established under the framework of the Joint ISO/IEC Technical Committee (JTC 1) on Information Technology with the mandate to develop standards for coded representation of moving pictures (video), associated sound (audio), and their combination when utilized for storage and retrieval on digital storage media (DSM). In April 1990, it became working Group 11 (WG 11) of JTC 1/SC 2, and in November 1991 WG 11 of JTC 1/SC 29. The original work items of MPEG consist of coding up to 1.5, 10, and 40 Mb/s, which were respectively nicknamed as MPEG-1, -2, and -3.

0003In broad terms, the MPEG standard addresses the compression/decompression of video and audio data, and the formation of a multiplexed common data stream that includes the compressed video and audio data plus any associated ancillary service data. The MPEG standard also addresses the synchronization of video, audio and ancillary data during playback of the decompressed signals. It is intended to cater to a wide variety of applications and services, including but not limited to television broadcasting and communications.

0004Specifically, the MPEG standard is a set of algorithms for compression and systemization, and techniques with well-defined rules and guidelines. The rules and guidelines allow variation in the values assigned to many of the parameters, providing for a broad range of products and interoperability. The MPEG data stream comprises two layers-system layer and compression layer. The system layer contains timing and other information needed to de-multiplex the audio, video and ancillary data streams, and to provide synchronization during playback. The compression layer includes the compressed audio and video data streams.

0005Although the MPEG standard is extremely flexible, there is a fundamental deficiency associated with the packet-oriented nature of the MPEG format, and that is no information about the position (or byte offset) of each video frame being encoded in the data stream. Even though such information can be deduced from the byte-rate of the data stream, but because the calculation of a frame position depends on a constant byte-rate, a problem may still arise when the byte-rate is non-existent, incorrectly encoded, or constantly changed due to the presence of several packs having varying rates. Certainly, the frame position can also be obtained by parsing the data stream. This, however, could be quite time consuming because the parsing must be perform from the beginning of the data stream every time when such information is needed.

0006Consequently, it would be desirable to provide a method for efficiently locating a frame position in MPEG data streams within a computer system.

DISCLOSURE OF THE INVENTION

0007In accordance with the method and system of the present invention, a frame number for a desired frame in an MPEG data stream is first obtained from a user or some other input. A byte-rate of the MPEG data stream and a byte position of a selected frame are then determined analytically. Next, a presentation time stamp that is nearest to the determined byte position of the selected frame is compared with a time position of the desired frame. If the presentation time stamp is not within a specified proximity to the time position of the desired frame, the process returns to determine another byte-rate for the data stream. However, if the presentation time stamp is within the specified proximity of the time position of the desired frame, the desired frame is located by searching a presentation time stamp that is closest to the time position of the desired frame.

BRIEF DESCRIPTION OF THE DRAWINGS

0008The invention will now be described, by way of example only, with reference to the accompanying drawings, in which: <ul id="ul0001" list-style="none"><li>Figure 1A is a pictorial diagram of a personal computer utilized by a preferred embodiment of the invention;</li><li>Figure 1B is a block diagram of the components for the personal computer depicted in Figure 1A;</li><li>Figure 2 is a block diagram of a typical MPEG decoder utilized by a preferred embodiment of the invention;</li><li>Figure 3 is a block diagram of the ISO/IEC 11172 data-stream structure; and</li><li>Figure 4 is a high-level logic flow diagram of a method for locating a frame position in an MPEG data stream within a computer system, according to a preferred embodiment of the invention.</li></ul>

DETAILED DESCRIPTION OF THE INVENTION

0009The present invention may be executed in a variety of computers under a number of different operating systems. The computer may be, for example, a personal computer, a mini-computer or a mainframe computer. In addition, the computer may be a stand-alone system or part of a network such as a local area network (LAN) or a wide area network (WAN). For the purpose of illustration, a preferred embodiment of the present invention, as described below, is implemented on a personal computer, such as the Aptiva series manufactured by International Business Machines Corporation. (Aptiva is a trademark of IBM Corporation).

0010Referring now to the drawings and in particular to Figure 1A, there is depicted a diagram of personal computer 10 which may be utilized by a preferred embodiment of the present invention. Personal computer 10 comprises processor unit 11, keyboard 12, mouse 13 and video display (or monitor) 14. Keyboard 12 and mouse 13 constitute user input devices, and video display 14 constitutes an output device. Mouse 13 is utilized to control cursor 15 displayed on screen 16 of video display 14. Personal computer 10 supports a Graphic User Interface (GUI) which allows a user to "point-and-shoot" by moving cursor 15 to an icon or specific location on screen 16 via mouse 13 and then press one of the buttons on mouse 13 to perform a user command.

0011Referring now to Figure 1B, there is illustrated a block diagram of the components for personal computer 10 in Figure 1A. Processor unit 11 includes system bus 21 to which various components are attached and by which communications among various components are accomplished. Microprocessor 22, connecting to system bus 21, is supported by read only memory (ROM) 23 and random access memory (RAM) 24, both of which are also connected to system bus 21. Microprocessor 22 in the International Business Machines Corporation's Aptiva computer series is one of the Intel family of microprocessors; however, other microprocessors manufactured by Hewlett Packard, Inc., Sun Microsystems, Motorola, Inc., and others may be applicable also. (Intel is a trademark of Intel Corp.)

0012ROM 23 contains, among other codes, the Basic Input/Output System (BIOS) which controls certain basic hardware operations, such as interactions of hard disk drive 26 and floppy disk drive 27. RAM 24 is the main memory within which the operating system having the present invention incorporated and other application programs are loaded. A memory management device 25 is connected to system bus 21 for controlling all Direct Memory Access (DMA) operations such as paging data between RAM 24 and hard disk drive 26 or floppy disk drive 27.

0013As shown in Figures 1A and 1B, a CD ROM drive 18 having a compact disk 19 inserted inside is installed within processor unit 11. In addition, an MPEG decoder 31 is also installed within processor unit 11 for decoding MPEG data stored in a digital storage medium (DSM) such as compact disk 19. Further, a modem 17 may be utilized to communicate with other data processing systems 270 across communications line 260.

0014To complete the description of processor unit 11, there are three input/output (I/O) controllers, namely, keyboard controller 28, mouse controller 29 and video controller 30, all of which are connected to system bus 21. As their names imply, keyboard controller 28 provides the hardware interface for keyboard 12, mouse controller 29 provides the hardware interface for mouse 13, and video controller 30 provides the hardware interface for video display 14.

0015Referring now to Figure 2, there is illustrated a block diagram of a typical MPEG decoder 31. MPEG decoder 31 comprises a medium specific decoder 40, a system decoder 42, an audio decoder 46 and a video decoder 48. System decoder 42, audio decoder 46 and video decoder 48 are all synchronized by clock 44. MPEG data from a digital storage medium is input to MPEG decoder 31 via medium specific decoder 40 which converts the MPEG data to an MPEG data stream specifically for system decoder 42. System decoder 42 then demultiplexes the MPEG data stream into two elementary streams for separately feeding to audio decoder 46 and video decoder 48, whose outputs are decoded audio signals and decoded video signals respectively.

0016Referring now to Figure 3, there is illustrated a block diagram of the ISO/IEC 11172 data-stream structure. The ISO 11172 data stream (or MPEG data stream) comprises a series of random access sub-streams 50, each having a pack start code 51 and a pack 52, and concludes with a pack end code 59. within each pack 52, there are system clock reference (SCR) 62, system header 64, and a variable number of packets 66a, 66n. Pack start code 51 is 32 bits in length, and it contains information for identifying each pack 52. SCR 62 is a snapshot of a system clock from an encoder (not shown). Because any SCR utilized by the audio, video and associated ancillary data in a decoder must have approximately the same value, SCR 62 is inserted into the MPEG data stream on a cyclic basis by the encoder and are extracted by system decoder 42. In fact, four bytes of each eight-byte SCR 62 contain a time stamp encoded by the encoder during the encoding stage intended for such synchronization purpose.

0017Typically, system header 64 only appears in the first pack of any MPEG data-stream, and it is a data structure that carries information summarizing the system characteristics of the multiplexed MPEG data stream. System header 64 contains information such as an audio bound and a video bound; however, for the purpose of the present invention, the most important element is the frame rate (not shown) stored within the video bound of system header 64. The frame rate is constant throughout for each MPEG data stream. Finally, all data, both audio and video, are stored within packet data 80 of packets 66a, 66n. In general, several packets 66a, 66n constitutes one frame.

0018In addition to packet data 80, each of packets 66a, 66n also includes a packet start code prefix 72, a stream identification 74, a packet length 76, and some other header data 78. Certain buffer information 92 and time stamps 94 are stored within the field of other header data 78 in each of packets 66a, 66n. Time stamps 94 can be either a presentation time-stamp (PTS) only or a PTS and a decoding time-stamp (DTS) both. The PTS indicates the time that a presentation unit is presented in system decoder 42 while the DTS indicates the time that an access unit is decoded in system decoder 42. A presentation unit is a decoded video picture, ancillary data, or audio time sequence, and PTSs are samples of the encoder system clock that are associated with some audio, video, or associated ancillary data presentation units. Thus, PTS represents the time at which the video picture is to be displayed, the starting playback time for the audio time sequence, or the time that the associated ancillary data must be achieved. PTSs are also inserted into the data stream on a cyclic basis by the encoder and are then extracted by system decoder 42. The ISO 11172 data-stream structure is described in more detail in ISO/IEC 11172-1, 1993.

0019Referring now to Figure 4, there is illustrated a high-level logic flow diagram of the method for locating a frame position in MPEG data streams within a computer system, according to a preferred embodiment of the invention. Starting at block 80, a frame number for a desired frame position, NewFramePos, may be determined from an input by a user or some other means. Then, at block 82, the number of bytes per frame, ByteRate, is determined by the following equation:<maths id="math0001" num=""><math display="block"><mrow><mtext mathvariant="italic">ByteRate =</mtext><mtext></mtext><mfrac><mrow><msub><mrow><mtext mathvariant="italic">ByteOffset</mtext></mrow><mrow><mtext mathvariant="italic">SCR2</mtext></mrow></msub><msub><mrow><mtext mathvariant="italic">-ByteOffset</mtext></mrow><mrow><mtext mathvariant="italic">SCR1</mtext></mrow></msub></mrow><mrow><mtext mathvariant="italic">SCR2-SCR1</mtext></mrow></mfrac><mtext mathvariant="italic">x</mtext><mfrac><mrow><mtext mathvariant="italic">ClockFreq</mtext></mrow><mrow><mtext mathvariant="italic">FrameRate</mtext></mrow></mfrac></mrow></math><img file="EP0786909A2_D0001.tif" /></maths>where:<dl id="dl0001" compact="compact"><dt>ByteOffsetSCR1 =</dt><dd>byte position of the first byte of last SCR from current byte position, w.r.t. to the beginning of the data stream</dd><dt>ByteOffsetSCR2 =</dt><dd>byte position of the first byte of next SCR from current byte position, w.r.t. to the beginning of the data stream</dd><dt>SCR1 =</dt><dd>last SCR value from the current byte position</dd><dt>SCR2 =</dt><dd>next SCR value from the current byte position</dd><dt>ClockFreq =</dt><dd>90 KHz</dd><dt>FrameRate =</dt><dd>frames per second</dd></dl> Typically, SCR 62 in each pack of data stream 50 is utilized to determine the byte arrival schedule at decoder 31 in order to accommodate the correct buffering in decoder 31. Because the values of SCR 62 are calculated based on the actual byte-rate during encoding of data stream 50, two consecutive SCR 62 values, SCR1 and SCR2, may be utilized to calculate the number of bytes per frame at any location within data stream 50. SCR1 is the value of a previous SCR from the current position in bytes, while SCR2 is the value of a next SCR from the current position in bytes. Further, ByteOffsetSCR1 is the byte position of a first byte of the previous SCR from the current position in bytes, while ByteOffsetSCR2 is the byte position of a first byte of the next SCR from the current position in bytes. As a preferred embodiment of the invention, ByteOffsetSCR1 may be obtained from a counter that keeps track of the byte count since the beginning of data stream 50, while ByteOffsetSCR2 may be obtained by moving forward from the current position to the next SCR. Clock frequency, ClockFreq, also from the encoder, is typically at 90 KHz as defined by the MPEG standard. The frame rate, FrameRate, may be extracted from the 5-bit video bound of system header 64.

0020At block 84, an estimate byte position for the desired frame, EstiBytePos, is determined by multiplying the difference between the frame number at the desired frame position, NewFramePos, and the frame number at the current position, CurrFramePos, to the number of bytes per frame, ByteRate (from block 82), and the product is then added to the byte position for the current frame, CurrBytePos. The steps described above can be expressed in equation form as follows:<maths id="math0002" num=""><math display="block"><mrow><mtext mathvariant="italic">EstiBytePos=(NewFramePos-CurrFramePos)xByteRate+CurrBytePos</mtext></mrow></math><img file="EP0786909A2_D0002.tif" /></maths>where:<dl id="dl0002" compact="compact"><dt>NewFramePos =</dt><dd>frame number of desired frame position</dd><dt>CurrFramePos =</dt><dd>frame number of current frame position</dd><dt>ByteRate =</dt><dd>number of bytes per frame</dd><dt>CurrBytePos =</dt><dd>current byte position</dd></dl>

0021Next, at block 86, a search is made by parsing forward and backward for the PTS nearest to the estimate byte position for the desired frame, EstiBytePos, found at block 84. Then, at block 88, the time position of the desired frame, NewTimePos, is determined by dividing the frame number of the desired frame, NewFramePos, by the frame rate, FrameRate, as follows:<maths id="math0003" num=""><math display="block"><mrow><mtext mathvariant="italic">NewTimePos=</mtext><mfrac><mrow><mtext mathvariant="italic">NewFramePos</mtext></mrow><mrow><mtext mathvariant="italic">FrameRate</mtext></mrow></mfrac></mrow></math><img file="EP0786909A2_D0003.tif" /></maths>where:<dl id="dl0003" compact="compact"><dt>NewFramePos =</dt><dd>frame number of desired frame position</dd><dt>FrameRate =</dt><dd>frames per second</dd></dl>

0022Subsequently, at block 90, a determination is made as to whether the nearest PTS as determined at block 86 is within a specified proximity to the time position of the desired frame, NewTimePos, as determined at block 88. As a preferred embodiment of the invention, the specified proximity is about one second, and probably should not be more than two second. If the nearest PTS is not within the specified proximity to NewTimePos, then the process returns back to block 82 in order to determine a new ByteRate. Otherwise, if the nearest PTS is within the specified proximity to NewTimePos, then move forward or backward to locate a frame with a PTS that is closest to NewTimePos, at block 92. This frame position, at which the desired frame is located, is probably different from the selected frame position obtained at block 86. Finally, at block 94, a backward search for the nearest I-frame is performed. All the frames subsequent to the I-frame can then be decoded, at block 96, until the desired frame position found at block 92 is reached.

0023As has been described, the present invention provides an improved method for locating a frame position in an MPEG data stream within a computer system.

Contents3

Every citation, both ways

Citations
Document	Relation	Office	Cited during
US10135900B2	Cited by	United States of America	Applicant
US7242851B2	Cited by	United States of America	Search report
WO03023785A2	Cited by	World Intellectual Property Organization (WIPO)	International search
US7975225B2	Cited by	United States of America	Applicant
US10382494B2	Cited by	United States of America	Applicant
US7212727B2	Cited by	United States of America	Applicant
EP1274086A2	Cited by	European Patent Office (EPO)	Search report
CN104298991A	Cited by	China	Search report
US7106749B1	Cited by	United States of America	Applicant
US7106749B1	Cited by	United States of America	Applicant
US9060190B2	Cited by	United States of America	Applicant
EP1274086A3	Cited by	European Patent Office (EPO)	Search report
US2024078784A1	Cited by	United States of America	Search report
US7756201B2	Cited by	United States of America	Applicant
EP1860884A1	Cited by	European Patent Office (EPO)	Search report
US7283880B2	Cited by	United States of America	Applicant
WO2017124897A1	Cited by	World Intellectual Property Organization (WIPO)	International search
EP2140371A1	Cited by	European Patent Office (EPO)	Search report
US9787725B2	Cited by	United States of America	Applicant
EP1056273A2	Cited by	European Patent Office (EPO)	Search report
US10911498B2	Cited by	United States of America	Applicant
EP1278193A1	Cited by	European Patent Office (EPO)	Search report
EP1278193A4	Cited by	European Patent Office (EPO)	Search report
US10834435B2	Cited by	United States of America	Applicant
WO2007138243A1	Cited by	World Intellectual Property Organization (WIPO)	International search
WO03010970A2	Cited by	World Intellectual Property Organization (WIPO)	International search
CN100350488C	Cited by	China	Search report
WO03023785A3	Cited by	World Intellectual Property Organization (WIPO)	International search
US9060190B2	Cited by	United States of America	Applicant
US7580416B2	Cited by	United States of America	Applicant
US9060190B2	Cited by	United States of America	Applicant
EP1860884A1	Cited by	European Patent Office (EPO)	Search report
EP2439741A1	Cited by	European Patent Office (EPO)	Search report
EP1056273A3	Cited by	European Patent Office (EPO)	Search report
US8611721B2	Cited by	United States of America	Applicant
WO03010970A3	Cited by	World Intellectual Property Organization (WIPO)	International search
WO2007138243A1	Cited by	World Intellectual Property Organization (WIPO)	International search
EP0634867A2	Cites	European Patent Office (EPO)	Search report
EP0677961A2	Cites	European Patent Office (EPO)	Search report

9 members in 5 offices; this record represents the family

This recordMember, by publication datePriority claim

Priority claims4

Priority claims
Document	Office	Kind	Date
59026496	United States of America	A	1996-01-23
590264	United States of America	–	1996-01-23
US19960590264	–	–	–
590264	–	–	–

Members9

Family members, oldest first
Document	Office	Kind
EP0786909A2This record	European Patent Office (EPO)	A2	1997-07-30
JPH09224244A	Japan	A	1997-08-26
US5715176A	United States of America	A	1998-02-03
EP0786909A3	European Patent Office (EPO)	A3	1999-07-28
KR100233353B1	Republic of Korea	B1	1999-12-01
JP3320327B2	Japan	B2	2002-09-03
EP0786909B1	European Patent Office (EPO)	B1	2006-11-02
DE69736871D1	Germany	D1	2006-12-14
DE69736871T2	Germany	T2	2007-04-26

24 legal events, as 4 offices reported them to INPADOC

Over the term

Point at a mark for the event

Events

Legal events, newest first
	Event	Code	Office
2017-05-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2017-02-08	Patent expired after termination of 20 yearsExpiredPE20	PE20	GB
2017-01-14	Expiry of rightR071	R071	DE
2016-05-31	Annual fee paid to national office [announced via postgrant information from national office to epo]GrantedPGFP	PGFP	EP
2016-04-29	Annual fee paid to national office [announced via postgrant information from national office to epo]GrantedPGFP	PGFP	EP
2012-12-31	Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25	PG25	EP
2012-10-19	Notification of lapseLapsedST	ST	FR
2011-05-31	Annual fee paid to national office [announced via postgrant information from national office to epo]GrantedPGFP	PGFP	EP
2009-01-07	Register noted 'licences of right' (sect. 46/1977)746	746	GB
2007-10-17	No opposition filedOpposition26N	26N	EP
2007-09-14	No opposition filed within time limitOppositionORIGINAL CODE: 0009261PLBE	PLBE	EP
2007-09-14	Information on the status of an ep patent application or granted ep patentGrantedSTATUS: NO OPPOSITION FILED WITHIN TIME LIMITSTAA	STAA	EP
2007-04-27	Fr: translation filedET	ET	EP
2006-12-14	Corresponds to:REF	REF	EP
2006-11-08	European patent grantedGrantedFG4D	FG4D	GB
2006-11-02	Designated contracting statesAK	AK	EP
2006-09-29	(expected) grantORIGINAL CODE: 0009210GRAA	GRAA	EP
2006-09-20	Grant fee paidORIGINAL CODE: EPIDOSNIGR3GRAS	GRAS	EP
2006-05-17	Despatch of communication of intention to grant a patentORIGINAL CODE: EPIDOSNIGR1GRAP	GRAP	EP
2000-03-22	Request for examination filed17P	17P	EP
1999-07-28	Designated contracting statesAK	AK	EP
1999-06-11	Search report despatchedORIGINAL CODE: 0009013PUAL	PUAL	EP
1997-07-30	Designated contracting statesAK	AK	EP
1997-06-13	Public reference made under article 153(3) epc to a published international application that has entered the european phaseORIGINAL CODE: 0009012PUAI	PUAI	EP