Automated traffic violation monitoring and reporting system

[0001] The present invention relates generally to traffic monitoring systems, and more particularly to a system for detecting and monitoring traffic violations.

[0002] Camera-based traffic monitoring systems are increasingly used by law enforcement and city authorities to enforce traffic rules and change unsafe driver behavior, such as exceeding speed limits, crossing red lights or stop signs, and making turns that do not comply with regulations. The most effective programs combine the consistent use of traffic cameras, aided by automated processing solutions that ensure the immediate handing of a traffic violation mandate, to other elements of the program including public education and specific road safety-oriented initiatives, such as drunk driving supervision programs and penalties related to loss of driving license. However, many current road traffic surveillance systems using photographic techniques have disadvantages that usually do not facilitate effective automation and checking of the photos required for effective use as legal proof.

[0003] Digital camera systems monitoring red light replace traditional 35 mm analog cameras and photographic techniques in obtaining photographic evidence of traffic violations. In the field of road traffic surveillance technology, obtaining vehicle offense data involves a trade-off between memory space requirements and image resolution. Usually, the misdemeanor is recorded as many still images of the vehicle along with relevant information such as speed, time of the offense and so on.

[0004] Recording of cases of non-compliance with red light has traditionally been done with cameras, either digital or based on wet film technology, or with video camera systems. These systems have many disadvantages. For example, still images usually do not provide enough information to assess the circumstances surrounding an offense. The vehicle forced to enter the intersection after the traffic light turns red when the privileged vehicle gives way will be depicted on the still images as the one who violated the rules and the driver will be charged if the emergency vehicle does not appear on the still images. Also at many intersections, vehicles may turn onto a red light if they stop first. Still images do not show the acceleration and speed of the vehicle and cannot determine whether the vehicle has acted unlawfully, i.e. without first stopping. In order to supervise speed, the vehicle speed must be determined from the vehicle detection device and reflected in the photo. Errors in the detected vehicle speed will not be visible in the picture, because the still images do not convey any impression of speed. Although many still images can be taken to show speed at two or more points, this solution leads to an increased number of images taken and memory requirements, and causes the camera to be busy for the duration of the image sequence.

[0005] Image resolution is critical to provide adequate information to analyze important details of the image, such as the identification of license plate data and

- driver's face. However, increasing the image resolution also increases the data storage requirements.

[0006] To address the problem of providing contextual and background data associated with a potential traffic violation in a place monitored by photographs, video has been introduced in some red light monitoring systems. However, the appearance of video has some significant disadvantages. First of all, when law enforcement authorities want to use video in their evidence set, problems related to data throughput and data storage increase significantly. Digital video technology generates data at a much higher speed than digital still image technology at the same resolution. Although the video material is used to identify and prosecute vehicles that violate traffic rules, the usually low resolution of current video systems makes it difficult to determine the fine details required for prosecution, such as the vehicle's license plate or driver's features. The low resolution problem also requires that the video camera be near the detected vehicle or that it move physically and track the vehicle, both of which are the main disadvantages when using automated road monitoring systems. Although high resolution video cameras can be used to identify and prosecute vehicles that violate traffic regulations, if the information from high resolution video cameras is stored digitally, the amount of file memory required makes it difficult or impractical to store and transfer such generated amounts information. This is especially true for systems that do not provide efficient video clips, but only fi y and send long loops of fixed video data.

[0007] The prior art that discloses the use of a video camera for the enforcement of traffic laws is presented in document WO-A-98/19284 which describes a traffic law enforcement system having one or more surveillance units for detecting speed and identifying the vehicle. Document WO-A-99/19284 also describes the use of sham units.

[0008] A typical start / stop capture mechanism available in almost all video capture systems is inadequate to meet the requirement to provide phim material both before and after the offense is detected. From the moment the offense is detected, it is too late to start the video capture sequence. In general, it is also difficult to predict the offense and prevent video capture. In addition, when video material from a video stream is recorded on magnetic tape, it takes time to extract information and finding a specific offense or incident cannot be done immediately.

[0009] Various suitable embodiments and features of the invention are defined in the appended claims.

[0010] The present invention combines still, high-resolution digital images and low-resolution video into one set of information to be used to record traffic violations in a way that minimizes data transmission and storage requirements.

[0011] The present invention includes a "before" and "after" video sequence that allows viewers to identify mitigating or aggravating circumstances just before or after the detection of a traffic offense.

[0012] Embodiments of the present invention provide means for visual verification of the speed of a detected vehicle without the use of multiple high resolution still images.

[0013] Embodiments of the present invention may provide means for easily searching for specific incidents or driver / car information from stored or archived data.

[0014] In a first embodiment, a system is provided as defined in the appended claim 1. [0015] In a second embodiment, there is provided a method of creating the main evidence of a road traffic offense at a traffic location as defined in the appended claim 7.

[0016] The above features may provide law enforcement authorities with a more complete record of events leading to or following the offense itself. This can help office staff better see the context of the offense or even detect subsequent offenses caused by the same vehicle. For example, a system based on still images will detect the car both before and after the line with a red light, but with video, employees investigating misdemeanors may also notice that the car has entered the intersection to give way to emergency vehicles or that the driver has also lost control of the car and became a participant in the accident.

[0017] The combination of still images and video material solves or mitigates the problems associated with the video requirement and the need for high resolution and low storage and transmission costs. As still images will continue to provide the high resolution necessary to obtain important detail data from the evidence set, video recording can be made using low resolution technology that will not overload the storage and data transmission systems.

[0018] Other features and advantages of embodiments of the present invention will become apparent through the accompanying drawings and the following detailed description.

[0019] The present invention is illustrated by way of example and non-limiting drawings in which like references point to similar elements and in which:

Fig. 1A is a block diagram that illustrates the entire traffic violation processing system in accordance with one embodiment of the present invention;

Fig. 1B is a table that shows some information transmitted along the data paths illustrated in Fig. 1A for an exemplary traffic monitoring and reporting case;

Fig. 1C illustrates the implementation of a camera system for monitoring traffic violations at a traffic location according to one embodiment of the present invention;

Fig. 2 illustrates the photographic image and accompanying reporting information provided by the camera system and data processing system of Fig. 1A according to one embodiment of the present invention;

Fig. 3A is an illustration of a block diagram of a camera intersection system with multiple CCD elements according to one embodiment of the present invention;

Fig. 3B illustrates the multi-component camera system of Fig. 3A in combination with a synchronous clock according to one embodiment of the present invention;

Fig. 4A illustrates a histogram of pixel intensity for an intersection image according to one embodiment of the present invention;

Fig. 4B illustrates the histogram of Fig. 4A with a license plate image extracted from a background scenery image;

Fig. 5 illustrates a set of offenses provided by an image processing system according to one embodiment of the present invention;

-4Fig. 6 is a flowchart that illustrates the steps performed by a central processor when information about an incident is received from a junction camera system according to one embodiment of the present invention;

Fig. 7 illustrates the DMV (Vehicle Registration Department) detail data area of ​​the verification screen according to one embodiment of the present invention;

Fig. 8 illustrates a DMV search screen according to one embodiment of the present invention; Fig. 9A illustrates an example of the police authorization module interface screen according to one embodiment;

Fig. 9B illustrates an example of a court interface screen formed by a court interface module according to one embodiment.

Fig. 9C illustrates a police authorization browsing interface that can be used by police officers to view photos and a video clip with an indicator;

Fig. 10 is a flowchart for creating traffic violation notifications according to one embodiment of the present invention;

Fig. 11 illustrates a preview of the notification displayed on the screen of the user interface according to one embodiment of the present invention;

Fig. 12 illustrates components of a traffic camera office offenses processing system in accordance with one embodiment of the present invention;

Fig. 13 illustrates components of an expert system for analyzing images according to one embodiment of the present invention;

Fig. 14 is a block diagram that illustrates the main components of the video camera system illustrated in Fig. 1A;

Fig. 15 is a flowchart that steps out the steps of making a video clip of an detected offense according to one embodiment of the present invention;

Fig. 16A illustrates a detection system using a single induction loop installed in a road surface;

Fig. 16B illustrates a detection system using two induction loops installed in a road surface;

Fig. 16C illustrates a detection system using an induction loop inserted between two piezoelectric strips installed in a road surface;

Fig. 16D illustrates a detection system using an induction loop inserted between two piezoelectric strips with an additional induction loop installed in the road surface;

Fig. 17 illustrates the detection of a vehicle by means of virtual video loops according to one embodiment of the present invention.

[0020] An automated system for monitoring and reporting traffic violations will now be described using both camera systems and video camera systems. In the following description, for the purpose of explanation, many specific details are given to provide an understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention can be used without these specific details. In other examples, well-known structures and devices are shown in block diagram to facilitate explanation.

The description of preferred embodiments is not intended to limit the scope of the claims appended hereto.

[0021] Fig. 1A is a block diagram that illustrates the entire traffic violation processing system according to one embodiment of the present invention. The main components of the traffic offense processing system 100 include the intersection camera system 102, the offense detector system 105, the data processing system 104, the police department interface system 106, the motor vehicle department interface 108, the court interface 110.

[0022] The red light camera system 102 includes one or more cameras 129 and one or more video cameras 122 located at or near the intersection or traffic site that is being monitored. When the alleged offender 101 commits an offense at the intersection detected by the offense detector 105, cameras monitoring the red light in the camera system of intersection 102 recognize and record the event. In one embodiment of the present invention, both the digital still photo as well as the video fragment, such as video lasting five to ten seconds and capturing the event, is recorded and sent to the data processing system 104. The data processing system 104 then performs various stages of data processing, to verify and check driver details and offenses. The data processing system 104 itself contains various components such as central processor 132, file server 134, database 136, verification module 138, quality assurance module 140 and notification printing module 142. The data processing system 104 receives data from various external sources such as intersection cameras and motor vehicle offices, and processes data for further investigation by law enforcement agencies.

[0023] As illustrated in Fig. 1A, various elements of driver and vehicle information are obtained by the data processing system 104 from selected authorities, such as a motor vehicle department through the motor vehicle department interface 108 and a police department through the police department interface 106. Usually this the information is extracted from still image data obtained by 120 cameras. Video data captured by video cameras 122 is provided to provide contextual information regarding the event. For this embodiment, the resolution of the video camera may be lower than the resolution of the cameras, because they are intended to provide general scenery data. This reduces the requirements for data storage and transmission compared to systems that use long, high-definition video clips.

[0024] In an alternative embodiment of the present invention, the identification information can be obtained from video data captured by video cameras 122. For this embodiment, still images are obtained from video clips, hence the resolution of the video camera system should be high enough to provide detailed information . The optional frame editor 133 in the data processing system can be used to extract and mark the appropriate frames to be processed as still video images. The detection system for system 100 may include a physical infraction detector 105 and / or a virtual loop detector 106 to result in the acquisition of still data or a video clip related to the offense.

[0025] Once the offense information is found to be convincing, it is provided by the court interface system 110 to the appropriate judicial authorities.

[0026] As illustrated in Fig. 1A, the infraction detector 105 may be included in a physical detection system that is located at an intersection, such as magnetic, optical or electric

-6system that detects the presence or movement of a vehicle through an intersection. If the vehicle is detected at the wrong time or at the wrong speed, the detector 105 will activate cameras and video cameras in the system 102 to capture the incident. In an alternative embodiment, the detection system for video cameras may be implemented by a virtual loop detector process 139. For this embodiment, the virtual loop or trigger is defined within the view field captured by the video cameras 122. When the vehicle is photographed or recorded on a video tape in this virtual place at the wrong time, a clock associated with the video clip based on the video material is started.

[0027] For the system shown in Fig. 1A, different data paths, numbered from 1 to 14, are provided between components and subcomponents of the system 100. Fig. 1B is a table that shows some information transmitted along these paths as a consequence of typical monitoring and reporting traffic offense. Table 150 in Fig. 1B together with the data paths of Fig. 1A represent the data flow process for the traffic violation processing system 100. As shown in Fig. 1A and Fig. 1B, data provided by the intersection camera system 102 includes still pictures 1A and video data 1B. There can be any number of still photos for the incident, usually four to six separate digital photos, and any length of the video clip of the incident, usually four to ten seconds of video accompanying the incident. Because still images and video clip are provided by separate camera systems 120 and 122, they can provide photographic data with different resolutions. To minimize bandwidth and data storage requirements, still images can be created and processed at a higher resolution to provide very accurate identification and evidence images, while video data may have lower resolution because they are primarily intended to provide background information.

[0028] If red light monitoring cameras in the intersection camera system 102 detect an offense incident, many images (usually four) of the incident along with associated data (such as vehicle time and speed) are captured and sent to central processor 132 of data processing system 104. These images and related data are the main evidence of the offense and are saved to the main image file server 134. The central processor creates compressed scenery images and incident details, sends them to database 136 for storage. In one embodiment, the offense is detected by using known wireless transmission techniques, such as radar or similar waves, or by light beam detection techniques, or similar techniques to determine if a vehicle is moving too fast or has passed a red light or stop sign. Alternatively, the violation is detected by using physical underground loops placed inside the road surface. The presence of a car near the loop at the wrong time relative to traffic lights or other controls will signal a potential traffic violation.

[0029] The images captured by cameras of the intersection camera system 120 typically include at least one image of the offending vehicle (i.e., red traffic light) and vehicle license plate and driver face images to provide car and driver identification information. The license plate and driver face images are sent from the main image file server to verification module 138. Based on the information about the vehicle's license plate, the details of the vehicle and its owner are then read from the appropriate department of motor vehicles 108 and sent to database 136. Along with the still images, a video file is taken with a violation by video cameras 122. Data videos are then associated with the corresponding still image data for review by authorities. This makes it possible to reduce the amount of data that needs to be created and transferred from about 80 megabytes of data (for current systems that only transfer high definition video data) to about 2.5 megabytes of data for low resolution video connection and high resolution still images resolution.

[0030] The incident details and compressed images stored in the database 136 are then sent to the quality assurance module 140. After the quality assurance module checks the incident data for accuracy and consistency, the details and compressed images are sent to the appropriate office police 106. If the police allow the notification to be identified to the identified driver, the details of the notification are sent to the appropriate court 110 by the data processing system 104. The incident notification and details are also sent from database 136 to the notification printing module 142 of the data processing system 104. Prepared notification it is then sent to the alleged offender 101 by the data processing system 104. Further correspondence, such as payment reminder letters, may be sent to the alleged offender from court 110. The alleged offender may then remit the fee or go to court to respond to the notice. The offense notification is then sent from court 110 to data processing system 104 and placed in database 136. This completes the data processing loop for a typical offense according to one embodiment of the present invention.

[0031] The structure and operation of the sub-components of each of the main components of the traffic violation processing system 100 will be described in more detail in the description below.

Intersection Camera System [0032] Typical surveillance use of the digital camera component 102 of the system 100 belongs to the red light detection area. For this application, cameras or cameras 120 of camera system 102 are strategically placed at an intersection to monitor and record incidents of non-compliance by drivers of red light. When a vehicle is detected approaching the stop line of the monitored lane, it is tracked and its speed is calculated. If vehicle entry at an intersection is detected against traffic lights, an evidentiary image set is made. The event of taking pictures and recording relevant detail data is referred to as an 'incident' which can be referred to as a potential misconduct. In one embodiment of the present invention, the evidence set includes four incident images, comprising the following: scenery shot A, which is a shot of the intersection scenery before the incident vehicle crosses the stop line; shot of scenery B, which is a shot of the intersection scenery, which shows that the incident vehicle did not comply with the traffic signals; enlarged facial shot that attempts to identify the driver of the incident vehicles; and an enlarged license plate shot that attempts to extract the vehicle's license plate area for the sole purpose of identifying the vehicle. In one embodiment, the still images are captured

- by 120 digital camera system they are in TIFF or JPG format, although other digital formats are also possible.

[0033] With respect to the potential offense, a lot of detail data is recorded for each image. These include the date and time of the incident, the location of the incident, the time that has elapsed since the red light turned on, and camera identification. The incident short video file is also recorded and associated with still image data.

[0034] The captured data is assigned a "digital signature", they are encrypted and then sent from the digital camera system 102 to central processor 132 in the data processing system 104. All four shots carry "stamped" details of the incident when they are transferred. In one embodiment, this "stamped information" is contained in a data bar that appears at the top of the image visible in the verification process 138 of the data processing system 104. Each of the four shots is individually identified as being of a particular type, e.g. shot A , B shot, face shot, whiteboard shot. FIG. 11 represents a notification to appear that includes photographic images and accompanying reporting information that is provided by the camera system and data processing system of Fig. 1A according to one embodiment of the present invention. As can be seen in Fig. 11, four photographs include a driver's face shot, a license plate shot and scenes shot A and B. Folding and creating the notice to appear illustrated in Fig. 11 will be described in more detail below.

[0035] Junction cameras can be remotely controlled to facilitate analytical system checks and test shots. For diagnostic tests, the log of test shots taken is recorded. Test shots can be treated as normal and exported to the data processing system to be added to the database as in the case of 'normal' shots. If it proves necessary to prove to the court that the camera system was operating correctly when a particular incident was detected, the test shots form part of the chain of evidence that is used to provide evidence of the cameras operating correctly.

[0036] Camera intersection systems are connected to each other on the detection side to provide the required coordination of the camera and flash. Each camera is strategically positioned to provide an optimal field of view for the desired captured image. The surveillance camera, which is equipped / connected with the vehicle tracking technology, is positioned so as to effectively record both the scenery images and the shot plate area shot. A supplementary camera can be positioned to take a picture of the driver of the vehicle violating traffic rules. The camera and processing systems are connected to each other using typical local network topologies. Camera system 102 may also be configured to send secure (encrypted) incident data and image information to data processing system 104 via a computer network line, such as a modem or telephone line.

[0037] Fig. 1C illustrates the implementation of an intersection camera system at an intersection according to one embodiment of the present invention. Cameras and processing system assembly are housed in a body 174 which is placed on a pole or other supporting structure 180 above the monitored location, usually in the vicinity of a traffic light or stop sign. The height and location of the camera system is selected to allow the appropriate field of view 182 of the monitored place. A loop detector 172 located in the roadway detects the abnormal presence or movement of the vehicle 170 at the monitored location. It is used to launch cameras to get photographic evidence of offenses. In one embodiment, the housing houses three separate digital cameras 176 and one video camera 178. Depending on the implementation limitations and capabilities of the system, different camera configurations can be used, such as one or kiika cameras and / or video cameras placed in one or many distributed locations around the place. If a video camera with sufficiently high resolution is used, one video camera can be used, thanks to which both video and still images can be obtained.

[0038] Parts of the data processing system 104 illustrated in Fig. 1A may be located inside the body 174. For example, a computer that includes a central processor 132 may be tightly coupled to cameras 176 and 178 inside the housing 174. Alternatively, the housing 174 may be configured to housing only cameras 176 and 178. In this case, a wired, wireless or telephone network connection can be used to connect cameras to the central processor and other components of the data processing system 104. This system can be provided in a separate housing on site or in a remote location, at a distance from the monitored location.

Camera system [0039] In a preferred embodiment of the present invention, the traffic violation processing system 100 uses digital camera technology for cameras 120. Such a digital camera system is directed to specific areas of interest by means of a system comprising kiika imaging elements. The advantage of this configuration is focusing on resolution where it is needed, while maintaining the premise that the obtained images are taken at the same time.

[0040] Imaging elements that are a Charge-Coupled Device (CCD) can be used for digital cameras. They usually provide a spatial and dynamic resolution that is equal to or better than the resolution of a 35 mm celluloid film. The camera system of intersection 102 uses a scalable multi-component digital camera system, designed specifically for surveillance applications. This camera system is specifically designed to solve problems with dynamic range resolution and imaging speed (i.e., frames per second) for the specific requirements of prosecution of offenses in which images are the main evidence.

[0041] A CCD is an image receiving device capable of converting light energy emitted or reflected by an object into an electric charge that is directly proportional to the intensity of the incident light. This payload or pixel can then be sampled and processed into the digital domain. Digital pixel information is cached and forwarded via local bus to RAM (Free Access Memory) in the host computer system where further processing and final storage take place.

[0042] The basic imaging requirement for prosecution is clear identification of the offense committed and identification of the offending vehicle. In a multi-camera system, all imaging elements must be synchronized and activated simultaneously to ensure that all captured images correspond to the same event, i.e. have an accurate time base.

[0043] Fig. 3A illustrates an intersection camera system with multiple CCD elements for use in cameras 120 according to one embodiment of the present invention. The camera system 300 in Fig. 3A illustrates a representative camera system comprising the main CCD 302 element and two subordinate CCD elements 304 and 306. The CCD elements 302, 304 and 306 convert the incoming light into an electric charge. The load is then carried through an analog shift register to provide a serial load data stream similar to a bucket brigade. For camera system 300, image data from the main CCD 302 is processed by the ADC (analog to digital converter) process 308 to create digital data streams 310. The image data from the two slave CCD cameras 304 and 306 are processed by the respective ADC processes 312 and 314 and fed to the multiplexer 316 to create digital data streams 318. [0044] The basic operation of the CCD in camera system 300 will now be described. For each camera, the image-sensing CCD area is set up in horizontal lines containing multiple pixels. When light enters the silicon in the image-sensing area, free electrons are generated and accumulated inside the photosensitive potential wells. The value of the charge accumulated in each pixel is a linear function of the incident light and exposure time. After exposure, the payload packages are transferred from the image area to the serial register at a speed of one line per clock cycle. After passing the image line to the serial register, the serial register output can be clocked until all payload packets leave the serial register through the buffer and amplification stage to create an analog signal. This signal is sampled using high speed ADC analog to digital converters to produce a digital image.

[0045] Color detection is achieved by laminating, on top of the image sensing area, a striped RGB color filter (red, green, blue). The strips are precisely aligned with respect to the sensing elements and the columns charged with the signal can be multlexed during reading into three separate registers with separate outputs corresponding to individual colors. Each red, green and blue pixel from the CCD is processed by a high-resolution analog-to-digital converter capable of sampling at high frequency. In the digital domain, the payload pixel is cached as it waits for the data transfer window to allow the host computer system to transfer it to the host computer RAM. [0046] In one embodiment of the present invention, the image data is transferred from CCD elements 302, 304 and 306 to the RAM of the host computer system via a PCI interface (Peripheral Component Interconnect) 320. For many current computer systems, PCI has become a local bus standard for connecting integrated circuits, expansion cards and processors. The original PCI architecture implements a 32-bit multiplied address and data bus.

[0047] According to the typical use of PCI in the camera system 300, communication between devices on the PCI bus occurs via a burst transfers mechanism. Serial transmission consists of establishing the relationship between the bus manager (l / O cycle - so that Ιπ ^ ιχ serial transmission becomes the bus manager) and the bus slave module (purpose). The length of the serial transmission is negotiated at the beginning of the transmission and can be any. At the end of the serial transmission, the recipient (target) ends the communication after receiving the previously determined quantity of information. Only one device being the bus manager can communicate via the bus at a time. Other devices cannot interrupt the serial transmission process because they do not have administrator status.

[0048] The integration of the CCD imaging device directly with the final computer processing system shortens the traditional process of acquiring digital images using video cameras, converting the composite analog signal to a digital image using a 'Frame Grabber' device and then imposing the resulting image to the host computer for processing. Image quality losses that occur as a result of digital-to-analog and then analog-to-digital processing in these systems limit their use for traffic control. In addition, video cameras are usually limited in resolution and dynamic range.

[0049] Dynamic resolution is an important feature of the camera system 300. Dynamic resolution defines the data size of each pixel after being digitized. The relationship is proportional to the CCD camera's ability to simultaneously represent very small and large levels of light intensity (i.e. signal-to-noise ratio, SNR) and is represented in decibels (dB). Accordingly, the ADC sampling is matched to show equivalent SNR.

[0050] The use of dynamic resolution in surveillance programs provides a mechanism for identifying vehicle number plates of reflective composites. When flash photography is used to reproduce high-quality images, the light energy that is directed towards the license plate area is reflected back at a level (high reflection effect) that is higher than the average intensity entering the camera. As a result, the effect of optical burning (i.e. excessive exposure) appears around the license plate area. [0051] The effect of optical burning or "array burning" is minimized by using a CCD and ADC system with a dynamic range that allows the resulting intensity spectrum to be distinguished. The image histogram will reveal all scenery and license plate details at opposite ends of the spectrum.

[0052] The license plate having the strongest intensity will appear at the highest levels, and the rest of the image in the rest of the spectrum, respectively. However, most computer systems, and in fact the human eye, can only distinguish between 256 levels (or 48 dB = 8 bits) of intensity. A typical 35mm celluloid film with a sensitivity of 100 ASA is thought to have 72 dB of equivalent dynamic resolution. This dynamic range can isolate 4096 intensity levels and is represented by a 12-bit word.

[0053] To reduce the amount of data and information held for evidentiary purposes, the "Slice histogram cutting" process can be used to reduce the total pixel data size from 12 bits to 8 bits by selecting only 256 of the available 4096 levels. The selection criterion will ensure that the visual integrity of the image is assured, but also normalizes the overall appearance so that the overexposed areas are in balance with the rest of the image. Ideally, the process should be a non-linear function that is inherently adaptive to counteract ambient and exposure conditions. Translation for speed and efficiency would be a mapping (or search) function.

[0054] Fig. 4A illustrates the histogram of pixel intensity for an intersection image according to one embodiment of the present invention, and Fig. 4B illustrates the histogram of Fig. 4A with the image of the license plate extracted from the other images that constitute the vehicle and the supporting scenery. Details of the digital imaging process that extracts the license plate image are described in US 6240217 entitled "Digital image processing" which has been assigned to the owner of the present invention and which is incorporated herein by reference. The histograms of Fig.

4A and Fig. 4B illustrate the intensity of individual pixels in a traffic violation image with pixel axis 402 and intensity axis 404. As illustrated in Fig. 4A, the individual components of the license plate are shown as elements 408 relative to the components of the background scenery 406 . Using compression and image extraction techniques, the pixel intensity of the license plate 408 is varied relative to the pixel intensity for the background scenery 406, as illustrated in Fig. 4B. In this way, the license plate is more readable in relation to the background scenery. It should be noted that the same technique could be applied to other images and image components so as to emphasize the driver's face in relation to the car.

[0055] As mentioned above, the typical surveillance application of the digital camera system illustrated in Fig. 3A is the detection area of ​​red light violation. The camera system is strategically placed at the intersection to monitor and record incidents of non-compliance by drivers with red lights. In this embodiment, the main evidence created is a set of two images. The first image shows a view of the intersection, which includes traffic lights of the monitored arrival, a vehicle committing an offense before crossing the line (usually a white line such as a pedestrian crossing) and a sufficient secondary scenery showing the driving conditions at the time of the offense. The second image usually presents the same field of view, but with a vehicle that committed an offense that completely crossed the line in black light.

[0056] The main area of ​​interest is the position of the vehicle before and after the intersection. Although the overall resolution for this image is not critical, there must be sufficient detail to distinguish the features of the intersection and the active phase of the traffic light. However, to identify the offending vehicle, license plate details and jurisdiction information must be legible. For 35mm cameras based on wet film technology, the actual spatial resolution must be 3072 x 2048 pixels. Even then, the license plate details only represent 5 percent of the total number of pixels.

[0057] The architecture of the digital camera system 300 allows synchronous operation of multiple imaging elements that all acquire specific areas of interest in the same time interval. The field of view of the main imaging element will cover the entire intersection, the front of the traffic light of the monitored approach and the relative position of the offending vehicle. Secondary imaging elements can be used to display the area of ​​the vehicle license plate.

[0058] To ensure synchronization of all imaging elements, the time generators for each CCD are simultaneously zeroed and timed by one source. Fig. 3B illustrates the camera system 200 of Fig. 3A in connection with a synchronous clock. Each of the three CCD elements 302, 304 and 306 has its output signal synchronized with the respective clock generator circuit 330, 332 and 334. The clock generator systems are controlled by a common clock 340 and reset signals 342. The effect is that each CCD acquires and emits an image simultaneously with other CCD cameras. One of the benefits of synchronous operation of CCD elements is that one flash can be fired with the resulting exposure recorded by all CCD elements.

[0059] In many situations, the vehicle detection system used to track and identify offending vehicles can provide real-world vehicle position information, such as lane, speed and direction, which can be used to narrow the field of view of the secondary imaging elements, thereby enabling sharper and a larger picture of the license plate area. On

- 13 example at an intersection or in a road environment with two lanes, one of the subordinate elements can be used to depict one lane and another can be used to depict another lane. The advantage of this system is that two slave cameras can share the same data path, because only either the first lane or the second lane will be imaged.

[0060] In many situations, more than one camera system (including host computer, imaging components and surveillance logic) may require complementary camera systems to provide additional or more optimal field of view of the offense. One such requirement is to acquire the image of the driver of the offending vehicle when the main detection camera shows the offending vehicle as it approaches the intersection. In such cases, it is impossible to achieve the required field of view, leading to the addition of a complementary camera system.

[0061] In one embodiment of the present invention, distributed computer and network technologies such as DCOM (Distributed Component Object Module) and equivalent CORBA (Common Object Request Broker Architecture) technology are implemented by the traffic supervision system 100 to provide mechanism for easy attachment of imaging elements. This allows you to efficiently increase the number of imaging elements while maintaining the ideology of a single camera surveillance system.

Video camera system [0062] For the system illustrated in Fig. 1A, the intersection camera system 102 includes a video camera system 122. As shown in Fig. 1C, this camera can be one digital video camera mounted with cameras in a specific location that provides sufficient field view of the monitored intersection or traffic. In an alternative embodiment, the video camera may be a system of two or more cameras, each of which provides a different field of view of the monitored place. The resulting video recordings can then be delivered separately to data processing system 104 or can be combined to create a composite video image.

[0063] Fig. 14 is a block diagram that illustrates the main components of a video camera system 122. In a 1400 system, the video camera 1402 is a digital video camera that creates video data in PAL, NTSC or other format that can then be processed to produce a video stream in a compressed form such as MPEG, MPEG2, Guicktime, AVI or similar format. In one embodiment, the video camera shoots film of a given place without interruption. Digital video data is stored in buffer 1404, which can be any type of memory (e.g. RAM, Ramdisk, tape and so on) that is sufficient to contain at least part of the video material shot by the camera. The 1406 detection system is connected to the 1402 video camera. When an offense is detected, the 1408 clock is started. The clock is programmed to stop after a predetermined period of time. At the end of the clock period, the buffer or "snapshot" buffer content is downloaded by the 1410 video recording device. The video recording device downloads the video file recorded by the video camera for that time period of the clock and the time period before the offense was detected. Buffer and video recording device are used to provide a clip about exec! "<^ <rnn and moments immediately before and after the offense. Therefore, to capture, for example, six seconds before and six seconds after the offense was detected, the buffer

- 141404 must keep at least twelve seconds of footage in memory. When an infraction is detected, the system starts a clock with a period of six seconds, after which it completes the buffer clip and places it in non-volatile memory such as hard disk 1412. This memory (hard disk) can also be used to store still images offenses. Therefore, the resulting video can be attached to a traditional evidence set provided by cameras.

[0064] Fig. 15 is a flowchart illustrating the steps of capturing a video file with the detected offense. according to one embodiment of the present invention. At step 1502, the video camera 1402 continuously records video segments of the monitored location. Video data is buffered in buffer 1404 at step 1504. Detection system 1406 detects a traffic offense at step 1506. Detection of an offense starts clock 1408 that measures a fixed period of time at step 1508. After this period of time has elapsed, the clock stays at step 1510. In step 1512, the video recording device 1410 downloads and cuts the video file from the buffer from a predetermined time before the violation to the end of the clock period. The video clip is then placed in a memory, such as hard disk 1412, and associated with camera offense data in step 1514.

[0065] As illustrated in Fig. 14, the video recording system includes detection system 1406 for detecting the occurrence of traffic violations. The detection system may include a physical loop or trigger wire embedded in the road surface to detect the incorrect presence of the vehicle. In one embodiment, the detection system uses one or more induction loops installed in one or more road pavement lanes of the monitored location. The loops can be a single induction loop sensor, a pair of induction loop sensors, or a single induction loop sensor inserted between a pair of piezoelectric sensors installed in the road surface. When a pair of induction loop sensors are used or when a single induction loop sensor is inserted between a pair of piezoelectric sensors, a second induction loop sensor, a "slave loop" following the first sensor, may also be used.

[0066] Fig. 16A illustrates a detection system using a single induction loop installed in a road surface. Fig. 16 B illustrates a detection system using two induction loops installed in a road surface. Fig. 16C illustrates a detection system using an induction loop inserted between two piezoelectric strips installed in a road surface. FIG. 16D illustrates a detection system using an induction loop inserted between two piezoelectric strips with an additional induction loop installed in the road surface.

[0067] For the single induction loop detection system illustrated in Fig. 16A, the vehicle 1602 is detected by detecting a change in a magnetic field near the induction loop sensor 1604. The start of a change in a magnetic field (increase in the induction loop sensor) means the position of the front of the vehicle above the induction loop sensor. Return to the initial magnetic field before the change (drop in the induction loop sensor) means the back of the car leaving the immediate vicinity of the induction loop sensor. When a change in the magnetic field (increase in the induction loop sensor) is detected and it does not return to normal within a certain period of time, it can be concluded that the vehicle has stopped over the induction loop sensor.

[0068] Knowing that the vehicle has stopped, the vehicle detection system has the ability to reject vehicles that have braked abruptly through the intersection stop line. These "false challengers" regarding

- 15 red light supervision would otherwise have to be rejected manually leading to an inefficient process of handing over tickets.

[0069] Fig. 16B illustrates a detection system using two induction loops installed in a road surface. When a pair of induction loop sensors are used, the vehicle 1602 is detected by detecting a change in the magnetic field near both induction loop sensors 1604 and 1606. The beginning of the magnetic field change for the first induction loop sensor 1604 means the position of the front of the vehicle above the induction loop sensor and the return to the initial magnetic field before the change means the rear of the vehicle leaving the direct court of the first induction loop sensor. The beginning of the magnetic field change of the second induction loop sensor 1606 means the position of the front of the vehicle and the return to the initial magnetic field before the change means the rear of the vehicle leaving the immediate vicinity of the second induction loop sensor.

[0070] By calculating the difference in time between the detection of the front or vehicle by each induction loop sensor and dividing by this time the distance between the induction loop sensors, the vehicle speed between the two induction loop sensors is obtained, i.e.

Vehicle Speed ​​(nVs) = Flap distance (my Time between loops (S) [0071] Similarly, by calculating the difference in time between detecting the rear of the vehicle on each induction loop sensor and dividing by this time the distance between the induction loop sensors, the vehicle speed between the two sensors black loops of induction loops.

[0072] Furthermore, by calculating the time between rise and fall at any induction loop sensor and multiplying it by the vehicle speed, an approximate length of the vehicle is obtained, that is:

Ptzybiżona Vehicle Length (m) = Vehicle Speed ​​(mS) x Time between rise and fall of loops (S)

This calculation can be made more accurately by subtracting the width of the induction loop sensor from the calculated length, i.e.

Vehicle Length (m) = [Pdk ^ part of the core (mtS) x Time between risefispispidenflag Loop width (m) [0073] When a change in the magnetic field is detected for one or both induction loop sensors and it does not return to normal state within a specified period of time , it can be said that the vehicle has stopped over the induction loop sensor.

[0074] Fig. 16C illustrates detection using induction loop 1604 inserted between two piezoelectric stripes 1608. When a single induction loop sensor is inserted between two piezoelectric stripes, vehicle 1602 is detected as in the case of the single loop detection system illustrated in Fig. 16A, i.e. . the beginning of the change in the magnetic field (increase in the induction loop sensor) means the position of the front of the vehicle and the return to the initial magnetic field before the change (decrease in the induction loop sensor) means the rear of the vehicle. When a vehicle passes over each piezoelectric sensor, its presence is detected by means of an electric signal or an impulse generated when the weight of the vehicle by the tires presses the stripes of the piezoelectric sensor 1608. Accurate

- the vehicle speed is determined by calculating the difference in time between the detection of the front axle passing over the piezoelectric sensors and dividing by that time the distance between the piezoelectric sensors giving the vehicle speed, i.e.

Vehicle speed (m / sj = Distance between piezoelectric sensors (myTime between piezoelectric sensors (s) [0075] As in the case of a sensor system with two induction loops, by calculating the time between the rise and fall of any induction loop sensor and multiplying it by the vehicle speed obtained is the approximate length of the vehicle, i.e.

Piezybiżcna Vehicle Length (m) = Vehicle Speed ​​(yyyy5) x Time between tie and slope of the loop (S)

This calculation can be made more accurately by subtracting the width of the induction loop sensor from the calculated length, i.e.

Vehicle length (m) = Vehicle speed (m6) x Time between half-loop and loop | -Shine <loops (m) [0076] The use of a single induction loop sensor inserted between two piezoelectric strips to detect the vehicle also provides the possibility to count the number of axles the vehicle has. An electrical signal or pulse is generated by the weight of each vehicle axle when it passes over a piezoelectric sensor. The number of pulses detected between the rise in the induction loop sensor and the decrease in the induction loop sensor is equal to the number of axles the vehicle has, i.e.

t = loop drop

Number of Vehicle Axles = Σ (hmxissyccuijii <apżoeekMrczne! Gc>) t = loop increase [0077] By calculating the number of axles the vehicle has and by calculating the length of the vehicle, the vehicle can then be classified into a specific type of vehicle according to standard, easily available charts and vehicle classification tables as a car, truck, bus, and the like. Thus, knowing the vehicle type, detection can be performed depending on the specific vehicle type. The vehicle type can be used to determine if the authorized vehicle uses a bus lane or transit route. The vehicle type can also be used to determine if the vehicle has exceeded the speed for a given type of vehicle, as trucks, cars and buses may have different speed limits. [0078] Fig. 16D illustrates a detection system using the piezoelectric strip system and the induction loop of Fig. 16C and an additional induction loop. When vehicle detection uses a pair of induction loop sensors or an induction loop sensor inserted between two piezoelectric stripes, an additional induction loop sensor 1606 can be added after the first and second induction loops for a pair of induction loops or after the first induction loop 1604 for an induction loop inserted between two 1608 piezoelectric strips to detect the vehicle elsewhere or in a different position after the first detection point. Additional vehicle detection provides the ability to determine the vehicle path after the first detection.

[0079] This system can be used to determine if a vehicle has entered an intersection against a red light after stopping in front of the stop line. It can also be used to determine if a vehicle has entered an intersection and stopped at an intersection.

[0080] In one embodiment, the loop sensor and / or piezoelectric sensor systems illustrated 16A-16D are embedded in the road surface with respect to marking, such as a stop sign or red light. In the event of an intersection, the detectors are usually placed on or at the pedestrian crossing with traffic lights. The actual placement of the sensors depends on the intersection layout. As shown in Fig. 14, detection of a vehicle passing through the intersection or monitored location by the sensor or sensors triggers the clock 1408, which controls the separation of the video clip from the video segment shot by video cameras 1402.

[0081] Other physical detection systems may be used to ensure the detection of an offense. For example, a light-based trigger can be used instead of or in conjunction with an induction loop / piezo belt to detect the presence of a vehicle.

[0082] In an alternative embodiment of the present invention, a virtual loop detector implemented by software or firmware is used as detection system 1406. In this case, data processing system 102 of Fig. 1A includes a virtual loop detection process 139. The process defines a virtual loop or a trigger line in the field of view that is continuously recorded by the video camera. When the vehicle is imaged in this virtual loop or on this line by a video camera in time not allowed by signaling or traffic lights, the clock 1408 starts. Digital image processing techniques can be used to define the virtual loop and detect the presence of the vehicle in this wrong video area at the wrong time or speed.

[0083] Fig. 17 illustrates vehicle detection by means of a virtual video loop according to one embodiment of the present invention. The example of Fig. 17 illustrates four separate video data frames 1700, 1710, 1730, 1730. The field of view of the video camera shows the area near the pedestrian crossing at intersection 1704 and traffic lights 1706. Car 1702 entering the intersection on a red light is visible. By means of digital image processing techniques, a virtual loop 1708 is defined or drawn in the intersection area, such as before a pedestrian crossing 1704. By using the 1708 virtual loop, it can be determined whether a 1702 car entered the intersection at the wrong time, i.e. when the light 1706 was red. The complete coverage of loop 1708 by car 1702 when the light has been red for a certain period of time, as shown in Box 1710, may result in the detection of an offense. At this point, the clock is started as illustrated in steps 1506 and 1508 in Fig. 15. It should be noted that depending on the layout of the monitored place and the capabilities of the camera and processing systems, one or more virtual loops can be defined in different places in relation to the pedestrian crossing (e.g. pedestrian crossing 1704).

[0084] Also as shown in Fig. 17, the header of frame 1709 is displayed at the top of all frames. As illustrated in Fig. 1A, the data processing system 104 may include a frame editor 133 that is separate from the direct connection between the camera system and the central processor. This frame editor allows the system to stamp each video frame with certain identification information or relevant facts. These may include time and place, light duration, car speed, direction of travel and other similar information items. Video frame information can also be used to determine certain facts about the incident, such as vehicle speed and possible acceleration or deceleration while driving through the site, using frame frequency and time information. For example, if the video clip lasts 12 seconds and the video camera spins 28 frames per second, the resulting kllp will contain 200 frames spaced 60 milliliters apart. As shown in Fig. 17, frame 1700 was filmed at 12: 59: 000, frame 1710 at 12: 59: 060, frame 1720 at 12: 59: 120, frame 1730 at 12: 59: 180, and so on This time information can then be used to determine the speed and acceleration of the vehicle using known distances for the site.

[0085] By correlating header information stamped on video frames with information associated with each still image of the event, a closely related set of evidence consisting of still image and video data can be combined and generated. Alternatively, in embodiments where a single video camera is used without cameras for the intersection camera, info ("the stamp allows the use of individual frames as still images, provided the video camera resolution is high enough to ensure readable identification of. To ensure the integrity of image data that is provided to authorities, the frame editing functions in the frame editor 133 can be limited to stamping data only to prevent unlawful manipulation or modification of real raw video data.

[0086] The detection system 1406, in a physical or virtual embodiment, can be used to run both video cameras 122 and digital cameras 120 for a system in which both types of cameras are used. When an offense is detected, the camera or cameras take a series of still pictures and the clock / video recording device process is carried out for the video camera material.

Data processing system [0087] As illustrated in Fig. 1A, images captured by the intersection camera system 102 are processed in a system data processing system 104. The data processing system 104 includes a central processor 132, main image file server 134, verification module 138, provisioning module quality 140, database 136 and notification printing module 142. In general, data processing system 102 largely processes digital still images provided by local cameras 102. Video cluster data provided by video cameras 122 is mainly provided to provide background contextual data for the moments of the incident to help viewers determine are there mitigating or aggravating occasions. The video camera therefore records the material both before and after the offense is detected. This provides law enforcement with a more complete record of the events leading to and following the offense, thus helping to better see the context of the offense. For example, video footage may show that a car has entered an intersection to give way to an emergency vehicle or police vehicle in response to an emergency, or that the car has been involved in a collision before or after entering the intersection.

[0088] The central processor 132 executes a main software program that implements a traffic violation monitoring and reporting system. Central processor 132 is designed to manage remote camera systems and receive, via modem, their incident data and

- 19 picture information. The central processor includes its own database for recording camera system information, but also sends information to main database 136 in data processing system 104 for each detected incident or test shot.

[0089] Fig. 6 is a flowchart that illustrates the steps that are performed by the central processor 132 when incident information is received from digital cameras of the intersection camera system 102 according to one embodiment of the present invention. At step 602, four images in the appropriate digital format (e.g. GIFF, TIFF or JPEG format) are placed on the main image file server 134 in an area that is regularly archived and which is read-only for verification users. These images are the main evidence that is digitally signed to prevent subsequent undetected manipulations. The four images typically contain two scenery images, a driver's face image, and a license plate image.

[0090] In step 604, compressed JPEG images are made of two scenery images. The incident record is then placed in the main database 136 with associated records containing two compressed scenery images and the address path of the face and table images in TIFF format in step 606. The incident record is assigned a unique incident number that is used to associate it with all other associated records throughout his entire life.

[0091] The verification module 138 within the data processing system 104 allows qualified operators to verify that all legal and task rules regarding the incident have been met by captured images and data. That is, operators verify that the incident is a legally justified offense and that the driver can be easily identified. In one embodiment of the present invention, when the user logs on to the verification module 138, he is presented with a screen which contains five main information areas. Fig. 2 illustrates the screen of the verification module for an exemplary incident according to one embodiment of the present invention. [0092] Incidents are queued to the verification station by incident number so that the oldest incidents are always processed first. Many verification application screens are also used in later processing applications, which may include quality assurance, hold queue, police queue authorization and offense viewer.

[0093] When the incident is loaded for the first time, the screen area 206 will display an enlarged view of the table. The user can then select command 208 to see an enlarged face shot. When displayed for the first time, the uncompressed TIFF images will be loaded from the file server using the stored image paths.

[0094] It should be noted that after verifying the incident, subsequent processing steps that use these images will load a compressed JPEG version of the images that have been placed in the database. This technique essentially improves system speed and keeps database file sizes to a minimum at the cost of a slight loss of image quality after the verification step.

[0095] To enable easier recognition in later processing steps, areas of interest for both the array and face shots can be enlarged by the verifying user. For this function, zoom control is provided. This control allows zoom, critical image review and allows you to adjust the intensity and contrast. The zoom control for face shots has an additional masking function to allow masking the identity of any passengers in the vehicle for privacy reasons. The enlarged images are used for all processing steps after the verification step. It should be noted that the main evidence images are not modified, only the compressed JPEG images that are stored in the database are processed.

[0096] When the incident is loaded for the first time, the main screen area 212 of the verification screen area will display the "A" scenery shot. The user may click button 218 to see the "B" shot. These images will be displayed in JPG format and loaded directly from the database. Shot A is taken when the vehicle crosses the stop line and shot B is taken after the vehicle has entered the intersection. As illustrated in Fig. 2, shot "B" is displayed.

[0097] In Fig. 2, the screen area 210 is the detail data block area. This area displays a representation of the incident details obtained on site and the incident number assigned to the details when the incident was inserted into the database by the central processor. Each image captured by the system has a data bar 212 at the top of each image to provide an additional level of security. The information in data block 210 must match the information in data bar 212. This ensures that images are not allocated incorrectly.

[0098] The image of Fig. 2 also includes a detail data area from the Department of Motor Vehicles (DMV) 216. In this area, the user enters the details of the vehicle license plate from the incident and performs the plate search in the DMV database. In general, the DMV search consists of many automatic stages, including searching the vehicle's registration number to get details about the registered owner (s), searching for the driver's personal details to retrieve the driver's driving license number for the registered owner obtained from the first search and Searching for the driver's driving license for full details of the driving license.

[0099] After a successful search, the detail data area of ​​DMV 216 of the verification screen of Fig. 2 will display some downloaded information. Fig. 7 illustrates the detail area of ​​DMV in more detail. The license plate and vehicle information is displayed in the upper half of the screen area 700. The name and address of the driver or company, if the vehicle is owned by the company, is displayed in the area of ​​screen 704 and information about the driving license of the driver is displayed in the area of ​​screen 706.

[0100] If any of the DMV search steps fail, the DMV search screen may be presented to the user. Fig. 8 illustrates a DMV search screen according to one embodiment of the present invention. The DMV 800 search screen allows the user to perform each of the three search steps in an incremental manner. The user has the option of entering various elements of information, such as the vehicle's registration number (license plate), personal details about the driver or the driver's license number. The vehicle registration number is entered and displayed in the 802 screen area, the vehicle details are entered and displayed in the 804 screen area, and the driver details are entered and displayed in the 806 screen area.

[0101] Use of the DMV search screen may be necessary if multiple records are obtained for either search by registration number or by personal details, i.e. if more than one owner has been registered for a hub vehicle if more than one person has the same name . The DMV search screen can also be used to change a user-defined search criterion in case the obtained owner records have been

-21 somewhat damaged, for example if the number "0" was placed in the name instead of the letter "O".

[0102] The detailed data obtained about the alleged offender will be forwarded to the appropriate fields in the lower half of the DMV 800 search screen when the user clicks the "Accept" button on the verification screen of Fig. 2. The user may perform multiple searches if he is not satisfied with the first results obtained . Each DMV search is logged for a given user and stamped with date / time. The search log can be made visible. [0103] This area in the lower right of the verification screen of Fig. 2 shows the buttons 218 corresponding to the different ways in which a user incident can be processed, i.e. how the incident status should be updated.

[0104] The user may click the "Pause" button to postpone the incident if there is insufficient information to accept or reject the incident. To postpone the incident, the user must also select the reason for the suspension from the displayed reasons for the suspension. The most common reason for the suspension is that the vehicle has no state registration. In this situation, the interstate search process must be completed.

[0105] If the user decides that the incident is not a valid offense or for some other reason cannot be issued to the alleged offender, the incident may be rejected using the 'Reject' button. In this case, the user will be presented with a rejection reason form to select the reason in the same way as for the reasons for the hold. [0106] The user may decide to resume the incident, which will remove all magnifications, masks and also clear all DMV details that may have been obtained. If the incident resumes, the incident history will reflect this and DMV searches will also be logged. The last option is to accept the incident as valid.

[0107] After one of the four choices is selected, the next incident will be displayed and the process will be repeated. The user will be able to view the incident history to provide the date and new comments for the incident.

[0108] In one embodiment of the present invention, the DMV 800 search form is also available from other applications. For example, the form may contain an interstate queue application, so that when another state issues information upon registration request sent to it, the user can enter registration details about the incident. This area of ​​the form can also be editable in the hold queue application once the incident has been 'verified' to get details about the name and address from the obtained DMV data about the registered owner. Usually it will not be editable in the hold queue application once the incident has been verified, i.e. when the incident has been suspended by the quality assurance module.

[0109] The screen illustrated in Fig. 2 may include a sub window that allows watching the video clip with an offense. After requesting access and playing the video clip, the system displays the video obtained from the video recording device. This usually includes a short video clip showing the circumstances of the offense, including kiika seconds before, during and after the offense. This allows the viewer to view the circumstances surrounding the offense.

Quality assurance process

[0110] The data processing system 104 of Fig. 1A also includes a quality assurance module (QA) 140. In one embodiment, the QA module uses the same user interface as the verification module, illustrated in Fig. 2. In the QA module, the user has no any image editing capabilities and may not change any of the details of the vehicle or alleged offender or perform DMV searches. All incidents that have the status "Accepted by verifier" or "Accepted by Operator Paused as verified" will be available for quality assurance. The system tracks users who are logged in to the GA module and does not queue any tasks to them that they "verified", whether in the verification application or the hold queue application.

[0111] When the quality assurance session begins, four images (table, face, scenery A, scenery B) in the compressed JPEG format are loaded from the database 136. The displayed table and face images are those that were processed in the verification step 138. At the beginning scenery A and enlarged shots of the board are displayed. The detail data block area is then filled in and the current status of the incident is displayed.

[0112] The user assesses the incident presented and can accept, reject or suspend the incident. Acceptance updates the incident status to "Accepted by verifier and QA". Rejection of incidents leads to displaying the reason for rejection form. The user selects the reason and confirms the incident status update to "Killed" (rejected). The user will be recorded in the log as the QA operator of the incident. No further action will be taken on this incident. [0113] If the user selects a hold, the reason for the hold is displayed and the incident status is updated to "Accepted by verifier, held by QA". The user will be saved in the diary as the QA operator of the incident. Since the incident was suspended by OA, the system will flag this condition and will prevent editing of the incident in the hold rail application, i.e. only incidents that have been held by the verification application are editable in the hold queue application. Being editable means being able to process face and whiteboard shots, perform DVM searches, or be able to edit details of an alleged offender on the DMV search screen.

[0114] In one embodiment of the present invention, the data processing system 104 includes a hold queue application. Incidents that may be important but require further clarification are queued in this application. The application starts with the display of the main hold queue screen, which presents a list of all held incidents that can be processed by the current user. The user can click on any item from the list and then click the appropriate command to display the same screen that is used by the verification application. Incidents can be suspended either in the verification module 138 or in the quality assurance module 140. Once the problem is resolved for the incident, the operator can then advance the incident by accepting or rejecting it. If the incident has been suspended during the verification phase, then the suspend operator becomes the actual verifier.

[0115] In one embodiment of the present invention, the data processing system also includes an interstate queue module. This module looks and works in the same way as a hold station that deals with other held incidents. For this application, the registration list can be printed or faxed to another state registration authority so that they can provide details by return fax. This would normally be done after the search filter has been introduced to list only those incidents from one jurisdiction that have not been evaluated. The user can then update the details of the incident by finding the appropriate incident. The incident can then be postponed to QA as usual.

Police Interface Modules [0116] The traffic violation monitoring and reporting system 100 of Fig. 1A also includes an interface to one or more police departments 106. Data processing application 104 provides police department 106 the ability to choose one of three modules. These are the police authorization module, offense viewer module and police reporting module.

[0117] An exemplary structure of the police screen interface screen of the authorization module is illustrated in Fig. 9A. The 900 interface screen provides a list of 902 incidents sorted by date and time with license plate numbers for offenses. All incidents accepted as valid by QA verification and process will be listed in (configurable) batches on the main screen of the police authorization application. The incidents will be listed creating a batch sorted by date and time of the incident, thus the oldest incident will first be presented to the police.

[0118] Relevant police officers will be able to view the details of individual incidents by selecting them and clicking on the appropriate command button, such as the 'show details' button 904. They will be represented on a non-editable screen, similar to the verification screen in Fig. 2. They can accept or reject a single incident from this screen. To maintain data integrity, the police will not be able to suspend the incident or view or enter commentators.

[0119] The user (police officer) assesses the incident and may decide to accept, reject or not carry out any action by canceling the incident assessment. If the user decides to accept the incident, the status of the incident is updated to "Ready for Notification Processing" in database 136 and the user is directed back to main llsty 902. If the user decides to reject the incident, the incident status is updated to "Killed" and the user is directed back to the main list 902. The incident is saved to the database as rejected by the police and the reason is recorded for reporting and checking purposes. No further action will be taken on this incident if the user chooses to cancel, the incident status will remain unchanged and the user will be directed back to the main list. [0120] For an authorized officer, it may be possible to review each incident on the list and take action on each of them individually, or he may at any stage return to the main list and decide to accept all other incidents listed on the list by selecting the "Accept All" function.

[0121] Within the police authorization application, the offense viewer module displays incident images for incidents that have been confirmed as offenses. This module will also be protected and only authorized police officers will have access to it. The user will use either the notification or vehicle registration number or the incident number as a search filter.

[0122] After entering the search parameter and performing the search, the system will display four incident images, a block of detail data and DMV detail data. Additional searches can be made from the main screen in the same way as the first search.

[0123] The police reporting module within the police authorization application allows the execution of reports for police activities. The police can then use these reports to prepare notices for the offender and similar activities. Available reports are presented as a list and can be viewed using the user interface of the police authorization application.

[0124] The police authorization application may also include a perpetrator notification report that presents the perpetrator reports in a list. The interface dialog window can prompt the user for the number of days and then display the report. The report will contain all notifications for which payment is delayed by the selected number of days.

[0125] An element of the dismissal report may also be included in the police authorization application. This report lists all notifications that were canceled because they were not processed within the time limits or due to appointment. Appointment occurs when an alleged offender appoints another person as a driver during an incident. In both cases, the previously issued notice must be removed from the court file. This report can be used as a list sent to the court asking you to dismiss canceled notifications.

[0126] The police authorization application also includes a notification module that enables the police department to issue and view notifications to appear to be issued to offenders.

[0127] Fig. 9C illustrates a police authorization viewing interface that can be used by police officers to view photos and video of an incident. As illustrated on screen 950, a specific incident can be selected from the list of incidents 952. Incidents can be sorted and searched using the appropriate input functions 954 and 956. Information about the incident is also provided in the area 958 of the screen. The main area of ​​the screen has four separate windows. Windows 960 and 962 show two still pictures of a place from different angles or at different times, and window 964 displays a license plate or other identification (e.g., driver's face) of the vehicle. Each still image can be a photo provided by each of the many cameras on site or they can be images from any camera taken at different times. Window 966 displays the incident video clip recorded by the video camera. The video clip is usually read by selecting the watch video 968 command. The screen in Fig. 9C has above all HL ^^ tri ^ 'into one possible arrangement of police authorization and browse screen, but many different layouts are possible. For example, the video window may be provided as a pop-up window on the main screen or it may be displayed in full screen to allow the operator to view the details of the video class.

Court interface [0128] The traffic violation monitoring and reporting system 100 also includes a court interface module 110 that allows the user to electronically submit detailed notification information to courts and then receive updates on the status of the notification from the courts. In one embodiment, this process is automatically managed by a third party scheduler by executing database scripts.

[0129] Fig. 9B illustrates a court interface screen generated by the court interface module 110 according to one embodiment of the present invention. The court interface screen 950 includes a screen area 952 that shows a list of notifications that have been approved and are ready to be sent to alleged offenders. The court interface screen 952 also includes a 954 screen area that allows access to files or documents received from the court. They may contain confirmed notices and decisions regarding notifications processed by the court. The 956 text display area may be provided to display messages associated with any incidents in the list in the 952 display area.

[0130] The manual court interface module can also be provided as a spare module if the automatic system fails or if unplanned actions are required. The manual court interface module allows you to start the following stages: generation of notification records from newly approved incidents of offenses, sending details of new notifications, receiving confirmations (editing report) of sent files and receiving weekly rulings. The database packages that are executed for each of these functions can either be started manually by clicking the interface command or automatically from a third party scheduler by executing scripts stored in the database. For each function, the function details are stored in a time stamp record in the log table with a unique session log identification number. The number of records processed or any errors encountered are also stored.

Creating a Notice [0131] In one embodiment of the present invention, the function of creating a notice is initiated either by the scheduler or appears automatically when the manual court interface screen is selected. Notification records are created by the notification print module 142 for incidents that have been authorized by the police. FIG. 10 is a flowchart that illustrates the steps of creating a notification according to one embodiment of the present invention. At step 1002, all traffic incident records that have the status "Ready to Process Notification" or "Ready to Process Warning" are identified.

[0132] For each incident found, the age of the incident is checked at step 1004. If it was found at step 1006 that too much time had passed since the incident occurred, the incident is rejected because it is too old to be issued at step 1008 This usually occurs because, depending on the jurisdiction, notifications must usually be sent to the alleged offender within a specified period of time (e.g. 15 days) from the date of the offense, date of updating the details of the address or date of appointment.

[0133] For every incident found that is in the allowed period, an offense notification record is created and a challenge number is assigned in step 1010. The notifications created will now have "New" status if the status was "Ready to process notification" or "New alert letter" if the status was "(Ready to process alert". A related offender record and offender address is created to store personal details and the address of the owner who was selected during the incident verification process.

[0134] Once the appropriate notices have been created, notices may be sent to the court. This function can be initiated either by the scheduler or manually by selecting the "Create Notification File" command from the 950 court interface screen. For this process, the system first searches all notifications with the appropriate status (e.g. New) and excludes all those that are behind old. The details of notifications are saved to a new export file (with a predefined name and location) in a format that is suitable for the court system. Notices that are too old have their status updated to "Send to police for dismissal." Other notifications will have their status updated to "Send to court". The system can display the number of notifications that have been updated to 'Send to court' and 'Send to police for dismissal'.

[0135] The export file created may have the text "ONLY EDITOR" in the header to indicate that the file must be checked for syntactic errors by the court system and that the edited report is to be created by the court system to serve as acknowledgment of receipt. The file processing procedure in the court system is initiated by a modem connection, which can be operated by the scheduler or manually by the operator.

[0136] If a notice is to be given to the perpetrator of an offense by a third party, non-judicial and non-police authorities, the court must acknowledge receipt of the notification before the third party prepares its printout and sends by post to the alleged offender. The notification processing module of the data processing system 104 provides a user interface screen that creates a list and previews the notifications to be printed. Such a preview of the notification is illustrated in Fig. 11.

[0137] In one embodiment of the present invention, printing the notification involves major steps. First, the current user is saved as the issuing user in the notification record and the notification status is updated to "Notification Printed" or "Warning Letter Printed", respectively. Files with two scene images, an enlarged image of the board and an enlarged face image, an image of the signature of the authorizing police officer and an image of the issuing user's signature are copied from database 136 to the data processing directory as image files (such as .jpg files).

[0138] The document is then viewed on screen to ensure that all images have been downloaded and later the document is printed on a printer. Note that a preview of a document that has not yet been printed may not display the details of the person issuing the notice, as it has not yet been issued.

[0139] Fig. 11 illustrates a notification preview displayed on the user interface screen according to one embodiment of the present invention. The following details appear on each notice to appear: name and address of the alleged offender, details of the incident, four images of the incident in the form recorded by the verification operator, place of the incident, time and date of the incident and information on payment of the fine. Also included is a section where the alleged offender may fill in the details of the person he or she may want to appoint as the driver of the vehicle at that time, and information about what the alleged offender wants to do if he or she disagrees with the prosecution. The notification may also contain a scanned signature of the police officer who authorized the incident to issue the offense and a scanned signature of the person who issued the notification.

[0140] Depending on the computer implementation, the report view function may also allow the user to manipulate the notification file, for example, printing the notification on the selected printer or exporting the notification to an HTML file or text file.

[0141] In one embodiment of the present invention, the alleged offender may claim to be innocent and then appoint another driver. There are two ways a person can do this. First, the notice of appearance has a part that a person can complete and return to the party that issued the notice, or the person can complete a certificate of innocence at the police station, and the police will forward it to the issuing party.

[0142] The data provided by the traffic offense monitoring and reporting system is legal evidence that can be used to find guilty offenders. In one embodiment of the present invention, the evidence kit includes a copy of the notification to appear in addition to other documents that are not necessarily created by the system. Such documents may contain information provided by the court, a chain of evidence attesting to the integrity of the images concerned, and a technology ruling.

Expert system for image analysis [0143] In one embodiment of the present invention, an image analysis system is implemented that automates data processing components. Image analysis is the process of discovering, identifying and understanding patterns that are relevant to the operation of image-based tasks. One such task is the ability to automatically locate and read information on the license plate in the evidence images. Here, an interesting pattern are the shapes of license plates and alphanumeric characters. The purpose of image analysis is to automatically locate these objects and perform character recognition with the operator's human accuracy.

[0144] An advantage of the image analysis system in the data processing system verification process would be that all details about the vehicle, owner and incident could be provided for visual verification, all filled out initially and thus requiring less or no manual data entry.

[0145] Image analysis elements can be categorized into three basic areas, low level processing, medium level processing and high level processing. These categories form the basis of the skeleton when describing various processes that are inherent components of an autonomous image analysis system.

[0146] Low level processing deals with functions that can be seen as automatic responses that do not require any intelligence on the part of the image analysis system. This classification will include compression t /! Ub image conversion, such as the use of a typical set of filters for image processing.

[0147] Medium level processing deals with the task of obtaining and characterizing components or zones in the image for lower level processing. This classification involves segmenting the image and describing it, i.e. extracting, obtaining, and categorizing objects within the image.

[0148] High level processing refers to the recognition and interpretation of the objects obtained. The use of intelligent behavior is most evident at this level because it attracts

- the ability to learn by example and to generalize this knowledge, so that it can be applied in new and different circumstances.

[0149] Image analysis systems using expert system technology can be used to accurately identify, obtain and translate interesting areas reflected or appearing on images recorded by the camera surveillance system of Fig. 1A. In general, technology requires the acquisition of knowledge through the process of obtaining, structuring and organizing knowledge from a single source so that it can be used in software. There are three main areas that are most important for knowledge acquisition that need to be considered when developing an expert system for image analysis. First, the field must be assessed to determine if the type of knowledge in the field is appropriate for an expert system for image analysis. Secondly, the source of knowledge must be identified and evaluated to ensure that the specific level of knowledge required by the expert system for image analysis is provided. Thirdly, specific knowledge acquisition techniques and participants must be identified.

[0150] The purpose of the expert system for image analysis is to accurately identify, obtain and translate optical data appearing in the photographic evidence acquired by any type of camera surveillance system.

[0151] Many film based camera systems optically reflect misdemeanor information on each photo. For example, the reflection of a camera speed supervision system on each image; information such as the measured speed and direction of travel of the offending vehicle, the speed zone and location monitored by the camera, the identifier of the operator supervising the implementation and the time and date of the offense. The process can also be used to identify and obtain vehicle registration plate details that can be used to identify the owner of the offending vehicle.

[0152] The knowledge base of an expert system for image analysis can be derived from a set of sources such as manuals, operating instructions and simulation models, although the basic knowledge comes from expert people. Expert people themselves do not have to be a source of technical information, but they can be operators or system users who make decisions based on known task processes rather than based on technical issues. This type of knowledge requested indirectly obtained by these experts provides a useful source for basic knowledge.

[0153] Knowledge acquisition includes several processes and methodologies for acquiring, identifying and obtaining knowledge. Although, in principle, knowledge is obtained from experts who provide a static core or benchmark, an expert system for image analysis can obtain its own dynamic knowledge by identifying trends or common motives, in fact derived from its own experience. The system achieves this ability through a unique feedback and tracking mechanism provided by the data processing system 104. The system has the ability to determine whether the information provided is correct in a relatively short time (in some cases, immediately using appropriate validation functions that can be included in the data obtained, such as a checksum). However, in traditional expert systems, the information obtained is based on the conclusions drawn from the input data set without any mechanism for checking the result, therefore, if the same input data is entered into the expert system, the same conclusions will be drawn. In traditional expert systems, knowledge acquisition is usually achieved

- by observing an expert solving real problems, through discussions, by building scenarios with experts who can be associated with different types of problems, creating rules based on interviews and solving problems using them and other similar methods. In addition to these methods of acquiring knowledge, an expert system for image analysis can also extract knowledge from the knowledge obtained, obtained through a certificate of verification of verification processes and rulings, allowing more than one result for the same set of input data, gaining access to external or other indirect input sources available in the field of the problem and other similar methods.

[0155] The image analysis expert system and image computer are the main components of the image processing system used in the office of the traffic camera system using the automatic offense processing system. The image computer provides the system with all information about the offense in electronic form, required when issuing a notification of the offense.

[0156] In the event of speeding, the image processing system will provide two digital images of each offense, one being the representative low resolution version of the original image, the other being the high resolution image only extracting the license plate area. In addition, textual details of the offense appearing in the acquired image are obtained using the optical character recognition (OCR) process.

[0157] Fig. 5 illustrates typical output of an overspeed monitoring camera provided by an image processing system according to one embodiment of the present invention. In Fig. 5, the resulting screen 500 contains several different image areas. The offense image is displayed in the 502 display area. An approximation image of the vehicle license plate is shown in the 504 display area and the details of the offense are displayed in the 506 display area. This information is checked and confirmed by two separate manual processes prior to the actual offense being issued. The traffic camera office offenses processing system typically includes a high-speed film scanner that provides images for computer images for processing under the control of a file arbitrator. Infringement information is automatically obtained by the image computer and placed in a database for manual verification and adjudication at the verification station.

[0158] Fig. 12 illustrates components of a traffic camera office offenses processing system in accordance with one embodiment of the present invention. Also in Fig. 12 are components which are included in the image processing system.

[0159] Unprocessed digital images of offenses are obtained either directly from digital field cameras or from a 35mm scanned digital film. File arbiter 1202 provides serial access to raw data of the offense. The image computer 1214 within the image processing system 1210 performs the main tasks of image analysis and is the basic interface between database 1208 and raw 1216 digital images. Verification Station 1206 provides a mechanism for visual manual determination of actual misdemeanor and information provided by the image processing system 1210. If the information provided is correct and the misdemeanor complies with all relevant task rules, then the misdemeanor is issued to the vehicle owner.

[0160] Surveillance station 1204 is used to check for any violation that may be rejected during the process of verifying and adjudicating task flow in the traffic camera office. Database 1208 may be a relational database, such as the Ingress ™ relational database system running on the UNIX ™ operating system on the HP-9000 ™ platform. It provides a central repository for all data containing data and images of offenses, audit certificate and archiving.

[0161] In one embodiment, the expert image analysis system 1220 provides an image processing system 1210 with human-like expert behavior, thereby equipping the image computer with essentially artificial intelligence to solve problems efficiently and effectively. [0162] Regardless of the type of surveillance, all offense images are returned to the traffic camera office for processing including all details of the offenses in electronic form and the camera settings and implementation log required by the operator to be able to respond. The speed camera settings and implementation log contain useful information about the actual implementation conditions and the environment, knowledge that can support the image analysis process.

[0163] File arbitrator 1202 detects a new image file and starts the image computer 1214 to start the image analysis process. The image computer then checks the image file, obtains an image area from the file surrounding the data block (containing the details of the offenses), splits it into segments and represents characters inside the data block, renews missing or damaged characters, and translates character objects in the text using an OCR process. Then the vehicle license plate is searched for the offense. Once it is found, the area is obtained for OCR, the details of the license plate are determined including jurisdiction. A compressed low-resolution JPEG image is then created to represent the entire image, and a compressed JPEG image of a high-resolution image is only made with the license plate area. The image collection and OCR text data are transferred to the database.

[0164] After the data has entered the database, it is presented at a verification station for visual confirmation and evaluation by a trained operator. The usual process of the operator is a simple affirmation given detailed offenses automatically obtained by the computer images. After confirming this data, details about the vehicle owner are searched and presented to check content and syntax. After confirming the details of the vehicle owner, the offense data is transferred to the quality system for examination and notification of actual misdemeanor.

[0165] Analyzing the process or workflow of the traffic camera office offenses processing system reveals kiika the possibility of obtaining and inferring knowledge for an expert system for image analysis. Knowledge acquisition has been taking place since the beginning of the surveillance processing cycle, even before the film hit the traffic camera office.

[0166] For example, the speed camera settings and deployment log provide the expert system for image analysis with useful dynamic or unstable knowledge of deployment configuration and environment that can be useful in obtaining a license plate and OCR process. Information describing weather conditions, traffic conditions and direction, the number of lanes monitored and the lane of the first vehicles committed offenses provide useful information for the image processing system. Although the acquired knowledge is stored temporarily (until the full implementation is successfully completed), archival information regarding the camera and the place of implementation can also be created / updated to help establish permanent or trending (i.e. location / camera profile).

[0167] After placing the movie data in the main database, an image analysis expert system can access this data when each image computer begins processing new image files. Since the first task of the image computer is to interpolate the data block area, an image analysis expert system can provide the image computer with the best location for the data block in the image. Accompanying this knowledge will also be the use of the best acquisition process and OCR (including best performance parameters).

[0168] In the event that the processing scenario provided is unsuccessful, an expert system for analyzing images may provide information for alternative acquisition and OCR processes. Both failures and successes are recorded by an image analysis expert system, improving the knowledge base and thus the efficiency and effectiveness of image processing. Here, the knowledge of success and failure is known in real time by checking the digital data block object.

[0169] Then, the image computer begins the process of searching and obtaining a license plate. Again, an image analysis expert system can instruct an image computer to perform this process using the best-performing algorithms and parameter scenario to date. Here, feedback about the success or failure of a process is delayed because there is no automatic mechanism for success / failure (as in the case of checking a digital data block object). However, the location of the license plate can be confirmed using the implementation log (for exceeding the allowed speed) for at least several first registered offenses. Here, the camera operator is required to register, for each frame number, which lane the offending vehicle was following.

[0170] However, until the offense is viewed at the verification station, the actual image analysis performed by the image computer cannot be checked and thus the image analysis expert system cannot acquire knowledge unless the verification priority has been placed on the first kiik images of each new film or implementation.

[0171] The actual verification process may also affect the knowledge acquisition process of the expert system for analyzing images by querying the verification operator with simple questions each time a correction is made to any part of the provided offense data. Alternative knowledge can be inferred by analyzing corrections and rejecting a task rule to determine why the selected process for this particular offense was unsuccessful. [0172] Fig. 13 and illustrates the functional components of an expert system for analyzing 1220 images according to one embodiment of the present invention. The retrieval module 1302 provides a knowledge database with real-time knowledge requested / provided by an image computer, with applied knowledge received directly from a verification station or analyzed by a system of inspection certificates or with direct knowledge obtained from a database processing traffic offenses of a traffic camera office.

[0173] The knowledge provider 1304 is the main interface for image computers and will supply the image computers with the necessary information and parameters to perform the required image processing tasks. [0174] The local database 1306 serves as the central repository for all knowledge, performance statistics, short- and long-term data, and configuration parameters for image computers. The local database also serves as a storage for the neural network learning set and character pattern.

[0175] The knowledge graphical user interface (GUI) provides the user with the ability to display, modify and delete knowledge and database data. The knowledge GUI also allows you to update configuration parameters, character patterns used by the OCR process, and neural network learning.

[0176] The image analysis expert system provides an image computer with a predefined scenario or set of rules to apply to achieve a successful image analysis result. Unlike other expert systems, there are relatively few combinations of processing scenarios because there is only a limited number of ways a block of data can be obtained from the offense image. However, the expert image analysis system of the present invention is generally able to perform parameter matching used by each process or rule and therefore has the ability to learn. This is achieved by prudent changes of these parameters and by the system tracking the results.

[0177] This mechanism for fine-tuning the scenarios (or in some cases using different scenarios together) is called "sampling". Sampling is a mechanism used by the expert system for analyzing images to successfully perform tests by applying different fine-tuning scenarios or image processing parameters, to improve performance. [0178] In one embodiment, this type of operation is performed at the beginning of a new deployment or movie, and randomly for each batch. The changes are tracked by the traffic camera office offenses processing system. Information about success or failure is analyzed enabling precise system tuning in real time. Although the knowledge obtained can only be used temporarily (i.e. for the current batch), trends can be recorded and if the need is static knowledge can be updated.

[0179] With reference to the image processing system, the "scenario" is a set of image processing rules that the image computer follows to create a successful image analysis result. The mechanism by which these rules are stored and the knowledge transmitted to the image computer depends on the level of sophistication employed by the image processing system.

[0180] Performance monitoring is a method of fine-tuning or detecting poor image analysis results. The mechanism used is simply a correlation and analysis of statistics obtained from real-time data enabling precise tuning, which may be due to slight differences or incorrect implementation conditions that were not provided as part of the basic knowledge. Scenario statistics are the second type of statistical data that can be correlated based on direct scenario results and scenario variants with different parameter values.

[0181] The main component of the expert system knowledge acquisition module for image analysis is the expert system that draws knowledge from the verification station. Knowledge, such as the usual OCR errors (i.e. characters that are regularly incorrectly recognized), incorrect selection of the license plate, incorrect dynamic retrieval of threshold values ​​and other such information, is used in the inference as a result of sampling.

[0182] An important requirement of this module, in particular when tracking images in sampling mode, is the correct identification of the image itself. The common theme or key must be used by the verification module, control system, database, image computer and expert subsystems for image analysis.

[0183] Access to the main traffic camera office database, processing offenses, can provide indirect knowledge for an image analysis expert system that cannot be obtained directly from an image or verification process. For example, implementation log information or other additional film and site information provide useful knowledge for an expert system for analyzing images and image computers.

[0184] The core of the image analysis expert system contains all image processing knowledge and image computer configuration / operational parameters. The local database contains both static and dynamic data. The structure of the database may vary depending on the form of knowledge and data. Character patterns and training sets of the neural network can also be placed in this database.

[0185] Although embodiments of the present invention have been described as implemented in traffic environments associated with intersections at traffic jams and regarding a red light and stop sign, it should be noted that alternative embodiments may be implemented in other traffic environments. For example, a road traffic monitoring and reporting system can be implemented and applied along a road section to determine if vehicles are exceeding speed.

[0186] In addition, examples of implementation may include facilities for issuing multiple offenses for one incident. For example, a camera that monitors a red light with speed tracking can detect and register a vehicle that is speeding and passing a red light. The notification may be in the form of separate notifications, one for the red light offense and one for the speeding offense or one notification recording all offenses.

Image protection [0187] Embodiments of the present invention include various methods for ensuring the security and integrity of digital images obtained at a target intersection. In one embodiment of the present invention, public key cryptographic techniques are used by the functionality of the digital imaging camera system. The original evidence of the offense is encrypted at the point of capture in the video camera system 102 of Fig. 1A. Because every pixel inside the CCD is unloaded outside the module, they are converted to a digital stream and encrypted in real time, preserving its original, unprocessed form. Using this process at this early stage eliminates the need for special purpose peripherals for storing, transmitting and manipulating data.

[0188] In one embodiment of the present invention, variants of known public key and secret key encryption systems are used to implement the digital cryptographic envelope for the digital traffic camera system. Each camera system is assigned a unique digital certificate, which is reproduced and whenever there is any change in the system. The certificate determines the relevant system details including the camera serial number and provides an identifiable public key for the camera system. This public key is used later

-34 to identify a specific source for each set of evidence that reaches the data processing system.

[0189] When the offense takes place, the camera system collects relevant evidence, which consists of many elements or 'properties', including various image files, speed data, offense time and the like. The camera system then uses all the details of its current unique digital certificate to create a hash function by applying proven public key cryptographic hash algorithms. The hash function is a one-sided equation that is used to 'sign' each property of an offense that occurs with its own unique digital signature.

[0190] The camera system then places each of the offense properties signed into the offense database and places it in the system server outbox (using, for example, the Microsoft ™ Message Queue server outbox). The server's outbox then divides all the information in the offenses database into smaller, easier to send packets, or 'mini-envelopes' of information. It then applies a different unique digital signature for each packet (using the above public key techniques).

[0191] When there is remote communication such as telephone, ISDN, optical fiber and the like between the location of the camera and the data processing system, the signed packets can be electronically forwarded via the Internet for processing using a virtual private network. In one embodiment, the data processing system server secures the transmission process using IPsec, a standard Internet protocol that is widely used to protect electronic transmissions over unprotected public networks.

[0192] When there is no remote communication with the camera location, the signed packages can either be copied to portable media (e.g. disks) for physical transport to the data processing system or copied to the operator's portable computer for transfer to the system.

[0193] Each signed packet is received in the data processing system by the data processing system server outbox, which decrypts the mini-envelopes of the packages and automatically checks the authenticity of their signatures. The original offense database is then reassembled from its various signed properties to restore the original offense file.

[0194] The unique digital signature on each property is then authenticated to identify the source of the property (thereby specifying the camera that originally acquired the evidence) and verify the integrity of that property (by confirming that its original digital signature is intact and unchanged). The original properties with their intact, authenticated digital signatures are then stored as the original database (i.e. main evidence) for the offense.

[0195] The data processing system then selects the data and image elements required to process the lawsuit, copies them and runs on copies. Original files with their intact, authenticated, digital signatures are stored separately as protected primary evidence of the offense. From now on, every access or access attempt is logged in the control chain, so that the entire lifetime of data related to the offense can be fully explained.

[0196] Any files with encoded signatures indicating that the evidence is forged or modified are not sent for processing. Processing can only be continued if the evidence has been confirmed as authentic. This encryption and authentication system is useful when implemented in jurisdictions that allow you to provide digital proof.

[0197] The use of digital signatures for traffic law surveillance to authenticate an offense provides a way to secure data integrity that is independent of the medium on which it is stored and / or through which it is transmitted. The process provides a mechanism for identifying the source of the data (i.e. camera system) and legality. [0198] As illustrated in the drawings of the present application and described herein, embodiments of the present invention may be implemented on one or more computers executing software instructions. According to one embodiment of the present invention, server and client computer systems send and receive data over a computer network or a regular line? telephone. The steps for accessing, downloading and manipulating data, and other embodiments of the present invention are implemented by central processing units (CPUs) on server and client computers executing sequences of instructions stored in memory. The memory can be random access memory (RAM), read-only memory (ROM), non-volatile memory such as a mass storage device or any combination of these devices. Execution of the instruction sequence causes the CPU to perform the steps according to the embodiments of the present invention.

[0199] The instructions may be loaded into the server or client computer's memory from the mass storage device or from one or more other computer systems via a network connection. For example, the client computer may send the instruction sequence to the server computer in response to messages sent to the client over the server network. When the server receives instructions over a network connection, it places the instructions in memory. The server can store instructions for later execution or it can execute instructions when it arrives over a network connection. In some cases, downloaded instructions can be directly handled by the CPU. In other cases, instructions may not be directly executed by the CPU and may instead be executed by an interpreter that interprets the instructions. In other embodiments, the hardware assembly may be used in place of or in conjunction with software instructions to implement the present invention. Therefore, the present invention is not limited to any specific combination of hardware and software or to any specific source for instructions executed by the server or client computers.

[0200] Above, a system has been described for automatically monitoring and reporting traffic violations that includes both still images and video data. Although the present invention has been described with reference to specific exemplary embodiments, it is obvious that various modifications and changes may be made to these embodiments without departing from the broader scope of the invention as set forth in the claims. For this reason, the description and drawings should be understood in an illustrative sense and not in a restrictive sense.

[0201] To the extent that the above-described embodiments of the invention are implemented, at least in part using a software-controlled data processing device, it should be noted that a computer program providing such program control and transmission, memory or other medium through which such computer program is provided, are understood as embodiments of the present invention.