Electronic vehicle indentification

TECHNICAL FIELD

This disclosure relates to electronic vehicle identification.

BACKGROUND

Communication facilities such as roads, bridges and tunnels generate road tolls, often the main source of income for many states and cities. A large number of cars, trucks and buses stopping at payment booths daily to pay a toll can cause significant problems. For example, these objects can restrict the flow of traffic causing traffic jams and lane changes, often increasing the likelihood of accidents and more often bottlenecks. In addition, many people may have delays in reaching their destination, and goods may be delayed in delivery on the market, and millions of gallons of fuel may be wasted because vehicles are idling. The environment may experience increased pollution as idle and slow moving vehicles emit pollutants (especially carbon dioxide and carbon monoxide) that can pose a significant threat to the health of drivers as well as payment booth operators.

Some payment booth systems may have a program that requires the driver to rent and then attach a radio transponder to the windshield of the vehicle connecting via radio frequency to the receiving units in the payment booths. However, such programs require drivers to search for the program and register it. These programs may require drivers to make a deposit with a credit card and execute automatic debit account instructions, which can effectively eliminate drivers with credit problems. These programs can also charge the user based on the minimum number of trips, regardless of the actual number of trips. For this reason, many drivers who travel infrequently on the toll road may have little benefit from investing time and money in participating in the program.

One problem with existing technology for vehicle identification is therefore that a transponder unit is required for each vehicle to be identified.

US 6747687 describes a vehicle entry and exit time system in which the passage of a vehicle through an entrance generates a "trigger z-tO" . The video camera at the entrance provides real-time photos of entering vehicles. The trigger-tO signal causes one video frame of the vehicle to be stored or "frame caught" in the entry subsystem. The entry time clock signal is used to indicate the time of the photo, which is then transmitted via a connection to the adjuster. The passage of the vehicle through the exit generates the signal "trip triggers cz-t1". The 116 video camera provides real-time photos of vehicles leaving. The t1 trigger signal causes one video frame of the vehicle to be stored or "frame caught" in the exit subsystem. The clock signal is used to indicate the time of the photo, which is then transmitted by connection to the matching. The result automatically provides the length of stay for each outgoing vehicle according to its time of entry. The match uses unique features on vehicle photos to determine compliance.

and

US2002 / 140577 describes a method of reading the registration plate affixed to a vehicle. The method includes determining whether a license plate photo is required, automatically processing the license plate photo in response to determining that the license plate photo is required, providing at least one confirmed photo, and determining whether to manually read the license plate photo by matching the license plate photo with at least one confirmed photo.

SUMMARY

Aspects and features of the present disclosure are detailed in the appended claims.

The present disclosure describes at least one vehicle identification method that allows automatic and electronic management of the payment of road tolls by vehicles passing through a road toll facility, without requiring vehicles to slow down or to have a transponder. The method may constitute at least part of the toll system. Such a system automatically identifies all or essentially all vehicles that pass through the toll object, and bills the owner of each identified vehicle for the toll incurred.

The existing technology for identifying vehicles without a transponder is reading the license plate (LPR).

The problem with existing LPR technology for identifying vehicles in the road toll system is, however, that due to the high number of vehicles passing through a typical road toll facility, this technology typically has a too high error rate for effective operation.

For example, the error rate for a typical LPR system may be approximately 1% while such an error rate may be acceptable for toll systems that only identify vehicles that are infringers, this error rate is usually too high for a toll system that tries identify each passing vehicle, not just the infringers, for toll collection. In such a system, a 1% error rate can result in a significant loss of income (e.g. a loss of 1,000 or more tolls per day).

In addition, typical LPR systems often show a balance between the number of vehicles identified (i.e., those vehicles for which the reading result exceeds the reading certainty threshold for assuming the correct ID) and the error rate. In an ideal world, this balance would be reflected in a binary certainty continuum, where the system always produces a reading certainty level of one when the reading result is correct and a reading certainty level of zero when the reading result is incorrect. In fact, however, the reading results are usually at least partially correct, and the system generates a confidence continuum having a wide range of confidence levels of, for example, from a level of one or about one (very probably correct) to a level of zero or about zero (very probably incorrect ). For this reason, the system is often required to set an arbitrary reading certainty threshold to determine which reading results will be considered valid. When the reading certainty threshold is set, all reading results having certainty levels above the threshold are considered correct, and all reading results having certainty levels below the threshold are considered incorrect. Setting the reading confidence threshold too high (e.g. to 95 or above) significantly reduces the possibility of error, but also disables many valid reading results, thus reducing income. Conversely, setting the reading certainty threshold too low (e.g. 3 or higher) increases the number of readings considered correct, but also significantly increases the number of errors, thereby increasing costs by introducing errors into a large number of accounts / bills that require a lot of time and effort to review and improvement. In the toll system, which identifies each passing vehicle, this balance is particularly problematic because it can result in a significant loss of benefit.

Furthermore, the toll system, which identifies each passing vehicle, identifies a much larger number of vehicles than a conventional toll system, which typically identifies only infringers. For this reason, such a road toll system attempts to identify each passing vehicle and is designed both to maximize income by very accurately identifying vehicles, and to reduce personnel costs by minimizing the need for manual vehicle identification and processing of account / invoice errors.

In one particular embodiment, for obtaining a lower vehicle identification error rate (and for obtaining a higher rate of automated identification), the toll system uses two vehicle identifiers to identify the target vehicle. Specifically, the toll system collects image data and / or sensor data for the target vehicle and extracts two vehicle identifiers from the collected data. The vehicle identifiers extracted from the collected data may include, for example, information from the license plate, vehicle fingerprint, laser signature, and target vehicle induction signature. In one particular embodiment, the first vehicle identifier is information from the license plate and the second vehicle identifier is the vehicle's fingerprint.

The toll system uses the first vehicle identifier to determine a set of one or more matching candidate vehicles by searching the vehicle register database and including only those vehicles associated with registers having data that match or closely match the first target vehicle identifier. The toll system uses a second target vehicle identifier to identify the target vehicle from among a set of matching candidate vehicles.

When the first vehicle identifier is information from the license plate and the second vehicle identifier is the fingerprint of the vehicle, the toll system can eliminate the problematic balance between the number of vehicles identified and the error rate typical of LPR systems by using LPR identification to identify a group of candidate vehicles, rather than for definitively identify the vehicle, and then applying a more accurate vehicle fingerprint match to ultimately identify the vehicle. For this reason, incorrect readings by the LPR system are eliminated during the final and more accurate identification of the fingerprint match. This toll system may therefore be able to obtain the most accurate identification results for a larger proportion of vehicles than would be obtained by reading the license plate alone.

In particular, the toll system gains access to registers of matching candidate vehicles and searches for one or more registers that have data sufficiently similar to the second target vehicle identifier to indicate a possible match. If no possible matches are found for the target vehicle among the set of matching candidate vehicles, the toll system may increase the size of the set by changing the matching criteria, and may again try to identify one or more possible matches for the target vehicle among the larger set of matching candidate vehicles. If no possible matches are still found, the toll system can enable the user to manually identify the destination vehicle by providing the user with access to the collected data for the target vehicle and access to databases internal and / or external to the toll system.

If one or more possible matches are found, the level of trust is determined for each possible match. If the level of confidence of a possible match exceeds the automatic trust threshold, the toll system automatically identifies the target vehicle without human intervention as a vehicle that matches the possible match. If the level of confidence of a possible match exceeds the threshold of a likely match, then the toll system presents the likely match to a human operator and allows the human operator to confirm or reject the likely match. If no automatic match is found or a likely match is confirmed, the toll system allows the user to manually identify the destination vehicle by providing the user with access to collected data for the destination vehicle and possible matches identified by the toll system, and access to internal databases and / or external to the road toll system.

In this way, the toll system typically achieves greater vehicle identification accuracy by requiring that the two vehicle identifiers be successfully matched for successful vehicle identification. What's more, the identification process can be faster because the matching of the second identifier is limited only to those vehicles with registers that successfully match the first vehicle identifier. Human operator intervention is also kept to a minimum by applying multiple confidence level thresholds.

Here, the vehicle is identified in the toll system by gaining access to the photo data for the first vehicle and obtaining the license plate data from the available photo data for the first vehicle. The set of registers is accessed. Each register includes vehicle registration plate data. The number plate data for the first vehicle is compared with the number plate data for the vehicles in the set of registers. Based on the results of the comparison of the number plates, a set of vehicles from vehicles having registers in the set of registers is identified. Access is given to the fingerprint data of the vehicle for the first vehicle. The fingerprint data for the first vehicle is based on image data for the first vehicle. Access is provided to the vehicle's fingerprint data for the vehicle in the vehicle set. Using the processing device, the vehicle fingerprint data for the first vehicle is compared with the vehicle fingerprint data for the vehicle in the vehicle set. The vehicle in the vehicle set is identified as the first vehicle based on the results of vehicle fingerprint data comparison.

Embodiments may include one or more of the following features. For example, comparing the license plate data for the first vehicle with the license plate data for the vehicles in the set of registers may include searching the vehicle register database within the range of registers that contain the number plate data that exactly matches the number plate data obtained for the first vehicle. Comparison of the license plate data for the first vehicle may further include performing an expanded search of the vehicle register database in the range of registers, which includes license plate data that closely matches the license plate data obtained for the first vehicle. The expanded search may depend on the fact that no vehicle identification registers were found that include license plate data that exactly matches the license plate data obtained for the first vehicle.

Comparing the license plate data for the first vehicle with the license plate data for the vehicles in the set of registers may include comparing the license plate data using predefined matching criteria. Predefined matching criteria can be changed to increase the number of vehicles in the identified set of vehicles. Changing the predetermined matching criteria for increasing the number of vehicles in an identified set of vehicles may depend on the failure to identify any vehicles in the set of vehicles as the first vehicle based on the results of vehicle fingerprint data comparison.

Vehicle identification in the toll system may further include capturing laser signature data or induction signature data for the first vehicle. The laser signature data may include data obtained using a laser to scan the first vehicle. The laser signature data may include one or more of: top electronic profile, number of axles of the first vehicle, and 3D image of the first vehicle.

Induction signature data may include data obtained using a loop system over which the first vehicle is passing. The induction signature data may include one or more of: number of axles of the first vehicle, engine type of the first vehicle, and type or class of the first vehicle.

Each register in the register set includes laser signature data or induction signature data for the vehicle. The vehicle identification in the toll system may further include comparing the laser signature data or induction signature data for the first vehicle with the laser signature data or induction signature data for the vehicles in the register set. Identifying a set of vehicles from vehicles having registers in the set of registers may include identifying a set of vehicles based on the comparison results of the license plate data and the results of the comparison of the laser signature data or the induction signature data.

Identification of a set of vehicles based on the results of a comparison of license plate data and results of a comparison of the laser signature or induction signature data may include determining the combined result of equivalent matching for each vehicle having a register in the set of registers and identifying the set of vehicles as a set of vehicles having combined results of the equivalent matching above a specified threshold. Each combined result of equivalent matching may include a weighted combination of a laser or induction signature matching result and a license plate matching result.

Identifying a vehicle in a vehicle set as a first vehicle may include identifying a vehicle as a first vehicle based on a comparison of the vehicle fingerprint data and the results of the laser signature or induction signature data. Identifying a vehicle in a vehicle set as the first vehicle based on a comparison of vehicle fingerprint data and comparison results of a laser signature or induction signature data may include determining a combined result of equivalent matching for a vehicle in a vehicle set and determining that the combined result of equivalent matching is above a specified threshold . The combined equivalent match result may include a weighted combination of the laser or induction signature matching result and the vehicle fingerprint matching result.

Identifying a vehicle in a vehicle set as a first vehicle may include identifying a vehicle as a first vehicle if a comparison of the vehicle's fingerprint data for the first vehicle with the fingerprint data for the vehicle in the vehicle set indicates a match having a confidence level that exceeds the confidence threshold. Identifying a vehicle in a set of vehicles as a first vehicle may include identifying a vehicle as a first vehicle without human intervention if the match confidence level exceeds the first confidence threshold and / or may include identifying a vehicle as a first vehicle if the match confidence level is less than the first level of trust but greater than the second confidence threshold and the human operator confirms the match. The human operator may confirm or reject the match by allowing the operator to see the image data for the first vehicle and allowing the human operator to interact with the user interface to indicate rejection or confirmation of the match. Identifying a vehicle in a set of vehicles as a first vehicle may include identifying a vehicle as a first vehicle if the match confidence level is less than the first and second confidence thresholds and the human operator manually identifies the vehicle as the first vehicle by accessing the image data for the first vehicle and the vehicle registry in set of registers. The human operator can manually identify the vehicle in the vehicle set as the first vehicle by enabling the human operator to access the image data for the first vehicle, allowing the human operator to access the vehicle's registry in the set of registers, and allowing the human operator to interact with the user interface to indicate positive identification of the first vehicle as a vehicle in the vehicle set. The human operator may be able to manually identify vehicles in a vehicle set as the first vehicle by enabling the human operator to access data stored in external system databases.

Identifying a vehicle in a vehicle set as the first vehicle may include identifying the vehicle by combining the vehicle identification number (VIN), laser signature data, induction signature data, and image data.

An apparatus for identifying a vehicle in a road toll system is described herein, an apparatus comprising a photo capture device configured to capture photo data for the first vehicle. The camera further includes one or more processing devices communicatively connected to each other and to a photo capture device. One or more processing devices are configured to obtain license plate data from captured image data for the first vehicle and to access the set of registers. Each register in the register set contains vehicle registration plate data. One or more processing devices are further configured to compare the license plate data for the first vehicle with the license plate data for the vehicles in the register set and to identify the set of vehicles from the vehicles having registers in the register set. The vehicle set is identified based on the results of the comparison of the license plate data. One or more processing devices are further configured to access vehicle fingerprint data for the first vehicle. Vehicle fingerprint data for the first vehicle is based on captured image data for the first vehicle. One or more processing devices are also configured to access vehicle fingerprint data for the vehicle in the vehicle set, compare fingerprint data for the first vehicle with fingerprint data for the vehicle in the vehicle set, and identify the vehicle in the vehicle set as the first vehicle based on results of vehicle fingerprint data comparison.

The above and other embodiments and features are described in detail below.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an embodiment of the electronic toll management system.

FIG. 2 is a flowchart of an example embodiment of an electronic road toll management system related to the management of tagged vehicle identifiers.

FIG. 3 is a flowchart of an embodiment of the electronic road toll management system related to toll management.

FIG. 4 is a flowchart of an embodiment of the electronic road toll management system related to toll management.

FIG. 5 is a flowchart of an embodiment of the electronic road toll management system associated with postal address verification.

FIG. 6 is a block diagram of an embodiment of the electronic toll management system.

FIG. 7 is a flowchart of an embodiment of the electronic road toll management system related to vehicle identification.

FIG. 8 is a flowchart of an embodiment of the electronic road toll management system associated with vehicle identification.

FIG. 9A-9C are a flowchart of an embodiment of the electronic road toll management system related to vehicle identification.

Similar reference symbols in different drawings indicate similar elements. DETAILED DESCRIPTION

FIG. 1 is a block diagram of an embodiment of the electronic toll management system 10. The system 10 is configured to capture the vehicle identifier 31 interacting with the object 28 and to notify external systems 34 of such interaction. For example, the system 10 may allow the toll authority to capture the vehicle identifier 31, such as information from the license plate, from a vehicle 30 traveling on the road with tolls, and then to notify the law enforcement authorities whether the captured vehicle identifier matches the license plate previously marked by the services law enforcement.

The toll management system 10 may also manage the toll on the person associated with the vehicle 32 based on the interaction between the vehicle 30 and the object 28. For example, the system 10 may capture information from the license plate of the vehicle 30 and identify the registered owner of the vehicle. The system will then provide the owner, via a communication channel such as the Internet, with an account to pay the fee or dispute regarding the fee. The toll management system 10 may send an invoice inviting page 32 to pay using a postal address that has been verified at one or more postal address sources. The system 10 may automatically capture a photo of the vehicle 30 upon activation by the interaction of the vehicle with the object. This image capture can be done using photo processing technology without the need for a radio transponder (e.g. RFID device) in the vehicle.

The electronic toll management system 10 includes a toll management computer 12 which can be configured in a decentralized or centralized manner. Although one computer 12 is shown, one or more computers may be configured to implement the disclosed techniques. Computer 12 is connected to object 28, which may charge a fee for interacting with the object. Examples of facility 28 include a toll facility (managed by a toll authority) such as a toll road, bridge with tolls, tunnel, parking facility, or other facility. The fee can be based on the interaction between vehicle 30 and object 28. Examples of interactions that may involve a fee include the distance traveled by the vehicle through the object, the period of time the vehicle is present on the object, the type of vehicle interacting with the object, the speed at which the vehicle moves through the object, and the type of interaction between vehicle and object.

Object 28 can process vehicles including cars, trucks, buses, or other vehicles. For easy explanation, the system 10 represents a single object 28 in interaction with one vehicle 30 and a person 32 associated with that vehicle. However, in other embodiments, the disclosed techniques may be configured to operate with one or more vehicles interacting with one or more objects extending at different geographical locations.

The toll management computer 12 includes a photo acquisition module 24 configured to detect the presence of a vehicle, acquire one or more vehicle photos, and send a photo (s) to the photo processing module 25 for further processing. Module 24 may include equipment that acquires photos based on the physical environment in which it is used. For example, for open road applications, photo acquisition equipment can be mounted above the road, on existing structures, or on purposely built gates. Some open road applications may also use equipment mounted in or next to the road. Lane based applications (or payment booths) may use equipment mounted on physical structures next to each road lane instead of or in addition to equipment mounted on the top or on the road.

The image acquisition module 24 may include image components such as vehicle sensors, cameras, digitizing systems, or other components. Vehicle sensors can detect the presence of a vehicle and provide a signal that activates the camera to capture one or more images of the vehicle. Vehicle sensors may include one or more of the following:

(1) Laser, sound, microwave devices - these devices commonly used in Intelligent Transport Systems (ITS) applications can recognize the presence of a vehicle and provide information on vehicle size, classification, and / or speed. These sensors can be configured to provide additional vehicle information that can be used to identify the vehicle and its use in a road toll facility, including travel time and compliance with traffic regulations.

(2) Loops - these sensors can detect the presence and type of vehicle by recognizing the presence of metal masses using a wire loop embedded in the road. Loops can be used to support more sophisticated sensors. Loops can also be used as a primary data source for vehicle detection, vehicle classification, camera launch, and delivery of vehicle signature data (e.g., based on the use of a loop system with an intelligent loop control program such as Diamond Consulting's IDRIS ® system from Buckinghamshire, United States

Kingdom).

(3) Trans-beam sensors - these sensors can emit a continuous beam across the road, and detect the presence of the vehicle based on beam breaks. This type of sensor can be used in installations where traffic is channeled into the toll booth lanes.

(4) Optical sensors - the vehicle can be recognized using cameras for continuous monitoring of road images for changes indicating the presence of the vehicle. These cameras can also be used to record photos for vehicle identification.

Cameras can be used to capture photos of vehicles and their identification features. For example, they can be used to generate a vehicle identifier such as a vehicle registration number based on a photo of the license plate. Cameras can be analog or digital, and can take one or more photos of each vehicle.

Digitizing systems convert photos into digital form. If analog cameras are used, the cameras can be connected to separate computerized digital equipment. This computer equipment may include a specialized processing device for analog-to-digital conversion or may be based on an output device installed on a general purpose computer that may perform additional functions such as image processing. Lighting can be used to provide adequate and consistent conditions for obtaining photos. Lighting may include gating lights or continuous lighting, and may emit visible and infrared light. When gating lights are used, they can be turned on by input from the vehicle sensor (s). Other sensors such as light sensors may be required to control the image acquisition module 24 and provide consistent results.

Once the photo acquisition module 24 has captured the vehicle photos, the photos can be sent to the photo processing module 25. The photo processing module 25 can be located in the same place as the photo acquisition module 24 and the photo computer 12, in a remote location, or in a combination of these places. Module 25 can process a single photo for each vehicle or multiple photos for each vehicle, depending on the functionality of the photo acquisition module 24 and / or business requirements (e.g., accuracy, jurisdictional requirements). If multiple photos are used, each photo can be processed and the results can be compared or combined to improve process accuracy. For example, more than one photograph of a rear registration plate, or photos of both license plates, front and rear, can be processed and the results compared to determine the most likely registration number and / or confidence level. Photo processing may include identifying the distinctive features of the vehicle (e.g., vehicle license plate) in the photo, and analyzing these features. Analysis may include optical character recognition (OCR), template matching, or other analytical techniques.

The toll management system 10 may include other systems capable of processing essentially in real time located at the location where the images are obtained to reduce data communication requirements. In an embodiment of local photo processing, the results can be compared to a list of authorized vehicles. If the vehicle is recognized as authorized, photos and / or data may be rejected rather than sent for further processing.

Photographs and data may be sent to a central processing facility such as an image database 14 working in conjunction with the billing device 22. This process may involve a computer network, but may also contain physical media from another computer located at the location of the acquisition (namely, the object 28). In general, information may be temporarily stored on a computer at the place where the images were acquired in case the network is unavailable.

Pictures received at a central location may not be processed. All raw photos can be taken care of as described above. Data resulting from the processing of the image (remote or central) can be divided into two categories. Data that meets the criteria applicable to the application or applicable to the jurisdiction of trust can be sent directly to the billing device 22. On the other hand, data results that do not meet the required confidence levels can be marked for additional processing. Additional processing may include, for example, determining whether multiple vehicle images are available, and independent image processing and comparison of results. This may include feature-by-feature comparisons of optical character recognition (OCR) results on a license plate photo. In another example, the photo (s) may be processed by one or more specialized algorithms for recognizing license plates of certain types or styles (such as tables from a specific jurisdiction). These algorithms can take into account the importance of characters in each position on the license plate, the expected effect of certain pattern features (such as background images), or other style specific criteria. The processed image may be sent based on the results of the pre-processing, or may include processing by all available algorithms to determine the highest level of trust.

Preliminary data can be compared to other available data to increase the level of trust. Such techniques include:

(1) Comparison of OCR processed license plate data with lists of valid license plate numbers within the billing system or vehicle registration authority of the relevant jurisdiction.

(2) Comparison of other data obtained from sensors at the shooting location (such as the size of the vehicle) to the known characteristics of the vehicle registered under the registration number recognized by the system, in recognized jurisdiction or in many jurisdictions.

(3) comparing the registration of other data to records from other places (e.g. records of the same or similar vehicle using other objects on the same day, or using the same object at different times).

(4) Comparison of vehicle fingerprint data with stored vehicle fingerprint data lists. The use of vehicle fingerprint data to identify a vehicle is described in detail below.

(5) Manual review of photos or data to confirm or delete the results of automatic processing.

If additional processing provides a result with a particular level of trust, the resulting data can then be sent to the billing device 22. If the required level of trust cannot be achieved, the data can be stored for future reference or deleted.

The billing device 22 processes the information captured during the interaction between the vehicle and the toll object, including the vehicle identifier as defined by the photo processing module 25 to create a transaction event corresponding to the interaction between the vehicle and the object. The device 22 may store a transaction event in an accounting database 16 for subsequent payment processing. For example, the billing device 22, alone or in combination with the customer management module 26 (described below), creates payment requests based on transaction events. Transaction event data may include individual fees based on the presence of the vehicle at specific points or facilities, or travel fees based on the origin of the vehicle and the destination associated with the facility. These transaction events can be compiled and invoiced, for example, by one or more of the following methods:

(1) Deduction of payment from an account established by the vehicle owner or user. For example, the accounting database 20 can be used to store the invoice record of each vehicle owner. In turn, each account entry may include a reference to more transaction events. A paper or electronic statement of payment can be issued and sent to the registered owner of the vehicle.

(2) Generating a paper bill and sending it to the vehicle owner using the postal address from the vehicle's registration record.

(3) Presentation of an electronic invoice for a specific invoice for the vehicle owner, kept either by computer 12 or a third party.

(4) Submission of the invoice to the competent vehicle registration authority or tax authorities, allowing the collection of a fee during the vehicle renewal process or during the tax collection process.

Billing can occur at regular intervals, or when transactions meet a certain threshold, such as the maximum time range or the maximum dollar amount of tolls due and other charges due. Owners can combine billing for many vehicles by setting up an account on the computer 12.

The client management module 26 may allow the user to interact with the toll management computer 12 through a communication channel such as a computer network (e.g., Internet, wired, wireless, etc.), telephone connection, or other channel. The user may include a person associated with the vehicle 22 (e.g., vehicle owner), a public or private body responsible for managing the facility 28, or another user. The customer management module 26 includes a combination of a computer hardware module and software configured to conduct customer interaction such as an account management module 26a, a dispute management module 26b and a fee processing module 26c. Module 26 uses secure access techniques such as encryption, firewalls, passwords, or other techniques.

The account management module 26a allows users such as vehicle drivers to create an account in the system 10, connect multiple vehicles to this account, review transactions for this account, review photos related to these transactions, and make payments to the account. In one embodiment, the user responsible for the facility may have access to accounting and aggregate information related to the vehicle drivers who have used the facility.

The 26b dispute management module can allow clients to challenge specific transactions on their accounts and resolve disputes using a computer 12 or third parties. Disputes may arise during the billing situation. Module 26b can help you resolve such disputes in an automated manner. Module 26b can provide the customer III with access to the "e-Solution" section of the website of the controlling / billing authority. Customers can set up a dispute and download a photo of their transaction, which is the subject of the dispute. If there is no match (namely, the customer's vehicle is not a vehicle in the photo frame), the invoice may be sent for third-party assessment, just like arbitration. In the most likely case, the photo will show that it was correctly billed for the customer's vehicle. Dispute management can use encrypted security, in which all text and photos are sent via a computer network (e.g., the Internet) using very strong encryption. Photographs of proof of presence may be embedded in the dispute resolution message as an electronic watermark.

The 26c fee processing module provides functionality for manual or electronic payment processing, depending on the transfer received. For example, if the transfer of the fee is in the form of a paper check, then scanning devices may be used to convert the paper information into an electronic format for further processing. On the other hand, if electronic payment has taken place, then standard electronic payment techniques may be used. The 26c fee processing module can support billing methods such as traditional mailing, electronic payment (e.g. using a credit card, debit card, smart card, or Automated Clearing House transactions), periodic issuing of bills (e.g. sending invoices monthly, quarterly, after reaching the threshold, or otherwise). The 26c fee processing module can support discounts and additional fees based on the frequency of use, method of payment, or time of use of the facility. The 26c toll processing module can also support toll collection methods such as traditional check processing, toll processing during extension of vehicle registration (with accrued interest), electronic payment, direct bank debit, credit cards, prepayment, customer initiated payments (as often as the customer wants), or provide discounts for various purposes.

The toll computer 12 communicates with external systems 34 using one or more communication techniques compatible with the system's communication interfaces. For example, communication interfaces may include computer networks such as the Internet, electronic data interchange (EDI), batch file transfers, message systems, or other interfaces. In one embodiment, external systems 34 include law enforcement services 36, postal authorities 38, vehicle registration authorities 40, insurance companies 42, service providers 44, financial systems 46 and internal security agencies 48. External systems 34 may include private or public organizations, which includes one or more geographical locations such as states, regions, countries, or other geographical locations.

The toll management computer 12 can connect and exchange information with law enforcement services 36. For example, when vehicles are identified, the computer can deliver transactions in substantially real-time to law enforcement systems, in formats specified by law enforcement services. Transactions may also be submitted for vehicles carrying hazardous materials or in violation of traffic rules (e.g. speeding, weight violations, missing number plates). If suitable sensors are in place (e.g. laser / sound / microwave sensors as described above, weight sensors, radiation sensors). Alternatively, vehicle entries can be compiled and forwarded in series, based on lists provided by law enforcement services.

A database of tagged vehicle identifiers 20 can be used to store lists provided by law enforcement services. The term "marked" refers to the concept that law enforcement services provided a list of vehicle identifiers that these services indicated (indicated) that they wanted the toll object to be monitored. For example, when a vehicle has been stolen and reported to the police, the police may send a list of tagged vehicle identifiers to database 20. When a vehicle marked by the police travels over the object, the image processing module 24 determines the vehicle identifier associated with the vehicle and determines through certain interfaces that this particular vehicle is being searched by the law enforcement authority. Law enforcement agencies may want to be immediately informed of the location of the vehicle (and driver), the time it was detected in that place and the direction it was heading. The computer 12 can notify the law enforcement mobile units in substantially real time. In addition, law enforcement authorities can automatically mark vehicles based on the expiration of registration, the occurrence of the date of the traffic court, or other incident. This, in turn, can stop illegal drivers off the road and increase the state's income.

The toll management computer 12 can connect to and exchange information with postal authorities 38. Since the disclosed techniques require road toll authorities to transform the collection of drivers' fees while traveling into collecting tolls in arrears, it is important that bills are sent to the appropriate driver / vehicle owner . To minimize the possibility of sending the invoice to the wrong person, computer 12 supports address compliance. For example, before sending the invoice, the computer 12 verifies that the address provided by the vehicle department matches the address provided by the postal authority. The vehicle database can then be updated with the most accurate address information associated with the vehicle owner. Since this occurs before sending the invoice, errors in the invoices can be reduced.

The toll management computer 12 can connect and exchange information with vehicle registration authorities 40. Registration authorities 40 provide an interface for exchanging information related to vehicle owners, owner addresses, vehicle features, or other information. Alternatively, it may be access to this information by third party data providers rather than via an interface to public vehicle registers. The accuracy of registers in various databases used by computer 12, including vehicle ownership and owner addresses, can be periodically verified against third party databases or government registers, including vehicle registers and address registers. This can help ensure the quality of property and address registers, and reduce errors in invoices and returned correspondence.

The toll management computer 12 can connect and exchange information with insurance companies 42. Insurance companies can mark vehicle identifiers in a similar way to law enforcement authorities 36. For example, the database of marked vehicle identifiers 20 may include vehicle license plate numbers with expired insurance indicating, that such a driver may drive illegally. The computer can notify law enforcement authorities, as well as insurance companies, whether a marked vehicle has been detected using a specific facility.

The toll management computer 12 can connect and exchange information with service providers 44. For example, the computer 12 may submit a set or interfaces in real time to delegate billing and charging functions to accounting service providers or collection agencies.

The toll management computer 12 can connect and exchange information with financial systems 46. For example, to manage payment and collection, the computer can connect to credit card processors, banks, electronic third party billing systems. Computer 12 can also exchange information with accounting systems.

The toll management computer 12 can connect and exchange information with the internal security agency 48. The internal security office can automatically provide a list of people for use in the database of the marked vehicle identifiers 20. For example, registered drivers who are on a visa in this country can be automatically marked when the visa expires. The computer 12 will then notify the internal security office 48 that the marked vehicle identifier associated with the person was detected while driving in the country, including information about the time and place of the vehicle.

As described above, data captured from the toll site flows to the image database, and is obtained from the image database by the billing device. In another embodiment, the toll computer detects, for each vehicle, the interaction between the vehicle and the toll object, takes pictures and generates a data record. The data log may include the date, time, and location of the transaction, reference to the photo file, and any other data available from the sensors on the object (e.g. speed, size). The photo can be forwarded to the photo processing module 25, which can generate a vehicle identifier, condition and confidence indicator for each vehicle.

This information can be added to the data register. (This process may appear after transmission to a central facility). The data log and photo file can be sent to a central facility. The image may be stored in an image database, and referenced if (a) additional processing is required to identify the vehicle, or (b) someone wants to verify the transaction. If the level of trust is sufficient, the data register can be submitted to the invoicing device, which can link it to the account and store it in the accounting database for subsequent invoicing. If no account exists, the vehicle ID is sent to the appropriate state registration authority or third party service provider to identify the owner and establish the account. This process may be delayed until a sufficient number of transactions are collected for the vehicle to justify the invoice. If the level of trust is not sufficient, additional processing may be performed as described elsewhere.

The techniques described above describe the data flow based on a single end-to-end transaction, then looping to the beginning. In another embodiment, some of the functions described may be event-driven or scheduled, and may operate independently of each other. For example, there may not be a flow of control of the final processes to take a picture of the vehicle. The process of taking a picture of a vehicle can be initiated by an event, including the presence of the vehicle at the toll site.

In another embodiment, the system may be used for traffic monitoring and accident management. For example, if a decrease in average vehicle speed is detected, the computer may send a message to the highway control facility alerting controllers about the possibility of an accident. Authorized inspectors may connect to the on-site toll equipment to review camera images and determine if a response is required.

The operation of the toll management system 10 is explained with reference to FIG. 2-5.

FIG. 2 is a flowchart of an embodiment of an electronic road toll management system, especially process 100 for managing tagged vehicle identifiers 20 provided by external systems 34. To illustrate, in one example, law enforcement services 36 generate a list of tagged vehicle identifiers (e.g. license plate numbers) of drivers sought by the services and that the services 36 want to be notified when such vehicles have been identified using the tolls 28.

Computer 12 obtains (block 102) marked vehicle identifiers from a side such as law enforcement services 36. In one embodiment, these vehicle identifiers can be stored in the vehicle identifier database 20 for further processing. Database 20 can be updated by new services as well as additional real-time information and / or batches. Law enforcement services accessed by a computer operate in many jurisdictions such as cities, cities, states, regions, countries, or other geographical purposes. Consequently, computer 12 can process vehicle information across many jurisdictions and on a national scale.

Computer 12 (block 104) takes a picture of the vehicle triggered by a transaction event based on the interaction between vehicle 30 and object 28. For example, photo acquisition module 24 can be used to obtain one or more pictures of a vehicle while traveling on an object such as a road toll. These photos can be stored in the image database 14 for further processing by the image processing module 25. Compression techniques can be applied to photos taken to help reduce the size of the database 14.

Computer 12 determines (block 106) the vehicle identifier based on the photo taken. For example, as discussed previously, the photo processing module 25 may apply photo analysis techniques to raw photos in an image database 14. These analytical techniques may extract a registration number from one or more photos of a vehicle license plate. The extracted vehicle identifiers can be stored in the vehicle identifier database 18 for further processing.

Computer 12 compares (block 108) the captured vehicle identifier with the designated vehicle identifier. For example, computer 12 may compare the captured license plate number from the vehicle identifier database 18 with the registration number from the database of vehicle identifiers 20. As discussed above, both automatic and manual techniques can be used to check the fit.

If the computer 12 detects a match (block 110) between registration numbers, then it checks (block 112) how the page associated with the tagged vehicle identifiers wants to be notified. This information may be stored in a vehicle identifier database 20 or other storage mechanism. On the other hand, if there is no match, the computer 12 resumes performing process 100 starting at block 102.

If the page indicates that it wants to be notified immediately (block 114), then the computer notifies (block 118) the person when a match occurs. In this example, a computer may notify law enforcement about essentially real-time matching using wireless communication techniques or via a computer network.

On the other hand, if a person does not want to be notified immediately (block 114), then the computer 12 stores (block 116) a match for later notification after meeting certain criteria. In one embodiment, specific criteria may include gathering a specific number to match and then send to match the law enforcement agencies batch.

When the page has been notified (block 118) about the match or the match has been stored for later notification (block 116), the computer 12 resumes the process 100 starting from block 102.

FIG. 3 is a flowchart of an embodiment of the electronic toll management system 10, especially the process 200 for managing payment from a person associated with a vehicle that has interacted with an object. To illustrate, in one example, it is assumed that the toll authority decides to adopt disclosed techniques for managing payment processing, including billing and collecting tolls on vehicles using its toll road.

Computer 12 takes (block 202) a photo of the vehicle triggered by a transaction event based on the interaction between the vehicle and the object. This function is similar to the process discussed above with respect to block 104 of FIG. 2. For example, the image acquisition module 24 may be used to obtain one or more images of a vehicle 30 when it crosses a road with tolls 28. These photos can be stored in the image database 14 for further processing by the image processing module 25.

Computer 12 determines (block 204) the vehicle identifier based on the photo taken. This function is also similar to the process discussed above with respect to block 106 of FIG. 2. For example, the photo processing module 25 may be used to extract a registration number from one or more vehicle license plate photos. These vehicle identifiers can be stored in the vehicle identifier database 18 for further processing.

Computer 12 identifies (block 206) a person associated with the vehicle identifier by searching the registers' databases. For example, the computer 12 may use the vehicle identifier from the vehicle identifier database 18 to search the vehicle registration authority database 40 to determine the registered vehicle owner associated with the vehicle identifier. Computer 12 can access vehicle information from one or more vehicle registration databases across many jurisdictions such as cities, cities, states, regions, countries, or other geographical locations. In one embodiment, the computer 12 may keep a copy of the registration information from multiple registration authorities for further processing. Alternatively, the computer 12 may have access to multiple registration authorities and obtain registration information based on the request. In any case, these techniques allow the computer 12 to process vehicle information across multiple jurisdictions, and thus to process vehicles nationally.

Computer 12 checks (block 208) whether to demand payment from a person associated with the vehicle identifier. The payment request may depend on the payment processing information associated with the registered owner. For example, bills can be sent to the registered owner periodically (e.g. monthly) when a certain amount is reached, or as otherwise agreed.

If the computer 12 determines that payment is required (block 210), then it requests (block 214) payment from a person associated with the vehicle identifier based on the transaction event. As discussed above, a payment request may be generated using traditional postal service techniques or electronic techniques such as electronic payment. The invoice amount may depend on information from a transaction event such as the nature of the interaction between the vehicle and the object. For example, a transaction event may indicate that the vehicle has arrived at a specific distance defined as the distance between the start and end point on the toll road. Accordingly, the amount of payment requested from the registered owner can be based on the distance traveled.

On the other hand, if the computer 12 determines that payment is not required (block 210), then it forwards (block 212) the transaction event to another party to manage the payment request. For example, the toll authority may decide that the computer 12 may perform image processing functions and the billing and toll collection should be performed by a third party such as external systems 34. In one embodiment, the computer 12 may connect to service providers 44 and financial systems 48 to perform all or part of the billing and payment management functions. When the transaction event has been forwarded to a third party, the computer 12 resumes the functions of the process 200 starting in block 202.

If the computer processes the charges, the computer 12 processes (block 216) the payment response from the person associated with the vehicle identifier. In one embodiment, the accounting database 16, in combination with the billing device 22 and the customer management module 26, can be used to perform the billing and collection functions. As discussed above, the payment processing module 26c may assist electronic or manual payment processing depending on the received transfer. For example, computer 12 may provide an account for performing electronic payment processing over a computer network such as the Internet. The computer can also receive traditional payment such as a check.

When the payment has been processed (block 216), the computer 12 resumes execution of the process 200 starting in block 202.

FIG. 4 is a flowchart of an embodiment of the electronic road toll management system 10, especially the process 300 for managing payment through a communication channel from a person associated with a vehicle that has interacted with an object. To illustrate, it should be assumed that the toll authority responsible for the toll road uses the disclosed techniques and that the registered owner wants to make payment for the use of the toll road in an efficient and automatic way.

Computer 12 provides (block 302) an account for a person associated with the vehicle identifier. In one embodiment, the computer 12 in combination with the account management module 26a can provide customers with a website to open an account for making electronic payment over a computer network such as the Internet. The website may also allow the customer to access and update account information such as payment history, amount due, preferred payment method, or other information.

Computer 12 receives (block 304) a request via a communication channel from a person to view the transaction event. For example, the account fee module 26a may execute this request by extracting information about the event of transactions related to the client account from the accounting database 16. The extracted information may include photographic data of the specific transaction concerning the client's vehicle and the toll booth.

Computer 12 sends (block 306) transaction event to person 32 via communication channel. Transaction event information may include vehicle photos and a vehicle identifier (namely a license plate). Such data can be encrypted to ensure secure transmission over the Internet. Standard communication protocols such as hypertext marking information (HTML) can be used to transmit information over the Internet.

Computer 12 determines (block 308) whether the person agrees to make the payment. For example, when a customer receives information about a transaction event, the customer can view the information to determine whether to make a payment based on whether the vehicle shown in the pictures is the customer's vehicle.

If computer 12 determines (block 310) that a person agrees to pay, then it processes (block 314) payment from the person by deducting the amount from the account based on the transaction event. For example, if the photographic information indicates that the transaction event data is accurate, then the customer can authorize payment as by making an electronic payment transaction.

On the other hand, if computer 12 determines (block 310) that a person does not agree to pay, then computer 12 processes (block 312) the contentious claim regarding payment from the person. In one embodiment, the dispute management module 26b can conduct a contentious request submitted by the client using on-line techniques. Module 26b may conduct specific transactions regarding the customer's account, including the involvement of a third party to resolve the dispute.

When the payment has been processed (block 314) or the dispute has been resolved (block 312), the computer 12 resumes the execution of the process 300 starting in block 304.

FIG. 5 is a flowchart of an embodiment of an electronic road toll management system, especially process 400 for reconciling postal addresses from various sources. To illustrate, it is assumed that the toll authority used the disclosed techniques for processing the toll related to the use of the toll object. Because the disclosed techniques involve the processing of a toll after some time since the vehicle was traveling through the toll authority, these techniques help to ensure that payment is sent to the correct address of the registered owner of the vehicle.

Computer 12 determines (block 402) that the request for payment has been sent to the person associated with the vehicle identifier. As explained above, for example, payment requests may be generated periodically or based on a threshold amount.

Computer 12 gains access (block 404) to the vehicle registration authority for the postal address of the person associated with the vehicle identifier. For example, computer 12 may access one or more databases related to vehicle registration authorities 40 to obtain information such as the postal address of the registered owner of the vehicle.

Computer 12 gains access (block 406) to the postal authority for the postal address of the person associated with the vehicle identifier. For example, computer 12 may access one or more databases associated with postal authorities 38 to obtain information such as the postal address of the registered vehicle owner.

Computer 12 compares (block 408) the postal address from the vehicle registration authority to the postal address from the postal authority. For example, a computer compares email addresses from two authorities to determine if there is a discrepancy between information from databases.

If computer 12 determines (block 410) that the addresses match, then it requests (block 414) payment from a person associated with the vehicle identifier using the postal address obtained from the postal authority. For example, the computer 12 may use the techniques described above to carry out payment processing, including billing and charging a fee from a registered owner.

On the other hand, if the computer 12 determines (block 410) that the addresses do not match, then it updates (block 412) the vehicle registration authority with a postal address from the postal authority. For example, computer 12 can update the databases related to vehicle registration authorities 40 with the correct postal address obtained from postal authorities 38. Such techniques can help reduce the likelihood of sending the invoice to the wrong postal address resulting in a shorter time of payment transfer.

When the vehicle registration authority has been upgraded (block 412) or payment has been requested (block 414), the computer 12 resumes the process 400 by starting in block 402 as explained above.

FIG. 6 is a block diagram of an embodiment of the electronic toll management system 600, which provides vehicle identification by extracting multiple vehicle identifiers for each vehicle that interacts with the toll object. The toll management system 600 includes a toll management computer 612. The toll management computer includes the image database 614, the accounting database 616, the vehicle identification database 618, the database of tagged vehicle identifiers 620, the billing device 622, the photo capture module 624, the photo processing module 625, and the customer management module 626. The toll management computer 612 connects or is integrated with the toll object 628, which interacts with the vehicle 630 and the person associated with the vehicle 632. The toll management computer 612 also connects to external systems 634.

Examples of each element included in the toll management system 600 of FIG. 6 are widely described with reference to FIG. 1. In particular, toll management computer 612, image database 614, accounting database 616, vehicle identification database 618, database of marked vehicle identifiers 620, bill issuing device 622, photo acquisition module 624, photo processing module 625, customer management module 626, and the toll object 628 usually have features comparable to and illustrate one possible embodiment, respectively, toll management computer 12 image database 14, accounting database 16, vehicle identification database 18, marked vehicle identifier database 20, billing device 22, photo acquisition module 24, photo processing module 25, customer management module 26, and facility toll 28 from FIG. 1. Similarly, vehicle 630, person associated with vehicle 632, and external systems 634 usually have features comparable to vehicle 30, person associated with vehicle 32, and external systems 34 of FIG. 1.

The vehicle identification database 618 includes the database of extracted identifiers 6181, the vehicle register database 6182, and the reading error database 6183. The functions of the databases 6181-6183 are described in more detail below.

System 600 is similar to system 10 and is configured to provide, for example, smaller amounts of vehicle identification errors by identifying each vehicle by using multiple vehicle identifiers. Two such identifiers are designated 631A and 631B. The vehicle identifier is preferably an identifier that uniquely or substantially uniquely identifies the vehicle, but may be an identifier that assists in the identification process by distinguishing the vehicle from another vehicle without necessarily having a unique identification of the vehicle. IDs 631A and 631B may be parts of vehicle 630, as suggested by FIG. 6, but not necessarily. For example, identifiers 631A and / or 631B may be generated by the image processing module 625 based on the features of the vehicle 630.

As previously described, one example of a vehicle identifier is information from the vehicle's license plate, such as the license plate number and status. The photo processing module 625 can determine vehicle license plate information from the license plate photo using OCR, template matching, and other analytical techniques. The number plate can contain any character, but is usually limited to alphanumeric characters. License plate information can usually be used to uniquely identify a vehicle.

Another example of a vehicle identifier is a vehicle detection tag as described in US Patent No. 6,747,687. The vehicle detection tag, referred to here as the vehicle's fingerprint, is a distilled set of data artifacts that represent the visual signature of the vehicle. The photo processing module 625 can generate vehicle fingerprints by processing the photo of the vehicle. However, to save processing time and storage needs, the generated fingerprint of the vehicle usually does not include normal "photographic" information that a person could recognize. Therefore, it is not usually possible to process the fingerprint of the vehicle to obtain the original photo of the vehicle. However, the fingerprints of some vehicles may include normal photographic information. A vehicle's fingerprint can usually be used to uniquely identify a vehicle.

In one embodiment, the camera in the photo acquisition module 624 takes a "still" photo of the rear of each vehicle that passes through the toll object 628. For each vehicle, the photo processing module 625 recognizes visual paths that are unique to the vehicle and reduces them into an imprint vehicle finger. Because the license plate is a very unique feature, the 625 photo processing module usually maximizes the use of the license plate in creating a fingerprint of a vehicle. Notably, the fingerprint of the vehicle also includes other vehicle parts in addition to the license plate and, therefore, vehicle identification by matching vehicle fingerprints is generally considered more accurate than identifying a vehicle by matching registration plate information. The vehicle fingerprint may include, for example, vehicle parts around the license plate and / or bumper parts and wheelbase.

Another example of a vehicle identifier is a vehicle signature generated using a laser scan (hereinafter referred to as a laser signature). The information from the laser signature that can be captured using a layered scan may include one or more electronic vehicle profiles from above, including the length, width, and height of the vehicle, the number of vehicle axles, and a 3D image of the vehicle. In one embodiment, the photo acquisition module 624 has two lasers per route lane, one is mounted above the road lane, and the other is attached along the road lane. A laser attached over a road lane typically scans the vehicle to capture the vehicle profile from above, and a laser attached along or over the road lane typically scans the vehicle to capture the number of vehicle axles. Together, two lasers can also generate a 3D image of the vehicle. The laser signature can be used to uniquely identify certain vehicles. For example, vehicles that have been modified to have a distinctive shape can be uniquely identified by a laser signature.

Another example of a vehicle identifier is a vehicle signature generated using a magnetic scan (hereinafter referred to as induction signature). The vehicle induction signature is a parameter that reflects the distribution of the metal in the vehicle and, therefore, can be used to classify the vehicle and, under certain circumstances, to uniquely identify the vehicle (e.g. if the metal distribution in a particular vehicle is unique to that vehicle due to unique modifications to that vehicle). The induction signature may include information that can be used to determine one or more of: the number of axles (and probably the number of tires) of the vehicle, the type of engine used in the vehicle, the type or class of vehicle. In one embodiment, the photo acquisition module 624 includes a pair of vehicle detection loops, an axle detection loop, and a camera trigger loop in each lane of road.

When one or more vehicle identifiers are extracted by the image processing module 625, the image processing module 625 stores the extracted vehicle identifiers in the extracted vehicle identifier database 6181. Ideally, the computer 612 will then be able to uniquely identify the owner of the vehicle by selecting a vehicle identifier that uniquely identifies the vehicle (eg. information from the license plate or fingerprint of the vehicle) and a search of one or more internal or external vehicle register databases to find a register containing a matching vehicle identifier. Unfortunately, the extraction of the vehicle identifier is an imperfect process. The extracted vehicle identifier may not match the actual vehicle identifier, and therefore may not uniquely identify the vehicle. An incorrectly or partially extracted vehicle identifier may not match the identifier of any vehicle, may match the identifier of the wrong vehicle, or may match the identifiers of more than one vehicle. To increase identification accuracy, system 612 computer 600 implements a multi-row identification process using two or more vehicle identifiers.

FIG. 7 is a flowchart of an example 700 double-row identification process that can be implemented to increase vehicle identification accuracy. Photographs and / or sensor data are captured for a vehicle that interacts with a toll object (hereinafter referred to as "target vehicle") and two vehicle identifiers are extracted from captured data (block 710). In one embodiment, only the image data is downloaded and the two extracted vehicle identifiers are the license plate number and the fingerprint of the vehicle. In another embodiment, the image data and induction sensor data are taken, and the extracted vehicle identifiers are vehicle fingerprint and induction signature.

One of the two extracted vehicle identifiers is designated as the first vehicle identifier and used to identify a set of one or more matching candidate vehicles (block 720). Typically, the vehicle identifier considered to be the least able to accurately and / or uniquely identify the target vehicle is designated as the first vehicle identifier. For example, if the two extracted vehicle identifiers were the number plate and fingerprint of the vehicle, the number plate would be marked as the first vehicle identifier due to the lower expected accuracy of vehicle identification by matching the number plate compared to matching the fingerprint. One or more matching candidate vehicles can be determined, for example, by accessing the vehicle register database and finding registers that contain vehicle identifiers that match or almost match the first vehicle identifier.

When a set of one or more matching candidate vehicles is specified, the target vehicle is identified from the set based on the second vehicle identifier (block 730). For example, if 12 candidate vehicles have been identified as matching a partially extracted license plate number, the target vehicle is identified by accessing vehicle fingerprints for each of the 12 candidate vehicles and determining which of 12 vehicle fingerprints matches the vehicle's fingerprint. If no match is found within the specified confidence threshold, manual vehicle identification may be used. In another embodiment, one or more larger sets (e.g., super sets) of matching candidate vehicles are determined sequentially or simultaneously by changing (e.g., loosening) the matching criteria and additional attempts are made to identify the target vehicle from each of one or more larger sets before resorting to manual identification.

In certain embodiments, the toll management system may be intentionally designed to identify a larger set of matching candidate vehicles during operation 720 to, for example, ensure that the expected lower accuracy of vehicle identification by the first identifier does not mistakenly exclude the target vehicle from the set of matching candidate vehicles. For example, if the first vehicle identifier is the number of the registration plate, the algorithm for reading the registration plate may be intentionally modified, for example, in two ways: (1) the criteria for matching the license plate reading algorithm can be loosened to allow the algorithm to generate a larger set of matching candidate vehicles, and (2) the license plate reading algorithm can be "re-tuned" by lowering the reading certainty threshold used to determine if the reading result is included in the set matching candidates. For example, the license plate reading algorithm may be loose to require only a matching vehicle to match a subset or fewer characters in the number of the registration plate extracted for the target vehicle. Additionally or alternatively, the reading certainty threshold may be lowered to allow previously suspected incorrect readings (namely, partial or low certainty readings) to include candidates in a set of matching vehicles.

The 700 double-row identification process ensures greater identification accuracy over the single-row / single identifier identification system by requiring that two vehicle identifiers be successfully matched for successful vehicle identification. Furthermore, the process 700 can provide a higher identification speed by limiting the matching of the second vehicle identifier to only those vehicle vehicles having registers that successfully match the first vehicle identifier. This can provide increased speed if, for example, the extracted second vehicle identifier consumes time to match with other such identifiers or if there is a large number of such other identifiers (namely, millions of identifiers for millions of vehicles in the vehicle database).

In another embodiment, two or more second identifiers are used to identify the target vehicle from among a set of matching candidate vehicles. Each of the second identifiers must match the same vehicle of the candidate at a certain level of trust for successful vehicle identification. Alternatively, the degree of matching of each of two or more second identifiers can be weighted and a combined result of equivalent matching can be generated. If the combined result of the equivalent match is above a certain threshold, the identification is considered successful.

In one embodiment, each second vehicle identifier is assigned a matching trust level number, which is in the range of 1 to 10, where 1 corresponds to a missing match and 10 corresponds to an exact match. Each vehicle identifier also has a weight value from 1 to 10 assigned, with larger weight values ​​assigned to vehicle identifiers considered to be more accurate in the unique vehicle identification. If, for example, the second vehicle identifiers are the laser signature and the information from the license plate, a weight of 6 can be allocated to the laser signature and a larger weight 9 can be assigned to the information from the license plate. If a combined equivalent match result of 100 is necessary to consider identification successful, and the license plate information matches a confidence level of 7 and the laser signature also matches a confidence level of 7, the combined equivalent result will be 7 * 6 + 7 * 9 = 105 and identification will be considered successful.

In another embodiment, two or more first vehicle identifiers are used to identify vehicles in the set of matching candidate vehicles. Each of the first vehicle identifiers for a possible candidate vehicle must match the target vehicle with a certain level of confidence so that the possible candidate vehicle is in the set of matching candidate vehicles. Alternatively, the degree of matching of each of the two or more first identifiers can be weighed and a combined result of equivalent matching can be generated. If the combined result of the equivalent match is above the specified threshold, the possible candidate vehicle is in the set of matching candidate vehicles.

In another embodiment, the second identifier is not used to uniquely identify the target vehicle among the vehicles in the set of matching candidate vehicles. Rather, the second identifier is used to generate a new and smaller set of matching candidate vehicles as a subset of the set identified using the first identifier, and then a third identifier is used to uniquely identify the target vehicle from this set of matching candidate vehicles. In another embodiment, multiple vehicle identifiers are used to further reduce the set of matching candidate vehicles and the target vehicle is uniquely identified from the successively reduced subset by using one or more final vehicle identifiers. In yet another embodiment, each of the plurality of vehicle identifiers is used to generate its own set of matching candidate vehicles by matching or close matching techniques, and the reduced set is a cross between all specified sets. In yet another embodiment, the reduced assembly is determined using a combination of the techniques described above.

FIG. 8 is a flowchart of an exemplary double-row identification process 800 that can be implemented to increase accuracy and / or automate vehicle identification. The 800 process is an implementation of the 700 process, where the first identifier is the license plate number and the second identifier is the vehicle's fingerprint. In particular, process 800 includes operations 810-830, and associated sub-operations that correspond to and illustrate one possible implementation of operations 710-730, respectively. For convenience, reference is made to the specific elements described with reference to FIG. 6 during process 800. However, similar methodologies can be used in other embodiments where different elements are used to determine the structure of the system, or where functionality is differently divided between the elements shown by

FIG. 6.

The image acquisition module 624 captures image data for the target vehicle based on the interaction between the target vehicle and the toll object 628 (block 812). In another embodiment, the image acquisition module 624 additionally or alternatively captures sensor data including, for example, laser scanning data and / or a loop sensor. The image processing module 625 obtains license plate data, including, for example, the whole or partial number of the license plate and the state for the target vehicle from captured image data (block 814). Optionally, the image processing module 625 may also determine the vehicle's fingerprint for the target vehicle from the image data. In another embodiment, the image processing module 625 may determine other vehicle signature data, such as, for example, laser and / or induction signature data, from image data and / or sensor data.

Computer 612 stores captured image data in an image database 614 and stores extracted license plate data in a database of extracted identifiers 6181. When applicable, the toll management computer 612 also stores the vehicle's fingerprint and other signature data, such as, for example, induction signature and / or laser signature in the extracted identifier database 6181.

Computer 612 accesses the vehicle identification register set from the vehicle register database 6182 (block 822). Each of the vehicle identification registers combines the vehicle owner / driver with vehicle identifier data. Computer 612 compares the extracted license plate data with the license plate data in the set of vehicle identification registers (block 824) and identifies the set of candidate vehicles from vehicles having registers in the set of registers (block 826). The comparison can be made using matching or close matching techniques.

Computer 612 accesses extracted vehicle fingerprint data for the target vehicle (832). If the vehicle's fingerprint has not yet been determined / extracted from the captured image data, the computer 612 calculates the vehicle's fingerprint and stores the vehicle's fingerprint in a database of retrieved vehicle identifiers 6181.

Computer 612 accesses the fingerprint data for the vehicle in the candidate vehicle set by gaining access to the appropriate vehicle identification register (block 834) and compares the vehicle fingerprint data for the target vehicle with the fingerprint data for the candidate vehicle (block 836). Computer 612 identifies the candidate vehicle as a target vehicle based on the results of vehicle fingerprint data comparison (block 838). If the vehicle's fingerprint data match a certain level of trust, the candidate vehicle is considered the target vehicle and the candidate vehicle owner / driver is considered the target vehicle owner / driver.

FIG. 9A-9C are a flowchart of an example 900 double-row identification process that can be implemented to increase the accuracy of vehicle identification while minimizing the need for manual vehicle identification. Process 900 is another embodiment of process 700, where the first identifier is the license plate number and the second identifier is the vehicle's fingerprint. In particular, process 900 includes operations 910-930, and associated sub-operations that correspond to and illustrate one possible implementation of operations 710-730, respectively. For convenience, reference is made to the specific elements described with reference to FIG. 6 during process 800.

However, similar methodologies can be used in other embodiments where different elements are used to determine the structure of the system, or where functionality is differently distributed between the elements shown in FIG. 6.

The image acquisition module 624 captures image and sensor data for the target vehicle (block 911). Side road sensors, for example, run cameras that take pictures of the front and rear of the target's vehicle. Other sensors may capture additional data used for vehicle classification / identification. For example, a laser scan can be used to specify laser signature data including height, width, length, number of axles, and vehicle dimensional profile. Sensors can also be used to determine transaction related data between the target vehicle and the toll object 628 such as, for example, vehicle weight, vehicle speed, and vehicle related transponder data.

The photo processing module 625 performs a license plate reading on captured image data, creates a vehicle fingerprint from captured image data, and optionally specifies other vehicle signature / classification data from captured sensor data (block 912). For example, the photo processing module 625 may use the automatic license plate reading algorithm to read one or more photos taken. The license plate reading algorithm can read the pictures taken, for example, in a priority order based on the readability of the plate and its location in the picture. The license plate reading results may include one or more license plate numbers, license plate status, license plate style, read confidence score, plate location in the photo, and plate size. The image processing module 625 may also use a visual signature extraction algorithm to generate a vehicle fingerprint for the target vehicle. The algorithm for extracting the visual signature may be similar to that developed by JAI-PULNiX Inc. from San Jose, California and described in US Patent No. 6,747,687. Computer 612 stores the photos taken in the image database 614 and stores the results of reading license plates, vehicle fingerprint, and other vehicle reference / classification data in the database of extracted 6181 vehicle identifiers.

The photo processing module 625 determines whether the photos taken have provided any partial or full reading results for the number plate and the vehicle's condition (block 913). If partial or full reading results have not been provided by the images taken, process 900 proceeds to operation 941 of the manual identification process 940.

If partial or full reading results for the registration number and condition of the vehicle of the target were provided by the pictures taken, computer 612 searches the vehicle register database 6182 and the reading error database 6183 for the exact (or partial or full) number of the registration plate (as read by the plate reader registration) (block 921)

The vehicle register database 6182 includes registers for all previously recognized vehicles and potentially includes registers for vehicles that can be expected to be seen. The 6182 vehicle register database is usually populated by a registration process during which the driver / owner of the vehicle saves the vehicle for automatic toll payment management. The driver / owner of the vehicle can register the vehicle for the automatic management of the payment of road tolls by driving the vehicle through a special registration lane at the road toll facility 628 and providing his customer service representative at the object 628 his or her identity and other contact details. The image acquisition module 624 and the image processing module 625 capture the license plate number, fingerprint, and other identification / classification data (e.g.

vehicle dimensions) of the user's vehicle when the vehicle exceeds object 628. The vehicle and owner identification data are stored in a new vehicle identification register associated with the newly registered vehicle and owner / driver.

Alternatively, the driver / owner can register a vehicle to automatically manage the payment of tolls simply by passing through facility 628 without stopping. Computer 612 captures image data and sensor data for the vehicle and attempts to identify the driver / owner by reading the license plate photo and searching for the reading results in an external 634 database (e.g. vehicle registration authorities). If the owner / driver has been identified, the computer 612 bills the owner / driver. Once the accounting relationship has been successfully established, the computer 612 officially registers the vehicle, generates as necessary vehicle fingerprint data and other reference / classification data from captured image data and sensor, and stores it in the vehicle identification register associated with the identified owner / driver.

In another embodiment, the computer 612 is configured to obtain greater accuracy in identifying the unregistered driver / owner by searching for the results of the license plate reading in the vehicle registration authority database (or other external system) and requesting the corresponding vehicle identification number (VIN) in the registration authority vehicles (or other external system). The 612 computer uses VIN to identify the make, model and year of the vehicle. The make, model and year of the vehicle can be used to determine the length, width, and height of the vehicle. The computer 612 can then determine the successful connection of the target vehicle to the vehicle registered in the vehicle registration body not only by comparing the license plate data but also by comparing the dimensions of the vehicle (as captured, for example, in the laser signature and / or induction signature). Typically, the computer 612 will consider the match to be successful if the result of the license plate reading for the target vehicle matches the license plate data for the vehicle registered by the vehicle registration authority within a certain threshold and the dimensions of both vehicles fit within a given tolerance.

The make, model and year of the vehicle can be used, for example, to determine the length, width, and height of the vehicle either by obtaining this information from a public database or from a third party database or, in addition or alternatively, by accessing the vehicle register database 6182 to obtain data on the length, width, and height from one or more vehicle identification registers corresponding to vehicles of the same make, model, and year as the target vehicle. Due to the fact that the vehicle dimensions may change if the vehicle has been modified, the length, width, and height obtained from the vehicle identification registers may differ for the vehicle. Therefore, the computer 612 may need to determine statistically valid dimensions for comparison by, for example, adopting the average or median dimensions of length, width, and height.

In one embodiment, the computer 612 identifies the vehicle in part by using an electronic signature, which includes a laser signature and / or an induction (namely, magnetic) signature. When a vehicle makes a transaction with a toll system, the electronic signature for the vehicle is captured. The photo and dimensions of the vehicle created by the laser (namely, the laser signature) and / or magnetic scan (namely, the induction signature) are compared against known dimensions and photos of vehicles based on the vehicle identification number (VIN), which were, for example, previously captured by toll system or through an external system. By comparing the photo and dimensions of the electronic signature to known vehicle dimensions based on VIN, the search for vehicle matching and associated VIN can be narrowed down. If, for example, the LPR for a vehicle has a low level of trust but the vehicle's electronic signature has been captured, the toll system can access the database as described above, known dimensions and photos of vehicles and associated VINs, and refer to the dimensions and photos of the electronic signature relative to the base data to identify the matching VIN of the vehicle or to identify potential match of candidate vehicles A / IN. The 6183 read error database connects previous erroneous readings to the correct vehicle identification registers. For example, when automatic vehicle identification fails, but manual vehicle identification is successful, the captured vehicle identification data (e.g. the result of the license plate reading) that led to the "error" (namely, identification failures) by the automated system are stored in the error register in the 6183 read error database, which is connected to the vehicle identification register that has been manually identified for the vehicle. In this way, when the same vehicle identification data is captured again at a later time, the computer 612 can successfully identify the vehicle automatically by accessing the error register in the 6183 read error database that identifies the correct vehicle identification register without requiring another manual vehicle identification .

An error log can also be generated and stored in the 6183 read error database when the automatic identification of the vehicle is successful based on a close match of the result of the incorrect reading of the license plate. For example, if the number plate "ABC123" is read as "ABC128" and the set of matching identified candidates is "ABC128", "ABC123", "ABG128" and "ABC128", which in turn gives the correct matching "ABC123", can be created an error register that automatically connects the result of the "ABC 128" license plate reading to the vehicle having the "ABC123" license plate number.

Computer 612 determines whether any vehicle identification registers correspond to the results of the license plate reading for the target vehicle (block 922). If no vehicle identification registers match the read results, the computer 612 performs an extended search (block 923).

Computer 612 performs an extended search by changing or loosening the criteria for a successful match or re-tuning the license plate reading algorithm. For example, computer 612 may perform an advanced search by one or more of the following: (1) comparing a subset of the license plate reading result with the license plate number characters stored in the 6182 vehicle register database (e.g. the last two characters of the license plate number may be omitted so that if the license plate number is "ABC123", all vehicles having license plate numbers "ABC1 **" are considered to be matching candidates, where "*" is a variable): (2) subset comparison the result of reading the number plate in reverse order with the number plate characters stored in the 6182 vehicle register database in reverse order (e.g. the last two characters of the number plate in reverse order may be omitted so that if the number plate is "ABC123", which in reverse order is "321CBA", all vehicles having number plates in reverse order "321C **" are considered matching candidates, where "*" is a variable; and (3) other close matching techniques involving the comparison of modified versions of the number plate reading results and the number plate numbers stored in the 6182 vehicle register database in which some of one or both are replaced and / or removed to reduce the impact of unread characters. For example, if the OCR algorithm does not indicate a level of trust above a certain threshold as a result of reading a mark on the license plate, that mark may be ignored. Additionally or alternatively, if the OCR algorithm indicates that the sign on the license plate can be one of two possible different characters, both alternative characters can be used in the extended search.

Computer 612 determines whether any vehicle identification registers correspond to the read results for the target vehicle after performing the expanded search (block 924). If no vehicle identification registers are found, process 900 proceeds to operation 941 of manual identification process 940 (block 924).

With reference to FIG. 9B, if the search or expanded search results in the identification of one or more vehicle identification registers, the computer 612 obtains the vehicle's fingerprint and optionally other vehicle reference / classification data from the identification records of the identified vehicles (block 931). Computer 612 compares the obtained fingerprint of the vehicle and optionally other vehicle signature / classification data for each matching candidate vehicle with the corresponding vehicle related data to identify one or more possible matches (block 932). The comparison of the vehicle's fingerprint can be made using a comparison algorithm identical or similar to the algorithm developed by JAI-PULNiX Inc. from San Jose, California and described in US Patent No. 6,747,687.

A possible match may be defined, for example, as a vehicle fingerprint match with a confidence score greater than or equal to a specified threshold and all or some other classification / reference data within the tolerance specified for each type of data. For example, if the fingerprint matching algorithm produces a result of the order of 1 to 1000, where 1 is no match and 1000 is a perfect match, then a result greater than or equal to 900 may be required for a successful match. In addition, if other classification / reference data include the height, width, and length of the target vehicle, then it may be required that the height, width, and length of the candidate vehicle be plus or minus four inches of the extracted height, width, and length of the target vehicle for a successful fit . One or more vehicle identification registers may be considered to correspond to vehicles that match the target vehicle as closely as possible.

Computer 612 determines whether a possible match is sufficient to automatically identify the vehicle without human intervention by determining the combined result of equivalent matching for each possible match and comparing the result with the specified automatic trust threshold (block 933). Computer 612 may, for example, determine the combined result of an equivalent match for each possible match in a manner similar to that previously described for process 700. Specifically, computer 612 may allocate the number of matching confidence levels to the matching fingerprint and, optionally, matching classification / signature data, allocate weight to each type of data, and calculate the combined result of the equivalent match by combining the weighted numbers of the match confidence level. If the combined result of the equivalent match exceeds the specified automatic trust threshold, the computer 612 will consider the target vehicle to be successfully identified and process 900 will proceed to operation 937 to record the transaction event between the identified vehicle and object 628. If more than one possible match exceeds the automatic confidence threshold, the automatic identification process may be faulty, and the 900 process may optionally proceed (not shown) to operation 941 of the 940 manual identification process.

If no possible match is considered sufficient to automatically identify the vehicle without human intervention, the computer 612 will determine if one or more possible matches meet the lower probable match threshold (block 934). The computer 612 may, for example, determine that a possible match meets the probable match threshold if the result of the combined equivalence match of the probable match is higher than the probable match threshold but lower than the automatic trust threshold.

If at least one possible match meets the likely match threshold, computer 612 allows the operator to perform a visual check of the match (block 935). Visual matching check is the process by which the computer 612 presents the operator with one or more photos of the target vehicle along with one or more reference photos related to the vehicle or vehicles that probably match the target vehicle. The operator quickly confirms or rejects every likely match with a simple yes or no by, for example, selecting the appropriate buttons on the user interface (block 936). The operator can optionally also provide a detailed explanation to justify his or her response.

If the match exceeds the automatic trust threshold or is visually confirmed by the operator by visually checking the match, the computer 612 creates an event log (namely, the interaction record between the positively identified vehicle target and object 628) as, for example, a billable or non-profit transaction (block 937). If the match has been confirmed by visual matching check, the computer 612 may optionally update the error reading database 6183 to include extracted vehicle identification data and a link linking the extracted vehicle identification data to the appropriate vehicle identification register (block 938).

Also referring to FIG. 9C, the computer 612 is configured to allow the operator to manually identify the target vehicle (block 941) in the following situations: (1) taken pictures of the target vehicle do not provide full or partial readings for the number plate and the state of the target vehicle (block 913); (2) no vehicle identification registers matching the results of the number plate readout for the target vehicle are searched after performing the extended search (block 924); (3) one or more possible matches are found, but the level of confidence in one or more possible matches, which is reflected in the results of the combined equivalence match, is below both the automatic confidence threshold and the likely match threshold (block 934); and (4) one or more possible matches are found, but the human operator rejects one or more likely matches by visual checking of the match (block 936).

The human operator attempts to manually identify the vehicle by (1) reading the registration plate (s), and (2) observing the details of the vehicle captured by the photo capture module 624, and (3) comparing the number plate data and the vehicle details with data available from 6182 vehicle register databases, 6183 read error databases, and / or external databases of 634 systems. License plates read by a human operator can be confirmed by comparison with the results of automatic reading of license plates and / or multiple entries by many human operators.

Manual identification can be considered successful if the manually collected data weighed against determinable criteria for a positive vehicle match exceeds the specified identification confidence threshold (block 942). This determination can be made by computer 612, the operator who provided the manual data, and / or a more qualified operator.

In one embodiment, if the vehicle cannot be automatically positively identified, and no close matches are found, one or more images of the vehicle are displayed to the first human reviewer. The first human reviewer browses the photos and manually specifies the number of the license plate that he thinks corresponds to the vehicle based on the photos. Due to this manual review by the first human reviewer, it is also subject to error (e.g. error in perception or typography), the license plate read by the first human reviewer is compared to the LPR database to determine if the number plate specified by the first human reviewer exists. In addition, if there is a database register having fingerprint data corresponding to the number plate read, a fingerprint comparison can also be made. If the reading results of the first human reviewer do not match any known LPR or vehicle results, one or more vehicle photos may be displayed for the second human reviewer. A second human reviewer browses the photos and manually specifies the number of the license plate that he thinks corresponds to the vehicle based on the photos. If the reading result of the second human reviewer differs from the reading result of the first human reviewer, it may be necessary for the third human reviewer to read, who is usually a more qualified reviewer. In summary, the reading of the first human reviewer is effectively a stepping stone for attempting automatic matching again. If automatic matching still fails, many human reviewers need to match the license plate reading for the reading to be accurate.

If the vehicle is not successfully identified, computer 612 creates an event log as an unidentified or unassigned transaction (block 943). If the vehicle is successfully identified, computer 612 creates an event log as, for example, an accountable or non-profit transaction (block 937). If the vehicle has never been identified before, the 612 computer can create a new vehicle identification register for the vehicle and its owner / driver in the 6182 vehicle register database. The 612 computer can also update the 6183 reading error database to include vehicle identification data and a link linking extracted data vehicle identification with the appropriate vehicle identification register (block 938).

The above applications are illustrative examples, and the disclosed techniques can be used in other applications. Furthermore, various aspects and disclosed techniques (including systems and processes) may be modified, combined in whole or in part with each other, supplemented, or removed to create additional embodiments.

The systems and techniques described here can be implemented in digital electronic circuitry assemblies, or in hardware, firmware, software, or a combination thereof. The systems and techniques described herein may be implemented as a computer program product, namely, a computer program mandrel embodied in an information medium, e.g. in a machine-readable storage device or in a distributed signal, for performing by or for controlling an operation, a data processing device, e.g. a programmable processor, computer, or multiple computers. The computer program may be saved in any form of programming language, including compiled or interpreted languages, and may be used in any form, including as an autonomous program or as a module, component, subroutine, or other entity suitable for use in a computing environment .

The computer program can be used for use on one computer or on many computers in one place or distributed in many places and connected to a communication network.

The method steps of the systems and techniques described herein may be performed by one or more programmable processors executing a computer program to perform the function of the invention by operating on input data and generating output. The method steps can also be carried out by, and the device according to the invention can be implemented as logical circuits for special purposes, e.g. FPGA (user programmable input array) or ASIC (special purpose integrated circuit).

Processors suitable for executing a computer program include, for example, general and specific application microprocessors, and any one or more processors of any type of digital computer. Generally, the processor will receive instructions and data from read-only memory or sparse memory or both. Typical components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, the computer will also include, or will be operably connected to receive or transfer data, or both, to one or more mass storage devices for storing data, e.g., magnetic, magnetic optical discs, or optical discs. Information media suitable to contain computer program and data instructions include all forms of non-volatile memory, including, for example, semiconductor memory devices, e.g. EPROM, EEPROM., Flash memory devices; magnetic disks such as internal hard disks and removable disks; magnetic optical discs; and CD-ROMs and DVD-ROMs. The processor and memory can be supplemented by, or incorporated into, special purpose logic circuitry.

To ensure user interaction, the systems and techniques described herein can be implemented into a computer having a display such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor for displaying information to the user and a keyboard and pointing device such as a mouse or ball manipulator, thanks to which the user can provide a batch to the computer. Other types of devices may also be provided to ensure interaction with the user; for example, the feedback provided to the user may be in any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and the batch from the user may be received in any form, including batch acoustically, by speech, or by touch.

The systems and techniques described herein can be implemented in a computing system that includes an internal element, e.g., a data server, or includes a software and hardware element, e.g., an application server, or includes an external element, e.g., a client computer having a graphical user interface or a Web browser, by which the user can interact with an embodiment of the invention, or any combination of such internal, software-hardware, or external elements. Elements of the system can be connected through any form or medium of digital data communication, e.g. a communication network. Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

The computing system may include clients and servers. The client and server are generally spaced apart and usually interact through the communication network.

Client and server relationships are created thanks to computer programs that work on appropriate computers and have client-server interdependence.

Proxy:

LAW AND PATENT OFFICE "BELLEPAT"

IzaMa Szychulska-Hawranek and Słowackiego 44, 37-700 Przemyśl tel. (016) 732-37-77 fax: (016) 675-02-87 mobile phone (0608) 503-081 e-mail: <a href="mailto:bellepat@op.pl">bellepat@op.pl</a> NIP'795-207-16-72 REGON: 1803505 6

river

<img file="PL1897065T3_D0001.tif" />

ENT mgr Izabela, Iska-Hawranek No. 3192