Method of surveilling intennet communication

Methods of monitoring Internet communications

BACKGROUND OF THE INVENTION The present invention relates to a method and system for enabling monitoring and monitoring of a network by analyzing data passing therethrough.

A large number of businesses are passing through todays computer networks, but not all businesses are harmless. Therefore, the owner or administrator of a given network may be very interested in being able to track or "listen" in real time in order to effectively monitor and/or protect the network. This monitoring or monitoring can be achieved by connecting a probe to the network to monitor data passing between two of multiple nodes (for example, user workstations) on the network.

In a system where the communication between two nodes is in the form of discrete packets, the network probe can "read" a data packet to collect information such as the source and destination addresses of the packet or the packet's protocol. In addition, statistics and related information can be calculated, such as the total or average number of traffic of a certain protocol type in a given time period, or the total number of packets sent to or from a node. This information can be reported to the system administrator in real time or stored for later analysis.

Various attempts have been made in this regard. For example, in Lincolnshire, USA, the software program "Clear View Network Window" of Clear Communications Company III states that it provides predictive/proactive maintenance, intelligent root cause analysis, and quality verification reports. However, the output is a network fault management design, which is different from "eavesdropping" on the communication between two nodes in the network. In this way, the clear observation system does not allow monitoring of data transmitted between two nodes in the network regarding content or characteristics.

The Livermore National Laboratory in Livermore, California, USA has developed a set of computer programs to protect the U.S. Department of Energy's computers by "error testing" data packets through a local area network. The National Aeronautics and Space Administration used one of these programs, called the "iWatch" program, to eavesdrop on a computer-hackable communication that broke into the computer systems of the U.S. Department of Defense and NASA. The iWatch program uses a network probe to read all packets passing through a network, and then "store" the information in a public database. Then, you can write a simple computer program to read the stored data and display only the predefined "interesting" pieces of information.

Whenever an interesting piece of information is found, the stored information is rescanned, and a certain number of characters at both ends of the "interesting" information are reported. These interesting characters are then re-observed to determine the content of the message and used as a guide for future monitoring activities.

The system is limited to historical analysis of user activities, and cannot complete the "eavesdropping" of all user activities and the complete simulation of user surfing activities.

Three major problems were encountered in obtaining continuous and reliable tracking: (a) Each browser did not report all activities performed on a web server. For example, when a browser loads a web page from its browser cache space or from a proxy server, it does not send the request to any "remote" web server through the computer space; (b) is designed to be a web server of a manufacturer Applications that implement certain characteristics are usually incompatible with those made by another manufacturer, because the browser interface mechanisms are different and are proprietary to each of them; and (c) each browser comes in a non-system order Send their request to the web server. In other words, for a given web server, the previous request has nothing to do with a subsequent request. In the processing of requests, the website has no control over the sequence of requests.

In an attempt to overcome these problems, US Patent No. 5,951,643 pointed out a mechanism for trusted organization and management of information for web synchronization and tracking in multiple consumer browsers.

However, the solution is limited to tracking the activities of certain users who agree to be "eavesdropped" and are willing to cooperate and connect to a host with a designated application.

Therefore, the main purpose of the present invention is to provide a monitoring and monitoring method and system that enables a network communication manufacturer to eavesdrop on any user connected to the network.

Another object of the present invention is to provide an eavesdropping method that enables a network communication manufacturer to monitor the activities of all users in real time during network communication.

Another object of the present invention is to enable website owners to monitor and eavesdrop on users who contact their website.

Summary of the invention

According to the present invention, there is provided a method for simulating the browser ("initial browser") activity of a given terminal by a network probe terminal ("terminal agent") to track network communication lines, the method includes the steps of: access Network communication line, trace the TCP/IP data packet routed through the communication line, select TCP/IP data packet involving a given IP address; ("identified data packet"), select from the identified data packet for Newly connected current request ("initial request"), select the current web page component indicating the new address from the identified data group ("new navigation component"), and divide the new navigation component into two categories, embedded objects or frames (" Wrong new component"), hyperlink ("correct new component"), divide the initial request into the initial request that matches the correct new component, does not match any new connected component and belongs to HTTP or as the "main request" The initial request of POST type and the initial request matching the error component as the "secondary request" select the HTML data file related to the primary request from the identified data group; ("the respective primary response"), according to the respective secondary In response, a "virtual secondary request is generated, a response related to the secondary virtual request is selected from the identified data packet ("respective secondary response), and the web page presentation on the terminal agent is simulated according to the respective secondary response.

Description of the drawings

These and other features and advantages of the present invention can be understood more clearly by describing several preferred embodiments with reference to the accompanying drawings. Among them: Figure 1 illustrates a typical network configuration in which the present invention can be implemented; Figure 2 illustrates the terminal proxy Operation scheme; Figure 3 illustrates the process of tracking and identifying TCP/IP data packets; Figure 4 is a flow chart for classifying TCP/IP requests; Figure 5 is a flow chart for simulating the creation of virtual secondary TCP/IP requests; and Figure 6 Explain the process of simulating initial browser activity.

The preferred embodiment discusses Fig. 1 in detail, assuming that the terminals 01, 02... are connected to the same communication line, where the communication line is used as an intranet ("Intranet") or an extranet such as the Internet. According to the present invention, it is recommended to connect a designated network probe (hereinafter referred to as "terminal agent") to the data communication line. Alternatively, the terminals 01, 02, etc. and the terminal agent may be connected to different data communication lines or located in different local networks.

The general scheme of terminal agent operation is shown in Figure 2.

The terminal agent faces all data frames passing through the communication line. These data frames contain information transferred between terminals or external data transfers to external sources such as Internet servers.

It is also assumed that the "owner" of the data communication line of a network such as an ISP or a private organization is interested in real-time monitoring of the actual communication activities of a given terminal while surfing the Internet.

The operation of the terminal agent is first to analyze the data frame to track TCP/IP data packets. As shown in Figure 3, data analysis is processed according to different protocol levels (see Internet Protocol RFC0793), that is, first analyze the local network protocol, filter external data transmission ("gateway level"), and then identify Internet Protocol (IP) data Frame, and finally detect the "host level" TCP ("Transmission Control Protocol") data packet.

Once the IP header of the data packet is analyzed, the IP addresses of the requesting terminal and the message destination are identified. The owner of the communication line can easily associate the IP address with the user terminal. Therefore, it is possible to filter out all other irrelevant data packets, and further restrict the processing of data (hereinafter referred to as "identified data packets") transmission of a selected terminal.

The identified data packet is also processed in accordance with the RFC079 specification that enables complete management and control of the data communication port.

According to known routines for managing TCP data communication ports, as handled by traditional browsers such as Inter Explorer, the terminal operating the browser is the initial source of all data transmission. For example, suppose the terminal sends out for YAHOO! The request of the homepage, the request is distributed to YAHOO through the network! server. In response, the server sends an HTML data file containing the YAHOO homepage component. Therefore, the browser sends a new request for receiving all components of the webpage by opening a new communication "virtual" port, where each port is used to send different components of the same webpage. An "outsider" terminal facing all data requests and their respective responses cannot request a complete YAHOO for example! A distinction is made between the initial "primary" request for the home page and the "secondary" request for receiving its components. In order to emulate the initial browser activity by an "outsider" probe terminal, it is essentially the same identification of the main request.

Figure 4 illustrates the process of distinguishing primary requests from secondary requests. The main requests come from different operations, such as inputting a new URL by the user, selecting a hyperlink, etc. Therefore, in order to detect the same one, the previous information sent to the same IP address must be analyzed. All new navigation components of the webpage received by the terminal (addressing the browser to a new location) are classified according to their type, all embedded objects, frames, etc. are marked as "error" components, and hyperlinks are marked as "correct" Components. All data are stored in the buffer response database for later use.

When a request for a new connection is identified according to TCP analysis, the request is checked according to the respective navigation component (RNC) entering the response buffer. If the RNC is marked as "error", the request is ignored; if the RNC is marked as "correct", the request is classified as primary; otherwise, if there is no RNC involved in the request, the connection type should be identified . If the connection is of the HTML type or "post" type, it is classified as the main request.

In order to observe and monitor the activity of a terminal, all "initial" browser activity must be reconstructed. For this, it is recommended to use a "virtual" browser. The virtual browser has all the functions of a "real" browser to download web pages from the Internet in real time. However, its connection to the Internet is virtual, because no actual data exchange with the Internet server is performed, but only the initial "real" browser activity is simulated.

Figure 5 shows the first function of the virtual browser. The browser is receiving all major requests from the "real" browser. These main requests and respective main responses from the Internet are analyzed and processed according to traditional browser operations. However, the results of secondary virtual requests (used to complete the process of downloading webpage components in traditional browsers) are not directly transmitted via the Internet to the appropriate server as usual, but are stored in a virtual "secondary" request buffer. In the database.

Although virtual browser connections are not "actual", all TCP protocol management for opening and controlling port connections is handled by the terminal agent as if these connections are "actual".

The final process of simulating and presenting the webpage on the virtual browser is further shown in Figure 6. All initial secondary responses through the communication line are analyzed and recorded in the response buffer database. The virtual requests are compared with the respective secondary responses stored in the incoming response buffer database in the order of their arrival. If the respective secondary responses are already in the buffer, these responses are transferred to the virtual browser and processed (according to traditional browser operations) to present visual images of the respective webpage components. As a result, the terminal agent simulates the precise process of downloading Internet web pages in real time, since it has been executed by the initial terminal.

If the respective response does not appear in the incoming response buffer database, the initial local cache activity is inferred. If the initial local cache is not used for the virtual request, its execution is suspended in the cache database until the initial secondary respective response arrives. Otherwise, if the actual local cache is used to refer to the response, the local cache of the virtual browser is checked, and if the respective secondary response is in the local cache, the respective response is transmitted to the virtual browser and It is processed as described above. If the respective responses are not in the virtual cache, any of the following alternatives can be applied. According to an alternative, the "passive" scheme of the terminal agent, no further action is taken to find the "missing" response, and an "error" message will appear on the agent terminal instead of the webpage component that appears on the actual terminal. According to this scheme, the simulation of the actual terminal is not completed, but the eavesdropping activity is not detectable. According to another scheme, the "active" scheme, the terminal agent addresses the web server to request a "missing" response. Although this solution enables the terminal agent to present a more accurate picture of the actual active activity, it is trackable to more experienced terminal users who can detect eavesdropping activity.

According to another implementation mode of the present invention, it is recommended not only to eavesdrop on related webpage data packets, but also related message data packets, such as e-mail or chat. The network initiates this kind of eavesdropping, and the same methods and principles as described above are applied to requests for receiving and sending messages instead of requests for web pages. The process of analyzing such requests and their respective responses is more pipelined, because there is no need to check cache memory activity, because by definition, this information is always new.

Finally, it should be understood that the above-mentioned embodiments are aimed at the Internet communication environment. However, the present invention can also be generally applied to computerized network communications in a broad sense, such as satellites, cellular, etc.

Although the above description contains many characteristics, they are not intended to limit the scope of the present invention, but rather serve as examples of preferred embodiments. Those skilled in the art can imagine other possible variations are also within the scope of the present invention. Therefore, the scope of the present invention should be determined not only by the illustrated embodiments, but also by the appended claims and their legal equivalents.