Selective event stream data storage based on historical stream data

RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No. 14/699,807, filed Apr. 29, 2015, which application is a continuation-in-part application of U.S. patent application Ser. No. 14/610,408, entitled “Grouping and Managing Event Streams Generated from Captured Network Data,” by inventors Fang I. Hsiao, Clayton S. Ching, Michael R. Dickey, Vladimir A. Shcherbakov, Nishant Teredesai and Cary Glen Noel, filed 30 Jan. 2015, issued as U.S. Pat. No. 10,360,196. U.S. patent application Ser. No. 14/610,408 is itself a continuation-in-part application of application Ser. No. 14/253,713, entitled “Distributed Processing of Network Data Using Remote Capture Agents,” by inventor Michael R. Dickey, filed 15 Apr. 2014, issued as U.S. Pat. No. 10,127,273. U.S. patent application Ser. No. 14/610,408 is also a continuation-in-part application of U.S. patent application Ser. No. 14/528,898, entitled “Protocol-Based Capture of Network Data Suing Remote Capture Agents,” by inventors Vladimir A. Shcherbakov and Michael R. Dickey, filed 30 Oct. 2014, issued as U.S. Pat. No. 9,838,512.

The subject matter of this application is also related to the subject matter in a co-pending non-provisional application by the same inventors as the instant application, entitled “Adjusting Network Data Storage Based on Event Stream Statistics,” having Ser. No. 14/699,787, and filed on 29 Apr. 2015.

BACKGROUND

Field

The disclosed embodiments relate to techniques for processing network data. More specifically, the disclosed embodiments relate to techniques for providing visualizations of statistics associated with captured network data.

Related Art

SUMMARY

Large volumes of performance and log data may be captured as “events,” wherein each event includes a collection of performance data and/or diagnostic information that is generated by a computer system and is correlated with a specific point in time. Events can be derived from “time-series event data,” wherein time-series data comprises a sequence of data points (e.g., performance measurements from a computer system) that are associated with successive points in time and are typically spaced at uniform time intervals. More specifically, an event stream of time-series event data may be generated from wire data, such as network packets, captured by a number of remote capture agents deployed across a network. The remote capture agents may be installed on physical servers and/or virtual machines on the network. As a result, the remote capture agents may avert the need to deploy and connect physical hardware to network TAPS or SPAN ports, thus allowing users to configure and change their data capture configuration on-the-fly rather than in fixed formats.

Configuration or management of event streams generated from network packets captured by the remote capture agents may be performed through a configuration server and/or GUI. The configuration server and/or GUI may allow a user (e.g., an administrator) to specify a protocol used by network packets from which an event stream is created. Because such protocol-based capture and analysis of network data may result in the capture of multiple protocols in a large number of event streams, event stream information for the event streams may be grouped by one or more event stream attributes (e.g., protocol, application, category, event stream lifecycle) in the GUI. Such grouping(s) of the event stream information may facilitate analysis, understanding, and management of the event streams by the user.

The configuration server and/or GUI may also include a number of mechanisms or user-interface elements that further assist the user with management and use of the event streams. First, the configuration server and/or GUI may enable the generation of a set of statistics from an event stream without subsequently storing and processing the event stream by one or more components on a network. Alternatively, the GUI may enable the selective storage and/or processing of at least a portion of the event stream based on the statistics, a storage limit associated with the time-series event data, and/or user input through the GUI.

Second, the GUI may display the statistics and/or one or more graphs containing values from the statistics, along with a value of a statistic based on a position of a cursor over the graph(s). For example, the GUI may display a bar chart and/or pie chart of index volume for one or more event streams across a pre-specified time range. The GUI may also display the value of the index volume represented by a segment of the bar chart and/or a slice of the pie chart over which the cursor is positioned. The GUI may further highlight the segment and/or slice and dim other portions of the chart. Consequently, the configuration server and/or GUI may improve user understanding and/or decision-making related to partial or complete storage, indexing, and/or processing of event streams.

Thus, the disclosed embodiments provide a system that facilitates the processing of network data. During operation, the system causes for display a graphical user interface (GUI) for configuring the generation of time-series event data from network packets captured by one or more remote capture agents. Next, the system causes for display, in the GUI, a first set of user-interface elements containing a set of statistics associated with one or more event streams that comprise the time-series event data. The system then causes for display, in the GUI, one or more graphs comprising one or more values from the set of statistics. Finally, the system causes for display, in the GUI, a value of a statistic from the set of statistics based on a position of a cursor over the one or more graphs.

In some embodiments, the system also updates the set of statistics and the one or more graphs in real-time with the time-series event data from the one or more remote capture agents.

In some embodiments, the system also causes for display, in the GUI, a second set of user-interface elements for changing a view of the one or more graphs.

In some embodiments, changing the view of the one or more graphs includes at least one of: <ul id="ul0001" list-style="none"><li id="ul0001-0001" num="0000"><ul id="ul0002" list-style="none"><li id="ul0002-0001" num="0014">(i) changing a scale of a graph;</li><li id="ul0002-0002" num="0015">(ii) changing a time range associated with the set of statistics used to generate the graph; and</li><li id="ul0002-0003" num="0016">(iii) changing a host associated with the time-series event data and the set of statistics.</li></ul></li></ul>

In some embodiments, the system also sorts the set of statistics by an attribute associated with the set of statistics.

In some embodiments, the attribute is at least one of a name, a total number of events, a total incoming traffic, a total outgoing traffic, a total traffic, and an index volume.

In some embodiments, the one or more graphs include a pie chart of an index volume across the one or more event streams.

In some embodiments, the pie chart is updated with one or more values of an index volume of an event stream based on a position of a cursor over the pie chart.

In some embodiments, the one or more values include a percentage of a total index volume associated with the event stream and an amount of data associated with the index volume of the event stream.

In some embodiments, the set of statistics include at least one of a total number of events, a total incoming traffic, a total outgoing traffic, a total traffic, and an index volume.

In some embodiments, the one or more graphs include a bar chart of an index volume of the one or more event streams over time.

In some embodiments, a bar in the bar chart includes one or more segments representing the index volume of the one or more event streams over a time interval.

In some embodiments, the bar chart is updated with a value of an index volume of an event stream based on a position of a cursor over the bar chart.

In some embodiments, a portion of the bar chart is highlighted based on a position of a cursor over a legend of the bar chart.

In some embodiments the GUI further includes a second set of user-interface elements for managing the one or more event streams, and managing the one or more event streams includes enabling the generation of the set of statistics from an event stream without transmitting the event stream over a network for subsequent storage and processing of the event stream by one or more components on the network.

In some embodiments, managing the one or more event streams comprises adjusting an amount of capture of the one or more event streams based on the set of statistics.

In some embodiments, the system also causes for display a value of a statistic from the set of statistics based on a position of a cursor over a legend associated with the one or more graphs.

In some embodiments, the value of the statistic includes the index volume associated with the position of the cursor over the one or more graph.

In some embodiments, the value of the statistic is at least one of a percentage of a total index volume associated with the one or more event streams and an amount of data associated with the index volume of the one or more event streams.

In some embodiments, the system also changes an appearance of the one or more graphs based on the position of the cursor over the one or more graphs.

In some embodiments, changing the appearance of the one or more graphs includes highlighting a portion of a graph based on the position of the cursor over the graph.

In some embodiments, changing the appearance of the one or more graphs includes dimming a portion of a graph based on the position of the cursor over the graph.

In some embodiments, changing the appearance of the one or more graphs includes highlighting a first portion of a graph and dimming a second portion of the graph based on the position of the cursor over the graph.

In some embodiments, the system also changes an appearance of the one or more graphs based on the position of the cursor over a legend associated with the one or more graphs.

In some embodiments changing the appearance of the one or more graphs includes highlighting a portion of a graph based on the position of the cursor over the legend.

In some embodiments, changing the appearance of the one or more graphs includes dimming a portion of a graph based on the position of the cursor over the legend.

In some embodiments, changing the appearance of the one or more graphs includes highlighting a first portion of a graph and dimming a second portion of the graph based on the position of the cursor over the legend.

In some embodiments, causing for display, in the GUI, the value of the statistic includes at least one of: <ul id="ul0003" list-style="none"><li id="ul0003-0001" num="0000"><ul id="ul0004" list-style="none"><li id="ul0004-0001" num="0041">(i) including the value of the statistic in a user-interface element of the GUI;</li><li id="ul0004-0002" num="0042">(ii) including a name of the statistic in the user-interface element;</li><li id="ul0004-0003" num="0043">(iii) including an identifier for the event stream in the user-interface element; and</li><li id="ul0004-0004" num="0044">(iv) displaying the user-interface element next to the position of the cursor.</li></ul></li></ul>

BRIEF DESCRIPTION OF THE FIGURES

The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

<figref idref="DRAWINGS">FIG. 1</figref> shows a block diagram of an exemplary event-processing system in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 2</figref> presents a flowchart illustrating how indexers process, index, and store data received from forwarders in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 3</figref> presents a flowchart illustrating how a search head and indexers perform a search query in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 4</figref> presents a block diagram of a system for processing search requests that uses extraction rules for field values in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 5</figref> illustrates an exemplary search query received from a client and executed by search peers in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 6A</figref> illustrates a search screen in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 6B</figref> illustrates a data summary dialog that enables a user to select various data sources in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 7A</figref> illustrates a key indicators view in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 7B</figref> illustrates an incident review dashboard in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 7C</figref> illustrates a proactive monitoring tree in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 7D</figref> illustrates a screen displaying both log data and performance data in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 8</figref> shows a schematic of a system in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 9A</figref> shows a remote capture agent in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 9B</figref> shows the protocol-based capture of network data using a remote capture agent in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 10</figref> shows a configuration server in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 11A</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 11B</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 11C</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 11D</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 11E</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 11F</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 12A</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 12B</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 13</figref> shows a flowchart illustrating the processing of network data in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 14</figref> shows a flowchart illustrating the process of using configuration information associated with a protocol classification to build an event stream from a packet flow in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 15</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 16</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 17A</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 17B</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 17C</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 17D</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 17E</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 18</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 19</figref> shows a flowchart illustrating the process of displaying event stream information represented by a grouping of the event streams by an event stream attribute in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 20</figref> presents a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 21</figref> presents a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 22</figref> presents a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 23A</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 23B</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 23C</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 24A</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 24B</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 24C</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 24D</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 24E</figref> shows an exemplary screenshot in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 25</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 26</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 27</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments.

<figref idref="DRAWINGS">FIG. 28</figref> shows a computer system in accordance with the disclosed embodiments.

In the figures, like reference numerals refer to the same figure elements.

DETAILED DESCRIPTION

The following description is presented to enable any person skilled in the art to make and use the embodiments, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Thus, the present invention is not limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

The data structures and code described in this detailed description are typically stored on a computer-readable storage medium, which may be any device or medium that can store code and/or data for use by a computer system. The computer-readable storage medium includes, but is not limited to, volatile memory, non-volatile memory, magnetic and optical storage devices such as disk drives, magnetic tape, CDs (compact discs), DVDs (digital versatile discs or digital video discs), or other media capable of storing code and/or data now known or later developed.

The methods and processes described in the detailed description section can be embodied as code and/or data, which can be stored in a computer-readable storage medium as described above. When a computer system reads and executes the code and/or data stored on the computer-readable storage medium, the computer system performs the methods and processes embodied as data structures and code and stored within the computer-readable storage medium.

Furthermore, methods and processes described herein can be included in hardware modules or apparatus. These modules or apparatus may include, but are not limited to, an application-specific integrated circuit (ASIC) chip, a field-programmable gate array (FPGA), a dedicated or shared processor that executes a particular software module or a piece of code at a particular time, and/or other programmable-logic devices now known or later developed. When the hardware modules or apparatus are activated, they perform the methods and processes included within them.

1.1 Overview

Modern data centers often comprise thousands of host computer systems that operate collectively to service requests from even larger numbers of remote clients. During operation, these data centers generate significant volumes of performance data and diagnostic information that can be analyzed to quickly diagnose performance problems. In order to reduce the size of this performance data, the data is typically pre-processed prior to being stored based on anticipated data-analysis needs. For example, pre-specified data items can be extracted from the performance data and stored in a database to facilitate efficient retrieval and analysis at search time. However, the rest of the performance data is not saved and is essentially discarded during pre-processing. As storage capacity becomes progressively cheaper and more plentiful, there are fewer incentives to discard this performance data and many reasons to keep it.

This plentiful storage capacity is presently making it feasible to store massive quantities of minimally processed performance data at “ingestion time” for later retrieval and analysis at “search time.” Note that performing the analysis operations at search time provides greater flexibility because it enables an analyst to search all of the performance data, instead of searching pre-specified data items that were stored at ingestion time. This enables the analyst to investigate different aspects of the performance data instead of being confined to the pre-specified set of data items that was selected at ingestion time.

However, analyzing massive quantities of heterogeneous performance data at search time can be a challenging task. A data center may generate heterogeneous performance data from thousands of different components, which can collectively generate tremendous volumes of performance data that can be time-consuming to analyze. For example, this performance data can include data from system logs, network packet data, sensor data, and data generated by various applications. Also, the unstructured nature of much of this performance data can pose additional challenges because of the difficulty of applying semantic meaning to unstructured data, and the difficulty of indexing and querying unstructured data using traditional database systems.

These challenges can be addressed by using an event-based system, such as the SPLUNK® ENTERPRISE system produced by Splunk Inc. of San Francisco, Calif., to store and process performance data. The SPLUNK® ENTERPRISE system is the leading platform for providing real-time operational intelligence that enables organizations to collect, index, and harness machine-generated data from various websites, applications, servers, networks, and mobile devices that power their businesses. The SPLUNK® ENTERPRISE system is particularly useful for analyzing unstructured performance data, which is commonly found in system log files. Although many of the techniques described herein are explained with reference to the SPLUNK® ENTERPRISE system, the techniques are also applicable to other types of data server systems.

In the SPLUNK® ENTERPRISE system, performance data is stored as “events,” wherein each event comprises a collection of performance data and/or diagnostic information that is generated by a computer system and is correlated with a specific point in time. Events can be derived from “time-series data,” wherein time-series data comprises a sequence of data points (e.g., performance measurements from a computer system) that are associated with successive points in time and are typically spaced at uniform time intervals. Events can also be derived from “structured” or “unstructured” data. Structured data has a predefined format, wherein specific data items with specific data formats reside at predefined locations in the data. For example, structured data can include data items stored in fields in a database table. In contrast, unstructured data does not have a predefined format. This means that unstructured data can comprise various data items having different data types that can reside at different locations. For example, when the data source is an operating system log, an event can include one or more lines from the operating system log containing raw data that includes different types of performance and diagnostic information associated with a specific point in time. Examples of data sources from which an event may be derived include, but are not limited to: web servers; application servers; databases; firewalls; routers; operating systems; and software applications that execute on computer systems, mobile devices, and sensors. The data generated by such data sources can be produced in various forms including, for example and without limitation, server log files, activity log files, configuration files, messages, network packet data, performance measurements and sensor measurements. An event typically includes a timestamp that may be derived from the raw data in the event, or may be determined through interpolation between temporally proximate events having known timestamps.

The SPLUNK® ENTERPRISE system also facilitates using a flexible schema to specify how to extract information from the event data, wherein the flexible schema may be developed and redefined as needed. Note that a flexible schema may be applied to event data “on the fly,” when it is needed (e.g., at search time), rather than at ingestion time of the data as in traditional database systems. Because the schema is not applied to event data until it is needed (e.g., at search time), it is referred to as a “late-binding schema.”

During operation, the SPLUNK® ENTERPRISE system starts with raw data, which can include unstructured data, machine data, performance measurements or other time-series data, such as data obtained from weblogs, syslogs, or sensor readings. It divides this raw data into “portions,” and optionally transforms the data to produce timestamped events. The system stores the timestamped events in a data store, and enables a user to run queries against the data store to retrieve events that meet specified criteria, such as containing certain keywords or having specific values in defined fields. Note that the term “field” refers to a location in the event data containing a value for a specific data item.

As noted above, the SPLUNK® ENTERPRISE system facilitates using a late-binding schema while performing queries on events. A late-binding schema specifies “extraction rules” that are applied to data in the events to extract values for specific fields. More specifically, the extraction rules for a field can include one or more instructions that specify how to extract a value for the field from the event data. An extraction rule can generally include any type of instruction for extracting values from data in events. In some cases, an extraction rule comprises a regular expression, in which case the rule is referred to as a “regex rule.”

In contrast to a conventional schema for a database system, a late-binding schema is not defined at data ingestion time. Instead, the late-binding schema can be developed on an ongoing basis until the time at which a query is actually executed. This means that extraction rules for the fields in a query may be provided in the query itself, or may be located during execution of the query. Hence, as an analyst learns more about the data in the events, the analyst can continue to refine the late-binding schema by adding new fields, deleting fields, or changing the field extraction rules until the next time the schema is used by a query. Because the SPLUNK® ENTERPRISE system maintains the underlying raw data and provides a late-binding schema for searching the raw data, it enables an analyst to investigate questions that arise as the analyst learns more about the events.

In the SPLUNK® ENTERPRISE system, a field extractor may be configured to automatically generate extraction rules for certain fields in the events when the events are being created, indexed, or stored, or possibly at a later time. Alternatively, a user may manually define extraction rules for fields using a variety of techniques.

Also, a number of “default fields” that specify metadata about the events rather than data in the events themselves can be created automatically. For example, such default fields can specify: a timestamp for the event data; a host from which the event data originated; a source of the event data; and a source type for the event data. These default fields may be determined automatically when the events are created, indexed or stored.

In some embodiments, a common field name may be used to reference two or more fields containing equivalent data items, even though the fields may be associated with different types of events that possibly have different data formats and different extraction rules. By enabling a common field name to be used to identify equivalent fields from different types of events generated by different data sources, the system facilitates use of a “common information model” (CIM) across the different data sources.

1.2 Data Server System

<figref idref="DRAWINGS">FIG. 1</figref> shows a block diagram of an exemplary event-processing system <b>100</b>, similar to the SPLUNK® ENTERPRISE system. System <b>100</b> includes one or more forwarders <b>101</b> that collect data obtained from a variety of different data sources <b>105</b>, and one or more indexers <b>102</b> that store, process, and/or perform operations on this data, wherein each indexer operates on data contained in a specific data store <b>103</b>. These forwarders and indexers can comprise separate computer systems in a data center, or may alternatively comprise separate processes executing on various computer systems in a data center.

During operation, forwarders <b>101</b> identify which indexers <b>102</b> will receive the collected data and then forward the data to the identified indexers <b>102</b>. Forwarders <b>101</b> can also perform operations to strip extraneous data and detect timestamps in the data. Forwarders <b>101</b> may next determine which indexers <b>102</b> will receive each data item and forward the data items to the determined indexers <b>102</b>. Indexers <b>102</b> may then provide the data for storage in one or more data stores <b>103</b>.

As mentioned above, the data may include streams, logs, database records, messages, archives, and/or other records containing time-series data. Time-series data refers to any data that can be associated with a timestamp. The data can be structured, unstructured, or semi-structured and come from files or directories. Unstructured data may include data, such as machine data and web logs, that is not organized to facilitate extraction of values for fields from the data.

Note that distributing data across different indexers facilitates parallel processing. This parallel processing can take place at data ingestion time, because multiple indexers can process the incoming data in parallel. The parallel processing can also take place at search time, because multiple indexers can search the data in parallel.

System <b>100</b> and the processes described below with respect to <figref idref="DRAWINGS">FIGS. 1-5</figref> are further described in “Exploring Splunk Search Processing Language (SPL) Primer and Cookbook,” by David Carasso, CITO Research, 2012, and in “Optimizing Data Analysis With a Semi-Structured Time-series Database,” by Ledion Bitincka, Archana Ganapathi, Stephen Sorkin, and Steve Zhang, SLAML, 2010, each of which is hereby incorporated herein by reference in its entirety for all purposes.

1.3 Data Ingestion

<figref idref="DRAWINGS">FIG. 2</figref> presents a flowchart illustrating how an indexer processes, indexes, and stores data received from forwarders in accordance with the disclosed embodiments. At block <b>201</b>, the indexer receives the data from the forwarder. Next, at block <b>202</b>, the indexer apportions the data into events. Note that the data can include lines of text that are separated by carriage returns or line breaks and an event may include one or more of these lines. During the apportioning process, the indexer can use heuristic rules to automatically determine the boundaries of the events, which for example coincide with line boundaries. These heuristic rules may be determined based on the source of the data, wherein the indexer can be explicitly informed about the source of the data or can infer the source of the data by examining the data. These heuristic rules can include regular expression-based rules or delimiter-based rules for determining event boundaries, wherein the event boundaries may be indicated by predefined characters or character strings. These predefined characters may include punctuation marks or other special characters including, for example, carriage returns, tabs, spaces or line breaks. In some cases, a user can fine-tune or configure the rules that the indexers use to determine event boundaries in order to adapt the rules to the user's specific requirements.

Next, the indexer determines a timestamp for each event at block <b>203</b>. As mentioned above, these timestamps can be determined by extracting the time directly from data in the event, or by interpolating the time based on timestamps from temporally proximate events. In some cases, a timestamp can be determined based on the time the data was received or generated. The indexer subsequently associates the determined timestamp with each event at block <b>204</b>, for example by storing the timestamp as metadata for each event.

Then, the system can apply transformations to data to be included in events at block <b>205</b>. For log data, such transformations can include removing a portion of an event (e.g., a portion used to define event boundaries, extraneous text, characters, etc.) or removing redundant portions of an event. Note that a user can specify portions to be removed using a regular expression or any other possible technique.

Next, a keyword index can optionally be generated to facilitate fast keyword searching for events. To build a keyword index, the indexer first identifies a set of keywords in block <b>206</b>. Then, at block <b>207</b> the indexer includes the identified keywords in an index, which associates each stored keyword with references to events containing that keyword (or to locations within events where that keyword is located). When an indexer subsequently receives a keyword-based query, the indexer can access the keyword index to quickly identify events containing the keyword.

In some embodiments, the keyword index may include entries for name-value pairs found in events, wherein a name-value pair can include a pair of keywords connected by a symbol, such as an equals sign or colon. In this way, events containing these name-value pairs can be quickly located. In some embodiments, fields can automatically be generated for some or all of the name-value pairs at the time of indexing. For example, if the string “dest=10.0.1.2” is found in an event, a field named “dest” may be created for the event, and assigned a value of “10.0.1.2.”

Finally, the indexer stores the events in a data store at block <b>208</b>, wherein a timestamp can be stored with each event to facilitate searching for events based on a time range. In some cases, the stored events are organized into a plurality of buckets, wherein each bucket stores events associated with a specific time range. This not only improves time-based searches, but it also allows events with recent timestamps that may have a higher likelihood of being accessed to be stored in faster memory to facilitate faster retrieval. For example, a bucket containing the most recent events can be stored as flash memory instead of on hard disk.

Each indexer <b>102</b> is responsible for storing and searching a subset of the events contained in a corresponding data store <b>103</b>. By distributing events among the indexers and data stores, the indexers can analyze events for a query in parallel, for example using map-reduce techniques, wherein each indexer returns partial responses for a subset of events to a search head that combines the results to produce an answer for the query. By storing events in buckets for specific time ranges, an indexer may further optimize searching by looking only in buckets for time ranges that are relevant to a query.

Moreover, events and buckets can also be replicated across different indexers and data stores to facilitate high availability and disaster recovery as is described in U.S. patent application Ser. No. 14/266,812 filed on 30 Apr. 2014, and in U.S. patent application Ser. No. 14/266,817 also filed on 30 Apr. 2014.

1.4 Query Processing

<figref idref="DRAWINGS">FIG. 3</figref> presents a flowchart illustrating how a search head and indexers perform a search query in accordance with the disclosed embodiments. At the start of this process, a search head receives a search query from a client at block <b>301</b>. Next, at block <b>302</b>, the search head analyzes the search query to determine what portions can be delegated to indexers and what portions need to be executed locally by the search head. At block <b>303</b>, the search head distributes the determined portions of the query to the indexers. Note that commands that operate on single events can be trivially delegated to the indexers, while commands that involve events from multiple indexers are harder to delegate.

Then, at block <b>304</b>, the indexers to which the query was distributed search their data stores for events that are responsive to the query. To determine which events are responsive to the query, the indexer searches for events that match the criteria specified in the query. This criteria can include matching keywords or specific values for certain fields. In a query that uses a late-binding schema, the searching operations in block <b>304</b> may involve using the late-binding schema to extract values for specified fields from events at the time the query is processed. Next, the indexers can either send the relevant events back to the search head, or use the events to calculate a partial result, and send the partial result back to the search head.

Finally, at block <b>305</b>, the search head combines the partial results and/or events received from the indexers to produce a final result for the query. This final result can comprise different types of data depending upon what the query is asking for. For example, the final results can include a listing of matching events returned by the query, or some type of visualization of data from the returned events. In another example, the final result can include one or more calculated values derived from the matching events.

Moreover, the results generated by system <b>100</b> can be returned to a client using different techniques. For example, one technique streams results back to a client in real-time as they are identified. Another technique waits to report results to the client until a complete set of results is ready to return to the client. Yet another technique streams interim results back to the client in real-time until a complete set of results is ready, and then returns the complete set of results to the client. In another technique, certain results are stored as “search jobs,” and the client may subsequently retrieve the results by referencing the search jobs.

The search head can also perform various operations to make the search more efficient. For example, before the search head starts executing a query, the search head can determine a time range for the query and a set of common keywords that all matching events must include. Next, the search head can use these parameters to query the indexers to obtain a superset of the eventual results. Then, during a filtering stage, the search head can perform field-extraction operations on the superset to produce a reduced set of search results.

1.5 Field Extraction

<figref idref="DRAWINGS">FIG. 4</figref> presents a block diagram illustrating how fields can be extracted during query processing in accordance with the disclosed embodiments. At the start of this process, a search query <b>402</b> is received at a query processor <b>404</b>. Query processor <b>404</b> includes various mechanisms for processing a query, wherein these mechanisms can reside in a search head <b>104</b> and/or an indexer <b>102</b>. Note that the exemplary search query <b>402</b> illustrated in <figref idref="DRAWINGS">FIG. 4</figref> is expressed in Search Processing Language (SPL), which is used in conjunction with the SPLUNK® ENTERPRISE system. SPL is a pipelined search language in which a set of inputs is operated on by a first command in a command line, and then a subsequent command following the pipe symbol “|” operates on the results produced by the first command, and so on for additional commands. Search query <b>402</b> can also be expressed in other query languages, such as the Structured Query Language (“SQL”) or any suitable query language.

Upon receiving search query <b>402</b>, query processor <b>404</b> sees that search query <b>402</b> includes two fields “IP” and “target.” Query processor <b>404</b> also determines that the values for the “IP” and “target” fields have not already been extracted from events in data store <b>414</b>, and consequently determines that query processor <b>404</b> needs to use extraction rules to extract values for the fields. Hence, query processor <b>404</b> performs a lookup for the extraction rules in a rule base <b>406</b>, wherein rule base <b>406</b> maps field names to corresponding extraction rules and obtains extraction rules <b>408</b>-<b>409</b>, wherein extraction rule <b>408</b> specifies how to extract a value for the “IP” field from an event, and extraction rule <b>409</b> specifies how to extract a value for the “target” field from an event. As is illustrated in <figref idref="DRAWINGS">FIG. 4</figref>, extraction rules <b>408</b>-<b>409</b> can comprise regular expressions that specify how to extract values for the relevant fields. Such regular-expression-based extraction rules are also referred to as “regex rules.” In addition to specifying how to extract field values, the extraction rules may also include instructions for deriving a field value by performing a function on a character string or value retrieved by the extraction rule. For example, a transformation rule may truncate a character string, or convert the character string into a different data format. In some cases, the query itself can specify one or more extraction rules.

Next, query processor <b>404</b> sends extraction rules <b>408</b>-<b>409</b> to a field extractor <b>412</b>, which applies extraction rules <b>408</b>-<b>409</b> to events <b>416</b>-<b>418</b> in a data store <b>414</b>. Note that data store <b>414</b> can include one or more data stores, and extraction rules <b>408</b>-<b>409</b> can be applied to large numbers of events in data store <b>414</b>, and are not meant to be limited to the three events <b>416</b>-<b>418</b> illustrated in <figref idref="DRAWINGS">FIG. 4</figref>. Moreover, the query processor <b>404</b> can instruct field extractor <b>412</b> to apply the extraction rules to all the events in a data store <b>414</b>, or to a subset of the events that have been filtered based on some criteria.

Next, field extractor <b>412</b> applies extraction rule <b>408</b> for the first command “Search IP=“10*” to events in data store <b>414</b> including events <b>416</b>-<b>418</b>. Extraction rule <b>408</b> is used to extract values for the IP address field from events in data store <b>414</b> by looking for a pattern of one or more digits, followed by a period, followed again by one or more digits, followed by another period, followed again by one or more digits, followed by another period, and followed again by one or more digits. Next, field extractor <b>412</b> returns field values <b>420</b> to query processor <b>404</b>, which uses the criterion IP=“10*” to look for IP addresses that start with “10”. Note that events <b>416</b> and <b>417</b> match this criterion, but event <b>418</b> does not, so the result set for the first command is events <b>416</b>-<b>417</b>.

Query processor <b>404</b> then sends events <b>416</b>-<b>417</b> to the next command “stats count target.” To process this command, query processor <b>404</b> causes field extractor <b>412</b> to apply extraction rule <b>409</b> to events <b>416</b>-<b>417</b>. Extraction rule <b>409</b> is used to extract values for the target field for events <b>416</b>-<b>417</b> by skipping the first four commas in events <b>416</b>-<b>417</b>, and then extracting all of the following characters until a comma or period is reached. Next, field extractor <b>412</b> returns field values <b>421</b> to query processor <b>404</b>, which executes the command “stats count target” to count the number of unique values contained in the target fields, which in this example produces the value “2” that is returned as a final result <b>422</b> for the query.

Note that query results can be returned to a client, a search head, or any other system component for further processing. In general, query results may include: a set of one or more events; a set of one or more values obtained from the events; a subset of the values; statistics calculated based on the values; a report containing the values; or a visualization, such as a graph or chart, generated from the values.

1.6 Exemplary Search Screen

<figref idref="DRAWINGS">FIG. 6A</figref> illustrates an exemplary search screen <b>600</b> in accordance with the disclosed embodiments. Search screen <b>600</b> includes a search bar <b>602</b> that accepts user input in the form of a search string. It also includes a time range picker <b>612</b> that enables the user to specify a time range for the search. For “historical searches” the user can select a specific time range, or alternatively a relative time range, such as “today,” “yesterday” or “last week.” For “real-time searches,” the user can select the size of a preceding time window to search for real-time events. Search screen <b>600</b> also initially displays a “data summary” dialog as is illustrated in <figref idref="DRAWINGS">FIG. 6B</figref> that enables the user to select different sources for the event data, for example by selecting specific hosts and log files.

After the search is executed, the search screen <b>600</b> can display the results through search results tabs <b>604</b>, wherein search results tabs <b>604</b> includes: an “events tab” that displays various information about events returned by the search; a “statistics tab” that displays statistics about the search results; and a “visualization tab” that displays various visualizations of the search results. The events tab illustrated in <figref idref="DRAWINGS">FIG. 6A</figref> displays a timeline <b>605</b> that graphically illustrates the number of events that occurred in one-hour intervals over the selected time range. It also displays an events list <b>608</b> that enables a user to view the raw data in each of the returned events. It additionally displays a fields sidebar <b>606</b> that includes statistics about occurrences of specific fields in the returned events, including “selected fields” that are pre-selected by the user, and “interesting fields” that are automatically selected by the system based on pre-specified criteria.

1.7 Acceleration Techniques

The above-described system provides significant flexibility by enabling a user to analyze massive quantities of minimally processed performance data “on the fly” at search time instead of storing pre-specified portions of the performance data in a database at ingestion time. This flexibility enables a user to see correlations in the performance data and perform subsequent queries to examine interesting aspects of the performance data that may not have been apparent at ingestion time.

However, performing extraction and analysis operations at search time can involve a large amount of data and require a large number of computational operations, which can cause considerable delays while processing the queries. Fortunately, a number of acceleration techniques have been developed to speed up analysis operations performed at search time. These techniques include: (1) performing search operations in parallel by formulating a search as a map-reduce computation; (2) using a keyword index; (3) using a high performance analytics store; and (4) accelerating the process of generating reports. These techniques are described in more detail below.

1.7.1 Map-Reduce Technique

To facilitate faster query processing, a query can be structured as a map-reduce computation, wherein the “map” operations are delegated to the indexers, while the corresponding “reduce” operations are performed locally at the search head. For example, <figref idref="DRAWINGS">FIG. 5</figref> illustrates how a search query <b>501</b> received from a client at search head <b>104</b> can split into two phases, including: (1) a “map phase” comprising subtasks <b>502</b> (e.g., data retrieval or simple filtering) that may be performed in parallel and are “mapped” to indexers <b>102</b> for execution, and (2) a “reduce phase” comprising a merging operation <b>503</b> to be executed by the search head when the results are ultimately collected from the indexers.

During operation, upon receiving search query <b>501</b>, search head <b>104</b> modifies search query <b>501</b> by substituting “stats” with “prestats” to produce search query <b>502</b>, and then distributes search query <b>502</b> to one or more distributed indexers, which are also referred to as “search peers.” Note that search queries may generally specify search criteria or operations to be performed on events that meet the search criteria. Search queries may also specify field names, as well as search criteria for the values in the fields or operations to be performed on the values in the fields. Moreover, the search head may distribute the full search query to the search peers as is illustrated in <figref idref="DRAWINGS">FIG. 3</figref>, or may alternatively distribute a modified version (e.g., a more restricted version) of the search query to the search peers. In this example, the indexers are responsible for producing the results and sending them to the search head. After the indexers return the results to the search head, the search head performs the merging operations <b>503</b> on the results. Note that by executing the computation in this way, the system effectively distributes the computational operations while minimizing data transfers.

1.7.2 Keyword Index

As described above with reference to the flow charts in <figref idref="DRAWINGS">FIGS. 2 and 3</figref>, event-processing system <b>100</b> can construct and maintain one or more keyword indices to facilitate rapidly identifying events containing specific keywords. This can greatly speed up the processing of queries involving specific keywords. As mentioned above, to build a keyword index, an indexer first identifies a set of keywords. Then, the indexer includes the identified keywords in an index, which associates each stored keyword with references to events containing that keyword, or to locations within events where that keyword is located. When an indexer subsequently receives a keyword-based query, the indexer can access the keyword index to quickly identify events containing the keyword.

1.7.3 High Performance Analytics Store

To speed up certain types of queries, some embodiments of system <b>100</b> make use of a high performance analytics store, which is referred to as a “summarization table,” that contains entries for specific field-value pairs. Each of these entries keeps track of instances of a specific value in a specific field in the event data and includes references to events containing the specific value in the specific field. For example, an exemplary entry in a summarization table can keep track of occurrences of the value “94107” in a “ZIP code” field of a set of events, wherein the entry includes references to all of the events that contain the value “94107” in the ZIP code field. This enables the system to quickly process queries that seek to determine how many events have a particular value for a particular field, because the system can examine the entry in the summarization table to count instances of the specific value in the field without having to go through the individual events or do extractions at search time. Also, if the system needs to process all events that have a specific field-value combination, the system can use the references in the summarization table entry to directly access the events to extract further information without having to search all of the events to find the specific field-value combination at search time.

In some embodiments, the system maintains a separate summarization table for each of the above-described time-specific buckets that stores events for a specific time range, wherein a bucket-specific summarization table includes entries for specific field-value combinations that occur in events in the specific bucket. Alternatively, the system can maintain a separate summarization table for each indexer, wherein the indexer-specific summarization table only includes entries for the events in a data store that is managed by the specific indexer.

The summarization table can be populated by running a “collection query” that scans a set of events to find instances of a specific field-value combination, or alternatively instances of all field-value combinations for a specific field. A collection query can be initiated by a user, or can be scheduled to occur automatically at specific time intervals. A collection query can also be automatically launched in response to a query that asks for a specific field-value combination.

In some cases, the summarization tables may not cover all of the events that are relevant to a query. In this case, the system can use the summarization tables to obtain partial results for the events that are covered by summarization tables, but may also have to search other events that are not covered by the summarization tables to produce additional results. These additional results can then be combined with the partial results to produce a final set of results for the query. This summarization table and associated techniques are described in more detail in U.S. Pat. No. 8,682,925, issued on Mar. 25, 2014.

1.7.4 Accelerating Report Generation

In some embodiments, a data server system such as the SPLUNK® ENTERPRISE system can accelerate the process of periodically generating updated reports based on query results. To accelerate this process, a summarization engine automatically examines the query to determine whether generation of updated reports can be accelerated by creating intermediate summaries. (This is possible if results from preceding time periods can be computed separately and combined to generate an updated report. In some cases, it is not possible to combine such incremental results, for example where a value in the report depends on relationships between events from different time periods.) If reports can be accelerated, the summarization engine periodically generates a summary covering data obtained during a latest non-overlapping time period. For example, where the query seeks events meeting a specified criteria, a summary for the time period includes only events within the time period that meet the specified criteria. Similarly, if the query seeks statistics calculated from the events, such as the number of events that match the specified criteria, then the summary for the time period includes the number of events in the period that match the specified criteria.

In parallel with the creation of the summaries, the summarization engine schedules the periodic updating of the report associated with the query. During each scheduled report update, the query engine determines whether intermediate summaries have been generated covering portions of the time period covered by the report update. If so, then the report is generated based on the information contained in the summaries. Also, if additional event data has been received and has not yet been summarized, and is required to generate the complete report, the query can be run on this additional event data. Then, the results returned by this query on the additional event data, along with the partial results obtained from the intermediate summaries, can be combined to generate the updated report. This process is repeated each time the report is updated. Alternatively, if the system stores events in buckets covering specific time ranges, then the summaries can be generated on a bucket-by-bucket basis. Note that producing intermediate summaries can save the work involved in re-running the query for previous time periods, so only the newer event data needs to be processed while generating an updated report. These report acceleration techniques are described in more detail in U.S. Pat. No. 8,589,403, issued on Nov. 19, 2013, and U.S. Pat. No. 8,412,696, issued on Apr. 2, 2011.

1.8 Security Features

The SPLUNK® ENTERPRISE platform provides various schemas, dashboards and visualizations that make it easy for developers to create applications to provide additional capabilities. One such application is the SPLUNK® APP FOR ENTERPRISE SECURITY, which performs monitoring and alerting operations and includes analytics to facilitate identifying both known and unknown security threats based on large volumes of data stored by the SPLUNK® ENTERPRISE system. This differs significantly from conventional Security Information and Event Management (SIEM) systems that lack the infrastructure to effectively store and analyze large volumes of security-related event data. Traditional SIEM systems typically use fixed schemas to extract data from pre-defined security-related fields at data ingestion time, wherein the extracted data is typically stored in a relational database. This data extraction process (and associated reduction in data size) that occurs at data ingestion time inevitably hampers future incident investigations, when all of the original data may be needed to determine the root cause of a security issue, or to detect the tiny fingerprints of an impending security threat.

In contrast, the SPLUNK® APP FOR ENTERPRISE SECURITY system stores large volumes of minimally processed security-related data at ingestion time for later retrieval and analysis at search time when a live security threat is being investigated. To facilitate this data retrieval process, the SPLUNK® APP FOR ENTERPRISE SECURITY provides pre-specified schemas for extracting relevant values from the different types of security-related event data, and also enables a user to define such schemas.

The SPLUNK® APP FOR ENTERPRISE SECURITY can process many types of security-related information. In general, this security-related information can include any information that can be used to identify security threats. For example, the security-related information can include network-related information, such as IP addresses, domain names, asset identifiers, network traffic volume, uniform resource locator strings, and source addresses. (The process of detecting security threats for network-related information is further described in U.S. patent application Ser. Nos. 13/956,252, and 13/956,262.) Security-related information can also include endpoint information, such as malware infection data and system configuration information, as well as access control information, such as login/logout information and access failure notifications. The security-related information can originate from various sources within a data center, such as hosts, virtual machines, storage devices and sensors. The security-related information can also originate from various sources in a network, such as routers, switches, email servers, proxy servers, gateways, firewalls and intrusion-detection systems.

During operation, the SPLUNK® APP FOR ENTERPRISE SECURITY facilitates detecting so-called “notable events” that are likely to indicate a security threat. These notable events can be detected in a number of ways: (1) an analyst can notice a correlation in the data and can manually identify a corresponding group of one or more events as “notable;” or (2) an analyst can define a “correlation search” specifying criteria for a notable event, and every time one or more events satisfy the criteria, the application can indicate that the one or more events are notable. An analyst can alternatively select a pre-defined correlation search provided by the application. Note that correlation searches can be run continuously or at regular intervals (e.g., every hour) to search for notable events. Upon detection, notable events can be stored in a dedicated “notable events index,” which can be subsequently accessed to generate various visualizations containing security-related information. Also, alerts can be generated to notify system operators when important notable events are discovered.

The SPLUNK® APP FOR ENTERPRISE SECURITY provides various visualizations to aid in discovering security threats, such as a “key indicators view” that enables a user to view security metrics of interest, such as counts of different types of notable events. For example, <figref idref="DRAWINGS">FIG. 7A</figref> illustrates an exemplary key indicators view <b>700</b> that comprises a dashboard, which can display a value <b>701</b>, for various security-related metrics, such as malware infections <b>702</b>. It can also display a change in a metric value <b>703</b>, which indicates that the number of malware infections increased by 63 during the preceding interval. Key indicators view <b>700</b> additionally displays a histogram panel <b>704</b> that displays a histogram of notable events organized by urgency values, and a histogram of notable events organized by time intervals. This key indicators view is described in further detail in pending U.S. patent application Ser. No. 13/956,338 filed Jul. 31, 2013.

These visualizations can also include an “incident review dashboard” that enables a user to view and act on “notable events.” These notable events can include: (1) a single event of high importance, such as any activity from a known web attacker; or (2) multiple events that collectively warrant review, such as a large number of authentication failures on a host followed by a successful authentication. For example, <figref idref="DRAWINGS">FIG. 7B</figref> illustrates an exemplary incident review dashboard <b>710</b> that includes a set of incident attribute fields <b>711</b> that, for example, enables a user to specify a time range field <b>712</b> for the displayed events. It also includes a timeline <b>713</b> that graphically illustrates the number of incidents that occurred in one-hour time intervals over the selected time range. It additionally displays an events list <b>714</b> that enables a user to view a list of all of the notable events that match the criteria in the incident attributes fields <b>711</b>. To facilitate identifying patterns among the notable events, each notable event can be associated with an urgency value (e.g., low, medium, high, critical), which is indicated in the incident review dashboard. The urgency value for a detected event can be determined based on the severity of the event and the priority of the system component associated with the event. The incident review dashboard is described further in “http://docs.splunk.com/Documentation/PCI/2.1.1/User/IncidentReviewdashboard.”

1.9 Data Center Monitoring

As mentioned above, the SPLUNK® ENTERPRISE platform provides various features that make it easy for developers to create various applications. One such application is the SPLUNK® APP FOR VMWARE®, which performs monitoring operations and includes analytics to facilitate diagnosing the root cause of performance problems in a data center based on large volumes of data stored by the SPLUNK® ENTERPRISE system.

This differs from conventional data-center-monitoring systems that lack the infrastructure to effectively store and analyze large volumes of performance information and log data obtained from the data center. In conventional data-center-monitoring systems, this performance data is typically pre-processed prior to being stored, for example by extracting pre-specified data items from the performance data and storing them in a database to facilitate subsequent retrieval and analysis at search time. However, the rest of the performance data is not saved and is essentially discarded during pre-processing. In contrast, the SPLUNK® APP FOR VMWARE® stores large volumes of minimally processed performance information and log data at ingestion time for later retrieval and analysis at search time when a live performance issue is being investigated.

The SPLUNK® APP FOR VMWARE® can process many types of performance-related information. In general, this performance-related information can include any type of performance-related data and log data produced by virtual machines and host computer systems in a data center. In addition to data obtained from various log files, this performance-related information can include values for performance metrics obtained through an application programming interface (API) provided as part of the vSphere Hypervisor™ system distributed by VMware, Inc., of Palo Alto, Calif. For example, these performance metrics can include: (1) CPU-related performance metrics; (2) disk-related performance metrics; (3) memory-related performance metrics; (4) network-related performance metrics; (5) energy-usage statistics; (6) data-traffic-related performance metrics; (7) overall system availability performance metrics; (8) cluster-related performance metrics; and (9) virtual machine performance statistics. For more details about such performance metrics, please see U.S. patent Ser. No. 14/167,316 filed 29 Jan. 2014, which is hereby incorporated herein by reference. Also, see “vSphere Monitoring and Performance,” Update 1, vSphere 5.5, EN-001357-00, http://pubs.vmware.com/vsphere-55/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-551-monitoring-performance-guide.pdf.

To facilitate retrieving information of interest from performance data and log files, the SPLUNK® APP FOR VMWARE® provides pre-specified schemas for extracting relevant values from different types of performance-related event data, and also enables a user to define such schemas.

The SPLUNK® APP FOR VMWARE® additionally provides various visualizations to facilitate detecting and diagnosing the root cause of performance problems. For example, one such visualization is a “proactive monitoring tree” that enables a user to easily view and understand relationships among various factors that affect the performance of a hierarchically structured computing system. This proactive monitoring tree enables a user to easily navigate the hierarchy by selectively expanding nodes representing various entities (e.g., virtual centers or computing clusters) to view performance information for lower-level nodes associated with lower-level entities (e.g., virtual machines or host systems). Exemplary node-expansion operations are illustrated in <figref idref="DRAWINGS">FIG. 7C</figref>, wherein nodes <b>733</b> and <b>734</b> are selectively expanded. Note that nodes <b>731</b>-<b>739</b> can be displayed using different patterns or colors to represent different performance states, such as a critical state, a warning state, a normal state or an unknown/offline state. The ease of navigation provided by selective expansion in combination with the associated performance-state information enables a user to quickly diagnose the root cause of a performance problem. The proactive monitoring tree is described in further detail in U.S. patent application Ser. No. 14/235,490 filed on 15 Apr. 2014, which is hereby incorporated herein by reference for all possible purposes.

The SPLUNK® APP FOR VMWARE® also provides a user interface that enables a user to select a specific time range and then view heterogeneous data, comprising events, log data and associated performance metrics, for the selected time range. For example, the screen illustrated in <figref idref="DRAWINGS">FIG. 7D</figref> displays a listing of recent “tasks and events” and a listing of recent “log entries” for a selected time range above a performance-metric graph for “average CPU core utilization” for the selected time range. Note that a user is able to operate pull-down menus <b>742</b> to selectively display different performance metric graphs for the selected time range. This enables the user to correlate trends in the performance-metric graph with corresponding event and log data to quickly determine the root cause of a performance problem. This user interface is described in more detail in U.S. patent application Ser. No. 14/167,316 filed on 29 Jan. 2014, which is hereby incorporated herein by reference for all possible purposes.

2.1 Managing Event Streams Generated from Captured Network Data

The disclosed embodiments provide a method and system for facilitating the processing of network data. As shown in <figref idref="DRAWINGS">FIG. 8</figref>, the network data may be captured using a data-processing system <b>800</b> in a distributed network environment. In the illustrated embodiment, system <b>800</b> includes a set of configuration servers <b>820</b> in communication with a set of remote capture agents <b>851</b>-<b>853</b> over one or more networks <b>890</b>.

Although system <b>800</b> only depicts three configuration servers <b>820</b> and three remote capture agents <b>851</b>-<b>853</b>, any number of configuration servers <b>820</b> and/or remote capture agents <b>851</b>-<b>853</b> may be configured to operate and/or communicate with one another within the data-processing system. For example, a single physical and/or virtual server may perform the functions of configuration servers <b>820</b>. Alternatively, multiple physical and/or virtual servers or network elements may be logically connected to provide the functionality of configuration servers <b>820</b>. The configuration server(s) may direct the activity of multiple distributed remote capture agents <b>851</b>-<b>853</b> installed on various client computing devices across one or more networks. In turn, remote capture agents <b>851</b>-<b>853</b> may be used to capture network data from multiple remote network data sources.

Further, embodiments described herein can be configured to capture network data in a cloud-based environment, such as cloud <b>840</b> depicted in the illustrated embodiment, and to generate events such as timestamped records of network activity from the network data. Remote capture agents <b>851</b>-<b>853</b> may capture network data originating from numerous distributed network servers, whether they are physical hardware servers or virtual machines running in cloud <b>840</b>. In cloud-based implementations, remote capture agents <b>851</b>-<b>853</b> will generally only have access to information that is communicated to and received from machines running in the cloud-based environment. This is because, in a cloud environment, there is generally no access to any of the physical network infrastructure, as cloud computing may utilize a “hosted services” delivery model where the physical network infrastructure is typically managed by a third party.

Embodiments further include the capability to separate the data capture technology into a standalone component that can be installed directly on client servers, which may be physical servers or virtual machines residing on a cloud-based network (e.g., cloud <b>840</b>), and used to capture and generate events for all network traffic that is transmitted in and out of the client servers. This eliminates the need to deploy and connect physical hardware to network TAPS or SPAN ports, thus allowing users to configure and change their data capture configuration on-the-fly rather than in fixed formats.

In the illustrated embodiment, remote capture agents <b>852</b>-<b>853</b> are in communication with network servers <b>830</b> residing in cloud <b>840</b>, and remote capture agent <b>851</b> is located in cloud <b>840</b>. Cloud <b>840</b> may represent any number of public and private clouds, and is not limited to any particular cloud configuration. Network servers <b>830</b> residing in cloud <b>840</b> may be physical servers and/or virtual machines in cloud <b>840</b>, and network traffic to and from network servers <b>830</b> may be monitored by remote capture agent <b>851</b> and/or other remote capture agents connected to network servers <b>830</b>. Further, remote capture agents <b>852</b>-<b>853</b> may also run in cloud <b>840</b> on physical servers and/or virtual machines. Those skilled in the art will appreciate that any number of remote capture agents may be included inside or outside of cloud <b>840</b>.

Remote capture agents <b>851</b>-<b>853</b> may analyze network packets received from the networks(s) to which remote capture agents <b>851</b>-<b>853</b> are connected to obtain network data from the network packets and generate a number of events from the network data. For example, each remote capture agent <b>851</b>-<b>853</b> may listen for network traffic on network interfaces available to the remote capture agent. Network packets transmitted to and/or from the network interfaces may be intercepted by the remote capture agent and analyzed, and relevant network data from the network packets may be used by the remote capture agent to create events related to the network data. Such events may be generated by aggregating network data from multiple network packets, or each event may be generated using the contents of only one network packet. A sequence of events from a remote capture agent may then be included in one or more event streams that are provided to other components of system <b>800</b>.

Configuration servers <b>820</b>, data storage servers <b>835</b>, and/or other network components may receive event data (e.g., event streams) from remote capture agents <b>851</b>-<b>853</b> and further process the event data before the event data is stored by data storage servers <b>835</b>. In the illustrated embodiment, configuration servers <b>820</b> may transmit event data to data storage servers <b>835</b> over a network <b>801</b> such as a local area network (LAN), wide area network (WAN), personal area network (PAN), virtual private network, intranet, mobile phone network (e.g., a cellular network), Wi-Fi network, Ethernet network, and/or other type of network that enables communication among computing devices. The event data may be received over a network (e.g., network <b>801</b>, network <b>890</b>) at one or more event indexers (see <figref idref="DRAWINGS">FIG. 1</figref>) associated with data storage servers <b>835</b>.

In addition, system <b>800</b> may include functionality to determine the types of network data collected and/or processed by each remote capture agent <b>851</b>-<b>853</b> to avoid data duplication at the indexers, data storage servers <b>835</b>, and/or other components of system <b>800</b>. For example, remote capture agents <b>852</b>-<b>853</b> may process network traffic from the same network. However, remote capture agent <b>852</b> may generate page view events from the network traffic, and remote capture agent <b>853</b> may generate request events (e.g., of HyperText Transfer Protocol (HTTP) requests and responses) from the network traffic.

In one or more embodiments, configuration servers <b>820</b> include configuration information that is used to configure the creation of events from network data on remote capture agents <b>851</b>-<b>853</b>. In addition, such configuration may occur dynamically during event processing (e.g., at runtime). Conversely, because most conventional network capture technologies target specific end uses, they have been designed to operate in a fixed way and generally cannot be modified dynamically or easily to address different and changing business needs.

At least certain embodiments are adapted to provide a distributed remote capture platform in which the times at which events are communicated to the configuration servers <b>820</b> and the fields to be included in the events are controlled by way of user-modifiable configuration rather than by “hard coding” fixed events with pre-determined fields for a given network capture mechanism. The remote configuration capability also enables additional in-memory processing (e.g., filtering, transformation, normalization, aggregation, etc.) on events at the point of capture (e.g., remote capture agents <b>851</b>-<b>853</b>) before the events are transmitted to other components of system <b>800</b>.

Configuration information stored at each configuration server <b>820</b> may be created and/or updated manually at the configuration server and/or at a network element in communication with the configuration server. For example, a user may upload a configuration file containing configuration information for a remote capture agent to one or more configuration servers <b>820</b> for subsequent propagation to the remote capture agent. Alternatively, the user may use a GUI to provide the configuration information, as described in further detail below with respect to <figref idref="DRAWINGS">FIGS. 11A-11D</figref>. The configuration information may further be provided by one or more applications running on a separate server or network element, such as data storage servers <b>835</b>.

Remote capture agents <b>851</b>-<b>853</b> may then use the configuration information to generate events from captured network packets. When changes in the configuration information at the configuration server are detected at the remote capture agents, logic in the remote capture agents may be automatically reconfigured in response. This means the remote capture agents may be dynamically configured to produce different events, transform the events, and/or communicate event streams to different components of system <b>800</b>. Dynamic configuration of the generation of events from captured network packets may also be performed by other components (e.g., configuration servers <b>820</b>, data storage servers <b>835</b>, etc.), in lieu of or in addition to the remote capture agents.

To detect changes in configuration information at configuration servers <b>820</b>, remote capture agents <b>851</b>-<b>853</b> may poll configuration servers <b>820</b> at periodic intervals for updates to the configuration information. The updates may then be pulled from configuration servers <b>820</b> by remote capture agents <b>851</b>-<b>853</b>. Conversely, updates to the configuration information may be pushed from configuration servers <b>820</b> to remote capture agents <b>851</b>-<b>853</b> at periodic intervals and/or when changes to the configuration information have been made.

In one embodiment, configuration servers <b>820</b> include a list of event streams generated by remote capture agents <b>851</b>-<b>853</b>, as well as the configuration information used to generate the event streams at remote capture agents <b>851</b>-<b>853</b>. The configuration information may include a unique identifier for each event stream, the types of events to be included in the event stream, one or more fields to be included in each event, and/or one or more filtering rules for filtering events to be included in the event stream. Using configuration information to dynamically modify network data capture by remote capture agents (e.g., remote capture agents <b>851</b>-<b>853</b>) is described in a co-pending non-provisional application by inventor Michael Dickey, entitled “Distributed Processing of Network Data Using Remote Capture Agents,” having Ser. No. 14/253,713, and filing date 15 Apr. 2014, issued as U.S. Pat. No. 10,127,273, which is incorporated herein by reference.

In one or more embodiments, system <b>800</b> includes functionality to perform protocol-based capture and analysis of network data using remote capture agents <b>851</b>-<b>853</b>. First, remote capture agents <b>851</b>-<b>853</b> may be configured to generate event streams from packet flows captured at remote capture agents <b>851</b>-<b>853</b> based on protocol classifications for the packet flows. Second, configuration servers <b>820</b> may include functionality to streamline the configuration of remote capture agents <b>851</b>-<b>853</b> in generating protocol-specific event streams. Third, configuration servers <b>820</b> and/or remote capture agents <b>851</b>-<b>853</b> may enable the use of capture triggers to capture additional network data based on the identification of potential security risks from previously generated event streams. Protocol-based capture and analysis of network data using remote capture agents is described in a co-pending non-provisional application by inventors Vladimir Shcherbakov and Michael Dickey and filed on the same day as the instant application, entitled “Protocol-Based Capture of Network Data Using Remote Capture Agents,” having Ser. No. 14/528,898, and filing date 30 October, issued as U.S. Pat. No. 9,838,512, which is incorporated herein by reference.

<figref idref="DRAWINGS">FIG. 9</figref> shows a remote capture agent <b>950</b> in accordance with the disclosed embodiments. In the illustrated embodiment, remote capture agent <b>950</b> is adapted to receive configuration information from one or more configuration servers <b>820</b> over network <b>801</b>. Remote capture agent <b>950</b> may be installed at a customer's premises on one or more of the customer's computing resources. Remote capture agent <b>950</b> may also be installed in a remote computing environment such as a cloud computing system. For example, remote capture agent <b>950</b> may be installed on a physical server and/or in a virtual computing environment (e.g., virtual machine) that is distributed across one or more physical machines.

Remote capture agent <b>950</b> includes a communications component <b>903</b> configured to communicate with network elements on one or more networks (e.g., network <b>801</b>) and send and receive network data (e.g., network packets) over the network(s). As depicted, communications component <b>903</b> may communicate with configuration servers <b>820</b> over network <b>801</b>. Communications component <b>903</b> may also communicate with one or more sources of network data, such as network servers <b>830</b> of <figref idref="DRAWINGS">FIG. 8</figref>.

Network data received at communications component <b>903</b> may be captured by a capture component <b>905</b> coupled with communications component <b>903</b>. Capture component <b>905</b> may capture some or all network data from communications component <b>903</b>. For example, capture component <b>905</b> may capture network data based on the sources and/or destinations of the network data, the types of the network data, the protocol associated with the network data, and/or other characteristics of the network data.

In addition, the network data may be captured based on configuration information stored in a configuration component <b>904</b> of remote capture agent <b>950</b>. As mentioned above, the configuration information may be received from configuration servers <b>820</b> over network <b>801</b>. The configuration information may then be used to dynamically configure or reconfigure remote capture agent <b>950</b> in real-time. For example, newly received configuration information in configuration component <b>904</b> may be used to configure the operation of remote capture agent <b>950</b> during processing of events from network data by remote capture agent <b>950</b>.

To dynamically configure remote capture agent <b>950</b>, configuration information received by configuration component <b>904</b> from configuration servers <b>820</b> may be provided to other components of remote capture agent <b>950</b>. More specifically, remote capture agent <b>950</b> includes an events generator <b>907</b> that receives network data from network data capture component <b>905</b> and generates events from the network data based on configuration information from configuration component <b>904</b>.

Using configuration information provided by configuration servers <b>820</b>, remote capture agent <b>950</b> can be instructed to perform any number of event-based processing operations. For example, the configuration information may specify the generation of event streams associated with network (e.g., HTTP, Simple Mail Transfer Protocol (SMTP), Domain Name System (DNS)) transactions, business transactions, errors, alerts, clickstream events, and/or other types of events. The configuration information may also describe custom fields to be included in the events, such as values associated with specific clickstream terms. The configuration information may include additional parameters related to the generation of event data, such as an interval between consecutive events and/or the inclusion of transactions and/or errors matching a given event in event data for the event. Configuration information for configuring the generation of event streams from network data captured by remote capture agents is further described in the above-referenced applications.

An events transformer <b>909</b> may further use the configuration information to transform some or all of the network data from capture component <b>905</b> and/or events from events generator <b>907</b> into one or more sets of transformed events. In one or more embodiments, transformations performed by events transformer <b>909</b> include aggregating, filtering, cleaning, and/or otherwise processing events from events generator <b>907</b>. Configuration information for the transformations may thus include a number of parameters that specify the types of transformations to be performed, the types of data on which the transformations are to be performed, and/or the formatting of the transformed data.

A rules comparison engine <b>908</b> in remote capture agent <b>950</b> may receive events from events generator <b>907</b> and compare one or more fields from the events to a set of filtering rules in the configuration information to determine whether to include the events in an event stream. For example, the configuration information may specify packet-level, protocol-level, and/or application-level filtering of event data from event streams generated by remote capture agent <b>950</b>.

Finally, a data enrichment component <b>911</b> may further transform event data into a different form or format based on the configuration information from configuration component <b>904</b>. For example, data enrichment component <b>911</b> may use the configuration information to normalize the data so that multiple representations of the same value (e.g., timestamps, measurements, etc.) are converted into the same value in transformed event data.

Data can be transformed by data enrichment component <b>911</b> in any number of ways. For example, remote capture agent <b>950</b> may reside on a client server in Cupertino, Calif., where all the laptops associated with the client server have been registered with the hostname of the client server. Remote capture agent <b>950</b> may use the registration data to look up an Internet Protocol (IP) address in a look-up table (LUT) that is associated with one or more network elements of the client server's local network. Remote capture agent <b>950</b> may then resolve a user's IP address into the name of the user's laptop, thereby enabling inclusion of the user's laptop name in transformed event data associated with the IP address. The transformed event data may then be communicated to configuration servers <b>820</b> and/or a central transformation server residing in San Francisco for further processing, indexing, and/or storage.

As mentioned above, remote capture agent <b>950</b> may perform protocol-based generation of event streams from network data. As shown in <figref idref="DRAWINGS">FIG. 9B</figref>, configuration component <b>904</b> may obtain protocol-specific configuration information (e.g., protocol-specific configuration information A <b>912</b>, protocol-specific configuration information B <b>914</b>) from one or more configuration servers (e.g., configuration servers <b>820</b>). For example, configuration information from the configuration server(s) may be transmitted over network <b>801</b> to communications component <b>903</b>, which provides the configuration information to configuration component <b>904</b> for storage and/or further processing.

Protocol-specific configuration information from configuration component <b>904</b> may be used to configure the generation of event streams (e.g., event stream C <b>932</b>, event stream D <b>934</b>, event stream E <b>940</b>, event stream F <b>942</b>) based on protocol classifications of network packets (e.g., network packets C <b>916</b>, network packets D <b>918</b>) captured by capture component <b>905</b>. For example, protocol-specific configuration information from configuration component <b>904</b> may specify the creation of event streams from the network packets based on the protocols used in the network packets, such as HTTP, DNS, SMTP, File Transfer Protocol (FTP), Server Message Block (SMB), Network File System (NFS), Internet Control Message Protocol (ICMP), email protocols, database protocols, and/or security protocols. Such event streams may include event attributes that are of interest to the respective protocols.

Before the event streams are generated from the network packets, capture component <b>905</b> may assemble the network packets into one or more packets flows (e.g., packet flow C <b>920</b>, packet flow D <b>922</b>). First, capture component <b>905</b> may identify the network packets in a given packet flow based on control information in the network packets. The packet flow may represent a communication path between a source and a destination (e.g., host, multicast group, broadcast domain, etc.) on the network. As a result, capture component <b>905</b> may identify network packets in the packet flow by examining network (e.g., IP) addresses, ports, sources, destinations, and/or transport protocols (e.g., Transmission Control Protocol (TCP), User Datagram Protocol (UDP), etc.) from the headers of the network packets.

Next, capture component <b>905</b> may assemble the packet flow from the network packets. For example, capture component <b>905</b> may assemble a TCP packet flow by rearranging out-of-order TCP packets. Conversely, capture component <b>905</b> may omit reordering of the network packets in the packet flow if the network packets use UDP and/or another protocol that does not provide for ordered packet transmission.

After the packet flow is assembled, capture component <b>905</b> and/or another component of remote capture agent <b>950</b> may detect encryption of the network packets in the packet flow by analyzing the byte signatures of the network packets' payloads. For example, the component may analyze the network packets' payloads for byte signatures that are indicative of Secure Sockets Layer (SSL) and/or Transport Layer Security (TLS) encryption. If the network packets are detected as encrypted, the component may decrypt the network packets. For example, the component may have access to private keys from an SSL server used by the network flow and perform decryption of the network packets to obtain plaintext payload data in the order in which the data was sent. Such access to private keys may be given to remote capture agent <b>950</b> by an administrator associated with the network flow, such as an administrator of the host from which the network packets are transmitted.

Events generator <b>907</b> may then obtain a protocol classification (e.g., protocol classification C <b>924</b>, protocol classification D <b>926</b>) for each packet flow identified, assembled, and/or decrypted by capture component <b>905</b>. For example, events generator <b>907</b> may use a protocol-decoding mechanism to analyze the headers and/or payloads of the network packets in the packet flow and return protocol identifiers of one or more protocols used in the network packets. The protocol-decoding mechanism may additionally provide metadata related to the protocols, such as metadata related to traffic volume, application usage, application performance, user and/or host identifiers, content (e.g., media, files, etc.), and/or file metadata (e.g., video codecs and bit rates).

Once the protocol classification is obtained for a packet flow, events generator <b>907</b> may use protocol-specific configuration information associated with the protocol classification from configuration component <b>904</b> to build an event stream (e.g., event stream C <b>932</b>, event stream D <b>934</b>) from the packet flow. As mentioned above and in the above-referenced application, the event stream may include time-series event data generated from network packets in the packet flow. To create the event stream, events generator <b>907</b> may obtain one or more event attributes associated with the protocol classification from the configuration information. Next, event generator <b>907</b> may extract the event attribute(s) from the network packets in the first packet flow. Events generator <b>907</b> may then include the extracted event attribute(s) in the event stream.

For example, events generator <b>907</b> may obtain a protocol classification of DNS for a packet flow from capture component <b>905</b> and protocol-specific configuration information for generating event streams from DNS traffic from configuration component <b>904</b>. The protocol-specific configuration information may specify the collection of event attributes such as the number of bytes transferred between the source and destination, network addresses and/or identifiers for the source and destination, DNS message type, DNS query type, return message, response time to a DNS request, DNS transaction identifier, and/or a transport layer protocol. In turn, events generator <b>907</b> may parse the protocol-specific configuration to identify the event attributes to be captured from the packet flow. Next, events generator <b>907</b> may extract the specified event attributes from the network packets in the packet flow and/or metadata received with the protocol classification of the packet flow and generate time-stamped event data from the extracted event attributes. Events generator <b>907</b> may then provide the time-stamped event data in an event stream to communications component <b>903</b> for transmission of the event stream over a network to one or more configuration servers, data storage servers, indexers, and/or other components for subsequent storage and processing of the event stream by the component(s).

As described above and in the above-referenced application, network data from capture component <b>905</b> and/or event data from events generator <b>907</b> may be transformed by events transformer <b>909</b> into transformed event data that is provided in lieu of or in addition to event data generated by events generator <b>907</b>. For example, events transformer <b>909</b> may aggregate, filter, clean, and/or otherwise process event attributes from events generator <b>907</b> to produce one or more sets of transformed event attributes (e.g., transformed event attributes 1 <b>936</b>, transformed event attributes z <b>938</b>). Events transformer <b>909</b> may then include the transformed event attributes into one or more additional event streams (e.g., event stream 1 <b>940</b>, event stream z <b>942</b>) that may be transmitted over the network for subsequent storage and processing of the event stream(s) by other components on the network. Such transformation of event data at remote capture agent <b>950</b> may offload subsequent processing of the event data at configuration servers and/or other components on the network. Moreover, if the transformation reduces the size of the event data (e.g., by aggregating the event data), network traffic between remote capture agent <b>950</b> and the other components may be reduced, along with the storage requirements associated with storing the event data at the other components.

As with protocol-based generation of event data by events generator <b>907</b>, events transformer <b>909</b> may use protocol-specific configuration information from configuration component <b>904</b> to transform network and/or event data from a given packet flow and/or event stream. For example, events transformer <b>909</b> may obtain protocol-specific configuration information for aggregating HTTP events and use the configuration information to generate aggregated HTTP events from HTTP events produced by events generator <b>907</b>. The configuration information may include one or more key attributes used to generate a unique key representing an aggregated event from the configuration information. For example, key attributes for generating an aggregated HTTP event may include the source and destination IP addresses and ports in a set of HTTP events. A different unique key and aggregated HTTP event may thus be generated for each unique combination of source and destination IP addresses and ports in the HTTP events.

The configuration information may also specify one or more aggregation attributes to be aggregated prior to inclusion in the aggregated event. For example, aggregation attributes for generating an aggregated HTTP event from HTTP event data may include the number of bytes and packets sent in each direction between the source and destination. Data represented by the aggregation attributes may be included in the aggregated HTTP event by summing, averaging, and/or calculating a summary statistic from the number of bytes and packets sent in each direction between the source and destination. Aggregation of event data is described in further detail below with respect to <figref idref="DRAWINGS">FIG. 11C</figref>.

<figref idref="DRAWINGS">FIG. 10</figref> shows a configuration server <b>1020</b> in accordance with the disclosed embodiments. As shown in the illustrated embodiment, configuration server <b>1020</b> is in communication with multiple remote capture agents <b>1050</b> over network <b>890</b>, and remote capture agents <b>1050</b> are distributed throughout network <b>890</b> and cloud <b>840</b>. Configuration server <b>1020</b> includes a communications component <b>1010</b> that receives events from remote capture agents <b>1050</b> over network <b>890</b> and/or from cloud <b>840</b>. Communications component <b>1010</b> may also communicate with one or more data storage servers, such as data storage servers <b>835</b> of <figref idref="DRAWINGS">FIG. 8</figref>.

Configuration server <b>1020</b> also includes a configuration component <b>1004</b> that stores configuration information for remote capture agents <b>1050</b>. As described above, the configuration information may specify the types of events to produce, data to be included in the events, and/or transformations to be applied to the data and/or events to produce transformed events. Some or all of the transformations may be specified in a set of filtering rules <b>1021</b> that may be applied to event data at remote capture agents <b>1050</b> to determine a subset of the event data to be included in one or more event streams that are sent to configuration server <b>1020</b> and/or other components.

Configuration information from configuration component <b>1004</b> may also be used to manage an event stream lifecycle of the event streams. The event stream lifecycle may be a permanent event stream lifecycle, in which generation of events in an event stream continues after the event stream's creation until the event stream is manually disabled, deleted, or otherwise inactivated. Conversely, the event stream lifecycle may be an ephemeral event stream lifecycle, in which events in an event stream are generated on a temporary basis, and the event stream has an end time at which the event stream is terminated. For example, an ephemeral event stream may be created by a capture trigger for generating additional time-series event data from the network packets on remote capture agents <b>1050</b> based on a security risk, as described above and in the above-referenced applications.

To distinguish between permanent and ephemeral event streams, the configuration information may include a parameter that identifies each event stream as “permanent” or “ephemeral.” The configuration information may also include attributes such as a start time and end time for each ephemeral event stream. Remote capture agents <b>1050</b> may begin generating time-series event data for the ephemeral event stream at the start time and terminate the ephemeral event stream at the end time.

Alternatively, the creation and termination of an ephemeral event stream may be managed by configuration component <b>1004</b> instead of remote capture agents <b>1050</b>. For example, configuration component <b>1004</b> may track the start and end times of an ephemeral event stream. At the start time of the ephemeral event stream, configuration component <b>1004</b> may provide remote capture agents <b>1050</b> with configuration information for the ephemeral event stream. At the end time of the ephemeral end stream, configuration component <b>1004</b> may remove all references to the ephemeral event stream from the configuration information and transmit the configuration information to remote capture agents <b>1050</b>. Because configuration component <b>1004</b> uses updates to the configuration information to create and terminate an ephemeral event stream, remote capture agents <b>1050</b> may not be required to distinguish between ephemeral event streams and permanent event streams.

Configuration information from configuration component <b>1004</b> may additionally be used to enable the generation of a set of statistics from an event stream without subsequently storing and processing the event stream by one or more components on a network. For example, the configuration information may allow an event stream to be captured in a “stats only” mode, in which an index volume (e.g., size of indexed event data), incoming traffic volume, outgoing traffic volume, and/or other statistics are generated from the event stream without subsequently indexing and/or storing the event stream. A user may use the “stats only” mode to understand and/or anticipate the volume of data that would be sent for indexing from the event stream without incurring additional costs and/or overhead associated with indexing the event stream. In turn, the “stats only” mode may allow the user to prioritize indexing of various event streams based on the index volumes of the event streams and/or a daily index volume limit associated with indexing the event streams.

The configuration information may also be used to selectively index some or all of an event stream based on a storage limit (e.g., index volume limit) associated with the capture of time-series event data by remote capture agents <b>1050</b> and/or the statistics generated from an enabled event stream or a “stats only” event stream. For example, the configuration information may specify the indexing of a portion of the event stream, up to the daily index volume limit associated with a license for indexing event streams captured by remote capture agents <b>1050</b>. In another example, the configuration information may specify indexing of the event stream based on a historical trend associated with the set of statistics generated from the event stream. In a third example, the configuration information may specify indexing of a sample and/or the entirety of the event stream during high traffic volume, low traffic volume, a notable event, and/or a capture trigger. In a fourth example, the configuration information may include one or more user settings for indexing some or all of the event stream, as described in further detail below.

Configuration server <b>1020</b> may also include a data processing component <b>1011</b> that performs additional processing of the event streams based on configuration information from configuration component <b>1004</b>. As discussed in the above example with respect to <figref idref="DRAWINGS">FIGS. 9A-9B</figref>, event data may be transformed at a remote capture agent (e.g., remote capture agent <b>950</b>) during resolution of the user's IP address into the name of the user's laptop. The transformed event data may be sent to configuration server <b>1020</b> and/or a transformation server for additional processing and/or transformation, such as taking the host name from the transformed event data, using an additional LUT to obtain a user identifier (user ID) of the person to which the laptop is registered, and further transforming the event data by including the user ID in the event data before forwarding the event data to a third server (e.g., a transformation server) for another round of processing.

In one or more embodiments, configuration server <b>1020</b> and remote capture agents <b>1050</b> include functionality to improve the management of event streams generated from captured network data, including event streams associated with various protocols, applications, and event stream lifecycles. As shown in <figref idref="DRAWINGS">FIG. 10</figref>, configuration server <b>1020</b> may provide a GUI <b>1025</b> that can be used to configure or reconfigure the information contained in configuration component <b>1004</b>. The configuration information from configuration component <b>1004</b> may then be propagated to remote capture agents <b>1050</b> and used by remote capture agents <b>1050</b> to generate time-series event data from network packets captured by remote capture agents <b>1050</b>.

GUI <b>1025</b> may include a number of features and/or mechanisms for facilitating the management of multiple event streams. First, GUI <b>1025</b> may group the event streams by one or more event stream attributes associated with the event streams. For example, GUI <b>1025</b> may allow a user to specify an event stream attribute such as a category of an event stream, a protocol used by network packets from which the event stream is generated, an application used to create the event stream, and/or an event stream lifecycle of the event stream. GUI <b>1025</b> may display event stream information for one or more subsets of the event streams represented by the grouping of the event streams by the specified event stream attribute. GUI <b>1025</b> may also group the event stream information by multiple event stream attributes. For example, GUI <b>1025</b> may group the event stream information by a first event stream attribute such as application, category, or protocol. GUI <b>1025</b> may then apply a second grouping of the event stream information by permanent or ephemeral event stream lifecycles. As a result, a user may view permanent event streams associated with a given category, application, or protocol separately from ephemeral event streams associated with the category, application or protocol. Grouping and managing event streams is described in further detail below with respect to <figref idref="DRAWINGS">FIGS. 17A-17C</figref>.

Second, GUI <b>1025</b> may display, along with the grouped event stream information, graphs of metrics associated with time-series event data in the event streams. For example, GUI <b>1025</b> may include a sparkline of network traffic over time for each event stream under a given grouping of event streams. GUI <b>1025</b> may also show a sparkline of aggregate network traffic and/or another aggregated metric for all event streams listed under the grouping. In-line visualizations of metrics related to event streams and/or captured network data is described in further detail below with respect to <figref idref="DRAWINGS">FIGS. 17A-17B</figref>.

Third, GUI <b>1025</b> may enable the management of ephemeral event streams. For example, GUI <b>1025</b> may allow a user to create a new ephemeral event stream, disable an existing event stream, delete an existing event stream, and/or modify an end time for terminating an existing event stream. Managing ephemeral event streams generated from captured network data is described in further detail below with respect to <figref idref="DRAWINGS">FIG. 17C</figref>.

Fourth, GUI <b>1025</b> may provide bidirectional linking of ephemeral event streams to creators of the ephemeral event streams. For example, GUI <b>1025</b> may include a hyperlink from event stream information for an ephemeral event stream to creation information for a creator of the ephemeral event stream, such as an application and/or capture trigger used to create the ephemeral event stream. GUI <b>1025</b> may also include another hyperlink from the creation information to the event stream information to facilitate understanding and analysis related to the context under which the ephemeral event stream was generated. Bidirectional linking of ephemeral event streams to creators of the ephemeral event streams is described in further detail below with respect to <figref idref="DRAWINGS">FIGS. 17C-17D</figref>.

Fifth, configuration server <b>1020</b> may provide a risk-identification mechanism <b>1007</b> for identifying a security risk from time-series event data generated by remote capture agents <b>1050</b>, as well as a capture trigger <b>1009</b> for generating additional time-series event data based on the security risk. For example, risk-identification mechanism <b>1007</b> may allow a user to view and/or search for events that may represent security risks through GUI <b>1025</b>. Risk-identification mechanism <b>1007</b> and/or GUI <b>1025</b> may also allow the user to set and/or activate capture trigger <b>1009</b> based on the events shown and/or found through risk-identification mechanism <b>1007</b> and/or GUI <b>1025</b>.

In particular, risk-identification mechanism <b>1007</b> and/or GUI <b>1025</b> may allow the user to manually activate capture trigger <b>1009</b> after discovering a potential security risk. In turn, the activated capture trigger <b>1009</b> may modify configuration information in configuration component <b>1004</b> that is propagated to remote capture agents <b>1050</b> to trigger the capture of additional network data by remote capture agents <b>1050</b>.

Alternatively, risk-identification mechanism <b>1007</b> may allow the user to create a search and/or recurring search for time-series event data that may match a security risk. If the search and/or recurring search finds time-series event data that matches the security risk, capture trigger <b>1009</b> may automatically be activated to enable the generation of additional time-series event data, such as event data containing one or more attributes associated with one or more protocols that facilitate analysis of the security risk. Such automatic activation of capture trigger <b>1009</b> may allow the additional event data to be generated immediately after a notable event is detected, thus averting the loss of captured network data that results from enabling additional network data capture only after a potential security risk is manually identified (e.g., by an analyst). Triggering the generation of additional time-series event data from network packets on remote agents based on potential security risks is described in further detail below with respect to <figref idref="DRAWINGS">FIGS. 12A-12B</figref>.

Sixth, GUI <b>1025</b> may be used to enable the generation of a set of statistics from an event stream without subsequently storing and processing the event stream. For example, GUI <b>1025</b> provide an option for enabling the “stats only” mode described above, which allows statistics associated with traffic volume to be generated from a given event stream without subsequently indexing and/or storing the event stream. After the statistics are generated, GUI <b>1025</b> may display the statistics to allow a user to decide whether to index the event stream or not without exceeding a storage limit (e.g., daily index volume limit) associated with the customer's license, such as a SPLUNK® license.

GUI <b>1025</b> may also allow the user to adjust the amount of the event stream to index based on the statistics and/or the storage limit. For example, GUI <b>1025</b> may provide a slider and/or other user-interface element for adjusting the amount (e.g., percentage) of the event stream to index. GUI <b>1025</b> may also provide a suggestion for setting the amount of storage of the event stream based on the set of statistics and the storage limit, a price associated with storing the event stream above the storage limit, an unused remainder of the storage limit, and/or other information that may facilitate intelligent selective indexing of the event stream. GUI <b>1025</b> may further include options for indexing some or all of the event stream during low traffic volume, high traffic volume, and/or other types of traffic or exceptional events in the event stream. Managing the generation of statistics from event streams and/or the selective indexing of the event streams is described in further detail below with respect to <figref idref="DRAWINGS">FIGS. 23A-23C</figref>.

Seventh, GUI <b>1025</b> may display one or more graphs containing one or more values from the statistics generated above, as well as a value of a statistic from the set of statistics based on the position of a cursor over the graph(s) and/or a legend associated with the graph(s). For example, GUI <b>1025</b> may include a “stream stats” dashboard that contains a table of statistics for enabled, “stats only,” and/or selectively indexed event streams. Within the same screen, GUI <b>1025</b> may include a bar chart of an aggregate index volume of the event streams over time, with segments in the bar chart representing the individual index volumes of the event streams over a time interval. The bar chart may be updated with a value of an event stream's index volume based on the position of the cursor over the bar chart. A portion of the bar chart may also be highlighted based on the position of the cursor over the legend of the bar chart. GUI <b>1025</b> may also include a pie chart of the index volume across the event streams. The pie chart may be updated with one or more values of an index volume of an event stream based on the position of the cursor over the pie chart. Visualizations of statistics associated with captured network data is described in further detail below with respect to <figref idref="DRAWINGS">FIGS. 24A-24E</figref>.

<figref idref="DRAWINGS">FIG. 11A</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 11A</figref> shows a screenshot of a GUI, such as GUI <b>1025</b> of <figref idref="DRAWINGS">FIG. 10</figref>. As described above, the GUI may be used to obtain configuration information that is used to configure the generation of event streams containing time-series event data at one or more remote capture agents distributed across a network.

As shown in <figref idref="DRAWINGS">FIG. 11A</figref>, the GUI includes a table with a set of columns <b>1102</b>-<b>1108</b> containing high-level information related to event streams that may be created using the configuration information. Each row of the table may represent an event stream, and rows of the table may be sorted by column <b>1102</b>.

Column <b>1102</b> shows an alphabetized or otherwise ordered or unordered list of names of the event streams, and column <b>1104</b> provides descriptions of the event streams. For example, columns <b>1102</b>-<b>1104</b> may include names and descriptions of event streams generated from HTTP, Dynamic Host Configuration Protocol (DHCP), DNS, FTP, email protocols, database protocols, NFS, Secure Message Block (SMB), security protocols, Session Initiation Protocol (SIP), TCP, and/or UDP network traffic. Columns <b>1102</b>-<b>1104</b> may thus indicate that event streams may be generated based on transport layer protocols, session layer protocols, presentation layer protocols, and/or application layer protocols.

A user may select a name of an event stream under column <b>1102</b> to access and/or update configuration information for configuring the generation of the event stream. For example, the user may select “DemoHTTP” in column <b>1102</b> to navigate to a screen of the GUI that allows the user to specify event attributes, filters, and/or aggregation information related to creating the “DemoHTTP” event stream, as discussed in further detail below with respect to <figref idref="DRAWINGS">FIGS. 11B-11E</figref>.

Column <b>1106</b> specifies whether each event stream is enabled or disabled. For example, column <b>1106</b> may indicate that the “AggregateHTTP,” “DemoHTTP,” “dns,” “ftp,” “mysql-query,” “sip,” “tcp,” and “udp” event streams are enabled. If an event stream is enabled, time-series event data may be included in the event stream based on the configuration information for the event stream.

Column <b>1108</b> specifies whether each event stream is cloned from an existing event stream. For example, column <b>1108</b> may indicate that the “AggregateHTTP” and “DemoHTTP” event streams have been cloned (e.g., copied) from other event streams, while the remaining event streams may be predefined with default event attributes.

The GUI also includes a user-interface element <b>1110</b> (e.g., “Clone Stream”). A user may select user-interface element <b>1110</b> to create a new event stream as a copy of an event stream listed in the GUI. After user-interface element <b>1110</b> is selected, an overlay may be displayed that allows the user to specify a name for the new event stream, a description of the new event stream, and an existing event stream from which the new event stream is to be cloned. The new event stream may then be created with the same event attributes and/or configuration options as the existing event stream, and the user may use the GUI to customize the new event stream as a variant of the existing event stream (e.g., by adding or removing event attributes, filters, and/or aggregation information).

<figref idref="DRAWINGS">FIG. 11B</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 11B</figref> shows a screenshot of the GUI of <figref idref="DRAWINGS">FIG. 11A</figref> after the user has selected “DemoHTTP” from column <b>1102</b>. In response to the selection, the GUI displays configuration information and/or configuration options for the “DemoHTTP” event stream.

Like the GUI of <figref idref="DRAWINGS">FIG. 11A</figref>, the GUI of <figref idref="DRAWINGS">FIG. 11B</figref> may include a table. Each row in the table may represent an event attribute that is eligible for inclusion in the event stream. For example, an event attribute may be included in the table if the event attribute can be obtained from network packets that include the protocol of the event stream. Columns <b>1112</b>-<b>1120</b> of the table may allow the user to use the event attributes to generate time-series event data that is included the event stream. First, column <b>1112</b> includes a series of checkboxes that allows the user to include individual event attributes in the event stream or exclude the event attributes from the event stream. If a checkbox is checked, the corresponding event attribute is added to the event stream, and the row representing the event attribute is shown with other included event attributes in an alphabetized list at the top of the table. If a checkbox is not checked, the corresponding event attribute is omitted from the event stream, and the row representing the event attribute is shown with other excluded event attributes in an alphabetized list following the list of included event attributes. Those skilled in the art will appreciate that the GUI may utilize other sortings and/or rankings of event attributes in columns <b>1112</b>-<b>1120</b>.

Columns <b>1114</b>-<b>1118</b> may provide information related to the event attributes. Column <b>1114</b> may show the names of the event attributes, column <b>1116</b> may provide a description of each event attribute, and column <b>1118</b> may provide a term representing the event attribute. In other words, columns <b>1114</b>-<b>1118</b> may allow the user to identify the event attributes and decide whether the event attributes should be included in the event stream.

Column <b>1120</b> may include a series of links labeled “Add.” The user may select one of the links to access a portion of the GUI that allows the user to set a filter for the corresponding event attribute. The filter may then be used in the generation of the event stream from network data. Creation of filters for generating event streams from network packets is described in further detail below with respect to <figref idref="DRAWINGS">FIGS. 11D-11E</figref>.

The GUI of <figref idref="DRAWINGS">FIG. 11B</figref> also includes information <b>1122</b> related to the event stream. For example, information <b>1122</b> may include the name (e.g., “DemoHTTP”) of the event stream, the protocol classification and/or type (e.g., “http.event”) of the event stream, and the number of filters (e.g., “0 filters configured”) set for the event stream. Information <b>1122</b> may also include a checkbox <b>1136</b> that identifies if the event stream contains aggregated event data. If checkbox <b>1136</b> is checked, the GUI may be updated with options associated with configuring the generation of an aggregated event stream, as described below with respect to <figref idref="DRAWINGS">FIG. 11C</figref>.

Finally, the GUI of <figref idref="DRAWINGS">FIG. 11B</figref> includes a set of user-interface elements <b>1124</b>-<b>1134</b> for managing the event stream. First, the user may select user-interface element <b>1124</b> (e.g., “Enabled”) to enable generation of the event stream from network data and user-interface element <b>1126</b> (e.g., “Disabled”) to disable the generation of the event stream from the network data.

Next, the user may select user-interface element <b>1128</b> (e.g., “Clone”) to clone the event stream and user-interface element <b>1130</b> (e.g., “Delete”) to delete the event stream. If the user selects user-interface element <b>1128</b>, the GUI may obtain a name and description for the cloned event stream from the user. Next, the GUI may copy the content of columns <b>1112</b>-<b>1120</b>, including configuration options (e.g., checkboxes in column <b>1112</b> and filters added using links in column <b>1120</b>) that have been changed but not yet saved by the user, to a new screen for configuring the generation of the cloned event stream.

If the user selects user-interface element <b>1130</b>, the GUI may remove the event stream from the table in <figref idref="DRAWINGS">FIG. 11A</figref>. In turn, a representation of the event stream may be removed from the configuration information to stop the generation of time-series event data in the event stream by one or more remote capture agents.

The user may select user-interface element <b>1132</b> (e.g., “Cancel”) to discharge changes to the configuration information made in the current screen of the GUI. Conversely, the user may select user-interface <b>1134</b> (e.g., “Save”) to propagate the changes to the configuration information, and in turn, update the generation of event data from network packets captured by the remote capture agents based on the changes.

<figref idref="DRAWINGS">FIG. 11C</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. In particular, <figref idref="DRAWINGS">FIG. 11C</figref> shows a screenshot of the GUI of <figref idref="DRAWINGS">FIG. 11B</figref> after checkbox <b>1136</b> has been checked. Because checkbox <b>1136</b> is checked, the GUI includes a number of user-interface elements for configuring the generation of an aggregated event stream. The aggregated event stream may include aggregated event data, which in turn may be generated by aggregating and/or extracting event attributes from one or more network packets in a packet flow. For example, an HTTP event may be generated from one to several HTTP packets representing an HTTP request/response pair. Event attributes from multiple HTTP events may then be aggregated into a single aggregated HTTP event to reduce the amount of event data generated from the network data without losing important attributes of the event data.

As shown in <figref idref="DRAWINGS">FIG. 11C</figref>, a new column <b>1138</b> is added to the table. Each row in column <b>1138</b> may include a pair of user-interface elements (e.g., buttons) that allow the user to identify the corresponding event attribute as a key attribute or an aggregation attribute. One or more key attributes may be used to generate a unique key representing each aggregated event, and one or more aggregation attributes may be aggregated prior to inclusion in the aggregated event. Some event attributes (e.g., “dest_ip,” “src_ip,” “uri_path”) may only be used as key attributes because the event attributes are not numeric in nature. On the other hand, event attributes that may be summed (e.g., “dest_port,” “status,” “bytes,” “bytes_in,” “bytes_out,” “time_taken”) may have numeric values.

Event attributes identified as key attributes in column <b>1138</b> may be sorted at the top of the table, followed by event attributes identified as aggregation attributes. Event attributes that are not included in the event stream (e.g., event attributes with unchecked checkboxes in column <b>1112</b>) may be shown below the aggregation attributes in the table. Alternatively, event attributes may be displayed in the table according to other sortings and/or rankings.

While sums are the only type of aggregation shown in the GUI of <figref idref="DRAWINGS">FIG. 11C</figref>, other types of aggregation may also be used to generate aggregated event data. For example, aggregated event streams may be created using minimums, maximums, averages, standard deviations, and/or other summary statistics of event attributes.

The GUI of <figref idref="DRAWINGS">FIG. 11C</figref> also includes a user-interface element <b>1140</b> (e.g., a text box) for obtaining an aggregation interval over which event attributes are to be aggregated into a single aggregated event. The aggregation interval may be increased to increase the amount of aggregation in the aggregated event stream and reduced to decrease the amount of aggregation in the aggregated event stream.

For example, column <b>1138</b> may indicate that the “dest_ip,” “dest_port,” “src_ip,” “status,” and “uri_path” event attributes are specified as key attributes and the “bytes,” “bytes_in,” “bytes_out,” and “time_taken” event attributes are specified as aggregation attributes. Similarly, an aggregation interval of 60 seconds may be obtained from user-interface element <b>1140</b>. As a result, the aggregated event stream may include aggregated events generated from event data over a 60-second interval. After each 60-second interval has passed, a separate aggregated event with a unique key may be generated for each unique combination of “dest_ip,” “dest_port,” “src_ip,” “status,” and “uri_path” key attributes encountered during the interval. Values of “bytes,” “bytes_in,” “bytes_out,” and “time_taken” for events within the interval that match the unique combination of key attributes may also be summed and/or otherwise aggregated into the aggregated event. Aggregated events generated from the configuration options may then be shown in the same GUI, as described in further detail below with respect to <figref idref="DRAWINGS">FIG. 11F</figref>.

Such configuration of event streams and/or aggregated event streams may allow network data to be captured at different levels of granularity and/or for different purposes. For example, an aggregated event stream may include all possible event attributes for the event stream to enable overall monitoring of network traffic. On the other hand, one or more unaggregated event streams may be created to capture specific types of network data at higher granularities than the aggregated event stream. In addition, multiple event streams may be created from the same packet flow and/or event data to provide multiple “views” of the packet flow and/or event data.

<figref idref="DRAWINGS">FIG. 11D</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 11D</figref> shows a screenshot of the GUI of <figref idref="DRAWINGS">FIGS. 11B-11C</figref> after an “Add” link in column <b>1120</b> is selected. For example, the GUI of <figref idref="DRAWINGS">FIG. 11D</figref> may be shown as an overlay on the screens of <figref idref="DRAWINGS">FIGS. 11B-11C</figref> to enable the addition of filters to configuration information for the event stream(s) and/or aggregated event stream(s) shown on the screens.

As with the screenshots of <figref idref="DRAWINGS">FIGS. 11A-11C</figref>, the GUI of <figref idref="DRAWINGS">FIG. 11D</figref> includes information and/or user-interface elements organized into a table. Rows of the table may represent filters for an event stream and/or aggregated event stream, and columns <b>1142</b>-<b>1150</b> of the table may facilitate identification and/or configuration of the filters.

First, column <b>1142</b> may provide a list of terms representing event attributes to which the filters are to be applied. For example, column <b>1142</b> may specify an “http.status” term representing the “status” event attribute and an “http.uri-stem” term representing the “uri_path” event attribute.

Column <b>1144</b> may be used to provide a comparison associated with each filter. For example, a user may select a cell under column <b>1144</b> to access a drop-down menu of possible comparisons for the corresponding filter. As shown in <figref idref="DRAWINGS">FIG. 11D</figref>, the second cell of column <b>1144</b> is selected to reveal a drop-down menu of comparisons for a string-based event attribute (e.g., “uri_path”). Within the drop-down menu, “Regular Expression” is selected, while other options for the comparison may include “False,” “True,” “Is defined,” “Is not defined,” “Not Regular Expression,” “Exactly matches,” “Does not exactly match,” “Contains,” “Does not contain,” “Starts with,” “Does not start with,” “Ends with,” “Does not end with,” “Ordered before,” “Not ordered before,” “Ordered after,” and “Not ordered after.” As a result, a number of comparisons may be made with string-based event attributes during filtering of network data by the string-based event attributes.

Column <b>1146</b> may allow the user to specify a value against which the comparison in column <b>1144</b> is made. Cells in column <b>1146</b> may be text-editable fields and/or other user-interface elements that accept user input. For example, the second cell of column <b>1146</b> may include a value of “admin” that is entered by the user. Consequently, the values in the second cells of columns <b>1144</b>-<b>1146</b> may be used to generate a filter that determines if the “uri_path” event attribute from network data matches a regular expression of “admin.” If the network data matches the regular expression, the network data may be used to generate event data, which may subsequently be used to generate aggregated event data. If the network data does not match the regular expression, generation of event data from the network data may be omitted.

Column <b>1148</b> may include a set of checkboxes with a “Match All” header. The user may check a checkbox in column <b>1148</b> to require each value in a multi-value event attribute to match the filter. For example, the user may check a checkbox in column <b>1148</b> for a filter that is applied to a checksum event attribute to ensure that each of multiple checksums in a given network packet and/or event satisfies the comparison in the filter.

Column <b>1150</b> may allow the user to delete filters from the configuration information. For example, the user may select a user-interface (e.g., an icon) in a cell of column <b>1150</b> to remove the corresponding filter from the configuration information.

The GUI also includes a set of user-interface elements <b>1152</b>-<b>1154</b> for determining the applicability of individual filters or all filters to the network data. For example, the user may select user-interface element <b>1152</b> (e.g., “All”) to apply the filters so that only data that matches all filters in the table is used to generate events. Conversely, the user may select user-interface element <b>1154</b> (e.g., “Any”) to apply the filters so that data matching any of the filters in the data is used to generate events. In other words, user-interface element <b>1152</b> may be selected to apply a logical conjunction to the filters, while user-interface element <b>1154</b> may be selected to apply a logical disjunction to the filters.

<figref idref="DRAWINGS">FIG. 11E</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. As with the screenshot of <figref idref="DRAWINGS">FIG. 11D</figref>, <figref idref="DRAWINGS">FIG. 11E</figref> shows a GUI for adding and/or managing filters for generating event data at one or more remote capture components.

Within the GUI of <figref idref="DRAWINGS">FIG. 11E</figref>, the first cell of column <b>1144</b> is selected. In turn, a drop-down menu of possible comparisons is shown for the corresponding filter. Because the filter relates to a numeric event attribute (e.g., an HTTP status code), comparisons in column <b>1144</b> may be numeric in nature. For example, the “Greater than” comparison is selected, while other possible comparisons may include “False,” “True,” “Is defined,” “Is not defined,” “Equals,” “Does not equal,” “Less than,” “Greater than or equal to,” and “Less than or equal to.” The differences in comparisons shown in <figref idref="DRAWINGS">FIG. 11E</figref> and <figref idref="DRAWINGS">FIG. 11D</figref> may ensure that comparisons that are meaningful and/or relevant to the types of event attributes specified in the filters are used with the filters.

<figref idref="DRAWINGS">FIG. 11F</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 11F</figref> shows a screenshot of a GUI, such as GUI <b>1025</b> of <figref idref="DRAWINGS">FIG. 10</figref>. The GUI of <figref idref="DRAWINGS">FIG. 11F</figref> may provide information related to aggregated events, such as aggregated events generated using the GUI of <figref idref="DRAWINGS">FIG. 11C</figref>.

As shown in <figref idref="DRAWINGS">FIG. 11F</figref>, a first column <b>1156</b> contains a timestamp of an aggregated event, and a second column <b>1158</b> shows the aggregated event. Within column <b>1158</b>, the aggregated event includes a number of event attributes. Some of the event attributes (e.g., “dest_ip,” “dest_port,” “src_ip,” “status,” “uri_path”) are key attributes that are used to uniquely identify the aggregated event, and other event attributes (e.g., “dest_port,” “status,” “bytes,” “bytes_in,” “bytes_out,” “time_taken”) may be numerically summed before the event attributes are included in the aggregated event.

<figref idref="DRAWINGS">FIG. 12A</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 12A</figref> shows a screenshot of a GUI, such as GUI <b>1025</b> of <figref idref="DRAWINGS">FIG. 10</figref>. The GUI may be used with a risk-identification mechanism and/or a capture trigger, such as risk-identification mechanism <b>1007</b> and capture trigger <b>1009</b> of <figref idref="DRAWINGS">FIG. 10</figref>.

The GUI of <figref idref="DRAWINGS">FIG. 12A</figref> may include a portion <b>1202</b> that represents the risk-identification mechanism. For example, portion <b>1202</b> may display a dashboard of time-series event data that represents security risks. The dashboard includes a number of potential security risks, such as “HTTP Errors,” “DNS Errors,” “Cloud Email,” “NFS Activity,” and “Threat List Activity.” Events that match one of the listed potential security risks may be represented as bars within a time interval represented by the horizontal dimension of the dashboard. For example, a security risk <b>1206</b> may be shown as a series of bars clustered around an interval of time under “DNS Errors” in portion <b>1202</b>.

On the other hand, the dashboard may lack data for other potential security risks because the data volume associated with capturing network data across all protocols and/or security risks may be too large to effectively store and/or consume. As a result, portion <b>1202</b> may indicate that no data is available (e.g., “Search returned no results”) for the “HTTP Errors,” “Cloud Email,” “NFS Activity,” and “Threat List Activity” security risks.

The GUI may also include a portion <b>1204</b> that represents a capture trigger for generating additional time-series event data based on identified security risks from portion <b>1202</b>. For example, portion <b>1204</b> may include a checkbox that allows a user to activate the capture trigger upon identifying security risk <b>1206</b> in portion <b>1202</b>. Portion <b>1204</b> may also include a first drop-down menu that allows the user to specify one or more protocols (e.g., “HTTP,” “DNS,” “All Email,” “NFS/SMB,” “All Protocols”) of additional time-series event data to be captured with the capture trigger. Portion <b>1204</b> may additionally include a second drop-down menu that allows the user to specify a period (e.g., “4 Hours”) over which the additional time-series event data is to be captured after the capture trigger is activated.

After the capture trigger is activated, configuration information on one or more remote capture agents used to generate the time-series event data may be updated to include the additional protocol(s) specified in portion <b>1204</b>. For example, configuration information for configuring the generation of additional event streams from the specified protocol(s) may be propagated to the remote capture agents, and the remote capture agents may use the configuration to create the event streams from network data and/or event data at the remote capture agents. The configuration information may include default event attributes for the protocol(s) and/or event attributes that may be of interest to the security assessment of network packet flows. For example, the configuration information may specify the generation of event data related to other security risks, such as the security risks shown in the dashboard. Once the event data is generated and/or indexed, the event data may be shown in the dashboard to facilitate verification, monitoring, and/or analysis of the security risk. After the pre-specified period obtained from portion <b>1204</b> has passed, the configuration information on the remote capture agents may be updated to disable the generation of the additional event streams and reduce the volume of network data captured by the remote capture agents.

As with the user interfaces of <figref idref="DRAWINGS">FIGS. 11A-11E</figref>, the user may add one or more filters that are applied during the generation of the additional time-series event data. For example, the user may use the user interfaces of <figref idref="DRAWINGS">FIGS. 11D-11E</figref> to add a filter for network and/or event data that exactly matches the IP address (e.g., 10.160.26.206) from which the security risk was detected. As a result, the additional time-series data may be generated only from network data containing the same source IP address. The user may also use the user interfaces of <figref idref="DRAWINGS">FIGS. 11A-11C</figref> to customize the collection of additional time-series event data by protocol and/or event attributes.

<figref idref="DRAWINGS">FIG. 12B</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. In particular, <figref idref="DRAWINGS">FIG. 12B</figref> shows a screenshot of a GUI, such as GUI <b>1025</b> of <figref idref="DRAWINGS">FIG. 10</figref>. Like the GUI of <figref idref="DRAWINGS">FIG. 12A</figref>, the GUI of <figref idref="DRAWINGS">FIG. 12B</figref> includes a first portion <b>1206</b> representing a risk-identification mechanism and a second portion <b>1208</b> representing a capture trigger.

Portion <b>1206</b> may allow a user to create a recurring search for time-series event data that matches a security risk. For example, portion <b>1206</b> may include user-interface elements for obtaining a domain, application context, description, search terms, time range (e.g., start and end times), and/or frequency (e.g., daily, hourly, every five minutes, etc.) for the recurring search. The user may use the user-interface elements of portion <b>1206</b> to specify a recurring search for an excessive number of failed login attempts in captured network and/or event data, which may represent brute force access behavior that constitutes a security risk.

Portion <b>1208</b> may allow the user to provide the capture trigger, which is automatically activated if the recurring search finds time-series event data that matches the security risk. As with portion <b>1204</b> of <figref idref="DRAWINGS">FIG. 12A</figref>, portion <b>1208</b> may allow the user to set the capture trigger, specify one or more protocols to be captured with the capture trigger, and/or a pre-specified period over which network data using the protocol(s) is to be captured.

After the user has finished defining the recurring search and capture trigger, the user may select a user-interface <b>1210</b> (e.g., “Save”) to save the recurring search and capture trigger. The capture trigger may then be activated without additional input from the user once an iteration of the recurring search identifies the security risk. Conversely, the user may select a user-interface element <b>1212</b> (e.g., “Cancel”) to exit the screen of <figref idref="DRAWINGS">FIG. 12B</figref> without creating the recurring search and/or capture trigger.

<figref idref="DRAWINGS">FIG. 13</figref> shows a flowchart illustrating the processing of network data in accordance with the disclosed embodiments. In one or more embodiments, one or more of the steps may be omitted, repeated, and/or performed in a different order. Accordingly, the specific arrangement of steps shown in <figref idref="DRAWINGS">FIG. 13</figref> should not be construed as limiting the scope of the embodiments.

Initially, configuration information is obtained at a remote capture agent from a configuration server over a network (operation <b>1302</b>). The remote capture agent may be located on a separate network from that of the configuration server. For example, the remote capture agent may be installed on a physical and/or virtual machine on a remote network and/or cloud. As discussed above, the remote capture agent and other remote capture agents may be used to capture network data from a set of remote networks in a distributed manner.

Next, the configuration information is used to configure the generation of event data from network packets captured by the remote capture agent during the runtime of the remote capture agent (operation <b>1304</b>). For example, the configuration information may be used to configure the remote capture agent to identify certain types of network packets, extract network data from the network packets, and/or include the network data in the event data.

The remote capture agent may identify network packets in a packet flow based on control information in the network packets (operation <b>1306</b>). For example, network packets between a source and destination may be identified based on source and/or destination network addresses, source and/or destination ports, and/or transport layer protocols in the headers of the network packets.

The remote capture agent may also assemble the packet flow from the network packets (operation <b>1308</b>) and/or decrypt the network packets upon detecting encryption of the network packets (operation <b>1310</b>). For example, the remote capture agent may rearrange out-of-order TCP packets into a TCP stream. The remote capture agent may also analyze the byte signatures of the network packets' payloads to identify encryption of the network packets and use an available private key to decrypt the network packets.

After the packet flow is identified, assembled and/or decrypted, the remote capture agent may obtain a protocol classification for the packet flow (operation <b>1312</b>). For example, the remote capture agent may provide network packets in the packet flow to a protocol-decoding mechanism and receive one or more protocol identifiers representing the protocols used by the network packets from the protocol-decoding mechanism.

Next, the remote capture agent may use configuration information associated with the protocol classification to build an event stream from the packet flow (operation <b>1314</b>), as described in further detail below with respect to <figref idref="DRAWINGS">FIG. 14</figref>. The remote capture agent may then transmit the event stream over a network for subsequent storage and processing of the event stream by one or more components on the network (operation <b>1316</b>). For example, the remote capture agent may transmit the event stream to one or more data storage servers, configuration servers, and/or indexers on the network.

An update to the configuration information may be received (operation <b>1316</b>). For example, the remote capture agent may receive an update to the configuration information after the configuration information is modified at a configuration server. If an update to the configuration information is received, the update is used to reconfigure the generation of time-series event data at the remote capture agent during runtime of the remote capture agent (operation <b>1320</b>). For example, the remote capture agent may be use the updated configuration information to generate one or more new event streams, discontinue the generation of one or more existing event streams, and/or modify the generation of one or more existing event streams.

The remote capture agent may continue to be used (operation <b>1322</b>) to capture network data. If the remote capture agent is to be used, packet flows captured by the remote capture agent are identified (operation <b>1306</b>), and network packets in the packet flows are assembled into the packet flows and/or decrypted (operations <b>1308</b>-<b>1310</b>). Protocol classifications for the packet flows are also obtained and used, along with configuration information associated with the protocol classifications, to build event streams from the packet flows (operations <b>1312</b>-<b>1314</b>). The event streams are then transmitted over the network (operation <b>1316</b>), and any updates to the configuration information are used to reconfigure the operation of the remote capture agent (operations <b>1318</b>-<b>1320</b>) during generation of the event streams. Capture of network data by the remote capture agent may continue until the remote capture agent is no longer used to generate event data from network data.

<figref idref="DRAWINGS">FIG. 14</figref> shows a flowchart illustrating the process of using configuration information associated with a protocol classification to build an event stream from a packet flow in accordance with the disclosed embodiments. In one or more embodiments, one or more of the steps may be omitted, repeated, and/or performed in a different order. Accordingly, the specific arrangement of steps shown in <figref idref="DRAWINGS">FIG. 14</figref> should not be construed as limiting the scope of the embodiments.

First, one or more event attributes associated with the protocol classification are obtained from the configuration information (operation <b>1402</b>). For example, the event attribute(s) may be obtained from a portion of the configuration information that specifies the generation of an event stream from network data matching the protocol classification.

Next, the event attribute(s) are extracted from network packets in the packet flow (operation <b>1404</b>). For example, the event attribute(s) may be used to generate event data from the network packets. The configuration information may optionally be used to transform the extracted event attribute(s) (operation <b>1406</b>). For example, the configuration information may be used to aggregate the event data into aggregated event data that reduces the volume of event data generated while retaining the important aspects of the event data.

Finally, the extracted and/or transformed event attributes are included in the event stream (operation <b>1408</b>). For example, the event stream may be include a series of events and/or aggregated events that contain event attributes that are relevant to the protocol classification of the network packets represented by the events.

<figref idref="DRAWINGS">FIG. 15</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments. In one or more embodiments, one or more of the steps may be omitted, repeated, and/or performed in a different order. Accordingly, the specific arrangement of steps shown in <figref idref="DRAWINGS">FIG. 15</figref> should not be construed as limiting the scope of the embodiments.

First, a GUI for obtaining configuration information for configuring the generation of time-series event data from network packets captured by one or more remote agents is provided (operation <b>1502</b>). The GUI may include a number of user-interface elements for streamlining the creation and/or update of the configuration information. First, the GUI may provide a set of user-interface elements for including one or more event attributes in the time-series event data of an event stream associated with a protocol classification of the network packets (operation <b>1504</b>). For example, the GUI may include a set of checkboxes that enable the selection of individual event attributes for inclusion in the time-series event data.

Second, the GUI may provide a set of user-interface elements for managing the event stream (operation <b>1506</b>) and/or obtaining the protocol classification for the event stream. For example, the GUI may include one or more user-interface elements for cloning the event stream from an existing event stream, which imparts the protocol classification of the existing event stream on the cloned event stream. The GUI may also include user-interface elements for deleting the event stream, enabling the event stream, and/or disabling the event stream.

Third, the GUI may provide a set of user-interface elements for filtering the network packets (operation <b>1508</b>) prior to generating the time-series event data from the network packets. Each filter may identify an event attribute, a comparison to be performed on the event attribute, and/or a value to which the event attribute is to be compared. For example, the filter may match the event attribute to a Boolean value (e.g., true or false), perform a numeric comparison (e.g., equals, greater, less than, greater than or equal to, less than or equal to), and/or verify the definition of (e.g., the existence of) the event attribute in network data. The filter may also compare the event attribute to a regular expression, perform an exact match of the event attribute to the value, perform a partial match of the event attribute to the value, and/or determine the event attribute's position in an ordering.

Fourth, the GUI may provide a set of user-interface elements for aggregating the event attribute(s) into aggregated event data that is included in the event stream (operation <b>1510</b>). For example, the GUI may provide user-interface elements for identifying event attributes as key attributes used to generate a key representing the aggregated event data and/or aggregation attributes to be aggregated prior to inclusion in the aggregated event data. The GUI may also include one or more user-interface elements for obtaining an aggregation interval over which the one or more event attributes are aggregated into the aggregated event data.

Finally, the event attribute(s), protocol classification, filtering information, and/or aggregation information obtained from the GUI are included in the configuration information (operation <b>1512</b>). The configuration information may then be used to configure the protocol-based capture, filtering, and/or aggregation of network data at the remote capture agent(s).

<figref idref="DRAWINGS">FIG. 16</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments. In one or more embodiments, one or more of the steps may be omitted, repeated, and/or performed in a different order. Accordingly, the specific arrangement of steps shown in <figref idref="DRAWINGS">FIG. 16</figref> should not be construed as limiting the scope of the embodiments.

Initially, a risk-identification mechanism for identifying a security risk from time-series event data generated from network packets captured by one or more remote capture agents distributed across a network is provided (operation <b>1602</b>). The risk-identification mechanism may include a GUI that displays an event of interest related to the security risk. For example, the GUI may show potential security risks in a dashboard and/or other visualization of the time-series event data. Alternatively, the risk-identification mechanism may include a search and/or recurring search for a subset of the time-series event data matching the security risk. For example, the risk-identification mechanism may include a search mechanism that allows a user to search for threats, attacks, errors, and/or other notable events in the time-series event data.

Next, a capture trigger for generation additional time-series data from the network packets on the remote capture agent(s) based on the security risk is provided (operation <b>1604</b>). The capture trigger may be received through one or more user-interface elements of a GUI, such as the same GUI used to provide the risk-identification mechanism. For example, the capture trigger may be activated in a portion of the GUI that is above, below, and/or next to a dashboard that displays security risks to the user. Alternatively, the capture trigger may be linked to a recurring search for time-series event data that matches a security risk. As a result, the capture trigger may automatically be activated once time-series event data matching the security risk is found.

After the capture trigger is activated, the capture trigger is used to configure the generation of the additional time-series event data from the network packets (operation <b>1606</b>). For example, activation of the capture trigger may result in the updating of configuration information for the remote capture agent(s), which causes the remote capture agent(s) to generate additional event streams containing event attributes associated with protocols that facilitate analysis of the security risk.

Finally, generation of the additional time-series event data is disabled after a pre-specified period has passed (operation <b>1608</b>). For example, generation of the additional time-series event data may be set to expire a number of hours or days after the capture trigger is activated. The expiry may be set by the user and/or based on a default expiration for security-based capture of additional network data from network packets.

<figref idref="DRAWINGS">FIG. 17A</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 17A</figref> shows a screenshot of a GUI, such as GUI <b>1025</b> of <figref idref="DRAWINGS">FIG. 10</figref>. As described above, the GUI may be used to obtain configuration information that is used to configure the generation of event streams containing time-series event data at one or more remote capture agents distributed across a network.

As shown in <figref idref="DRAWINGS">FIG. 17A</figref>, the GUI includes a table with a set of columns <b>1708</b>-<b>1724</b>. Columns <b>1710</b>-<b>1718</b> may include high-level event stream information related to event streams that are created and/or managed using the configuration information. Each row of the table may represent an event stream, and rows of the table may be sorted by column <b>1710</b>.

Column <b>1710</b> shows an alphabetized list of names of the event streams, and column <b>1714</b> may specify a protocol associated with each event stream. For example, columns <b>1710</b> and <b>1714</b> may include names and/or protocols of event streams generated from HTTP, Dynamic Host Configuration Protocol (DHCP), DNS, FTP, email protocols, database protocols, NFS, Secure Message Block (SMB), security protocols, Session Initiation Protocol (SIP), TCP, and/or UDP network traffic. In other words, the event streams may be generated based on transport layer protocols, session layer protocols, presentation layer protocols, and/or application layer protocols.

A user may select a name of an event stream under column <b>1710</b> to access and/or update configuration information for configuring the generation of the event stream. For example, the user may select “Stream_A” in column <b>1710</b> to navigate to a screen of the GUI that allows the user to specify event attributes, filters, and/or aggregation information related to creating the “Stream_A” event stream.

Column <b>1712</b> specifies a type indicating whether each event stream is cloned from an existing event stream. For example, column <b>1712</b> may indicate that the “Stream_C” and “Stream_D” streams have been cloned (e.g., copied) from other event streams, while the remaining event streams may be predefined with default event attributes.

Column <b>1716</b> shows an application associated with each event stream, and column <b>1718</b> includes a description of each event stream. For example, column <b>1716</b> may include the names of applications used to create the event streams (e.g., “Stream,” “Enterprise Security,” etc.), and column <b>1718</b> may include descriptions that are generated by the applications and/or users of the applications.

The table may also include a column <b>1722</b> that specifies a status indicating whether each event stream is enabled or disabled. For example, column <b>1722</b> may indicate that the “Stream_A,” “Stream_B,” “Stream_C,” and “Stream_E” event streams are enabled. If an event stream is enabled, time-series event data may be included in the event stream based on the configuration information for the event stream. If an event stream is disabled, time-series event data may not be generated for the event stream.

Event streams in the table may further be sorted by information in other columns of the table and/or randomly. For example, the user may select the column header of a given column (e.g., columns <b>1708</b>-<b>1724</b>) to order the displayed event stream information by the information represented by the column. Alternatively, event streams in the table may be randomly sorted and/or sorted by an attribute that is not explicitly displayed in a column of the table.

The GUI also includes a user-interface element <b>1730</b> (e.g., “Clone Stream”). A user may select user-interface element <b>1730</b> to create a new event stream as a copy of an event stream listed in the GUI. After user-interface element <b>1730</b> is selected, an overlay may be displayed that allows the user to specify a name for the new event stream, a description of the new event stream, and an existing event stream from which the new event stream is to be cloned. The new event stream may then be created with the same event attributes and/or configuration options as the existing event stream, and the GUI may navigate the user to a new screen for customizing the new event stream as a variant of the existing event stream (e.g., by adding or removing event attributes, filters, and/or aggregation information).

As mentioned above, the GUI may include functionality to group event information for the event streams by one or more event stream attributes. In particular, the GUI may provide a user-interface element <b>1704</b> for specifying an event stream attribute by which event streams are to be grouped. For example, user-interface element <b>1704</b> may be a drop-down menu that allows the user to specify grouping of the event streams by “Protocol,” “Category,” or “Apps” (e.g., applications). As shown in <figref idref="DRAWINGS">FIG. 17A</figref>, the “Protocol” event stream attribute is specified in user-interface element <b>1704</b>. In response to the selection of “Protocol” in user-interface element <b>1704</b>, the GUI may display a list <b>1702</b> of possible values for the “Protocol” event stream attribute. For example, list <b>1702</b> may include protocols such as HTTP, FTP, TCP, UDP, and SMTP.

The user may select a protocol name from list <b>1702</b> to view event stream information for a subset of the event streams matching the protocol (e.g., event streams containing time-series event data generated from network packets classified as using the protocol). Because “HTTP” is selected in list <b>1702</b>, the table may show event stream information for event streams that match the “HTTP” protocol classification, as indicated in column <b>1714</b>. The user may select another protocol name from list <b>1702</b> to view event streams associated with another protocol represented by the protocol name, or the user may select “All” in list <b>1702</b> to view all event streams, regardless of the event streams' protocol classifications.

The GUI may additionally group the event stream information by an event stream lifecycle of the event streams. In particular, the GUI may include two user-interface elements <b>1766</b>-<b>1768</b> for specifying an event stream lifecycle. The user may select user-interface element <b>1766</b> (e.g., “Permanent”) to view event stream information for permanent event streams and user-interface element <b>1768</b> (e.g., “Ephemeral”) to view event stream information for ephemeral event streams. Selection of one user-interface element <b>1766</b>-<b>1768</b> may result in the automatic deselection of the other user-interface element. In response to the selection of either user-interface element <b>1766</b> or user-interface element <b>1768</b>, the GUI may further group event stream information shown in the table by the event stream lifecycle represented by the selected user-interface element. For example, the GUI may show only permanent event streams that match the “HTTP” protocol classification in the table of <figref idref="DRAWINGS">FIG. 17A</figref> because user-interface element <b>1766</b> and “HTTP” are selected.

To further facilitate analysis and/or management of the event streams, column <b>1720</b> of the table may include graphs of metrics associated with the event streams inline with event stream information for the event streams. The graphs may be generated using time-series event data for the event streams. For example, column <b>1720</b> may show, for each event stream represented by a row in the table, a sparkline of network traffic over time for the event stream. Alternatively, column <b>1720</b> may show graphs and/or sparklines of other metrics, such as a number of events and/or a number of notable events over time. As with other columns in the table, column <b>1720</b> may be updated based on user interaction with user-interface elements <b>1704</b> and <b>1766</b>-<b>1768</b> and list <b>1702</b>. For example, the selection of one or more other groupings using user-interface elements <b>1704</b> and <b>1766</b>-<b>1768</b> and list <b>1702</b> may trigger the display of event stream information and graphs in the table for event streams matching the other grouping(s).

The user may click on a graph in column <b>1720</b> to navigate to a screen containing a larger version of the graph and one or more user-interface elements for changing a view of the graph. For example, after the user selects a graph in column <b>1720</b>, the GUI may navigate the user to a dashboard with a more detailed version of the graph that includes a scale and/or labeled axes. The dashboard may also include scrollbars, sliders, buttons, and/or other user-interface elements that allow the user to change the scale along one or both axes, scroll across different portions of the data (e.g., different time ranges), and/or view data from multiple event streams in the same graph.

The GUI may also include a user-interface element <b>1732</b> that shows an aggregated value of the metric in the graphs of column <b>1720</b>. For example, user-interface element <b>1732</b> may include a sparkline of aggregate network traffic, events, and/or notable events over time for the event streams represented by the rows of the table. The aggregate metric may be calculated as a sum, average, and/or other summary statistic.

User-interface element <b>1732</b> may also display a numeric value of the aggregate metric (e.g., “154 Mb/s”) over the time spanned by the sparkline. For example, the numeric value shown to the left of the sparkline in user-interface element <b>1732</b> may represent a value of aggregate network traffic at a time represented by a given point in the sparkline. The user may position a cursor at different points along the sparkline to view different values of the aggregate network traffic represented by the points. Similarly, the user may position the cursor at different points in the graphs of column <b>1720</b> to trigger the display of numeric values of network traffic at times represented by those points.

As with graphs in column <b>1720</b>, the graph in user-interface element <b>1732</b> may be generated or updated based on the event stream information shown in the table. For example, the selection of “UDP” in list <b>1702</b> may cause the GUI to display event stream information for event streams matching the “UDP” protocol classification. In turn, graphs in column <b>1720</b> and user-interface element <b>1732</b> may be updated to reflect network traffic, events, notable events, network bandwidth, total bandwidth, protocol-based bandwidth, and/or other metrics associated with the “UDP” event streams.

The graphs in column <b>1720</b> and/or user-interface element <b>1732</b> may further be updated in real-time with time-series event data as the time-series event data is received from one more remote capture agents. For example, sparklines and/or other graphical representations in column <b>1720</b> and user-interface element <b>1732</b> may shift as the time window spanned by the sparklines advances and additional time-series event data is collected within the time window.

In addition to displaying event stream information and graphs for one or more groupings of event streams, the GUI of <figref idref="DRAWINGS">FIG. 17A</figref> may enable management of the event streams through a column <b>1724</b> that allows the user to perform one or more actions on individual event streams. Each row of the table may include a user-interface element in column <b>1724</b> that, when selected, activates a drop-down menu of possible actions to be applied to the corresponding event stream.

Within the GUI, the user-interface element in the first row of column <b>1724</b> may be selected. As a result, a drop-down menu may be displayed below the user-interface element with a set of options, including “Disable,” “Clone,” and “Delete.” The user may select “Disable” to disable generation of the event stream from network data. Alternatively, the “Disable” option may be replaced with an “Enable” option if the event stream (e.g., “Stream_D”) is already disabled to allow the user to enable generation of the event stream from network data.

The user may select “Clone” to create a new event stream as a copy of the event stream. If “Clone” is selected, the GUI may obtain a name and description for the cloned event stream. The GUI may then copy configuration information for the event stream to a new screen for configuring the cloned event stream. As described in the above-referenced application, configuration of new and/or cloned event streams may include one or more event attributes to be included in the event stream, filtering network packets prior to generating the event stream from the network packets, and/or aggregating the event attribute(s) into aggregated event data that is included in the event stream.

The user may select “Delete” to delete the event stream. If “Delete” is selected, the GUI may remove event stream information for the event stream from the table. In turn, a representation of the event stream may be removed from the configuration information to stop the generation of time-series event data in the event stream by one or more remote capture agents.

The GUI may additionally include a user-interface element <b>1726</b> (e.g., “Bulk Edit”) that allows the user to apply an action associated with managing an event stream to multiple event streams. The user may use a set of checkboxes in column <b>1708</b> to select one or more event streams to which the action is to be applied. The user may then select user-interface element <b>1726</b> to access a drop-down menu containing a set of possible actions to apply to the selected event streams. For example, the drop-down menu may include options for enabling, disabling, and deleting the selected event streams, which are similar to options in the drop-down menu of user-interface elements in column <b>1724</b>.

Finally, the GUI may include a user-interface element <b>1728</b> that allows the user to search for event streams. The user may type one or more keywords into a text box provided by user-interface element <b>1728</b>, and the GUI may match the keyword(s) to the names, descriptions, and/or other event stream attributes of the event streams in the table. Event stream information for event streams that do not match the keyword(s) may be removed from the table while the search is in effect. User-interface element <b>1728</b> may thus provide another mechanism by which event stream information in the table can be grouped and/or filtered. Consequently, the GUI of <figref idref="DRAWINGS">FIG. 17A</figref> may allow the user to create, find, and/or manage event streams across multiple applications, categories, keywords, and/or protocols that may be relevant to the user's interests or goals.

<figref idref="DRAWINGS">FIG. 17B</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 17B</figref> shows the GUI of <figref idref="DRAWINGS">FIG. 17A</figref> after the event stream attribute in user-interface element <b>1704</b> is changed from “Protocol” to “Category.” In response to the change, the GUI may update list <b>1702</b> with possible values for the “Category” event stream attribute. For example, list <b>1702</b> may include different technological categories of network data represented by the event streams, such as “Infrastructure,” “Networking,” “File Transfer,” “Web,” and “Email.” As with the GUI of <figref idref="DRAWINGS">FIG. 17A</figref>, the user may select “All” in list <b>1702</b> to view all event streams, regardless of the categories to which the event streams belong.

Other categories not shown in list <b>1702</b> may include, but are not limited to, messaging, authentication, database, telephony, and/or network management. Finally, categories in list <b>1702</b> may include one or more user-created values. For example, the GUI may provide one or more user-interface elements that allow the user to specify a name of a new category, along with one or more event stream attributes of event streams to be included under the new category.

Within list <b>1702</b>, “Networking” is selected. As a result, the table may include names, types, protocols, applications, descriptions, and/or other event stream information for permanent event streams in a networking category, such as event streams associated with networking protocols (e.g., DHCP, DNS, TCP, UDP). Sparklines in column <b>1720</b> and user-interface element <b>1732</b> may also be updated to reflect metrics and an aggregated metric associated with the event streams represented by the rows of the table in <figref idref="DRAWINGS">FIG. 17B</figref>, respectively.

<figref idref="DRAWINGS">FIG. 17C</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 17C</figref> shows the GUI of <figref idref="DRAWINGS">FIG. 17B</figref> after user-interface element <b>1768</b> has been selected and the event stream attribute in user-interface element <b>1704</b> is changed from “Category” to “Apps.” In the GUI of <figref idref="DRAWINGS">FIG. 17C</figref>, list <b>1702</b> includes possible values for the “Apps” (e.g., applications) event stream attribute. For example, list <b>1702</b> may include names of applications (e.g., “Stream,” “Enterprise Security”) associated with event streams, such as applications from which the event streams were created. Within list <b>1702</b>, “Enterprise Security” is selected. The user may select another application name from list <b>1702</b> to view event streams associated with another application represented by the application name, or the user may select “All” in list <b>1702</b> to view all event streams, regardless of the applications associated with the event streams.

Because user-interface element <b>1768</b> is also selected, the table includes event stream information for ephemeral event streams. In other words, the GUI may group event streams by application and event stream lifecycle so that event stream information for event streams that match both the “Enterprise Security” application name and the ephemeral event stream lifecycle is shown in the table.

The user may apply an additional grouping or filter to event stream information shown in the table by performing a search using user-interface element <b>1728</b>. For example, the user may type one or more keywords into a text box provided by user-interface element <b>1728</b>, and the GUI may match the keyword(s) to the names, descriptions, and/or other event stream attributes of the ephemeral event streams in the table. Event stream information for event streams that do not match the keyword(s) may be removed from the table while the search is in effect.

As shown in <figref idref="DRAWINGS">FIG. 17C</figref>, the table includes a different set of columns <b>1734</b>-<b>1750</b> from the table of <figref idref="DRAWINGS">FIGS. 17A-17B</figref>. Unlike columns <b>1710</b>-<b>1718</b> of <figref idref="DRAWINGS">FIGS. 17A-17B</figref>, columns <b>1734</b>-<b>1746</b> may include event stream information that is relevant to ephemeral event streams instead of permanent event streams. Column <b>1734</b> may show an alphabetized or otherwise ordered or unordered list of names of groups of ephemeral event streams, and column <b>1736</b> may show the number of event streams in each group. The user may select a user-interface element in a row of column <b>1750</b> to expand the table to show event stream information for ephemeral event streams in the group represented by the row. For example, the user may select the user-interface element in the first row of column <b>1750</b> to view event stream information for <b>120</b> ephemeral event streams belonging to the group named “Group_A,” as discussed in further detail below with respect to <figref idref="DRAWINGS">FIG. 17D</figref>.

The user may also select a value in column <b>1734</b> to view time-series event data for the corresponding ephemeral event stream or group of ephemeral event streams. For example, selection of the “Group_A” value in column <b>1734</b> may cause the GUI to navigate to a screen showing events and the corresponding timestamps of the ephemeral event streams of “Group_A,” graphs of metrics related to the events, and/or other information associated with the events.

Column <b>1738</b> may show the names of applications used to create the ephemeral event streams. Because “Enterprise Security” is selected in list <b>1702</b>, all values in column <b>1738</b> are matched to the “Enterprise Security” application name, and event stream information for ephemeral event streams associated with other applications (e.g., “Stream”) is omitted from the table.

In addition, column <b>1738</b> may allow the user to navigate from the event stream information for a given ephemeral event stream to creation information for a creator of the ephemeral event stream. For example, each application name in column <b>1738</b> may include a hyperlink to a screen of the GUI for interacting with the application represented by the application name. The screen may show user-interface elements and/or information that provides context for the creation of the ephemeral event stream. As a result, column <b>1738</b> may link the portion of the GUI used to manage the ephemeral event stream to the portion of the GUI used to create the ephemeral event stream, which is described in further detail below with respect to <figref idref="DRAWINGS">FIG. 17D</figref>.

Columns <b>1740</b>-<b>1744</b> show start times, end times, and times remaining for the ephemeral event streams, respectively. The start times may represent times at which generation of time-series event data for the corresponding ephemeral event streams was initiated. For example, each start time may be a time at which an ephemeral event stream was created by a capture trigger for generating additional time-series event data based on a security risk and/or an application that collects time-series event data from a number of sources for subsequent analysis and/or correlation.

The end times may be times at which generation of time-series event data for the corresponding ephemeral event streams is to end. For example, each end time may be a time that is a pre-specified number of minutes, hours, and/or days from the corresponding start time. The amount of time spanned between the start and end time may thus represent the duration of the ephemeral event stream, which may be selected by a capture trigger, application, and/or user interacting with the capture trigger or application. Once the end time for an ephemeral event stream is reached, the ephemeral event stream is terminated.

The times remaining for the ephemeral event streams may indicate the amount of time left in the lifetimes of the ephemeral event streams. For example, each value in column <b>1744</b> may represent a “countdown” to the end time of the corresponding ephemeral event stream shown in column <b>1742</b>.

Column <b>1746</b> may provide a status indicating whether each ephemeral event stream or group of ephemeral event streams is enabled or disabled. For example, column <b>1746</b> may indicate that the “Group_A,” “Group_B,” and “Group_E” groups of ephemeral event streams are enabled. Such enabling or disabling of ephemeral event streams may be independent of the creation or termination of the ephemeral event streams. For example, an ephemeral event stream may be created at the start time of the ephemeral event stream by updating one or more remote capture agents with configuration information for the ephemeral event stream. Between the start and end times of the ephemeral event stream, the ephemeral event stream may be disabled to stop the generation of time-series event data for the ephemeral event stream and/or re-enabled to resume the generation of time-series event data for the ephemeral event stream. Once the end time of the ephemeral event stream is reached, the ephemeral event stream may be terminated, and a representation of the event stream may be removed from the configuration information to stop the generation of time-series event data in the event stream by the remote capture agent(s).

Like the GUIs of <figref idref="DRAWINGS">FIGS. 17A-17B</figref>, the GUI may provide a number of mechanisms for managing the ephemeral event streams. First, a column <b>1748</b> in the table may allow the user to perform one or more actions on individual event streams. Each row of the table may include a user-interface element in column <b>1748</b> that, when selected, activates a drop-down menu of possible actions to be applied to the corresponding event stream.

Within the GUI, the user-interface element in the first row of column <b>1748</b> may be selected. As a result, a drop-down menu may be displayed below the user-interface element with a set of options, including “Disable,” “Delete,” and “Modify End Time.” The user may select “Disable” to disable generation of the ephemeral event stream from network data before the end time of the ephemeral event stream is reached. Alternatively, the “Disable” option may be replaced with an “Enable” option if the event stream (e.g., “Stream_D”) is already disabled to allow the user to enable generation of the event stream from network data before the end time of the ephemeral event stream is reached.

The user may select “Delete” to delete the ephemeral event stream. If “Delete” is selected, the GUI may remove the ephemeral event stream from the table, even if the end time of the ephemeral event stream has not been reached. In turn, a representation of the event stream may be removed from the configuration information to stop the generation of time-series event data in the event stream by one or more remote capture agents.

The user may select “Modify End Time” to modify the end time of the ephemeral event stream shown in column <b>1742</b>. If “Modify End Time” is selected, the GUI may display an overlay that allows the user to specify a new end time for the ephemeral event stream as a date and time and/or a number of minutes, hours, and/or days by which the existing end time should be extended or reduced.

Second, user-interface element <b>1726</b> (e.g., “Bulk Edit”) may be used to apply an action associated with managing the event streams to multiple ephemeral event streams. The user may use a set of checkboxes in column <b>1708</b> to select the event streams to which the action is to be applied. The user may then select user-interface element <b>1726</b> to access a drop-down menu containing a set of possible actions to apply to the selected event streams. For example, the drop-down menu may include options for enabling, disabling, and deleting the selected event streams, which are similar to options in the drop-down menu of user-interface elements in column <b>1746</b>.

Third, user-interface element <b>1730</b> may allow the user to create a new ephemeral event stream as a copy of an existing ephemeral event stream. After user-interface element <b>1730</b> is selected, an overlay may be displayed that includes user-interface elements for specifying a name for the new ephemeral event stream, a description of the new event stream, and an existing ephemeral event stream from which the new ephemeral event stream is to be cloned. The new ephemeral event stream may be created with the same event attributes and/or configuration options as the existing ephemeral event stream, including the same end time and/or duration as the existing ephemeral event stream. The GUI may then show a new screen that allows the user to customize the new ephemeral event stream as a variant of the existing ephemeral event stream.

<figref idref="DRAWINGS">FIG. 17D</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 17D</figref> shows the GUI of <figref idref="DRAWINGS">FIG. 17C</figref> after the user-interface element in the first row of column <b>1750</b> has been selected. In response to the selected user-interface element, the table includes additional event stream information for ephemeral event streams in the group represented by the first row in the table. As shown in <figref idref="DRAWINGS">FIG. 17D</figref>, the additional event stream information includes an additional grouping of ephemeral event streams in the “Group_A” group by protocol. For example, the GUI may indicate that the 120 ephemeral event streams in the group are further grouped into 80 ephemeral event streams for capturing HTTP network packets, 20 ephemeral event streams for capturing FTP network packets, and 20 ephemeral event streams for capturing UDP packets.

All ephemeral event streams in the group may be created by the same application (e.g., “Enterprise Security”) and have the same start and end times. As a result, the ephemeral event streams may be created by the application for the same purpose or similar purposes. For example, the “Enterprise Security” application may create 120 ephemeral event streams for generating additional time-series event data from network packets based on a security risk.

<figref idref="DRAWINGS">FIG. 17E</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. In particular, <figref idref="DRAWINGS">FIG. 17E</figref> shows the GUI of <figref idref="DRAWINGS">FIG. 17C</figref> after a hyperlink (e.g., “Enterprise Security”) in the second row of column <b>1738</b> has been selected. The hyperlink may navigate the user from a screen for managing the ephemeral event stream represented by the second row of the table to a screen containing creation information for a creator of the ephemeral event stream.

The GUI of <figref idref="DRAWINGS">FIG. 17E</figref> may show a creator name (e.g., “Enterprise Security: Asset Investigator”) of the creator. For example, the name may specify an application (e.g., “Enterprise Security”) and/or a feature of the application (e.g., “Asset Investigator”) used to create the ephemeral event stream. The GUI may also include a portion <b>1758</b> that shows a trigger condition for creating or activating the ephemeral event stream. For example, portion <b>1758</b> may be a risk-identification mechanism that displays a dashboard of time-series event data representing security risks. The dashboard includes a number of potential security risks, such as “HTTP Errors,” “DNS Errors,” “Cloud Email,” “NFS Activity,” and “Threat List Activity.” Events that match one of the listed potential security risks may be represented as bars within a time interval represented by the horizontal dimension of the dashboard. For example, a security risk <b>1752</b> may be shown as a series of bars clustered around an interval of time under “DNS Errors” in portion <b>1758</b>. The presence of security risk <b>1752</b> in portion <b>1758</b> may indicate that the trigger condition for creating the ephemeral event stream includes a potential security risk <b>1752</b>, as discovered using portion <b>1758</b> in the “Enterprise Security” application. To enable identification of the trigger condition, portion <b>1758</b> may replicate the timescale and data (e.g., security risk <b>1752</b>) seen by the user at the time at which the ephemeral event stream was created using the “Enterprise Security” application.

Below portion <b>1758</b>, the GUI may display additional creation information <b>1754</b> describing the creator of the ephemeral event stream. For example, creation information <b>1754</b> may include a start time (e.g., “2014/09/01 12:00:00”), duration (e.g., “7 days”), and/or protocol (e.g., “HTTP”) associated with network data capture by the ephemeral event stream. Creation information <b>1754</b> may describe a capture trigger for generating additional time-series event data based on identified security risks from portion <b>1758</b>. For example, creation information <b>1754</b> may be submitted through one or more user-interface elements shown below portion <b>1758</b> in the “Enterprise Security” application to trigger the capture of additional time-series event data in response to security risk <b>1752</b>. After creation information <b>1754</b> is submitted to the GUI, the information may be used to configure the generation of the ephemeral event stream at one or more remote capture agents.

The GUI may also include a hyperlink <b>1756</b> (e.g., “Go to Stream Configuration”) that navigates the user back to event stream information for the ephemeral event stream. For example, the user may select hyperlink <b>1756</b> to view the event stream information within the GUI of <figref idref="DRAWINGS">FIG. 17C</figref>. Hyperlinks in the GUIs of <figref idref="DRAWINGS">FIGS. 17C-17D</figref> may thus provide a mechanism for bidirectional navigation between the event stream information and the creation information. Such bidirectional linking may allow the user to establish the context for creating the ephemeral event stream as well as the current state of the ephemeral event stream, thus improving analysis, understanding, and management of ephemeral event streams from multiple disparate creators.

<figref idref="DRAWINGS">FIG. 18</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 18</figref> shows a flowchart of grouping and managing event streams generated from captured network data. In one or more embodiments, one or more of the steps may be omitted, repeated, and/or performed in a different order. Accordingly, the specific arrangement of steps shown in <figref idref="DRAWINGS">FIG. 18</figref> should not be construed as limiting the scope of the embodiments.

Initially, a GUI is provided on a computer system for configuring the generation of time-series event data from network packets captured by one or more remote capture agents (operation <b>1802</b>). The GUI may include a number of user-interface elements for streamlining the creation, management, and/or update of the configuration information.

First, the GUI may provide a set of user-interface elements for specifying a grouping of a set of event streams containing time-series event data by an event stream attribute (operation <b>1804</b>). For example, the GUI may enable grouping of the event streams by a category (e.g., web, infrastructure, networking, file transfer, email, messaging, authentication, database, telephony, network management, user-created value, etc.) and/or a protocol used by the network packets (e.g., transport layer protocol, session layer protocol, presentation layer protocol, application layer protocol). The GUI may also enable grouping of the event streams by applications used to create the event streams (e.g., based on application name) and/or event stream lifecycles of the event stream (e.g., permanent or ephemeral).

Next, the GUI may display a set of user-interface elements containing event stream information for one or more subsets of the event streams represented by the grouping of the event streams by the event stream attribute (operation <b>1806</b>). Grouping of displayed event stream information by event stream attributes is described in further detail below with respect to <figref idref="DRAWINGS">FIG. 19</figref>.

Finally, the GUI may provide a set of user-interface elements for managing the event streams (operation <b>1808</b>). For example, the GUI may be used to clone a new event stream from an existing event stream, create an event stream, delete an event stream, enable an event stream, disable an event stream, and/or modify an end time of an ephemeral event stream, as discussed above with respect to <figref idref="DRAWINGS">FIGS. 17A-17E</figref>.

<figref idref="DRAWINGS">FIG. 19</figref> shows a flowchart illustrating the process of displaying event stream information represented by a grouping of the event streams by an event stream attribute in accordance with the disclosed embodiments. In one or more embodiments, one or more of the steps may be omitted, repeated, and/or performed in a different order. Accordingly, the specific arrangement of steps shown in <figref idref="DRAWINGS">FIG. 19</figref> should not be construed as limiting the scope of the embodiments.

First, one or more values of an event stream attribute are displayed (operation <b>1902</b>), and displayed event stream information is grouped into one or more subsets of the event streams based on the value(s) of the event stream attribute (operation <b>1904</b>). For example, a user may specify the type of event stream attribute to group by, and a GUI may display one or more categories, protocols, application names, and/or other values of the event stream attribute in a list. After a given value of the event stream attribute is selected from the list, the GUI may show event stream information matching the selected value in a table next to the list.

The event stream information may be grouped by an additional event stream attribute (operation <b>1906</b>). If the event stream information is not to be grouped by an additional event stream attribute, grouping of the displayed event stream information by the first event stream attribute is maintained.

If the event stream information is to be grouped by an additional event stream attribute, one or more values of the additional event stream attribute are displayed (operation <b>1902</b>), and the displayed event stream information is further grouped into one or more additional subsets of the event streams based on the value(s) of the additional event stream attribute (operation <b>1904</b>). Continuing with the above example, the event stream information in the table may additionally be grouped and/or filtered by an event stream lifecycle of the event streams, which may be permanent or ephemeral. If a permanent event stream lifecycle is selected (e.g., through the GUI), event stream information for permanent event streams that match the value of the first event stream attribute (e.g., category, protocol, application) is shown. Such event stream information may include a name, a type, a protocol, an application, a description, a status, and/or a graph of a metric associated with the time-series event data of the event streams. If an ephemeral event stream lifecycle is selected, event stream information for ephemeral event streams that match the value of the first event stream attribute is shown. Such event stream information may include a name, a number of event streams, an application, a start time, an end time, a time remaining, and/or a status.

The displayed event stream information may continue to be grouped by additional event stream attributes (operation <b>1906</b>) to further facilitate the creation, search, and/or management of event streams across multiple applications, categories, protocols, and/or other event stream attributes. For example, the displayed event stream information may be grouped by category, protocol, keyword, and/or event stream lifecycle to allow a user to find event streams associated with a given category, protocol, keyword, and/or event stream lifecycle. For each event stream attribute by which the event stream information is to be grouped, one or more values of the event stream attribute are displayed (operation <b>1902</b>). The displayed event stream information, which may already be grouped by one or more other event stream attributes, is then further grouped or filtered into one or more subsets of the event streams based on the value(s) of the event stream attribute (operation <b>1904</b>). Grouping of displayed event stream information by values of event stream attributes may continue until the displayed event stream information has been grouped by values for all relevant event stream attributes.

<figref idref="DRAWINGS">FIG. 20</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 20</figref> shows a flowchart of providing inline visualizations of metrics related to captured network data. In one or more embodiments, one or more of the steps may be omitted, repeated, and/or performed in a different order. Accordingly, the specific arrangement of steps shown in <figref idref="DRAWINGS">FIG. 20</figref> should not be construed as limiting the scope of the embodiments.

Initially, a set of event streams is obtained from one or more remote capture agents over one or more networks (operation <b>2002</b>). The event streams may include time-series event data generated from network packets captured by the remote capture agent(s). Next, event stream information for each event stream and a graph of a metric associated with the time-series event data in the event stream are displayed within a GUI on a computer system (operation <b>2004</b>). The graph may include a sparkline, bar graph, line chart, histogram, and/or other type of visualization of the metric that is shown in line with the event stream information. The metric may include network traffic, a number of events, and/or a number of notable events (e.g., security risks).

A subset of event streams associated with a grouping of the event streams by an event stream attribute is also obtained (operation <b>2006</b>). For example, the subset of event streams may match a value of one or more event stream attributes. Alternatively, the subset of event streams may include all event streams if the event streams are matched to all possible values of the event stream attribute(s). Next, the metric is aggregated across the subset of the event streams (operation <b>2008</b>), and a graph of the aggregated metric across the event streams is displayed within the GUI (operation <b>2010</b>). For example, the metric may be aggregated as a sum, average, and/or other summary statistic, and the graph of the aggregated metric may include a sparkline and/or other visual representation of the aggregated metric over time and/or another dimension.

While the graphs for individual event streams and the aggregated metric across the event streams are displayed, the graphs are updated in real-time with time-series event data from the remote capture agent(s) (operation <b>2012</b>). For example, sparklines representing individual and aggregate network traffic over time may “advance” to reflect newly received time-series event data from the remote capture agent(s). The graph(s) are also updated with the value of the metrics or aggregated metric based on the position of a cursor over the graph(s) (operation <b>2014</b>). For example, the numeric value of a metric (e.g., network traffic, number of events, number of notable events, etc.) at a given point in time may be displayed in response to the positioning of a cursor over that point in time in the graph.

Finally, event stream information for the subset of the event streams is displayed (operation <b>2016</b>). For example, the event stream information may be displayed in a table, and the graphs for individual event streams may be shown in a column of the table. The graph of the aggregated metric may be displayed in a different part of the GUI, and the graphs may be updated based on the event streams and/or groupings shown in the table.

<figref idref="DRAWINGS">FIG. 21</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 21</figref> shows a flowchart of managing ephemeral event streams generated from captured network data. In one or more embodiments, one or more of the steps may be omitted, repeated, and/or performed in a different order. Accordingly, the specific arrangement of steps shown in <figref idref="DRAWINGS">FIG. 21</figref> should not be construed as limiting the scope of the embodiments.

First, a GUI is provided on a computer system for obtaining configuration information for configuring the generation of time-series event data from network packets captured by one or more remote capture agents (operation <b>2102</b>). Next, a subset of one or more ephemeral event streams associated with a grouping of the ephemeral event stream(s) by an event stream attribute is obtained (operation <b>2104</b>). For example, the subset of ephemeral event stream(s) may be associated with a grouping of the ephemeral event streams by one or more categories, applications and/or protocols. The ephemeral event streams may be used to temporarily generate time-series event data from network packets captured by the remote capture agent(s).

The GUI is used to display event stream information for the ephemeral event stream(s) (operation <b>2106</b>), along with a set of user-interface elements for managing the ephemeral event stream(s) (operation <b>2108</b>). The user-interface elements may be used to disable an ephemeral event stream, delete an ephemeral event stream, and/or modify an end time for terminating the ephemeral event stream. The event stream information may include a name, number of event streams, application, start time, end time, time remaining, and/or status.

The GUI also includes a set of user-interface elements for creating an ephemeral event stream (operation <b>2110</b>), as well as a mechanism for applying an action associated with managing the ephemeral event stream(s) to a set of selected ephemeral event streams (operation <b>2112</b>). For example, the GUI may enable the creation of an ephemeral event stream as a copy (e.g., clone) of an existing ephemeral event stream. The GUI may also allow multiple ephemeral event streams to be enabled, disabled, and/or deleted.

The configuration information is updated based on input received through the GUI (operation <b>2114</b>) and provided over the network to the remote capture agent(s) (operation <b>2116</b>). The configuration information may then be used to configure the generation of the time-series event data at the remote capture agent(s) during runtime of the remote capture agent(s). For example, the configuration information may be used to create, delete, enable, disable, and/or modify the end times of one or more ephemeral event streams.

<figref idref="DRAWINGS">FIG. 22</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 22</figref> shows a flowchart of bidirectional linking of ephemeral event streams to creators of the ephemeral event streams. In one or more embodiments, one or more of the steps may be omitted, repeated, and/or performed in a different order. Accordingly, the specific arrangement of steps shown in <figref idref="DRAWINGS">FIG. 22</figref> should not be construed as limiting the scope of the embodiments.

First, a GUI is provided on a computer system for obtaining configuration information for configuring the generation of time-series event data from network packets captured by one or more remote capture agents (operation <b>2202</b>). Next, a subset of one or more ephemeral event streams associated with a grouping of the ephemeral event stream(s) by an event stream attribute is obtained (operation <b>2204</b>), and event stream information for the ephemeral event stream(s) is displayed in a GUI (operation <b>2206</b>), as described above.

The GUI is used to provide a mechanism for navigating between the event stream information and creation information for one or more creators of the ephemeral event stream(s) (operation <b>2208</b>). The mechanism may include a hyperlink from the event stream information to the creation information for a creator of an ephemeral event stream and/or a hyperlink from the creation information back to the event stream information. The creation information may include a creator name, a protocol, a duration of the ephemeral event stream, and/or a trigger condition for activating the ephemeral event stream. Creators of ephemeral event streams may include applications for monitoring network traffic captured by the remote capture agent(s) and/or capture triggers for generating additional time-series event data from network packets on the remote capture agent(s) based on a security risk.

The GUI also includes a set of user-interface elements containing the time-series event data (operation <b>2210</b>). For example, the GUI may show individual events and the associated timestamps, graphs of metrics associated with the events, and/or other representations of events in the ephemeral event stream(s). Consequently, the GUI may facilitate understanding and analysis of both the content and context of the ephemeral event streams.

<figref idref="DRAWINGS">FIG. 23A</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 23A</figref> shows a screenshot of a GUI, such as GUI <b>1025</b> of <figref idref="DRAWINGS">FIG. 10</figref>. As described above, the GUI may be used to obtain configuration information that is used to configure the generation of event streams containing time-series event data at one or more remote capture agents distributed across a network.

Like the GUI of <figref idref="DRAWINGS">FIGS. 17A-17B</figref>, the GUI of <figref idref="DRAWINGS">FIG. 23A</figref> may include a table with a set of columns <b>2308</b>-<b>2324</b>. Event streams in the table may be sorted by information in columns <b>2308</b>-<b>2324</b> of the table, other attributes of the event streams, and/or randomly. For example, the user may select the column header of a given column (e.g., columns <b>2308</b>-<b>2324</b>) to order the displayed event stream information by the information represented by the column. Alternatively, event streams in the table may be randomly sorted and/or sorted by an attribute that is not explicitly displayed in a column of the table.

Column <b>2308</b> may include a series of checkboxes that can be used to select one or more event streams to which an action is to be applied using a “Bulk Edit” mechanism, as described above with respect to <figref idref="DRAWINGS">FIG. 17A</figref>. Columns <b>2310</b>-<b>2318</b> may include high-level event stream information related to event streams that are created and/or managed using the configuration information. Column <b>2310</b> may show a list of names of the event streams, column <b>2312</b> may specify a type indicating whether each event stream is cloned from an existing event stream, and column <b>2314</b> may specify a protocol associated with each event stream. Column <b>2316</b> may show an application associated with each event stream, column <b>2318</b> may provide a description of each event stream, and column <b>2320</b> may include inline graphs (e.g., sparklines) of metrics associated with the event streams. The GUI may also include a user-interface element <b>2304</b> that shows an aggregated value of the metric in the graphs of column <b>2320</b>, as well as a numeric value of the aggregate metric (e.g., “˜317.4 MB/s”) over the time spanned by the sparkline.

Column <b>2322</b> may specify a status of each event stream, and column <b>2324</b> may be used by a user to perform one or more actions on individual event streams. As shown in FIG. <b>23</b>A, column <b>2322</b> indicates a status of “Enabled” for the “mysql” event stream, “Stats-only” for the “Stream_sip” event stream, and “Disabled” for all other event streams.

Within the GUI, the user-interface element in the second row of column <b>2324</b> may be selected. As a result, a drop-down menu may be displayed below the user-interface element with a set of options, including “Stats Only,” “Enable,” and “Clone.” The user may select the “Enable” option to enable generation of the corresponding event stream from network data and the “Clone” option to create a new event stream as a copy of the event stream. The user may select the “Stats Only” option to enable the generation of a set of statistics from the event stream without requiring the subsequent storage and processing (e.g., indexing) of the event stream. If the “Stats Only” option is selected, network data related to the event stream may be captured by one or more remote capture agents, and the statistics may be generated from the captured network data and displayed within the GUI without indexing or storing the event stream, as described in further detail below with respect to <figref idref="DRAWINGS">FIGS. 24A-24E</figref>. Alternatively, partial indexing of the event stream in “Stats Only” mode may be configured using another screen of the GUI, as described below with respect to <figref idref="DRAWINGS">FIG. 23C</figref>.

<figref idref="DRAWINGS">FIG. 23B</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. The screenshot of <figref idref="DRAWINGS">FIG. 23B</figref> may be provided by a GUI for obtaining configuration information that is used to configure the generation of event streams containing time-series event data at one or more remote capture agents distributed across a network. More specifically, the GUI of <figref idref="DRAWINGS">FIG. 23B</figref> includes a screen for specifying configuration information for configuring the generation of time-series event data in an event stream named “Stream_sip.” For example, the screen of <figref idref="DRAWINGS">FIG. 23B</figref> may be shown after a user selects the first row of column <b>2310</b> in the GUI of <figref idref="DRAWINGS">FIG. 23A</figref>.

Like the GUI of <figref idref="DRAWINGS">FIG. 11B</figref>, the GUI of <figref idref="DRAWINGS">FIG. 23B</figref> includes a table. Each row in the table may represent an event attribute that is eligible for inclusion in the event stream. For example, an event attribute may be included in the table if the event attribute can be obtained from network packets that include the protocol of the event stream. Columns <b>2332</b>-<b>2342</b> of the table may allow the user to use the event attributes to generate time-series event data that is included the event stream. First, column <b>2332</b> includes a series of checkboxes that allows the user to include individual event attributes in the event stream or exclude the event attributes from the event stream. If a checkbox is checked, the corresponding event attribute is added to the event stream, and the row representing the event attribute is shown with other included event attributes in an alphabetized list at the top of the table. If a checkbox is not checked, the corresponding event attribute is omitted from the event stream, and the row representing the event attribute is shown with other excluded event attributes in an alphabetized list following the list of included event attributes. Those skilled in the art will appreciate that the GUI may utilize other sortings and/or rankings of event attributes in columns <b>2332</b>-<b>2342</b>.

Columns <b>2334</b>-<b>2340</b> may provide information related to the event attributes. Column <b>2334</b> may show the names of the event attributes, column <b>2336</b> may provide a description of each event attribute, column <b>2338</b> may provide a type of each the event attribute (e.g., “Original” or “Extracted” using an extraction rule), and column <b>2340</b> may provide a term representing the event attribute. In other words, columns <b>2334</b>-<b>2340</b> may allow the user to identify the event attributes and decide whether the event attributes should be included in the event stream.

Column <b>2342</b> may be used to apply one or more actions to the event attributes represented by the rows of the table. The user may select each row in column <b>2342</b> to set a filter and/or create an extraction rule for the corresponding event attribute. The filter and/or extraction rule may then be used in the generation, processing, and/or querying of the event stream.

The GUI of <figref idref="DRAWINGS">FIG. 23B</figref> also includes a set of user-interface elements <b>2344</b>-<b>2348</b> for managing the event stream. The user may select user-interface element <b>2344</b> (e.g., “Enabled”) to enable generation of the event stream from network data and user-interface element <b>2346</b> (e.g., “Disabled”) to disable the generation of the event stream from the network data. The user may select user-interface element <b>2348</b> (e.g., “Stats Only”) to enable the generation of a set of statistics from the event stream without requiring the subsequent storage and processing the event stream by indexers, data stores, and/or other components on the network. The GUI of <figref idref="DRAWINGS">FIG. 23B</figref> may thus provide a mechanism for generating a set of statistics from an un-indexed and/or partially indexed event stream, in lieu of or in addition to drop-down menu <b>2302</b> in the GUI of <figref idref="DRAWINGS">FIG. 23A</figref>.

<figref idref="DRAWINGS">FIG. 23C</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. As mentioned above, the screenshot of <figref idref="DRAWINGS">FIG. 23C</figref> may be provided by a GUI (e.g., GUI <b>1025</b> of <figref idref="DRAWINGS">FIG. 10</figref>) that is used to manage an event stream. More specifically, the GUI of <figref idref="DRAWINGS">FIG. 23C</figref> may be used to configure the selective indexing of an event stream after the event stream is placed into a “stats only” mode. For example, the GUI of <figref idref="DRAWINGS">FIG. 23C</figref> may be shown after the “Stats Only” option is selected for the “tcp” stream using the GUI of <figref idref="DRAWINGS">FIG. 23A</figref> or <figref idref="DRAWINGS">FIG. 23B</figref>.

As shown in <figref idref="DRAWINGS">FIG. 23C</figref>, the GUI includes a portion <b>2350</b> that contains information and/or statistics associated with the “tcp” event stream. For example, portion <b>2350</b> may specify the name of the event stream and include a total number of events (e.g., “15764”), which represents the number of events generated from network data associated with the event stream over a given period. Portion <b>2350</b> may also include a total incoming traffic in MB (e.g., “6.31”), a total outgoing traffic in MB (e.g., “44.13”), and a total traffic in MB (e.g., “50.44”) for the network data over the same period. Finally, portion <b>2350</b> may specify an index volume in MB (e.g., “17.52”) for the event stream, which represents the estimated indexed size of the 15764 events in the event stream.

The statistics in portion <b>2350</b> may be generated without indexing and storing the entirety of the event stream. For example, the statistics may be generated from the event stream by one or more remote capture agents and/or configuration servers. As a result, some or all of the event stream may be omitted from subsequent storage and processing by forwarders, indexers, data stores, and/or other network components.

Next, the GUI includes a portion <b>2352</b> that is used to obtain a user setting for the amount to index in the event stream. Portion <b>2352</b> includes a slider that allows the user to specify a percentage (e.g., “40%”) and/or amount (e.g., “7 MB”) of the event stream to index. The user may move the slider to the right to increase the percentage up to a maximum of 100% (e.g., 17.52 MB), or the user may move the slider to the left to decrease the percentage down to a minimum of 0% (e.g., 0 MB).

Portion <b>2352</b> is followed by information <b>2354</b> related to an unused storage limit associated with the time-series event data in the event stream. For example, information <b>2354</b> may provide the storage limit as a “daily index volume limit” (e.g., “1 GB”) for a given user account or license, along with an average unused portion of the daily storage limit (e.g., “25 MB”) that can be used to index other data. Information <b>2354</b> is followed by a suggestion <b>2356</b> for setting the percentage of the event stream to store (e.g., index) based on the set of statistics in portion <b>2352</b>, a historical trend associated with the statistics, and/or the storage limit information <b>2352</b>. For example, suggestion <b>2356</b> may recommend indexing up to 85% of the “tcp” event stream based on fluctuations in the index volume of the event stream and the average unused index volume limit associated with a given license.

Suggestion <b>2356</b> may be followed by information <b>2358</b> that provides a price associated with capturing the event stream above the storage limit. For example, information <b>2358</b> may indicate an increase of $1800 a year for increasing the daily index volume limit to 2 GB and an increase of $8200 a year for increasing the daily index volume limit to 10 GB.

The GUI may additionally provide a set of options <b>2360</b>-<b>2374</b> for configuring the selective indexing of the event stream. Options <b>2360</b>-<b>2374</b> may be associated with checkboxes in the GUI. The user may enable an option by selecting the corresponding checkbox and disable the option by deselecting the checkbox.

Option <b>2360</b> may enable the automatic adjustment of the indexing of the event stream based on the average unused daily index volume limit. If option <b>2360</b> is enabled, the percentage specified in the slider of portion <b>2352</b> may be ignored, and configuration information associated with the event stream may be updated to automatically adjust the indexing of the event stream to stay within the daily unused index volume limit.

Option <b>2362</b> may enable indexing of the entire event stream during high traffic volume, and option <b>2364</b> may enable indexing of a sample of the event stream during high traffic volume. Conversely, option <b>2366</b> may enable indexing of the entire event stream during low traffic volume. Options <b>2362</b>-<b>2374</b> may thus be used to fine-tune the indexing of one or more portions of the event stream during various network traffic conditions associated with the event stream.

Option <b>2368</b> may enable indexing of the event stream during a daily period that is defined by a user-editable start time (e.g., “6:30 pm”) and end time (e.g., “7:30 pm”). The user may edit the start and end times by interacting with a text box, drop-down menu, and/or other user-interface elements in which the start and end times are displayed.

Option <b>2370</b> may enable indexing of the event stream during predefined public holidays and retail shopping days. For example, option <b>2370</b> may be selected to trigger indexing of the event stream during days such as Thanksgiving, Christmas, Black Friday, Cyber Monday, and/or other days associated with deviations from standard user and/or network traffic patterns.

Option <b>2372</b> may enable indexing of the event stream during user-editable custom days (e.g., “January 31, April 30, July 31, October 31”). The user may add edit the list of custom days by interacting with a text box, drop-down menu, and/or other user-interface element in which the custom days are displayed.

Option <b>2374</b> may enable indexing of the event stream based on compliance standards such as Payment Card Industry (PCI) data security standards, Health Insurance Portability and Accountability Act (HIPAA) requirements, and/or Sarbanes-Oxley rules. The user may select a user-interface element <b>2378</b> (e.g., “compliance standards”) associated with option <b>2374</b> to navigate to a different screen of the GUI for configuring compliance-based indexing of the event stream. For example, the user may select user-interface element <b>2378</b> to reach a screen that allows the user to select one or more standards with which to comply and/or specify one or more associated with compliance with the selected standards during indexing of the event stream.

After the user has finished setting options associated with selective indexing of the event stream, the user may select a user-interface element <b>2382</b> (e.g., “Save”) to commit the options to configuration information that is used by remote capture agents, configuration servers, forwarders, indexers, data stores, and/or other distributed network components to capture and subsequently process the event stream. Conversely, the user may select a user-interface element <b>2380</b> (e.g., “Cancel”) to exit the screen of <figref idref="DRAWINGS">FIG. 23C</figref> without saving the options in the configuration information.

<figref idref="DRAWINGS">FIG. 24A</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. In particular, <figref idref="DRAWINGS">FIG. 24A</figref> shows a screenshot of a GUI for obtaining configuration information that is used to configure the generation of event streams containing time-series event data at one or more remote capture agents distributed across a network, such as GUI <b>1025</b> of <figref idref="DRAWINGS">FIG. 10</figref>.

The GUI of <figref idref="DRAWINGS">FIG. 24A</figref> includes a table of statistics for a set of event streams. As described above, the statistics may be generated from the event streams independently of subsequent storage and processing of the event streams by forwarders, indexers, data stores, and/or other components on a network, such as network <b>801</b> of <figref idref="DRAWINGS">FIG. 8</figref>.

The table includes a set of columns <b>2406</b>-<b>2416</b>. Each column <b>2406</b>-<b>2416</b> may provide a different statistic for event streams represented by rows in the table, and rows in the table may be sorted by different column values by selecting the corresponding column headers (e.g. “Stream id,” “Total Events,” “Total Incoming Traffic (MB),” “Total Outgoing Traffic (MB),” “Total Traffic (MB),” “Splunk Index Volume (MB)”) of columns <b>2406</b>-<b>2416</b>. Alternatively, rows in the table may be sorted by other attributes and/or randomly. Elements of rows in the table may also be selected to navigate to a different screen containing additional information related to the event streams and/or statistics, as described in further detail below with respect to <figref idref="DRAWINGS">FIG. 24E</figref>.

Column <b>2406</b> may show a list of names of the event streams, such as “tcp,” “http,” and “mysql.” Column <b>2408</b> may show a total number of events collected from each event stream over a pre-specified time period, column <b>2410</b> may show a total incoming traffic for the event stream over the same time period, and column <b>2412</b> may show a total outgoing traffic for the event stream over the same time period. Column <b>2414</b> may show a total traffic (e.g., incoming and outgoing combined) for the event stream over the time period, and column <b>2416</b> may show an index volume of the event stream over the time period. Columns <b>2408</b>-<b>2414</b> may thus represent statistics for network traffic associated with each event stream, while column <b>2416</b> may provide index volumes related to the potential indexing of time-series event data in the event stream after the time-series event data is extracted from the network traffic.

The GUI of <figref idref="DRAWINGS">FIG. 24A</figref> also includes two graphs <b>2402</b>-<b>2404</b>. Graph <b>2402</b> may be a bar chart of index volumes of event streams in the table. Each bar in the bar chart may contain one or more segments representing the index volume of a corresponding event stream over a given time range. As shown in <figref idref="DRAWINGS">FIG. 24A</figref>, bars in the bar chart may be divided into different colored segments, which represent different event streams collected over the corresponding time ranges. The height of each bar in the bar chart may represent a total index volume over a minute, and each colored segment in the bar chart may represent the contribution of the corresponding event stream to the total index volume. As a result, longer segments in a bar may take up more of the total index volume, while shorter segments in the bar may take up less of the total index volume.

The bar chart also includes a legend <b>2422</b> that maps colors in the bar chart to specific event streams. Legend <b>2422</b> may thus allow a user to identify event streams represented by different colored segments in the bar chart. As discussed below with respect to <figref idref="DRAWINGS">FIGS. 24B-24C</figref>, the appearance of the bar chart may also change based on the position of a cursor over the bar chart and/or legend <b>2422</b>.

Graph <b>2404</b> may be a pie chart of index volume across the event streams. Each colored “slice” of the pie chart may represent the proportion of the total index volume occupied by the corresponding event stream over a given period. For example, the pie chart of <figref idref="DRAWINGS">FIG. 24A</figref> may indicate that the “http” and “tcp” event streams occupy relatively larger portions of the total index volume than other event streams over the same period.

Graph <b>2404</b> also includes a user-interface element <b>2424</b> that displays a number of values of an index volume of the “tcp” event stream. The appearance and/or location of user-interface element <b>2424</b> may change based on the position of the cursor over the pie chart. For example, user-interface element <b>2424</b> may be shown when the cursor is positioned over the slice of the pie chart that represents the “tcp” event stream. As the cursor is moved over other slices of the pie chart, the position of user-interface element <b>2424</b> may shift to be adjacent to the slice over which the cursor is currently positioned. At the same time, values in user-interface element <b>2424</b> may be updated to reflect statistics associated with the corresponding slice of the pie chart.

More specifically, user-interface element <b>2424</b> may include the name of the corresponding event stream (e.g., “tcp”), an amount of data associated with the index volume of the event stream in MB (e.g., “417.96”), and a percentage of a total index volume associated with the event stream (e.g., “35.642%”). As the cursor is positioned over other slices in the pie chart, user-interface element <b>2424</b> may be updated with the names, index volume data amounts, and index volume percentages of the corresponding event streams. Consequently, the appearance of the pie chart may change with the location of the cursor within graph <b>2404</b>.

The user may provide other types of user input to graph <b>2404</b> and/or user-interface element <b>2424</b> to further update the appearance of the GUI. For example, the user may click or double-click on user-interface element <b>2424</b> to navigate to a screen of the user interface that displays a search term, event streams, and/or additional data associated with the “tcp” event stream and/or other event streams, such as the screen of <figref idref="DRAWINGS">FIG. 24E</figref>. Other areas of the GUI of <figref idref="DRAWINGS">FIG. 24A</figref> may be updated based on user input associated with user-interface element <b>2424</b>. For example, the portions (e.g., bar segments) of graph <b>2402</b> representing the “tcp” event stream in graph <b>2402</b> may be highlighted, and other segments of graph <b>2402</b> may be dimmed, in response to the position of cursor over user-interface element <b>2424</b> and/or the clicking or double-clicking of user-interface element <b>2424</b>. In another example, statistics and/or other data associated with the “tcp” event stream may be displayed in one or both graphs <b>2402</b>-<b>2404</b> in response to user input associated with user-interface element <b>2424</b>.

The GUI additionally includes a number of user-interface elements <b>2418</b>-<b>2420</b> for changing a view of graphs <b>2402</b>-<b>2404</b>. User-interface element <b>2418</b> may allow the user to change the scale and/or time range associated with data in graphs <b>2402</b>-<b>2404</b>. For example, user-interface element <b>2418</b> may allow the user to select a pre-specified, relative, real-time, absolute, and/or custom time period spanned by the index volumes shown in graphs <b>2402</b>-<b>2404</b>. Selection of time ranges associated with graphs of statistics associated with event streams is described in further detail below with respect to <figref idref="DRAWINGS">FIG. 24D</figref>.

User-interface element <b>2420</b> may be used to filter statistics in the table and graphs <b>2402</b>-<b>2404</b> by a host from which the event streams are collected. For example, the user may select user-interface element <b>2420</b> to access a drop-down menu that contains a list of hosts from which the event streams are captured. The user may select one or more of the hosts to update columns <b>2406</b>-<b>2416</b> in the table and graphs <b>2402</b>-<b>2404</b> with statistics associated with the hosts, or the user may select an “all” option to display statistics associated with all hosts from which network data is captured. The user may also search for hosts by name using a form field in the drop-down menu.

Graphs <b>2402</b>-<b>2404</b> may be updated in real-time with time-series event data as the time-series event data is received from one more remote capture agents. For example, the bar chart of graph <b>2402</b> may shift as a real-time time range specified in user-interface element <b>2418</b> advances and additional time-series event data is collected within the time range. Similarly, slices in the pie chart of graph <b>2404</b> may change size to reflect changes in the proportions of index volumes in the event streams within the time range.

<figref idref="DRAWINGS">FIG. 24B</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 24B</figref> shows the GUI of <figref idref="DRAWINGS">FIG. 24A</figref> after the cursor is positioned over a portion <b>2426</b> of legend <b>2422</b> in graph <b>2402</b>. Portion <b>2426</b> may map the “mysql” event stream to a particular color (e.g., red). As a result, portion <b>2426</b> may indicate that all red segments in graph <b>2402</b> correspond to the contribution of the “mysql” event stream to the total index volume for bars in graph <b>2402</b>.

In response to the position of the cursor over portion <b>2426</b>, the appearance of bar chart may be changed so that the red segments in bars of the bar chart are highlighted and other segments corresponding to other portions of legend <b>2422</b> are dimmed. By positioning the cursor over portion <b>2426</b>, the user may identify the locations and/or relative sizes of segments representing the “mysql” event stream in the bar chart. In turn, the user may be able to analyze trends and/or patterns associated with the “mysql” event stream using the highlighted segments.

The user may provide other types of user input to portion <b>2426</b> to further update the appearance of the GUI. For example, the user may click or double-click on portion <b>2426</b> to navigate to a screen of the user interface that displays a search term, event streams, and/or additional data associated with the “mysql” event stream and/or other event streams, such as the screen of <figref idref="DRAWINGS">FIG. 24E</figref>. Other areas of the GUI of <figref idref="DRAWINGS">FIG. 24B</figref> may be updated based on user input associated with portion <b>2426</b>. For example, the slice representing the “mysql” event stream in graph <b>2404</b> may be highlighted, and other slices of graph <b>2404</b> may be dimmed, in response to the position of cursor over portion <b>2426</b> and/or the clicking or double-clicking of portion <b>2426</b>. In another example, statistics and/or other data associated with the “mysql” event stream may be displayed in one or both graphs <b>2402</b>-<b>2404</b> in response to user input associated with portion <b>2426</b>.

The user may change the highlighting and/or dimming of the bar chart by positioning the cursor over other portions of legend <b>2426</b>. As the cursor is placed over a given portion of legend <b>2426</b>, colored segments in the bar chart represented by the portion may be highlighted, and other segments in the bar chart may be dimmed. On the other hand, if the cursor is positioned over a colored segment in a bar of the bar chart instead of legend <b>2422</b>, the appearance of the bar chart may be updated differently, as described in further detail below with respect to <figref idref="DRAWINGS">FIG. 24C</figref>.

<figref idref="DRAWINGS">FIG. 24C</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. In particular, <figref idref="DRAWINGS">FIG. 24C</figref> shows the GUI of <figref idref="DRAWINGS">FIGS. 24A-24B</figref> after the cursor is positioned over a segment <b>2428</b> in a bar of the bar chart corresponding to graph <b>2402</b> instead of a portion of legend <b>2422</b>. Segment <b>2428</b> may represent the contribution of a given event stream (e.g., “http”) to the total index volume over a given time interval (e.g., one minute) in the bar chart.

The positioning of the cursor over segment <b>2428</b> may cause the segment to be highlighted and other segments in the bar chart to be dimmed. A portion of legend <b>2422</b> containing a mapping of the event stream represented by the segment to a color in the bar chart (e.g., yellow) may also be highlighted, while other portions of legend <b>2422</b> may be dimmed. In addition, the bar chart may be updated with a user-interface element <b>2446</b> that displays information related to the index volume of the “http” event stream. For example, user-interface element <b>2446</b> may display the index volume of the event stream (e.g., “0.16629”) represented by the highlighted segment in the bar chart in MB. User-interface element <b>2446</b> may also identify the time interval (e.g., “Apr. 2, 2015 12:44 PM”) during which the index volume represented by the highlighted segment was produced.

As with portion <b>2426</b> of <figref idref="DRAWINGS">FIG. 24B</figref>, additional user input to segment <b>2428</b> may result in changes to the appearance of the GUI. For example, the user may click or double-click on segment <b>2428</b> to navigate to a screen of the user interface that displays a search term, event streams, and/or additional data associated with the “http” event stream and/or segment <b>2428</b>, such as the screen of <figref idref="DRAWINGS">FIG. 24E</figref>. Other areas of the GUI of <figref idref="DRAWINGS">FIG. 24C</figref> may be updated based on user input associated with segment <b>2428</b>. For example, the slice representing the “http” event stream in graph <b>2404</b> may be highlighted, and other slices of graph <b>2404</b> may be dimmed, in response to the position of cursor over segment <b>2428</b> and/or the clicking or double-clicking of segment <b>2428</b>. In another example, statistics and/or other data associated with the “http” event stream and/or segment <b>2428</b> may be displayed in one or both graphs <b>2402</b>-<b>2404</b> in response to user input associated with portion <b>2426</b>.

The appearance and/or location of user-interface element <b>2446</b> and the bar chart may change based on the position of the cursor over the bar chart. For example, user-interface element <b>2446</b> may be shown next to a given segment in the bar chart over which the cursor is currently positioned. The segment and corresponding portion of legend <b>2422</b> may be highlighted, and other segments in the bar chart and/or other portions of legend <b>2422</b> may be dimmed. Values in user-interface element <b>2446</b> may also be updated to reflect the time interval, index volume, and/or other information or statistics associated with the segment.

A user may thus place the cursor over a given segment in the bar chart to obtain information and/or statistics associated with the segment. In turn, the information and/or statistics may provide the user with a more thorough understanding of index volumes of individual event streams a particular points in time.

<figref idref="DRAWINGS">FIG. 24D</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 24D</figref> shows the GUI of <figref idref="DRAWINGS">FIGS. 24A-24C</figref> after the user has selected user-interface element <b>2418</b>. In response to the selection of user-interface element <b>2418</b>, the GUI may display a drop-down menu containing a number of sections <b>2430</b>-<b>2440</b> for specifying a time range associated with the time-series event data and/or statistics used to generate graphs <b>2402</b>-<b>2404</b> and populate columns <b>2406</b>-<b>2416</b> in the table.

Section <b>2430</b> may include a number of preset time ranges spanned by data in graphs <b>2402</b>-<b>2404</b> and columns <b>2406</b>-<b>2416</b>. For example, section <b>2430</b> may include a number of user-interface elements for specifying real-time windows (e.g., 30 seconds, one minute, five minutes, 30 minutes, one hour), relative time ranges (e.g., same or previous minutes, hours, days, weeks, months, years), and/or all time. A user may select a preset time range listed under section <b>2430</b> to efficiently update graphs <b>2402</b>-<b>2404</b> and columns <b>2406</b>-<b>2416</b> with data spanning the time range.

Section <b>2432</b> may be used to define a relative time range for data in graphs <b>2402</b>-<b>2404</b> and columns <b>2406</b>-<b>2416</b>. For example, section <b>2432</b> may include a number of user-interface elements for defining a start time and an end time of the time range spanned by data in graphs <b>2402</b>-<b>2404</b> and columns <b>2406</b>-<b>2416</b>.

Section <b>2434</b> may be used to define a “Real-time” interval that includes the current time as an end time. For example, section <b>2434</b> may allow the user to specify a time window spanning a number of minutes, hours, days, weeks, and/or months before the current time. Unlike the preset times in section <b>2430</b>, section <b>2434</b> may allow the time window to span an arbitrary amount of time up to the current time.

Section <b>2436</b> may be used to define a date range representing the time range, and section <b>2438</b> may be used to define a date and time range representing the time range. For example, sections <b>2436</b>-<b>2438</b> may include user-interface elements that allow the user to specify dates (e.g., day, month, year) and/or timestamps representing the start and end times of the time range.

Finally, section <b>2440</b> may be used to define an advanced time range. For example, section <b>2440</b> may include a number of user-interface elements for entering values of the start and end times for the time range using one or more supported time notations, such as epoch time and/or relative time notation.

Once a time range is selected and/or defined using one or more sections <b>2430</b>-<b>2440</b> of user-interface element <b>2418</b>, graphs <b>2402</b>-<b>2404</b> and columns <b>2406</b>-<b>2416</b> may be updated with data spanning the time range. As a result, user-interface element <b>2418</b> may be used to change a view of graphs <b>2402</b>-<b>2404</b> and data in the table.

<figref idref="DRAWINGS">FIG. 24E</figref> shows an exemplary screenshot in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 24E</figref> shows the GUI of <figref idref="DRAWINGS">FIGS. 24A-24D</figref> after any element of the table is selected. In response to the selection, the GUI may navigate to a screen containing a user-interface element <b>2442</b> (e.g., a text box) that displays an editable search term for retrieving data associated with graphs <b>2402</b>-<b>2404</b> and columns <b>2406</b>-<b>2416</b> of the table. A user may manually modify the search term within user-interface element <b>2442</b> to change the data included in graphs <b>2402</b>-<b>2404</b> and columns <b>2406</b>-<b>2416</b>. For example, the user may modify the statistics, time range, event streams, and/or other information to be retrieved, generated, and/or visualized using the search term.

The GUI of <figref idref="DRAWINGS">FIG. 24E</figref> also includes a user-interface element <b>2444</b> (e.g., a drop-down menu) that is displayed next to the first element of column <b>2406</b> (e.g., “mysql”) in the table. User-interface element <b>2444</b> may be shown after the first element of column <b>2406</b> is selected by a user. User-interface element <b>2444</b> may include options that are used to configure the search term in user-interface element <b>2442</b> and, in turn, the data in columns <b>2406</b>-<b>2416</b> and/or graphs <b>2402</b>-<b>2404</b>. For example, user-interface element <b>2444</b> may include a first option (e.g., “View events”) that allows the user to search for and view events in the “mysql” event stream in a separate screen of the GUI, such as the screen of <figref idref="DRAWINGS">FIG. 11F</figref>. User-interface element <b>2444</b> may include a second option (e.g., “Other events”) that allows the user to search for and view events in other event streams but not the “mysql” event stream in a separate screen of the GUI. User-interface element <b>2444</b> may include a third option (e.g., “Exclude from results”) that excludes data associated with the “mysql” event stream from data shown in columns <b>2406</b>-<b>2416</b>. Finally, user-interface element <b>244</b> may include a fourth option (e.g., “New search”) that clears the search term in user-interface element <b>2442</b> and allows the user to specify a new search term using user-interface element <b>2442</b>.

Those skilled in the art will appreciate that the GUI of <figref idref="DRAWINGS">FIGS. 24A-24E</figref> may include other types of information, graphs, and/or variations on graphs <b>2402</b>-<b>2404</b>. First, the GUI may include line charts, histograms, scatter plots, timelines, and/or other visualizations of statistics associated with the capture and/or indexing of event streams. Second, the GUI may change the views and/or appearances of graphs <b>2402</b>-<b>2404</b> in other ways. For example, the GUI may show segments in the bar chart corresponding to graph <b>2402</b> in a side-by-side fashion within each time interval instead of stacking the segments on top of one another to form a single bar in the time interval. In another example, graphs <b>2402</b>-<b>2404</b> may be highlighted, dimmed, and/or shown with different types of statistics based on cursor positions and/or other user input into the GUI. In yet another example, the GUI may include user-interface elements that allow the user to change the scale of one or both graphs <b>2402</b>-<b>2404</b> and/or perform additional filtering of data in graphs <b>2402</b>-<b>2404</b>. Third, the table and/or graphs <b>2402</b>-<b>2404</b> may include additional information, filters, and/or statistics related to the event streams. For example, the table may include a column that identifies each event stream as enabled or “stats only” and/or a column that identifies the percentage and/or amount of indexing of the event stream, up to an entirety of the event stream. In another example, the table may include columns that specify the number of packets in each event stream, the number of packets indexed in the event stream, and/or the number of packets not included in indexing of the event stream. In yet another example, data in graphs <b>2402</b>-<b>2404</b> may be filtered by capture mode (e.g., enabled, “stats only,” partially indexed), sorted by amount or percentage of indexing, and/or sorted by amount or percentage not indexed.

<figref idref="DRAWINGS">FIG. 25</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 25</figref> shows a flowchart of adjusting network capture based on event stream statistics through a GUI, such as GUI <b>1025</b> of <figref idref="DRAWINGS">FIG. 10</figref>. In one or more embodiments, one or more of the steps may be omitted, repeated, and/or performed in a different order. Accordingly, the specific arrangement of steps shown in <figref idref="DRAWINGS">FIG. 25</figref> should not be construed as limiting the scope of the embodiments.

First, a GUI for obtaining configuration information for configuring the generation of time-series event data from network packets captured by one or more remote capture agents is displayed on a computer system (operation <b>2502</b>). The GUI may include a number of user-interface elements for streamlining the creation, management, and/or update of the configuration information. Next, a set of statistics is generated in real-time with time-series event data from one or more remote capture agents (operation <b>2504</b>). The statistics may include a total number of events, a total incoming traffic, a total outgoing traffic, a total traffic, and/or an index volume for the time-series event data.

The statistics are also aggregated across a number of event streams (operation <b>2506</b>). For example, the statistics may be aggregated to obtain a total number of events, total incoming traffic, total outgoing traffic, total traffic, and/or total index volume for all event streams collected under a given license and/or user account. The statistics and/or aggregated statistics are then displayed in a set of user-interface elements within the GUI (operation <b>2508</b>). For example, the statistics and/or aggregated statistics may be shown in one or more tables or lists within the GUI. The statistics and/or aggregated statistics may be generated and/or displayed for event streams that are not subsequently stored, event streams that are subsequently partially stored, and/or event streams that are subsequently stored in entirety. The displayed statistics and/or aggregated statistics may also be filtered and/or sorted by the amount of storage of the event streams.

A set of user-interface for managing the event streams, including enabling the generation of the statistics from an event stream without subsequently storing and processing at least a first portion of the event stream by one or more components on a network, is also displayed in the GUI (operation <b>2510</b>). For example, the GUI may include a number of buttons, drop-down menus, and/or other user-interface elements that allow a user to enable, disable, or clone individual event streams. The user-interface elements may also allow the user to place each event stream in a “stats-only” mode, in which time-series event data in the event stream is collected and statistics are generated from the time-series event data without indexing and storing some or all of the time-series event data.

Another set of user-interface elements for storing at least a second portion of the event stream based on the statistics and/or a storage limit associated with the time-series event data is also provided in the GUI (operation <b>2512</b>). For example, the user-interface elements may allow the user to specify an amount and/or percentage of data to index in the event stream and/or enable a setting that automatically adjusts the indexing of the event stream based on the statistics and/or storage limit (e.g., storing the event stream if the size of the event stream and other event streams associated with the same license does not exceed the storage limit). The user-interface elements may also include options for indexing some or all of the event stream during high traffic volume, light traffic volume, and/or another condition associated with the event stream.

A suggestion for storing at least a second portion of the event stream based on the statistics and/or storage limit is additionally displayed in the GUI (operation <b>2514</b>), along with a price associated with storing the event stream above the storage limit (operation <b>2516</b>). For example, the GUI may suggest a maximum amount or percentage of the event stream to index to prevent the user from exceeding the storage limit (e.g., daily index volume limit) associated with a given license or user account. As an alternative or addition to the suggested maximum, the GUI may display one or more pricing options or plans for increasing the storage limit so that the entire event stream can be indexed.

The configuration information is updated based on input received through the GUI (operation <b>2518</b>) and provided over the network to the remote capture agent(s) (operation <b>2520</b>). The configuration information may then be used to configure the generation of the time-series event data at the remote capture agent(s) during runtime of the remote capture agent(s). For example, the configuration information may be used to generate statistics for a number of event streams without subsequently indexing the event streams, or the configuration information may be used to perform selective indexing of the event streams.

<figref idref="DRAWINGS">FIG. 26</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 26</figref> shows a flowchart of performing selective indexing of an event stream. In one or more embodiments, one or more of the steps may be omitted, repeated, and/or performed in a different order. Accordingly, the specific arrangement of steps shown in <figref idref="DRAWINGS">FIG. 26</figref> should not be construed as limiting the scope of the embodiments.

First, a GUI for obtaining configuration information for configuring the generation of time-series event data from network packets captured by one or more remote capture agents is displayed on a computer system (operation <b>2602</b>). The GUI may include a set of user-interface elements for managing one or more event streams containing the time-series event data, including selective indexing of the event streams.

Next, a set of statistics is generated from the time-series event data (operation <b>2604</b>). The statistics may include a total number of events, a total incoming traffic, a total outgoing traffic, a total traffic, and/or an index volume for the time-series event data in one or more of the event streams.

The configuration information is then updated to trigger the subsequent storage and processing of at least a portion of the event stream by one or more components on a network based on the statistics, a storage limit associated with the time-series event data, an index volume of the event stream, a historical trend associated with the statistics, and/or user input through the GUI (operation <b>2606</b>). For example, automatic adjustment of the indexing of an event stream may be enabled through the GUI. Next, the statistics may be periodically generated and used to establish the index volume of the event stream and/or a historical trend associated with the index volume. The index volume and/or historical trend may be compared with the unused storage limit of the time-series event data and used to determine an amount of the event stream that can be indexed and/or stored without exceeding the storage limit. The configuration information may then be updated to trigger the indexing and/or storage of the determined amount and/or an amount that is set by a user through the GUI.

<figref idref="DRAWINGS">FIG. 27</figref> shows a flowchart illustrating the process of facilitating the processing of network data in accordance with the disclosed embodiments. More specifically, <figref idref="DRAWINGS">FIG. 27</figref> shows a flowchart of providing visualizations of statistics associated with captured network data. In one or more embodiments, one or more of the steps may be omitted, repeated, and/or performed in a different order. Accordingly, the specific arrangement of steps shown in <figref idref="DRAWINGS">FIG. 27</figref> should not be construed as limiting the scope of the embodiments.

First, a GUI for obtaining configuration information for configuring the generation of time-series event data from network packets captured by one or more remote capture agents is displayed on a computer system (operation <b>2702</b>). Next, a set of user-interface elements containing statistics associated with one or more event streams containing the time-series event data is displayed in the GUI (operation <b>2704</b>). For example, the statistics may be displayed in a list, table, and/or other formatted text within the GUI. The statistics are sorted by an attribute associated with the set of statistics (operation <b>2706</b>). For example, rows in a table containing the statistics may be sorted by columns of that table that represent a name, a total number of events, a total incoming traffic, a total outgoing traffic, a total traffic, an index volume of the event streams, and/or the amount of storage of the event streams. Alternatively, rows in the table may reflect a different and/or random ordering of the event streams.

One or more graphs containing one or more values from the set of statistics is also displayed in the GUI (operation <b>2708</b>), along with a set of user-interface elements for changing a view of the graph(s) (operation <b>2710</b>). For example, the graph(s) may include a bar chart of index volumes of the event streams over time and/or a pie chart of index volume across the event streams. Bars in the bar chart may include segments representing the individual index volumes of event streams over given time intervals within a time range, and “slices” of the pie chart may represent the contributions of individual event streams to the total index volume over the time range. The bar and/or pie chart may be displayed with one or more user-interface elements for changing the scale, time range, and/or host associated with the statistics shown in the graph(s) and/or GUI.

The GUI further displays a value of a statistic and/or changes the appearance of the graph(s) based on a position of a cursor over the graph(s) and/or a legend associated with the graph(s) (operations <b>2712</b>-<b>2714</b>). Continuing with the above example, the GUI may display one or more values (e.g., amount of data associated with index volume, percentage of total index volume, etc.) of an index volume for a segment of a bar chart and/or a slice of a pie chart over which the cursor is positioned. During display of the value of the statistic, the GUI may include the value of the statistic in a user-interface element of the GUI, include a name of the statistic in the user-interface element, include an identifier for the event stream in the user-interface element, and/or display the user-interface element next to the position of the cursor. The GUI may also highlight the segment and/or slice and dim other portions of the chart. If the cursor is positioned over a portion of a legend associated with the bar or pie chart, one or more segments in the bar or pie chart associated with the portion of the legend may be highlighted, and other parts of the bar or pie chart may be dimmed.

Finally, the statistics and graph(s) are updated in real-time with the time-series event data from the remote capture agent(s) (operation <b>2716</b>). For example, bars in the bar chart and/or slices in the pie chart may be updated with different segments and/or slices as the time window spanned by the bar chart advances and additional time-series event data is collected within the time window. Similarly, the values of statistics shown based on the position of the cursor may be updated using the additional time-series event data.

<figref idref="DRAWINGS">FIG. 28</figref> shows a computer system <b>2800</b> in accordance with the disclosed embodiments. Computer system <b>2800</b> includes a processor <b>2802</b>, memory <b>2804</b>, storage <b>2806</b>, and/or other components found in electronic computing devices. Processor <b>2802</b> may support parallel processing and/or multi-threaded operation with other processors in computer system <b>2800</b>. Computer system <b>2800</b> may also include input/output (I/O) devices such as a keyboard <b>2808</b>, a mouse <b>2810</b>, and a display <b>2812</b>.

Computer system <b>2800</b> may include functionality to execute various components of the present embodiments. In particular, computer system <b>2800</b> may include an operating system (not shown) that coordinates the use of hardware and software resources on computer system <b>2800</b>, as well as one or more applications that perform specialized tasks for the user. To perform tasks for the user, applications may obtain the use of hardware resources on computer system <b>2800</b> from the operating system, as well as interact with the user through a hardware and/or software framework provided by the operating system.

In one or more embodiments, computer system <b>2800</b> provides a system for facilitating the processing of network data. The system may include a configuration server. The configuration server may provide a GUI for obtaining configuration information for configuring the generation of time-series event data from network packets captured by a remote capture agent. The GUI may include a set of user-interface elements for managing one or more event streams containing the time-series event data, which includes enabling the generation of a set of statistics from an event stream without subsequently storing and processing the event stream by one or more components on a network. The GUI may also include the set of statistics and/or one or more graphs containing one or more values from the set of statistics. The statistics and/or graph(s) may be updated in real-time with time-series event data from the remote capture agent(s). Input received through the GUI may be used to update configuration information, which is provided over a network to the remote capture agent(s) and used to configure the generation of time-series event data at the remote capture agent(s) during runtime of the remote capture agent(s).

In addition, one or more components of computer system <b>2800</b> may be remotely located and connected to the other components over a network. Portions of the present embodiments (e.g., remote capture agent, configuration server, GUI, etc.) may also be located on different nodes of a distributed system that implements the embodiments. For example, the present embodiments may be implemented using a cloud computing system that manages the creation, update, and deletion of event streams at a set of distributed remote capture agents, as well as the generation of statistics from the event streams independently of subsequent processing and storage of the event streams by one or more components on the network.

The foregoing descriptions of various embodiments have been presented only for purposes of illustration and description. They are not intended to be exhaustive or to limit the present invention to the forms disclosed. Accordingly, many modifications and variations will be apparent to practitioners skilled in the art. Additionally, the above disclosure is not intended to limit the present invention.