Statistical Processing in Splunk

---

Statistical Processing

A data series is a sequence of related data points that are plotted in a visualization.

• types:
  • single-series
    • compares values of a single data category
  • multi-series
    • compares values of 2 or more data categories
  • time-series
    • compares values over time
• transforming commands can be used to organize results into a statistical table into a data series that can be visualized

Chart Command

chart command is used to take results from a search and format it into a table that can be displayed as a visualization.

• specify the y-axis with a stats function <stats-fun>(<wc-field>)
  • wc-field> is a field with numeric values; supports wildcards
  • <stats-func> is a supported statistical function
• Specify x-axis with over <row-split>
• Further split data by including by <column-split>
• control behavior with span, limit, useother, and usenull
• can only be spit over two fields

Timechart Command

Timechart command performs stats aggregations against time.

• time is always the x-axis
• <stats-func>(<field>) applies a function to a single field
  • always the y-axis
  • if using count function, a field does not need to be specified
• <split-by-field? to further split results of the aggregation
  • only one split can be specified
  • each distinct value of <split-by-field> will become a series
• can use span and limit
• by <field> becomes the different series charted (lines)
• multi-series mode in Format separates each series into their own y-axis

Top Command

Top command finds the most common values for given fields in a results set.

• by groups results together based on shared fields
• by default, returns top 10 results
  • change with limit
• showperc=f turns off percent
• countfield renames the count column

Rare Command

rare command returns the least common values of a results set.

• has the same options as the top command

Stats Command

stats command produces statistics from search results using functions.

• ...| stats <stats-func>(<wc-field>) [as <wc-field>] [by <field-list>]
• returns a results table
• group results with by clause

Stats Functions

• 4 categories of statistical functions:
  • Aggregate
    • summarizes event values to create a single value
  • Event Order
    • returns values from fields based on processing order
  • Multivalue
    • returns lists of values for a field as a multivalue entry
  • Time
    • returns values based on time
• distinct_count() (dc())
  • counts unique values in a given field
• min, max, avg
• values function lists all unique values for a given field
• list lists all values found in a given field, including duplicates

Eval Command

eval command performs calculations on values in data.

• ...| eval <field1>=<expression1>[, <field2>=<expression2>]
• creates new temporary values that do not get added to index
• supports multiple operators:
  • arithmetic
  • concatenation
  • boolean
  • comparison
• field values are case-sensitive
• string vales must be double-quoted
• field names must be unquoted or single-quoted
• use a . instead of + to concatenate strings and numbers
  • avoid data type conflicts
• does not need a function, can be used without

Example

| eval Sales = "$".tostring(Sales,"commas")

Eval Functions

Evaluation functions evaluate an expression based on events and returns a result.

• 11 categories of evaluation functions:
  • Conversion
  • Comparison and Conditional
  • Mathematical
  • Informational
  • Statistical
  • Text
  • etc.
• pow(x, y)
• round(x, y)
• max(x1, x2,...)
• min(x1,x2,...)
• random()
  • no arguments
  • returns a pseudo-random integer

Eval Conversion Functions

• eval can be used as a function within the stats function
  • ...| stats count(eval(vendor=Google)) as Google

Rename Command

rename command renames a field.

• syntax: ...| rename <field> as <newfield>
• once a command is renamed, it must be referenced by the new name later in the same search query
• can use wildcards to rename multiple fields that match a pattern

Sort Command

sort displays a fields values in ascending or descending order.

• syntax: ...| sort (-|+) <field> [limit=<int> | <int>]
• sorts ascending order by default
• limit results to specified integer
• determines field type, then determines sorting type:
  • alphabetic strings = lexicographic sort
  • numeric = numerical sort
  • combination = lexicographic or numeric sort based on first character