Graphs

This section gives a detailed guide to the Graph view, which is where you can graph values in your logs, such as server response times or page sizes. You can graph values from an individual log, aggregate multiple servers / logs, and use search terms to narrow the log messages to graph.

Accessing Graphs

There are various ways to access Scalyr's graphing tools. The most common workflow involves using Search view to select the field or metric you wish to graph (see (1) through (3), (6), and (7) below). Selecting any of the graphing options from the Field list will take you to the Graph view (see (7a) below).

Another way to enter Graph view is by clicking on "Expand Graph" from the bar chart of log volume on the Search view page. This takes you to a graph of your log volume (matches per second). You can then use the tools explained below to modify the graph, including the field and graph type.

If you have already saved a graph as an alert, or to a dashboard, click on Alerts or Dashboards to view that graph. From there you can also access the Dashboard or Alert JSON to add, change, or modify your existing graphs.

Quick Reference

(1) To search for a specific word or phrase, type it here. This determines which log messages are reflected in the graph. Numbers, punctuation, or phrases must be enclosed in quotes. Sample searches:

Search Meaning
error To search for a word or part of a word, just type it
"/blog" Punctuation must be enclosed in quotes
"customer 1309" Multi-word phrases must also be enclosed in quotes
userId = 1309 Matching on a parsed field
time > 0.5 Numeric comparison on a parsed field

See Query Language Reference for a full description of the Scalyr query language.

(2) As you type search text into the box, it is parsed and presented in a form that makes your search easier to read and understand. Different parts of your search text such as fields, operators, and values are highlighted in order to visually differentiate them. For example, in the search text "bytes > 5000", each of the three components will be a different color.

(3) Click here to specify the time range to graph. The following options will appear:

(3a) Click on a preset to quickly graph that time range.

(3b) Enter the start time for your graph. You can enter a time (e.g. 14:30 or 5:05 AM), a date (May 23), or date and time (5/14/2016 2:00 PM), using a wide variety of date and time formats. You can also enter shortcuts like "5h" to indicate five hours ago. See Time Syntax Reference for a complete list of options.

(3c) Enter the end time for your graph. You can use any of the formats explained in (3b). You can also enter a shortcut beginning with + to specify the amount of time you'd like to search, e.g. +24h or +1d to graph a one-day period beginning at the From time.

(4) Use these buttons to move forward or backward one half-graph at a time.

(5) Use this button to view the raw log messages matching your search.

(6) Use these fields to search a specific server or log file. If you're using Kubernetes these will allow you to search cluster and controller name, respectively. You can use a single * as a wildcard at the beginning or end (but not the middle) of the server or log file name.

(7) This area lists the fields the parser found in the log messages matching your search. The top 100 fields are arranged alphabetically in a scrollable window (All Fields). Click the dropdown and switch to Top Fields to view the most common fields first.

The number next to each field indicates how many distinct values appear in that field. (If there are more than a few hundred distinct values, the number shown will be an estimate.)

Click on a field to bring up a list of its most common values:

(7a) Depending on the type of data, various graphing options appear as buttons:

  • Graph Values graphs the selected field over time.
  • #Matches graphs matching events per second, broken down by the selected field.
  • Distribution graphs a distribution of the selected field.

For more information on Scalyr's graphing tools, see Graphs.

(7b) You can click on a value to restrict your graph to events having that field value. You can also use the `==` and `!=` symbols to include (or exclude) these values from your search.

(7c) The bars provide a visual indication of how often each value appears, while the numbers provide more precision. Note that these are often estimates due to sampling (see (7d) below).

You can click on a value to restrict your graph to events having that field value.

(7d) Information concerning estimated values is located here. (We sample to achieve a statistically valid number, using a two-pass method. In the first pass (first bullet point), we query a 1% subsample to estimate the total number of matches. In the second pass (second bullet point), we sample the data with a rate based on information obtained in the first pass.)

(7e) If the field has too many values to display on one screen, click "see more" to show up to a maximum of 200 values.

(8) This shows the name of the field you're graphing.

(9) This area shows a graph of the specified field.

(10) Use the chart type drop down to select the type of chart you'd like to see. If you choose Stacked Bar Chart you'll also be able to select the time interval for the bars, e.g. 1 minute or 1 hour. If you have a graph with a huge difference between highest and lowest values you may want to switch to a logarithmic y-axis.

(11) This area lists the functions which you can select for your graph. Check one or more boxes to select different functions of the graphed field. The available functions are:

Function Value
Average The average of all values in each time period. For instance, if you are graphing server response times, this will show the average response time.
Minimum The smallest value in each time period.
Maximum The largest value in each time period.
Sum/sec The "smoothed" sum of all values per second. For instance, if you are graphing the response-size field in a web access log, this will give the response bandwidth in bytes per second. (We divide the time period of your graph into a number of time spans, sum all values per time span, and then divide by the time span in seconds to get an average sum per second, per time span. Note that graphed values are exact over brief time periods (100 seconds, for example), and effectively smoothed over longer time periods.)
10th %ile Shows the 10th percentile of all values in each time period.
50th %ile Shows the 50th percentile (median) of all values in each time period.
90th %ile Shows the 90th percentile of all values in each time period.
95th %ile Shows the 95th percentile of all values in each time period.
99th %ile Shows the 99th percentile of all values in each time period.
99.9th %ile Shows the 99.9th percentile of all values in each time period.

You can click and drag in the graph to select a time range. The legend then shows aggregate statistics for the entire time range. A "Zoom to selection" button will appear; click this button to zoom in to the selected time range.

(12) These statistics apply only when you have dragged to select a time range in the graph. The deltas show information about the slope, or rate of change, of your graph. For instance, if you're viewing a graph of free disk space, the delta tells you how quickly disk space is being consumed.

  • "Change" shows the change in value from one end of the graph to the other.
  • "Change/hour" shows the average change in value per hour.
  • "Change/sec" shows the average change in value per second.

Deltas are computed based on average values in the first and last time periods of the graph, even if you have chosen to display a different function (such as minimum or maximum).

(13) Click the Compare dropdown to graph a previous period with the current period.

(14) Click "Break Down By" to create a breakdown graph. This graph type breaks down log volume or a plot of a field by another field. For example, when graphing data from a web access log, you could break it down by URL or user-agent. The breakdown graph below filters for logs where `$status == "failure"` (14a), breaks the results down by server (14b), and then presents the information as a stacked bar chart (14c):

Note that breakdown graphs can timeout when they require searching through large amounts of log data. Whenever possible, we recommend the use of standard graphs when saving to Alerts or Dashboards. See Timeout Tips for more information.

(15) Click the "Save" button to display the following actions for your current search:

  • Save Graph: Opens a dialog box that lets you save the graph to either your personal or team's list of saved graphs, which are also available in the main Search menu at the top of the page.
  • Save as Alert: Create a new alerting rule, which will trigger if the number of matches to your current search goes above or below a level you specify.
  • Save to Dashboard: Add this search to an existing dashboard, or start a new dashboard with this search.
  • Download as PNG: Saves the current graph as a PNG file and downloads it to your default Downloads folder.

When possible, we recommend the use of standard graphs when saving to Alerts or Dashboards. See Timeout Tips for more information.

(16) Click the "Share" button in the left-center of the search bar to display the following Share actions for your current search:

  • Copy Link: Opens a modal window where you can copy a link to this search with relative time references replaced by absolute (e.g., instead of the searching the previous hour, it would search 8 a.m. to 9 a.m.).
  • Add to Shared Search List: Opens a dialog box that lets you save the active search query to either your personal or team's list of saved searches; the team list is selected by default. Saved searches are available in the Search menu.