Histogram of API Response Times

If you are the SRE on call for an API, the question is never what the average response time is. It is how slow the slowest requests get, and how many people meet them. An average of 90 milliseconds can sit comfortably on top of a handful of requests that take a full second, and the customers who hit those are the ones who write in.

A histogram answers that by counting. It sorts every request into a bucket of response times and draws one bar per bucket, so the pile of fast requests shows on the left and the long tail trails off to the right. The chart above is bound to the grid beneath it, one row per request, so filtering the rows redraws the chart. Filter the Endpoint column to /orders and its slow requests are all that is left to read; switch to /login and compare the tails.

Why a histogram and not an average

An average, or a bar chart of averages per endpoint, folds every request into one number and the slow ones vanish into it. A histogram keeps all of them, so a tail that is one request in thirty is still a bar you can see.

The chart over a grid

The chart reads the grid’s rows, so there is nothing to keep in step. A histogram takes one column, y, and counts how many rows fall in each bucket. Here is the example in plain JavaScript, with a dozen inline requests standing in for your own rows.

import { createGrid } from '@toclocoinc/lattice-grid';
import '@toclocoinc/lattice-grid/css';
import { createChart } from '@toclocoinc/lattice-grid/modules/charts';

const rows = [
  { id: 1, endpoint: '/search', ms: 42 }, { id: 2, endpoint: '/search', ms: 51 },
  { id: 3, endpoint: '/search', ms: 58 }, { id: 4, endpoint: '/search', ms: 240 },
  { id: 5, endpoint: '/orders', ms: 95 }, { id: 6, endpoint: '/orders', ms: 118 },
  { id: 7, endpoint: '/orders', ms: 142 }, { id: 8, endpoint: '/orders', ms: 960 },
  { id: 9, endpoint: '/login', ms: 32 }, { id: 10, endpoint: '/login', ms: 38 },
  { id: 11, endpoint: '/login', ms: 45 }, { id: 12, endpoint: '/login', ms: 320 },
];

const gridEl = document.createElement('div');
const chartEl = document.createElement('div');
gridEl.style.height = '260px';
chartEl.style.height = '320px';
document.body.append(gridEl, chartEl);

const grid = createGrid(gridEl, {
  rowKey: 'id',
  rows,
  columns: [
    { field: 'endpoint', title: 'Endpoint', filter: { type: 'set' } },
    { field: 'ms', title: 'Response (ms)', type: 'number' },
  ],
});

createChart({ grid, container: chartEl, type: 'histogram', y: 'ms', buckets: 12,
  title: 'Response times in milliseconds' });

The same thing in React

The React adapter is a factory, so you build the component once at module scope. The ref’s grid is the real grid, which is what the chart binds to.

import React, { useEffect, useRef } from 'react';
import { createGrid } from '@toclocoinc/lattice-grid';
import '@toclocoinc/lattice-grid/css';
import { createLatticeGrid } from '@toclocoinc/lattice-grid/modules/react';
import { createChart } from '@toclocoinc/lattice-grid/modules/charts';

const LatticeGrid = createLatticeGrid({ React, createGrid });
const rows = [
  { id: 1, endpoint: '/search', ms: 42 }, { id: 2, endpoint: '/search', ms: 58 },
  { id: 3, endpoint: '/search', ms: 240 }, { id: 4, endpoint: '/orders', ms: 95 },
  { id: 5, endpoint: '/orders', ms: 142 }, { id: 6, endpoint: '/orders', ms: 960 },
  { id: 7, endpoint: '/login', ms: 32 }, { id: 8, endpoint: '/login', ms: 45 },
];
const columns = [
  { field: 'endpoint', title: 'Endpoint', filter: { type: 'set' } },
  { field: 'ms', title: 'Response (ms)', type: 'number' },
];

export function ResponseTimes() {
  const gridRef = useRef(null);
  const chartRef = useRef(null);
  useEffect(() => {
    const chart = createChart({ grid: gridRef.current.grid, container: chartRef.current,
      type: 'histogram', y: 'ms', buckets: 12, title: 'Response times in milliseconds' });
    return () => chart.destroy();
  }, []);
  return (<>
    <LatticeGrid ref={gridRef} rowKey="id" columns={columns} rows={rows} style={{ height: 220 }} />
    <div ref={chartRef} style={{ height: 280 }} />
  </>);
}

The options used

These are the rows of the charts reference for the options in the listings above.

Choosing the buckets

Twelve buckets is the default, and it is a fair start. Too few and the tail is swallowed into the last bar; too many and a thin sample looks like noise. The reference also describes a curve option, which overlays a density curve on a histogram so you can tell what is in the data from what is in the binning. Whatever the setting, the filter on the grid still drives it: narrow the rows to one endpoint, or to the requests after a deploy, and the buckets are counted again from what is left.

Next

See the histogram beside a box plot in the distribution chart demo, read the full list of chart types on the charts page, or follow the how-to for a chart that follows the grid.