developer guide

Statistics, capability and intervals

What the grid knows about its own numbers: column profiling, process control and capability, confidence intervals, and the tiles that put a figure above the table.

Developer guide › Statistics, capability and intervals

Profiling and statistics

grid.statistics answers questions about the rows the filters left, so every figure describes what the user is looking at rather than the whole table.

Everything worth knowing about a column, in one pass

const p = grid.statistics.profile('capacity');

p.count; p.missing; p.distinct;
p.min; p.q1; p.median; p.q3; p.max;
p.mean; p.stdDev;
p.outliers;      // by the interquartile rule
p.histogram;     // bins, ready to draw
p.alerts;        // what is worth looking at

alerts is the part that saves time: a column that never varies, a key that turns out not to be unique, a fifth of the rows missing. A profile that reports only numbers leaves the reader to notice those, and readers reliably do not.

Method	Answers
reduce	A column through any of the thirty-eight named kernels, or one of your own.
correlation	Pearson's r between two columns. `spearman` resists an outlier; `kendall` is tau-b.
regression	A least-squares fit of one column on another, with slope, intercept and r².
weightedAverage	One column averaged by another. `weightedQuantile` for the median and beyond.
series	How a column varies along an ordering. `by` is required and never guessed.
shadow	What the grid knows about a row over time: updates, delta, rate, rank, percentile, streak.

The filters are part of the question. Every one of these reads the filtered rows. Narrow the grid and the statistics narrow with it, which is the behaviour you want when the filter is the analysis.

Kernels see arrival order, not display order. Anything order-dependent takes an explicit by rather than inferring one from the current sort, so the answer does not change when a user clicks a column header.

Process control and capability

Whether a process sits inside the tolerance it was given, and whether it is behaving or drifting. The tolerance is declared once, on the column.

The specification lives with the column

{ id: 'diameter', type: 'number', spec: { lower: 9.95, upper: 10.05, target: 10 } }

Asking for the capability

const c = grid.statistics.capability('diameter', { rules: 'nelson' });

c.cp; c.cpk;         // short-term spread, from the moving range
c.pp; c.ppk;         // overall spread
c.outOfSpec;         // parts outside the customer's tolerance
c.limits;            // { centre, upper, lower, sigma }
c.violations;        // [{ index, rule, description }, …]
c.interval;          // a confidence interval for cpk

One declared tolerance, so nothing can disagree. The indices, the charts and any conditional format all read the same spec. A tolerance passed separately to each is a tolerance that eventually differs between them, and a capability report that contradicts the cell colouring is worse than neither.

Cp and Cpk use short-term variation, Pp and Ppk overall. The first pair comes from the moving range, which is what the process can do when it is behaving; the second from the whole spread, which is what it actually delivered. Ppk well below Cpk is the signal that the process drifted rather than that it is incapable.

A baseline finds a shift instead of absorbing it. baseline: 30 fixes the limits over the first thirty readings. Limits recomputed over all the data widen to accommodate the very shift you are looking for, and then report no violation.

The point estimate alone overstates the case. A Cpk of 1.35 measured on thirty parts has a lower bound below 1.0, so a process that has "passed" a 1.33 requirement on thirty parts has demonstrated very little. interval is reported alongside it for that reason.

Drawing it

Three chart types complete the picture, and they read the same specification.

Type	Shows
control	Readings against the centre line and control limits, with every rule break numbered.
movingRange	The companion chart: variation between consecutive readings.
capability	The distribution against the tolerance, with a curve for each of the two spreads.

Rule breaks are numbered rather than merely marked, under Western Electric's four rules or Nelson's eight. The two sets number differently, so the chart names which it applied: a "rule 3" that could mean either is not a finding anyone can act on.

Confidence intervals

How firmly the data pins a figure down. An interval narrows as the grid does, because it describes the filtered rows and not the whole table.

A mean and a rate

grid.statistics.interval('capacity');
// { lower, upper, mean, n, confidence }, by Student's t

grid.statistics.interval('status', {
  kind: 'proportion',
  where: (v) => v === 'failed',
});
// Wilson score, which stays sensible at small n and near 0 or 1

Intervals are also available on a regression slope and on a capability index. Each uses the method that suits it: Student's t for a mean, the Wilson score for a proportion, and Bissell's approximation for Cpk.

The line the product draws. Lattice quantifies uncertainty. It does not adjudicate hypotheses: there are no p-values and no significance tests. An interval says how precisely a figure is known and leaves the judgement where it belongs. A tool that returns a verdict invites it to be read as one, and a grid is the wrong place for that.

Wilson, not the textbook formula. The normal approximation gives bounds below zero and above one at small counts, which is visibly wrong to anyone who reads it. The Wilson score stays inside the interval it is describing.

Statistic tiles

A headline figure over a grid, with the change since a baseline, a tone from thresholds, and the interval underneath.

A tile that follows the grid

import { createStat } from '@toclocoinc/lattice-grid';

createStat({
  grid,
  container: tile,
  title: 'Total capacity',
  value: { of: 'capacity', fn: 'sum' },
  baseline: (g) => lastMonth,
  bands: { good: 5000, warn: 3000, direction: 'up' },
  interval: (v, g) => g.statistics.interval('capacity'),
});

bands and goodWhen judge different things. goodWhen says whether a rise is good news, and colours the change indicator. bands judge the value itself. They are separate because a Cpk of 0.9 is bad news whether it rose or fell to get there.

The tile follows the grid by default. Filter the grid and the figure updates. scope chooses filtered, all or selected rows; live: false detaches it and leaves refresh() to you.

value also takes show, which reports a field from the row holding an extreme rather than the extreme itself: { of: 'sales', fn: 'max', show: 'rep' } is the name of the best rep. It needs min or max, because no single row holds an average.