the statistics layer · v1.103.2
JavaScript Statistics for a Data Grid
The grid already knows what the data says. When the grid holds the data in memory, it holds every number on the page and recomputes over them as the filters change. The statistics layer is that knowledge made reachable: a median or a 95th percentile in the totals row, a full column profile in a panel, and columns that track how a value has moved since you opened the page. Every figure is computed over the rows the grid holds, through the same column handles the totals row uses, so a median in the panel and a median in the footer are the same number by the same definition. Point the grid at a windowed remote or pushdown source instead and these figures describe the rows currently loaded, the view in front of you; for statistics over an entire dataset, either pull the whole matching set in or, over a DuckDB connection, push the aggregation down to the engine so the figures are computed over everything.
reductions in the totals row
More than a sum at the bottom.
A totals row usually means one thing: a sum. Here every numeric column names its own
reduction, so one footer reads a median price, a 95th-percentile bid, the deviation of the
day's change and the spread of volume at once. The reductions are kernels, more than twenty
of them, and they run over the filtered rows: narrow to one desk and every figure narrows
with it. Quantiles use the R type 7 definition, the one a spreadsheet's PERCENTILE
agrees with, so the number is the one a reviewer expects rather than one of the nine other
reasonable answers.
The same column, drawn: a histogram of the distribution the footer is reducing, with a density curve over the bars so the shape the median and the deviation describe is one you can see.
shadow columns
A column that remembers.
A shadow column is declared against another column and maintained by the grid. It has no field in the data, and it is not a computed column either, because its value depends on what happened before: the value the row had when the page loaded, how many times it has changed since, which way it is trending. It is a real column throughout, sortable, filterable, groupable, exportable, saveable into a view. The feed below is live, because a delta against a still data set is a column of zeroes.
Delta and percent change are measured against the value at page load, not the previous tick. Moves counts every change since; Streak counts consecutive moves in one direction and resets on a turn, because "seven rises" means something that "seven changes" does not.
the gesture nothing else does
Sort by places moved. Watch the top movers.
The second family of shadow columns answers where a row sits among the others rather than what it did before: rank, percentile, quartile, and the one that matters, places moved since the baseline. Because it is a real column, sorting by it is one gesture, and the result is a live top-movers view that reorders itself as the feed runs. Rank is computed over every tracked row rather than the filtered set, so the order does not shift under you as you filter, and the column never disagrees with itself between two views. "Mark all" in the rail resets every baseline to now, and the race starts again.
capability against a tolerance
The process, and the limits it has never heard of.
Declare the customer's tolerance on a column and the capability figures, a control chart and any out-of-tolerance rule all read the same limits. The chart draws two sets of lines that mean different things: the control limits, three sigma from the process itself, and the spec limits, dashed, which the process has never heard of. A point breaking a rule is marked.
is this column normal?
A straight line, or a story.
A Q-Q plot puts a column's quantiles against a normal distribution's. Points on the line are normal; a bend at the ends is a skew or a heavy tail. The reference line runs through the quartiles rather than being least-squares fitted, because a fitted line is dragged by the very tails you are inspecting.
every pair at once
Which columns move together.
Before the two-column questions below, the whole matrix: a correlogram reads every numeric column against every other and shades each cell by the correlation, so the pairs that move together and the pairs that do not are one glance rather than a search. It reads the filtered rows like everything else here, so narrowing the grid recomputes the matrix.
numbers you can reconcile
A figure that agrees with the spreadsheet beside it.
A number is only worth having if it survives being checked against the tools your team already trusts. Every figure here is computed by a named method, so it reconciles rather than starts an argument.
-
A median that matches the spreadsheet
Quantiles use the R type 7 definition, the same one NumPy computes by default and Excel's
PERCENTILE.INCreturns. So a median or a 95th percentile here lands on the value a stakeholder gets in their own sheet, not one of the nine other reasonable answers. -
A correlation that refuses to invent one
Run a correlation against a column that never varies and you get nothing back rather than a confident-looking number that means nothing. Every correlation also says how many rows it was drawn from, because a figure over forty pairs is a different claim from one over a thousand, and the count is part of the answer.
-
Sample or population, said out loud
A deviation states which one it is rather than leaving you to guess, with the sample form as the default and the population form there by name. The label travels with the figure, so a reviewer knows exactly what they are checking.
-
The same definitions when the engine computes them
These figures are computed client-side, and they reconcile with R, pandas and Excel because they use named, documented definitions. When you push the aggregation down to a DuckDB connection, the engine is asked for those same definitions, so the common statistics match: a quantile is the R type 7 the spreadsheet returns, a deviation is the sample form. Two differences are stated rather than hidden. The statistical
modeis computed by the engine's own convention and can disagree at the edges, so it stays client-side under the recommended policy, and aweightedQuantileis always computed in the browser because there is no faithful way to express it in SQL.
why the numbers hold up
Numbers you can put in front of someone.
A figure on a screen is a claim, and a claim you cannot defend is worse than no figure at all. These are the guarantees behind the numbers here, in the terms that matter when a reviewer, an auditor or your own spreadsheet checks them.
-
A spread that can never come back nonsense
A batch variance or standard deviation here is computed by Welford's method, which accumulates the deviation directly rather than subtracting two large, nearly equal sums of squares. The textbook shortcut can return a negative variance on a near-constant column; this cannot. The number is numerically stable and never negative, so a deviation on tightly clustered data is a real figure rather than an artefact of how it was computed.
-
A percentile that matches R, pandas, Excel and DuckDB
Quantiles use the R type 7 definition, the same one pandas and NumPy compute by default, Excel's
PERCENTILE.INCreturns and DuckDB'squantile_contuses. So a median or a 95th percentile here reconciles with the tool the reader opens next, rather than being one of the nine other reasonable answers a quantile can take. -
Sample by default, the way the engines mean it
A variance or standard deviation is the sample form with Bessel's
n − 1correction by default, matchingvar_sampandstddev_sampin SQL and the default in R and pandas. The population form is there by name when you want it, so the figure means the same thing on both sides of the check. -
The engine agrees, and the two exceptions are named
Push the aggregation down to a DuckDB connection and the common statistics compute identically, because the grid asks the engine for the same named definitions it uses itself. Two figures are documented as exceptions rather than left to surprise you: the statistical
modefollows the engine's own convention and can differ at the edges, so it stays client-side under the recommended policy, andweightedQuantileis always computed in the browser because there is no faithful way to express it in SQL. -
No figure without its scope
Every statistic here is computed over a stated set of rows, never an unstated one. When the grid holds the data in memory, that set is the rows the filter matches. On a windowed remote or pushdown source, a statistic describes the rows currently loaded, and in development the grid warns you by name the moment a figure would be computed over only that window, so a window number is never quietly passed off as a whole-dataset one. You always know which rows a number is about.
two columns from the API
And the figures a panel cannot hold.
The panel and the totals row cover a column at a time. grid.statistics
answers the two-column and whole-series questions directly: correlation and regression,
weighted averages and quantiles, volatility and drawdown over a date-ordered series. Declare
a tolerance on a column and the process-capability figures, a control chart and any
out-of-tolerance rule all read the same limits, so Cpk well above
Ppk tells you the process drifted rather than that it is merely
variable.
// two columns at a time
grid.statistics.correlation('spend', 'revenue');
grid.statistics.regression('spend', 'revenue');
grid.statistics.weightedAverage('price', 'qty');
// a whole date-ordered series
grid.statistics.series('price', { by: 'date' });
// { volatility, growth, maxDrawdown, and more }
// declare a spec, get capability + a control chart
{ field: 'mm', type: 'number',
spec: { lower: 9.5, upper: 10.8, target: 10 } }
grid.statistics.capability('mm');
// { cp, cpk, pp, ppk, defectRate, violations, and more }four desks, four grids
The same layer, in the room it was built for.
Each of these is a realistic grid where the types and the statistics belong together rather than being demonstrated at. Every one is live, and repaints in the low hundreds of milliseconds with a feed running.
-
A trading book
Money as a custom unit, shadow columns for change and rank, a running P&L and a live price feed, sorted by the top movers.
-
Analytical chemistry
Molarity, ppb and dose types, with one contaminated batch the outlier and z-score rules find and nothing else.
-
Manufacturing and process
A machined bore with a tolerance driving capability, drifting in one shift so Cpk and Ppk separate.
-
Fleet and network
Address, data-size and rate types, latency long-tailed so the 99th percentile is far from the median.
Process capability → Column profiling → Variation over time → Selection statistics → Running totals → The API → All of it in one application →