A grid over DuckDB, in the browser

DuckDB is having a moment, and the moment is earned. An analytical engine that runs anywhere, reads Parquet without a load step, and now compiles to WebAssembly so it runs inside a browser tab is a genuinely new shape of tool. The obvious thing to do with it is what people are doing: query a Parquet file where the data already is, without standing up a warehouse to ask one question.

The less obvious thing is what you put in front of it. An engine that can answer a filtered, sorted, paged query over a hundred thousand rows in a few milliseconds deserves a grid that will not throw that away by pulling every row into the page and sorting it in JavaScript. The grid above reads a hundred-thousand-row Parquet file with DuckDB running in the tab you are reading this in, with no server anywhere behind it. It pulls the whole matching set in once and then groups it by plant, so every subtotal and the grand total you see are figures over all hundred thousand rows, not the page on screen.

You bring the engine

The first thing to say is that Lattice Grid does not ship DuckDB, and the bundle is the same size whether you use this or not. You create the connection and hand it over. In the browser that connection comes from @duckdb/duckdb-wasm, which is exactly what the demo above loads. On a server it is any DuckDB client driven through your own endpoint. The adapter is the same code either way, and it imports nothing to read the results: DuckDB returns Arrow tables and the grid reads them through their own accessors, so the Arrow library never enters your bundle either.

That separation is the point. The engine is the caller’s, the connection is the caller’s, and the grid is a driver over whatever you handed it. What you query is a from expression, which can be a table, a view, or a scan written inline. read_parquet('s3://bucket/*.parquet') is as valid there as a table name, so the grid can sit directly over a Parquet file without anything loading it into a table first.

One query can do the work

The adapter’s native mode is to keep nothing local. Sort a column, filter a field, or scroll to the next page, and it builds one SQL statement and asks DuckDB. The filter tree becomes a WHERE clause, with its and and or groups intact. A multi-column sort becomes ORDER BY in the order you set. The page you are looking at becomes LIMIT and OFFSET, so only the window on screen ever comes back. And the total number of matching rows rides along in the same statement through a window count, so the grid knows it is row two hundred of some larger number without a second round trip to find out how large.

The grid above does not run in that mode, and this is the honest place to say so: it pulls the whole matching set in once so it can group and total over all of it, which is a different trade and the one the next section is about. To watch the per-interaction query instead, the live pushdown demo leaves the whole-set pull off and prints the statement above the grid as you interact. Sort a column and watch the ORDER BY change; add a filter and watch the WHERE clause and the timing move. What you read there is the exact query the adapter generated, not an illustration of one.

Filter values are bound, never pasted in

A filter is the one part of a grid a stranger can steer, so the values a user types are sent to DuckDB through prepared statements rather than dropped into the SQL text. Column names, which cannot be parameterised that way, are refused unless they are plain identifiers. If a connection has no way to prepare a statement, the adapter declines to send the filter at all rather than interpolate it, and says so once. There is no path where a user’s input reaches the engine as concatenated SQL.

Grouping and totals over the whole set

The one honest gap in the pure-pushdown story used to be aggregation. Filtering, sorting, paging and the count go to DuckDB, but a group-by or a total, left to its own devices, would summarise only the page the grid had loaded. That is a window statistic dressed as a whole-dataset one, and it is the wrong number to put in a footer. There are now two ways to get the right one, and they suit different jobs.

The first is to pull the whole matching set in. Turn the full-dataset pull on and the grid asks DuckDB for everything the filter matches, once, and holds it. From then on the grouping, the subtotals and the statistics layer all compute over the entire set rather than a page, and browsing is instant because the rows are already here. This is what the grid above does: it holds all hundred thousand rows and groups them by plant, so the count, the average bore, the average cycle and the in-spec rate in every subtotal and in the grand total are figures over the whole set. The trade is plain, and worth stating: once the set is held, the grid is memory-resident, so it is no longer issuing a fresh query per interaction. It is a whole-dataset analytics view, not a live pushdown one.

The second is to push the aggregation down to DuckDB. Rather than pull rows in, the grid asks the engine for the figures directly, ungrouped or with a GROUP BY, and gets back just the numbers. The engine computes the count, the sums and averages, the deviation, the quantiles over the whole matching set, and the browsing story is untouched: filter, sort, paging and the count still push down, and only the visible window still comes back. This keeps the dataset where it is and is the mode to reach for when it is too large to hold. One difference is documented rather than hidden: the statistical mode is computed by the engine’s own convention and can disagree with the grid’s at the edges, and a weighted quantile stays in the browser because there is no faithful way to express it in SQL. The common statistics match, because the engine is asked for the same definitions the grid uses.

So this is not a claim that the grid magically summarises a dataset it never saw. It pushes down the parts that decide whether a large dataset feels fast, and for the aggregates it either pulls the set in or asks the engine for the figures, whichever fits the size of the data in front of you.

Why the pairing is worth it

The reason to reach for this is not novelty. It is that the usual answer to “the dataset is too big for the client” is a server-side row model, where every sort is a round trip and every filter is an endpoint someone has to write and maintain. That is a real answer for genuinely enormous data. For an analytical dataset that fits in a Parquet file, DuckDB in the tab plus a grid that pushes its query down gets you the same responsiveness with no backend to build. Point the grid at the file, and the engine that is having its moment does the heavy lifting while the grid stays out of its way.

See it running, a hundred thousand rows pulled into the browser and grouped with subtotals over all of them, with no server. The live pushdown demo shows the other mode, one SQL query per interaction with the statement on screen, and the DuckDB guide covers the full adapter.