The Pushdown Data Grid: One Query, Any Engine

Most data grids have one idea about data: give me an array. That works until the array is a table with thirty million rows in it, and then the choices are a paging API you write yourself, a server you maintain to do the filtering, or a grid that quietly loads more than it should.

Lattice Grid has a different idea. The grid speaks one query, the source translates it for whatever engine sits behind it, and the browser holds only the rows you can see.

One query, any engine

Everything a user does in the grid, a filter, a sort, a page, a group, a pivot, a total, becomes one portable request. A pushdown source hands that request to an adapter, and the adapter turns it into the engine’s own language: SQL for DuckDB and ClickHouse, Query DSL for Elasticsearch, SPL for Splunk. The engine answers, the grid paints, and nothing else moved.

const source = LatticeGrid.createPushdownSource({
  adapter: LatticeGrid.clickhouseAdapter({ url, table: 'uk_price_paid' }),
  compute: LatticeGrid,
  pageSize: 100,
});
const grid = LatticeGrid.createGrid(el, { source, columns, filterRow: true });

That is the whole integration. The table behind it holds 28 million property sales. Filter the town, sort by price, group by county: each one runs in ClickHouse, and the browser never holds more than a hundred rows at a time.

What gets pushed

The adapter declares what the engine can do, and the source plans around it. The SQL and search adapters push:

• Filters, including text, number, date, set and quick search conditions, and the AND/OR groups the filter builder produces.
• Sorting on any sortable field, with the engine’s own null placement.
• Paging and counts, past any result window limit the engine has.
• Grouping with per group totals, and lazy expansion of a group’s children.
• Aggregates for KPI tiles, charts and the statistics panel.
• Pivots, answered from grouped aggregates with the column set discovered from the engine’s reply.
• The value list behind a column’s filter menu: the top 200 values by count, straight from the engine, with a search that asks again.

When an engine can’t do something, the source says so. source.lastPlan().unpushed names what it finished in the browser, and a request the engine can’t express is refused by name rather than answered with the wrong rows. A slow query you can diagnose beats a fast query that lies.

The backends

Engine	What it’s for
DuckDB in the browser	Parquet and CSV files, local or on object storage, read over HTTPS range requests. A 203 MB GeoParquet extract shows its first rows in seconds with no server at all.
MotherDuck	The same DuckDB adapter over a cloud connection, so a shared warehouse behaves like a local file.
ClickHouse	Analytical tables in the hundreds of millions of rows, pivoted by property type in the engine, with query parameters and a read only default.
Elasticsearch and OpenSearch	Logs and documents, with search_after past the result window and terms aggregations for grouping.
Splunk	Searches as jobs, paged, cancelled when superseded, with time filters as earliest and latest.
REST, OData, GraphQL	Whatever your endpoint supports. The source pushes that part and finishes the rest itself, and tells you which was which.

Adding a backend is an adapter, not a new grid. An adapter is a translation layer with a declared capability list, and the source does the planning, paging, cancellation and aggregate routing for all of them.

The same source, every viewer

The grid is not the only thing that reads a pushdown source. KPI tiles, charts, the statistics panel and map layers on deck.gl or Leaflet all ask the same source for their aggregates, so they follow the grid’s filter without a line of wiring. Filter the grid to one airline and the arcs on the map are that airline’s routes, the tiles recount, and the chart redraws, each from one engine query.

Headless, when there’s no table to show

Sometimes the visualisation is the point and the table is scaffolding. A headless grid is the same query engine without the rendering: create it against a pushdown source, set its filters and grouping in code, and hand its rows and aggregates to a chart, a map or your own component. It’s how a dashboard can run six charts off one ClickHouse table with no grid on screen, and how a page can read a Parquet file into a map with the table a click away.

const headless = LatticeGrid.createHeadlessGrid({ source, columns });
headless.filters.set({ col: 'county', op: 'eq', value: 'DEVON' });
const byType = await source.aggregate(
  { filters: headless.filters.get(), sort: [], range: null, groupBy: ['type'] },
  [{ id: 'avgPrice', col: 'price', fn: 'avg' }],
);

Try it

• ClickHouse, live on 28 million rows
• Your file never leaves the browser, DuckDB over a dropped 10 million row Parquet
• UK flight routes, a grid driving a deck.gl arc map
• Splunk

The guides for each adapter are under Sources in the docs, and every capability on this page is in the reference with a runnable example.