duckdb and parquet
Build an interactive frontend for DuckDB.
Turn DuckDB and Parquet datasets into interactive grids, charts and analytical applications. Push the query to the data. Return only what the user needs.
Explore the live demo Query source documentation →
Vanilla JavaScript. Zero runtime dependencies. Lattice ships no database of its own: it uses the connection your page already has.
Open the full demo → The same idea over a log lake → How pushdown works →
the wrong way round
Don't move millions of rows just to filter them.
A client-side grid has to be given its rows before it can do anything with them. Every filter, every sort, every total means the whole set has to reach the browser first, so the size of the screen your users get is capped by the size of the payload they are willing to wait for. That is a strange bargain to accept when the data already lives somewhere that answers questions like this for a living.
What a grid usually asks for
Send me everything, and I will work out what to show. The transfer, the memory and the wait all scale with the dataset, whether or not the user ever looks past the first screen.
What a query source asks for
Send me the hundred rows I am about to paint, and tell me how many matched. Filtering, sorting, paging and totals become one statement answered where the data already sits, and the answer that comes back is the size of the screen, not the size of the dataset.
The demo above is that difference made visible. The panel beside the grid prints the statement DuckDB was given, with the values bound to it and the time it took in your own browser, and the line underneath is the whole point in two numbers: how many rows matched, and how many came back.
a clean division of labour
DuckDB does the data work. Lattice does the application work.
You already chose your engine. Lattice does not want to be a second one, and it does not ship a copy of yours: you hand it the connection your page already made, and it imports nothing of its own. What it adds is everything above the query, which is where the engineering time actually goes.
DuckDB's side of the line
Reading Parquet, matching the filter, ordering the result, counting the matches and computing the summaries. It is extremely good at all of it, over files that would never fit in a tab.
Lattice's side of the line
Turning what the user does into that query and what comes back into a screen: columns and types, filter and sort controls, virtual scrolling, grouping and totals, charts and KPI tiles, selection, keyboard and screen-reader support, saved views, and export.
Neither side has to know much about the other. Point the same screen at a table in your warehouse instead of a file in the browser and nothing above the connection changes. See it over a private log lake, where the credential stays on the connection and never reaches the grid.
from parquet to application
Four steps, and none of them are yours to write.
A file, an engine, a source and a screen. The two in the middle are configuration, and the one at the end is an ordinary Lattice grid that has no idea where its rows came from.
Your Parquet, or your table
A file on object storage, a directory of partitioned files, or a table in the warehouse you already run. The source takes a from expression and runs it as written, so anything your engine can select from is a valid starting point.
Your DuckDB connection
In the browser, on a server, or against a warehouse behind an endpoint. The adapter takes the connection you made and asks the engine to describe the columns, so a date filter is typed and cast the way that engine expects rather than guessed at.
A query source
One line that turns the grid's state into a statement. It tells you what it pushed and what, if anything, it could not, so you are never guessing whether a screen is describing the whole set or the page that happens to be loaded.
An ordinary application
A grid configured exactly as it would be over rows held in memory, with charts and KPI tiles fed from the summaries the engine returns over the whole matching set. Nothing about your columns, your renderers or your layout changes because the data got bigger.
what becomes a query
The things users do all day, answered by the engine.
Everything below leaves the browser as part of one statement. Values are bound as parameters rather than pasted into text, which is what makes the same code safe to point at a real warehouse.
Filtering
Equals and not equals, greater and less than, in a list, between two bounds and outside them, contains, starts with and ends with, blank and not blank. Combine them with and and or as deeply as you like and it stays one statement. Dates and timestamps are typed from the engine's own description of the column and cast for it.
Sorting and paging
Sort on several columns at once and the order is applied to the whole matching set, not to the rows that happen to be loaded. Scrolling asks for the next window, so what comes back is always the size of the screen rather than the size of the dataset.
Counting
The number of matching rows comes back inside the same statement that returns the page, so the count and the rows can never disagree with each other.
Summaries and grouping
Totals, averages, minimums and maximums, medians, quartiles and percentiles, standard deviation and variance, distinct counts. Group by a column and the engine returns a summary per group and a grand total, all computed over everything that matches.
the same source, every engine
Not only DuckDB.
The query source is the general shape and DuckDB is one adapter for it. An adapter says what its engine can answer, and the source pushes exactly that much and finishes anything left over in the browser rather than quietly returning the wrong rows.
Query engines and lakes
DuckDB over Parquet, in the browser or on a server, and the object storage your data already sits on.
Services and endpoints
An OData service, a REST endpoint or your own API, each behind the same source contract. See the source documentation for what each one pushes.
Your dataset is already fast. Give it a frontend that is too.
Development on localhost needs no licence key, so you can point a grid at your own file or your own table and see the real screen before you decide anything. Licence per domain when you ship.