demo D293
Triage security logs at scale in the browser, no server
A day of firewall, DNS, auth and IDS events in a Parquet file, queried by DuckDB in the tab: each filter is a WHERE with the whole-set count, the top talkers come from a GROUP BY over everything that matches, and the SQL is on screen
duckdbAdapter · source.aggregate() · lastPlan()
This grid triages a day of firewall, DNS, authentication and intrusion events held in a Parquet file, with DuckDB running in the browser and no server behind it. Each preset becomes a WHERE clause with the whole-set match count, the top sources come from a GROUP BY over everything that matches, and the SQL DuckDB ran is on screen, so an analyst can see exactly what was asked of the data.
The configuration
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@toclocoinc/lattice-grid@1.47.0/lattice-grid.min.css">
<script src="https://cdn.jsdelivr.net/npm/@toclocoinc/lattice-grid@1.47.0/lattice-grid.min.js"></script>
<div id="grid" style="height: 560px"></div>
<div id="top"></div>
<pre id="sql"></pre>
<script>
// Lattice ships no engine. The page starts DuckDB-Wasm itself and hands the
// adapter a live connection; the grid imports none of it and makes no
// network calls of its own. DuckDB fetches the Parquet bytes it needs.
(async () => {
const duckdb = await import('https://cdn.jsdelivr.net/npm/@duckdb/duckdb-wasm@1.29.0/+esm');
const bundle = await duckdb.selectBundle(duckdb.getJsDelivrBundles());
const worker = await duckdb.createWorker(bundle.mainWorker);
const db = new duckdb.AsyncDuckDB(new duckdb.ConsoleLogger(duckdb.LogLevel.ERROR), worker);
await db.instantiate(bundle.mainModule, bundle.pthreadWorker);
const connection = await db.connect();
// The bundled sample: a day of events in one Parquet file on this origin.
const adapter = LatticeGrid.duckdbAdapter({
connection,
from: "read_parquet('/demo-data/security-events.parquet')",
});
// Your own log lake instead: give DuckDB the credential as its own SECRET
// on the connection, before the grid sees it, then point from at the bucket.
// The grid never sees the credential; it runs your from expression as-is.
//
// await connection.query(
// "CREATE SECRET logs (TYPE s3, PROVIDER credential_chain, REGION 'eu-west-2')"
// );
// const adapter = LatticeGrid.duckdbAdapter({
// connection,
// from: "read_parquet('s3://your-bucket/logs/2026/*.parquet')",
// });
const source = LatticeGrid.createPushdownSource({
adapter,
compute: LatticeGrid, // the grid finishes anything SQL could not express
pageSize: 200, // only the visible window comes back, per interaction
// count and sum are verified identical between DuckDB and the grid, so
// this policy lets source.aggregate() run them in the engine.
aggregates: { default: 'engine-if-identical' },
});
const grid = LatticeGrid.createGrid(document.getElementById('grid'), {
rowKey: 'event_id',
toolPanel: { side: 'left', panels: ['filters', 'columns'] },
columns: [
{ field: 'ts', title: 'Time (UTC)', type: 'timestamp', typeOptions: { timeZone: 'UTC' } },
{ field: 'action', title: 'Action', filter: { type: 'set' } },
{ field: 'severity', title: 'Severity', filter: { type: 'set' } },
{ field: 'event_type', title: 'Type', filter: { type: 'set' } },
{ field: 'src_ip', title: 'Source', type: 'ipv4' },
{ field: 'dst_ip', title: 'Destination', type: 'ipv4' },
{ field: 'dst_port', title: 'Dst port', type: 'text' },
{ field: 'protocol', title: 'Proto', filter: { type: 'set' } },
{ field: 'bytes', title: 'Bytes', type: 'bytes' },
{ field: 'rule_id', title: 'Rule', filter: { type: 'set' } },
],
source, // each filter, sort and page is one SQL statement DuckDB answers
});
// A preset is an ordinary filter tree; the adapter binds every value.
grid.filters.set({ col: 'action', op: 'eq', value: 'deny' });
// Top talkers over the WHOLE matching set: a GROUP BY pushed to DuckDB,
// computed over every row the filter matches, not the loaded page.
const top = await source.aggregate(
{ filters: grid.filters.get(), sort: [], range: null, groupBy: ['src_ip'] },
[{ id: 'events', col: 'event_id', fn: 'count' }, { id: 'bytes', col: 'bytes', fn: 'sum' }],
);
// grouping mirrors SQL GROUPING(): 0 is a real group, 1 is the ROLLUP total.
const ranked = top.groups.filter((g) => g.grouping[0] === 0)
.sort((a, b) => b.values.events - a.values.events).slice(0, 10);
document.getElementById('top').textContent = ranked
.map((g) => g.keys[0] + ' ' + g.values.events + ' events ' + g.values.bytes + ' bytes').join('\n');
// The split, after every query: what DuckDB did and what the grid did.
const plan = source.lastPlan();
document.getElementById('sql').textContent =
'pushed: ' + JSON.stringify(plan.pushed.filters) + '\n' +
'left for the grid: ' + (plan.unpushed.join(', ') || 'nothing') + '\n' +
'aggregates by the engine: ' + plan.aggregates.engine.map((a) => a.fn).join(', ');
})();
</script>Triage a security log without standing up a query service
Security logs land as Parquet: VPC Flow Logs, CloudTrail, WAF and DNS resolver logs, and anything a security lake writes in OCSF. This demo shows Lattice Grid working as an interactive triage front end over exactly that kind of file. The page starts DuckDB in the browser, hands the grid’s DuckDB adapter a live connection, and from then on every filter, sort and page the analyst asks for is one SQL statement the engine answers. Only the visible window of rows comes back, and the count of everything that matched rides along in the same statement, so the headline figure is always the whole set, never the loaded page.
The ranked panel is the part a security analyst usually has to leave the grid to get. Ask for the top sources, rules or destination ports and the grid pushes a GROUP BY to DuckDB, computed over every event the current filter matches. Pick a row in that panel and it becomes one more condition, so pivoting from “who is noisiest” to “show me that host” is one click. Two small charts read the same aggregate results through the Data Router, so the picture by hour and the top ten redraw with every filter.
The panel under the grid prints the statements DuckDB actually ran, with their bound values and how long each took, beside the grid’s own account of what was pushed down and what, if anything, it finished itself. Filter values travel as prepared-statement parameters, never pasted into SQL. Every address in the file is synthetic: private ranges inside, the reserved documentation ranges outside.
How do I query security logs in S3 from the browser with DuckDB?
Start DuckDB-Wasm in the page, give the connection a SECRET for the bucket (an ambient credential chain or an explicit key), and hand that connection to duckdbAdapter with from set to read_parquet('s3://your-bucket/logs/2026/*.parquet'). Wrap it in createPushdownSource and pass it to createGrid. The grid pushes the filter tree, the sort, the page and the count down as SQL; the credential stays on the connection and the grid never sees it.
How do I get top talkers over the whole matching set rather than the loaded page?
Set an aggregates policy on the source, then call source.aggregate(request, stats) with a groupBy. Under engine-if-identical the statistics whose engine result is verified identical to the grid’s own, count and sum among them, are computed by DuckDB over every row the filter matches, and only the group figures come back. source.lastPlan().aggregates reports which statistics the engine computed and which the grid did.
What is the sensible size for in-browser DuckDB?
A bounded slice: a day of logs, or one bucket prefix. The in-tab engine reads only the byte ranges a query needs, and the whole-set pull is memory-guarded rather than silently truncated. Larger scans belong to a server-side DuckDB behind your own endpoint, which the same adapter drives unchanged.