security log triage
Triage security logs in the browser, straight from the Parquet you already keep.
Your logs already land in S3 as Parquet: VPC Flow Logs, CloudTrail, WAF and DNS resolver logs, and whatever a security lake writes in OCSF. DuckDB reads those same files, in the tab or on a server, with no per-query service in between. Put Lattice Grid over that connection and an analyst gets a screen that filters, sorts, pages and counts a day of events as SQL pushed to the engine, ranks the top talkers over everything that matches, and shows the query it ran.
Below: a hundred thousand synthetic events in one Parquet file, queried by DuckDB running in your browser. Nothing here talks to a server of ours.
where the logs already are
The lake is built. The front end is the missing part.
Most security teams have already done the hard part. The logs are collected, partitioned and written as Parquet in a bucket, and Athena answers questions about them with serverless SQL. What that stack does not give an analyst is a screen to work in: something to narrow, sort, scroll and rank a day of events interactively, without writing a query for every step.
Landed as Parquet, queried with SQL
VPC Flow Logs, CloudTrail, WAF logs, Route 53 resolver logs and Amazon Security Lake all write columnar Parquet to S3, often partitioned by day. Athena scans it on demand. That is the right shape for retention, for audit, and for the wide question that touches months of data at once.
The same files, read by DuckDB
DuckDB reads Parquet natively, straight from S3, and it runs in the browser as duckdb-wasm as readily as on a server. It reads only the byte ranges a query touches. Point it at a day's prefix and the whole file never has to be downloaded, let alone loaded into a table first.
Lattice Grid is the front end over that connection. It is a client library that imports no engine: you create the DuckDB connection, hand it over, and the grid turns every interaction into one SQL statement the engine answers. The result is a triage screen over the log lake you already have, alongside the batch queries you already run.
what the analyst gets
Filter, count, rank, and see the query.
Filter, sort and page, pushed down
Denied only, high and critical, one source address, a port range, a time window: each condition becomes a WHERE clause DuckDB answers, the sort an ORDER BY, the scroll a LIMIT and OFFSET. Only the visible page comes back, and the count of everything that matched arrives in the same round trip, so the headline number is always the whole set.
Top talkers over everything that matches
Who is noisiest, which rule fires most, which port is being hit: a GROUP BY pushed to the engine and computed over every matching event, not the two hundred rows on screen. Count, sum, min, max and the quantiles are verified to match the grid's own definitions, so a figure computed by DuckDB is the figure the grid would have produced.
The SQL, on screen
A security analyst wants to see the query, not trust a widget. The statement DuckDB ran, its
bound values and its timing sit under the grid, and source.lastPlan()
reports what was pushed to the engine and what, if anything, the grid finished itself. Nothing
is hidden behind "server-side".
No server, no network calls of its own
The grid talks to a database connection you made. It makes no HTTP calls of its own: DuckDB fetches the byte ranges each query needs, and nothing passes through a service of ours. In the browser that means a log file can be triaged with no back end at all.
Credentials stay DuckDB's
A private bucket is read through a SECRET on the connection, picked up from the ambient credential chain or given explicitly, set before the grid sees the connection. The grid never sees the credential and does nothing with it. Filter values travel as prepared-statement parameters, never pasted into SQL.
The same adapter behind your endpoint
When the slice outgrows a tab, move DuckDB to a server and drive it through your own endpoint. The adapter takes a connection and does not care where it came from, so the grid, the pushdown and the visible plan are unchanged. Nothing about the screen has to be rebuilt.
pointed at your bucket
One SECRET, one from expression, one grid.
The demo above reads a Parquet file served beside the page. Your own lake is the same three
lines with a different from. The credential goes on the DuckDB
connection first, as DuckDB's own SECRET, so by the time the grid
receives the connection the bucket is already readable and the grid has nothing to hold.
createPushdownSource wraps the adapter and takes the aggregates
policy. engine-if-identical pushes only the statistics whose engine
result is verified identical to the grid's, which covers the counts, sums and quantiles a
triage screen needs, and keeps anything that could differ client-side.
source.aggregate with a groupBy is the
top-talkers call, and source.lastPlan() is the receipt.
Querying a private bucket, in the guide → Whole-set statistics, in the guide →
import * as duckdb from '@duckdb/duckdb-wasm';
import { createGrid, createPushdownSource, duckdbAdapter } from '@toclocoinc/lattice-grid';
import * as compute from '@toclocoinc/lattice-grid';
const db = await makeDuckDb(); // your duckdb-wasm bootstrap
const connection = await db.connect();
// The credential is DuckDB's, on the connection. The grid never sees it.
await connection.query(`
CREATE SECRET logs (
TYPE s3,
PROVIDER credential_chain,
REGION 'eu-west-2'
);
`);
const source = createPushdownSource({
adapter: duckdbAdapter({
connection,
from: "read_parquet('s3://your-bucket/vpc-flow-logs/2026/09/07/*.parquet')",
}),
compute,
pageSize: 200,
// count, sum, min, max and the quantiles are verified identical between
// DuckDB and the grid, so they run in the engine over the whole match.
aggregates: { default: 'engine-if-identical' },
});
createGrid(el, { rowKey: 'event_id', columns, source });
// Top talkers over everything the current filter matches, computed by DuckDB.
const { groups } = await source.aggregate(
{ filters: grid.filters.get(), sort: [], range: null, groupBy: ['srcaddr'] },
[{ id: 'flows', col: 'event_id', fn: 'count' }, { id: 'bytes', col: 'bytes', fn: 'sum' }],
);
// After any query: what reached DuckDB and what, if anything, the grid did.
source.lastPlan();where the line is
A bounded slice in the tab. The wide scan stays where it belongs.
This is a complement to your batch query engine, and the honest scope is worth stating plainly.
In the browser: a day, or a prefix
DuckDB in a tab suits a bounded slice: a day's logs, one bucket prefix, the window an incident is being worked in. Windowed queries stay light because only the visible page comes back. If you turn on the whole-set pull for client-side grouping, it is memory-guarded: a set past the configured row or byte ceiling is refused with a visible error, never truncated to look complete. The documentation example caps that pull at 250,000 rows.
Across the lake: Athena, or DuckDB on a server
A scan across months of partitions, or across the whole lake, is a job for Athena or for a server-side DuckDB behind your own endpoint. The same adapter drives the server-side engine, so the screen an analyst learned on a day's slice is the screen that fronts the bigger question. Nothing about the choice of engine leaks into the grid.
The grid never trades a right answer for a faster one. A figure it cannot compute over the whole matching set is not presented as if it had been, and a partial result over residual work is refused rather than shown. That is the property a security team needs from a screen it will act on.
questions
Asked before choosing this.
Can I analyse security logs stored as Parquet in S3 without Athena?
Yes, for interactive triage of a bounded slice. DuckDB reads the same Parquet files Athena does, in the browser through duckdb-wasm or on a server, with no per-query service in the path. Lattice Grid sits over that connection and pushes each filter, sort, page and count down as SQL, with top-N aggregates computed over the whole matching set. Large scans across a whole lake remain a job for Athena or a server-side DuckDB behind your own endpoint, which the same adapter drives.
How do credentials for a private bucket work?
They are DuckDB’s own configuration, not a grid option. You create a SECRET on the connection, from the ambient credential chain or an explicit key, before handing the connection to the grid. The grid then runs your from expression as-is. It never sees the credential and makes no HTTP calls of its own; DuckDB fetches the byte ranges each query needs.
How much log data suits DuckDB running in the browser?
A bounded slice: one day of logs, or one bucket prefix. Windowed queries stay light because only the visible page comes back, and the whole-set pull is memory-guarded, refusing a set past the configured row or byte ceiling with a visible error rather than truncating it. The documentation example caps that pull at 250,000 rows. Petabyte-scale scans belong to Athena or a server-side DuckDB behind your endpoint.
Put a grid over the logs you already keep.
Open the demo, filter it hard and read the SQL. Then swap the from for
your own bucket.