demo D231

A hundred thousand rows in DuckDB, no server in Web Component

DuckDB reads a Parquet file in the browser and the whole set is pulled in, so the plant subtotals and the grand total are computed over all 100,000 rows

createPushdownSource · duckdbAdapter · fullDataset

This grid reads a Parquet file with DuckDB in the browser and pulls the whole hundred-thousand-row set through it, so the subtotals and grand total are computed over every row. It shows analytical figures that cover the full dataset, with no server involved.

Building…

Loading a live grid…

This is the web component version. A lattice-grid custom element renders the grid with no build step, its configuration set as a property, so it works on a page that has no bundler at all.

The configuration

<link rel="stylesheet" href="https://cdn.latticegrid.dev/lattice-grid@1.107.0/lattice-grid.min.css">

<script type="module">
  import { defineLatticeGrid } from 'https://cdn.latticegrid.dev/lattice-grid@1.107.0/modules/webcomponent.esm.min.js';
  defineLatticeGrid();
</script>

<lattice-grid row-key="reading_id" style="display: block; height: 560px"></lattice-grid>

<script type="module">
  import { duckdbAdapter, createPushdownSource } from 'https://cdn.latticegrid.dev/lattice-grid@1.107.0/modules/webcomponent.esm.min.js';

  // The web component module carries the grid; DuckDB is your own engine. The
  // page starts DuckDB-Wasm itself and hands the adapter a live connection, so
  // reaching for pushdown never grows the module you load.
  import * as duckdb from 'https://cdn.jsdelivr.net/npm/@duckdb/duckdb-wasm@1.29.0/+esm';
  const bundle = await duckdb.selectBundle(duckdb.getJsDelivrBundles());
  const worker = await duckdb.createWorker(bundle.mainWorker);
  const db = new duckdb.AsyncDuckDB(new duckdb.ConsoleLogger(duckdb.LogLevel.ERROR), worker);
  await db.instantiate(bundle.mainModule, bundle.pthreadWorker);
  const connection = await db.connect();

  // from is any SQL table expression. read_parquet over an HTTP URL streams the
  // file, so a host that honours range requests lets DuckDB read only the bytes a
  // query touches rather than the whole file.
  const adapter = duckdbAdapter({
    connection,
    from: "read_parquet('/demo-data/readings.parquet')",
  });

  // The adapter answers every filter and sort on this grid in SQL, so only the
  // visible window comes back on each interaction.
  const source = createPushdownSource({ adapter, pageSize: 200 });

  const el = document.querySelector('lattice-grid');
  el.config = {
    toolPanel: { side: 'left', panels: ['filters', 'columns'] },
    columns: [
      { field: 'reading_id', title: '#', type: 'number', layout: { width: 110 } },
      { field: 'plant', title: 'Plant', filter: { type: 'set' } },
      { field: 'line', title: 'Line', filter: { type: 'set' } },
      { field: 'product', title: 'Product' },
      { field: 'bore_mm', title: 'Bore', type: 'millimetres',
        spec: { lower: 9.95, upper: 10.05, target: 10 } },
      { field: 'cycle_seconds', title: 'Cycle', type: 'seconds' },
      { field: 'in_spec', title: 'In spec', type: 'boolean' },
    ],
    source,   // a hundred thousand rows; DuckDB answers each filter and sort in SQL
  };
</script>