Getting started
Make a repository, write a first version with DuckDB, and read it back.
This page makes a repository in a folder, writes a table into it and reads the result as a reader would. The same commands work on S3 and R2: pass s3://bucket/prefix/ instead of a folder.
Install
pip install 'fdl-cli[s3]'
The package installs the fdl command and the fdl Python package. Python 3.10 or later is needed. Leave out [s3] if you only use folders.
Make a repository
fdl init ./towns/
init writes two files and nothing else:
towns/
fdl.json {"fdl": "0.1", "id": "5f0c…", "data_dir": "data/"}
HEAD ref: refs/heads/main
HEAD names the default branch, main. The branch itself does not exist yet: there is no version for it to point at.
fdl init makes the repository where readers will find it, as git init --bare does on a server. Your own place to write is the working metadata, made next.
Clone a working metadata file
You never write to the repository directly. You write to a working metadata file, a DuckLake metadata file in a SQLite file on your own disk, and turn it into a version when you are done.
mkdir work && cd work
fdl clone ../towns/ lake.sqlite
Before the first version, clone makes an empty metadata. Later it copies the metadata of the version main holds. It prints the statement to attach the working metadata with:
ATTACH 'ducklake:lake.sqlite' AS lake (
DATA_PATH '/abs/path/towns/data/', OVERRIDE_DATA_PATH true,
META_TYPE 'sqlite', DATA_INLINING_ROW_LIMIT 0,
META_JOURNAL_MODE 'WAL', BUSY_TIMEOUT 5000
);
Beside the working metadata, lake.sqlite.fdl.json remembers the repository, the branch and the version the clone was made from.
Write
Use any DuckDB client. The Parquet files go straight into the repository’s data/ folder.
CREATE TABLE lake.main.towns AS
SELECT * FROM '../towns.csv';
DATA_INLINING_ROW_LIMIT 0 makes DuckLake write every row to Parquet. Left at its default, DuckLake keeps small tables inside the metadata, and a version has to carry its rows in data files that a reader can fetch.
Declare
Tell FDL which columns identify a row. A merge matches rows by the primary key, and readers can look rows up by it:
fdl declare primary-key lake.sqlite main.towns code
Declarations stay with the working metadata until the commit, which checks them against the tables (see Declarations). They are optional: a version without them holds the tables alone.
Commit and push
fdl commit lake.sqlite -m "First towns" --author alice
fdl push lake.sqlite
commit freezes the working metadata into a version on your disk. push measures each table into a small summary (a profile) that readers can show without reading the data, uploads the version with them and moves main to it, and prints refs/heads/main -> 019a3c4e-… (version 1). The repository now holds:
towns/
fdl.json
HEAD
refs/heads/main 019a3c4e-…
versions/019a3c4e-…/
metadata.db
tables.json
profiles/main/towns.json
version.json
tables/main/towns/ducklake-….parquet
fdl log ../towns/
1 019a3c4e-… 2026-11-01T09:00:00Z alice
First towns
Read
fdl url ../towns/
prints the metadata of the version main holds. Attach it read-only, and tell DuckDB where the data files are:
ATTACH 'ducklake:../towns/versions/019a3c4e-…/metadata.db' AS towns (
DATA_PATH '../towns/data/', OVERRIDE_DATA_PATH true, READ_ONLY
);
SELECT count(*) FROM towns.main.towns;
A version’s metadata never changes, so a reader may cache it forever. Only HEAD and the files under refs/ change.
Make a second version
fdl clone ../towns/ lake.sqlite
INSERT INTO lake.main.towns SELECT * FROM '../towns-2026.csv';
fdl commit lake.sqlite -m "Add 2026" --author alice
fdl push lake.sqlite
The new version lists the first as its parent. Both metadata files name the first Parquet file; only the second names the new one.
If someone else pushed in between, the push is refused, because the new version would not be built on what main holds now. Clone again and redo the change. Versions and refs explains why.