Your data, in your own bucket

Your data is yours. If you ever leave, or if Overscore were ever gone, you should have all of it, in a form you can read without us. So any bucket you send files to can also hold a complete copy of your warehouse, written every night after our own backup.

Turn it on

On Destinations, Places, open your Amazon S3 or Google Cloud Storage bucket (add one first if you have none: Sending data out) and turn on Copy everything here, nightly. It takes an admin. Turning it on reads the bucket and writes one small file, overscore/README.md, so a key that cannot write is caught now, not at night. One bucket at a time. Copy now, beside the switch, runs the same copy once, right away, whether the switch is on or off, and the panel shows it going.

The key needs to list, write and delete under <prefix>/overscore/: for S3, s3:ListBucket, s3:PutObject and s3:DeleteObject; for Google Cloud Storage, Storage Object Admin on the bucket. A file send alone needs only the right to write.

What lands there

• lake/: every table's data as Parquet files, lake/<dataset>/<table>/.
• files/ and originals/: every file you uploaded or emailed in, every version, and the file behind each table loaded from one.
• _catalog/latest.sql.gz: the database that says which file belongs to which table, with your models' SQL, metrics, dashboards' code and notes in it; and beside it the record of every file's size and hash.
• README.md: what is there and how to read it, with your own folder names in it.

It is one copy, written over each night: last night's warehouse and nothing older. The first night copies everything; after that, only what changed. A file the warehouse no longer holds (one you deleted, a model's replaced copies after 3 days) is removed from the copy the next night, so it is never bigger than your warehouse. Nothing outside overscore/ is ever touched. Turning the switch off stops the copy and leaves the bucket as it is: nothing is deleted.

Your stored passwords and keys for other systems are not in the copy: they are encrypted with a key that stays with Overscore, so they would be unreadable there. The copy is your bucket's, at your bucket's cost; it does not count toward your plan's storage.

When a copy fails, the panel says what your bucket answered, a card reaches the Inbox (as A run failed, to whoever hears that on Settings, Alerts), and it is tried again within the hour. A Copy now that fails is said on the panel and not tried again by itself.

Read it without us

Each table's files, with DuckDB alone (the README writes these lines with your real bucket and table names):

INSTALL httpfs; LOAD httpfs;
CREATE SECRET mine (TYPE s3, KEY_ID '<your key id>', SECRET '<your secret>', REGION '<your region>');
select columns(c -> c not in ('file_path', 'pos', 'filename'))
  from read_parquet('s3://<bucket>/<prefix>/overscore/lake/<dataset>/<table>/*.parquet', union_by_name = true, filename = true)
  where filename not like '%-delete.parquet';

That reads every row ever written to the table, old versions and deleted rows included, since a table keeps its history as more files. For each table as it stood last night, load the catalog into any Postgres, point it at your bucket with the update lines the README gives (they name where the files were and where they are now), and attach it:

createdb overscore
gunzip -c latest.sql.gz | psql overscore
# then the README's update lines, then in DuckDB:
INSTALL ducklake; INSTALL postgres;
ATTACH 'ducklake:postgres:dbname=overscore' AS lake (METADATA_SCHEMA 'public', READ_ONLY);
select * from lake.<dataset>.<table>;
select * from lake.<dataset>.<table> AT (TIMESTAMP => '2026-10-20 02:00:00+00');

Overscore does exactly this every week on its own backup of your warehouse, so it is known to work. Take everything (Take everything with you) gives the same as files you open in a browser and a dbt project, which is easier to read than the tables.

From a terminal

ovs destination set "<bucket place>" --copy-everything       # on, from an admin's own laptop
ovs destination set "<bucket place>" --no-copy-everything    # off; nothing in the bucket is deleted
ovs destinations                                              # says "copies everything nightly" on that place

No agent turns it on: the switch is a person's, since it points the whole company's data at a bucket.