Connect a source

A key you give here (an API key, a service account's key file, a database password, a webhook's signing secret, a Google sign-in) is encrypted before it is kept, with a key that is not on the machine, and never shown again: not to you, not to an agent, not in a backup. A service account's key file is chosen as a file and read once in your browser. See Security.

On Sources, press Add a source, pick the system and follow the guide beside its form. Connecting is two steps, as in Fivetran. Test the connection tries the key, or your sign-in, and says Connected with what it found (a Shopify store's name, what it holds and how far back its orders go; the Google Ads accounts you ticked; a sheet's columns) and the tables it will pull, or says plainly what is wrong; nothing is saved yet. Start the initial sync saves the source and starts its first pull on its own, and takes you straight to the source's page, where the Syncs tab shows each table as it lands, with its rows so far. You can leave the page and come back: the pull keeps going, and the page shows where it is. Once it has landed, the page ends with Describe these tables while a table it landed has no words: the chat opens beside the page and drafts what each table is and what its columns mean, one table at a time, asking you about the few columns a wrong guess would cost most, and your answers file them as yours. The offer stays on the Tables tab until every table has its words. Nothing is drafted until you press it. If the first pull fails, it is tried again at its next scheduled time, the source's page shows the failed run and what went wrong, and whatever had already arrived is kept for the retry. When a sync breaks later, a card shows up in your inbox saying what broke and, usually, the fix. An uploaded file is one step, since the file is the pull.

A waiting pull checks access again before it fetches rows. If its key has expired or been revoked, or the person who started it has been removed, it records a failed run and fetches nothing. Someone with access can start it again. A Google Sheet still reads the sheet as the person who connected it.

When the first sync lands, Overscore proposes the source's obvious metrics as cards in your Inbox, each with its definition written out and last month's figure, or what stands now for one with no date (Stripe's MRR). Confirm makes it a metric with a receipt, and Not this removes it for good. Nothing is defined until you say so. One whose definition makes a choice says it and asks ("Trialing and past-due subscriptions are left out. Is that right?"), so Confirm is your answer to it. A metric you already have, by its name or by its parts, is never proposed, and a source connected again proposes nothing. Each system's guide lists the metrics it proposes, under More detail. Stripe's money starters give no dollar figure when the selected rows contain another currency or no currency. Choose one currency or a conversion rule first.

A source that ships a starting page (Pipedrive's is Pipeline: every open deal by stage, what each is worth, when it should close, and who to call) offers it at the end of its page once the first pull has landed. Make the page saves it as a draft only builders see, with the saved queries it reads, and opens the builder on it; change it there and publish it as any draft. Not now takes the offer away for that source. Nothing is made on its own, nothing is published until you publish it, and a source with no starting page is never asked. From a terminal, ovs dashboard start pipedrive makes it any time.

On Free you connect up to three of the apps here, each pulled once a day at your company's own time, and by hand once an hour; databases (your Postgres, a copy of BigQuery), webhooks and sources of your own are for Self-serve and Done for you. See Limits and plans.

Each system's setup steps sit beside its form, with how long it takes, and name each box the way the form does. More detail, under them, has the terminal commands, what lands, and how to keep it running.

Sources lists each one you have: its name and dataset, how it is doing, its last sync, its schedule and the rows that sync landed. Click a row to open its page. The ··· at the end of a row has what its page has: Sync now, See its tables, Settings, Pull everything again and Disconnect. Pull everything again and Disconnect ask first; the tables stay in Warehouse either way. On a phone a row opens its page, where all of them are, Disconnect under Settings.

A source's page has a Tables tab with every table it lands: the ones you turn on and off, and under each the tables that come with it, which land and stop with it (Shopify's orders bring their order lines, refunds and fulfillments, its products their variants). The row on Sources and the tab count all of them, the number Data shows for the dataset. Each shows the rows it holds now, as Data shows them, and when its syncs last landed anything; what each sync landed is in that sync's log, under Syncs.

Search the wall of sources by name. When nothing is called what you typed, the page says so and offers Add a source of your own, which opens the builder with that name already in the box (on Free, which connects its own apps alone, it offers Clear search). Emptying the box and choosing All shows every source again.

The builder reads that system's API docs and walks you through making its key step by step, in that system's own words, above the field you paste the key into. The source's guide keeps the same steps for whoever replaces the key later.

A file someone emails

For a report someone already sends by email, like a weekly spreadsheet from a supplier or another office. On Sources, press A file someone emails (the third big card, beside Upload a file and Add a Google Sheet). Give it a name, the addresses that may send to it, what to expect (a spreadsheet, a CSV or a PDF, and if you like a pattern its file name always fits), the dataset it goes in, and who to tell when something goes wrong (the file is late, an email is turned away, or what reads it fails). Press Make the address: it is the name with a short tail, like riverfront-weekly-k7m4xq@in.overscore.tech, with Copy beside it.

Two ways to get mail to it: ask the sender to add the address to the email they already send, or set a Gmail filter on your own inbox that forwards those emails to it. Gmail first mails the address a code before it will forward; the code shows on the form in full, so you can finish the filter there. The form waits for the first email and says when it came, from whom, and that version 1 of the file is in Files, or, for an email with no file on it, that it was kept.

Each email is kept whole, in Files, as an email. One attachment on it that fits what you said to expect becomes the next version of the file, and whatever reads that file runs again. Past weeks can go in by hand (ovs upload put and the file's name), in any order: a reader that goes by the report's own date takes an older week as the past it is. If your company's machine is asleep when an email comes, it is kept for it and handed over as soon as the machine is awake, in the order the emails came; the source's page says how many wait ("1 email waiting for the machine"). Nothing is turned back to the sender for it. A logo or a signature in the email is never a file. The same file again is noted and changes nothing. An email with no file that fits, or more than one, is kept like the rest, with its words, and no new version lands. Nothing waits for you to choose: what that email means is up to what reads the emails (below).

Mail from an address not on the list, or not signed by the domain it claims to come from, is turned away and listed on the source's page as turned away; the people you named hear about it once a day at most. A forward from Gmail counts when the original sender is on the list, and so does one a Gmail filter sends when the person whose filter it is is on the list (Gmail sends it from their address with a tag after a +). In Sources the source is a row like a sheet's: when its last email came and from whom, and how many came. Its ··· has Copy the address, See it in Files, Pause and Disconnect. Pause turns its mail away until you resume; Disconnect stops the address taking mail, and the emails and the file stay in Files.

How often it comes is a choice on the form: every week (the usual), every month, or no set schedule. The week, or the month, is counted from the last email that came to it (with a file on it or not; a report typed into the email counts) or file you put in by hand. A day past it with nothing, the source shows Behind in Sources, gets one row in your inbox like any source that falls behind, and the people you named get one email: which source, when the last one came, when this one was expected. Nobody is mailed twice the same day, and the row closes itself when the next email comes. An email that came without the file you need is for what reads the emails to catch, with a check of its own. A source on no set schedule, or paused, is never late.

If what reads the file fails on a new version (the model's script stops, or one of its checks fails), nothing from that version goes in: the model is not updated, and a dashboard built on it keeps showing the numbers it had, with the date they were last updated. A model that reads that model is checked in the same turn, as soon as the file lands. The people you named get one email for that version, with the model's name, the version and what went wrong: for a check, which check and what it found ("2026's months add to $9,606,109; the sheet's year-to-date says $9,606,108 (cell B112)"). Your inbox gets one card saying the same. Running the same version again mails nobody; the next version that passes goes in as usual. When the new figures are right (they restated a month), press Those rows are fine, update it on the card or on the model's page: it is built again with those rows let in, who pressed it and when is on the record, and the next week's file is checked against the new numbers.

A Python model can read the emails themselves: every email the address kept, with who sent it, when, its subject, its words line by line and its files (emails(), under Models). Each new email runs it. That is how a report someone types into the body of an email becomes a table, and how an email that should have brought the spreadsheet and did not is caught the moment it comes: a check on that model ("The newest email has the spreadsheet on it") fails, nothing goes in, and the people you named get one email saying which email it was, beside one card in your inbox. Save such a model once the first email has come (Gmail's forwarding code counts).

The source's page says, in one line under its name, what reads it last did: ran a few minutes ago and made so many rows, or stopped, and why, with See the run. Below it every email is listed with what came on it (its files by name and size, or its text only) and what it became: a version of the file, the same file again, kept, or turned away and why. Open one to read its words and download the whole email.

How far back

Shopify and QuickBooks ask how far back to go when you connect them. All of its history is the default: the first pull takes everything the system hands out. From a day starts the dated records there: Shopify's orders, QuickBooks' invoices, payments, bills and the rest of its transactions. Customers, products, vendors, items and accounts always come whole. After the first pull, every pull takes what changed, whatever the day. The source's page says how far back each table went ("Orders: back to 2 Jan 2024"), and from ovs: ovs connect shopify shop=… token=… --start 2024-01-01.

Shopify hands out only the last 60 days of orders to an app without the read_all_orders scope, and says nothing about the rest. Testing the connection says so ("Only the last 60 days of orders"); tick the scope in the app, install it again and paste the new token. A first pull that the day's row limit stops goes on the next day from the page where it stopped, not from the start.

When a sync breaks

A source that refuses its key, or answers with an error, gets one card in your inbox with what looks wrong and the fix, not one card an hour, and the card closes itself when the next sync works. A source that lands nothing for longer than usual gets one card too. Longer than usual means longer than its longest quiet spell in the last two weeks, and at least three pulls in a row. So a website with no visitors overnight does not get a card every night. The card stays until a pull lands rows: a pull that again finds nothing new leaves it open. Nothing is lost while it's broken: a sync only moves its mark when it finishes clean, so when you paste a new key it catches up on everything since the last good sync, right then. One table failing doesn't stop the others; only the source refusing the key itself stops the rest, since every table would be refused the same way. Each sync of Shopify, QuickBooks or Stripe asks from five minutes before the last one began, so a change made while a sync was running, or in the same second as the last one it saw, is picked up by the next; what comes twice is the same record, kept once. One pull of a source runs at a time. Sync now while one is going says so, A pull of it is already going, and starts nothing, the hourly turn waits for the next hour, and a database copy is the same. Sync now, Pull now and Pull everything again run on one of your company's workers, ahead of the scheduled work, never on the machine that answers your page. The page answers at once, Syncing on one of your workers, and its sync card says it is waiting for a worker, then starting on worker-2, then fills in table by table as they land. A pull started any other way runs on a worker too: an agent's first pull, Run it again now or Pull it again from the start on a card in your Inbox, a database copy you start with ovs copy, and POST /pipelines/{dataset}/{name}/run. Each waits for its worker and answers with what landed and where it ran, pulled on worker-2. A file you load, rows you send and a table you fill in are written by the machine that answers you, while you wait. Every source and every model is graded against its own schedule every five minutes: on time, running late once it is a quarter of a schedule overdue, behind past two; behind gets one row in your inbox, however many times it happens, and the row closes itself when it catches up; it shows on the source's page under what reads it, and we hear about it in our own daily report even when no run failed. A rate limit is waited out, not reported, and each table's line in the run's log says its pages, its rows and how many requests were tried again and why ("3 requests retried (429 x2, connection dropped x1)"). If a sync is paused longer than the source keeps its changes (Stripe keeps 30 days), everything is listed again so no new or changed record is missed; a delete made in that gap is not seen. A first pull too big for the day's allowance goes on the next day from the page where it stopped, not from the start, and the sync after that asks again for anything changed since the first day began, so a change to a page it read that day is not missed; if the source no longer takes that page, that sync says so and the next one starts the table again from its last mark. Every sync is recorded step by step, one per table, with its rows and what went wrong; a build one per model and per test, with what each one read. The Runs screen (under Models) lists your own work as it ran, by hand or on a schedule: model builds, syncs, checks, sends. The app's own housekeeping (upkeep, expiry, backups, the nightly checks) and a sheet's pulls, which live on the sheet's own page under Files, are behind "Show everything". A run's page draws its steps on a time line, an arrow from each step to the ones that read what it wrote, with what went wrong on the step that failed; GET /runs/{id} returns the same steps.

Replace a key or password

Open the source’s Settings. A BigQuery mirror checks the new service account key against its saved project and dataset; its chosen tables, new-table policy, schedule and paused state stay as they were. Google Ads offers Reconnect Google and its account picker with saved accounts ticked; accounts that lost access stay visible until you restore access or deliberately remove them. A Shopify token-refusal card’s Replace the key button opens the right store’s Settings.

Postgres has a separate password form: only the password changes, after the saved connection is checked and its login is found unable to write. Its mode control can move a live link to copy mode. Once copied tables exist, add a live connection with another dataset name; those copies stay here.

Every connection has a name

The first box on every connect form is the connection's name, filled in with a suggestion. The name is where it lands: a Shopify connection called shopify puts its tables in the shopify dataset, and a second store you call shopify_uk lands in shopify_uk. A warehouse you copy from (BigQuery) uses the name as a prefix, one dataset per dataset of theirs: a connection called bigquery_finance lands its quickbooks dataset as bigquery_finance_quickbooks, so it never collides with the QuickBooks connector or with a second BigQuery. The form shows where every dataset will land and says so if a name is already taken, before you press Connect. A connection only ever writes into datasets it made, and its name is set once, when you connect. Open the row with that dataset name in Sources: its page, failures, pause, key replacement and disconnect belong to that connection. Two stores never share these controls.

What lands, and what's behind it

One table per thing the source has, under its plain name, typed: shopify.orders, shopify.customers, shopify.products, with _synced_at on every row. Money is exact, and every moment is the instant in UTC, whichever zone the source wrote it in; a metric counts it in your company's day (Settings, time zone). A record the source deleted leaves its table, and its items leave theirs; a refund, a void or a cancelled order is not a delete, it is the record's newest state (A row deleted at the source, below, says when, source by source). Lists inside a record (an order's line items) are their own tables (shopify.order_lines). Fields are discovered across all landed rows, including new fields that arrive long after setup. Each is built after every sync and shows under Models as a built-in model: open it to read the SQL, copy it, or build on it; nobody saves over it. Every copy the source ever sent is kept behind the scenes in a dataset whose name starts with an underscore (_shopify.orders): out of Warehouse, Ask and the agent's lists unless an admin turns on Behind the scenes in Settings, Company, and always there to query by name. The deletes are kept there too, each with when it went, so a total from before a delete can be worked out again.

A row deleted at the source

Something deleted where it came from (a test order, a duplicate contact, an invoice entered twice) is learned one of three ways: the source tells us, we look each night at which records it still has, or the whole table is copied again. The record then leaves its table, and your numbers, at the next sync that learns of it, and a number's receipt says how many it left out. Nothing is thrown away at once. Behind the scenes the copy keeps the record with when it went (_deleted_at, unless its line names another), and a table copied whole keeps its earlier versions for 30 days.

• Files and Google Sheets: read whole, so a row taken out is gone at the next pull or upload. Earlier files stay in Warehouse, Files.
• Stripe: a deleted customer, product, price, coupon or draft invoice leaves at the next sync, or in seconds with the webhook. Charges, payments and payouts are never deleted. A cancelled subscription stays, marked cancelled.
• Shopify: a deleted order, customer or product leaves at once with the deletion webhooks, otherwise by morning.
• QuickBooks Online: a deleted invoice, bill or payment leaves at the next sync. A voided one stays, at zero.
• BigQuery: a deleted row stays in the copied table with _deleted_at filled, so add where _deleted_at is null. Its guide says which tables see deletes.
• HubSpot: an archived (deleted) record leaves at the next sync, marked archived_at behind the scenes.
• Pipedrive: a deleted deal leaves at the next sync, marked is_deleted, and a person, organization, product or activity by the next morning. A pipeline, stage or user leaves when Pipedrive marks it deleted.
• Postgres: copied, a deleted row leaves at the next copy and stays in _app.orders (with no primary key, it just goes). Live, it is gone at once.
• Webhooks, Slack too: nothing is taken out. A reported delete arrives as one more row.
• Anything with an API: a record deleted there stays as it last came.
• Google Ads and Meta Ads: nothing is taken out. A removed campaign, ad or keyword stays, and so do its days, since that money was spent.
• PostHog: nothing is taken out. An event or person deleted in PostHog stays as exported.

Shopify

Ten minutes. In your Shopify admin, allow custom apps, create one called Overscore with the four read-only scopes (orders, all orders, customers, products), install it, and copy the token that starts with shpat_. Paste it with your shop handle. Orders, customers and products come in with full history.

Stripe

Five minutes. In the Stripe dashboard make a restricted key with every resource set to Read and nothing on Write; it starts with rk_live_. Paste it. Charges, customers, subscriptions (the cancelled ones too), invoices and refunds come in, and their changes (a refund, a new plan, a cancellation) update within the hour through Stripe's event feed.

QuickBooks Online

Twenty minutes the first time, because Intuit makes you create a developer app and sign in once. The in-app guide walks through it click by click. You end up with four values to paste; after that it syncs invoices, payments, customers, vendors and the chart of accounts. For an Intuit sandbox, tick This is a sandbox company. That choice stays with the connection through later pulls and key replacement.

Pipedrive

Two minutes. In Pipedrive click your name at the top right, then Personal preferences, then API, and copy your personal API token. Paste it. Deals (with the products on each), people, organizations, activities, products, pipelines, stages and users come in, every custom field as a column named after the field, a dropdown's choice as its label. The token reads as you, so the tables hold what you can see in Pipedrive.

BigQuery

A mirror: the tables you tick are copied here on a schedule, so questions and dashboards never run up a BigQuery bill. Each sync reads the cheapest way the table allows: nothing when BigQuery says it hasn't changed, only the changes when change history is on, only the partitions that moved for a table partitioned by day, otherwise the whole table through the Storage Read API. Google Analytics's one-table-per-day export is one table here with a table_date column, and a new day comes on its own. Selecting every table defaults to copying tables that appear later. Leaving a table out defaults to asking first with a card in the inbox. The checkbox below each dataset lets you override that choice; choosing a new key resets those overrides.

Google Ads

Nothing to paste. Click Connect Google Ads, sign in with the address you use for Google Ads, and tick the ad accounts to read: your own, and any under a manager account you hold. Overscore reads them as you and never changes a campaign or a budget.

What lands: each account, campaign, ad group, ad and keyword, and one row for each of them for each day it ran, with cost, impressions, clicks, conversions and their value as Google Ads shows them, in the account's own currency and time zone. Each level is its own table; they don't add up to each other, so use the one at the level of your question. The first pull reaches back two years. A clicks table has every click Google tagged in the last 90 days, with its gclid and the campaign, ad and keyword it was on: with your website analytics connected, a visit joins back to the ad that brought it in one query.

It refreshes every hour at the fastest. Google Ads keeps revising the last few weeks as late conversions are credited, so the last 30 days are pulled again once a day and the numbers follow.

And the other way: under Destinations, a table of leads or sales goes back to Google Ads as conversions, each counted against the click that brought the visitor, so Google's bidding learns which clicks turn into business. It starts in test mode, where Google checks every row and records none, and goes live when you say.

PostHog

PostHog sends an export of your events every hour to a drop-off of your own, a storage bucket that holds only what PostHog sends, and Overscore picks up each new file from there. An admin makes the drop-off on the PostHog form with Make my drop-off: it shows the endpoint, the region, the bucket, the access key ID and the secret, each with Copy, the secret this once (Make a new secret gives another, and the old one stops working). Under PostHog sends, the guide beside the form has PostHog's clicks one by one: Data, Destinations, New destination, S3-compatible, and each field with the value to put in it. Under Overscore picks up, the endpoint, bucket and key ID come filled in, and you paste the secret, test the connection and start the initial sync.

It remembers the files it has read, so an export PostHog sends late for an older hour still lands, and a file PostHog changes is read again. The events and people tables keep one copy of each.

Your own database

Postgres, wherever it runs. Two ways: live attaches it read-only and queries run against it in place (point it at a read replica, never the primary), or copy copies the tables you name every hour, which is right when the database is busy or you want history: app.orders shows the rows your database has now, and behind the scenes _app.orders keeps every row ever copied, a row deleted there with when it went (_deleted_at). A table the login cannot read is named on the run while the others are copied, and a column dropped in your database is said on the run and in the table's note. The schema you name is a plain name: letters, digits and underscores. When a live database is down, a question that reads it says so and everything else goes on; it is tried again within a minute and works again by itself once it is back.

Either way you make a login that can only read, and Overscore refuses any other: it never needs to write to your database, so it will not hold a login that could. Run this in your database as an admin, with your own password, and connect with that login rather than the one your app uses:

create role overscore_reader login password '…';
grant connect on database app to overscore_reader;
grant usage on schema public to overscore_reader;
grant select on all tables in schema public to overscore_reader;
alter default privileges in schema public grant select on tables to overscore_reader;

When you connect, Overscore asks your database what the login may do. A superuser, a login that owns the tables, or one that can insert, update, delete or create anything is refused: nothing is saved, and the message says which right it found and gives these lines with your own database and schema in them. On Neon, make the login in the SQL Editor; one made on the Roles page can write.

The login is looked at again every night. If it has since been given a right to write, the connection is paused: a live one stops answering and a copy stops copying, both say why, and a card in your Inbox names the right and gives the line that takes it away. Run that in your database and press Run it again on the card; a copy also starts again by itself at its next scheduled time.

An agent's key that only reads a dataset sees its sources and how they are doing, and changes none of it: which tables a connector pulls, its key, its schedule, pausing, syncing and disconnecting all take read and write on that dataset. A schedule belongs to its dataset, so a key that writes another dataset cannot save over it. A source's settings come back without the keys and passwords written into them; a key that can write the dataset sees only that one is set.

Spreadsheets and files

At the top of Sources, above everything else you can connect, are two big cards: Upload a file and Add a Google Sheet. A CSV, Excel, JSON or Parquet file becomes a table, with the file itself kept behind it; a PDF or a Word file is kept and its text read. A Google Sheet becomes a table that refreshes on its own (on Free, once a day with the connected systems), and it is listed with your other sources, with its schedule and its last pull. A pull that finds the sheet as it was writes nothing and says Unchanged. Upload a file asks for the file first: a spreadsheet's first rows are shown with what each column was read as (text, number, date, yes/no), which you can change, and then you say which dataset it goes in. Once the file is in, you are taken to the table it made. Every file you gave is kept under Warehouse, Files: see The warehouse.

A Google Sheet can stay private. Click Connect Google, sign in, and pick the sheet in Google's own file picker, or paste the sheet's link and confirm it there. Renaming a tab doesn't break anything: Overscore follows the tab, not its name. Overscore reads only the sheets you pick, and reads them as you: each sheet says who it is read as. If your Google access ends, the sheet pauses and says so, the table keeps its last rows, and connecting Google again catches it up. You can disconnect under Settings, Account.

Or paste a link to a sheet that anyone with the link can open. A sheet added that way can be moved to your own sign-in later with Read this as me instead.

Webhooks and other apps

Any app that can post JSON can post to a table. On Sources, press Add a source, then Webhook, to make an address; paste it into the other app, and rows land as they happen. Stripe and other signed senders are checked, and fakes are kept in a log with the reason. A signed Stripe or Shopify address also fills the table an event or a topic names, and that table's own signature is checked too; an address with no signature only ever fills its own table. Slack channels can be kept the same way. A delivery is kept safe the moment it arrives, even while your warehouse is starting or being updated, and the tables that read what it brought are rebuilt as soon as it lands, on a worker of your company's own, so a dashboard over them moves with each sale. One that cannot land is tried again for hours, and one refused for a bad signature or body is listed with the reason among the address's refusals (ovs hook-log). Retries through hooks.overscore.tech keep one copy of a captured delivery, including a replay from the queue. A sender posting again starts a new delivery. Count those by the sender's own event or record id.

An address that begins with https://hooks.overscore.tech/ is caught at the edge, by Cloudflare, and handed to your company's worker. An address made before it, at your company's own name, keeps working.

Tell a webhook address how often to expect rows (Expect rows every, on its form or its page's Settings) and Overscore watches it like a source, late once that long passes with no rows and behind a quarter of that later, with one Inbox card that closes itself when rows come again. Leave it unset (No expectation) for an address that posts whenever it likes.

What to say to it