Sending data out
Rows sent to Meta, Google Ads, HubSpot, your own BigQuery or a URL of yours, and your files copied to your own bucket: what goes, what is hashed, how to test it, and how to stop it.
A destination is a place your data goes: Meta, Google Ads, HubSpot, your own BigQuery, a bucket of yours on Google Cloud Storage or Amazon S3, or a web address of your own. A send is a table, or a query, sent there: which column is the row's id, which columns go, and when (after the table changes, on a schedule, or only when you press Send). Only rows that changed since they last went are sent. Each row needs an id of its own: a column that repeats, like an email two customers share, is refused when you save, with how many rows share one and an example. To a bucket, a send is one of your files instead: every version of it, copied as it lands.
Destinations has three tabs. Sending lists every send, across every place: what it sends, where to, its status (Live, Testing, Paused, or a dot and what is wrong) and when it last sent. Places lists the places themselves, and Add a place is there. Sent is every row that left; a long place or send name is cut with dots there, with the whole name when you point at it, so when each row left is always in view. A send the assistant set up for you in a chat has Open the send beside its line in that chat.
A destination is on the map at the right edge, after the table it sends, and on that model's page under Read by. A table's page lists the send among what reads it.
On Free nothing is sent out: Destinations says so in place of Add a place and Send something, and a send kept from Self-serve or Done for you waits until the company is on one of them again.
Before anything goes
Click a send and it opens beside the list: how many rows are waiting, and the next one exactly as it will leave, each column with what it goes as and whether it leaves hashed or as it is. To Meta and Google Ads, rows too old for them are not counted as waiting: they are set aside and never sent, and the number of them sits beside the count. Under that are its last sends, each marked Live, or Test (Checked, for Google Ads), and any rows that can't be sent, with why. The buttons are under all of it: Send now, Edit, Pause and Remove.
Send something makes a new one: it asks which place first, then the table or query, the id column, the columns and when. Saved, it opens the same way, with the rows waiting and the next one as it will leave, before anything has gone.
What goes to each place
- Meta: each row is one event, such as a purchase or a lead. Emails, phones, names, gender, birth date, city, state, zip, country and your own customer id are hashed here before they leave, the way Meta asks; a state or country written out becomes its two-letter code. Meta's click and browser ids, the value, the currency and the order id go as they are, as Meta asks. Anything else goes as it is, so a column that looks personal is refused there until you pick the hashed field for it. Meta takes events up to 7 days old (a sale in a shop, up to 62); older rows are set aside and never sent. Each row goes once.
- Google Ads: each row is one conversion on a click Google tagged. The click id, the time, the value and the currency go; each row goes once.
- HubSpot: each row writes its columns as properties on the contact, company, deal or ticket with that id. A property that does not exist yet is created.
- A URL of yours: every batch of changed rows is posted as JSON, signed with a secret you hold and the time it was sent, so your side can refuse a copy posted later.
- Your BigQuery: the table lands in a dataset of your project, under its Overscore name unless the place names one, so everything behind a dashboard can sit in your own warehouse. With an id column, the changed rows are merged by it; with none (tick No id column when you make the send), the whole table is written again each time it changes. The table is made from the columns' types when it is not there yet. You choose the key file of a service account with BigQuery Data Editor and BigQuery Job User on the project; it is read once in your browser, kept encrypted on your company's own machine, and never shown again.
- Your bucket (Google Cloud Storage, or Amazon S3 and anything that speaks S3): the original files of one of your kept files, an upload or an email source's file, every version as it lands, under the prefix you give as
prefix/file name/version/file name. A version is copied once. For Google you choose a service account's key file with Storage Object Creator on the bucket; for S3 an access key id and secret that may put objects in it, the region, and an endpoint if it is not Amazon's. Every key and secret a place holds is kept encrypted on your company's own machine and never shown again. A bucket place can also hold a copy of your whole warehouse, written there every night: Your data, in your own bucket.
Testing first
- Meta: with a test code, a send takes only the 5 newest rows, as test events you can watch arrive in Events Manager. Meta keeps test events and counts them like any other, so those 5 are remembered and never sent again. When they look right, open the place on Places and press Go live; the rest go.
- Google Ads: until you press Go live on the place, Google checks each row and records none of them.
- HubSpot and a URL: there is no test mode at the other end. Read what would be sent first, and start with a query that returns a few rows.
- Your BigQuery or your bucket: Check it on the place reads the dataset or the bucket with the key you pasted and writes nothing; a send then writes for real.
Who may send
- Adding a place, renaming it and removing it take an admin, or an agent's key that can write.
- Sending customers' details to Meta or Google Ads takes an admin who is signed in or on their own laptop, never an agent's key: adding the place, taking it live, and adding a send to it.
- Anyone who can change a table can add a send of it to HubSpot, your BigQuery or a URL of yours; anyone who can change the dataset a file is kept in can send its versions to your bucket.
- A send reads what its maker was allowed to see, whoever presses Send. If its maker leaves, or their key is revoked, the send pauses and a card in the inbox says so.
Stopping and changing
- Pause stops a send: nothing goes on its schedule, after a change, or when someone presses Send, until it is resumed.
- Edit opens the same form, filled in, and changes the send in place, keeping its record of what went. To Meta or Google Ads nothing is sent twice. To HubSpot or a URL, a column given a new field name is sent again so it lands under the new name.
- Removing a send keeps the record too: add it back to the same place, with the same table and id column, and it carries on where it stopped.
- A place's own panel (Places, then click it) has its settings, Check it, Go live while it is testing, Rename and Remove. Removing a place stops its sends; nothing already sent is taken back.
- When a token lapses, an admin presses Replace the token on the place's panel (for Google Ads, Connect Google again; for your BigQuery or bucket, Replace the key file or the secret); the new one is checked at once.
When a row does not go
- If the place was busy or refused the sign-in, the send fails, a card says so in the inbox, its status says what went wrong, and the rows go again next time. A refused sign-in is mended on the place's panel: Replace the token.
- If one row cannot go as it is (no time on it, nobody it says who, a click Google does not know), it is set aside with the reason and the rest go. It is tried again when the row changes, or when you press Retry beside it in the send's panel.
What left
Every row that leaves is written down and kept: when, where to, whether it was live or a test, and which fields went hashed. Removing the send or the place does not remove the record. On the Sent tab, find a row by its id to answer what was sent about one order or one customer, and when; pick a place, or how far back. Click a row to see each column as it went. Under the rows is every change made: a place added, taken live or removed, a send added, changed, paused or removed, and who did it.