Moving data between environments

plumix sync pull copies the deployed site’s data into your local database. plumix sync push copies your local data to the deployed site. Either way the target’s data in each chosen scope is replaced, never merged.

Overview

A sync moves whole tables, one named scope at a time, with every row under the id it had on the source. It never matches rows by slug or title: content refers to other rows by id inside JSON (a menu item’s meta.entryId, a revision’s slug), so renumbering would break those references. ADR 0039 records why.

The runtime does the moving. A runtime adapter’s commands module exports sync, and a runtime without one fails with sync_runtime_unsupported, naming it. Runtime Adapters covers the export.

Quickstart

See what a pull would change, then pull. On Cloudflare, --production names the deployed database (see On Cloudflare):

plumix sync pull --url https://example.com --production --dry-run
plumix sync pull --url https://example.com --production

Push your local content and media, confirming at the prompt:

plumix sync push --scope content --scope media --url https://example.com --production

Commands

plumix sync pull|push [--scope <scope>...] [--url <origin>] [--token <token>] [--binding <name>] [--dry-run] [--yes] [--no-rewrite-urls]

--url is the deployed site’s origin. One end of a sync is always the deployed site, so it is required; without it the command fails with sync_remote_url_required.
--token is the API token a runtime that calls the deployed site uses, as plumix mail test --remote does. Without it the token is PLUMIX_API_TOKEN, else, on a terminal, one the site’s device flow mints: the command prints a link and a code to approve, and never stores the token. With neither and no terminal it fails with sync_remote_token_missing. The token’s user must hold settings:manage, and the token must carry no scopes: one minted with a scope list or a preset is refused. Cloudflare reaches D1 through wrangler instead and asks for one only for the settle after a sync (see After a sync), resolved the same way.
CF_ACCESS_CLIENT_ID and CF_ACCESS_CLIENT_SECRET get the CLI past Cloudflare Access. Set both to an Access service token’s client ID and secret, and every request the command sends the site carries them. A request Access redirects to its sign-in page fails with remote_behind_access.
--scope names a scope, once per scope. Without one, the sync moves content.
--binding picks the database on a runtime that can have more than one, as plumix migrate --binding does.
--dry-run prints what the sync would change and changes nothing.
--yes skips a push’s confirmation prompt.
--no-rewrite-urls leaves the source’s URLs in content as they are. See URLs.
A runtime may add the options that name its deployed database, as plumix migrate takes them: --production, --preview and --env <name> on Cloudflare.

A sync runs these steps in order:

A push without --yes or --dry-run, with no terminal to confirm in, fails with sync_push_unconfirmed before anything is opened.
The scopes are resolved. An unknown one fails with sync_unknown_scope, and a scope named without the one it needs fails with sync_scope_requires.
Both ends open, and their migrations are compared. When any owner’s head differs, the sync fails with sync_migrations_differ, naming the owner and the end that is behind. Run plumix migrate on that end first, with the target option that names the deployed database (--production on Cloudflare).
The source is exported into a dump, which is then read back and checked.
The target runs the import as a dry run, and the command prints, per table, the rows replaced, added and removed, then the rows outside the sync it would remove or clear, then the emails of the authors it would add as disabled users, then how many values it rewrites in each table (see URLs).
With --dry-run it prints Dry run: nothing was changed. and stops.
A push without --yes asks for confirmation. Answering no changes nothing.
A push takes a snapshot of the deployed site and prints how to restore it. When the import or a step after it fails, it prints that again.
The target imports the dump.
The target settles: every sync.after handler runs, then the whole CDN is purged. See After a sync.

A pull takes no snapshot and asks for no confirmation: it only ever overwrites your local database.

On Node and Bun

@plumix/runtime-node and @plumix/runtime-bun export sync. Their local end is the SQLite file nodeSqlite() or bunSqlite() names, resolved against the project root, and the site’s storage slot. Its import is one transaction, so a failed one changes nothing.

Their remote end calls the deployed site’s sync routes with the API token. Every route answers 401 without a token, and 403 when the token carries scopes or its user lacks settings:manage:

GET /_plumix/api/v1/system/sync/heads answers { heads }, each tracking table’s newest migration.
GET /_plumix/api/v1/system/sync/export?scopes=content,media streams the scopes’ rows as NDJSON: each table, then the authors, then the objects, ending in {"end":true}.
POST /_plumix/api/v1/system/sync/import?scopes=…&dryRun=true|false takes the dump as NDJSON in the order an import reads it and answers with the import’s report. It is all or nothing: a dump at other migration heads gets 409 before anything is written, and a body that ends before its last line rolls the import back.
POST /_plumix/api/v1/system/sync/snapshot copies the database file beside itself with VACUUM INTO, as <file>.snapshot-<timestamp>, and answers { id, restoreHint }. A push prints the hint, which names the copy and the command that restores it with the site stopped.

The routes exist on every runtime. A database slot that cannot open itself for a sync, such as D1, answers 501.

Scopes

Each scope is whole tables. No scope splits a table with a filter, because a split would leave rows on the target that point at rows the sync removed.

content, the default: terms, entries (every entry type, so menu items, revisions and media rows too), entry_term and settings.
media: every object in the site’s storage slot. The target’s objects the source lacks are deleted.
users: users and credentials. It travels only with content, because an entry refuses the deletion of its author. --scope users alone fails with sync_scope_requires.
plugin:<id>: the tables a plugin’s schema declares. A plugin whose tables reference content’s tables travels only with content. A plugin that owns no tables is not a scope.

Tables load parents first: users, credentials, terms, entries, entry_term, settings, then each plugin’s tables in config order. Tables a trigger maintains, like entry_changes, are in no scope; their triggers refill them as the rows land.

Auth state never travels. sessions, api_tokens, auth_tokens, device_codes, oauth_accounts, oauth_grants, oauth_authorization_codes and oauth_refresh_tokens are in no scope and never in a dump.

References

Ids travel verbatim. A row keeps its id on the target.
Users are matched by email. Without the users scope, every column that references a user (an entry’s author, a comment’s author) is rewritten to the target’s user with the same email. An author the target lacks is added as a disabled user, with the source’s email, slug, name, avatar and role, and empty meta. A slug the target already has gets -2, -3, and so on. Nobody can sign in as an added user until an admin enables them. A user id stored inside JSON, rather than in a column that references users, is not rewritten.
Rows outside the sync that reference a replaced row through their own foreign key are removed, or cleared where the key sets null: comments on replaced entries, for example. The dry run counts them.
Replacing users keeps each of the target’s sessions and API tokens whose user’s email is in the dump, re-pointed to that user, and removes the rest. A pull of users also keeps every local admin: a remote user with the admin’s email is imported as an admin and holds the admin’s local passkeys instead of their own, which are bound to the deployed site and cannot sign in locally. An admin the deployed site lacks is imported under a free id, with their passkeys.

URLs

A sync points the source’s URLs at the target, so a pulled site links to itself rather than to production. The source is the dump manifest’s origin and basePath; the target is the target end’s origin (--url for the deployed site, auth.passkey.origin for the local one) and the site’s basePath.

Where. Only inside the string values of core’s JSON content: entries.content (block trees and their rich-text links), entries.meta, terms.meta, users.meta and settings.value. Each value is parsed and walked; nothing is found and replaced in raw SQL or text, so a serialized structure can’t break. Every other column, a plugin’s tables included, and every object key is left alone.
What. An absolute URL whose origin and base path match the source’s exactly, wherever it sits in a string, an HTML block’s href included. With the source at https://prod.example and a basePath of /site, https://prod.example/site/a becomes http://localhost:5173/site/a on a pull, and https://prod.example/other, https://prod.example.com/, another domain and a relative URL stay as they are.
Report. The dry run’s summary ends with each table’s count of rewritten values.

A value nothing matched in keeps its bytes. --no-rewrite-urls turns the rewrite off, and leaves every value byte-identical.

The dump

A sync writes its dump under .plumix/sync/<direction>-<timestamp>/ in your project, which the scaffold’s .gitignore already ignores. It is deleted after a sync that succeeds. When a sync fails, the dump is kept and its path is printed.

manifest.json holds format: 1, the source’s origin and basePath, the scopes, each table with its row count in load order, the number of authors and objects, and each migration owner’s head.
<table>.ndjson holds one row per line. A value is a JSON string, number or null; a BLOB is {"$blob":"<base64>"}.
authors.ndjson holds the id, email, slug, name, avatar_url and role of every user the dump’s rows reference. It is written only when users is not among the scopes.
objects/index.ndjson lists each object as { key, file, contentType, size }, and objects/<file> holds its bytes. Files are numbered, never named after their key.

A dump is checked whenever it is read. An unknown format, a malformed manifest, a missing table file or a count that disagrees with the manifest fails with sync_dump_invalid, naming the file.

While the export runs, progress.json records the dump’s source, scopes and heads and each table written whole. A pull goes on from the newest dump a failed pull left whose source, scopes and heads match its own. It prints Resuming from the dump at <path>: <tables> already exported, keeps those tables and the objects the index already lists without reading them again, and exports the rest. When that dump’s export had finished, it imports the dump without exporting at all. Delete the directory to start a pull afresh. A push never resumes: a failed one is undone with the snapshot it took.

After a sync

The target settles once per sync, on a pull and a push alike. Core’s settleSync emits sync.after, then purges the whole CDN through purgeAll. Each step runs whatever the others did. @plumix/plugin-search starts a rebuild of its index, and @plumix/plugin-og deletes its stored cards, when content was among the scopes.

When a step fails, the command fails with sync_settle_failed, says the data was replaced, and lists every failed step. A CDN that cannot purge everything is not a failure: the command says that cached pages refresh as their ttl runs out.

On the deployed site, the settle runs inside the site through POST /_plumix/api/v1/system/sync/settle, with a body of { scopes, direction }. It needs an API token with no scopes whose user holds settings:manage, answers 401 without a token and 403 for a scoped token or without the capability, and answers with the report: { cdn, failures }, where cdn is purged, cannot-purge, failed or none.

The CLI calls it with the token --token passes, else PLUMIX_API_TOKEN, else, on a terminal, a token the site’s device flow mints once you approve it in the browser. On Cloudflare, where the import needs no token, a push with none of them fails with sync_remote_token_missing after its import. A refused call fails with sync_settle_refused, which says the data was replaced. Either way, pass the token and sync again to repeat the steps after it.

On Cloudflare

@plumix/runtime-cloudflare’s commands module exports the transport. The local end is the D1 database and the storage slot plumix dev uses, under .wrangler/state/v3, opened through wrangler’s getPlatformProxy. The remote end is the deployed database --production, --preview or --env <name> names, resolved from the wrangler config as plumix migrate resolves it (see Targets), and the deployed bucket the storage slot binds. Without one of them the sync fails with sync_remote_target_missing before it changes anything. --binding picks the D1 binding as migrate does.

plumix sync pull --url https://example.com --production
plumix sync push --url https://qa.example.com --env qa --yes
plumix import content.jsonl --url https://example.com --production

Target line. The remote end prints its target first, such as → production (top level) · DB · site-prod, with the note a shared previews database carries.
Export. A pull exports each table through D1’s export API, as wrangler d1 export --remote --no-schema --table <table> runs it, so it needs wrangler login or an API token, as wrangler does. D1 serves no queries while an export runs. The local end reads its tables directly.
Import. The import is one script, a statement per line: the deletes, the added authors, then every row. A push runs it through D1’s import API, as wrangler d1 execute --remote --file does, which loads it all or leaves the database as it was. Nothing splits a push into batches. A pull runs the script on the local database in exec calls of whole lines up to 128 MB each, since no string holds a whole site; each call is its own transaction, so a pull that fails partway leaves the local database part-replaced, and the next pull imports its kept dump again. Tables load parents first, and a row whose parent is in its own table, such as an entry’s parent_id, is written after that parent, so every call’s foreign keys hold when it commits.
Snapshot. A push first takes a D1 Time Travel bookmark and prints wrangler d1 time-travel restore <database> --bookmark=<bookmark>, which restores the database as it was before the import. Time Travel restores the database only, not the bucket’s objects.
Media. The media scope copies every object of the storage slot, then deletes the target’s others. Without a storage slot it fails with sync_no_storage. KV holds no scope’s data and never travels.
Settle. The local end settles inside the site’s own handler, on the platform proxy’s bindings alone. When the site cannot reach them from the command line, it fails with sync_site_unreachable and says the data was replaced.
Progress. Each table prints its row count as it is exported and imported, and each object its key.

D1 caps one SQL statement at 100 KB, so a row whose INSERT is larger fails the push’s import, which then leaves the database as it was. D1’s export writes a line break in a string as replace('…\n…','\n',char(10)), so a pulled value holding both a line break and the two characters \n comes back with that \n turned into a line break too.

Importing content

plumix import loads content from files instead of from another environment, so a site moving from another stack can seed itself without its own scripts. It takes a sync dump directory, or a file of authoring records:

plumix import content.jsonl --dry-run
plumix import content.jsonl
plumix import content.jsonl --url https://example.com

plumix import <dir|file.jsonl> [--url <origin>] [--token <token>] [--binding <name>] [--dry-run] [--yes] [--no-rewrite-urls]

Without --url the import loads your local database. --url names the deployed site instead, --token the API token as for plumix sync, and then a push’s rules hold: the import asks for confirmation unless --yes is passed, fails with import_unconfirmed when there is no terminal to ask in, and takes a snapshot and prints how to restore it before it writes. Either way it runs the same dry run first and prints its counts, --dry-run stops there, and after the import the target settles as after a sync.

The authoring format

An authoring file holds one JSON record per line. A blank line is skipped. A term:

{ "type": "term", "taxonomy": "category", "slug": "news", "name": "News" }

An entry:

{
  "type": "entry",
  "entryType": "post",
  "slug": "hello",
  "title": "Hello",
  "author": "alice@example.com",
  "content": { "blocks": [] },
  "meta": { "subtitle": "A first post" },
  "terms": [{ "taxonomy": "category", "slug": "news" }]
}

A term takes taxonomy, slug and name, and may take description and meta.
An entry takes entryType, slug, title and author, and may take content (the stored block tree), excerpt, status (published, the default, or draft), publishedAt (an ISO timestamp; a published entry without one is published at the time of the import), meta and terms.
author is an email, matched to a user by email. An author the site lacks is added as a disabled user with the author role.
terms refers to terms by taxonomy and slug, each one a term record in the same file. Ids are assigned on import, in file order.

Validation

Every record is checked against the site before anything is opened:

its entry type and taxonomy are registered, and so is each taxonomy it refers to;
each meta key is a field the entry type or taxonomy registers, its value passes that field as a publish checks it, and every required field is there;
no two entries share a type and slug, and no two terms a taxonomy and slug;
every term an entry refers to is in the file.

When any record fails, the import fails with import_records_invalid, listing each failure as <file>:<line>: <reason>, and writes nothing.

What it replaces

An authoring file replaces the content scope’s terms, entries and entry_term. It holds no settings, so the site keeps its own settings. A sync dump replaces the scopes its manifest names, exactly as a sync would, and is checked the same way: its format, its counts, and its migration heads against the target’s. Its source’s URLs are rewritten to the target’s, as in URLs; --no-rewrite-urls turns that off. Authoring records name no source, so nothing in them is rewritten.

CLI Reference lists sync beside the other commands. CDN covers purgeAll. Runtime Adapters covers the sync export a runtime implements and describeSyncContract, the suite it is held to.

Next steps

Run plumix sync pull --dry-run against your deployed site to see what a pull would change before you rely on it.