dctl sync

Make the destination identical to the source. Deletes from destination.

Synopsis#

dctl sync deletes files you did not name. It is the only verb in the transfer family that does, and that single property drives everything about how it behaves. After a successful sync, DEST contains exactly what SOURCE contains: files missing at the destination are transferred, files that differ are re-transferred, and files present only at the destination are removed.

The blast radius is the whole destination tree. dctl sync ./photos /srv/photos with an empty or wrong ./photos is an instruction to delete every photo, and from inside the process an unmounted volume and an empty directory are the same syscall result. Several guards below exist specifically for that case, but the first line of defence is --dry-run, and the habit worth building is to never run a sync you have not previewed.

If you want to add and update without ever removing, use copy. That is the whole difference between the two verbs.

The second run is cheap#

This is the property that makes sync worth putting in a cron job, and it is worth stating explicitly because it did not hold until recently.

Every write records the source's own modification time at the destination — in a vault's index row and inside the sealed object's own metadata, on a filesystem or SFTP host as the file's mtime, on B2 as the documented src_last_modified_millis field. So the default size-and-time comparison finds an unchanged tree unchanged, and the second run transfers nothing and fetches nothing:

$ dctl sync ./src backup:
       Files: 3 / 3
$ dctl sync ./src backup:
       Files: 0 / 0
      Checks: 3 / 3
     Skipped: 3 (unchanged)

dctl check ./src backup: immediately afterwards reports all match, from the same comparison with the same tolerance — the two verbs read --modify-window from one place, so they cannot disagree about a tree one of them has just written.

Three things this does not cover, stated here rather than found on a bill:

• S3 and R2. The source's time is stored as x-amz-meta-mtime, but ListObjectsV2 does not return user metadata and a transfer compares against the listing — so a sync to those two providers still re-transfers an unchanged tree. Use --size-only there.
• Objects written before this build. A bucket's existing objects carry no recorded source time, so the first run against one re-uploads the tree once. Every run after that skips.
• An edit that changes neither the size nor the timestamp. Invisible to the default comparison by construction — the same trade rclone and rsync make. --checksum is the answer, and against a vault destination it costs nothing on the vault's side because the index already holds the plaintext BLAKE3.

The safety guards#

sync refuses rather than guesses when the arguments cannot mean what they say. Each refusal below is a usage error (exit 1) raised before anything is modified.

• A single-file source. dctl sync photo.jpg backups/ reads as "make backups/ contain exactly photo.jpg", which means deleting everything else in it. Almost nobody means that, so it is refused and the hint names the two verbs that do: copy to add the file without deleting anything, copyto to write it to an exact destination name.

• An empty source against a non-empty destination. A source that listed zero files, when the destination holds some, is refused — this is the unmounted volume case. --force is the escape hatch, because deliberately emptying a tree is a real thing to want and DCTL's job is to make it explicit rather than impossible. To remove a tree on purpose, purge is the command that says so in its name.

  That guard is the last line, not the first: it only fires because the listing is honest about what it found. A source root reached through a symbolic link used to list as empty — /data -> /mnt/disk/data walked to nothing — and sync --force then deleted every object at the destination and exited 0 with Errors: 0. The walk now resolves the root it was given; links found inside the tree are still never followed. See copy → Omissions are announced.

• Source and destination are the same place. Structural equality catches the obvious case and canonicalisation catches photos versus ./photos and a symlinked duplicate. A sync onto itself would be a race between listing a tree and deleting from it.

• A destination that is already a file. A tree cannot be written inside a file; the hint points at copyto/moveto.

Two more safeguards are not refusals but noise, deliberately:

• The mass-deletion warning. When a sync would remove half or more of the files it found at the destination, an unconditional warning goes to stderr — not gated behind -v, because it is the last chance to notice a typo: warning: this would delete 1 of the 2 files at the destination. It never blocks. Emptying a tree is legitimate; being quiet about it is not.
• Every filter is applied to both listings. --include, --exclude, --filter-from, --files-from, --min-size, --max-size and --max-depth all go through one engine, and the same engine on both sides. This matters more here than anywhere else in the tool: a rule that reached only the source would make sync see every excluded destination file as an extra and delete it. Because a rule hides a file on both sides, it is neither transferred nor deleted — dctl sync src dst --exclude 'archive/**' protects archive/ at the destination rather than emptying it. A pattern that will not compile is a usage error before anything is listed. dctl sync src dst --min-size 100 leaves a 4-byte destination file alone rather than treating it as an extra.

The confirmation#

sync is classified destructive, and asks for approval when — and only when — the computed plan actually removes something. A sync that only adds files does not prompt, because there is nothing destructive about it.

Note what the default is: running without --interactive does not prompt. Typing the command is taken as consent, because prompting by default would break every script. --interactive asks on stderr and requires the exact word yes; with no terminal available it fails with a usage error rather than hanging. --force approves without asking, and is also what unlocks the empty-source guard. A declined confirmation exits 25 (cancelled), never a silent zero — a command that declined to do its work has not succeeded.

When the deletions happen#

sync both adds and removes, so it is the only verb where ordering is a user-visible choice. The three modes are rclone's, and each answers a different question:

The three flags are mutually exclusive and clap rejects any combination of them at parse time, before either side is enumerated. Whatever the mode, the plan is fixed before execution begins: the executor reorders the entries it was given, it never recomputes them. The list a --dry-run printed is the list that gets performed.

--no-traverse is not offered#

dctl sync --no-traverse is a parse error, not a no-op. Skipping the destination listing is a sensible optimisation for copy, where the listing only decides what to skip. For sync the listing is where the deletions come from, so the flag would either do nothing or delete nothing. A flag that parses and is then ignored is a defect, so the parser does not accept it.

The verified-write contract#

Sync's transfers are the family's transfers, unchanged, and copy documents them in full. In short: a file bound for a vault is sealed, written with a verified write — the stored object compared against the hash of what was sent — and then committed to the index, and the index commit is what makes it count as stored. A file bound for a local path is written and fsynced before success is reported. A mismatch between what was sent and what the backend stored aborts before the index commit: nothing is committed, nothing is reported as transferred, and the run exits 20 (checksum_mismatch).

The interaction with deletion is the part to understand. A destination file being replaced is only replaced by a commit that verified; a destination file being removed is removed because the source does not have it, which is a listing fact rather than a transfer outcome. Under --delete-after no removal happens until every replacement has committed, which is why that mode is the one to reach for when the destination is irreplaceable.

Per-file failures are counted and survived: the loop continues and the errors downgrade the exit code to 6 (partial_failure), never rolled up into success. A failed deletion is counted as an error but is never counted as a deletion — the "files deleted" number in the summary means files that are actually gone. Fatal failures (locked vault, index error, cancellation) stop the run rather than producing one identical error per remaining file.

--dry-run is authoritative#

The plan is a pure function of two listings and a policy — no I/O, no clock, no mutation — and the same value is either printed for review or handed to the executor. Every planned deletion is shown, in every output format, because a deletion nobody could review before it happened is not a deletion anybody approved. dctl sync src dst --dry-run --json | jq '.actions[] | select(.action == "delete")' is a supported workflow: plan data goes to stdout, and progress, warnings and the summary go to stderr where they cannot corrupt it.

What runs today#

Between two local paths, sync runs for real — transfers and deletions, in all three orderings. Files missing at the destination are written and flushed, files that differ are rewritten, and files the source does not have are removed from the destination. The mass-deletion warning, the confirmation, the guards and --dry-run all sit in front of that unchanged.

A sync with a vault on one side runs too. Both directions work: a vault is enumerated through crate::source, the same reader dctl ls uses, so the listing an operator reads before approving a deletion describes the sync that follows. dctl sync archive: /srv/mirror makes the local tree match the vault, removing what the vault does not hold; dctl sync /srv/photos archive: does the reverse.

Remote-to-remote is still refused, at connect time rather than part-way through a tree, and the message distinguishes two different waits. With a sealed end it needs a re-encrypting transfer dctl-core does not expose, and no PLAN.md §11 phase schedules one. With two plain ends nothing needs re-sealing and nothing is waiting on the core: the dctl-cli engine holds one backend and one local side, and no phase names that either. Sync down to a local path first, then sync that up.

The refusals copy documents apply here unchanged, all exit 7 and all before anything is removed: a plain write into a directory that holds a vault, and --checksum against a plain object store, whose provider checksum is not the plaintext hash a vault records. A size ceiling used to be among them and no longer is: the engine streams, so there is no size at which a transfer stops fitting in memory.

--immutable is honoured, at plan time, and it is strictest here: sync is the verb that both replaces and removes, so a plan containing any update or delete fails the whole run with exit 7 before anything moves, naming the paths. Only additions are allowed — which is to say that under --immutable, sync degrades to copy and refuses outright rather than quietly doing so. All three --delete-* orderings are equally refused; the ordering only decides when deletions happen, not whether they are permitted.

--transfers, --bwlimit and --retries are parsed and not consulted. Files move one at a time.

dctl sync SOURCE DEST [flags]

Examples#

The stderr run summary and the structured ERROR log line are omitted below except where they are the point. A REMOTE:PATH on either side stops with exit 7 before anything happens; see What runs today.

Preview before anything else. -v adds the one-line shape summary; the mass-deletion warning appears regardless of verbosity:

$ dctl sync /mnt/media/incoming /srv/archive/incoming --dry-run --size-only -v
2 to copy, 0 to update, 1 to delete, 1 unchanged (1.91 MiB)
warning: this would delete 1 of the 2 files at the destination
Action      Size  Path
------  --------  ---------
copy    1.91 MiB  big.mov
copy         3 B  sub/b.txt
delete       2 B  stale.txt

Review the deletions machine-side before approving a real run. This is the artefact worth attaching to a change ticket — it names both endpoints, so it is still meaningful pulled out of a CI log a month later:

$ dctl sync /mnt/media/incoming /srv/archive/incoming --dry-run --size-only --json
{
  "command": "sync",
  "source": "/mnt/media/incoming",
  "destination": "/srv/archive/incoming",
  "dry_run": true,
  "summary": { "copy": 2, "update": 0, "delete": 1, "skip": 1, "mkdir": 0, "bytes": 2000003 },
  "actions": [
    { "action": "copy", "source": "big.mov", "dest": "big.mov", "size": 2000000, "reason": "missing-at-destination" },
    { "action": "copy", "source": "sub/b.txt", "dest": "sub/b.txt", "size": 3, "reason": "missing-at-destination" },
    { "action": "delete", "source": "", "dest": "stale.txt", "size": 2, "reason": "not-at-source" }
  ]
}

Run the sync, removing nothing until every replacement is durable. --delete-after is the mode to reach for when the destination is the only copy:

$ dctl sync /mnt/media/incoming /srv/archive/incoming --delete-after --size-only -v
2 to copy, 0 to update, 1 to delete, 1 unchanged (1.91 MiB)
warning: this would delete 1 of the 2 files at the destination
 
 Transferred: 1.91 MiB / 1.91 MiB, 100%, 44 MiB/s
    Verified: 1.91 MiB checksum-matched
       Files: 2 / 2
      Checks: 4 / 4
     Skipped: 1 (unchanged)
     Deleted: 1
      Errors: 0
     Elapsed: 0s
$ echo $?
0

Sync a Windows working drive. E:\projects\apollo is a local path — a one-character prefix is a drive letter on every platform, never a remote called E:

$ dctl sync E:\projects\apollo F:\backup\apollo --delete-after --progress

Mirror a vault onto local disk. The vault is listed like any other side, which is what makes the deletions computable at all:

$ dctl sync archive: /srv/mirror --force
warning: this would delete 1 of the 1 files at the destination
 Transferred: 293.0 KiB / 293.0 KiB, 100%, 34.8 KiB/s
    Verified: 293.0 KiB checksum-matched
       Files: 4 / 4
      Checks: 5 / 5
     Deleted: 1
      Errors: 0
     Elapsed: 8s

The empty-source guard. This is the difference between a typo and a restore:

$ dctl sync /mnt/backup /srv/archive/photos
error: '/mnt/backup' contains no files, so '/srv/archive/photos' would delete all 2 of them
warning: An unmounted volume and an empty directory look identical from here.
Check the source, or pass --force if the destination really should be emptied.
To remove a tree deliberately, use `dctl purge`.
$ echo $?
1

A single-file source is refused, with the two verbs that do mean it named in the hint:

$ dctl sync photo.jpg b2prod:bucket/media
error: 'photo.jpg' is a file, so 'sync' would make 'b2prod:bucket/media' contain nothing else
warning: Use 'copy' to add the file without deleting anything, or 'copyto' to
write it to an exact destination name.
$ echo $?
1

A pattern filter protects the destination rather than emptying it. The rule is applied to both listings, so archive/ is invisible on either side and is neither transferred nor deleted — which is the whole reason one engine answers for both:

$ dctl sync /srv/src /srv/dst --exclude 'archive/**' --dry-run

--no-traverse is not a flag this command has:

$ dctl sync /srv/src /srv/dst --no-traverse
error: unexpected argument '--no-traverse' found
$ echo $?
1

Two answers to "when is data deleted?" is not a question resolved by precedence:

$ dctl sync /srv/src /srv/dst --delete-before --delete-after
error: the argument '--delete-before' cannot be used with '--delete-after'
$ echo $?
1

An already-identical destination needs neither a prompt nor an engine:

$ dctl sync /srv/src /srv/dst --size-only -v
0 to copy, 0 to update, 0 to delete, 1 unchanged (0 B)
nothing to do: the destination already matches the source
$ echo $?
0

Options#

      --create-empty-src-dirs  Recreate empty source directories at the destination
      --ignore-existing        Skip files that already exist at the destination, without comparing them
      --update                 Skip files where the destination is newer than the source
      --delete-before          Delete destination files before transferring
      --delete-during          Delete destination files during the transfer. The default
      --delete-after           Delete destination files after transferring everything
  -h, --help                   Print help (see more with '--help')
  -V, --version                Print version

Positional arguments:

There is deliberately no --no-traverse; see the Synopsis.

--ignore-existing and --update change only whether an existing destination file is overwritten. Neither protects a destination file that the source does not have at all — that file is an extra, and an extra is deleted. If you need files at the destination to survive, you need copy, not a flag.

Options inherited from parent commands#

Every global flag is accepted; see ../GLOBAL_FLAGS.md for the full list. The ones that matter here:

Exit codes#

See ../EXIT_CODES.md for the full contract.

20 (checksum_mismatch), 21 (integrity_failure), 22 (vault_locked) and 23 (index_error) are part of this command's contract and are not reachable today: all four require a vault, and every vault-involving sync stops at enumeration with exit 7. They become reachable when remote listing lands, with the meanings copy gives them.

See also#

• dctl copy — add and update without ever removing. The safe verb.
• dctl move — transfer, then delete the source, never the destination.
• dctl check — see the differences between two sides without changing either.
• dctl purge — remove a tree deliberately, rather than as a side effect of an empty source.
• dctl delete — remove objects by filter, keeping the directory structure.
• dctl restore — the command you need if a sync deleted the wrong thing.