dctl move

Move files, deleting the source only after a verified, durable commit.

Synopsis#

dctl move is copy plus one step: after each file is durably committed at the destination, its source is deleted. Everything else — how SOURCE and DEST are parsed, how the two sides are enumerated, which files count as identical, what the plan looks like — is the same shared machinery, so move and copy cannot drift apart about what needs transferring.

The ordering is the product. PLAN.md §6 step 7 reads: after the commit is durable, delete the source. That single sentence is why a move interrupted by a crash, a network failure, a bad checksum or a Ctrl-C leaves the source file exactly where it was, and it is what makes DCTL safe to point at the only copy of something.

The guarantee is encoded rather than documented. Source deletion happens on the far side of the transfer's result, per file: the transfer must return Ok before the deletion is even reachable, and there is no second pass over the plan to batch the deletions into. At every instant the process could be killed, each individual file is either still at the source or durably committed at the destination. Never neither, and never "moved" on the strength of an upload that had not yet been verified.

What "durable commit" means concretely, because the promise is only as good as the thing it names:

• Into a vault, the file is sealed, written to the backend with a verified write — the stored object is compared against the hash of what was sent — and its index record is committed. dctl-core returns Ok only after the index commit, and only then does the source deletion become reachable. There is no window in which the object is stored but uncommitted.
• To a local path, the destination file is written and fsynced to stable storage, and only then is the source removed. A power cut between the two leaves both copies, which is the direction that loses nothing.

Two consequences worth stating plainly:

• A checksum mismatch aborts before the commit, so the source survives. For a vault destination the comparison is inside the verified write, so a mismatch aborts during the upload — the index is never touched, nothing is deleted, and the run exits 20 (checksum_mismatch) rather than a generic error, so a script can tell corruption apart from a timeout.
• A file that transferred but whose source deletion failed is reported as an error. The data is safe, but the operator now has two copies rather than one and needs to know it. Failure messages for a removal always name the side — source a.mov — because for move the difference between a failed destination write and a failed source removal is the whole story. A source that was already gone is not an error: a retried move must not fail because the first attempt succeeded.

move never deletes at the destination. It adds to DEST and removes from SOURCE. A file that exists only at the destination is left alone, exactly as under copy. The verb that removes destination extras is sync, and confusing the two is the mistake this separation exists to prevent.

Both arguments are containers, as in copy. dctl move ./scratch vault:films/2024 moves the contents of ./scratch under vault:films/2024, preserving relative paths. A single-file SOURCE is allowed and lands inside DEST under its own name. When DEST should be the object's new name instead, use moveto. A vault destination is not reachable from move at all today — see What runs today — and the two ways vault addressing still differs from the above are documented under copy's What runs today.

Destructive classification. move is registered as a destructive command, which controls the confirmation gate: under --interactive it asks before it acts, and --force approves without asking. Note what the default is — running without either flag does not prompt. Typing the command is taken as consent, because prompting by default would break every script. --interactive with no terminal available is a usage error rather than a hang, which is what an unattended job needs.

A no-op move deletes nothing. If every file is already at the destination, the command says so and stops:

nothing to move: every file is already at the destination

Deleting the sources on that basis would be defensible, but it is not what the plan said would happen — and a move that removes files without transferring them is precisely the surprise this command exists not to spring. If you want the sources gone regardless, remove them explicitly with delete or purge.

Comparison, filters and failure policy are the family's, identical to copy: size and modification time by default with a one-second tolerance, --checksum or --size-only to change it, --ignore-existing and --update to skip. What "identical" means matters more here than anywhere else, because what this command does with it is delete the source — so it is worth reading copy's note on the one edit the default cannot see (same size, same timestamp) and reaching for --checksum when a source is rewritten by something that preserves timestamps; every filter honoured through one engine (--include, --exclude, --filter-from, --files-from, --min-size, --max-size, --max-depth); per-file failures counted and survived (exit 6), fatal failures stopping the run. Symbolic links are never followed and are counted and warned about — a link is not moved, and therefore is not deleted either.

--dry-run is authoritative and deletes nothing. The plan is computed without touching either side, and the same value is either printed or executed. A dry run never reaches the confirmation and never reaches the reaper.

What runs today#

Filesystem to filesystem, move works end to end. Each planned file is read, written, flushed, and only then removed from the source. Files the plan skipped keep their sources. That is the whole command, running for real.

Moving into a vault does not work yet, and fails safely. The upload path itself is finished — copy uses it — but move opens a second vault session for its reaper, and the index database allows a single writer, so the run stops at connect time with exit 23 (index_error):

error: index database error: Database already open. Cannot acquire lock.
warning: The index is a rebuildable cache: `dctl index rebuild` rescans object
headers.

That happens before any file is transferred and before any source is touched, which is the outcome the ordering guarantee is supposed to produce when something goes wrong. Until it is fixed, move into a vault in two steps: dctl copy the tree up, verify it, then remove the source deliberately with delete.

Moving out of a vault plans and runs. A REMOTE:PATH source is enumerated through crate::source, the same reader dctl ls uses, so the listing and the move agree about what is there; each object is fetched, authenticated, written durably, and only then removed from the vault. Remote-to-remote is still refused by the engine at connect time, naming which of two gaps applies: a sealed end needs a re-encrypting transfer dctl-core does not expose (no PLAN.md §11 phase schedules one), and two plain ends need only a dctl-cli engine that holds two backends (no phase names that either).

The refusals copy documents apply here unchanged, all exit 7 and all before anything is deleted: a plain write into a directory that holds a vault, and --checksum against a plain object store, whose provider checksum is not the plaintext hash a vault records. A size ceiling used to be among them and no longer is: the engine streams, so there is no size at which a move stops fitting in memory.

--immutable is honoured, at plan time: a plan containing an update — an existing destination object being replaced — fails the whole run with exit 7 before any byte moves, and the message names the paths. It governs the destination only. It does not protect the source, whose removal is what move means; use copy for a transfer that leaves the source in place. --immutable with --no-traverse is a usage error (exit 1), because an unlisted destination cannot be checked for overwrites.

--transfers, --bwlimit and --retries are parsed and not consulted. Files move one at a time.

dctl move SOURCE DEST [flags]

Examples#

The stderr run summary and the structured ERROR log line are omitted below except where they are the point.

Move finished renders between two local trees. Each file's source is removed only after that file's destination write is flushed to stable storage, so an interrupted run leaves the remainder in ./scratch/renders:

$ dctl move ./scratch/renders /srv/archive/renders --progress -v
2 to copy, 0 to update, 0 to delete, 0 unchanged (1.91 MiB)
 
 Transferred: 1.91 MiB / 1.91 MiB, 100%, 44 MiB/s
    Verified: 1.91 MiB checksum-matched
       Files: 2 / 2
      Checks: 2 / 2
      Errors: 0
     Elapsed: 0s
$ ls ./scratch/renders
$ echo $?
0

Moving into a vault stops at connect time, with both copies intact — see What runs today for the two-step alternative:

$ dctl move ./scratch/renders archive: --no-traverse --force
error: index database error: Database already open. Cannot acquire lock.
$ ls ./scratch/renders
a1.mov  a2.mov
$ echo $?
23

Always preview a move. --dry-run computes the same plan the real run would execute and touches neither side:

$ dctl move /mnt/ingest /srv/archive/ingest --dry-run -v
2 to copy, 0 to update, 0 to delete, 1 unchanged (1.91 MiB)
Action      Size  Path
------  --------  ---------
copy    1.91 MiB  big.mov
copy         3 B  sub/b.txt

Note that the plan shows no delete rows. The source removals are not plan entries — they are step 7 of each transfer, and they happen only for the files the plan lists as copy or update. The one unchanged file above is skipped, and its source is therefore not deleted.

Move from a Windows staging drive. A one-character prefix is a drive letter on every platform, so D:\ingest\2024-06-01 is a local path and never a remote called D:

$ dctl move D:\ingest\2024-06-01 D:\archive\2024-06-01 --dry-run

Approve a destructive run without a terminal. --interactive prompts and requires typing yes; with stdin not attached to a terminal it refuses rather than hanging, which is what an unattended job needs:

$ dctl move /mnt/ingest /srv/archive/ingest --interactive < /dev/null
error: cannot confirm 'move (deleting the source of) files from' on '/mnt/ingest': no terminal available
warning: Pass --force to approve destructive actions non-interactively.
$ echo $?
1

$ dctl move /mnt/ingest /srv/archive/ingest --force

Nothing to move is a success, and it leaves the sources alone:

$ dctl move /srv/staging /srv/archive --size-only -v
0 to copy, 0 to update, 0 to delete, 1 unchanged (0 B)
nothing to move: every file is already at the destination
$ echo $?
0

A file too large for the whole-buffer engine is refused rather than attempted, and the refusal is fatal — so a-small.txt, earlier in plan order, has already been moved and removed from the source, while everything after z-huge.bin is untouched on both sides:

$ dctl move /mnt/ingest /srv/archive --force
$ echo $?
7

Options#

      --create-empty-src-dirs  Recreate empty source directories at the destination
      --ignore-existing        Skip files that already exist at the destination, without comparing them
      --update                 Skip files where the destination is newer than the source
      --no-traverse            Do not list the destination; assume every source file is missing there
  -h, --help                   Print help (see more with '--help')
  -V, --version                Print version

Positional arguments:

--ignore-existing and --update deserve a second look here, because under move a skip is also a non-deletion: a file the plan skips is neither transferred nor removed from the source. That is the intended reading — the source is only ever deleted for a file whose own commit succeeded.

Options inherited from parent commands#

Every global flag is accepted; see ../GLOBAL_FLAGS.md for the full list. The ones that change what this command does:

Exit codes#

See ../EXIT_CODES.md for the full contract.

Exit 20 is the one that carries the specific promise: the destination stored the wrong bytes, nothing was committed, and the source is untouched.

See also#

• dctl copy — the same transfer without the source deletion.
• dctl moveto — move one thing to an exact destination name.
• dctl sync — make the destination identical, deleting destination extras.
• dctl delete — remove objects without transferring them anywhere.
• dctl verify — prove afterwards that what arrived is intact.