DedupCommando v0.9.0-beta.4
For anyone running 0.9.0-beta.3. Most of this release is about the checkpoint database,
dedcom.db: how its move journal stores pathnames, and what the shape guard accepts before it
writes anything. One fix changes what you see on disk and in the duplicate list.
Order and names on disk no longer depend on the machine
Two habits made one tree come out differently on two machines. Directory order was taken from
readdir, and a file name was rebuilt through a lossy string that maps distinct bytes onto one.
Siblings are now ordered by their raw bytes, and a name keeps the bytes it had. A failed read no
longer counts as an answer to whether two files are duplicates, and a merge that leaves the source
behind is not reported as clean.
The checkpoint moves to schema v6
The first start of this build upgrades dedcom.db to schema v6 in one transaction. The
move_event journal used to pass pathnames through a lossy UTF-8 conversion, so two names that
differ only in bytes outside UTF-8 collapsed into one record. v6 stores both pathnames as raw bytes
and marks each row with path_fidelity: 1 for rows this build wrote, 0 for rows carried over from
the v5 journal.
The table is rebuilt only after its stored definition and everything attached to it (indexes, triggers, views, foreign keys) prove to be what this product wrote. Anything else stops the upgrade with a message, and the data and the schema version stay as they were.
The shape guard refuses more before it writes
Before the database is switched to WAL, the guard now refuses:
- a product name held in another letter case. SQLite resolves names without regard to ASCII case,
the guard compared them byte for byte, and a foreign
SCAN_ROOTcould be adopted; - two objects under one product name, such as a table and a trigger;
- a hidden or generated column under a product column name, and a column of a later schema in a checkpoint that declares an older one;
- a virtual or shadow table (FTS, R*Tree and the like) under a product table name. It could carry the right column names and be taken for the product's own.
A refused file keeps its bytes. Objects under names the product does not use, virtual tables included, are left alone.
Also in this release
The manual matches this release: the F9 menu, the wording of the size figure on the confirmation
and summary screens, and the session-retention default. A test now holds the manual to what the
code does. Two paths gained tests: the order of a walk over several roots that cannot be keyed, and
a refused stat while listing files of the same size.
What it does not establish
- Journal rows carried over from v5 keep the text v5 had. Bytes lost then are not recovered, and those rows are not claimed to be exact.
- The journal is still best-effort: a move whose record could not be written is not reported.
- The declared types and defaults of required columns are not compared, and a
WITHOUT ROWIDorSTRICTtable under a product name passes as an ordinary table.
On upgrade
Back up dedcom.db first with the procedure in the manual (§12, "Backing up and restoring
dedcom.db"). A plain cp misses the -wal file. beta.2 and beta.3 refuse to open a v6 database
before writing to it, so going back means restoring that backup.
Published from docs/release-notes/v0.9.0-beta.4.md at v0.9.2.