Hashes, pins and history
Status: accepted · decided 2026-11-13 · tracked in core#268 · part of Migrating stored IR across format versions
The question. A migration rewrites what hashes cover. Which hashes may change, which must not, how does a dependency pin keep meaning what it meant, and what do compatibility checks and the changelog say about a history that was migrated?
The hashes that depend on the IR
| Hash | Covers | Computed | Under a migration |
|---|---|---|---|
| Object (blob, tree, commit) | the object's serialized bytes | at write | changes exactly where it has to: below |
Dependency pin, hash = "blake3:…" in config.idp |
scheme v2 (core#261): the CBOR of every schema's frozen units, and the dependency's frozen config units, sorted, minus PublishRegistry and Dependency |
from a fresh compile of the dependency, checked against the pin | not in the store, never rewritten, bridged |
manifest blob |
the content hash verified when a dependency was vendored, as text | at build | carried verbatim. Nothing in core reads it today |
schema_ir_hash, the handshake's ir_hash |
the CBOR of one schema's units, folded to 64 bits | embedded in generated code | follows the units' bytes. Not stored; generated code isn't migrated |
shape_hash, param_hash |
a protocol's canonical wire shape; an encoding's (kind, name, params) |
when facts are read | must not change: peers negotiate on them |
The last is the point of the table: only the first three are stored, and only the first changes by construction.
Object hashes
A hash covers bytes, and a commit holds its tree's hash and its parents'. So a changed payload changes its blob, every
tree on the way up, its commit, and every later commit, because each holds its parent's hash. release.since is a
hash too, and so are the second parents a reconcile leaves.
- What may change: a payload whose encoding changed, and everything above it. For a format that touches only
schema_<n>blobs, that is those blobs, the trees above them, and the whole commit chain. - What must not: a payload whose bytes are the same in N and C keeps its hash, and is not rewritten. A step that moves one field of one unit changes that unit's blob and no other.
- Who maps them: the engine, bottom-up, from the model, where references are version ids. A step author never sees a hash. The old-to-new map is a bijection on commits, and is checked.
- Outside the store: a CI log, a note, or
comline doc's cache (.comline-doc.jsonis keyed by commit hash) may hold an old one.comline store migrate --mapwrites the map, and caches should key on the IR format as well.
Pins
A pin is hashed from IR bytes: schema_units_cbor is to_cbor of the units. So a format change that alters those
bytes for a dependency changes its pin, and the scheme tag (comline-dep-content-v2) doesn't know about the IR format.
Core#261 handled the case of a scheme change, and its message ("this pin was made by an older comline") is the
precedent. A format change is the same event with no message, and the earlier format changes very probably did it
(not checked) without anyone being told.
Two things make that unacceptable for migrations. A migration that turns every pin in a repository into a mismatch is a migration that breaks the build it was meant to save. And the offline proposal makes the hash the identity of a dependency: caches and signed bundles are keyed by it, so a hash that moves with core's internal format would invalidate both.
Decided: a pin made under format N, by scheme S, still verifies under C when the dependency's content can be expressed in N, and a pin is never accepted otherwise.
- To check a pin that doesn't match, core takes the dependency's compiled model and applies
downthrough each step back to N. If every step's witness is defined on it, the content has nothing N couldn't record. It encodes the result with N's codec, hashes it as scheme S did, and compares it with the pin. - A match is exact or it isn't one. Equality of the hash, of the content N would have recorded. Nothing is weaker: the content has no information the pin didn't cover, so every guarantee the pin gave is still given.
- If the content uses something N lacks (a dependency that now has chain markers, with format 4), the witness is
not defined, no older pin can match, and the error says what is new: "this pin was made under IR format 3; the
dependency now records chains that it couldn't have covered. Review it and re-pin:
hash = "…"". The command that writes a re-pin exists already as a request (comline add --repin, or the futureupdate, core#261). config.idpis never rewritten by a migration or by this check.- Cost is only on a mismatch, as
mismatch_messageis today, and only through the formats that still have steps.
A package that doesn't use what format 4 added therefore keeps every pin through the bump, with no action, and the
tests show it (proof): golden-pins.json holds each fixture dependency's pin as the historical core
computed it, and the current core must verify it through the bridge. A package that does use it is asked, once, and
told why.
| A. Verify through the witness (proposed) | B. A pin form independent of the IR | C. Re-pin on every format change | |
|---|---|---|---|
| An unchanged dependency, after a bump | keeps its pin | keeps its pin | every pin breaks |
| A new scheme | no | yes: a v3, so every pin changes once | no |
| A second definition of "content" to keep in step with the IR | no | yes | no |
| Proof | golden pins from the old cores | golden pins from the form | none needed, and none possible |
| Costs | down must be defined on a compile, not only on a stored version |
the form must also carry new information the day the IR does | the offline proposal's bundles and caches |
Decided: a pin that verifies through the witness is accepted, with a one-line notice that suggests a re-pin
("this pin was made under IR format N and still verifies; re-pin to move to the current one: hash = "…""). It is never
silent and never a failure. Accepting is sound by the argument above: the content has nothing the pin didn't cover. The
re-pin is the user's, by hand or with comline add --repin; nothing rewrites config.idp. Option B stays available if
the bridge turns out too costly to keep through many formats.
The pin gate in the proof makes the move visible: a change that moves the fresh pin of a fixture dependency has to say so in the pull request.
History
Compatibility, diff and the changelog compare versions, and a migrated history is not the same as a freshly built
one in what it records. The principle is:
Compare from inputs; recompute outputs; keep what only history can say.
| What a version holds | Examples | In a migrated history |
|---|---|---|
| Inputs | the schema units, the vendored dependencies' schemas and configs, config, docs |
carried. If a version never kept one, it is unseen, not empty |
| Outputs of a rule | a protocol's given-up guarantees and their causes, the encodings in play, a shape hash | recomputed when read, by the core that reads, under the current spec. Where stored (for pinning an acknowledgement, or to be quick), the stored value is a record and not an input to comparisons |
| History itself | version, release records, a protocol's line, a function's since, wire numbers |
carried as data. They depend on the chain, not one version, and they are promises that peers may already hold |
Version numbers in particular are kept even where today's rules would have numbered the history differently. A
migration is not a rebuild, and a version that was released as 0.4.0 was.
What a comparison does
- Both sides are recomputed under the same, current spec. Granular
@unverifiedsaysdiffrecomputes both versions under the newer spec, so a loss the spec learned to see is in both and is no change. A format-3 version whose protocol never hadbounded_sizein its frozen set (the guarantee didn't exist) is not "narrower" than today's: it is recomputed, and gives it up like its successor does. - The spec and core that wrote a version are, today, recorded nowhere: the decision in the vocabulary is that each
frozen version records the spec it was frozen against, and that isn't built. Decided: each stored version records
the core version and the spec version that wrote it, as root-tree entries (absent meaning unknown), in the first
release of the framework, so every version built after it has them. A version without them, a migrated one included, reads as
unknown, is recomputed under the current spec like any older one, and a ledger row can be told from the record instead of guessed. - What can't be recomputed is
unseen. If a rule needs an input the version lacks (a dependency's config, a type's width), the comparison lists it in the sameComparison::unseenthat core#262 added: the version and the path, with the reason from theunknownentry. A rule that needs a fact it can't see reads the strictest (the same wayEncodingsdoes when it isn'tcomplete), and the bump is at least a patch: a comparison that couldn't see part of what it compares doesn't say that nothing changed. It can be higher, since "strictest" can be a breaking change: a type of unknown width against a known one is possibly breaking. - The changelog shows the same: a change it found, and, as lines of their own, what it could not compare, in the words of
the reason.
comline doc --version Vfor a version with unknowns fails closed on what it would print from an unknown, asdocdoes for any consumer (format 4).
| Case | Result |
|---|---|
| A package migrated from format 3, source unchanged | Nothing found; see the first build |
| The spec has since added a guarantee an old protocol gives up | Recomputed on both sides: no change, no false major |
| A version from before dependency configs were kept (core#262) | unseen: Dependency("acme"), at least a patch: as now |
| A version that went through a lossy step, with an array whose length was lost, compared to today's | the path listed as unseen; possibly breaking |
A version whose release says 0.4.0 |
0.4.0, whatever the current rules would number it |
The first build after a migration
A migrated head lacks what the current core knows, so a fresh build of the unchanged working tree differs from it,
by exactly the unknowns. Without care, the first comline build makes a commit, with a bump of at least a patch, to
say that nothing changed.
Proposed: the build's "same as before" test compares modulo unknown: if the working tree equals the head at every field the head knows, nothing is committed and no version moves. What the build learned is not written back, since history is append-only, and is not needed: it is recomputed when read. Any real change is compared with the unknowns listed as unseen, as above.