One protocol core: a C ABI and WASM from one crate
Status: accepted · part of Shared runtime core · stage 2 · rollout steps 1–3 done (see What's built); steps 4–6 are future work
The question: what exactly the shared core owns, and what its boundary looks like, so that ten or more languages can bind it with no loss of correctness, conformance or performance.
The constraint this page answers to, as the user set it on 2026-10-25:
lets presume that comline in the future will have a lot of languages to be supported, possibly 10+, so nothing like correctness, conformance, performance and so on can be lost, there is in no moment that accepting lowering any standard will be possible to accept
So every choice below is judged on all three at once. Where a host makes a cost unavoidable, the page says so, and the stage 1 gates decide by measurement.
What's built
Built 2026-10-25 and 2026-10-26, in runtime except where noted. These
are rollout steps 1–3. The as-built API is the
C ABI reference.
Step 1: the core changes
- JSON-RPC without
std(runtime#49): ajsonrpcfeature that needsallocand nostd. Request ids are written by a stack writer, not throughto_string. - Body-bytes entry points (runtime#50):
encode_request_bodyonFramingandConnectionFraming.Callis unchanged. - The stream layer and the handshake step (runtime#51):
contract::stream:DEFAULT_MAX_FRAME(16 MiB),put_prefixandsplit_frame;CountBuf;HandshakeFrame::from_factsandcheck_bytes;RuntimeError::FrameTooLarge.- Fixed room for waiting calls (runtime#52):
State::with_capacityandunordered_with_capacity;RuntimeError::BusyandBufferTooSmall;State::encode_request_body, which commits nothing on overflow.- Frame limits in the transports (runtime#55): the transport owns the limit, with one default for everyone. See the frame-limit design.
- The simulator (simulator#29)
frames through the runtime's JSON-RPC and
encode_request_body. Its vendored JSON-RPC copy andPreencodedare deleted. It still keeps its own request ids instead of usingState, because it plays both ends on one simulated clock.
Step 2: the C ABI (runtime#53), crate
comline-runtime-ffi:
- ABI 0.1, with
include/comline.hgenerated by cbindgen 0.29.2 and checked in. CI fails when it drifts. - A
cdyliband astaticlibnatively, and the same exports on wasm32, pluscomline_wasm_reserve_inandcomline_wasm_reserve_out. - Status codes, an opaque
comline_connection, a 48-bytecomline_event, theNAME_ESCAPEDflag andcomline_method_name_unescape. - Every function runs under
catch_unwindand poisons the handle on a panic. Nothing allocates aftercomline_connection_new, exceptjsonrpc@1decoding (see Found along the way). A call with overlapping buffers, or a length overisize::MAX, is refused. - CI jobs
ffiandffi miri.
Step 3: proving the boundary
- The vectors, from Rust, C and Node (runtime#56):
the
export_vectorsexample, a C harness (static, dynamic, and under ASan and UBSan), the JS gluejs/comline.mjswithcomline.d.mts(nowasm-bindgen), and a Node harness. The same 65 cases pass in all three. - Fuzzing and a differential test (runtime#57): the
ffi_feedandffi_vs_corefuzz targets, andtests/differential.rs. - Benchmarks (runtime#59): the C ABI's calls against the same work on the core, gated with the runtime's.
- Cheaper crossing (runtime#60): every ABI call costs fewer instructions.
Verification has the detail of step 3's checks.
The main ruleset requires ffi, ffi miri, wasm32 core and the fuzz
targets; see Branch rules.
What changed from the proposal
The implementation changed these, each checked against the source. Everything else on this page is as proposed.
- Method names follow runtime#45. The sketch said
namemust be an identifier, and that a received method with an escape fails to decode. As built: nameis any UTF-8 text, andjsonrpc@1writes it escaped as serde_json does. A name that isn't UTF-8 isCOMLINE_ERR_INVALID_ARGUMENT;- a received name is kept as the frame spelled it, and an event sets
COMLINE_EVENT_NAME_ESCAPEDwhen it has escapes; comline_method_name_unescapeturns it into UTF-8.Callis unchanged. The proposal borrowedCall::name(&'a str). The newencode_request_body(call_id, name: &str, …)takes the name as a borrowed&strinstead, soCallkeeps&'static strand nothing cascades into generated code.COMLINE_ERR_BUSY(23) ismax_waiting's overflow. The proposal left it open whether it is an error or the binding's job to queue. It is an error. The request spends no id and writes nothing, and queueing is the binding's choice. A one-way call needs no room, andmax_waiting0 serves requests and sends one-way calls only. The code wasn't in the sketch.BUFFER_TOO_SMALLreports the exact size with the connection unchanged.CountBufmeasures the frame, andState::encode_request_bodycommits no id and no waiting call on overflow.*writtencounts a stream's prefix too.start,encode_reply,handshake_encodeandcomline_method_name_unescapereport their sizes the same way.- The frame-limit design. It follows runtime#55:
- the transport owns the limit, with one default, 16 MiB, declared once in
contract::stream; - in the ABI the limit is
comline_config.max_frame: 0 is the default, and it may be raised tou32::MAXless the prefix; - sending over the limit is recoverable: nothing is written, no id is spent, the connection works on;
- receiving over it closes the transport, in stream mode. In message mode the frame is refused and the connection works on;
- the Rust drivers answer an oversized handler reply with a
Fault. The ABI reportsCOMLINE_ERR_FRAME_TOO_LARGEfromencode_replyand leaves that choice to the host; - there is no
config.idpkey for it. - One
encode_reply, built on the envelope. The lockstepFraminghas three (encode_response_ok,_errand_fault). The ABI has one, asConnectionFraming::encode_replydoes, which takes the envelope kind. AFaulttakes no body. - One-way depends on the mode, and a connection uses only the connection framings.
datagram@1is lockstep and has no connection form, socomline_connection_newreturnsCOMLINE_ERR_UNKNOWN_FRAMINGfor it. The ABI carriesdatagram@2andjsonrpc@1.datagram@2has no way to say a call is one-way. Onjsonrpc@1a one-way call is a notification, with noid.- An event sets
COMLINE_EVENT_ONE_WAYonly when the frame says so, never from a request id of 0. - Which fields each framing reads is stated per function and set out in
the reference.
datagram@2readsprotocolandcall_id.jsonrpc@1readsname, writes an error'serror_idas the JSON-RPCcode, and writes a fault as-32603. - JSON-RPC without
stdis confirmed.serde_jsonwithraw_valuebuilds withoutstd, and CI'swasm32 corejob buildsalloc,jsonrpcforwasm32-unknown-unknown. - Smaller differences from the sketch, all in
comline.h: comline_statusis anint32_twith constants, not an enum;- names are
const uint8_t *with a length, notconst char *; max_waitingis auint32_tandmax_frameauint64_t;comline_confighas a_reservedbyte that must be 0;comline_eventcarriescapabilities, for the handshake event;- a
comline_handshakestruct is whatcomline_handshake_checkreturns; - the constants are
COMLINE_DEFAULT_MAX_FRAMEandCOMLINE_MAX_FRAME_CEILING, where the sketch hadCOMLINE_MAX_FRAME. - The JS glue does not catch a panic. The sketch said it would. The trap is
thrown as a
WebAssembly.RuntimeErrorand the host discards the instance. The glue also reserves its regions on every call, not up front, and returns copies, so no view of wasm memory outlives a call. - Deferred:
comline_connection_sizeandcomline_connection_initare a later addition;comline@2's negotiation comes before 1.0 if it changes the event struct.
Found along the way
- runtime#58:
jsonrpc@1decoding allocates aftercomline_connection_new. The ABI promises no allocation, but the framing parses with serde_json, which allocates: - on any refused frame, because it boxes its errors;
- when a member name has an escape;
- when a value nests brackets three deep, as in
"params":[[1]], which is an ordinary frame.
The planned fix is a hand-written strict JSON scanner in the core that
allocates nothing, checked against serde_json by a differential fuzz target.
It is deferred: Comline's own formats come first, and JSON is left for
later. The exemption is narrow and written down: the test
jsonrpc_decode_allocates_nothing is ignored, and the fuzz targets and the
differential proptest let through only those cases, on jsonrpc@1 feeds.
Every datagram@2 feed and every encode must still allocate nothing.
The rest of this page is the proposal as written. Where the code has since changed it, the text says so.
Where things stand
As of the proposal, before the work in What's built. The simulator
and the core have both moved since: simulator#29 deleted the vendored JSON-RPC
and Preencoded named below.
The core already exists. It is runtime/core (crate comline-runtime) in
ComlineProject/runtime, and most of what a shared core needs is already
sans-IO, no_std and allocation-free.
Features (runtime/core/Cargo.toml): contract and packed@1 need only
serde's core traits. alloc adds the connection state, client and server.
std adds JSON, MessagePack, the in-memory and TCP transports, JSON-RPC and
package_abi. async and tokio add the async driver.
| Piece | Where | Tier | I/O |
|---|---|---|---|
Handshake frame: encode, decode, check; name_hash (FNV-1a-64) |
contract/handshake.rs (190) |
no_std, no alloc |
none |
Envelope (Ok / Err{id} / Fault, tag byte) |
contract/envelope.rs (121) |
no_std, no alloc |
none |
datagram@1 |
contract/framing.rs (208) |
no_std, no alloc |
none |
datagram@2 (kind byte, protocol varint) |
contract/connection_framing.rs (255) |
no_std, no alloc |
none |
BufMut, SliceBuf (a fixed buffer that records overflow) |
contract/buf.rs (80) |
no_std, no alloc |
none |
RuntimeError |
contract/error.rs (56) |
no_std, no alloc |
none |
jsonrpc@1 |
framing.rs (351) |
std (serde_json with RawValue) |
none |
Connection State: request ids, waiting stack, frame → Event::{Request, Reply, Stale} |
connection.rs, lines 44–176 |
alloc (waiting: Vec<u64>) |
none |
ComlineHandshake (the exchange) |
handshake.rs (144) |
alloc |
Transport-bound |
Blocking Connection, Client, Server |
connection.rs, client.rs, serve.rs |
alloc |
Transport-bound |
AsyncConnection |
async_connection.rs (918) |
std + futures |
async |
InMemory, Tcp (u32 LE length prefix, MAX_FRAME 16 MiB) |
transport.rs (226) |
std |
I/O |
Tokio Tcp (same prefix, MAX_FRAME declared again) |
async_connection.rs |
tokio |
I/O |
package_abi (abi_stable + libloading) |
package_abi/ |
std |
n/a |
What this shows:
- The protocol logic is already sans-IO.
State's own doc says it: "No I/O: bytes in,Events out". The blocking and async drivers share it. Everything host-bound (transports, the handshake exchange, both drivers, timers) sits on top and is not part of the proposal. - Four things stop it from being bound as it is:
- Framing
encode_requestis generic overP: SerializeandW: WireFormat. A C caller has bytes, not aSerializevalue. Call::nameis&'static str. A binding's names live in the host.jsonrpc@1sits behindstd, for module organisation and not a real need, and it allocates per encode (u64_bytesgoes throughto_string).- Byte-level limits are per transport.
MAX_FRAMEis declared twice in Rust, and other languages pick their own.
The simulator is the working precedent
(ComlineProject/simulator, Cargo.toml):
- It pins
comline-runtimeatrev = "6a208d78…"withdefault-features = false, features = ["alloc"]. The comment says why: "stdpullslibloading(dylib) which won't build for wasm32." - It uses
contract(the handshake, envelopes and both datagram framings) andformat::Packed, and builds towasm32-unknown-unknownwithwasm-bindgen(scripts/build-wasm.sh). - It hits three of the four gaps above and works around each one:
Preencoded(src/format.rs) is aWireFormatwhoseencodeignores the value and writes bytes encoded earlier. The engine encodes a call's body from the schema first, then hands&()plusPreencodedto the framing (src/engine.rs, around line 506). This is exactly the pattern a C ABI needs, because codecs live in generated code.- It vendors JSON-RPC (
src/framing.rs): "the same code the runtime ships, but the runtime keeps it behindstd(module organisation, not a realstdneed)". That is a copy, which is the drift this proposal is against. - It carries names separately. Its
WireFramingtakes(call_id, name)because the name "comes from a runtime-parsedProtocolShape(not a&'static strthe waycontract::Callwants it)". - It keeps its own request ids instead of using
State, soStatehas not been bound outside Rust yet.
What the core owns
The rule: the core owns everything that is bytes and integers, and nothing that is a host value.
- The handshake.
- Encode, decode and check of the 31-byte
comline@1frame. name_hash.- A sans-IO form of the exchange: "write our frame", then "check the peer's frame". The binding moves the bytes.
- Envelopes: the tag byte, the
Errordinal, andFault. - All connection framings:
datagram@2,jsonrpc@1, and later versions as they are written. (datagram@1is lockstep and has no connection form; see What changed.) - Request and reply encode go around a body encoded elsewhere (the
Preencodedshape, made a first-class entry point). - Decode returns the body as a span of the input.
- The connection state machine (
State): - request ids, in both directions;
- the waiting stack, or the unordered set for an async host;
- frame →
Request/Reply/Stale, including the "a reply that skips the innermost waiting call is a framing error" rule; abandonon timeout. The host's clock decides when; the core decides what a late reply means.- Byte-level limits.
- The stream length prefix (
u32LE) and the default frame limit, declared once, incontract::stream. - The varint rules: shortest form only, and fits in
u16. - JSON-RPC id range. Today these are per transport and per language. In the core they are one constant each.
What it doesn't own
- Codecs. Encoding a value of a generated type is generated native code in each language, from one layout plan. See stage 3. A schema-driven codec in the core is rejected below.
- Transports. Sockets, WebSockets, pipes and stdio stay in the host. The core sees frames, or a stream buffer the host filled.
- Async. No futures, callbacks, threads or timers cross the boundary. A
host drives the state machine from its own loop: asyncio, goroutines,
CompletableFuture,Task, promises. - Generated types, dispatch tables and handlers. These are generated per language. The core sees a protocol number, a call id or name, and body bytes.
- Per-connection state (
Extensions) and interceptors. These are host objects.
The C ABI
Crate layout
runtime/ ComlineProject/runtime
├── runtime/core/ comline-runtime: the reference, unchanged in role
└── runtime/ffi/ comline-runtime-ffi
├── Cargo.toml crate-type = ["cdylib", "staticlib", "rlib"]
├── cbindgen.toml
├── include/comline.h generated by cbindgen, checked in
├── src/lib.rs the extern "C" surface: handles, spans, codes
├── js/ ES module glue for the wasm32 build, plus .d.mts
│ └── test/ the Node harness
├── tests/ Rust tests, the C harness (tests/c), the differential test
├── examples/ export_vectors: the vectors, written out for C and Node
└── bench/ the instruction-count benches (a package of its own)
- One crate, two outputs. On native targets it builds a
cdylib(fordlopen-style hosts: Python, Java, C#, Ruby, Dart, PHP, LuaJIT) and astaticlib(for hosts that link at build time: Go with cgo, Swift, Zig, C/C++). Onwasm32-unknown-unknownthe sameextern "C"functions become the module's exports. - No
wasm-bindgenon this surface. The API is integers and pointers, which WASM exports carry as they are. Without a JS-specific layer, the.wasmis the C ABI. A non-JS WASM host (wazero in Go, wasmtime) then calls the same functions with the same semantics. The simulator useswasm-bindgenbecause its surface is JS objects; this one isn't. - It depends on
comline-runtimewithoutstd(default-features = false, features = ["alloc", "jsonrpc"]; see What changes in the core). Sopackage_abinever enters the build, and wasm32 works before stage 5. The FFI crate itself usesstdforcatch_unwind, and wasm32 hasstd. - The header is generated, then checked in.
- cbindgen is pinned (0.29.2), and CI's
ffijob fails wheninclude/comline.hdiffers from a fresh run, as the docs'cliandidldrift jobs do. - Reviewers then see every ABI change as a header diff.
- Why in
runtimeand not its own repo: the FFI layer is a thin translation ofcomline-runtime. Splitting it would put every core change through a two-repo cascade. The core's tests, vectors and fuzz targets already live there.
The API
The header and the C ABI reference hold the as-built API. The C sketch that stood here is gone. Its shape held:
- a version,
comline_abi_version(), and a header that carries it; - an opaque
comline_connection, made from acomline_config; comline_connection_feed,comline_connection_encode_requestandcomline_connection_encode_reply, one whole frame per call;- event spans as offsets into the caller's input;
- append-only status codes.
What changed lists where the built API differs from the sketch.
The rest of the surface is small:
comline_connection_start(conn, out, cap, written)writes this side's handshake frame.comline_connection_abandon(conn, request_id)gives up on a call after a timeout.comline_connection_is_waiting(conn, waiting).- Free functions with no handle:
comline_name_hash,comline_handshake_encodeandcomline_handshake_check, for hosts that run the handshake themselves;comline_supports; andcomline_method_name_unescape.
A call from Python, Go or Java is then:
generated codec: value -> body bytes (native, stage 3)
core: body -> frame (1 crossing)
host: write(frame) (native I/O, native async)
host: read() -> buffer
core: buffer -> event + spans (1 crossing)
generated codec: buffer[span] -> value (native, stage 3)
In Rust, comline-runtime keeps calling State and the framings directly.
The FFI layer is for every other language, and Rust never pays for it.
Rules
One crossing per frame, never per field.
- Each of feed, encode_request and encode_reply handles one whole
frame.
- Nothing calls back into the host: no callbacks, no host allocator, no
per-field accessor.
- An event is a plain struct the host reads with its own field loads.
No allocation crosses the boundary.
- Every output buffer is the caller's (out, cap, *written).
- Every input is borrowed for the duration of the call only.
- The one thing the library allocates is the handle. It does so once, in
comline_connection_new, sized by max_waiting, and frees it in
comline_connection_free. The host never frees library memory and the
library never frees host memory.
- After the handle exists, the happy path allocates nothing.
- State::with_capacity reserves the waiting calls' room in
comline_connection_new (a Vec reserved to max_waiting, not an
array), and jsonrpc@1 writes ids on the stack. Both apply to Rust
callers too, under
§4.6's budget
("zero allocations on the happy path").
- One exemption, as built: jsonrpc@1 decoding allocates through
serde_json. See Found along the way.
- A host with no allocator story of its own (embedded C, Zig) can get a
later comline_connection_size(cfg) / comline_connection_init(mem, …)
pair. That is a later addition, not part of the ABI as built.
Panics never cross.
- Since Rust 1.81, a panic unwinding out of an extern "C" function aborts
the process. That is never undefined behaviour, but aborting a Python
interpreter or a JVM is not acceptable either.
- Policy: the release profile keeps panic = "unwind". Every exported
function body runs inside catch_unwind, which costs nothing when there is
no panic.
- A caught panic marks the handle poisoned and returns
COMLINE_ERR_POISONED.
- Every later call on that handle returns the same code, so the binding
closes the connection.
- On wasm32, a panic is a trap (WebAssembly.RuntimeError). The JS glue
does not catch it: the trap reaches the host, which discards the instance.
Unwinding on wasm32 needs the exception-handling proposal, which this does
not depend on.
- Panics are bugs, not a control path. The decode paths are already
written as Option-returning split_at_checked chains. The ffi_feed and
ffi_vs_core fuzz targets run against the FFI entry points and fail if a
panic crosses.
Thread safety.
- A comline_connection* is movable, not shareable. Any thread may use
it, but calls on one handle must not overlap. The binding enforces this
however its host does: a mutex in Go, confinement or a lock in Java and C#,
a lock per handle in Python (the GIL alone is not enough on free-threaded
builds).
- Different handles are independent. The FFI layer keeps no global mutable
state. next_connection_id's static atomic lives in the blocking driver,
which the FFI doesn't use.
- The free functions are pure and re-entrant.
ABI versioning.
- The ABI has its own version, separate from the crate version
(comline-runtime is 0.1.0) and from encoding versions (datagram@2).
- The major changes when any existing signature, struct layout or code
meaning changes.
- The minor changes when functions, codes or flag bits are added.
- The header carries COMLINE_ABI_MAJOR / _MINOR.
comline_abi_version() returns the library's.
- At load, before any other call, a binding refuses a library
- whose major differs from its header's, or
- whose minor is lower than its header's.
It refuses with an error naming both versions. It never refuses lazily on
first use.
- Before 1.0, a minor counts as a major. A binding built against 0.n
accepts only 0.n, so nothing pre-release is mistaken for compatible.
- The freeze comes with the project's 1.0 announcement, not with a
milestone of the ABI's own (phases). The ABI stays
0.x until then.
- Structs are frozen within a major. New data comes through new
functions or new flag bits. A struct_size field is not used, because
every binding would have to honour it.
- Status codes and flag bits are append-only. A binding maps an unknown
status to a generic error carrying the number. RuntimeError is already
#[non_exhaustive], for the same reason.
- comline_supports(name, len) answers whether this library build carries a
framing or handshake name ("datagram@2", "jsonrpc@1", "comline@1",
"none"). Wire formats are the host's codecs, so none is listed. A binding
can then refuse a schema's @framing at setup, not at the first frame.
Zero-copy where the host allows.
- Input is borrowed. Events point into it by comline_span offsets, so
the core never copies a body out.
- The datagram framings already return params and envelopes as subslices
of the frame.
- jsonrpc@1 already borrows params, result, error.data and
method through RawValue and &str.
- Offsets, not pointers. A span stays valid if the host moves or
reallocates its buffer between the call and the decode. A garbage-collected
host (Java heap arrays, .NET with pinning only during the call, Go)
usually does exactly that.
- u32 spans are enough: the default frame limit is 16 MiB, and a
connection's limit can't pass u32::MAX less the prefix. They are also the
same width on wasm32.
- Two JSON-RPC edge cases surface as flags, not copies:
- Absent params or result decode as the literal null (a static
slice, not part of the input). The event sets BODY_ABSENT with an empty
span, and the JSON codec reads null. A null the frame spelled out sets
it too, since the framing can't tell the two apart.
- serde_json will not borrow a string that contains escapes, so a
method with an escape failed to decode. As built
(runtime#45): the
name is kept as the frame spelled it and the event sets NAME_ESCAPED;
comline_method_name_unescape reads it. The found-request-method-*
vectors pin it.
- Output is written once, straight into the caller's out. The body
passed to encode_* is copied exactly once, into the frame. That is the
same single copy the Rust path makes when Packed encodes into the send
buffer.
What changes in the core
All of these are small. Each one also helps Rust callers.
| Change | Why | Status |
|---|---|---|
Body-bytes entry points on Framing / ConnectionFraming (encode_request_body(…, body: &[u8], …)), the generic Serialize form becoming a wrapper |
The Preencoded shim, made first-class. The simulator can drop its copy. |
Done (runtime#50). The generic form is kept as it was; the simulator dropped Preencoded (simulator#29). |
Call::name borrowed (&'a str) rather than &'static str |
Host-owned names; the simulator's WireFraming workaround goes away. |
Not done, not needed. encode_request_body takes name: &str, so Call is unchanged. |
jsonrpc@1 behind alloc or its own jsonrpc feature, not std; ids written without to_string |
The simulator vendors it today. Whether serde_json's raw_value builds without std has to be checked first. |
Done (runtime#49): a jsonrpc feature. raw_value builds without std. |
State.waiting with a capacity fixed at construction |
No allocation on the happy path; an overflow becomes an error code, not a reallocation. | Done (runtime#52): State::with_capacity; overflow is RuntimeError::Busy. |
A sans-IO stream deframer: the u32 LE prefix plus one MAX_FRAME constant |
Replaces the two declarations in transport.rs and the tokio Tcp. |
Done (runtime#51): contract::stream. Both transports read it (runtime#55). |
A sans-IO handshake step (ours.encode, check(peer_bytes)) beside ComlineHandshake |
ComlineHandshake::exchange is Transport-bound. |
Done (runtime#51): HandshakeFrame::from_facts and check_bytes. |
A counting BufMut |
So BUFFER_TOO_SMALL can report the exact size needed without mutating state. |
Done (runtime#51, #52): CountBuf, and BufMut::overflowed. |
WASM
The same API, compiled to wasm32-unknown-unknown.
- comline-runtime-ffi exports the comline_* functions as they are.
- js/ holds a small ES module, hand-written, with no generated glue. It
- instantiates the module, and checks its ABI version and the
comline_event layout;
- reserves an input region and an output region in linear memory, through
two wasm-only exports (comline_wasm_reserve_in(len),
comline_wasm_reserve_out(len)) that return pointers into memory the
module owns. It does so on every call, which costs nothing when the
region is already big enough;
- reads comline_event with a DataView.
- A size reference point: the simulator's 2026-09-03 spike, built on
comline-runtime with alloc only, was an 88 KB .wasm.
The copy is the honest cost.
| Step | Cost |
|---|---|
Bytes from the socket (a Uint8Array on the JS heap) into linear memory |
one set(), a memcpy linear in frame size, per frame in |
| The call | a JS→WASM call; small in current engines, not free |
| Spans back | read through DataView; no copy |
| The body, for the generated JS codec | As built: a subarray of the Uint8Array the host fed, at the span's offset. The glue hands out no view of wasm memory. |
| A frame out | the codec's body is copied in, the frame is copied out: two memcpys |
- Views on
memory.bufferare invalidated when memory grows. As built, the glue takes fresh views after every reserve and returns copies, so a generated codec holds none across calls. - For small frames this can lose to hand-written TypeScript framing,
which works on the socket's
Uint8Arraywith no copy. Adatagram@2request header is 12 bytes for a protocol number below 128; the copy and the call are fixed costs around very little work. - That is why stage 4 moves TypeScript piece by piece, and only where the stage 1 gates show the core is at least as fast for that host. The browser has no native option, so there the comparison is WASM against pure TypeScript only.
The component model and WIT: a later option, not the plan. The WASM component model would let a WIT interface generate bindings for many host languages. As of this writing:
- Mature enough to rely on:
- WASI 0.2 (Preview 2, January 2024) and the component model in wasmtime, on the Rust host side;
wit-bindgenfor Rust guests;jcofor running components in Node and, after transpiling, in browsers.- Not mature enough to rely on:
- Host-side component support outside Rust and JS is uneven: Python, Go, Java, .NET, Ruby and PHP each depend on a third-party engine binding, at varying stages.
- Async in the component model is WASI 0.3 work, still in progress.
- Browsers have no native component support; it goes through
jco. - The canonical ABI copies values between instances. That is the cost this page avoids with spans.
So a WIT surface would not cover the languages Comline targets today without
also keeping the C ABI. Re-assess after stage 2 ships. If the WIT
toolchain has caught up by then, a WIT world mirroring comline.h is an
addition, not a replacement.
The package_abi leftover.
- runtime/core/src/package_abi pulls abi_stable and libloading under
std, which is why comline-runtime with std doesn't build for wasm32.
- The FFI crate avoids it by not enabling std, but the trap stays for
anyone else.
- Stage 5 removes it, or splits it into its own crate. It belongs to dylib
mode, which is a different idea from this core.
Per-language bindings
The rule: - Every language binds the core, natively or through WASM. - A pure implementation is allowed only where both native and WASM lose to it by measurement, against the stage 1 gates. - A pure implementation is held to everything the core is held to: - the byte-exact vectors; - differential fuzzing against the Rust core, with the same malformed inputs refused; - the benchmark gates. - It is an exception that each language's CI has to keep earning, not a default.
| Language | Mechanism | Ships as | Known costs and caveats |
|---|---|---|---|
| C, C++ | comline.h directly |
static or shared lib + header | None beyond the ABI rules; C++ can add an RAII wrapper. |
| Zig | @cImport("comline.h"), static link |
static lib + header, or a build.zig dependency |
None notable. |
| Rust | comline-runtime directly, no FFI |
crate | The reference; never goes through the C ABI. |
| Python | cffi (API mode, compiled) over the C ABI; pyo3 only if gates show it wins |
wheels per platform, via maturin or cibuildwheel |
ctypes is rejected: per-call overhead is far higher. pyo3 would bind the Rust API, not the C ABI, so it would be a second surface and has to earn that by measurement. Free-threaded builds need the per-handle lock. |
| Go | cgo, static link | module with vendored .a per GOOS/GOARCH, or a build step |
Each cgo call has a fixed overhead (tens of nanoseconds), so one crossing per frame matters most here. cgo needs a C toolchain and complicates cross-compiling. Without cgo (CGO_ENABLED=0, which many Go shops require): the same .wasm in wazero (a pure-Go runtime), or a pure Go implementation under the rule above. Measurement picks. |
| Java | Panama FFM (java.lang.foreign, final in Java 22), header via jextract |
jar with native libs per platform | JDK 21 and older need JNI or JNA. Android has no FFM; it uses JNI. Heap arrays need copying to a MemorySegment, or the host reads into off-heap segments from the start. |
| Kotlin | JVM: as Java. Android: JNI. Native: cinterop |
as Java; .aar for Android |
Three mechanisms for one language; the JVM one comes first. |
| C# / .NET | P/Invoke, [LibraryImport] source-generated marshalling |
NuGet with runtimes/<rid>/native/ |
Blittable structs and spans keep marshalling at zero; Span<byte> pins only for the call. |
| Swift | the C header through a module map | SwiftPM binaryTarget (XCFramework) on Apple; static lib on Linux |
XCFramework covers device and simulator slices. |
| Ruby | ffi gem; a native extension if gates require |
platform gems | ffi goes through libffi; per-call cost is higher than a compiled extension. |
| Dart / Flutter | dart:ffi, header via ffigen |
package with native libs; Dart's build hooks ("native assets") where available | Check the build-hooks status when implementing; Flutter mobile needs the Android and iOS triples. |
| PHP | FFI extension (PHP 7.4+) |
.so/.dll + header in the package |
ffi.enable defaults to preload, so FFI is off outside the CLI unless preloaded. Per-call FFI overhead is high; a pure implementation may win here, and the gates decide. |
| Lua | LuaJIT: ffi.cdef. PUC Lua: a small C module over the ABI |
LuaRocks rock | LuaJIT's FFI is fast; PUC Lua needs the C module. Luau has no FFI, so the embedder exposes it. |
| Node, Bun, Deno | napi over the C ABI (static link into a .node addon); WASM as a fallback |
npm, one package per platform under optionalDependencies |
napi calls are cheap, but the stage 4 small-frame question applies. Bun and Deno load napi addons to differing degrees; WASM works everywhere. |
| Browser | WASM only | npm, or the .wasm + js/ glue |
The linear-memory copy, stated above. Pure TypeScript is the alternative, and the gates decide. |
Distribution
This is a sketch and an open question. Nothing in it is built.
- Releases, registries and prebuilt binaries follow the post-1.0 rules on the phases page.
- Until then, bindings pin a runtime revision.
Targets. Prebuilt binaries per triple, from runtime's release
workflow:
x86_64-unknown-linux-gnu,aarch64-unknown-linux-gnu, and their-musltwins;x86_64-apple-darwin,aarch64-apple-darwin;x86_64-pc-windows-msvc,aarch64-pc-windows-msvc;aarch64-linux-android,armv7-linux-androideabi;aarch64-apple-ios,aarch64-apple-ios-sim;wasm32-unknown-unknown.
How each ecosystem ships them:
| Ecosystem | Pattern |
|---|---|
| Python | wheels tagged per platform (manylinux, musllinux, macOS, Windows) |
| npm | one package per platform, selected by os/cpu under optionalDependencies, plus a WASM fallback package |
| Maven | one jar per platform classifier, or one jar holding every platform's library, extracted at load |
| NuGet | runtimes/<rid>/native/ in one package |
| Go | static libs per GOOS/GOARCH committed to the module with #cgo LDFLAGS per platform. A build step that needs cargo is the alternative, and a worse experience. |
| SwiftPM | binaryTarget with a checksummed XCFramework |
| Ruby | platform gems |
| Dart, PHP, Lua | native libs inside the package; specifics per ecosystem |
Reproducibility.
- Release builds use a pinned stable toolchain. The runtime repo pins stable
1.98.0 in rust-toolchain.toml (runtime#30),
which replaced an unpinned nightly.
- Builds run with --locked, --remap-path-prefix and a fixed
SOURCE_DATE_EPOCH, and the release publishes checksums.
- Each binding records the core release it bundles in a file, as
VECTORS_REV does for conformance today. A language bumps it on purpose.
Open:
- whether binaries are released from runtime once and pulled by each
language's packaging, or built in each comline-<lang> repo from a pinned
rev;
- signing;
- the Go repository size if every platform's .a is committed;
- how bumping the bundled core rev is ordered across repos, as
comline-core and STD_REV bumps already are.
How it fits one repository per language
The 2026-08-31 decision still holds:
runtimeholds the core,comline-runtime-ffi,comline.h, the conformance vectors and the fuzz targets, and releases the prebuilt libraries.- Each
comline-<lang>holds: - that language's thin binding (load, version check, spans to host slices, the per-handle lock);
- its generator backend;
- its CI with the stage 1 gates. The binding and the generated code it serves still change together in one CI.
comline-typescript'sruntime/shrinks toward a binding as stage 4 moves pieces, where the gates allow.comline-rustis unaffected; Rust uses the crate.
This answers the open seam in
core-target-contract.md §4.7,
"the per-language-runtime ↔ core-runtime seam". It does not answer §5:
that is dylib mode, a generated package as a loadable object, and it
stays a separate idea.
Alternatives rejected under the constraint
| Alternative | Why it's rejected |
|---|---|
WASM everywhere. One .wasm, every host embeds an engine. |
Performance: outside JS, a WASM call through an embedded engine (wasmtime, wazero) costs more than a native call, and every frame is copied into linear memory and out. Distribution: every host ships an engine, itself a native dependency, so nothing is saved. WASM stays for the hosts that need it: browsers, and Go without cgo if it measures better there. |
| Per-language reimplementation. The status quo. | Correctness and conformance: the copies drift. Five interop bugs in TypeScript were found the day the corpus landed, and comline-typescript#57 is open. Every wire feature this month was built twice. At ten languages that is ten copies of every fix. Kept only as the measured exception above. |
A schema-driven codec in the core. Hosts hand values to the core, which encodes them from the schema, as the simulator's packed.rs does over serde_json::Value. |
Performance: every call converts host values into a core representation, field by field, across the boundary. That is many crossings per call, or a second serialization to get one. It is slower than native generated code in every language. Codecs are stage 3, generated per type. The simulator does this because it is schema-driven by nature, not for speed. |
abi_stable (as package_abi used). |
It is a Rust-to-Rust stable ABI: its types (RVec, prefix types) are for Rust on both sides. A C, Go or Java consumer can't use them, and the C ABI is what those consumers need. This is how the 2023–24 attempt stalled. |
wasm-bindgen as the WASM surface. |
It produces a JS-only surface. The .wasm would then not be the C ABI, and Go-without-cgo through wazero would need a second surface. |
One crossing per field or per event poll (a feed, then poll until empty). |
Two or more crossings per frame where one does. On cgo, PHP and Ruby ffi the fixed per-call cost dominates small frames. feed returns the frame's event directly; a stream buffer holding several frames is fed again from consumed. |
Open questions
Settled by the build:
- Handle memory for no-allocator hosts:
comline_connection_size/comline_connection_initare a later addition. max_waitingoverflow: an error to the caller ofencode_request,COMLINE_ERR_BUSY. Queueing is the binding's job.- Whether
jsonrpc@1'sserde_jsondependency fits the no-stdFFI build: it does, as is. Its decoding allocates, which breaks the ABI's no-allocation rule; that is runtime#58, deferred.
Still open:
- Whether
comline@2's negotiation (protocol numbers handed out at the handshake) changes the event struct. If so, it should land before 1.0, when the ABI freezes. pyo3vscffifor Python, and whether a second binding surface is ever worth it. Decided by gates, but the default needs picking.- Go without cgo: wazero against pure Go, measured on the same benchmarks.
- Distribution: everything flagged in Distribution.
Rollout
Steps 1–3 are done (What's built).
- The core changes in
comline-runtime, each with vectors: done. Body-bytes framing entry points,jsonrpc@1out ofstd, fixed-capacitywaiting, the stream deframer with one default limit, and the sans-IO handshake step. BorrowedCall::namewas not needed. The simulator moved onto them and deleted its vendored JSON-RPC andPreencoded. comline-runtime-ffiwithcomline.hat ABI0.x: done. Native and wasm32 builds, the header drift check, and FFI fuzz targets in stage 1's harness.- A C test binding in
runtimeCI runs the conformance vectors through the C ABI and the wasm32 build: done. C and Node replay the same 65 cases as Rust. This proves the boundary before any language depends on it. - The first real binding: TypeScript, through WASM (stage 4). Each piece moves only where instruction counts show the core costs no more than the TypeScript it replaces. napi and Go (cgo, the most expensive call) come later, chosen by those numbers.
- Bindings pin a runtime revision. Releases and prebuilt binaries come with the 1.0 rules (phases).
- Further languages each bring a binding and a generator backend in their own repo.
What it touches
- Runtime:
- the
comline-runtimechanges above; - the new
runtime/fficrate, header and JS glue; - FFI fuzz targets;
- the release workflow;
- the workspace
Cargo.toml, where theruntime-langs/cstub (anaddfunction) was removed by runtime#53. - Simulator: dropped its vendored JSON-RPC and the
Preencodedshim (simulator#29). - comline-typescript: stage 4, behind the gates.
- Each future
comline-<lang>: a binding, its packaging, and the gates. - Docs:
- the runtime guide's "no FFI boundary";
core-target-contract.md§4.7 (answered here) and §4.6'sno_stdlayering table;- a
comline.hreference page: done, the C ABI reference.