Skip to main content

Collections get a real ABI, closures get capture inference, and dispatch goes through one authority

· 9 min read
daily-progress: Part 9 of 11

A HashMap<string, int> has been working for a while. A HashMap<Point, int> — where Point is a record you defined — has not, and the reason is unglamorous: the string-keyed map knew exactly how big a string handle was and how to drop one, and a map keyed on an arbitrary record knew neither.

That’s the gap these two days were about. The string-keyed map is a special case that happens to be easy. Making the general case work means the runtime has to be told, for every element type, how wide a slot is, how to align it, how to hash it, how to compare two of them, and how to drop and clone one. None of that is knowable from inside the runtime — it lives in the compiler, which means it has to be handed across the ABI boundary as data.

What a layout descriptor carries

The new map (1d55e78a) stores opaque key and value blobs in one byte array and delegates identity to thunks the compiler supplies. The slot geometry is computed up front:

key_offset = align_up(1, key_align)
val_offset = align_up(key_offset + key_size, val_align)
stride     = align_up(val_offset + val_size, max(key_align, val_align))

That’s the whole reason a record key works now. The map doesn’t know what a Point is — it knows the key is key_size bytes at key_align alignment, and it strides through the entries array by stride. A Point { x: i64, y: i64 } is 16 bytes at 8-byte alignment; a record with a bool and an i64 packs differently, and the descriptor carries that difference instead of the runtime guessing.

The descriptor also carries function pointers: a hash thunk, an equality thunk, and — added a few commits later (83189329) — drop and clone thunks, so the map can tear down or duplicate elements without knowing their shape. Those descriptors are snapshotted by value into the map at construction, not held by reference, because the alternative is a map outliving the global it points at.

Here’s the surface that drives all of it:

record Point {
    x: i64,
    y: i64
}
 
fn main() -> i64 {
    let cells: HashMap<Point, i64> = HashMap.new();
    cells.insert(Point { x: 1, y: 2 }, 10);
    cells.insert(Point { x: 3, y: 4 }, 20);
    let present: bool = cells.contains_key(Point { x: 1, y: 2 });
    let n: i64 = cells.len();
    if present {
        return n;
    }
    return 0;
}

There’s nothing in that program that says “layout-backed.” The checker decides. The rule (c5abc899) is that a record is admitted as a key — or as a HashSet element — only if it’s hash-eligible: every field is itself hash-eligible, recursively, with no floats and no non-comparable members. If Point had an f64 field, this program would be rejected at the checker boundary, not miscompiled into a map that hashes bit patterns of NaN. The hash-eligibility predicate shares the same aggregate walker as the equality check (59cf3ef8), so the two answers can’t drift apart.

HashSet got the same treatment by being almost nothing (2506aac7): it’s the layout map with a zero-sized value. The value layout is size = 0, align = 1, the existing ZST gate in the map constructor covers it, and every set operation delegates straight through. There was no reason to write a second hash table.

The codegen side (c9e10824) wires HashMap.new and HashSet.new to the right constructor, and — the part I want to flag — routes the drop helper by key kind:

HashMap { key: UserRecord, .. } -> hew_hashmap_free_layout
HashMap { key: _, .. }          -> hew_hashmap_free_impl   // legacy string-keyed

The discriminator is “is the key a user record,” which the checker already decided when it admitted the layout constructor. So “user-record key” and “layout-backed” are the same fact, established once, read in two places.

Two honest gaps here. The clone path still routes through the legacy string-keyed helper while drop now consumes layout-keyed handles — the layout-keyed clone runtime entry isn’t wired yet. And non-actor map/set locals aren’t yet routed through the drop dispatcher; actor-state fields are. So a map you store in actor state gets cleaned up correctly today, and a map you hold in a plain local function variable does not. That’s a leak with a known address, not a mystery.

[Update, June 2026: both gaps closed before v0.5.0 shipped — clone now routes through layout-keyed entry points (hew_hashmap_clone_layout and its set sibling, superseding the legacy helpers), and local HashMap/HashSet handles drop at scope exit.]

Closures stop guessing how they capture

A closure that reads a captured variable and a closure that mutates one are different things, and until now the checker treated capture as a single mode. It now infers (0b955839).

The finalizer scans the closure body once. If a captured binding appears in a mutating position — an assignment, a compound op, a mutating method call — the capture is BorrowMut. If it’s only read and isn’t Copy, it’s Borrow. Copy and explicit move are unchanged. Each capture also records which rule fired, so a later pass can tell an inferred borrow from an explicit move without re-deriving it.

fn run() -> i64 {
    let base: i64 = 100;
    let owned = move |delta: i64| base + delta;     // explicit move
    let label: string = "subtotal";
    let tagged = |delta: i64| label.len() + delta;  // inferred borrow
    return owned(5) + tagged(2);
}

base is Copy, so the explicit move costs nothing. label is where the inference earns its keep: it’s a string, non-Copy, read but never mutated, so the capture comes out Borrow — the closure shares the string instead of stealing it. That program returns 115 ("subtotal" is 8 bytes).

The other half is escape classification. The checker now produces one fact per closure: Local if it can prove the closure is only ever called directly inside the block that introduced it, Escapes if it’s returned, stored, or handed to a higher-order function, and Forked for closures used inside fork bodies. Local requires positive proof, not absence of evidence — a closure is local only when every use is a direct call, which is the kind of thing that’s safe to keep on the stack.

There’s a diagnostic in here that I’m proud of and haven’t fully cashed in yet: NonSyncMutCaptureCrossesSuspend. If a closure captures a non-Sync binding by mutable borrow and the body has a suspend point — an await, a channel recv — that’s the shape of a data race waiting for an auto-lock pass that doesn’t exist yet. The diagnostic fires now so the seam is named; the pass that subscribes to it lands later. None of this couples to the lowerer yet either — it’s checker-only metadata sitting on the type-check output, waiting for the MIR side to consume it. The ClosureEnvLayout foundation it’ll consume got published (fc1a6285) and frozen (873638d1) in the same window, so the producer and the consumer agree on a shape before either one is load-bearing.

dyn Trait values become real fat pointers

Dynamic dispatch through dyn Trait typechecked before these two days, but the value-level representation was fail-closed — the type existed, constructing one errored out. That’s now wired end to end.

record Circle {
    radius: i64
}
 
record Rectangle {
    w: i64,
    h: i64
}
 
trait Describe {
    fn describe(val: Self) -> string;
}
 
impl Describe for Circle {
    fn describe(c: Circle) -> string { "circle" }
}
 
impl Describe for Rectangle {
    fn describe(r: Rectangle) -> string { "rectangle" }
}
 
fn print_shape(shape: dyn Describe) {
    println(shape.describe());
}
 
fn main() -> i64 {
    print_shape(Circle { radius: 1 });
    print_shape(Rectangle { w: 2, h: 3 });
    return 0;
}

Running it prints circle, then rectangle.

A dyn Describe is a fat pointer: %hew.dyn.fat_ptr = { ptr, ptr }, data word first, vtable word second (6e6456ad). Passing a Circle where a dyn Describe is expected (53c795cf) assembles that pair — the data pointer from the source’s stack slot, the vtable pointer from a per-(trait, type) global. Those vtable globals get emitted as real definitions (e3d4655d), each slot pointing at an erased self thunk (687808dc) — a generated function that takes the data word as an opaque pointer, casts it back to the concrete type, and calls the real method. The vtable also carries a drop-in-place entry (f717cd45) so a dyn value can be torn down without knowing what’s behind it.

The piece I keep coming back to is that the vtable identity — (trait_name, concrete_type, vtable_entries) — is computed by the checker and survives to codegen unchanged. Codegen doesn’t re-infer which methods go in which slots; it reads the checker’s answer through a shared symbol-mangling function so both sides name the same global. That theme — compute the fact once, in the checker, and carry it across the boundary — is the whole arc of these two days, and it’s about to get a name.

One authority for “how does this call lower”

Here’s the thing I noticed that reframed the rest of the work. The collection method calls, the trait method calls, and the actor sends each had their own path through the compiler for deciding how a call lowers — which symbol, which calling convention, whether the receiver is consumed. Three subsystems, three opinions, and no single place to look when one of them was wrong.

ResolvedCall + ImplRegistry (b9de1e93) is the start of collapsing that into one structure. A ResolvedCall is the checker’s complete answer to “what does this call site do”: the impl it dispatches to, the runtime calling convention (ByValue, ByRef, ByRefMut, ConsumeReceiver), and whether it goes direct or through a runtime shim. The resolver is fail-closed — it returns a ResolvedCall or a structured LookupError, never a silent None that some downstream default arm papers over.

There’s a design constraint baked in that I want to call out, because it’s a bet about the future. The registry is pure data — no Rust closures, no boxed predicates stored inside it. Bound satisfaction is decided by a predicate passed in at the call boundary, not by a function living in the registry. The enforcement is almost funny: every type in the registry derives Serialize, and a Box<dyn Fn> can’t be serialized, so if anyone tries to stash a callback in there the build breaks. The reason is that I want this registry to eventually be expressible in Hew source itself, and that only stays mechanical if it never grows a host-language closure.

This stage is substrate only. The field is on the type-check output, populated by its own test, and nothing in HIR, MIR, or codegen reads it yet. The three existing dispatch paths still run. Standing up the authority and migrating the readers onto it are deliberately separate — the second is where the actual de-duplication happens, and doing it before the shape is proven would mean migrating onto something that might still move. The collection-dispatch resolver entries got populated right after (af50c201) so there’s a real consumer being fed, even if no production code consumes the output yet.

[Update, June 2026: the readers arrived. HIR lowering now consumes ResolvedCall, and the typed runtime-call descriptors it feeds landed for v0.5.0.]

Generic actors, const generics, and a cancellation type

Three smaller things landed alongside the substrate.

Spawning a generic actor used to silently lose its type arguments — spawn Foo<T>() parsed the <T>, threw it away, and produced an argument-less spawn. That’s fixed across parser, formatter, and checker (6a551a92):

actor Buffer<T> {
    let cap: i64;
 
    receive fn capacity() -> i64 {
        cap
    }
}
 
fn main() -> i64 {
    let buf = spawn Buffer<i64>(cap: 8);
    match await buf.capacity() {
        .Ok(n) => n,
        .Err(_e) => 0,
    }
}

The checker now resolves the turbofish into real types, checks the count against the actor’s arity, enforces trait bounds, and records the resolved arguments for the monomorphisation pass to pick up. Omitting the turbofish on a generic actor is a diagnostic — spawn Buffer(cap: 8) tells you to supply spawn Buffer<T>(...) — instead of a phantom empty spawn. The honest limitation is bigger than I first wrote down: all of that is checker-side. The runtime path for a generic actor isn’t wired yet — constructor arguments don’t reach the spawned actor’s fields, so the buffer above type-checks, runs, and then capacity() answers with whatever happened to be in that memory instead of 8. The same actor without the <T> returns 8 fine. That’s a silent miscompile, the exact failure mode this whole window is supposed to eliminate, so consider it flagged as a gap rather than a feature: the spawn-site resolution is real; making the runtime honour it is the next layer.

Const-generic parameters on machines (a432058d) got their parser surface, a checker side-table, and — the part that was more work than I expected — a small constexpr sub-engine that evaluates <const N: usize = 16> defaults and arithmetic over them (the seven operators, all with checked arithmetic). HIR and codegen population wait on the machine-monomorphisation substrate, which also landed in this window (4515d2d0), so the const value has a real ConstValue::Usize type and a mangling rule even though no machine declaration uses one end-to-end yet.

And CancellationToken became a frontend value type (e031a3be), with is_cancelled() lowering to a runtime observation (1401e8aa):

fn process(token: CancellationToken) -> i64 {
    var done: i64 = 0;
    var i: i64 = 0;
    while i < 1000 {
        if token.is_cancelled() {
            break;
        }
        done = done + 1;
        i = i + 1;
    }
    return done;
}

That checks and the observation lowers. What doesn’t lower yet is the creation path — the extern "rt" scope-handle plumbing that hands you a token in the first place reports E_NOT_YET_IMPLEMENTED at the MIR boundary, because the per-symbol runtime producers for the scope handles haven’t been wired. So you can take a token and poll it; you can’t yet stand one up from source. The type and the observation are the foundation; the scope wiring is the layer on top, and it’s not there.

If there’s a pattern across these two days it’s that one: build the descriptor, the fact, the authority — the thing the future feature reads — and be honest that the reader is sometimes still a commit or two away.

Hew Notes

Language development, engineering articles, releases and community projects. About once a month, with an extra issue for a significant release. Written in English.

Subscribe to Hew Notes