Implement reference types for .NET embeddings.

[peterhuene]

Now that the reference types proposal is implemented in Wasmtime, let's implement it for .NET embeddings!

This means fully supporting the ref values, ref value types, handling reference types in host functions, handling reference types when invoking Wasm functions, and properly rooting .NET GC heap references when passing in and out of WebAssembly.

[peterhuene]

Once this is implemented, we can release a 0.19.0 NuGet package.

[fitzgen]

FWIW, here is what it took to do this in the python API: bytecodealliance/wasmtime-py@b8532a4

We don't really integrate with the python reference counting collector, we just manually pin data that is referenced by an externref in a global map.

[peterhuene]

It'll be something similar for .NET, although we won't need to do any mapping ourselves.

We'll allocate a GCHandle that will act as the extern ref's value; the finalizer will free the GCHandle, allowing for collection provided the .NET object is no longer reachable. The .NET object won't need to be pinned as the GCHandle can track it and WebAssembly never needs the physical address of the object on the .NET GC heap.

[peterhuene]

I've got extern refs working for .NET embeddings, but I'm noticing that the finalizer isn't being invoked:

• Host calls wasmtime_externref_new_with_finalizer, the internal ref count should be 1 for the returned value.
• Pass reference into Wasm function (reference not stored in a table or global, simply passed to host function).
• Host calls wasm_val_delete on the reference value. This should be driving the reference count to 0 and thus invoking the finalizer.

I haven't debugged it yet, so I'll take a look on Monday.

[peterhuene]

So I'm seeing 5 increments and 4 decrements between a wasm_func_call invocation passing a ref value:

Creating extern ref with `wasmtime_externref_new_with_finalizer`.
Calling function...
Extern ref count: 1 -> 2
Extern ref count: 2 -> 3
Extern ref count: 3 -> 4
Extern ref count: 4 -> 5
Extern ref count: 5 -> 6
Extern ref count: 6 -> 5
Hello from C#, Peter!
Extern ref count: 5 -> 4
Extern ref count: 4 -> 3
Extern ref count: 3 -> 2
Wasm function returned...
Deleting previously created value with `wasm_val_delete`.
Extern ref count: 2 -> 1

The code:

using var engine = new EngineBuilder()
    .WithReferenceTypes(true)
    .Build();

using var module = Module.FromText(engine, "externref", @"
(module
  (import """" ""hello"" (func $.hello (param externref)))
  (func (export ""run"") (param externref)
    local.get 0
    call $.hello
  )
)");

using var host = new Host(engine);

host.DefineFunction(
    "",
    "hello",
    (string name) => Console.WriteLine($"Hello from C#, {name}!")
);

using dynamic instance = host.Instantiate(module);
instance.run("Peter");

And the value is thereby "leaked" (actually held on to by Wasmtime). I'm capturing the ref count stacks now.

[peterhuene]

increment: /home/peterhuene/src/wasmtime/crates/c-api/src/func.rs:213 (_wasmtime_func_call)
increment: /home/peterhuene/src/wasmtime/crates/wasmtime/src/func.rs:597 (Func::call)
increment: /home/peterhuene/src/wasmtime/crates/wasmtime/src/func.rs:279 (Func::new)
increment: /home/peterhuene/src/wasmtime/crates/c-api/src/func.rs:91 (create_function)
increment: /home/peterhuene/src/wasmtime/crates/c-api/src/ref.rs:129 (wasmtime_externref_data)

decrement: /home/peterhuene/src/wasmtime/crates/c-api/src/ref.rs:142 (wasmtime_externref_data)
decrement: /home/peterhuene/src/wasmtime/crates/c-api/src/func.rs:105 (create_function)
decrement: /home/peterhuene/src/wasmtime/crates/wasmtime/src/func.rs:311 (Func::new)
decrement: /home/peterhuene/src/wasmtime/crates/c-api/src/func.rs:247 (_wasmtime_func_call)

Looks like there's no matching decrement for the clone in Func::call. Still digging.

[peterhuene]

I see, I didn't expect the call arguments to end up in the activations table as the reference value is always rooted by the caller to wasm_func_call. This is problematic as there is no way to manually trigger the GC from the C API (afaik), not that we would want to do it after every call to wasm_func_call anyway.

@fitzgen thoughts on the above?

[peterhuene]

I deleted the previous comment regarding a leak on dropping the store; there was actually a reference to the store alive from the .NET side because a Dispose call was missing (DefineFunction returns a disposable object, thus the store was only dropping if a .NET GC occurred). When the store drops, the extern ref is finalized as expected.

[fitzgen]

We can add a wasmtime.h function to do GC, or you can pass in a bunch of references into Wasm in a loop until GC is triggered.

FWIW, I didn't test GC reclaimation in wasmtime-py, since we have some tests for that in Rust, but on further reflection it probably does make sense to test the interaction with host objects and finalization.

[fitzgen]

Also, should StoreInner::drop do a GC?

I ask because the extern ref is definitely leaked even if it's in the activation table when the store gets dropped by the host.

This is interesting, I would expect that it would be cleaned up via the activations table running Drops on its elements. I'll see if I can make a Rust-only reproducer for this.

Oh nevermind, just saw your update.

[fitzgen]

Here is a PR to add a C API to do GC: bytecodealliance/wasmtime#2052

[peterhuene]

For my understanding, why are wasm_func_call extern ref arguments put in the activation table? Given the extern refs don't end up in a table or a global in this case, it is a little surprising to me that Wasmtime itself would keep a reference alive independently of the target Wasm code.

I think exposing a GC-triggering method in the .NET API and letting the users manually collect if desired (they'll know best when to do so given their use case) make the most sense.

[fitzgen]

For my understanding, why are wasm_func_call extern ref arguments put in the activation table? Given the extern refs don't end up in a table or a global in this case, it is a little surprising to me that Wasmtime itself would keep a reference alive independently of the target Wasm code.

Yeah, this is due to the deferred reference counting for references inside Wasm frames, where we don't actually increment/decrement the ref counts. Instead we over-eagerly keep them alive until we do a GC, when we find the precise set that need to be alive, and reclaim the references that were in the over approximation. See https://github.com/bytecodealliance/wasmtime/blob/main/crates/runtime/src/externref.rs#L61-L100 for more details.