user-defined allocators

[pnkfelix]

We will want a user-defined allocator API in our standard library. It has been moved off of the 1.0 requirements, but it remains an important post 1.0 task.

See for reference:

• Implement allocators, integrate with box, smart pointers, containers rust#12038
• RFC: Allocator trait #39
• Allocator RFC, take II #244
• WIP: Add Faux allocator API rust#19320
• Allocators, take III #1398

Also, it is good to be aware of whatever happens with placement new and the box syntax: See e.g. #413 (though that is arguably a draft RFC that has been filed as an issue).

I've played around with the concept a bit in Jurily/rust-allocator, but many of my use cases would be suboptimal with panic. While it might make sense to kill everything when we run out of global heap, it would be unnecessarily fragile for size-limited pools (sandbox) or allocators that don't implement some operations (arena cannot reallocate).

What exactly is the scope of this API?

[Gankra]

All Collections and SmahtPtr types should absolutely support the custom allocator API, as far as I'm aware. This results in some... nasty... details as you're finding.

[reem]

I don't think it's reasonable to expect allocation errors to bubble up all the way through to users of collections and smart pointers since in the vast majority of cases the only reasonable thing to do is abort or panic and we'll just end up with a million meaningless and annoying unwraps all over most rust code. Remember, for 99.9% of Rust code which will just use the default allocator, an allocation failure means OOM and handling anything under OOM is basically impossible.

[thestinger]

an allocation failure means OOM and handling anything under OOM is basically impossible.

There's often a sane way for the application to handle it, and in some niches it must be handled. Rust isn't entirely ruling out those niches by doing this because the alloc and core crates are (mostly) usable with that constraint but it isn't a non-issue.

[reem]

@thestinger The allocator API should definitely support handling OOM - I'm just saying it's unreasonable for most collections and smart pointers to propagate those errors in any way other than panicking or aborting. You could still write custom collections or smart pointers which exposed the error.

@reem You can do anything under OOM as long as you don't allocate more memory. For example, you can free memory or wait for some other thread to do so.

I find it a perfectly reasonable expectation from a modern systems language to not need to fork the standard library for the cases where that's important. It's not like we don't have enough precedents on how to solve this. We even have default type params now.

The main problem is that bubbling up errors from multiple sources is an unsolved ergonomic problem in libstd, which is just hilarious since it has access to Result, try! and i32.

[pythonesque]

I agree with @reem. Rust can't enforce a constraint like "doesn't allocate memory" in destructors for generic collection types, and panicking can allocate memory too, so if you really need to handle OOM I don't think you can rely on the standard library anyway.

[thestinger]

Rust can't enforce a constraint like "doesn't allocate memory" in destructors for generic collection types

The inability to enforce error handling in the type system certainly doesn't prevent handling that class of error correctly. We can't enforce that other types of errors are handled correctly either...

and panicking can allocate memory too

This means that panicking is not a solution to handling out-of-memory errors, which is already a given considering that the niches where this needs to be handled are often the same ones where unwinding isn't supported / tolerable.

don't think you can rely on the standard library anyway.

The core crate already works fine, as does most of the alloc crate. The alloc crate could be improved to fully handle this situation via alternative methods, as could other areas of the standard libraries.

[pythonesque]

My point was that for an application that cares about OOM, it won't be able to use any data structures without auditing the code to make sure it can't allocate memory in destructors, since Rust can't enforce that behavior. Once that code is audited, it has to be forked anyway, unless there are actual guarantees that that won't occur in the future. The standard library (outside of libcore and liballoc) could be changed to guarantee it wouldn't allocate in destructors as long as the generic destructors didn't allocate, I guess. I'm pretty sure that's not currently a guarantee.

[thestinger]

My point was that for an application that cares about OOM, it won't be able to use any data structures without auditing the code to make sure it can't allocate memory in destructors, since Rust can't enforce that behavior.

I don't know why you're getting caught up on destructors. Rust can't enforce effects for any functions but that doesn't mean that effects can't be documented / guaranteed.

Once that code is audited, it has to be forked anyway, unless there are actual guarantees that that won't occur in the future.

No, it just has to make the guarantee that it won't allocate. It's not possible to write any code at all if you're unable to trust APIs to uphold their contracts... this is no different than any other part of the API.

The standard library (outside of libcore and liballoc) could be changed to guarantee it wouldn't allocate in destructors as long as the generic destructors didn't allocate

The core crate doesn't allocate. There's nothing special about destructors when it comes to this. There's nothing stopping any crate from documenting the API surface that's either free of dynamic allocations or that returns errors on allocation failures.

I'm pretty sure that's not currently a guarantee.

It's already a strong guarantee in core. The fact that it isn't documented elsewhere is a flaw to fix, not a reason to avoid working on it...

[thestinger]

"it's not implemented" makes no sense as the rationale for "it shouldn't be implemented"

[pythonesque]

Okay, you've convinced me, as long as the guarantee only holds if unwinding is turned off.

[rrichardson]

Are there plans for Box to be able to reference the allocator that allocated it? I am thinking of arena style allocators. Drop would need to be self.myalloc.free(self.ptr) instead of free(self.ptr)

[pnkfelix]

@rrichardson one example from RFC PR 244 shows embedding "the allocator" into the allocated object. (And then the embedded "allocator" is really just a reference to the actual shared state of the allocator itself.)

I don't know how much the eventual allocator design will continue to look like RFC PR 244, but I think we do intend to continue to allow the encoding of such back-references (with the expectation that many allocators will be represented by zero-sized types to avoid unnecessary overhead in the allocated state).

[nmyers12]

We have a simulation benchmark, with plausible memory usage patterns, that in some circumstances runs eight times faster with custom allocators than with global malloc, on modern hardware. The program spends a negligible amount of time actually allocating and freeing memory; effectively all of the runtime difference is the result of better or worse cache locality resulting from the elements used together being allocated together, despite being allocated at widely different times.

The simulation benchmark models a system composed of subsystems that are activated, one at a time, in random order. When a subsystem is activated, it runs for a while touching memory allocated during previous activations, and doing some further allocating and freeing before returning control to the dispatcher. This usage pattern is far from unusual in real systems: imagine a web service that handles various POSTs that touch different database tables, or a client with multiple (tab or window) workspaces.

Fully integrating allocators into a foundation library has profound architectural costs, particularly as regards safety. E.g., moving an object with parts that live in one allocator into a container constructed to use another to has typically catastrophic consequences. The need to store an allocator pointer in the container and in elements adds space overhead and cache pressure. Sharing an allocator object among different containers or elements in a container presents complex ownership questions. Nonetheless, the profound performance consequences (8x!) justify quite a lot of pain. These consequences are not likely to fade away with future changes in system design.