`slice::from_raw_parts` returns a different address in const context for u8

[mxk]

Possibly as a result of some u8-specific optimizations, a const &[u8] returned by std::slice::from_raw_parts has a different address than all other uses of the same expression. Forum discussion.

I tried this code:

Playground

use std::ptr::NonNull;
use std::slice::from_raw_parts;

const PTR_U8: *const u8 = NonNull::dangling().as_ptr();
const CONST_U8_REF: &[u8] = unsafe { from_raw_parts(PTR_U8, 0) };
const CONST_U8_PTR: *const u8 = unsafe { from_raw_parts(PTR_U8, 0).as_ptr() };
static STATIC_U8_REF: &[u8] = unsafe { from_raw_parts(PTR_U8, 0) };

const PTR_U16: *const u16 = NonNull::dangling().as_ptr();
const CONST_U16_REF: &[u16] = unsafe { from_raw_parts(PTR_U16, 0) };

const fn const_u8_fn() -> &'static [u8] {
    unsafe { from_raw_parts(PTR_U8, 0) }
}

fn main() {
    let ptr_u8 = unsafe { from_raw_parts(PTR_U8, 0) }.as_ptr();
    let ptr_u16 = unsafe { from_raw_parts(PTR_U16, 0) }.as_ptr();

    assert_eq!(ptr_u8, PTR_U8); // OK
    assert_eq!(ptr_u8, CONST_U8_PTR); // OK
    assert_eq!(ptr_u8, const_u8_fn().as_ptr()); // OK
    assert_eq!(ptr_u8, STATIC_U8_REF.as_ptr()); // OK
    assert_eq!(ptr_u16, CONST_U16_REF.as_ptr()); // OK
    assert_eq!(ptr_u8, CONST_U8_REF.as_ptr()); // ERROR
}

I expected to see this happen: All assertions should pass.

Instead, this happened:

thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `0x1`,
 right: `0x55f434eaa038`', src/main.rs:25:5

Meta

rustc --version --verbose:

rustc 1.65.0 (897e37553 2022-11-02)
binary: rustc
commit-hash: 897e37553bba8b42751c67658967889d11ecd120
commit-date: 2022-11-02
host: x86_64-pc-windows-msvc
release: 1.65.0
LLVM version: 15.0.0

Backtrace

thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `0x1`,
 right: `0x55c4dc25b038`', src/main.rs:25:5
stack backtrace:
   0: rust_begin_unwind
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/panicking.rs:142:14
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/panicking.rs:181:5
   4: playground::main
             at ./[src/main.rs:25](https://play.rust-lang.org/#):5
   5: core::ops::function::FnOnce::call_once
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/ops/function.rs:248:5

[steffahn]

@rustbot label A-const-eval, T-compiler, A-slice

In the linked URLO discussion, @LegionMammal978 mentioned some part of the compiler that seems relevant for why this behavior only affects u8 slices in particular, the code history is a bit hard to navigate in the git blame, but AFAICT, @oli-obk and @RalfJung are involved.

Maybe they also have more insight into whether this kind of behavior classifies as intended or bug, since there definitely are cases where reference addresses of const-eval results being soft-of “unstable” is intended behavior, albeit surprising behavior to some, but I feel like those cases are usually different in that a reference to some actual static(-promoted) data and memory is created.

[oli-obk]

It's not promotion related: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=b89aaee74c5da29370ba7107576adc7a

So I'm guessing some sort of codegen issue, because otherwise turning the reference back to a pointer in const land would cause the same issue, but it doesn't: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=6ac87b271a9b495b6497015d95b57815

[scottmcm]

Minimized, and emphasizing the same thing oli just showed about this being related to the const item somehow, not a problem in MIR interpreting itself:

assert_eq!(
    // as_ptr INside the `const{}`
    (const { (unsafe { std::slice::from_raw_parts(3 as *const u8, 0) }).as_ptr() }),
    std::ptr::invalid(3),
); // PASSES
assert_eq!(
    // as_ptr OUTside the `const{}`
    (const { (unsafe { std::slice::from_raw_parts(7 as *const u8, 0) }) }).as_ptr(),
    std::ptr::invalid(7),
); // FAILS, 0x56229d3aa00b != 0x7

https://play.rust-lang.org/?version=nightly&mode=release&edition=2021&gist=5544bc2edee6a4276a57cbeedbb12212

[LegionMammal978]

It seems like the whole point of op_to_const() is to make this transformation, which occurs a bit further down in the function:

                debug!("ScalarPair(a: {:?}, b: {:?})", a, b);
                // We know `offset` is relative to the allocation, so we can use `into_parts`.
                let (data, start) = match a.to_pointer(ecx).unwrap().into_parts() {
                    (Some(alloc_id), offset) => {
                        (ecx.tcx.global_alloc(alloc_id).unwrap_memory(), offset.bytes())
                    }
                    (None, _offset) => (
                        ecx.tcx.intern_const_alloc(Allocation::from_bytes_byte_aligned_immutable(
                            b"" as &[u8],
                        )),
                        0,
                    ),
                };
                let len = b.to_machine_usize(ecx).unwrap();
                let start = start.try_into().unwrap();
                let len: usize = len.try_into().unwrap();
                ConstValue::Slice { data, start, end: start + len }

That is, a &[u8] or &str without an AllocId (e.g., one from an invalid pointer) is converted to an arbitrary allocation storing b"" as &[u8]. Looking up the call graph, this operation appears to be invoked by eval_to_const_value_raw() and valtree_to_const_value().

So the reference isn't transformed every time it's used, only in certain contexts, where the slice has to exit const-eval-land and become a real instantiated value. For instance, using a slice as a const-generic parameter triggers the same behavior (Playground):

#![allow(incomplete_features)]
#![feature(adt_const_params)]
use std::slice::from_raw_parts;

struct AsPtr<const SLICE: &'static [u8]>;
impl<const SLICE: &'static [u8]> AsPtr<SLICE> {
    const PTR: *const u8 = SLICE.as_ptr();
}

fn main() {
    let ptr = AsPtr::<{ unsafe { from_raw_parts(1 as *const u8, 0) } }>::PTR;
    println!("{ptr:p}");
}

Notably, this only occurs when the slice is the entire constant value. The address isn't modified if the slice is a field of a struct, enum, or array.

[oli-obk]

Considering that we got valtrees now, this will become more common. All addresses in type level constants are now de- or duplicated.

For these ones that are not used in type level consts, we should just remove the ConstValue::Slice variant, as it is not needed anymore except potentially for perf, but we may be able to preserve that

[RalfJung]

op_to_const is a big hack and the sooner we can get rid of it the better...

For the specific case of u8 slices, IIRC that was needed because one can pattern-match on them. How close is the pattern matching logic to being ported away from ConstValue, to valtrees? Once that is the case, some cleanup should be possible here.