str::from_bytes

[jesse99]

The semantics of this should be tightened up (or probably even better new function(s) with better names should be added). Is the buffer supposed to be utf-8? 7-bit ASCII?

Also it should probably stop if it hits a NUL character. One place where this is annoying is inet_ntop which writes an ASCII representation of an address to a user provided buffer. With the way from_str works now you have to write silly code like:

    alt vec::position(buffer, {|c| c == 0u8})
    {
        option::some(i)
        {
            str::from_bytes(vec::slice(buffer, 0u, i))
        }
        option::none
        {
            str::from_bytes(buffer)
        }
    }

[brson]

The vector argument to from_bytes must be valid UTF-8. Needs better docs.

For converting from C strings usually str::unsafe::from_c_str is more appropriate. It takes an unsafe pointer and stops at nulls.

[jesse99]

I actually have a [u8] buffer so from_c_str won't work without a cast. unsafe::from_bytes did stop at null characters though the docs don't specify how it is supposed to behave.

[graydon]

The docs for from_bytes say that it fails when provided with invalid UTF-8.
The docs for from_c_str say that it consumes a null-terminated C string, and only takes a pointer.
unsafe::from_bytes does not stop at a null byte. Can you provide a testcase showing that occurring? It should not do so.

[jesse99]

Not sure why I thought str::unsafe::from_bytes stopped at nulls; it's working as you described. I'm don't know why str::from_bytes adds null bytes to the string though. It says that it converts "bytes to a UTF-8 string" and nulls are not valid utf-8.

Which gets back to my original point: why is the safe interface even providing methods for operating on byte arrays? It's not like you can safely do anything with an arbitrary byte sequence so why not be more explicit and more useful and have from_utf8 instead?

[brson]

from_bytes could be renamed to from_utf8 to be consistent with from_utf16

[bblum]

There is also the confusing from_bytes_with_null, which also does not stop at nulls, but requires that the last character be null. Nobody outside of std::str appears to use it.

@graydon, not sure which way you mean it should or shouldn't do, but a test case:

use std::str;

fn main() {
    let a = ~[65, 65, 65, 0, 65, 65];
    println(str::from_bytes(a));
}

Prints 5 "A"s.

I propose we (a) remove from_bytes_with_null, (b) rename from_bytes in its current incarnation to from_utf8_ignore_null, and (c) add a stops-at-null version called from_utf8.

[erickt]

cc'ing #7039, where I'm doing some cleanup of the bytes-to-str cleanup. @brson: I'll rename from_bytes to from_utf8.

[emberian]

Still relevant. @erickt still going to do the rename?

[erickt]

@cmr: yep, it's next up on my plate once #8296 lands.

[thestinger]

@jesse99: \0 is most definitely valid UTF-8, so it's only possible to represent a subset of UTF-8 in a C string

the string module doesn't need any special handling of \0 beyond conversion to and from C strings

[thestinger]

Replaced with #8985, there's nothing else to do beyond renaming.