Generalize get_nullable_type to accept types that have an all-ones bit
pattern as their sentry "null" value.
This will allow [`OwnedFd`], [`BorrowedFd`], [`OwnedSocket`], and
[`BorrowedSocket`] to be marked with
`#[rustc_nonnull_optimization_guaranteed]`, which will allow
`Option<OwnedFd>`, `Option<BorrowedFd>`, `Option<OwnedSocket>`, and
`Option<BorrowedSocket>` to be used in FFI declarations, as described
in the [I/O safety RFC].
For example, it will allow a function like `open` on Unix and `WSASocketW`
on Windows to be declared using `Option<OwnedFd>` and `Option<OwnedSocket>`
return types, respectively.
The actual change to add `#[rustc_nonnull_optimization_guaranteed]`
to the abovementioned types will be a separate PR, as it'll depend on
having this patch in the stage0 compiler.
Also, update the diagnostics to mention that "niche optimizations" are
used in libstd as well as libcore, as `rustc_layout_scalar_valid_range_start`
and `rustc_layout_scalar_valid_range_end` are already in use in libstd.
[`OwnedFd`]: c9dc44be24/library/std/src/os/fd/owned.rs (L49)
[`BorrowedFd`]: c9dc44be24/library/std/src/os/fd/owned.rs (L29)
[`OwnedSocket`]: c9dc44be24/library/std/src/os/windows/io/socket.rs (L51)
[`BorrowedSocket`]: c9dc44be24/library/std/src/os/windows/io/socket.rs (L29)
[I/O safety RFC]: https://github.com/rust-lang/rfcs/blob/master/text/3128-io-safety.md#ownedfd-and-borrowedfdfd-1
This is step 2 towards fixing #77548.
In the codegen and codegen-units test suites, the `//` comment markers
were kept in order not to affect any source locations. This is because
these tests cannot be automatically `--bless`ed.
This allows types like Option<NonZeroU8> to be used in FFI without triggering the improper_ctypes lint. This works by changing the is_repr_nullable_ptr function to consider an enum E to be FFI-safe if:
- E has no explicit #[repr(...)].
- It only has two variants.
- One of those variants is empty (meaning it has no fields).
- The other variant has only one field.
- That field is one of the following:
- &T
- &mut T
- extern "C" fn
- core::num::NonZero*
- core::ptr::NonNull<T>
- #[repr(transparent)] struct wrapper around one of the types in this list.
- The size of E and its field are both known and are both the same size (implying E is participating in the nonnull optimization).