Resolver test/debug cleanup
This is several small things salvaged from abandoned PRs and implemented on top of #7011
In working on this I noted that the prop tests are very sensitive to whether backtrace are turned on. Maybe we should set that env to 0 for that builder?
These tests take a good amount of time to run locally and they're also
causing a lot of dependencies to get pulled into rust-lang/rust, so
let's have a separate crate that we just test on our own CI
