Improve integration of the http server introduced by the http-registry feature.
Now the same HTTP server is used for serving downloads, the index, and
the API.
This makes it easier to write tests that deal with authentication and
http registries.
This is intended to help prevent the following scenario from happening:
1. Old cargo builds an index cache.
2. Old cargo finds an index entry it cannot parse, skipping it,
and saving the cache without the entry.
3. New cargo loads the cache with the missing entry, and never sees
the new entries that it understands.
This may result in more cache thrashing, but that seems better than
having new cargos missing entries.
