Due to libgit2 not supporting HTTP proxies, the custom transport API of the
library must be used to reimplement the HTTP transport with proxy support. The
git2-curl crate implements this transport on top of the curl-rust crate. By
using libcurl we gain all sorts of proxy support for free.
This transport is not used by default, however, as it is not well battle tested
and the architecture is not currently ideal (download the entire repo into
memory on a clone). Only when an HTTP proxy is present is the new transport
used.
The other drawback of git2-curl is that it does not currently support
authentication. If a private git repository is cloned or authentication is
required then it will generate an error instead of correctly asking for
credentials.
Closes#636
Whenever a build script or a plugin depends on a native dynamic library, the
dylib search path will be required to include the path to the generated dynamic
library. Currently Cargo does not augment the search path, causing load errors
or execution errors whenever builds are attempted in this case.
This commit remedies the situation by ensuring that dynamic libraries show up in
dylib search paths. This commit is necessary to fully migrate the remaining
tests from the old build command infrastructure to build scripts.
This commit includes a laundry list of updates and tweaks to reflect the current
API of the registry:
* `registry.host` has been renamed to `registry.index`
* New top-level manifest keys are now accepted:
* `homepage` - url
* `documentation` - url
* `repository` - url
* `description` - a markdown-less blurb
* `license` - string (verified by the registry on upload)
* `keywords` - string array
* `readme` - string pointing at a file
* Authors are now uploaded to the registry
* The upload format to the registry has changed to a body json payload
* Unpacking tarballs respects the executable bit for scripts and such.
* Downloading now follows redirects to go to S3.
* The download URL for a package has changed slightly.
* Verify path dependencies have a version listed when being uploaded
* Rename `upload` to `publish`
* Rename `ops::cargo_upload` to `ops::registry`
* Add a new `registry` package for interoperating with the registry
* Add the ability to modify owners via `cargo owner`
* Add a `readme` key to the manifest, and upload its contents to the registry.
* Add the ability to yank crates and their versions
* When packaging a library, verify that it builds from the packaged source by
unpacking the tarball and simulate running `cargo build` inside of it.
This commit extends the support in cargo's resolver to start resolving packages
with multiple versions as well as package requirements. This is a crucial step
forward when impelmenting the cargo registry as multiple versions will be
uploaded to the registry quite quickly!
This implements a fairly naive solution which should at least help cargo get out
the gates initially. This impelments a depth-first-search of the pacakage
dependency graph with a few sorting heuristics along the way to help out
resolution as it goes along.
Resolution errors will likely improve over time, but this commit does make an
effort to try to get some good error messages right off the bat.
It's looking more likely like plugins will not make it into the stable channel
of Rust, so this commits removes Cargo's personal dependence on the two
plugin-based pieces of functionality it was using:
1. Uses of the `regex!` macro now go through `Regex::new`.
2. Uses of the `docopt!` macro now go through `deriving(Decodable)` instead.
# cargo upload
The cargo-upload command will take the local package and upload it to the
specified registry. The local package is uploaded as a tarball compressed with
gzip under maximum compression. Most of this is done by just delegating to
`cargo package` The host to upload to is specified, in order of priority, by a
command line `--host` flag, the `registry.host` config key, and then the default
registry. The default registry is still `example.com`
The registry itself is still a work in progress, but the general plumbing for a
command such as this would look like:
1. Ensure the local package has been compressed into an archive.
2. Fetch the relevant registry and login token from config files.
3. Ensure all dependencies for a package are listed as coming from the same
registry.
4. Upload the archive to the registry with the login token.
5. The registry will verify the package is under 2MB (configurable).
6. The registry will upload the archive to S3, calculating a checksum in the
process.
7. The registry will add an entry to the registry's index (a git repository).
The entry will include the name of the package, the version uploaded, the
checksum of the upload, and then the list of dependencies (name/version req)
8. The local `cargo upload` command will succeed.
# cargo login
Uploading requires a token from the api server, and this token follows the same
config chain for the host except that there is no fallback. To implement login,
the `cargo login` command is used. With 0 arguments, the command will request
that a site be visited for a login token, and with an argument it will set the
argument as the new login token.
The `util::config` module was modified to allow writing configuration as well as
reading it. The support is a little lacking in that comments are blown away, but
the support is there at least.
# RegistrySource
An implementation of `RegistrySource` has been created (deleting the old
`DummyRegistrySource`). This implementation only needs a URL to be constructed,
and it is assumed that the URL is running an instance of the cargo registry.
## RegistrySource::update
Currently this will unconditionally update the registry's index (a git
repository). Tuning is necessary to prevent updating the index each time (more
coming soon).
## RegistrySource::query
This is called in the resolve phase of cargo. This function is given a
dependency to query for, and the source will simply look into the index to see
if any package with the name is present. If found, the package's index file will
be loaded and parsed into a list of summaries.
The main optimization of this function is to not require the entire registry to
ever be resident in memory. Instead, only necessary packages are loaded into
memory and parsed.
## RegistrySource::download
This is also called during the resolve phase of cargo, but only when a package
has been selected to be built (actually resolved). This phase of the source will
actually download and unpack the tarball for the package.
Currently a configuration file is located in the root of a registry's index
describing the root url to download packages from.
This function is optimized for two different metrics:
1. If a tarball is downloaded, it is not downloaded again. It is assumed that
once a tarball is successfully downloaded it will never change.
2. If the unpacking destination has a `.cargo-ok` file, it is assumed that the
unpacking has already occurred and does not need to happen again.
With these in place, a rebuild should take almost no time at all.
## RegistrySource::get
This function is simply implemented in terms of a PathSource's `get` function by
creating a `PathSource` for all unpacked tarballs as part of the `download`
stage.
## Filesystem layout
There are a few new directories as part of the `.cargo` home folder:
* `.cargo/registry/index/$hostname-$hash` - This is the directory containing the
actual index of the registry. `$hostname` comes from its url, and `$hash` is
the hash of the entire url.
* `.cargo/registry/cache/$hostname-$hash/$pkg-$vers.tar.gz` - This is a
directory used to cache the downloads of packages from the registry.
* `.cargo/registry/src/$hostname-$hash/$pkg-$vers` - This is the location of the
unpacked packages. They will be compiled from this location.
# New Dependencies
Cargo has picked up a new dependency on the `curl-rust` package in order to send
HTTP requests to the registry as well as send HTTP requests to download
tarballs.
This command will assemble the current package into a tarball ready for
uploading to the cargo registry. Currently no further verification is done
beyond packaging the local repository into a tarball, but in the future this
could execute other operations such as api stability tools.
This commit removes all distributed executables except for `cargo`. All
builtin subcommands are implemented through library calls, and the fallback
methods are retained to maintain extensability through new subcommands.
Closes#393
In general relying on external programs is dicey and tricky as they're very
different across systems in both how they're used as well as what versions
you'll find. Instead of binding to the least common denominator of CLI, we can
code against an exact version of libgit2.
This introduces a build-time dependency on cmake which libgit2 requires to build
itself, which is unfortunate, but thankfully it's only a build time dep. The
build process for libgit2 also automatically detects as many system libraries as
possible to use (if available), falling back to bundled versions if not
available. I have currently not figured how to control this, so the link-config
package is used to build libgit2 which requires that pkg-config be installed to
build cargo as well.
Closes#138
The hammer library currently has some shortcomings such as the inability to
document individual options. Additionally our handling with hammer of extra
arguments is dodgy at best currently.
This commit moves the repository to BurntSushi's docopt.rs library which seems
to more feature-complete at this time. Additionally, docopt has the great
benefit of a "document once, use everywhere" documentation strategy.
This migration solves two primary issues:
* Comprehensive and useful CLI documentation
* Gracefully handling flavorful combinations of arguments in odd combinations
Closes#218
