Find a file
bors 92f3d9a6b4 Auto merge of #23820 - sfackler:fast_read_to_end, r=alexcrichton
with_end_to_cap is enormously expensive now that it's initializing
memory since it involves 64k allocation + memset on every call. This is
most noticable when calling read_to_end on very small readers, where the
new version if **4 orders of magnitude** faster.

BufReader also depended on with_end_to_cap so I've rewritten it in its
original form.

As a bonus, converted the buffered IO struct Debug impls to use the
debug builders.

I first came across this in sfackler/rust-postgres#106 where a user reported a 10x performance regression. A call to read_to_end turned out to be the culprit: 9cd413d42c.

The new version differs from the old in a couple of ways. The buffer size used is now adaptive. It starts at 32 bytes and doubles each time EOF hasn't been reached up to a limit of 64k. In addition, the buffer is only truncated when EOF or an error has been reached, rather than after every call to read as was the case for the old implementation.

I wrote up a benchmark to compare the old version and new version: https://gist.github.com/sfackler/e979711b0ee2f2063462

It tests a couple of different cases: a high bandwidth reader, a low bandwidth reader, and a low bandwidth reader that won't return more than 10k per call to `read`. The high bandwidth reader should be analagous to use cases when reading from e.g. a `BufReader` or `Vec`, and the low bandwidth readers should be analogous to reading from something like a `TcpStream`.

Of special note, reads from a high bandwith reader containing 4 bytes are now *4,495 times faster*. 
```
~/foo ❯ cargo bench
   Compiling foo v0.0.1 (file:///home/sfackler/foo)
     Running target/release/foo-7498d7dd7faecf5c

running 13 tests
test test_new ... ignored
test new_delay_4      ... bench:    230768 ns/iter (+/- 14812)
test new_delay_4_cap  ... bench:    231421 ns/iter (+/- 7211)
test new_delay_5m     ... bench:  14495370 ns/iter (+/- 4008648)
test new_delay_5m_cap ... bench:  73127954 ns/iter (+/- 59908587)
test new_nodelay_4    ... bench:        83 ns/iter (+/- 2)
test new_nodelay_5m   ... bench:  12527237 ns/iter (+/- 335243)
test std_delay_4      ... bench:    373095 ns/iter (+/- 12613)
test std_delay_4_cap  ... bench:    374190 ns/iter (+/- 19611)
test std_delay_5m     ... bench:  17356012 ns/iter (+/- 15906588)
test std_delay_5m_cap ... bench: 883555035 ns/iter (+/- 205559857)
test std_nodelay_4    ... bench:    144937 ns/iter (+/- 2448)
test std_nodelay_5m   ... bench:  16095893 ns/iter (+/- 3315116)

test result: ok. 0 passed; 0 failed; 1 ignored; 12 measured
```

r? @alexcrichton
2015-03-29 19:47:18 +00:00
man environment variables -> environment 2015-03-20 10:53:51 -04:00
mk make it use gdb instead of lldb when mac-android cross compile 2015-03-25 18:12:35 +09:00
src Auto merge of #23820 - sfackler:fast_read_to_end, r=alexcrichton 2015-03-29 19:47:18 +00:00
.gitattributes
.gitignore gitignore: Add the autogenerated/downloaded unicode data files. 2014-08-03 17:32:53 +10:00
.gitmodules Use rust-installer for installation 2014-12-11 17:14:17 -08:00
.mailmap Update .mailmap 2014-10-23 23:01:31 +02:00
.travis.yml Allow travis to use newer-faster infrastructure for building. http://blog.travis-ci.com/2014-12-17-faster-builds-with-container-based-infrastructure/ 2015-01-01 02:00:29 -05:00
AUTHORS.txt Correct Phil Dawes email address 2015-03-28 22:26:00 +00:00
configure configure: Fix detection of 32-bit Linux userspace 2015-03-23 14:27:09 -07:00
CONTRIBUTING.md Fixes "Code of Conduct" Link 2015-02-18 20:53:00 -07:00
COPYRIGHT copyright: update paths and entries 2015-01-22 15:45:47 +01:00
LICENSE-APACHE
LICENSE-MIT Bump LICENSE copyright year 2015-01-17 10:51:07 -05:00
Makefile.in Rollup merge of #23510 - lgvz:makefile, r=Manishearth 2015-03-20 12:43:13 +05:30
README.md Strip trailing whitespace 2015-03-15 11:25:43 -07:00
RELEASES.md Fix frow_raw/from_raw typo 2015-02-21 13:14:17 +01:00

The Rust Programming Language

This is a compiler for Rust, including standard libraries, tools and documentation. Rust is a systems programming language that is fast, memory safe and multithreaded, but does not employ a garbage collector or otherwise impose significant runtime overhead.

Quick Start

Read "Installing Rust" from The Book.

Building from Source

  1. Make sure you have installed the dependencies:

    • g++ 4.7 or clang++ 3.x
    • python 2.6 or later (but not 3.x)
    • GNU make 3.81 or later
    • curl
    • git
  2. Clone the source with git:

    $ git clone https://github.com/rust-lang/rust.git
    $ cd rust
    
  1. Build and install:

    $ ./configure
    $ make && make install
    

    Note: You may need to use sudo make install if you do not normally have permission to modify the destination directory. The install locations can be adjusted by passing a --prefix argument to configure. Various other options are also supported pass --help for more information on them.

    When complete, make install will place several programs into /usr/local/bin: rustc, the Rust compiler, and rustdoc, the API-documentation tool. This install does not include Cargo, Rust's package manager, which you may also want to build.

Building on Windows

MSYS2 can be used to easily build Rust on Windows:

  1. Grab the latest MSYS2 installer and go through the installer.

  2. From the MSYS2 terminal, install the mingw64 toolchain and other required tools.

    # Choose one based on platform:
    $ pacman -S mingw-w64-i686-toolchain
    $ pacman -S mingw-w64-x86_64-toolchain
    
    $ pacman -S base-devel
    
  3. Run mingw32_shell.bat or mingw64_shell.bat from wherever you installed MYSY2 (i.e. C:\msys), depending on whether you want 32-bit or 64-bit Rust.

  4. Navigate to Rust's source code, configure and build it:

    $ ./configure
    $ make && make install
    

Notes

Since the Rust compiler is written in Rust, it must be built by a precompiled "snapshot" version of itself (made in an earlier state of development). As such, source builds require a connection to the Internet, to fetch snapshots, and an OS that can execute the available snapshot binaries.

Snapshot binaries are currently built and tested on several platforms:

  • Windows (7, 8, Server 2008 R2), x86 and x86-64 (64-bit support added in Rust 0.12.0)
  • Linux (2.6.18 or later, various distributions), x86 and x86-64
  • OSX 10.7 (Lion) or greater, x86 and x86-64

You may find that other platforms work, but these are our officially supported build environments that are most likely to work.

Rust currently needs about 1.5 GiB of RAM to build without swapping; if it hits swap, it will take a very long time to build.

There is more advice about hacking on Rust in CONTRIBUTING.md.

Getting Help

The Rust community congregates in a few places:

Contributing

To contribute to Rust, please see CONTRIBUTING.

Rust has an IRC culture and most real-time collaboration happens in a variety of channels on Mozilla's IRC network, irc.mozilla.org. The most popular channel is #rust, a venue for general discussion about Rust, and a good place to ask for help.

License

Rust is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0), with portions covered by various BSD-like licenses.

See LICENSE-APACHE, LICENSE-MIT, and COPYRIGHT for details.