The go1 benchmark suite does a lot of work at package init time, which
makes it take quite a while to run even if you're not running any of
the benchmarks, or if you're only running a subset of them. This leads
to an awkward workaround in dist test to compile but not run the
package, unlike roughly all other packages. It also reduces isolation
between benchmarks by affecting the starting heap size of all
benchmarks.
Fix this by initializing all data required by a benchmark when that
benchmark runs, and keeping it local so it gets freed by the GC and
doesn't leak between benchmarks. Now, none of the benchmarks depend on
global state.
Re-initializing the data on each benchmark run does add overhead to an
actual benchmark run, as each benchmark function is called several
times with different values of b.N. A full run of all benchmarks at
the default -benchtime=1s now takes ~10% longer; higher -benchtimes
would be less. It would be quite difficult to cache this data between
invocations of the same benchmark function without leaking between
different benchmarks and affecting GC overheads, as the testing
package doesn't provide any mechanism for this.
This reduces the time to run the binary with no benchmarks from 1.5
seconds to 10 ms, and also reduces the memory required to do this from
342 MiB to 17 MiB.
To make sure data was not leaking between different benchmarks, I ran
the benchmarks with -shuffle=on. The variance remained low: mostly
under 3%. A few benchmarks had higher variance, but in all cases it
was similar to the variance between this change.
This CL naturally changes the measured performance of several of the
benchmarks because it dramatically changes the heap size and hence GC
overheads. However, going forward the benchmarks should be much better
isolated.
For #37486.
Change-Id: I252ebea703a9560706cc1990dc5ad22d1927c7a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/443336
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Update references missed in CL 263142.
For #41190
Change-Id: I778760a6a69bd0440fec0848bdef539c9ccb4ee1
GitHub-Last-Rev: dda42b09ff
GitHub-Pull-Request: golang/go#42874
Reviewed-on: https://go-review.googlesource.com/c/go/+/273946
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Cherry Zhang <cherryyz@google.com>
cmd/dist executes 'go test' within this directory, so it needs a
go.mod file to tell the compiler what package path to use in
diagnostic and debug information.
Updates #30228
Change-Id: Ia313ac06bc0ec4631d415faa20c56cce2ac8dbc5
Reviewed-on: https://go-review.googlesource.com/c/go/+/165498
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This is necessary to avoid disrupting the go1 suite and gives
us a place to put other tests of basic compiler function and
correctness.
Change-Id: I36933819ff2bfe6a2121fff2be9a98efd2123d9a
Reviewed-on: https://go-review.googlesource.com/22597
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Also rename the go parser test to GoParse so it doesn't grab the globally useful Parse name.
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/7732044
As discussed in
https://groups.google.com/d/msg/golang-dev/Na9XE6mcQyY/zbeBI7R-vnoJ
Here is a static copy of the go/parser benchmark. I ended up using
fancy encodings because the original parser.go had a number of `s
scattered throughout which made it hard to embed the source directly.
Curiously on my laptop this benchmark always scores roughly 10% higher
than the standalone benchmark. This may be down to the generation of
the fasta data set triggering the cpu governor to raise the cpu speed.
However the benchmark is consistent with itself across multiple runs.
R=golang-dev, minux.ma, rsc
CC=golang-dev
https://golang.org/cl/6305055
As discussed on golang-dev, reduce the size of the fasta
dataset to make it possible to run the go1 benchmarks on
small ARM systems.
Also, remove the 25m suffix from fasta data and Revcomp.
linux/arm: pandaboard OMAP4
BenchmarkBinaryTree17 1 70892426000 ns/op
BenchmarkFannkuch11 1 35712066000 ns/op
BenchmarkGobDecode 10 137146000 ns/op 5.60 MB/s
BenchmarkGobEncode 50 64953000 ns/op 11.82 MB/s
BenchmarkGzip 1 5675690000 ns/op 3.42 MB/s
BenchmarkGunzip 1 1207001000 ns/op 16.08 MB/s
BenchmarkJSONEncode 5 860424800 ns/op 2.26 MB/s
BenchmarkJSONDecode 1 3321839000 ns/op 0.58 MB/s
BenchmarkMandelbrot200 50 45893560 ns/op
BenchmarkRevcomp 10 135220300 ns/op 18.80 MB/s
BenchmarkTemplate 1 6385681000 ns/op 0.30 MB/s
R=rsc, minux.ma, dsymonds
CC=golang-dev
https://golang.org/cl/6278048
We can't depend on init() order, and certainly we don't want to
register all future benchmarks that use jsonbytes or jsondata to init()
in json_test.go, so we use a more general solution: make generation of
jsonbytes and jsondata their own function so that the compiler will take
care of the order.
R=golang-dev, dave, rsc
CC=golang-dev
https://golang.org/cl/6282046
Also delete gotest, since it's messy to fix and slated for deletion anyway.
A couple of things outside src can't be tested any more. "go test" will be
fixed and these tests will be re-enabled. They're noisy for now.
Fixes#284.
R=rsc
CC=golang-dev
https://golang.org/cl/5598049
I have included a few important microbenchmarks,
but the overall intent is to have mostly end-to-end
benchmarks timing real world operations.
The jsondata.go file is a summary of agl's
activity in various open source repositories.
It gets used as test data for many of the benchmarks.
Everything links into one binary (even the test data)
so that it is easy to run the benchmarks on many
computers: there is just one file to copy around.
R=golang-dev, r, bradfitz, adg, r
CC=golang-dev
https://golang.org/cl/5484071
