Commit graph

49771 commits

Author SHA1 Message Date
Dan Scales 8854368cb0 cmd/compile: deal with blank nodes with typeparam type during stenciling
Deal correctly with a blank local variable with type param type. This is
a special case, because a blank local variable is not in the fn.Dcl
list. In this case, we must explicitly create a new blank node with the
correct substituted type, so we have correct types if the blank local
variable has an initializing assignment.

Fixes #48602

Change-Id: I903ea44b29934e180404e32800773b7309bf297b
Reviewed-on: https://go-review.googlesource.com/c/go/+/352117
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Dan Scales <danscales@google.com>
2021-09-25 01:24:46 +00:00
Jay Conrod d60ad1e068 testing: address feedback for dev.fuzz merge
Based on comments in CL 348469.

Note that with this change, F.Fuzz no longer calls
runtime.Goexit. This simplifies our logic and makes F.Fuzz more
predictable.

Change-Id: I6c3c65b0e8e8f261621cbe2f17375e8164ef60a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/351316
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-09-24 21:42:31 +00:00
Daniel Martí 1ce6fd03b8 cmd/gofmt: format files in parallel
gofmt is pretty heavily CPU-bound, since parsing and formatting 1MiB
of Go code takes much longer than reading that amount of bytes from
disk. However, parsing and manipulating a large Go source file is very
difficult to parallelize, so we continue to process each file in its
own goroutine.

A Go module may contain a large number of Go source files, so we need
to bound the amount of work in flight. However, because the
distribution of sizes for Go source files varies widely — from tiny
doc.go files containing a single package comment all the way up to
massive API wrappers generated by automated tools — the amount of
time, work, and memory overhead needed to process each file also
varies. To account for this variability, we limit the in-flight work
by bytes of input rather than by number of files. That allows us to
make progress on many small files while we wait for work on a handful
of large files to complete.

The gofmt tool has a well-defined output format on stdout, which was
previously deterministic. We keep it deterministic by printing the
results of each file in order, using a lazily-synchronized io.Writer
(loosly inspired by Haskell's IO monad). After a file has been
formatted in memory, we keep it in memory (again, limited by the
corresponding number of input bytes) until the output for all previous
files has been flushed. This adds a bit of latency compared to
emitting the output in nondeterministic order, but a little extra
latency seems worth the cost to preserve output stability.

This change is based on Daniel Martí's work in CL 284139, but using a
weighted semaphore and ephemeral goroutines instead of a worker pool
and batches. Benchmark results are similar, and I find the concurrency
in this approach a bit easier to reason about.

In the batching-based approach, the batch size allows us to "look
ahead" to find large files and start processing them early. To keep
the CPUs saturated and prevent stragglers, we would need to tune the
batch size to be about the same as the largest input files. If the
batch size is set too high, a large batch of small files could turn
into a straggler, but if the batch size is set too low, the largest
files in the data set won't be started early enough and we'll end up
with a large-file straggler.

One possible alternative would be to sort by file size instead of
batching: identify all of the files to be processed, sort from largest
to smallest, and then process the largest files first so that the
"tail" of processing covers the smallest files. However, that approach
would still fail to saturate available CPU when disk latency is high,
would require buffering an arbitrary amount of metadata in order to
sort by size, and (perhaps most importantly!) would not allow the
`gofmt` binary to preserve the same (deterministic) output order that
it has today.

In contrast, with a semaphore we can produce the same deterministic
output as ever using only one tuning parameter: the memory footprint,
expressed as a rough lower bound on the amount of RAM available per
thread. While we're below the memory limit, we can run arbitrarily
many disk operations arbitrarily far ahead, and process the results of
those operations whenever they become avaliable. Then it's up to the
kernel (not us) to schedule the disk operations for throughput and
latency, and it's up to the runtime (not us) to schedule the
goroutines so that they complete quickly.

In practice, even a modest assumption of a few megabytes per thread
seems to provide a nice speedup, and it should scale reasonably even
to machines with vastly different ratios of CPU to disk. (In practice,
I expect that most 'gofmt' invocations will work with files on at most
one physical disk, so the CPU:disk ratio should vary more-or-less
directly with the thread count, whereas the CPU:memory ratio is
more-or-less independent of thread count.)

name \ time/op         baseline.txt  284139.txt    simplified.txt
GofmtGorootCmd           11.9s ± 2%     2.7s ± 3%       2.8s ± 5%

name \ user-time/op    baseline.txt  284139.txt    simplified.txt
GofmtGorootCmd           13.5s ± 2%    14.4s ± 1%      14.7s ± 1%

name \ sys-time/op     baseline.txt  284139.txt    simplified.txt
GofmtGorootCmd           465ms ± 8%    229ms ±28%      232ms ±31%

name \ peak-RSS-bytes  baseline.txt  284139.txt    simplified.txt
GofmtGorootCmd          77.7MB ± 4%  162.2MB ±10%    192.9MB ±15%

For #43566

Change-Id: I4ba251eb4d188a3bd1901039086be57f0b341910
Reviewed-on: https://go-review.googlesource.com/c/go/+/317975
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2021-09-24 21:38:56 +00:00
Jay Conrod 7e4fb8b3ef cmd/go: make 'go mod why -m' work in inconsistent, pruned module
'go mod why -m' works by listing modules matching command line
arguments, then loading "all" packages and finding which of the listed
modules provide packages imported by the main module.

If go.mod is inconsistent (that is, a requirement has a lower version
than MVS would select when the module graph is loaded) and pruned
(that is, the module graph is only loaded when necessary), then
modload.ListModules may return modules with different versions than
would be selected in modload.LoadPackages.

'go mod why -m' was too strict about this, mapping module paths and
versions to packages. With this fix, it maps module paths without
versions to packages.

Fixes #48613

Change-Id: I836c46289bb647d6c46ec65e7589531da532d5e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/352115
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-09-24 21:38:42 +00:00
Cuong Manh Le cfd0868fc3 cmd/compile: fix delayTransform condition
The delayTransform only checks whether ir.CurFunc is generic function or
not. but when compiling a non-generic closure inside a generic function,
we also want to delay the transformation, which delayTransform fails to
detect, since when ir.CurFunc is the closure, not the top level function.

Instead, we must rely on irgen.topFuncIsGeneric field to decide whether
to delay the transformation, the same logic with what is being done for
not adding closure inside a generic function to g.target.Decls list.

Fixes #48609

Change-Id: I5bf5592027d112fe8b19c92eb906add424c46507
Reviewed-on: https://go-review.googlesource.com/c/go/+/351855
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
2021-09-24 19:40:03 +00:00
Cuong Manh Le c94543b85f cmd/compile: move all usage of delayTransform out of helpers.go
So next CL will make delayTransform to become irgen's method, because
the delay transform logic also depends on irgen.topFuncIsGeneric field.

For #48609

Change-Id: I660ed19856bd06c3b6f4279a9184db96175dea2d
Reviewed-on: https://go-review.googlesource.com/c/go/+/351854
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
2021-09-24 19:39:49 +00:00
Dan Scales f6b5ffb5e1 cmd/compile: fix crawler for unexported fields with instantiated types
In markType() in crawler.go, mark the type of a unexported field if it
is a fully-instantiated type, since we create and instantiate the
methods of any fully-instantiated type that we see during import. As
before, we still do not mark the type of an unexported field if that
type is not generic. Fixes #48454 and most recent issue described in
48337. The included test is similar to the case in 48454.

Fixes #48454
Fixes #48337

Change-Id: I77a2a62b9e2647876facfa6f004201e8f699c905
Reviewed-on: https://go-review.googlesource.com/c/go/+/351315
Trust: Dan Scales <danscales@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
2021-09-24 18:21:14 +00:00
Dan Scales 812c99f86a cmd/compile: fix case in dictPass where OMETHVALUE should become ODOTMETH
When I separate out the dictionary transformations to dictPass, I missed
duplicating a conditional that deals with OMETHVALUE nodes that are
actually called. We create the OMETHVALUE when transforming bounds
function reference (before we know whether that reference will be
called), and we need to call transformDot() again to convert the
OMETHVALUE to ODOTMETH if the reference is actually called (the usual
case). Without this change, we leave the OMETHVALUE in, and extra *-fm
are created and used unncessarily.

Also, fixed a few places where we were missing ir.MarkFunc(), which sets
the class of a function node properly.

Change-Id: I6b02613039b16b507b44525faa2cd7031afb6982
Reviewed-on: https://go-review.googlesource.com/c/go/+/352069
Trust: Dan Scales <danscales@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2021-09-24 18:11:24 +00:00
Jay Conrod b00222fcdd cmd/go: refactor {Allow,Disallow}WriteGoMod to ExplicitWriteGoMod
Subcommands may now set the global flag modload.ExplicitWriteGoMod
instead of calling {Allow,Disallow}WriteGoMod.

When ExplicitWriteGoMod is false (default), modload.LoadPackages and
ListModules will either update go.mod and go.sum or report an error if
they need to be updated, depending on cfg.BuildMod.

When ExplicitWriteGoMod is true, commands must explicitly call
modload.WriteGoMod to update go.mod and go.sum or report an
error. Commands that perform some operation after loading the build
list (like downloading zips or building packages) and commands that
load packages multiple times should set this. For now, only 'go get'
and 'go mod download' set this.

This CL is a pure refactor: no change in behavior is expected.
There are some other minor changes in here, too: commitRequirements no
longer sets the global requirements: that should be done separately first.

Change-Id: I69942a808bb177faf7904a53aaf2d4ac68500e82
Reviewed-on: https://go-review.googlesource.com/c/go/+/349600
Trust: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-09-24 17:53:04 +00:00
Jay Conrod 584afc2928 cmd/go: test that graph, verify, and why don't write go.mod or go.sum
They should also not report an error if these files need to be
updated. These commands are used for debugging, so it's important that
they still work when go.mod and go.sum are incomplete.

For #40775

Change-Id: I1b731599e5a4510f47827b9812525636a7402bf4
Reviewed-on: https://go-review.googlesource.com/c/go/+/351468
Trust: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-09-24 17:52:54 +00:00
Jay Conrod d5d3f80013 cmd/go: adjust comments on why fuzzing instrumentation is disabled
For #48504
Related #14565

Change-Id: Ibe43c75224525c4b80dbb66a1b6e0d688e47e2e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/351314
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-09-24 17:23:19 +00:00
Cherry Mui 217507eb03 runtime: set vdsoSP to caller's SP consistently
m.vdsoSP should be set to the SP of the caller of nanotime1,
instead of the SP of nanotime1 itself, which matches m.vdsoPC.
Otherwise the unmatched vdsoPC and vdsoSP would make the stack
trace look like recursive.

We already do it correctly on AMD64, 386, and RISCV64. This CL
fixes the rest.

Fixes #47324.

Change-Id: I98b6fcfbe9fc6bdd28b8fe2a1299b7c505371dd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/337590
Trust: Cherry Mui <cherryyz@google.com>
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2021-09-24 14:52:47 +00:00
Joel Sing fe8347b61a cmd/compile: optimise immediate operands with constants on riscv64
Instructions with immediates can be precomputed when operating on a
constant - do so for SLTI/SLTIU, SLLI/SRLI/SRAI, NEG/NEGW, ANDI, ORI
and ADDI. Additionally, optimise ANDI and ORI when the immediate is
all ones or all zeroes.

In particular, the RISCV64 logical left and right shift rules
(Lsh*x*/Rsh*Ux*) produce sequences that check if the shift amount
exceeds 64 and if so returns zero. When the shift amount is a
constant we can precompute and eliminate the filter entirely.

Likewise the arithmetic right shift rules produce sequences that
check if the shift amount exceeds 64 and if so, ensures that the
lower six bits of the shift are all ones. When the shift amount
is a constant we can precompute the shift value.

Arithmetic right shift sequences like:

   117fc:       00100513                li      a0,1
   11800:       04053593                sltiu   a1,a0,64
   11804:       fff58593                addi    a1,a1,-1
   11808:       0015e593                ori     a1,a1,1
   1180c:       40b45433                sra     s0,s0,a1

Are now a single srai instruction:

   117fc:       40145413                srai    s0,s0,0x1

Likewise for logical left shift (and logical right shift):

   1d560:       01100413                li      s0,17
   1d564:       04043413                sltiu   s0,s0,64
   1d568:       40800433                neg     s0,s0
   1d56c:       01131493                slli    s1,t1,0x11
   1d570:       0084f433                and     s0,s1,s0

Which are now a single slli (or srli) instruction:

   1d120:       01131413                slli    s0,t1,0x11

This removes more than 30,000 instructions from the Go binary and
should improve performance in a variety of areas - of note
runtime.makemap_small drops from 48 to 36 instructions. Similar
gains exist in at least other parts of runtime and math/bits.

Change-Id: I33f6f3d1fd36d9ff1bda706997162bfe4bb859b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/350689
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Michael Munday <mike.munday@lowrisc.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-09-24 10:51:48 +00:00
Joel Sing d413908320 test/codegen: add shift tests for RISCV64
Add tests for shift by constant, masked shifts and bounded shifts. While here,
sort tests by architecture and keep order of tests consistent (lsh, rshU, rsh).

Change-Id: I512d64196f34df9cb2884e8c0f6adcf9dd88b0fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/351289
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Michael Munday <mike.munday@lowrisc.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
2021-09-24 07:22:13 +00:00
Robert Griesemer 242d02dd5e cmd/compile/internal/types2: assume generic code for std lib
Change-Id: Ib24890af8caa02af61358cadac6637574d62ff52
Reviewed-on: https://go-review.googlesource.com/c/go/+/351792
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
2021-09-24 04:59:50 +00:00
Cuong Manh Le 9ce601df6a cmd/go: move gc concurrency level computation near gcflags
So after constructing "args" variable, "gcflags" is not used anywhere.
It makes the code easier to maintain, and prevent subtle bug like #48490.

Change-Id: I41653536480880a8a6f9fbf6cfa8a461b6fb3208
Reviewed-on: https://go-review.googlesource.com/c/go/+/351849
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
2021-09-24 04:32:28 +00:00
Robert Griesemer aa878ee49b cmd/compile/internal/syntax: assume generic code for std lib
Also: improve some error message prints in testSyntaxErrors.
Change-Id: Iaa1d642398fa82975fefb4bde54f476dd5229eb5
Reviewed-on: https://go-review.googlesource.com/c/go/+/351791
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-09-24 04:20:30 +00:00
Meng Zhuo 483533df9e runtime: using wyrand for fastrand
For modern 64-bit CPU architecture multiplier is faster than xorshift

darwin/amd64
name              old time/op  new time/op  delta
Fastrand          2.13ns ± 1%  1.78ns ± 1%  -16.47%  (p=0.000 n=9+10)
FastrandHashiter  32.5ns ± 4%  32.1ns ± 3%     ~     (p=0.277 n=8+9)
Fastrandn/2       2.16ns ± 1%  1.99ns ± 1%   -7.53%  (p=0.000 n=10+10)
Fastrandn/3       2.13ns ± 3%  2.00ns ± 1%   -5.88%  (p=0.000 n=10+10)
Fastrandn/4       2.08ns ± 2%  1.98ns ± 2%   -4.71%  (p=0.000 n=10+10)
Fastrandn/5       2.08ns ± 2%  1.98ns ± 1%   -4.90%  (p=0.000 n=10+9)

linux/mips64le
name              old time/op  new time/op  delta
Fastrand          12.1ns ± 0%  10.8ns ± 1%  -10.58%  (p=0.000 n=8+10)
FastrandHashiter   105ns ± 1%   105ns ± 1%     ~     (p=0.138 n=10+10)
Fastrandn/2       16.9ns ± 0%  16.4ns ± 4%   -2.84%  (p=0.020 n=10+10)
Fastrandn/3       16.9ns ± 0%  16.4ns ± 3%   -3.03%  (p=0.000 n=10+10)
Fastrandn/4       16.9ns ± 0%  16.5ns ± 2%   -2.01%  (p=0.002 n=8+10)
Fastrandn/5       16.9ns ± 0%  16.4ns ± 3%   -2.70%  (p=0.000 n=8+10)

linux/riscv64
name              old time/op  new time/op  delta
Fastrand          22.7ns ± 0%  12.7ns ±19%  -44.09%  (p=0.000 n=9+10)
FastrandHashiter   255ns ± 4%   250ns ± 7%     ~     (p=0.363 n=10+10)
Fastrandn/2       31.8ns ± 2%  28.5ns ±13%  -10.45%  (p=0.000 n=10+10)
Fastrandn/3       33.0ns ± 2%  27.4ns ± 8%  -17.16%  (p=0.000 n=9+10)
Fastrandn/4       29.6ns ± 3%  28.2ns ± 5%   -4.81%  (p=0.000 n=8+9)
Fastrandn/5       33.4ns ± 3%  26.5ns ± 9%  -20.49%  (p=0.000 n=8+10)

Change-Id: I88ac69625ef923f8be66647e3361e3be986de002
Reviewed-on: https://go-review.googlesource.com/c/go/+/337350
Trust: Meng Zhuo <mzh@golangcn.org>
Run-TryBot: Meng Zhuo <mzh@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2021-09-24 01:25:24 +00:00
Robert Griesemer 0f1159bf54 go/types: delay union element checks
This is a clean port of CL 351969 from types2 to go/types
with a minor adjustment for error handling (provide an error
code).

For #46461.

Change-Id: I493dde12d8ccf86aa33f4913ac6e82f2eb459088
Reviewed-on: https://go-review.googlesource.com/c/go/+/351971
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
2021-09-24 01:18:01 +00:00
Ian Lance Taylor 4dd5f0994f constraints: new package
The constraint packages defined a set of useful constraints to be used
with type parameters.

Fixes #45458

Change-Id: Id4f4e6c55debb90e6b10ea0dbe2319be1e888746
Reviewed-on: https://go-review.googlesource.com/c/go/+/349709
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-09-24 00:39:31 +00:00
Robert Griesemer c90ead97ad cmd/compile/internal/types2: delay union element checks
We cannot determine the underlying type right when parsing
a union term since it may lead to types that are not yet
fully set up.

Fixes #46461.

Change-Id: I1fcadb1dcef2160be2f088a4a34e99dbab01da67
Reviewed-on: https://go-review.googlesource.com/c/go/+/351969
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
2021-09-24 00:30:36 +00:00
Daniel Martí 4e308d73ba cmd/go: refer to the right package in a test
The test checks that two packages aren't non-test dependencies.
There's a copy-paste typo, however.
When net/http is unexpectedly found as a dependendency,
we instead mention the other package in the error message.

Change-Id: I3232c6252255c839e08efa048f2232c192d0fb85
Reviewed-on: https://go-review.googlesource.com/c/go/+/351372
Trust: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-09-23 21:59:30 +00:00
Bryan C. Mills 93f964b412 cmd/compile/internal/ssa: remove workarounds for #43938
The cmd/go bug this worked around should be fixed as of CL 351329.

Fixes #43938
Fixes #48550

Change-Id: Ida930e7ee33d44d89556b9b8bbc3c26bb53697b2
Reviewed-on: https://go-review.googlesource.com/c/go/+/351529
Trust: Bryan C. Mills <bcmills@google.com>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-09-23 20:15:19 +00:00
Robert Griesemer 02913aa51c test/fixedbugs: adjust test case (fix longtest builders)
For #33232.

Change-Id: Id95a92bfdad91e3ccde9f5654c3b1b02ca95f6ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/351731
Trust: Robert Griesemer <gri@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
2021-09-23 19:41:46 +00:00
Robert Griesemer ddb5a42b25 cmd/compile/internal/types2: avoid "declared but not used" errors for invalid code
Agressively mark all LHS variables in assignments as used if there
is any error in the (entire) assignment. This reduces the number of
spurious "declared but not used" errors in programs that are invalid
in the first place. This behavior is closer to the behavior of the
compiler's original type checker (types1) and lets us remove lines
of the form "_ = variable" just to satisfy test cases. It also makes
more important errors visible by not crowding them out.

Remove the Checker.useLHS function and use Checker.use instead:
useLHS didn't evaluate top-level variables, but we actually want
them to be evaluated in an error scenario so that they are getting
used (and thus we don't get the "declared but not used" error).

Fixes #42937.

Change-Id: Idda460f6b81c66735bf9fd597c54188949bf12b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/351730
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
2021-09-23 19:41:45 +00:00
Robert Griesemer c0766d2cd0 go/types: avoid "declared but not used errors" for invalid variable initializations
This is a partial port of CL 351669 from types2 to go/types; it
only copies the fix for variable usage.

Eventually we may want to use the compiler error messages for assignment
errors everywhere, but that doesn't need to happen now.

Change-Id: I62b024b1b29cc27c29d919de3de44f944f6e2b4d
Reviewed-on: https://go-review.googlesource.com/c/go/+/351670
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
2021-09-23 19:41:44 +00:00
Robert Griesemer 0626ac064d cmd/compile: restore original assignment error messages
When used with the compiler, types2 will report assignment error
messages that closely match what the compiler type checker (types1)
produces.

Also, mark lhs variables as used in invalid variable initializations
to avoid a class of follow-on errors.

Fixes #48558.

Change-Id: I92d1de006c66b3a2364bb1bea773a312963afe75
Reviewed-on: https://go-review.googlesource.com/c/go/+/351669
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
2021-09-23 19:41:41 +00:00
Cuong Manh Le 2fc7697da4 Revert "cmd/go: insert goroot to the hash of build cache when the packages include C files"
This reverts commit abbfec2829.

Reason to revert: breaks darwin builders.

Updates #48319

Change-Id: I50c957a6a3f46ffcdaf972bdbb0574867ddc9486
Reviewed-on: https://go-review.googlesource.com/c/go/+/351851
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-09-23 17:47:39 +00:00
Robert Findley 13f3c57cef go/types,types2: disallow illegal cycles through Unions
Checker.validType was not considering Unions when looking for illegal
cycles.

Fixes #48582

Change-Id: I11ad0279eeaaa56bb6d5731b0572c1c3a0c459eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/351829
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-09-23 17:04:30 +00:00
Joe Tsai 5961134fa5 crypto: avoid escaping Hash.Sum on generic architectures
For architectures without a specialized implementation (e.g. arm),
the generic implementation allocates because it does:

	var block = blockGeneric

which causes the compiler to give up trying to analyze block
even though it is technically only ever one implementation.
Instead of a variable, declare a function that wraps blockGeneric.

We apply this fix to md5, sha1, and sha256,
while sha512 already had the equivalent change.
We add a test to all hashing packages to ensure no allocations.

Credit goes to Cuong Manh Le for more specifically identifying
the problem and Keith Randal for suggesting a concrete solution.

Fixes #48055

Change-Id: I1a6a2e028038e051c83fd72b10a8bf4d210df57d
Reviewed-on: https://go-review.googlesource.com/c/go/+/346209
Trust: Joe Tsai <joetsai@digital-static.net>
Run-TryBot: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-09-23 16:54:46 +00:00
Daniel Martí 335e72bcb6 cmd/go: fix indentation of -workfile help text
While reading the help text for #48576,
I noticed that the output was misaligned.

Turns out it's because two lines have space indentation,
while the rest use tab indentation. Fix that.

Change-Id: Ie7c473d892ca13852fa2134f3cdef21e9210e02e
Reviewed-on: https://go-review.googlesource.com/c/go/+/351750
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-09-23 16:08:00 +00:00
Lynn Boger 24c2ee7b65 cmd/compile: enable reg args and add duffcopy support on ppc64x
This adds support for duffcopy on ppc64x and updates the
ssa/config.go file to enable register args and recognize
the duffDevice is available on ppc64x.

Change-Id: Ifc472cc9cc19c9a80e468fb52078c75f7dd44d36
Reviewed-on: https://go-review.googlesource.com/c/go/+/351490
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-09-23 15:51:39 +00:00
zhouguangyuan abbfec2829 cmd/go: insert goroot to the hash of build cache when the packages include C files
There are some absolute paths in the object file of the packages include C files. The path in C objects file can't be rewritten by linker. The goroot must be used as input for the hash when the packages include C files. So that the debug_info of the binary is correctly.

Fixes #48319

Change-Id: I659a3d6d71c4e49fff83f5bcf53a0a417e552a93
Reviewed-on: https://go-review.googlesource.com/c/go/+/348991
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Meng Zhuo <mzh@golangcn.org>
Trust: Jay Conrod <jayconrod@google.com>
2021-09-23 15:10:56 +00:00
Russ Cox d0dd26a88c html/template, text/template: implement break and continue for range loops
Break and continue for range loops was accepted as a proposal in June 2017.
It was implemented in CL 66410 (Oct 2017)
but then rolled back in CL 92155 (Feb 2018)
because html/template changes had not been implemented.

This CL reimplements break and continue in text/template
and then adds support for them in html/template as well.

Fixes #20531.

Change-Id: I05330482a976f1c078b4b49c2287bd9031bb7616
Reviewed-on: https://go-review.googlesource.com/c/go/+/321491
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2021-09-23 02:52:10 +00:00
Matthew Dempsky 93453233bd cmd/compile: fix unsafe.Add with small-size offsets operands
Like other builtin functions, unsafe.Add's len operand is allowed to
be variable sized. However, unlike other builtins, it doesn't get
lowered to a runtime function call, so we never end up coercing it to
a specific type. As a result, we could end up constructing an OpAddPtr
value but with a less-than-ptr-sized addend operand.

This CL fixes this by always coercing the second operand to uintptr
during SSA construction.

Theoretically, we could do this during walk instead, but the frontend
doesn't allow converting negative constants to uintptr.

Fixes #48536.

Change-Id: Ib0619ea79df58b256b250fec967a6d3c8afea631
Reviewed-on: https://go-review.googlesource.com/c/go/+/351592
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
2021-09-23 00:14:22 +00:00
Dan Scales ee69b09424 cmd/compile: break out constants for local and global dictionary prefixes
Create constant LocalDictName for the pname/refix for dictionary
parameters or local variables, and constant GlobalDictPrefix for the
prefix for names of global dictionaries. I wanted to make sure these
constants were set up as we add more reference to dictionaries for
debugging, etc.

Change-Id: Ide801f842383300a2699c96943ec06decaecc358
Reviewed-on: https://go-review.googlesource.com/c/go/+/351450
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-09-22 23:17:55 +00:00
Zvonimir Pavlinovic 1319b1476e cmd/go/internal/test: pass only analysis flags to vet
In go test vet=x, x should be off, all, or one of the analyses supported
by vet. All other flags should not be passed to vet. This CL maintains a
list of supported vet analyzers by running go tool vet -flags and
parsing the flag info to figure out the names of the supported analyzers
and their aliases.

Fixes #47309

Change-Id: I16ade8024301ad4aee5ad45aa92cf63b63dbc2d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/341334
Trust: Zvonimir Pavlinovic <zpavlinovic@google.com>
Run-TryBot: Zvonimir Pavlinovic <zpavlinovic@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-09-22 22:55:16 +00:00
Lynn Boger 333f3de2a1 cmd/compile: update PPC64 CALL* ops for register ABI
Allow the CALL ops to take variable number of arguments.

Change-Id: I6753c2fcb434ce2ebbce51dd595fbcbb39a60bed
Reviewed-on: https://go-review.googlesource.com/c/go/+/351589
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
2021-09-22 22:50:41 +00:00
jiahua wang 1537f14db5 io: fix spelling in documentation
Change-Id: Ie23a9f1300a803d9c713e82b0d892dd90333ca7b
Reviewed-on: https://go-review.googlesource.com/c/go/+/351371
Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-09-22 21:47:18 +00:00
Robert Griesemer 41bb7446dc cmd/compile/internal/types2: don't panic if targs don't match tparams when substituting
This is a clean port of CL 351337 from go/types to types2.

Change-Id: I974bf79fcc1ec0016c38e4c0b361d05f7b44e649
Reviewed-on: https://go-review.googlesource.com/c/go/+/351466
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-09-22 20:53:48 +00:00
Cherry Mui 5b75a9b2c3 cmd/compile: update comments with ABI aliases/wrappers
ABI aliases are gone.

Change-Id: I0f5676d8730cac14b7495dd6c0c1e08ca85a1c77
Reviewed-on: https://go-review.googlesource.com/c/go/+/351531
Trust: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-09-22 19:37:42 +00:00
Robert Griesemer 1e57748892 cmd/compile, go/types: allow any anywhere (as a type)
Adjust types2 and go/types and some test cases.

Because `any` is not treated specially anymore in constraint
position we get additional errors in constraints if `any` is
used before Go1.18 (in addition to the error that type parameter
lists are not permitted before Go1.18).

Fixes #33232.

Change-Id: I85590c6094b07c3e494fef319e3a38d0217cf6f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/351456
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
2021-09-22 19:19:49 +00:00
Roland Shoemaker 7a03ca65b3 internal/fuzz,cmd/compile: don't add race instrumentation to counters
Don't add race detector instrumentation to the fuzzing counters,
allowing usage of -race without immediately triggering the
detector. Also fixes a minor race in contextReader.Read.

Fixes #48307

Change-Id: Idb2cfeaa4283f8a74473b4bac6cd68eed577e943
Reviewed-on: https://go-review.googlesource.com/c/go/+/351453
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-09-22 18:44:36 +00:00
Cherry Mui ccfc41eee0 cmd/compile: check out-of-range shifts on ARM and ARM64
When encoding ARM or ARM64 shifted register operand, check that
the shift is in range.

Change-Id: If0014933bfd0a1b8eaaa01e0220a6eeb17ab9f40
Reviewed-on: https://go-review.googlesource.com/c/go/+/351530
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2021-09-22 18:32:21 +00:00
Elias Naur 635e49388b cmd/cgo: add go:notinheap annotation to Windows handle types
Fixes #42018

Change-Id: I6a40f3effe860e67a45fca2e8ab86f3e9887ffee
Reviewed-on: https://go-review.googlesource.com/c/go/+/350070
Trust: Elias Naur <mail@eliasnaur.com>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
2021-09-22 18:10:24 +00:00
Archana R e128749be8 hash/crc32: improve performance of ppc64SlicingUpdateBy8 on ppc64le
Reduce the number of instructions in the short loop of
ppc64SlicingUpdateBy8 function by combining MOVWZ and SRD into a SRD
with appropriate parameters performing the same operation and remove
MOVWZ R7,R7 from the loop
This change produces the following improvements on POWER9. None of the
other tests regress. Improvments on other POWERPC platforms similar.

name                                     old time/op    new time/op
delta

CRC32/poly=IEEE/size=15/align=0            80.5ns ± 0%    70.6ns ± 0%
-12%
CRC32/poly=IEEE/size=15/align=1            80.5ns ± 0%    70.6ns ± 0%
-12%
CRC32/poly=IEEE/size=512/align=1            151ns ± 0%     139ns ± 0%
-7%
CRC32/poly=IEEE/size=1kB/align=1            167ns ± 0%     155ns ± 0%
-7%
CRC32/poly=Castagnoli/size=15/align=0      80.2ns ± 0%    70.5ns ± 0%
-12%
CRC32/poly=Castagnoli/size=15/align=1      80.2ns ± 0%    70.5ns ± 0%
-12%
CRC32/poly=Castagnoli/size=512/align=1      150ns ± 0%     139ns ± 0%
-7%
CRC32/poly=Castagnoli/size=1kB/align=1      166ns ± 0%     155ns ± 0%
-6%

Change-Id: I424709041c30d1c637b595d0845e3ae78dc3e0a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/350989
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-09-22 16:53:49 +00:00
Cherry Mui 74ba70be68 cmd/link: use keyed literal for AbiSymKindToSymKind
The correspondence is clearer. Also avoid accidental mismatch.

Change-Id: Ibd9a31cfcd7c2301614f3496cac67cbae99972a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/351334
Trust: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-09-22 16:24:17 +00:00
Cherry Mui 55d22f7490 cmd/link: remove ABI aliases
It is now gone.

Change-Id: I59f68b324af706476695de2f291dd3aa5734e192
Reviewed-on: https://go-review.googlesource.com/c/go/+/351332
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-09-22 15:49:52 +00:00
Cherry Mui a630783ba0 cmd/internal/obj: remove ABI aliases from object file
Change-Id: I8a51f054e017e0116dee4e435b60c08d72e998e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/351331
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-09-22 15:49:42 +00:00
Bryan C. Mills 5ee32ff252 cmd/go: proceed with GOPATH unset if the command doesn't use it
For #43938

Change-Id: I0937b9bb6de3d29d7242ee61f053d4803277dc0f
Reviewed-on: https://go-review.googlesource.com/c/go/+/351329
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-09-22 15:46:33 +00:00