Commit graph

59599 commits

Author SHA1 Message Date
Russ Cox 966609ad9e time: avoid stale receives after Timer/Ticker Stop/Reset return
A proposal discussion in mid-2020 on #37196 decided to change
time.Timer and time.Ticker so that their Stop and Reset methods
guarantee that no old value (corresponding to the previous configuration
of the Timer or Ticker) will be received after the method returns.

The trivial way to do this is to make the Timer/Ticker channels
unbuffered, create a goroutine per Timer/Ticker feeding the channel,
and then coordinate with that goroutine during Stop/Reset.
Since Stop/Reset coordinate with the goroutine and the channel
is unbuffered, there is no possibility of a stale value being sent
after Stop/Reset returns.

Of course, we do not want an extra goroutine per Timer/Ticker,
but that's still a good semantic model: behave like the channels
are unbuffered and fed by a coordinating goroutine.

The actual implementation is more effort but behaves like the model.
Specifically, the timer channel has a 1-element buffer like it always has,
but len(t.C) and cap(t.C) are special-cased to return 0 anyway, so user
code cannot see what's in the buffer except with a receive.
Stop/Reset lock out any stale sends and then clear any pending send
from the buffer.

Some programs will change behavior. For example:

	package main

	import "time"

	func main() {
		t := time.NewTimer(2 * time.Second)
		time.Sleep(3 * time.Second)
		if t.Reset(2*time.Second) != false {
			panic("expected timer to have fired")
		}
		<-t.C
		<-t.C
	}

This program (from #11513) sleeps 3s after setting a 2s timer,
resets the timer, and expects Reset to return false: the Reset is too
late and the send has already occurred. It then expects to receive
two values: the one from before the Reset, and the one from after
the Reset.

With an unbuffered timer channel, it should be clear that no value
can be sent during the time.Sleep, so the time.Reset returns true,
indicating that the Reset stopped the timer from going off.
Then there is only one value to receive from t.C: the one from after the Reset.

In 2015, I used the above example as an argument against this change.

Note that a correct version of the program would be:

	func main() {
		t := time.NewTimer(2 * time.Second)
		time.Sleep(3 * time.Second)
		if !t.Reset(2*time.Second) {
			<-t.C
		}
		<-t.C
	}

This works with either semantics, by heeding t.Reset's result.
The change should not affect correct programs.

However, one way that the change would be visible is when programs
use len(t.C) (instead of a non-blocking receive) to poll whether the timer
has triggered already. We might legitimately worry about breaking such
programs.

In 2020, discussing #37196, Bryan Mills and I surveyed programs using
len on timer channels. These are exceedingly rare to start with; nearly all
the uses are buggy; and all the buggy programs would be fixed by the new
semantics. The details are at [1].

To further reduce the impact of this change, this CL adds a temporary
GODEBUG setting, which we didn't know about yet in 2015 and 2020.
Specifically, asynctimerchan=1 disables the change and is the default
for main programs in modules that use a Go version before 1.23.
We hope to be able to retire this setting after the minimum 2-year window.
Setting asynctimerchan=1 also disables the garbage collection change
from CL 568341, although users shouldn't need to know that since
it is not a semantically visible change (unless we have bugs!).

As an undocumented bonus that we do not officially support,
asynctimerchan=2 disables the channel buffer change but keeps
the garbage collection change. This may help while we are
shaking out bugs in either of them.

Fixes #37196.

[1] https://github.com/golang/go/issues/37196#issuecomment-641698749

Change-Id: I8925d3fb2b86b2ae87fd2acd055011cbf7bd5916
Reviewed-on: https://go-review.googlesource.com/c/go/+/568341
Reviewed-by: Austin Clements <austin@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-14 18:25:25 +00:00
Kévin Dunglas 0159150a4a cmd/go: update docs for go test -benchmem
Mention that the allocation counter doesn't count allocations made using
C.malloc (cgo) or in C.

Change-Id: I03c23b6d8cfde68c04a75732a0d3c05eedae2276
GitHub-Last-Rev: 2d805b3322
GitHub-Pull-Request: golang/go#65430
Reviewed-on: https://go-review.googlesource.com/c/go/+/560397
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: qiulaidongfeng <2645477756@qq.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2024-03-14 17:03:49 +00:00
apocelipes c841ba3a3e math/big: use built-in clear to simplify code
Change-Id: I07c3a498ce1e462c3d1703d77e7d7824e9334651
GitHub-Last-Rev: 2ba8c4c705
GitHub-Pull-Request: golang/go#66312
Reviewed-on: https://go-review.googlesource.com/c/go/+/571636
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-14 17:02:38 +00:00
Michael Knyszek 1304d9843e Revert "cmd/go/internal/test: add 'tests' vet check to 'go test' suite"
This reverts commit f1d60500bc.

Reason for revert: Broke the longtest builders.

Change-Id: I5f3510c8ffc24fae5e71fac0a2dbda01ecfe5d5c
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/571695
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-14 16:58:10 +00:00
guoguangwu 64017d10e1 time: replace time.Now().Sub call with time.Since in test
Change-Id: I56ca2d11637d60c6b0656fdc1d900a2384aba141
GitHub-Last-Rev: 686e02db77
GitHub-Pull-Request: golang/go#66264
Reviewed-on: https://go-review.googlesource.com/c/go/+/570916
Reviewed-by: qiulaidongfeng <2645477756@qq.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-14 16:32:52 +00:00
guoguangwu 90a2dcf1db cmd/compile/internal/syntax: replace bytes.Compare call with bytes.Equal
Change-Id: I783e02e215efaebf4936146c6aaa032634fdfa64
GitHub-Last-Rev: 24680a73ee
GitHub-Pull-Request: golang/go#66304
Reviewed-on: https://go-review.googlesource.com/c/go/+/571595
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-03-14 16:30:19 +00:00
Russ Cox 376be64922 encoding/gob: make x509.Certificate marshalable again
The OID type is not exported data like most of the other x509 structs.
Using it in x509.Certificate made Certificate not gob-compatible anymore,
which breaks real-world code. As a temporary fix, make gob ignore
that field, making it work as well as it did in Go 1.21.

For Go 1.23, we anticipate adding a proper fix and removing the gob
workaround. See #65633 and #66249 for more details.

For #66249.
Fixes #65633.

Change-Id: Idd1431d15063b3009e15d0565cd3120b9fa13f61
Reviewed-on: https://go-review.googlesource.com/c/go/+/571095
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2024-03-14 16:06:11 +00:00
Russ Cox 4a1038fa52 Revert "net/url: consider an empty base Path as equivalent to / in JoinPath"
This reverts commit a46285f8c2 (CL 469935).

Reason for revert: This breaks a variety of code inside Google
that seem representative of possible external real-world usage.

If we roll this forward again we should include a GODEBUG like
urljoinpathslash=0 to go back to the old behavior.

Change-Id: I6cd8e9888a0c088669dc5634418372252289e074
Reviewed-on: https://go-review.googlesource.com/c/go/+/571655
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
2024-03-14 16:01:25 +00:00
Than McIntosh be58fd0070 Revert: "cmd/link: add option to enable full RELRO for ELF"
This reverts https://go.dev/cl/c/go/+/473495.

Reason for revert: breaks some Google-internal tests.

This revert will be temporary until we can gather more info on the
nature of the failures and hopefully develop an upstream test case,
etc.

Updates #45681.

Change-Id: Ib628ddc53bc5489e4f76c0f4ad809b75e899102c
Reviewed-on: https://go-review.googlesource.com/c/go/+/571415
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-03-14 14:41:04 +00:00
guoguangwu 1e20af021f cmd/go/internal/modcmd: fix typo in comment
Change-Id: I331c46083e9608227615183ba7e25f6299669341
GitHub-Last-Rev: 0cb78ae1c1
GitHub-Pull-Request: golang/go#66305
Reviewed-on: https://go-review.googlesource.com/c/go/+/571536
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
2024-03-14 14:19:59 +00:00
Russ Cox 303aa921c5 Revert "net/http: remove superfluous newline on redirects"
This reverts commit 2b58355ef6.

Reason for revert: This breaks tons of tests for no real reason.

Change-Id: I89773f48cf983c0b6346e46c37a0ebbe2620e3b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/571675
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-14 14:18:44 +00:00
Russ Cox 2965dc9895 runtime: fix lost sleep causing TestZeroTimer flakes
Classic operating system kernel mistake: if you start using
per-CPU data without disabling interrupts on the CPU,
and then an interrupt reschedules the process onto a different
CPU, now you're using the wrong CPU's per-CPU data.
The same thing happens in Go if you use per-M or per-P
data structures while not holding a lock nor using acquirem.

In the original timer.modify before CL 564977, I had been
very careful about this during the "unlock t; lock ts" dance,
only calling releasem after ts was locked. That made sure
we used the right ts. The refactoring of that code into its
own helper function in CL 564977 missed that nuance.

The code

    ts := &getg().m.p.p.ptr().timers
    ts.lock()

was now executing without holding any locks nor acquirem.
If the goroutine changed its M or P between deciding which
ts to use and actually locking that ts, the code would proceed
to add the timer t to some other P's timers. If the P was idle
by then, the scheduler could have already checked it for timers
and not notice the newly added timer when deciding when the
next timer should trigger.

The solution is to do what the old code correctly did, namely
acquirem before deciding which ts to use, rather than assume
getg().m.p won't change before ts.lock can complete.
This CL does that.

Before CL 564977,

	stress ./time.test -test.run='ZeroTimer/impl=(func|cache)' -test.timeout=3m -test.count=20

ran without failure for over an hour on my laptop.
Starting in CL 564977, it consistently failed within a few minutes.
After this CL, it now runs without failure for over an hour again.

Fixes #66006.

Change-Id: Ib9e7ccaa0f22a326ce3fdef2b9a92f7f0bdafcbf
Reviewed-on: https://go-review.googlesource.com/c/go/+/571196
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-03-14 14:01:19 +00:00
Matthew Dempsky 9159d71a37 Revert "cmd/compile/internal/inline: refactor fixpoint algorithm"
This reverts commit 28e0052ee7.

Reason for revert: #66261

Change-Id: I9dfc8946c41e504c97ecad752971d760ae7a7416
Reviewed-on: https://go-review.googlesource.com/c/go/+/571555
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2024-03-14 13:11:09 +00:00
Alan Donovan f1d60500bc cmd/go/internal/test: add 'tests' vet check to 'go test' suite
The tests analyser reports structural problems in test
declarations. Presumably most of these would be caught by
go test itself, which compiles and runs (some subset of) the
tests, but Benchmark and Fuzz functions are executed less
frequently and may benefit more from static checks.

Also, reflect the change in go test help message.

+ release note

Fixes golang/go#44251

Change-Id: If5b9dee6d18fa0bc4de7f5f5f549eddeae953fc2
Reviewed-on: https://go-review.googlesource.com/c/go/+/529816
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Alan Donovan <adonovan@google.com>
2024-03-14 03:12:27 +00:00
Dominik Honnef 6133c1e4e2 internal/trace/v2: support old trace format
Add support for traces from Go 1.11–1.19 by converting old traces to the
Go 1.22 format on the fly.

We import Gotraceui's trace parser, which is an optimized parser based
on Go 1.19's internal/trace package, and further modify it for the needs
of the conversion process.

With the optimized parser, loading old traces using the new API is twice
as fast and uses less total memory than 'go tool trace' did in older
versions.

The new parser does not, however, support traces from versions older
than 1.11.

This commit does not update cmd/trace to use the new API for old traces.

Change-Id: If9380aa515e29445ff624274d1760ee945ca4816
Reviewed-on: https://go-review.googlesource.com/c/go/+/557356
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-14 02:20:08 +00:00
Russ Cox 508bb17edd time: garbage collect unstopped Tickers and Timers
From the beginning of Go, the time package has had a gotcha:
if you use a select on <-time.After(1*time.Minute), even if the select
finishes immediately because some other case is ready, the underlying
timer from time.After keeps running until the minute is over. This
pins the timer in the timer heap, which keeps it from being garbage
collected and in extreme cases also slows down timer operations.
The lack of garbage collection is the more important problem.

The docs for After warn against this scenario and suggest using
NewTimer with a call to Stop after the select instead, purely to work
around this garbage collection problem.

Oddly, the docs for NewTimer and NewTicker do not mention this
problem, but they have the same issue: they cannot be collected until
either they are Stopped or, in the case of Timer, the timer expires.
(Tickers repeat, so they never expire.) People have built up a shared
knowledge that timers and tickers need to defer t.Stop even though the
docs do not mention this (it is somewhat implied by the After docs).

This CL fixes the garbage collection problem, so that a timer that is
unreferenced can be GC'ed immediately, even if it is still running.
The approach is to only insert the timer into the heap when some
channel operation is blocked on it; the last channel operation to stop
using the timer takes it back out of the heap. When a timer's channel
is no longer referenced, there are no channel operations blocked on
it, so it's not in the heap, so it can be GC'ed immediately.

This CL adds an undocumented GODEBUG asynctimerchan=1
that will disable the change. The documentation happens in
the CL 568341.

Fixes #8898.
Fixes #61542.

Change-Id: Ieb303b6de1fb3527d3256135151a9e983f3c27e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/512355
Reviewed-by: Austin Clements <austin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
2024-03-13 21:36:04 +00:00
Russ Cox 74a0e3160d time: clean up benchmarks
Comparing BenchmarkStop against very old commits like
CL 13094043, I was very confused about how timers had
gotten almost 10X slower since 2013.

It turns out that CL 68060043 introduced a factor of 1000
in the benchmark cost, by counting batches of 1000 as 1 op
instead of 1000 ops, and timers have actually gotten
dramatically faster since 2013, with the addition of per-P
timer heaps and other optimizations.

This CL rewrites the benchmarks to use testing.PB directly,
so that the factor of 1000 disappears, and "/op" really means "/op".
In the few tests that need to run in batches for one reason or
another, add "1000" to the name to make clear that batches
are being run.

Change-Id: I27ed74d1e420934982e4205aad4f218cdfc42509
Reviewed-on: https://go-review.googlesource.com/c/go/+/570495
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-13 20:57:28 +00:00
qmuntal 418e6d559e os,internal/godebugs: add missing IncNonDefault calls
Fixes #66215

Change-Id: Id7de15feabe08f66c048dc114c09494813c9febc
Reviewed-on: https://go-review.googlesource.com/c/go/+/570695
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-03-13 19:10:34 +00:00
kwakubiney 381ba9f64c encoding/binary: cache struct sizes to speed up Read and Write for slice of structs.
A lot of allocations happen in dataSize due to reflection.

Cache the result of the function when encoding a
slice of structs similar to what is done for struct types
so that subsequent calls to dataSize can avoid allocations.

                        │   old.txt   │            new.txt            │
                        │   sec/op    │   sec/op     vs base          │
WriteSlice1000Structs-2   846.7µ ± 4%   856.4µ ± 3%  ~ (p=0.602 n=20)

                        │   old.txt    │            new.txt             │
                        │     B/s      │     B/s       vs base          │
WriteSlice1000Structs-2   84.48Mi ± 4%   83.52Mi ± 3%  ~ (p=0.602 n=20)

                        │   old.txt    │               new.txt               │
                        │     B/op     │     B/op      vs base               │
WriteSlice1000Structs-2   80.18Ki ± 0%   80.06Ki ± 0%  -0.15% (p=0.000 n=20)

                        │   old.txt   │              new.txt               │
                        │  allocs/op  │ allocs/op   vs base                │
WriteSlice1000Structs-2   16.000 ± 0%   1.000 ± 0%  -93.75% (p=0.000 n=2

                       │   old.txt   │              new.txt               │
                       │   sec/op    │   sec/op     vs base               │
ReadSlice1000Structs-2   847.4µ ± 4%   821.1µ ± 3%  -3.10% (p=0.012 n=20)

                       │   old.txt    │               new.txt               │
                       │     B/s      │     B/s       vs base               │
ReadSlice1000Structs-2   84.40Mi ± 4%   87.11Mi ± 3%  +3.20% (p=0.012 n=20)

                       │   old.txt    │               new.txt               │
                       │     B/op     │     B/op      vs base               │
ReadSlice1000Structs-2   80.12Ki ± 0%   80.00Ki ± 0%  -0.15% (p=0.000 n=20)

                       │   old.txt   │              new.txt               │
                       │  allocs/op  │ allocs/op   vs base                │
ReadSlice1000Structs-2   16.000 ± 0%   1.000 ± 0%  -93.75% (p=0.000 n=20)

Fixes #66253

Change-Id: I8227e61306db1fe103489ea4fee2429247c3debc
Reviewed-on: https://go-review.googlesource.com/c/go/+/570855
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-03-13 18:32:53 +00:00
Michael Matloob 4f07bb3953 cmd/go: change some counter names
Primarily, this change removes the cmd/ prefix on the go command
counter names. The 'error' counter is changed to 'errors' reflecting
that it's a bucket that contains multiple errors. the switch-exec and
select-exec counters are moved into a 'toolchain' grouping.

For #58894

Change-Id: Id6e0e7a0b4a5e42a0aef04b1210d2bb5256eb6c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/570736
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-13 18:24:48 +00:00
Russ Cox 38723f2b38 runtime: add minimal debug tracing of runtime timers
Toggled by a compile-time const, so there should be no
runtime footprint in ordinary builds.

Change-Id: I7751847524f4fda3853388d3e5a18188bd737c27
Reviewed-on: https://go-review.googlesource.com/c/go/+/570336
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Austin Clements <austin@google.com>
2024-03-13 18:22:21 +00:00
Russ Cox 6135651867 runtime: clean up timer state
The timers had evolved to the point where the state was stored as follows:

	if timer in heap:
	    state has timerHeaped set
	    if heap timer is stale:
	        heap deadline in t.when
	        real deadline in t.nextWhen
	        state has timerNextWhen set
	    else:
	        real deadline in t.when
	        t.nextWhen unset
	else:
	    real deadline in t.when
	    t.nextWhen unset

That made it hard to find the real deadline and just hard to think about everything.
The new state is:

	real deadline in t.when (always)
	if timer in heap:
	    state has timerHeaped set
	    heap deadline in t.whenHeap
	    if heap timer is stale:
	        state has timerModified set

Separately, the 'state' word itself was being used as a lock
and state bits because the code started with CAS loops,
which we abstracted into the lock/unlock methods step by step.
At this point, we can switch to a real lock, making sure to
publish the one boolean needed by timers fast paths
at each unlock.

All this simplifies various logic considerably.

Change-Id: I35766204f7a26d999206bd56cc0db60ad1b17cbe
Reviewed-on: https://go-review.googlesource.com/c/go/+/570335
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2024-03-13 17:06:22 +00:00
Russ Cox b2e9221089 runtime: fix another lock ordering problem
https://logs.chromium.org/logs/golang/buildbucket/cr-buildbucket/8753622336585847105/+/u/step/11/log/2
shows a staticlockranking crash with pollcache (defaulted to LEAF)
being held during a write barrier, which got unlucky and acquired
wbufSpans, triggering a lock ordering throw.

My change in https://go-review.googlesource.com/c/go/+/570335/13/src/runtime/netpoll.go
around line 700 caused batching of many write barriers on the first
call rather than having just a few write barriers on each call,
making the crash much more likely, but the ordering problem
appears to have always existed. We just never allocated enough
pollDescs to trigger it.

Change-Id: Icb5e8340a5027dd4f7535a5ef02b2868476539e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/571195
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2024-03-13 17:00:15 +00:00
Paul E. Murphy 6e5398bad1 cmd/asm,cmd/compile: generate less instructions for most 32 bit constant adds on ppc64x
For GOPPC64 < 10 targets, most large 32 bit constants (those
exceeding int16 capacity) can be added using two instructions
instead of 3.

This cannot be done for values greater than 0x7FFF7FFF, so this
must be done during asm preprocessing as the optab matching
rules cannot differentiate this special case.

Likewise, constants 0x8000 <= x < 0x10000 are not converted. The
assembler currently generates 2 instructions sequences for these
constants.

Change-Id: I1ccc839c6c28fc32f15d286b2e52e2d22a2a06d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/568116
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2024-03-13 13:58:44 +00:00
Keith Randall a46ecdca36 cmd/compile: fix sign/zero-extension removal
When an opcode generates a known high bit state (typically, a sub-word
operation that zeros the high bits), we can remove any subsequent
extension operation that would be a no-op.

x = (OP ...)
y = (ZeroExt32to64 x)

If OP zeros the high 32 bits, then we can replace y with x, as the
zero extension doesn't do anything.

However, x in this situation normally has a sub-word-sized type.  The
semantics of values in registers is typically that the high bits
beyond the value's type size are junk. So although the opcode
generating x *currently* zeros the high bits, after x is rewritten to
another opcode it may not - rewrites of sub-word-sized values can
trash the high bits.

To fix, move the extension-removing rules to late lower. That ensures
that their arguments won't be rewritten to change their high bits.

I am also worried about spilling and restoring. Spilling and restoring
doesn't preserve the high bits, but instead sets them to a known value
(often 0, but in some cases it could be sign-extended).  I am unable
to come up with a case that would cause a problem here, so leaving for
another time.

Fixes #66066

Change-Id: I3b5c091b3b3278ccbb7f11beda8b56f4b6d3fde7
Reviewed-on: https://go-review.googlesource.com/c/go/+/568616
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-12 19:38:41 +00:00
qmuntal 34d28ba932 internal/syscall/windows: implement SupportUnixSocket by enumerating protocols
windows.SupportUnixSocket is currently implemented using a Windows
version check. This approach is not reliable, see #27943 and #28061.
Also, it uses the undocumented RtlGetNtVersionNumbers API, which
we should try to avoid.

This PR implements SupportUnixSocket by enumerating the available
protocols and checking for AF_UNIX support.

Cq-Include-Trybots: luci.golang.try:gotip-windows-arm64
Change-Id: I76cd635067309f09571ad0eac4a5699450a2709a
Reviewed-on: https://go-review.googlesource.com/c/go/+/570075
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-12 17:24:57 +00:00
guoguangwu 293fadffe7 cmd/covdata: fix typo in comment
Change-Id: I14d2e58e36feb17a52fdc376a4562628e0da6698
GitHub-Last-Rev: 7e5d056dc5
GitHub-Pull-Request: golang/go#66262
Reviewed-on: https://go-review.googlesource.com/c/go/+/570896
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-12 17:17:47 +00:00
Sam Thanawalla 5eb87c3941 cmd/go: check case-insensitive path collisions for go mod vendor.
Fixes: #38571
Change-Id: Iec1cd1532ff17f7d943149f9b6a79e7fd419d179
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/570775
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2024-03-12 17:04:57 +00:00
apocelipes f83102cf71 strconv: use generics to reduce redundant helper functions
Benchstat shows there are no noticeable performance changes here.

Change-Id: If2250334fe6664986f044cbaabfa1bfc84f871f7
GitHub-Last-Rev: d41a498d54
GitHub-Pull-Request: golang/go#66266
Reviewed-on: https://go-review.googlesource.com/c/go/+/570935
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
2024-03-12 16:49:33 +00:00
Bryan C. Mills 2ab9218c86 cmd/go/internal/modfetch: pass "-c" arguments to git before subcommands
I accidentally transposed the arguments in CL 556358, causing the
shallow 'git fetch' attempt to always fail. That didn't break any
tests because we fall back to a full fetch, which works for nearly all
real Git servers, and we didn't have a test that checked for shallow
fetches.

Tested manually using:
	GOPROXY=direct go mod download -x -json gerrit.wikimedia.org/r/mediawiki@v0.0.0-20240202145822-67da0cbcfdf7

(I'm still thinking about how to add a proper regression test.)

Fixes #66147.

Change-Id: I0bb17283bae856f369fd24f29375e507d0999933
Cq-Include-Trybots: luci.golang.try:gotip-darwin-amd64-longtest,gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/569422
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2024-03-11 22:10:38 +00:00
sivchari a18aa0e3d1 all: gofmt
These files are not formatted by gofmt. Thus, run gofmt to format them.

Change-Id: Iea9650e64b1f47cf82739f3a8a34f47740a96455
Reviewed-on: https://go-review.googlesource.com/c/go/+/570398
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-03-11 19:54:31 +00:00
Jes Cok d85d11f101 all.bash: allow spaces in $GOTOOLDIR to print build info
For consistency with all.bat: "%GOTOOLDIR%/dist" banner

Fixes #66061

Change-Id: I3387003a77a5fe82fe132e7aba472b06dd9068f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/570395
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-11 19:53:58 +00:00
guoguangwu 80157f4cff io: close PipeReader in test
Change-Id: I33858efc00dff02432f28f1e5a94aeea261a5bad
GitHub-Last-Rev: 98861f8d6e
GitHub-Pull-Request: golang/go#66230
Reviewed-on: https://go-review.googlesource.com/c/go/+/570357
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-03-11 19:49:15 +00:00
apocelipes b4b9746e51 strconv: use slices.BinarySearch to simplify makeisprint.go
Change-Id: I9886a99f730b7616f6f8a5e6154e1beb7d3c79e6
GitHub-Last-Rev: 3f9dc77073
GitHub-Pull-Request: golang/go#66242
Reviewed-on: https://go-review.googlesource.com/c/go/+/570535
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2024-03-11 18:56:17 +00:00
Michael Matloob edbb5a13b4 cmd/go: provide a better error message when there's no go directive
On Go 1.21+ it's an error for a workspace to contain a module with a
version newer than the workspace's stated go version. If the workspace
doesn't explicitly have a go version it's explicitly 1.18. So if a
workspace without a go directive contains a module whose go directive
is newer on it's always an error for 1.21+. In the error, before this
CL the error would read "module <path> listed in go.work requires go
>= <version>, but go.work lists go 1.18". After this change the second
clause would read "but go.work implicitly requires go 1.18.

Fixes #66207

Change-Id: I44680880162a82e5cee9cfc8655d6774add6f762
Reviewed-on: https://go-review.googlesource.com/c/go/+/570735
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-11 18:35:47 +00:00
Kevin Burke 3a41bfac9b net/http/httptest: add NewRequestWithContext
This matches the net/http API.

Updates #59473.

Change-Id: I99917cef3ed42a0b4a2b39230b492be00da8bbfd
Reviewed-on: https://go-review.googlesource.com/c/go/+/548355
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-03-11 18:09:14 +00:00
胡玮文 c8e4f8c775 net/http: support socks5h proxy schema
Extend the net/http Transport to recognize the 'socks5h' schema as an
alias for 'socks5'. Traditionally, the 'socks5h' schema indicates that
the hostname should be resolved by the proxy server, which is behavior
already implemented in Go for 'socks5'.

Fixes #24135

Change-Id: I0a6a92bbd282a3200dc4dc7b47a9b0628f931783
Reviewed-on: https://go-review.googlesource.com/c/go/+/569977
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2024-03-11 18:07:37 +00:00
qiulaidongfeng ec4baaca39 time: fix typo in BenchmarkReset
Change-Id: I1dbd1c5aa26f458cdac7a3f0ca974254a069311f
GitHub-Last-Rev: da481ba7a9
GitHub-Pull-Request: golang/go#66219
Reviewed-on: https://go-review.googlesource.com/c/go/+/569897
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-03-11 17:11:35 +00:00
guoguangwu a3eb3b8148 compress/gzip: close writer in test
Change-Id: I12bc9287106f1492cbc9e74b4163cce97c957d31
GitHub-Last-Rev: cda1b48fe3
GitHub-Pull-Request: golang/go#66185
Reviewed-on: https://go-review.googlesource.com/c/go/+/569896
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-11 17:10:06 +00:00
guoguangwu 3e1a6925a1 os: close pipe in test
Change-Id: Ic8b06c6fd9fc6a30b26f4e4614aa40b5cad3a5e7
GitHub-Last-Rev: 8397a8b30c
GitHub-Pull-Request: golang/go#66240
Reviewed-on: https://go-review.googlesource.com/c/go/+/570515
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-03-11 17:08:40 +00:00
apocelipes ed08d01dc3 image: use built-in clear to simplify code
Change-Id: Id34936a115baaf61e4268582c6d9a2027494c385
GitHub-Last-Rev: 5fe455b7d2
GitHub-Pull-Request: golang/go#66244
Reviewed-on: https://go-review.googlesource.com/c/go/+/570555
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
2024-03-11 17:08:05 +00:00
guoguangwu e3d4491ad6 cmd/preprofile: fix typo in comment
Change-Id: Ib44e9e6345fa8df7f46bc9cbdc19ad8ba73c8b83
GitHub-Last-Rev: 5a37ad7988
GitHub-Pull-Request: golang/go#66233
Reviewed-on: https://go-review.googlesource.com/c/go/+/570415
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-03-11 17:08:02 +00:00
guoguangwu b28b4248f9 cmd/trace/v2: fix typo in comment
Change-Id: Icbf295e668335945084616a88c3ea2cef1bb2527
GitHub-Last-Rev: 0341d0fea7
GitHub-Pull-Request: golang/go#66229
Reviewed-on: https://go-review.googlesource.com/c/go/+/570356
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-03-11 17:07:59 +00:00
guoguangwu 78755f6b8c cmd/compile: use raw strings to avoid double escapes
Change-Id: I39917b90b67f630f8212853c0a201635960275cb
GitHub-Last-Rev: fe886534b4
GitHub-Pull-Request: golang/go#66180
Reviewed-on: https://go-review.googlesource.com/c/go/+/569975
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2024-03-11 05:51:32 +00:00
Russ Cox 48b10c9af7 net/http: update bundle
go install golang.org/x/tools/cmd/bundle@latest
go generate net/http

This fixes the longtest builders, which broke at CL 570156.

Change-Id: I85e6a1c20bd0080228400a561efd750342ae2d67
Reviewed-on: https://go-review.googlesource.com/c/go/+/570276
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-10 07:39:08 +00:00
Joel Sing 1e433915ce cmd/link,debug/elf: mark Go binaries with no branch target CFI on openbsd
OpenBSD enables Indirect Branch Tracking (IBT) on amd64 and Branch Target
Identification (BTI) on arm64, where hardware permits. Since Go generated
binaries do not currently support IBT or BTI, temporarily mark them with
PT_OPENBSD_NOBTCFI which prevents branch target CFI from being enforced
on execution. This should be removed as soon asn IBT and BTI support are
available.

Fixes #66040
Updates #66054

Change-Id: I91ac05736e6942c54502bef4b8815eb8740d2d5e
Reviewed-on: https://go-review.googlesource.com/c/go/+/568435
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Josh Rickmar <jrick@zettaport.com>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Joel Sing <joel@sing.id.au>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2024-03-10 04:13:26 +00:00
Russ Cox 47d4295f92 cmd/go: respect GODEBUG in test cache
GODEBUG affects test execution but was not being tracked.

Fixes #66213.
Fixes #65436.

Change-Id: I3ac3c397f0c6fa46cd9be0d22d03020d0632f64f
Reviewed-on: https://go-review.googlesource.com/c/go/+/570259
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
2024-03-09 22:23:53 +00:00
Russ Cox aa511b426d time: disable TestLongAdjustTimers on android/ios
The simulators are too slow.

Change-Id: I0aaf2304ad0881c74886ff3185c09614de2aae63
Reviewed-on: https://go-review.googlesource.com/c/go/+/570236
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
2024-03-09 22:19:48 +00:00
Russ Cox 74726defe9 internal/godebugs: test for use of IncNonDefault
A few recent godebugs are missing IncNonDefault uses.
Test for that, so that people remember to do it.
Filed bugs for the missing ones.

For #66215.
For #66216.
For #66217.

Change-Id: Ia3fd10fd108e1b003bb30a8bc2f83995c768fab6
Reviewed-on: https://go-review.googlesource.com/c/go/+/570275
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
2024-03-09 14:19:39 +00:00
Hana (Hyang-Ah) Kim 065c5d220e cmd/go: check checkCounters counter read only on supported platforms
Telemetry counters writing is disabled on certain platforms.
See x/telemetry/internal/telemetry.DisabledOnPlatform.

For #66205

Change-Id: I833e15ae33fb27e09d67fc77b921498476237176
Reviewed-on: https://go-review.googlesource.com/c/go/+/570196
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-09 04:24:30 +00:00