Consider this code:
func f(*int)
func g() {
p := new(int)
f(p)
}
where f is an assembly function.
In general liveness analysis assumes that during the call to f, p is dead
in this frame. If f has retained p, p will be found alive in f's frame and keep
the new(int) from being garbage collected. This is all correct and works.
We use the Go func declaration for f to give the assembly function
liveness information (the arguments are assumed live for the entire call).
Now consider this code:
func h1() {
p := new(int)
syscall.Syscall(1, 2, 3, uintptr(unsafe.Pointer(p)))
}
Here syscall.Syscall is taking the place of f, but because its arguments
are uintptr, the liveness analysis and the garbage collector ignore them.
Since p is no longer live in h once the call starts, if the garbage collector
scans the stack while the system call is blocked, it will find no reference
to the new(int) and reclaim it. If the kernel is going to write to *p once
the call finishes, reclaiming the memory is a mistake.
We can't change the arguments or the liveness information for
syscall.Syscall itself, both for compatibility and because sometimes the
arguments really are integers, and the garbage collector will get quite upset
if it finds an integer where it expects a pointer. The problem is that
these arguments are fundamentally untyped.
The solution we have taken in the syscall package's wrappers in past
releases is to insert a call to a dummy function named "use", to make
it look like the argument is live during the call to syscall.Syscall:
func h2() {
p := new(int)
syscall.Syscall(1, 2, 3, uintptr(unsafe.Pointer(p)))
use(unsafe.Pointer(p))
}
Keeping p alive during the call means that if the garbage collector
scans the stack during the system call now, it will find the reference to p.
Unfortunately, this approach is not available to users outside syscall,
because 'use' is unexported, and people also have to realize they need
to use it and do so. There is much existing code using syscall.Syscall
without a 'use'-like function. That code will fail very occasionally in
mysterious ways (see #13372).
This CL fixes all that existing code by making the compiler do the right
thing automatically, without any code modifications. That is, it takes h1
above, which is incorrect code today, and makes it correct code.
Specifically, if the compiler sees a foreign func definition (one
without a body) that has uintptr arguments, it marks those arguments
as "unsafe uintptrs". If it later sees the function being called
with uintptr(unsafe.Pointer(x)) as an argument, it arranges to mark x
as having escaped, and it makes sure to hold x in a live temporary
variable until the call returns, so that the garbage collector cannot
reclaim whatever heap memory x points to.
For now I am leaving the explicit calls to use in package syscall,
but they can be removed early in a future cycle (likely Go 1.7).
The rule has no effect on escape analysis, only on liveness analysis.
Fixes#13372.
Change-Id: I2addb83f70d08db08c64d394f9d06ff0a063c500
Reviewed-on: https://go-review.googlesource.com/18584
Reviewed-by: Ian Lance Taylor <iant@golang.org>
CMOVs were not introduced until P6. We need 386 to run on
Pentium MMX.
Fixes#13923
Change-Id: Iee9572cd83e64c3a1336bc1e6b300b048fbcc996
Reviewed-on: https://go-review.googlesource.com/18621
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
This is the equivalent of https://golang.org/cl/18549 for
the binary importer (which is usually not used because by
default the gc compiler produces the traditional textual
export format).
For #13898.
Change-Id: Idb6b515f2ee49e6d0362c71846994b0bd4dae8f7
Reviewed-on: https://go-review.googlesource.com/18598
Reviewed-by: Alan Donovan <adonovan@google.com>
Run-TryBot: Robert Griesemer <gri@golang.org>
GCC 4.8 exits 1 on an unrecognized option, but GCC 4.4 and 4.5 exit 0.
I didn't check other versions, or try to figure out just when this
changed.
Fixes#13937.
Change-Id: If193e9053fbb535999c9bde99f430f465a8c7c57
Reviewed-on: https://go-review.googlesource.com/18597
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The package of anonymous fields is the package in which they were
declared, not the package of the anonymous field's type. Was correct
before and incorrectly changed with https://golang.org/cl/18549.
Change-Id: I9fd5bfbe9d0498c8733b6ca7b134a85defe16113
Reviewed-on: https://go-review.googlesource.com/18596
Reviewed-by: Alan Donovan <adonovan@google.com>
This makes lldb willing to debug them.
The minimum version is hard-coded at OS X 10.7,
because that is the minimum that Go requires.
For more control over the version, users can
use linkmode=external and pass the relevant flags to the host linker.
Fixes#12941.
Change-Id: I20027be8aa034d07dd2a3326828f75170afe905f
Reviewed-on: https://go-review.googlesource.com/18588
Reviewed-by: Ian Lance Taylor <iant@golang.org>
In gc export data, exported struct field and interface method names appear
in unqualified form (i.e., w/o package name). The (gc)importer assumed that
unqualified exported names automatically belong to the package being imported.
This is not the case if the field or method belongs to a struct or interface
that was declared in another package and re-exported.
The issue becomes visible if a type T (say an interface with a method M)
is declared in a package A, indirectly re-exported by a package B (which
imports A), and then imported in C. If C imports both A and B, if A is
imported before B, T.M gets associated with the correct package A. If B
is imported before A, T.M appears to be exported by B (even though T itself
is correctly marked as coming from A). If T.M is imported again via the
import of A if gets dropped (as it should) because it was imported already.
The fix is to pass down the parent package when we parse imported types
so that the importer can use the correct package when creating fields
and methods.
Fixes#13898.
Change-Id: I7ec2ee2dda15859c582b65db221c3841899776e1
Reviewed-on: https://go-review.googlesource.com/18549
Reviewed-by: Alan Donovan <adonovan@google.com>
* Enable c-shared buildmode on darwin/386
* dyld does not support text relocation on i386. Add -read_only_relocs suppress flag to linker
Fixes#13904
Change-Id: I9adbd20d3f36ce9bbccf1bffb746b391780d088f
Reviewed-on: https://go-review.googlesource.com/18500
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Updates x/net/http2 to git rev c93a9b4f2a for https://golang.org/cl/18474
Forgot to submit this four days ago.
Change-Id: Id96ab164ec765911c31874cca39b44aa55e80153
Reviewed-on: https://go-review.googlesource.com/18574
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
When the Transport was creating an bound HTTP connection (protocol
unknown initially) and then ends up deciding it doesn't need it, a
goroutine sits around to clean up whatever the result was. That
goroutine made the false assumption that the result was always an
HTTP/1 connection or an error. It may also be an alternate protocol
in which case the *persistConn.conn net.Conn field is nil, and the
alt field is non-nil.
Fixes#13839
Change-Id: Ia4972e5eb1ad53fa00410b3466d4129c753e0871
Reviewed-on: https://go-review.googlesource.com/18573
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The fucomi* opcodes were only introduced for the Pentium Pro.
They do not exist for an MMX Pentium. Use the fucom* instructions
instead and move the condition codes from the fp flags register to
the integer flags register explicitly.
The use of fucomi* opcodes in ggen.go was introduced in 1.5 (CL 8738).
The bad ops were generated for 64-bit floating-point comparisons.
The use of fucomi* opcodes in gsubr.go dates back to at least 1.1.
The bad ops were generated for float{32,64} to uint64 conversions.
Fixes#13923
Change-Id: I5290599f5edea8abf8fb18036f44fa78bd1fc9e6
Reviewed-on: https://go-review.googlesource.com/18590
Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Install pkg.h rather than libpkg.h.
Link against -lc.
Fixes#13860.
Change-Id: I4e429426f8363712a5dbbd2655b9aab802ab2888
Reviewed-on: https://go-review.googlesource.com/18592
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Add several instructions that were used via BYTE and use them.
Instructions added: PEXTRB, PEXTRD, PEXTRQ, PINSRB, XGETBV, POPCNT.
Change-Id: I5a80cd390dc01f3555dbbe856a475f74b5e6df65
Reviewed-on: https://go-review.googlesource.com/18593
Run-TryBot: Ilya Tocar <ilya.tocar@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
This change applies the fix for #13564 to Plan 9 and Windows.
Also enables Lookup API test cases on builders.
Updates #13564.
Change-Id: I863f03c7cb6fbe58b3a55223bfa0ac5f9bf9c3df
Reviewed-on: https://go-review.googlesource.com/18559
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Conn.Close sends an encrypted "close notify" to signal secure EOF.
But writing that involves acquiring mutexes (handshake mutex + the
c.out mutex) and writing to the network. But if the reason we're
calling Conn.Close is because the network is already being
problematic, then Close might block, waiting for one of those mutexes.
Instead of blocking, and instead of introducing new API (at least for
now), distinguish between a normal Close (one that sends a secure EOF)
and a resource-releasing destructor-style Close based on whether there
are existing Write calls in-flight.
Because io.Writer and io.Closer aren't defined with respect to
concurrent usage, a Close with active Writes is already undefined, and
should only be used during teardown after failures (e.g. deadlines or
cancelations by HTTP users). A normal user will do a Write then
serially do a Close, and things are unchanged for that case.
This should fix the leaked goroutines and hung net/http.Transport
requests when there are network errors while making TLS requests.
Change-Id: If3f8c69d6fdcebf8c70227f41ad042ccc3f20ac9
Reviewed-on: https://go-review.googlesource.com/18572
Reviewed-by: Adam Langley <agl@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Brief background on "why heap allocate". Things can be
forced to the heap for the following reasons:
1) address published, hence lifetime unknown.
2) size unknown/too large, cannot be stack allocated
3) multiplicity unknown/too large, cannot be stack allocated
4) reachable from heap (not necessarily published)
The bug here is a case of failing to enforce 4) when an
object Y was reachable from a heap allocation X forced
because of 3). It was found in the case of a closure
allocated within a loop (X) and assigned to a variable
outside the loop (multiplicity unknown) where the closure
also captured a map (Y) declared outside the loop (reachable
from heap). Note the variable declared outside the loop (Y)
is not published, has known size, and known multiplicity
(one). The only reason for heap allocation is that it was
reached from a heap allocated item (X), but because that was
not forced by publication, it has to be tracked by loop
level, but escape-loop level was not tracked and thus a bug
results.
The fix is that when a heap allocation is newly discovered,
use its looplevel as the minimum loop level for downstream
escape flooding.
Every attempt to generalize this bug to X-in-loop-
references-Y-outside loop succeeded, so the fix was aimed
to be general. Anywhere that loop level forces heap
allocation, the loop level is tracked. This is not yet
tested for all possible X and Y, but it is correctness-
conservative and because it caused only one trivial
regression in the escape tests, it is probably also
performance-conservative.
The new test checks the following:
1) in the map case, that if fn escapes, so does the map.
2) in the map case, if fn does not escape, neither does the map.
3) in the &x case, that if fn escapes, so does &x.
4) in the &x case, if fn does not escape, neither does &x.
Fixes#13799.
Change-Id: Ie280bef2bb86ec869c7c206789d0b68f080c3fdb
Reviewed-on: https://go-review.googlesource.com/18234
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
A bit cleanuppy for 1.6 maybe, but something I happened to notice.
Change-Id: I70f3b48445f4f527d67f7b202b6171195440b09f
Reviewed-on: https://go-review.googlesource.com/18550
Reviewed-by: Russ Cox <rsc@golang.org>
[Repeat of CL 18343 with build fixes.]
Before, NumGoroutine counted system goroutines and Stack (usually) didn't show them,
which was inconsistent and confusing.
To resolve which way they should be consistent, it seems like
package main
import "runtime"
func main() { println(runtime.NumGoroutine()) }
should print 1 regardless of internal runtime details. Make it so.
Fixes#11706.
Change-Id: If26749fec06aa0ff84311f7941b88d140552e81d
Reviewed-on: https://go-review.googlesource.com/18432
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
Many browsers now support schemeless URLs in the Location headers
and also it is allowed in the draft HTTP/1.1 specification (see
http://stackoverflow.com/q/4831741#comment25926312_4831741), but
Go standard library lacks support for them.
This patch implements schemeless URLs support in http.Redirect().
Since url.Parse() correctly handles schemeless URLs, I've just added
an extra condition to verify URL's Host part in the absoulute/relative
check in the http.Redirect function.
Also I've moved oldpath variable initialization inside the block
of code where it is used.
Change-Id: Ib8a6347816a83e16576f00c4aa13224a89d610b5
Reviewed-on: https://go-review.googlesource.com/14172
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It would certainly be a mistake to invoke a write barrier while
greying an object.
Change-Id: I34445a15ab09655ea8a3628a507df56aea61e618
Reviewed-on: https://go-review.googlesource.com/18533
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It used to be the case that repeatedly getting one GC pointer and
enqueuing one GC pointer could cause contention on the work buffers as
each operation passed over the boundary of a work buffer. As of
b6c0934, we use a two buffer cache that prevents this sort of
contention.
Change-Id: I4f1111623f76df9c5493dd9124dec1e0bfaf53b7
Reviewed-on: https://go-review.googlesource.com/18532
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This comment is probably a hold-over from when the heap bitmap was
interleaved and the shift was 0, 2, 4, or 6. Now the shift is 0, 1, 2,
or 3.
Change-Id: I096ec729e1ca31b708455c98b573dd961d16aaee
Reviewed-on: https://go-review.googlesource.com/18531
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
Go fails to build on a system which has PIE enabled by default like this:
/usr/bin/ld: -r and -pie may not be used together
collect2: error: ld returned 1 exit status
The only system I know that has this property right now is Ubuntu Xenial
running on s390x, which is hardly the most accessible system, but it's planned
to enable this on amd64 soon too. The fix is to pass -no-pie along with -Wl,-r
to the compiler, but unfortunately that flag is very new as well. So this does
a test compile of a trivial file to see if the flag is supported.
Change-Id: I1345571142b7c3a96212e43297d19e84ec4a3d41
Reviewed-on: https://go-review.googlesource.com/18359
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
There are reports of corruption. Let's disable it for now (for Go 1.6,
especially) until we can investigate and fix properly.
Update #13892
Change-Id: I557275e5142fe616e8a4f89c00ffafb830eb3b78
Reviewed-on: https://go-review.googlesource.com/18540
Reviewed-by: Dave Cheney <dave@cheney.net>
Per suggestion from adonovan.
Change-Id: Icbb4d2f201590bc94672b8d8141b6e7901e11dc5
Reviewed-on: https://go-review.googlesource.com/18510
Reviewed-by: Robert Griesemer <gri@golang.org>
In the presence of vendored packages, the path found in a package
declaration may not be the path at which the package imported from
srcDir was found. Use the correct package path.
Change-Id: I74496c3cdf82a5dbd6a5bd189bb3cd0ca103fd52
Reviewed-on: https://go-review.googlesource.com/18460
Reviewed-by: Alan Donovan <adonovan@google.com>
Fixes#13881.
Change-Id: Idff77db381640184ddd2b65022133bb226168800
Reviewed-on: https://go-review.googlesource.com/18449
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Currently, due to an oversight, we only balance work buffers
in background and idle workers and not in assists. As a
result, in assist-heavy workloads, assists are likely to tie
up large work buffers in per-P caches increasing the
likelihood that the global list will be empty. This increases
the likelihood that other GC workers will exit and assists
will block, slowing down the system as a whole. Fix this by
eagerly balancing work buffers as soon as the assists notice
that the global buffers are empty. This makes it much more
likely that work will be immediately available to other
workers and assists.
This change reduces the garbage benchmark time by 39% and
fixes the regresssion seen at CL 15893 golang.org/cl/15893.
Garbage benchmark times before and after this CL.
Before GOPERF-METRIC:time=4427020
After GOPERF-METRIC:time=2721645
Fixes#13827
Change-Id: I9cb531fb873bab4b69ce9c1617e30df6c49cdcfe
Reviewed-on: https://go-review.googlesource.com/18341
Reviewed-by: Austin Clements <austin@google.com>
The AESNI GCM code decrypts and authenticates concurrently and so
overwrites the destination buffer even in the case of an authentication
failure.
This change updates the documentation to make that clear and also
mimics that behaviour in the generic code so that different platforms
act identically.
Fixes#13886
Change-Id: Idc54e51f01e27b0fc60c1745d50bb4c099d37e94
Reviewed-on: https://go-review.googlesource.com/18480
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
mips64 builder and one machine of the mips64le builder has small amount
of memory. Since CL 18199, they have been running slowly, as more
processes were launched in running 'test' directory, and a lot of swap
were used. This CL brings all.bash from 5h back to 3h on Loongson 2E
with 512 MB memory.
Change-Id: I4a22e239a542a99ba5986753205d8cd1f4b3d3c6
Reviewed-on: https://go-review.googlesource.com/18483
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Updates http2 to x/net git rev 0e6d34ef942 for https://golang.org/cl/18472
which means we'll get to delete a ton of grpc-go code and just use the
standard library's HTTP client instead.
Also, the comments in this CL aren't entirely accurate it turns out.
RFC 2616 says:
"The Trailer header field can be used to indicate which header fields
are included in a trailer (see section 14.40)."
And 14.40:
" An HTTP/1.1 message SHOULD include a Trailer header field in a
message using chunked transfer-coding with a non-empty trailer. Doing
so allows the recipient to know which header fields to expect in the
trailer.
If no Trailer header field is present, the trailer SHOULD NOT include
any header fields. See section 3.6.1 for restrictions on the use of
trailer fields in a "chunked" transfer-coding."
So it's really a SHOULD more than a MUST.
And gRPC (at least Google's server) doesn't predeclare "grpc-status"
ahead of time in a Trailer Header, so we'll be lenient. We were too
strict anyway. It's also not a concern for the Go client we have a
different place to populate the Trailers, and it won't confuse clients
which aren't looking for them. The ResponseWriter server side is more
complicated (and strict), though, since we don't want to widen the
ResponseWriter interface. So the Go server still requires that you
predeclare Trailers.
Change-Id: Ia2defc11a2469fb8570ecfabb8453537121084eb
Reviewed-on: https://go-review.googlesource.com/18473
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
