languages/go
languages/go
1
0
Fork 0
mirror of https://github.com/golang/go synced 2024-09-15 22:20:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

[dev.typeparams] Merge branch 'master' into dev.typeparams

Browse source 
Change-Id: Ie8e697895b1d04ab79d35ef5a886f005be209756
This commit is contained in:
Robert Griesemer 2020-11-19 17:01:31 -08:00
parent 0123c9b321 c72a448881
commit b1ae0a0646
378 changed files with 18600 additions and 6479 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
doc/articles/race_detector.html
View file

@ -418,7 +418,7 @@ close(c)
<code>linux/amd64</code>, <code>linux/ppc64le</code>,
<code>linux/arm64</code>, <code>freebsd/amd64</code>,
<code>netbsd/amd64</code>, <code>darwin/amd64</code>,
and <code>windows/amd64</code>.
<code>darwin/arm64</code>, and <code>windows/amd64</code>.
</p>
<h2 id="Runtime_Overheads">Runtime Overhead</h2>

130
doc/go1.16.html
View file

@ -31,6 +31,29 @@ Do not send CLs removing the interior tags from such phrases.
<h2 id="ports">Ports</h2>
<h3 id="darwin">Darwin</h3>
<p><!-- golang.org/issue/38485, golang.org/issue/41385, many CLs -->
Go 1.16 adds support of 64-bit ARM architecture on macOS (also known as
Apple Silicon) with <code>GOOS=darwin</code>, <code>GOARCH=arm64</code>.
Like the <code>darwin/amd64</code> port, the <code>darwin/arm64</code>
port supports cgo, internal and external linking, <code>c-archive</code>,
<code>c-shared</code>, and <code>pie</code> build modes, and the race
detector.
</p>
<p>
The iOS port, which was previously <code>darwin/arm64</code>, is now
moved to <code>ios/arm64</code>. <code>GOOS=ios</code> implies the
<code>darwin</code> build tag, just as <code>GOOS=android</code>
implies the <code>linux</code> build tag.
</p>
<p><!-- golang.org/issue/42100, CL 263798 -->
The <code>ios/amd64</code> port is added, targetting the iOS simulator
running on AMD64-based macOS.
</p>
<h3 id="netbsd">NetBSD</h3>
<p><!-- golang.org/issue/30824 -->
@ -59,6 +82,15 @@ Do not send CLs removing the interior tags from such phrases.
<h4 id="modules">Modules</h4>
<p><!-- golang.org/issue/40728 -->
Build commands like <code>go</code> <code>build</code> and <code>go</code>
<code>test</code> no longer modify <code>go.mod</code> and <code>go.sum</code>
by default. Instead, they report an error if a module requirement or checksum
needs to be added or updated (as if the <code>-mod=readonly</code> flag were
used). Module requirements and sums may be adjusted with <code>go</code>
<code>mod</code> <code>tidy</code> or <code>go</code> <code>get</code>.
</p>
<p><!-- golang.org/issue/40276 -->
<code>go</code> <code>install</code> now accepts arguments with
version suffixes (for example, <code>go</code> <code>install</code>
@ -71,6 +103,16 @@ Do not send CLs removing the interior tags from such phrases.
TODO: write and link to blog post
</p>
<p><!-- golang.org/issue/40276 -->
<code>go</code> <code>install</code>, with or without a version suffix (as
described above), is now the recommended way to build and install packages in
module mode. <code>go</code> <code>get</code> should be used with the
<code>-d</code> flag to adjust the current module's dependencies without
building packages, and use of <code>go</code> <code>get</code> to build and
install packages is deprecated. In a future release, the <code>-d</code> flag
will always be enabled.
</p>
<p><!-- golang.org/issue/24031 -->
<code>retract</code> directives may now be used in a <code>go.mod</code> file
to indicate that certain published versions of the module should not be used
@ -110,6 +152,16 @@ Do not send CLs removing the interior tags from such phrases.
See <code>go</code> <code>help</code> <code>environment</code> for details.
</p>
<h4 id="go-get"><code>go</code> <code>get</code></h4>
<p><!-- golang.org/cl/263267 -->
<code>go</code> <code>get</code> <code>example.com/mod@patch</code> now
requires that some version of <code>example.com/mod</code> already be
required by the main module.
(However, <code>go</code> <code>get</code> <code>-u=patch</code> continues
to patch even newly-added dependencies.)
</p>
<h4 id="all-pattern">The <code>all</code> pattern</h4>
<p><!-- golang.org/cl/240623 -->
@ -131,6 +183,18 @@ Do not send CLs removing the interior tags from such phrases.
being built.
</p>
<h4 id="i-flag">The <code>-i</code> build flag</h4>
<p><!-- golang.org/issue/41696 -->
The <code>-i</code> flag accepted by <code>go</code> <code>build</code>,
<code>go</code> <code>install</code>, and <code>go</code> <code>test</code> is
now deprecated. The <code>-i</code> flag instructs the <code>go</code> command
to install packages imported by packages named on the command line. Since
the build cache was introduced in Go 1.10, the <code>-i</code> flag no longer
has a significant effect on build times, and it causes errors when the install
directory is not writable.
</p>
<h4 id="list-buildid">The <code>list</code> command</h4>
<p><!-- golang.org/cl/263542 -->
@ -227,12 +291,52 @@ Do not send CLs removing the interior tags from such phrases.
<p><!-- CL 256897 -->
I/O operations on closing or closed TLS connections can now be detected using
the new <a href="/pkg/net/#ErrClosed">ErrClosed</a> error. A typical use
would be <code>errors.Is(err, net.ErrClosed)</code>. In earlier releases
the new <a href="/pkg/net/#ErrClosed">ErrClosed</a> error. A typical use
would be <code>errors.Is(err, net.ErrClosed)</code>. In earlier releases
the only way to reliably detect this case was to match the string returned
by the <code>Error</code> method with <code>"tls: use of closed connection"</code>.
</p>
<p><!-- CL 266037 -->
A default deadline is set in <a href="/pkg/crypto/tls/#Conn.Close">Close</a>
before sending the close notify alert, in order to prevent blocking
indefinitely.
</p>
<p><!-- CL 246338 -->
<a href="/pkg/crypto/tls#Conn.HandshakeContext">(*Conn).HandshakeContext</a> was added to
allow the user to control cancellation of an in-progress TLS Handshake.
The context provided is propagated into the
<a href="/pkg/crypto/tls#ClientHelloInfo">ClientHelloInfo</a>
and <a href="/pkg/crypto/tls#CertificateRequestInfo">CertificateRequestInfo</a>
structs and accessible through the new
<a href="/pkg/crypto/tls#ClientHelloInfo.Context">(*ClientHelloInfo).Context</a>
and
<a href="/pkg/crypto/tls#CertificateRequestInfo.Context">
(*CertificateRequestInfo).Context
</a> methods respectively. Canceling the context after the handshake has finished
has no effect.
</p>
<p><!-- CL 239748 -->
Clients now ensure that the server selects
<a href="/pkg/crypto/tls/#ConnectionState.NegotiatedProtocol">
an ALPN protocol</a> from
<a href="/pkg/crypto/tls/#Config.NextProtos">
the list advertised by the client</a>.
</p>
<p><!-- CL 262857 -->
TLS servers will now prefer other AEAD cipher suites (such as ChaCha20Poly1305)
over AES-GCM cipher suites if either the client or server doesn't have AES hardware
support, unless the application set both
<a href="/pkg/crypto/tls/#Config.PreferServerCipherSuites"><code>Config.PreferServerCipherSuites</code></a>
and <a href="/pkg/crypto/tls/#Config.CipherSuites"><code>Config.CipherSuites</code></a>
or there are no other AEAD cipher suites supported.
The client is assumed not to have AES hardware support if it does not signal a
preference for AES-GCM cipher suites.
</p>
<h3 id="crypto/x509"><a href="/pkg/crypto/x509">crypto/x509</a></h3>
<p><!-- CL 235078 -->
@ -250,6 +354,13 @@ Do not send CLs removing the interior tags from such phrases.
of a malformed certificate.
</p>
<p><!-- CL 233163 -->
A number of additional fields have been added to the
<a href="/pkg/crypto/x509/#CertificateRequest">CertificateRequest</a> type.
These fields are now parsed in <a href="/pkg/crypto/x509/#ParseCertificateRequest">ParseCertificateRequest</a>
and marshalled in <a href="/pkg/crypto/x509/#CreateCertificateRequest">CreateCertificateRequest</a>.
</p>
<h3 id="encoding/json"><a href="/pkg/encoding/json">encoding/json</a></h3>
<p><!-- CL 263619 -->
@ -367,6 +478,13 @@ Do not send CLs removing the interior tags from such phrases.
Cookies set with <code>SameSiteDefaultMode</code> now behave according to the current
spec (no attribute is set) instead of generating a SameSite key without a value.
</p>
<p><!-- CL 246338 -->
The <a href="/pkg/net/http/"><code>net/http</code></a> package now uses the new
<a href="/pkg/crypto/tls#Conn.HandshakeContext"><code>(*tls.Conn).HandshakeContext</code></a>
with the <a href="/pkg/net/http/#Request"><code>Request</code></a> context
when performing TLS handshakes in the client or server.
</p>
</dd>
</dl><!-- net/http -->
@ -378,6 +496,14 @@ Do not send CLs removing the interior tags from such phrases.
</dd>
</dl><!-- runtime/debug -->
<dl id="syscall"><dt><a href="/pkg/syscall/">syscall</a></dt>
<dd>
<p><!-- CL 261917 -->
<a href="/pkg/syscall/#SysProcAttr"><code>SysProcAttr</code></a> on Windows has a new NoInheritHandles field that disables inheriting handles when creating a new process.
</p>
</dd>
</dl><!-- syscall -->
<dl id="strconv"><dt><a href="/pkg/strconv/">strconv</a></dt>
<dd>
<p><!-- CL 260858 -->

216
misc/cgo/errors/badsym_test.go Normal file
View file

@ -0,0 +1,216 @@
// Copyright 2020 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package errorstest
import (
"bytes"
"io/ioutil"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"unicode"
)
// A manually modified object file could pass unexpected characters
// into the files generated by cgo.
const magicInput = "abcdefghijklmnopqrstuvwxyz0123"
const magicReplace = "\n//go:cgo_ldflag \"-badflag\"\n//"
const cSymbol = "BadSymbol" + magicInput + "Name"
const cDefSource = "int " + cSymbol + " = 1;"
const cRefSource = "extern int " + cSymbol + "; int F() { return " + cSymbol + "; }"
// goSource is the source code for the trivial Go file we use.
// We will replace TMPDIR with the temporary directory name.
const goSource = `
package main
// #cgo LDFLAGS: TMPDIR/cbad.o TMPDIR/cbad.so
// extern int F();
import "C"
func main() {
println(C.F())
}
`
func TestBadSymbol(t *testing.T) {
dir := t.TempDir()
mkdir := func(base string) string {
ret := filepath.Join(dir, base)
if err := os.Mkdir(ret, 0755); err != nil {
t.Fatal(err)
}
return ret
}
cdir := mkdir("c")
godir := mkdir("go")
makeFile := func(mdir, base, source string) string {
ret := filepath.Join(mdir, base)
if err := ioutil.WriteFile(ret, []byte(source), 0644); err != nil {
t.Fatal(err)
}
return ret
}
cDefFile := makeFile(cdir, "cdef.c", cDefSource)
cRefFile := makeFile(cdir, "cref.c", cRefSource)
ccCmd := cCompilerCmd(t)
cCompile := func(arg, base, src string) string {
out := filepath.Join(cdir, base)
run := append(ccCmd, arg, "-o", out, src)
output, err := exec.Command(run[0], run[1:]...).CombinedOutput()
if err != nil {
t.Log(run)
t.Logf("%s", output)
t.Fatal(err)
}
if err := os.Remove(src); err != nil {
t.Fatal(err)
}
return out
}
// Build a shared library that defines a symbol whose name
// contains magicInput.
cShared := cCompile("-shared", "c.so", cDefFile)
// Build an object file that refers to the symbol whose name
// contains magicInput.
cObj := cCompile("-c", "c.o", cRefFile)
// Rewrite the shared library and the object file, replacing
// magicInput with magicReplace. This will have the effect of
// introducing a symbol whose name looks like a cgo command.
// The cgo tool will use that name when it generates the
// _cgo_import.go file, thus smuggling a magic //go:cgo_ldflag
// pragma into a Go file. We used to not check the pragmas in
// _cgo_import.go.
rewrite := func(from, to string) {
obj, err := ioutil.ReadFile(from)
if err != nil {
t.Fatal(err)
}
if bytes.Count(obj, []byte(magicInput)) == 0 {
t.Fatalf("%s: did not find magic string", from)
}
if len(magicInput) != len(magicReplace) {
t.Fatalf("internal test error: different magic lengths: %d != %d", len(magicInput), len(magicReplace))
}
obj = bytes.ReplaceAll(obj, []byte(magicInput), []byte(magicReplace))
if err := ioutil.WriteFile(to, obj, 0644); err != nil {
t.Fatal(err)
}
}
cBadShared := filepath.Join(godir, "cbad.so")
rewrite(cShared, cBadShared)
cBadObj := filepath.Join(godir, "cbad.o")
rewrite(cObj, cBadObj)
goSourceBadObject := strings.ReplaceAll(goSource, "TMPDIR", godir)
makeFile(godir, "go.go", goSourceBadObject)
makeFile(godir, "go.mod", "module badsym")
// Try to build our little package.
cmd := exec.Command("go", "build", "-ldflags=-v")
cmd.Dir = godir
output, err := cmd.CombinedOutput()
// The build should fail, but we want it to fail because we
// detected the error, not because we passed a bad flag to the
// C linker.
if err == nil {
t.Errorf("go build succeeded unexpectedly")
}
t.Logf("%s", output)
for _, line := range bytes.Split(output, []byte("\n")) {
if bytes.Contains(line, []byte("dynamic symbol")) && bytes.Contains(line, []byte("contains unsupported character")) {
// This is the error from cgo.
continue
}
// We passed -ldflags=-v to see the external linker invocation,
// which should not include -badflag.
if bytes.Contains(line, []byte("-badflag")) {
t.Error("output should not mention -badflag")
}
// Also check for compiler errors, just in case.
// GCC says "unrecognized command line option".
// clang says "unknown argument".
if bytes.Contains(line, []byte("unrecognized")) || bytes.Contains(output, []byte("unknown")) {
t.Error("problem should have been caught before invoking C linker")
}
}
}
func cCompilerCmd(t *testing.T) []string {
cc := []string{goEnv(t, "CC")}
out := goEnv(t, "GOGCCFLAGS")
quote := '\000'
start := 0
lastSpace := true
backslash := false
s := string(out)
for i, c := range s {
if quote == '\000' && unicode.IsSpace(c) {
if !lastSpace {
cc = append(cc, s[start:i])
lastSpace = true
}
} else {
if lastSpace {
start = i
lastSpace = false
}
if quote == '\000' && !backslash && (c == '"' || c == '\'') {
quote = c
backslash = false
} else if !backslash && quote == c {
quote = '\000'
} else if (quote == '\000' || quote == '"') && !backslash && c == '\\' {
backslash = true
} else {
backslash = false
}
}
}
if !lastSpace {
cc = append(cc, s[start:])
}
return cc
}
func goEnv(t *testing.T, key string) string {
out, err := exec.Command("go", "env", key).CombinedOutput()
if err != nil {
t.Logf("go env %s\n", key)
t.Logf("%s", out)
t.Fatal(err)
}
return strings.TrimSpace(string(out))
}

80
misc/cgo/test/issue1435.go
View file

@ -62,28 +62,60 @@ import "C"
// compareStatus is used to confirm the contents of the thread
// specific status files match expectations.
func compareStatus(filter, expect string) error {
expected := filter + "\t" + expect
expected := filter + expect
pid := syscall.Getpid()
fs, err := ioutil.ReadDir(fmt.Sprintf("/proc/%d/task", pid))
if err != nil {
return fmt.Errorf("unable to find %d tasks: %v", pid, err)
}
expectedProc := fmt.Sprintf("Pid:\t%d", pid)
foundAThread := false
for _, f := range fs {
tf := fmt.Sprintf("/proc/%s/status", f.Name())
d, err := ioutil.ReadFile(tf)
if err != nil {
return fmt.Errorf("unable to read %q: %v", tf, err)
// There are a surprising number of ways this
// can error out on linux. We've seen all of
// the following, so treat any error here as
// equivalent to the "process is gone":
// os.IsNotExist(err),
// "... : no such process",
// "... : bad file descriptor.
continue
}
lines := strings.Split(string(d), "\n")
for _, line := range lines {
// Different kernel vintages pad differently.
line = strings.TrimSpace(line)
if strings.HasPrefix(line, "Pid:\t") {
// On loaded systems, it is possible
// for a TID to be reused really
// quickly. As such, we need to
// validate that the thread status
// info we just read is a task of the
// same process PID as we are
// currently running, and not a
// recently terminated thread
// resurfaced in a different process.
if line != expectedProc {
break
}
// Fall through in the unlikely case
// that filter at some point is
// "Pid:\t".
}
if strings.HasPrefix(line, filter) {
if line != expected {
return fmt.Errorf("%s %s (bad)\n", tf, line)
return fmt.Errorf("%q got:%q want:%q (bad) [pid=%d file:'%s' %v]\n", tf, line, expected, pid, string(d), expectedProc)
}
foundAThread = true
break
}
}
}
if !foundAThread {
return fmt.Errorf("found no thread /proc/<TID>/status files for process %q", expectedProc)
}
return nil
}
@ -110,34 +142,34 @@ func test1435(t *testing.T) {
fn func() error
filter, expect string
}{
{call: "Setegid(1)", fn: func() error { return syscall.Setegid(1) }, filter: "Gid:", expect: "0\t1\t0\t1"},
{call: "Setegid(0)", fn: func() error { return syscall.Setegid(0) }, filter: "Gid:", expect: "0\t0\t0\t0"},
{call: "Setegid(1)", fn: func() error { return syscall.Setegid(1) }, filter: "Gid:", expect: "\t0\t1\t0\t1"},
{call: "Setegid(0)", fn: func() error { return syscall.Setegid(0) }, filter: "Gid:", expect: "\t0\t0\t0\t0"},
{call: "Seteuid(1)", fn: func() error { return syscall.Seteuid(1) }, filter: "Uid:", expect: "0\t1\t0\t1"},
{call: "Setuid(0)", fn: func() error { return syscall.Setuid(0) }, filter: "Uid:", expect: "0\t0\t0\t0"},
{call: "Seteuid(1)", fn: func() error { return syscall.Seteuid(1) }, filter: "Uid:", expect: "\t0\t1\t0\t1"},
{call: "Setuid(0)", fn: func() error { return syscall.Setuid(0) }, filter: "Uid:", expect: "\t0\t0\t0\t0"},
{call: "Setgid(1)", fn: func() error { return syscall.Setgid(1) }, filter: "Gid:", expect: "1\t1\t1\t1"},
{call: "Setgid(0)", fn: func() error { return syscall.Setgid(0) }, filter: "Gid:", expect: "0\t0\t0\t0"},
{call: "Setgid(1)", fn: func() error { return syscall.Setgid(1) }, filter: "Gid:", expect: "\t1\t1\t1\t1"},
{call: "Setgid(0)", fn: func() error { return syscall.Setgid(0) }, filter: "Gid:", expect: "\t0\t0\t0\t0"},
{call: "Setgroups([]int{0,1,2,3})", fn: func() error { return syscall.Setgroups([]int{0, 1, 2, 3}) }, filter: "Groups:", expect: "0 1 2 3 "},
{call: "Setgroups(nil)", fn: func() error { return syscall.Setgroups(nil) }, filter: "Groups:", expect: " "},
{call: "Setgroups([]int{0})", fn: func() error { return syscall.Setgroups([]int{0}) }, filter: "Groups:", expect: "0 "},
{call: "Setgroups([]int{0,1,2,3})", fn: func() error { return syscall.Setgroups([]int{0, 1, 2, 3}) }, filter: "Groups:", expect: "\t0 1 2 3"},
{call: "Setgroups(nil)", fn: func() error { return syscall.Setgroups(nil) }, filter: "Groups:", expect: ""},
{call: "Setgroups([]int{0})", fn: func() error { return syscall.Setgroups([]int{0}) }, filter: "Groups:", expect: "\t0"},
{call: "Setregid(101,0)", fn: func() error { return syscall.Setregid(101, 0) }, filter: "Gid:", expect: "101\t0\t0\t0"},
{call: "Setregid(0,102)", fn: func() error { return syscall.Setregid(0, 102) }, filter: "Gid:", expect: "0\t102\t102\t102"},
{call: "Setregid(0,0)", fn: func() error { return syscall.Setregid(0, 0) }, filter: "Gid:", expect: "0\t0\t0\t0"},
{call: "Setregid(101,0)", fn: func() error { return syscall.Setregid(101, 0) }, filter: "Gid:", expect: "\t101\t0\t0\t0"},
{call: "Setregid(0,102)", fn: func() error { return syscall.Setregid(0, 102) }, filter: "Gid:", expect: "\t0\t102\t102\t102"},
{call: "Setregid(0,0)", fn: func() error { return syscall.Setregid(0, 0) }, filter: "Gid:", expect: "\t0\t0\t0\t0"},
{call: "Setreuid(1,0)", fn: func() error { return syscall.Setreuid(1, 0) }, filter: "Uid:", expect: "1\t0\t0\t0"},
{call: "Setreuid(0,2)", fn: func() error { return syscall.Setreuid(0, 2) }, filter: "Uid:", expect: "0\t2\t2\t2"},
{call: "Setreuid(0,0)", fn: func() error { return syscall.Setreuid(0, 0) }, filter: "Uid:", expect: "0\t0\t0\t0"},
{call: "Setreuid(1,0)", fn: func() error { return syscall.Setreuid(1, 0) }, filter: "Uid:", expect: "\t1\t0\t0\t0"},
{call: "Setreuid(0,2)", fn: func() error { return syscall.Setreuid(0, 2) }, filter: "Uid:", expect: "\t0\t2\t2\t2"},
{call: "Setreuid(0,0)", fn: func() error { return syscall.Setreuid(0, 0) }, filter: "Uid:", expect: "\t0\t0\t0\t0"},
{call: "Setresgid(101,0,102)", fn: func() error { return syscall.Setresgid(101, 0, 102) }, filter: "Gid:", expect: "101\t0\t102\t0"},
{call: "Setresgid(0,102,101)", fn: func() error { return syscall.Setresgid(0, 102, 101) }, filter: "Gid:", expect: "0\t102\t101\t102"},
{call: "Setresgid(0,0,0)", fn: func() error { return syscall.Setresgid(0, 0, 0) }, filter: "Gid:", expect: "0\t0\t0\t0"},
{call: "Setresgid(101,0,102)", fn: func() error { return syscall.Setresgid(101, 0, 102) }, filter: "Gid:", expect: "\t101\t0\t102\t0"},
{call: "Setresgid(0,102,101)", fn: func() error { return syscall.Setresgid(0, 102, 101) }, filter: "Gid:", expect: "\t0\t102\t101\t102"},
{call: "Setresgid(0,0,0)", fn: func() error { return syscall.Setresgid(0, 0, 0) }, filter: "Gid:", expect: "\t0\t0\t0\t0"},
{call: "Setresuid(1,0,2)", fn: func() error { return syscall.Setresuid(1, 0, 2) }, filter: "Uid:", expect: "1\t0\t2\t0"},
{call: "Setresuid(0,2,1)", fn: func() error { return syscall.Setresuid(0, 2, 1) }, filter: "Uid:", expect: "0\t2\t1\t2"},
{call: "Setresuid(0,0,0)", fn: func() error { return syscall.Setresuid(0, 0, 0) }, filter: "Uid:", expect: "0\t0\t0\t0"},
{call: "Setresuid(1,0,2)", fn: func() error { return syscall.Setresuid(1, 0, 2) }, filter: "Uid:", expect: "\t1\t0\t2\t0"},
{call: "Setresuid(0,2,1)", fn: func() error { return syscall.Setresuid(0, 2, 1) }, filter: "Uid:", expect: "\t0\t2\t1\t2"},
{call: "Setresuid(0,0,0)", fn: func() error { return syscall.Setresuid(0, 0, 0) }, filter: "Uid:", expect: "\t0\t0\t0\t0"},
}
for i, v := range vs {

15
misc/cgo/test/issue42495.go Normal file
View file

@ -0,0 +1,15 @@
// Copyright 2020 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package cgotest
// typedef struct { } T42495A;
// typedef struct { int x[0]; } T42495B;
import "C"
//export Issue42495A
func Issue42495A(C.T42495A) {}
//export Issue42495B
func Issue42495B(C.T42495B) {}

14
misc/cgo/testplugin/plugin_test.go
View file

@ -196,3 +196,17 @@ func TestIssue25756(t *testing.T) {
})
}
}
func TestMethod(t *testing.T) {
// Exported symbol's method must be live.
goCmd(t, "build", "-buildmode=plugin", "-o", "plugin.so", "./method/plugin.go")
goCmd(t, "build", "-o", "method.exe", "./method/main.go")
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "./method.exe")
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("%s: %v\n%s", strings.Join(cmd.Args, " "), err, out)
}
}

26
misc/cgo/testplugin/testdata/method/main.go vendored Normal file
View file

@ -0,0 +1,26 @@
// Copyright 2020 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Issue 42579: methods of symbols exported from plugin must be live.
package main
import (
"plugin"
"reflect"
)
func main() {
p, err := plugin.Open("plugin.so")
if err != nil {
panic(err)
}
x, err := p.Lookup("X")
if err != nil {
panic(err)
}
reflect.ValueOf(x).Elem().MethodByName("M").Call(nil)
}

13
misc/cgo/testplugin/testdata/method/plugin.go vendored Normal file
View file

@ -0,0 +1,13 @@
// Copyright 2020 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
func main() {}
type T int
func (T) M() { println("M") }
var X T

2
src/archive/zip/reader.go
View file

@ -695,7 +695,7 @@ func fileEntryLess(x, y string) bool {
}
// Open opens the named file in the ZIP archive,
// using the semantics of io.FS.Open:
// using the semantics of fs.FS.Open:
// paths are always slash separated, with no
// leading / or ../ elements.
func (r *Reader) Open(name string) (fs.File, error) {

7
src/bytes/example_test.go
View file

@ -30,6 +30,13 @@ func ExampleBuffer_reader() {
// Output: Gophers rule!
}
func ExampleBuffer_Bytes() {
buf := bytes.Buffer{}
buf.Write([]byte{'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd'})
os.Stdout.Write(buf.Bytes())
// Output: hello world
}
func ExampleBuffer_Grow() {
var b bytes.Buffer
b.Grow(64)

64
src/cmd/asm/internal/asm/testdata/arm64.s vendored
View file

@ -681,38 +681,38 @@ again:
LDADDLH R5, (RSP), R7 // e7036578
LDADDLB R5, (R6), R7 // c7006538
LDADDLB R5, (RSP), R7 // e7036538
LDANDAD R5, (R6), R7 // c710a5f8
LDANDAD R5, (RSP), R7 // e713a5f8
LDANDAW R5, (R6), R7 // c710a5b8
LDANDAW R5, (RSP), R7 // e713a5b8
LDANDAH R5, (R6), R7 // c710a578
LDANDAH R5, (RSP), R7 // e713a578
LDANDAB R5, (R6), R7 // c710a538
LDANDAB R5, (RSP), R7 // e713a538
LDANDALD R5, (R6), R7 // c710e5f8
LDANDALD R5, (RSP), R7 // e713e5f8
LDANDALW R5, (R6), R7 // c710e5b8
LDANDALW R5, (RSP), R7 // e713e5b8
LDANDALH R5, (R6), R7 // c710e578
LDANDALH R5, (RSP), R7 // e713e578
LDANDALB R5, (R6), R7 // c710e538
LDANDALB R5, (RSP), R7 // e713e538
LDANDD R5, (R6), R7 // c71025f8
LDANDD R5, (RSP), R7 // e71325f8
LDANDW R5, (R6), R7 // c71025b8
LDANDW R5, (RSP), R7 // e71325b8
LDANDH R5, (R6), R7 // c7102578
LDANDH R5, (RSP), R7 // e7132578
LDANDB R5, (R6), R7 // c7102538
LDANDB R5, (RSP), R7 // e7132538
LDANDLD R5, (R6), R7 // c71065f8
LDANDLD R5, (RSP), R7 // e71365f8
LDANDLW R5, (R6), R7 // c71065b8
LDANDLW R5, (RSP), R7 // e71365b8
LDANDLH R5, (R6), R7 // c7106578
LDANDLH R5, (RSP), R7 // e7136578
LDANDLB R5, (R6), R7 // c7106538
LDANDLB R5, (RSP), R7 // e7136538
LDCLRAD R5, (R6), R7 // c710a5f8
LDCLRAD R5, (RSP), R7 // e713a5f8
LDCLRAW R5, (R6), R7 // c710a5b8
LDCLRAW R5, (RSP), R7 // e713a5b8
LDCLRAH R5, (R6), R7 // c710a578
LDCLRAH R5, (RSP), R7 // e713a578
LDCLRAB R5, (R6), R7 // c710a538
LDCLRAB R5, (RSP), R7 // e713a538
LDCLRALD R5, (R6), R7 // c710e5f8
LDCLRALD R5, (RSP), R7 // e713e5f8
LDCLRALW R5, (R6), R7 // c710e5b8
LDCLRALW R5, (RSP), R7 // e713e5b8
LDCLRALH R5, (R6), R7 // c710e578
LDCLRALH R5, (RSP), R7 // e713e578
LDCLRALB R5, (R6), R7 // c710e538
LDCLRALB R5, (RSP), R7 // e713e538
LDCLRD R5, (R6), R7 // c71025f8
LDCLRD R5, (RSP), R7 // e71325f8
LDCLRW R5, (R6), R7 // c71025b8
LDCLRW R5, (RSP), R7 // e71325b8
LDCLRH R5, (R6), R7 // c7102578
LDCLRH R5, (RSP), R7 // e7132578
LDCLRB R5, (R6), R7 // c7102538
LDCLRB R5, (RSP), R7 // e7132538
LDCLRLD R5, (R6), R7 // c71065f8
LDCLRLD R5, (RSP), R7 // e71365f8
LDCLRLW R5, (R6), R7 // c71065b8
LDCLRLW R5, (RSP), R7 // e71365b8
LDCLRLH R5, (R6), R7 // c7106578
LDCLRLH R5, (RSP), R7 // e7136578
LDCLRLB R5, (R6), R7 // c7106538
LDCLRLB R5, (RSP), R7 // e7136538
LDEORAD R5, (R6), R7 // c720a5f8
LDEORAD R5, (RSP), R7 // e723a5f8
LDEORAW R5, (R6), R7 // c720a5b8

48
src/cmd/asm/internal/asm/testdata/arm64error.s vendored
View file

@ -123,14 +123,14 @@ TEXT errors(SB),$0
LDADDLW R5, (R6), ZR // ERROR "illegal destination register"
LDADDLH R5, (R6), ZR // ERROR "illegal destination register"
LDADDLB R5, (R6), ZR // ERROR "illegal destination register"
LDANDD R5, (R6), ZR // ERROR "illegal destination register"
LDANDW R5, (R6), ZR // ERROR "illegal destination register"
LDANDH R5, (R6), ZR // ERROR "illegal destination register"
LDANDB R5, (R6), ZR // ERROR "illegal destination register"
LDANDLD R5, (R6), ZR // ERROR "illegal destination register"
LDANDLW R5, (R6), ZR // ERROR "illegal destination register"
LDANDLH R5, (R6), ZR // ERROR "illegal destination register"
LDANDLB R5, (R6), ZR // ERROR "illegal destination register"
LDCLRD R5, (R6), ZR // ERROR "illegal destination register"
LDCLRW R5, (R6), ZR // ERROR "illegal destination register"
LDCLRH R5, (R6), ZR // ERROR "illegal destination register"
LDCLRB R5, (R6), ZR // ERROR "illegal destination register"
LDCLRLD R5, (R6), ZR // ERROR "illegal destination register"
LDCLRLW R5, (R6), ZR // ERROR "illegal destination register"
LDCLRLH R5, (R6), ZR // ERROR "illegal destination register"
LDCLRLB R5, (R6), ZR // ERROR "illegal destination register"
LDEORD R5, (R6), ZR // ERROR "illegal destination register"
LDEORW R5, (R6), ZR // ERROR "illegal destination register"
LDEORH R5, (R6), ZR // ERROR "illegal destination register"
@ -163,22 +163,22 @@ TEXT errors(SB),$0
LDADDLW R5, (R6), RSP // ERROR "illegal destination register"
LDADDLH R5, (R6), RSP // ERROR "illegal destination register"
LDADDLB R5, (R6), RSP // ERROR "illegal destination register"
LDANDAD R5, (R6), RSP // ERROR "illegal destination register"
LDANDAW R5, (R6), RSP // ERROR "illegal destination register"
LDANDAH R5, (R6), RSP // ERROR "illegal destination register"
LDANDAB R5, (R6), RSP // ERROR "illegal destination register"
LDANDALD R5, (R6), RSP // ERROR "illegal destination register"
LDANDALW R5, (R6), RSP // ERROR "illegal destination register"
LDANDALH R5, (R6), RSP // ERROR "illegal destination register"
LDANDALB R5, (R6), RSP // ERROR "illegal destination register"
LDANDD R5, (R6), RSP // ERROR "illegal destination register"
LDANDW R5, (R6), RSP // ERROR "illegal destination register"
LDANDH R5, (R6), RSP // ERROR "illegal destination register"
LDANDB R5, (R6), RSP // ERROR "illegal destination register"
LDANDLD R5, (R6), RSP // ERROR "illegal destination register"
LDANDLW R5, (R6), RSP // ERROR "illegal destination register"
LDANDLH R5, (R6), RSP // ERROR "illegal destination register"
LDANDLB R5, (R6), RSP // ERROR "illegal destination register"
LDCLRAD R5, (R6), RSP // ERROR "illegal destination register"
LDCLRAW R5, (R6), RSP // ERROR "illegal destination register"
LDCLRAH R5, (R6), RSP // ERROR "illegal destination register"
LDCLRAB R5, (R6), RSP // ERROR "illegal destination register"
LDCLRALD R5, (R6), RSP // ERROR "illegal destination register"
LDCLRALW R5, (R6), RSP // ERROR "illegal destination register"
LDCLRALH R5, (R6), RSP // ERROR "illegal destination register"
LDCLRALB R5, (R6), RSP // ERROR "illegal destination register"
LDCLRD R5, (R6), RSP // ERROR "illegal destination register"
LDCLRW R5, (R6), RSP // ERROR "illegal destination register"
LDCLRH R5, (R6), RSP // ERROR "illegal destination register"
LDCLRB R5, (R6), RSP // ERROR "illegal destination register"
LDCLRLD R5, (R6), RSP // ERROR "illegal destination register"
LDCLRLW R5, (R6), RSP // ERROR "illegal destination register"
LDCLRLH R5, (R6), RSP // ERROR "illegal destination register"
LDCLRLB R5, (R6), RSP // ERROR "illegal destination register"
LDEORAD R5, (R6), RSP // ERROR "illegal destination register"
LDEORAW R5, (R6), RSP // ERROR "illegal destination register"
LDEORAH R5, (R6), RSP // ERROR "illegal destination register"

2
src/cmd/asm/internal/asm/testdata/ppc64.s vendored
View file

@ -282,7 +282,9 @@ TEXT asmtest(SB),DUPOK|NOSPLIT,$0
RLWMI $7, R3, $65535, R6 // 50663c3e
RLWMICC $7, R3, $65535, R6 // 50663c3f
RLWNM $3, R4, $7, R6 // 54861f7e
RLWNM R3, R4, $7, R6 // 5c861f7e
RLWNMCC $3, R4, $7, R6 // 54861f7f
RLWNMCC R3, R4, $7, R6 // 5c861f7f
RLDMI $0, R4, $7, R6 // 7886076c
RLDMICC $0, R4, $7, R6 // 7886076d
RLDIMI $0, R4, $7, R6 // 788601cc

2
src/cmd/asm/internal/asm/testdata/s390x.s vendored
View file

@ -412,6 +412,8 @@ TEXT main·foo(SB),DUPOK|NOSPLIT,$16-0 // TEXT main.foo(SB), DUPOK|NOSPLIT, $16-
UNDEF // 00000000
NOPH // 0700
SYNC // 07e0
// vector add and sub instructions
VAB V3, V4, V4 // e743400000f3
VAH V3, V4, V4 // e743400010f3

33
src/cmd/cgo/out.go
View file

@ -337,6 +337,8 @@ func dynimport(obj string) {
if s.Version != "" {
targ += "#" + s.Version
}
checkImportSymName(s.Name)
checkImportSymName(targ)
fmt.Fprintf(stdout, "//go:cgo_import_dynamic %s %s %q\n", s.Name, targ, s.Library)
}
lib, _ := f.ImportedLibraries()
@ -352,6 +354,7 @@ func dynimport(obj string) {
if len(s) > 0 && s[0] == '_' {
s = s[1:]
}
checkImportSymName(s)
fmt.Fprintf(stdout, "//go:cgo_import_dynamic %s %s %q\n", s, s, "")
}
lib, _ := f.ImportedLibraries()
@ -366,6 +369,8 @@ func dynimport(obj string) {
for _, s := range sym {
ss := strings.Split(s, ":")
name := strings.Split(ss[0], "@")[0]
checkImportSymName(name)
checkImportSymName(ss[0])
fmt.Fprintf(stdout, "//go:cgo_import_dynamic %s %s %q\n", name, ss[0], strings.ToLower(ss[1]))
}
return
@ -383,6 +388,7 @@ func dynimport(obj string) {
// Go symbols.
continue
}
checkImportSymName(s.Name)
fmt.Fprintf(stdout, "//go:cgo_import_dynamic %s %s %q\n", s.Name, s.Name, s.Library)
}
lib, err := f.ImportedLibraries()
@ -398,6 +404,23 @@ func dynimport(obj string) {
fatalf("cannot parse %s as ELF, Mach-O, PE or XCOFF", obj)
}
// checkImportSymName checks a symbol name we are going to emit as part
// of a //go:cgo_import_dynamic pragma. These names come from object
// files, so they may be corrupt. We are going to emit them unquoted,
// so while they don't need to be valid symbol names (and in some cases,
// involving symbol versions, they won't be) they must contain only
// graphic characters and must not contain Go comments.
func checkImportSymName(s string) {
for _, c := range s {
if !unicode.IsGraphic(c) || unicode.IsSpace(c) {
fatalf("dynamic symbol %q contains unsupported character", s)
}
}
if strings.Index(s, "//") >= 0 || strings.Index(s, "/*") >= 0 {
fatalf("dynamic symbol %q contains Go comment")
}
}
// Construct a gcc struct matching the gc argument frame.
// Assumes that in gcc, char is 1 byte, short 2 bytes, int 4 bytes, long long 8 bytes.
// These assumptions are checked by the gccProlog.
@ -962,7 +985,15 @@ func (p *Package) writeExports(fgo2, fm, fgcc, fgcch io.Writer) {
// The results part of the argument structure must be
// initialized to 0 so the write barriers generated by
// the assignments to these fields in Go are safe.
fmt.Fprintf(fgcc, "\t%s %v _cgo_a = {0};\n", ctype, p.packedAttribute())
//
// We use a local static variable to get the zeroed
// value of the argument type. This avoids including
// string.h for memset, and is also robust to C++
// types with constructors. Both GCC and LLVM optimize
// this into just zeroing _cgo_a.
fmt.Fprintf(fgcc, "\ttypedef %s %v _cgo_argtype;\n", ctype, p.packedAttribute())
fmt.Fprintf(fgcc, "\tstatic _cgo_argtype _cgo_zero;\n")
fmt.Fprintf(fgcc, "\t_cgo_argtype _cgo_a = _cgo_zero;\n")
if gccResult != "void" && (len(fntype.Results.List) > 1 || len(fntype.Results.List[0].Names) > 1) {
fmt.Fprintf(fgcc, "\t%s r;\n", gccResult)
}

4
src/cmd/compile/internal/amd64/ssa.go
View file

@ -76,7 +76,7 @@ func storeByType(t *types.Type) obj.As {
return x86.AMOVQ
}
}
panic("bad store type")
panic(fmt.Sprintf("bad store type %v", t))
}
// moveByType returns the reg->reg move instruction of the given type.
@ -101,7 +101,7 @@ func moveByType(t *types.Type) obj.As {
case 16:
return x86.AMOVUPS // int128s are in SSE registers
default:
panic(fmt.Sprintf("bad int register width %d:%s", t.Size(), t))
panic(fmt.Sprintf("bad int register width %d:%v", t.Size(), t))
}
}
}

125
src/cmd/compile/internal/arm64/ssa.go
View file

@ -581,6 +581,24 @@ func ssaGenValue(s *gc.SSAGenState, v *ssa.Value) {
p2.From.Reg = arm64.REGTMP
p2.To.Type = obj.TYPE_BRANCH
gc.Patch(p2, p)
case ssa.OpARM64LoweredAtomicExchange64Variant,
ssa.OpARM64LoweredAtomicExchange32Variant:
swap := arm64.ASWPALD
if v.Op == ssa.OpARM64LoweredAtomicExchange32Variant {
swap = arm64.ASWPALW
}
r0 := v.Args[0].Reg()
r1 := v.Args[1].Reg()
out := v.Reg0()
// SWPALD Rarg1, (Rarg0), Rout
p := s.Prog(swap)
p.From.Type = obj.TYPE_REG
p.From.Reg = r1
p.To.Type = obj.TYPE_MEM
p.To.Reg = r0
p.RegTo2 = out
case ssa.OpARM64LoweredAtomicAdd64,
ssa.OpARM64LoweredAtomicAdd32:
// LDAXR (Rarg0), Rout
@ -687,6 +705,56 @@ func ssaGenValue(s *gc.SSAGenState, v *ssa.Value) {
p5.To.Type = obj.TYPE_REG
p5.To.Reg = out
gc.Patch(p2, p5)
case ssa.OpARM64LoweredAtomicCas64Variant,
ssa.OpARM64LoweredAtomicCas32Variant:
// Rarg0: ptr
// Rarg1: old
// Rarg2: new
// MOV Rarg1, Rtmp
// CASAL Rtmp, (Rarg0), Rarg2
// CMP Rarg1, Rtmp
// CSET EQ, Rout
cas := arm64.ACASALD
cmp := arm64.ACMP
mov := arm64.AMOVD
if v.Op == ssa.OpARM64LoweredAtomicCas32Variant {
cas = arm64.ACASALW
cmp = arm64.ACMPW
mov = arm64.AMOVW
}
r0 := v.Args[0].Reg()
r1 := v.Args[1].Reg()
r2 := v.Args[2].Reg()
out := v.Reg0()
// MOV Rarg1, Rtmp
p := s.Prog(mov)
p.From.Type = obj.TYPE_REG
p.From.Reg = r1
p.To.Type = obj.TYPE_REG
p.To.Reg = arm64.REGTMP
// CASAL Rtmp, (Rarg0), Rarg2
p1 := s.Prog(cas)
p1.From.Type = obj.TYPE_REG
p1.From.Reg = arm64.REGTMP
p1.To.Type = obj.TYPE_MEM
p1.To.Reg = r0
p1.RegTo2 = r2
// CMP Rarg1, Rtmp
p2 := s.Prog(cmp)
p2.From.Type = obj.TYPE_REG
p2.From.Reg = r1
p2.Reg = arm64.REGTMP
// CSET EQ, Rout
p3 := s.Prog(arm64.ACSET)
p3.From.Type = obj.TYPE_REG
p3.From.Reg = arm64.COND_EQ
p3.To.Type = obj.TYPE_REG
p3.To.Reg = out
case ssa.OpARM64LoweredAtomicAnd8,
ssa.OpARM64LoweredAtomicAnd32,
ssa.OpARM64LoweredAtomicOr8,
@ -725,6 +793,63 @@ func ssaGenValue(s *gc.SSAGenState, v *ssa.Value) {
p3.From.Reg = arm64.REGTMP
p3.To.Type = obj.TYPE_BRANCH
gc.Patch(p3, p)
case ssa.OpARM64LoweredAtomicAnd8Variant,
ssa.OpARM64LoweredAtomicAnd32Variant:
atomic_clear := arm64.ALDCLRALW
if v.Op == ssa.OpARM64LoweredAtomicAnd8Variant {
atomic_clear = arm64.ALDCLRALB
}
r0 := v.Args[0].Reg()
r1 := v.Args[1].Reg()
out := v.Reg0()
// MNV Rarg1 Rtemp
p := s.Prog(arm64.AMVN)
p.From.Type = obj.TYPE_REG
p.From.Reg = r1
p.To.Type = obj.TYPE_REG
p.To.Reg = arm64.REGTMP
// LDCLRALW Rtemp, (Rarg0), Rout
p1 := s.Prog(atomic_clear)
p1.From.Type = obj.TYPE_REG
p1.From.Reg = arm64.REGTMP
p1.To.Type = obj.TYPE_MEM
p1.To.Reg = r0
p1.RegTo2 = out
// AND Rarg1, Rout
p2 := s.Prog(arm64.AAND)
p2.From.Type = obj.TYPE_REG
p2.From.Reg = r1
p2.To.Type = obj.TYPE_REG
p2.To.Reg = out
case ssa.OpARM64LoweredAtomicOr8Variant,
ssa.OpARM64LoweredAtomicOr32Variant:
atomic_or := arm64.ALDORALW
if v.Op == ssa.OpARM64LoweredAtomicOr8Variant {
atomic_or = arm64.ALDORALB
}
r0 := v.Args[0].Reg()
r1 := v.Args[1].Reg()
out := v.Reg0()
// LDORALW Rarg1, (Rarg0), Rout
p := s.Prog(atomic_or)
p.From.Type = obj.TYPE_REG
p.From.Reg = r1
p.To.Type = obj.TYPE_MEM
p.To.Reg = r0
p.RegTo2 = out
// ORR Rarg1, Rout
p2 := s.Prog(arm64.AORR)
p2.From.Type = obj.TYPE_REG
p2.From.Reg = r1
p2.To.Type = obj.TYPE_REG
p2.To.Reg = out
case ssa.OpARM64MOVBreg,
ssa.OpARM64MOVBUreg,
ssa.OpARM64MOVHreg,

4
src/cmd/compile/internal/gc/closure.go
View file

@ -71,6 +71,10 @@ func (p *noder) funcLit(expr *syntax.FuncLit) *Node {
return clo
}
// typecheckclosure typechecks an OCLOSURE node. It also creates the named
// function associated with the closure.
// TODO: This creation of the named function should probably really be done in a
// separate pass from type-checking.
func typecheckclosure(clo *Node, top int) {
xfunc := clo.Func.Closure
// Set current associated iota value, so iota can be used inside

2
src/cmd/compile/internal/gc/dcl.go
View file

@ -257,6 +257,8 @@ func symfield(s *types.Sym, typ *types.Type) *Node {
// oldname returns the Node that declares symbol s in the current scope.
// If no such Node currently exists, an ONONAME Node is returned instead.
// Automatically creates a new closure variable if the referenced symbol was
// declared in a different (containing) function.
func oldname(s *types.Sym) *Node {
n := asNode(s.Def)
if n == nil {

92
src/cmd/compile/internal/gc/dwinl.go
View file

@ -8,6 +8,7 @@ import (
"cmd/internal/dwarf"
"cmd/internal/obj"
"cmd/internal/src"
"fmt"
"strings"
)
@ -170,12 +171,32 @@ func assembleInlines(fnsym *obj.LSym, dwVars []*dwarf.Var) dwarf.InlCalls {
addRange(inlcalls.Calls, start, fnsym.Size, curii, imap)
}
// Issue 33188: if II foo is a child of II bar, then ensure that
// bar's ranges include the ranges of foo (the loop above will produce
// disjoint ranges).
for k, c := range inlcalls.Calls {
if c.Root {
unifyCallRanges(inlcalls, k)
}
}
// Debugging
if Debug_gendwarfinl != 0 {
dumpInlCalls(inlcalls)
dumpInlVars(dwVars)
}
// Perform a consistency check on inlined routine PC ranges
// produced by unifyCallRanges above. In particular, complain in
// cases where you have A -> B -> C (e.g. C is inlined into B, and
// B is inlined into A) and the ranges for B are not enclosed
// within the ranges for A, or C within B.
for k, c := range inlcalls.Calls {
if c.Root {
checkInlCall(fnsym.Name, inlcalls, fnsym.Size, k, -1)
}
}
return inlcalls
}
@ -355,3 +376,74 @@ func dumpInlVars(dwvars []*dwarf.Var) {
Ctxt.Logf("V%d: %s CI:%d II:%d IA:%d %s\n", i, dwv.Name, dwv.ChildIndex, dwv.InlIndex-1, ia, typ)
}
}
func rangesContains(par []dwarf.Range, rng dwarf.Range) (bool, string) {
for _, r := range par {
if rng.Start >= r.Start && rng.End <= r.End {
return true, ""
}
}
msg := fmt.Sprintf("range [%d,%d) not contained in {", rng.Start, rng.End)
for _, r := range par {
msg += fmt.Sprintf(" [%d,%d)", r.Start, r.End)
}
msg += " }"
return false, msg
}
func rangesContainsAll(parent, child []dwarf.Range) (bool, string) {
for _, r := range child {
c, m := rangesContains(parent, r)
if !c {
return false, m
}
}
return true, ""
}
// checkInlCall verifies that the PC ranges for inline info 'idx' are
// enclosed/contained within the ranges of its parent inline (or if
// this is a root/toplevel inline, checks that the ranges fall within
// the extent of the top level function). A panic is issued if a
// malformed range is found.
func checkInlCall(funcName string, inlCalls dwarf.InlCalls, funcSize int64, idx, parentIdx int) {
// Callee
ic := inlCalls.Calls[idx]
callee := Ctxt.InlTree.InlinedFunction(ic.InlIndex).Name
calleeRanges := ic.Ranges
// Caller
caller := funcName
parentRanges := []dwarf.Range{dwarf.Range{Start: int64(0), End: funcSize}}
if parentIdx != -1 {
pic := inlCalls.Calls[parentIdx]
caller = Ctxt.InlTree.InlinedFunction(pic.InlIndex).Name
parentRanges = pic.Ranges
}
// Callee ranges contained in caller ranges?
c, m := rangesContainsAll(parentRanges, calleeRanges)
if !c {
Fatalf("** malformed inlined routine range in %s: caller %s callee %s II=%d %s\n", funcName, caller, callee, idx, m)
}
// Now visit kids
for _, k := range ic.Children {
checkInlCall(funcName, inlCalls, funcSize, k, idx)
}
}
// unifyCallRanges ensures that the ranges for a given inline
// transitively include all of the ranges for its child inlines.
func unifyCallRanges(inlcalls dwarf.InlCalls, idx int) {
ic := &inlcalls.Calls[idx]
for _, childIdx := range ic.Children {
// First make sure child ranges are unified.
unifyCallRanges(inlcalls, childIdx)
// Then merge child ranges into ranges for this inline.
cic := inlcalls.Calls[childIdx]
ic.Ranges = dwarf.MergeRanges(ic.Ranges, cic.Ranges)
}
}

8
src/cmd/compile/internal/gc/fmt.go
View file

@ -419,13 +419,15 @@ func (n *Node) format(s fmt.State, verb rune, mode fmtMode) {
func (n *Node) jconv(s fmt.State, flag FmtFlag) {
c := flag & FmtShort
// Useful to see which nodes in an AST printout are actually identical
fmt.Fprintf(s, " p(%p)", n)
// Useful to see which nodes in a Node Dump/dumplist are actually identical
if Debug_dumpptrs != 0 {
fmt.Fprintf(s, " p(%p)", n)
}
if c == 0 && n.Name != nil && n.Name.Vargen != 0 {
fmt.Fprintf(s, " g(%d)", n.Name.Vargen)
}
if c == 0 && n.Name != nil && n.Name.Defn != nil {
if Debug_dumpptrs != 0 && c == 0 && n.Name != nil && n.Name.Defn != nil {
// Useful to see where Defn is set and what node it points to
fmt.Fprintf(s, " defn(%p)", n.Name.Defn)
}

6
src/cmd/compile/internal/gc/gsubr.go
View file

@ -302,6 +302,12 @@ func ggloblnod(nam *Node) {
if nam.Name.LibfuzzerExtraCounter() {
s.Type = objabi.SLIBFUZZER_EXTRA_COUNTER
}
if nam.Sym.Linkname != "" {
// Make sure linkname'd symbol is non-package. When a symbol is
// both imported and linkname'd, s.Pkg may not set to "_" in
// types.Sym.Linksym because LSym already exists. Set it here.
s.Pkg = "_"
}
}
func ggloblsym(s *obj.LSym, width int32, flags int16) {

40
src/cmd/compile/internal/gc/iexport.go
View file

@ -1138,13 +1138,10 @@ func (w *exportWriter) stmt(n *Node) {
w.pos(n.Pos)
w.stmtList(n.Ninit)
w.exprsOrNil(n.Left, nil)
w.stmtList(n.List)
w.caseList(n)
case OCASE:
w.op(OCASE)
w.pos(n.Pos)
w.stmtList(n.List)
w.stmtList(n.Nbody)
// case OCASE:
// handled by caseList
case OFALL:
w.op(OFALL)
@ -1168,6 +1165,24 @@ func (w *exportWriter) stmt(n *Node) {
}
}
func (w *exportWriter) caseList(sw *Node) {
namedTypeSwitch := sw.Op == OSWITCH && sw.Left != nil && sw.Left.Op == OTYPESW && sw.Left.Left != nil
cases := sw.List.Slice()
w.uint64(uint64(len(cases)))
for _, cas := range cases {
if cas.Op != OCASE {
Fatalf("expected OCASE, got %v", cas)
}
w.pos(cas.Pos)
w.stmtList(cas.List)
if namedTypeSwitch {
w.localName(cas.Rlist.First())
}
w.stmtList(cas.Nbody)
}
}
func (w *exportWriter) exprList(list Nodes) {
for _, n := range list.Slice() {
w.expr(n)
@ -1232,6 +1247,19 @@ func (w *exportWriter) expr(n *Node) {
w.op(OTYPE)
w.typ(n.Type)
case OTYPESW:
w.op(OTYPESW)
w.pos(n.Pos)
var s *types.Sym
if n.Left != nil {
if n.Left.Op != ONONAME {
Fatalf("expected ONONAME, got %v", n.Left)
}
s = n.Left.Sym
}
w.localIdent(s, 0) // declared pseudo-variable, if any
w.exprsOrNil(n.Right, nil)
// case OTARRAY, OTMAP, OTCHAN, OTSTRUCT, OTINTER, OTFUNC:
// should have been resolved by typechecking - handled by default case

41
src/cmd/compile/internal/gc/iimport.go
View file

@ -784,6 +784,28 @@ func (r *importReader) stmtList() []*Node {
return list
}
func (r *importReader) caseList(sw *Node) []*Node {
namedTypeSwitch := sw.Op == OSWITCH && sw.Left != nil && sw.Left.Op == OTYPESW && sw.Left.Left != nil
cases := make([]*Node, r.uint64())
for i := range cases {
cas := nodl(r.pos(), OCASE, nil, nil)
cas.List.Set(r.stmtList())
if namedTypeSwitch {
// Note: per-case variables will have distinct, dotted
// names after import. That's okay: swt.go only needs
// Sym for diagnostics anyway.
caseVar := newnamel(cas.Pos, r.ident())
declare(caseVar, dclcontext)
cas.Rlist.Set1(caseVar)
caseVar.Name.Defn = sw.Left
}
cas.Nbody.Set(r.stmtList())
cases[i] = cas
}
return cases
}
func (r *importReader) exprList() []*Node {
var list []*Node
for {
@ -831,6 +853,14 @@ func (r *importReader) node() *Node {
case OTYPE:
return typenod(r.typ())
case OTYPESW:
n := nodl(r.pos(), OTYPESW, nil, nil)
if s := r.ident(); s != nil {
n.Left = npos(n.Pos, newnoname(s))
}
n.Right, _ = r.exprsOrNil()
return n
// case OTARRAY, OTMAP, OTCHAN, OTSTRUCT, OTINTER, OTFUNC:
// unreachable - should have been resolved by typechecking
@ -1025,16 +1055,11 @@ func (r *importReader) node() *Node {
n := nodl(r.pos(), op, nil, nil)
n.Ninit.Set(r.stmtList())
n.Left, _ = r.exprsOrNil()
n.List.Set(r.stmtList())
n.List.Set(r.caseList(n))
return n
case OCASE:
n := nodl(r.pos(), OCASE, nil, nil)
n.List.Set(r.exprList())
// TODO(gri) eventually we must declare variables for type switch
// statements (type switch statements are not yet exported)
n.Nbody.Set(r.stmtList())
return n
// case OCASE:
// handled by caseList
case OFALL:
n := nodl(r.pos(), OFALL, nil, nil)

42
src/cmd/compile/internal/gc/inl.go
View file

@ -94,10 +94,11 @@ func typecheckinl(fn *Node) {
typecheckslice(fn.Func.Inl.Body, ctxStmt)
Curfn = savefn
// During typechecking, declarations are added to
// Curfn.Func.Dcl. Move them to Inl.Dcl for consistency with
// how local functions behave. (Append because typecheckinl
// may be called multiple times.)
// During expandInline (which imports fn.Func.Inl.Body),
// declarations are added to fn.Func.Dcl by funcHdr(). Move them
// to fn.Func.Inl.Dcl for consistency with how local functions
// behave. (Append because typecheckinl may be called multiple
// times.)
fn.Func.Inl.Dcl = append(fn.Func.Inl.Dcl, fn.Func.Dcl...)
fn.Func.Dcl = nil
@ -392,13 +393,9 @@ func (v *hairyVisitor) visit(n *Node) bool {
v.reason = "call to recover"
return true
case OCALLPART:
// OCALLPART is inlineable, but no extra cost to the budget
case OCLOSURE,
ORANGE,
OSELECT,
OTYPESW,
OGO,
ODEFER,
ODCLTYPE, // can't print yet
@ -452,9 +449,9 @@ func (v *hairyVisitor) visit(n *Node) bool {
v.visitList(n.Ninit) || v.visitList(n.Nbody)
}
// Inlcopy and inlcopylist recursively copy the body of a function.
// Any name-like node of non-local class is marked for re-export by adding it to
// the exportlist.
// inlcopylist (together with inlcopy) recursively copies a list of nodes, except
// that it keeps the same ONAME, OTYPE, and OLITERAL nodes. It is used for copying
// the body and dcls of an inlineable function.
func inlcopylist(ll []*Node) []*Node {
s := make([]*Node, 0, len(ll))
for _, n := range ll {
@ -893,10 +890,10 @@ func inlParam(t *types.Field, as *Node, inlvars map[*Node]*Node) *Node {
var inlgen int
// If n is a call, and fn is a function with an inlinable body,
// return an OINLCALL.
// On return ninit has the parameter assignments, the nbody is the
// inlined function body and list, rlist contain the input, output
// If n is a call node (OCALLFUNC or OCALLMETH), and fn is an ONAME node for a
// function with an inlinable body, return an OINLCALL node that can replace n.
// The returned node's Ninit has the parameter assignments, the Nbody is the
// inlined function body, and (List, Rlist) contain the (input, output)
// parameters.
// The result of mkinlcall MUST be assigned back to n, e.g.
// n.Left = mkinlcall(n.Left, fn, isddd)
@ -966,6 +963,21 @@ func mkinlcall(n, fn *Node, maxCost int32, inlMap map[*Node]bool) *Node {
ninit := n.Ninit
// For normal function calls, the function callee expression
// may contain side effects (e.g., added by addinit during
// inlconv2expr or inlconv2list). Make sure to preserve these,
// if necessary (#42703).
if n.Op == OCALLFUNC {
callee := n.Left
for callee.Op == OCONVNOP {
ninit.AppendNodes(&callee.Ninit)
callee = callee.Left
}
if callee.Op != ONAME && callee.Op != OCLOSURE {
Fatalf("unexpected callee expression: %v", callee)
}
}
// Make temp names to use instead of the originals.
inlvars := make(map[*Node]*Node)

5
src/cmd/compile/internal/gc/main.go
View file

@ -46,6 +46,7 @@ var (
Debug_closure int
Debug_compilelater int
debug_dclstack int
Debug_dumpptrs int
Debug_libfuzzer int
Debug_panic int
Debug_slice int
@ -75,6 +76,7 @@ var debugtab = []struct {
{"compilelater", "compile functions as late as possible", &Debug_compilelater},
{"disablenil", "disable nil checks", &disable_checknil},
{"dclstack", "run internal dclstack check", &debug_dclstack},
{"dumpptrs", "show Node pointer values in Dump/dumplist output", &Debug_dumpptrs},
{"gcprog", "print dump of GC programs", &Debug_gcprog},
{"libfuzzer", "coverage instrumentation for libfuzzer", &Debug_libfuzzer},
{"nil", "print information about nil checks", &Debug_checknil},
@ -89,6 +91,7 @@ var debugtab = []struct {
{"dwarfinl", "print information about DWARF inlined function creation", &Debug_gendwarfinl},
{"softfloat", "force compiler to emit soft-float code", &Debug_softfloat},
{"defer", "print information about defer compilation", &Debug_defer},
{"fieldtrack", "enable fieldtracking", &objabi.Fieldtrack_enabled},
}
const debugHelpHeader = `usage: -d arg[,arg]* and arg is <key>[=<value>]
@ -130,7 +133,7 @@ func hidePanic() {
// supportsDynlink reports whether or not the code generator for the given
// architecture supports the -shared and -dynlink flags.
func supportsDynlink(arch *sys.Arch) bool {
return arch.InFamily(sys.AMD64, sys.ARM, sys.ARM64, sys.I386, sys.PPC64, sys.S390X)
return arch.InFamily(sys.AMD64, sys.ARM, sys.ARM64, sys.I386, sys.PPC64, sys.RISCV64, sys.S390X)
}
// timing data for compiler phases

6
src/cmd/compile/internal/gc/reflect.go
View file

@ -1591,8 +1591,12 @@ func dumptabs() {
// typ typeOff // pointer to symbol
// }
nsym := dname(p.s.Name, "", nil, true)
tsym := dtypesym(p.t)
ot = dsymptrOff(s, ot, nsym)
ot = dsymptrOff(s, ot, dtypesym(p.t))
ot = dsymptrOff(s, ot, tsym)
// Plugin exports symbols as interfaces. Mark their types
// as UsedInIface.
tsym.Set(obj.AttrUsedInIface, true)
}
ggloblsym(s, int32(ot), int16(obj.RODATA))

133
src/cmd/compile/internal/gc/ssa.go
View file

@ -3458,14 +3458,64 @@ func init() {
s.vars[&memVar] = s.newValue1(ssa.OpSelect1, types.TypeMem, v)
return s.newValue1(ssa.OpSelect0, types.Types[TUINT32], v)
},
sys.AMD64, sys.ARM64, sys.MIPS, sys.MIPS64, sys.PPC64, sys.RISCV64, sys.S390X)
sys.AMD64, sys.MIPS, sys.MIPS64, sys.PPC64, sys.RISCV64, sys.S390X)
addF("runtime/internal/atomic", "Xchg64",
func(s *state, n *Node, args []*ssa.Value) *ssa.Value {
v := s.newValue3(ssa.OpAtomicExchange64, types.NewTuple(types.Types[TUINT64], types.TypeMem), args[0], args[1], s.mem())
s.vars[&memVar] = s.newValue1(ssa.OpSelect1, types.TypeMem, v)
return s.newValue1(ssa.OpSelect0, types.Types[TUINT64], v)
},
sys.AMD64, sys.ARM64, sys.MIPS64, sys.PPC64, sys.RISCV64, sys.S390X)
sys.AMD64, sys.MIPS64, sys.PPC64, sys.RISCV64, sys.S390X)
type atomicOpEmitter func(s *state, n *Node, args []*ssa.Value, op ssa.Op, typ types.EType)
makeAtomicGuardedIntrinsicARM64 := func(op0, op1 ssa.Op, typ, rtyp types.EType, emit atomicOpEmitter) intrinsicBuilder {
return func(s *state, n *Node, args []*ssa.Value) *ssa.Value {
// Target Atomic feature is identified by dynamic detection
addr := s.entryNewValue1A(ssa.OpAddr, types.Types[TBOOL].PtrTo(), arm64HasATOMICS, s.sb)
v := s.load(types.Types[TBOOL], addr)
b := s.endBlock()
b.Kind = ssa.BlockIf
b.SetControl(v)
bTrue := s.f.NewBlock(ssa.BlockPlain)
bFalse := s.f.NewBlock(ssa.BlockPlain)
bEnd := s.f.NewBlock(ssa.BlockPlain)
b.AddEdgeTo(bTrue)
b.AddEdgeTo(bFalse)
b.Likely = ssa.BranchLikely
// We have atomic instructions - use it directly.
s.startBlock(bTrue)
emit(s, n, args, op1, typ)
s.endBlock().AddEdgeTo(bEnd)
// Use original instruction sequence.
s.startBlock(bFalse)
emit(s, n, args, op0, typ)
s.endBlock().AddEdgeTo(bEnd)
// Merge results.
s.startBlock(bEnd)
if rtyp == TNIL {
return nil
} else {
return s.variable(n, types.Types[rtyp])
}
}
}
atomicXchgXaddEmitterARM64 := func(s *state, n *Node, args []*ssa.Value, op ssa.Op, typ types.EType) {
v := s.newValue3(op, types.NewTuple(types.Types[typ], types.TypeMem), args[0], args[1], s.mem())
s.vars[&memVar] = s.newValue1(ssa.OpSelect1, types.TypeMem, v)
s.vars[n] = s.newValue1(ssa.OpSelect0, types.Types[typ], v)
}
addF("runtime/internal/atomic", "Xchg",
makeAtomicGuardedIntrinsicARM64(ssa.OpAtomicExchange32, ssa.OpAtomicExchange32Variant, TUINT32, TUINT32, atomicXchgXaddEmitterARM64),
sys.ARM64)
addF("runtime/internal/atomic", "Xchg64",
makeAtomicGuardedIntrinsicARM64(ssa.OpAtomicExchange64, ssa.OpAtomicExchange64Variant, TUINT64, TUINT64, atomicXchgXaddEmitterARM64),
sys.ARM64)
addF("runtime/internal/atomic", "Xadd",
func(s *state, n *Node, args []*ssa.Value) *ssa.Value {
@ -3482,46 +3532,11 @@ func init() {
},
sys.AMD64, sys.MIPS64, sys.PPC64, sys.RISCV64, sys.S390X)
makeXaddARM64 := func(op0 ssa.Op, op1 ssa.Op, ty types.EType) func(s *state, n *Node, args []*ssa.Value) *ssa.Value {
return func(s *state, n *Node, args []*ssa.Value) *ssa.Value {
// Target Atomic feature is identified by dynamic detection
addr := s.entryNewValue1A(ssa.OpAddr, types.Types[TBOOL].PtrTo(), arm64HasATOMICS, s.sb)
v := s.load(types.Types[TBOOL], addr)
b := s.endBlock()
b.Kind = ssa.BlockIf
b.SetControl(v)
bTrue := s.f.NewBlock(ssa.BlockPlain)
bFalse := s.f.NewBlock(ssa.BlockPlain)
bEnd := s.f.NewBlock(ssa.BlockPlain)
b.AddEdgeTo(bTrue)
b.AddEdgeTo(bFalse)
b.Likely = ssa.BranchUnlikely // most machines don't have Atomics nowadays
// We have atomic instructions - use it directly.
s.startBlock(bTrue)
v0 := s.newValue3(op1, types.NewTuple(types.Types[ty], types.TypeMem), args[0], args[1], s.mem())
s.vars[&memVar] = s.newValue1(ssa.OpSelect1, types.TypeMem, v0)
s.vars[n] = s.newValue1(ssa.OpSelect0, types.Types[ty], v0)
s.endBlock().AddEdgeTo(bEnd)
// Use original instruction sequence.
s.startBlock(bFalse)
v1 := s.newValue3(op0, types.NewTuple(types.Types[ty], types.TypeMem), args[0], args[1], s.mem())
s.vars[&memVar] = s.newValue1(ssa.OpSelect1, types.TypeMem, v1)
s.vars[n] = s.newValue1(ssa.OpSelect0, types.Types[ty], v1)
s.endBlock().AddEdgeTo(bEnd)
// Merge results.
s.startBlock(bEnd)
return s.variable(n, types.Types[ty])
}
}
addF("runtime/internal/atomic", "Xadd",
makeXaddARM64(ssa.OpAtomicAdd32, ssa.OpAtomicAdd32Variant, TUINT32),
makeAtomicGuardedIntrinsicARM64(ssa.OpAtomicAdd32, ssa.OpAtomicAdd32Variant, TUINT32, TUINT32, atomicXchgXaddEmitterARM64),
sys.ARM64)
addF("runtime/internal/atomic", "Xadd64",
makeXaddARM64(ssa.OpAtomicAdd64, ssa.OpAtomicAdd64Variant, TUINT64),
makeAtomicGuardedIntrinsicARM64(ssa.OpAtomicAdd64, ssa.OpAtomicAdd64Variant, TUINT64, TUINT64, atomicXchgXaddEmitterARM64),
sys.ARM64)
addF("runtime/internal/atomic", "Cas",
@ -3530,14 +3545,14 @@ func init() {
s.vars[&memVar] = s.newValue1(ssa.OpSelect1, types.TypeMem, v)
return s.newValue1(ssa.OpSelect0, types.Types[TBOOL], v)
},
sys.AMD64, sys.ARM64, sys.MIPS, sys.MIPS64, sys.PPC64, sys.RISCV64, sys.S390X)
sys.AMD64, sys.MIPS, sys.MIPS64, sys.PPC64, sys.RISCV64, sys.S390X)
addF("runtime/internal/atomic", "Cas64",
func(s *state, n *Node, args []*ssa.Value) *ssa.Value {
v := s.newValue4(ssa.OpAtomicCompareAndSwap64, types.NewTuple(types.Types[TBOOL], types.TypeMem), args[0], args[1], args[2], s.mem())
s.vars[&memVar] = s.newValue1(ssa.OpSelect1, types.TypeMem, v)
return s.newValue1(ssa.OpSelect0, types.Types[TBOOL], v)
},
sys.AMD64, sys.ARM64, sys.MIPS64, sys.PPC64, sys.RISCV64, sys.S390X)
sys.AMD64, sys.MIPS64, sys.PPC64, sys.RISCV64, sys.S390X)
addF("runtime/internal/atomic", "CasRel",
func(s *state, n *Node, args []*ssa.Value) *ssa.Value {
v := s.newValue4(ssa.OpAtomicCompareAndSwap32, types.NewTuple(types.Types[TBOOL], types.TypeMem), args[0], args[1], args[2], s.mem())
@ -3546,18 +3561,31 @@ func init() {
},
sys.PPC64)
atomicCasEmitterARM64 := func(s *state, n *Node, args []*ssa.Value, op ssa.Op, typ types.EType) {
v := s.newValue4(op, types.NewTuple(types.Types[TBOOL], types.TypeMem), args[0], args[1], args[2], s.mem())
s.vars[&memVar] = s.newValue1(ssa.OpSelect1, types.TypeMem, v)
s.vars[n] = s.newValue1(ssa.OpSelect0, types.Types[typ], v)
}
addF("runtime/internal/atomic", "Cas",
makeAtomicGuardedIntrinsicARM64(ssa.OpAtomicCompareAndSwap32, ssa.OpAtomicCompareAndSwap32Variant, TUINT32, TBOOL, atomicCasEmitterARM64),
sys.ARM64)
addF("runtime/internal/atomic", "Cas64",
makeAtomicGuardedIntrinsicARM64(ssa.OpAtomicCompareAndSwap64, ssa.OpAtomicCompareAndSwap64Variant, TUINT64, TBOOL, atomicCasEmitterARM64),
sys.ARM64)
addF("runtime/internal/atomic", "And8",
func(s *state, n *Node, args []*ssa.Value) *ssa.Value {
s.vars[&memVar] = s.newValue3(ssa.OpAtomicAnd8, types.TypeMem, args[0], args[1], s.mem())
return nil
},
sys.AMD64, sys.ARM64, sys.MIPS, sys.PPC64, sys.S390X)
sys.AMD64, sys.MIPS, sys.PPC64, sys.S390X)
addF("runtime/internal/atomic", "And",
func(s *state, n *Node, args []*ssa.Value) *ssa.Value {
s.vars[&memVar] = s.newValue3(ssa.OpAtomicAnd32, types.TypeMem, args[0], args[1], s.mem())
return nil
},
sys.AMD64, sys.ARM64, sys.MIPS, sys.PPC64, sys.S390X)
sys.AMD64, sys.MIPS, sys.PPC64, sys.S390X)
addF("runtime/internal/atomic", "Or8",
func(s *state, n *Node, args []*ssa.Value) *ssa.Value {
s.vars[&memVar] = s.newValue3(ssa.OpAtomicOr8, types.TypeMem, args[0], args[1], s.mem())
@ -3569,7 +3597,24 @@ func init() {
s.vars[&memVar] = s.newValue3(ssa.OpAtomicOr32, types.TypeMem, args[0], args[1], s.mem())
return nil
},
sys.AMD64, sys.ARM64, sys.MIPS, sys.PPC64, sys.S390X)
sys.AMD64, sys.MIPS, sys.PPC64, sys.S390X)
atomicAndOrEmitterARM64 := func(s *state, n *Node, args []*ssa.Value, op ssa.Op, typ types.EType) {
s.vars[&memVar] = s.newValue3(op, types.TypeMem, args[0], args[1], s.mem())
}
addF("runtime/internal/atomic", "And8",
makeAtomicGuardedIntrinsicARM64(ssa.OpAtomicAnd8, ssa.OpAtomicAnd8Variant, TNIL, TNIL, atomicAndOrEmitterARM64),
sys.ARM64)
addF("runtime/internal/atomic", "And",
makeAtomicGuardedIntrinsicARM64(ssa.OpAtomicAnd32, ssa.OpAtomicAnd32Variant, TNIL, TNIL, atomicAndOrEmitterARM64),
sys.ARM64)
addF("runtime/internal/atomic", "Or8",
makeAtomicGuardedIntrinsicARM64(ssa.OpAtomicOr8, ssa.OpAtomicOr8Variant, TNIL, TNIL, atomicAndOrEmitterARM64),
sys.ARM64)
addF("runtime/internal/atomic", "Or",
makeAtomicGuardedIntrinsicARM64(ssa.OpAtomicOr32, ssa.OpAtomicOr32Variant, TNIL, TNIL, atomicAndOrEmitterARM64),
sys.ARM64)
alias("runtime/internal/atomic", "Loadint64", "runtime/internal/atomic", "Load64", all...)
alias("runtime/internal/atomic", "Xaddint64", "runtime/internal/atomic", "Xadd64", all...)

68
src/cmd/compile/internal/gc/syntax.go
View file

@ -344,14 +344,22 @@ func (n *Node) CanBeAnSSASym() {
// Name holds Node fields used only by named nodes (ONAME, OTYPE, OPACK, OLABEL, some OLITERAL).
type Name struct {
Pack *Node // real package for import . names
Pkg *types.Pkg // pkg for OPACK nodes
Defn *Node // initializing assignment
Curfn *Node // function for local variables
Param *Param // additional fields for ONAME, OTYPE
Decldepth int32 // declaration loop depth, increased for every loop or label
Vargen int32 // unique name for ONAME within a function. Function outputs are numbered starting at one.
flags bitset16
Pack *Node // real package for import . names
Pkg *types.Pkg // pkg for OPACK nodes
// For a local variable (not param) or extern, the initializing assignment (OAS or OAS2).
// For a closure var, the ONAME node of the outer captured variable
Defn *Node
// The ODCLFUNC node (for a static function/method or a closure) in which
// local variable or param is declared.
Curfn *Node
Param *Param // additional fields for ONAME, OTYPE
Decldepth int32 // declaration loop depth, increased for every loop or label
// Unique number for ONAME nodes within a function. Function outputs
// (results) are numbered starting at one, followed by function inputs
// (parameters), and then local variables. Vargen is used to distinguish
// local variables/params with the same name.
Vargen int32
flags bitset16
}
const (
@ -608,10 +616,16 @@ func (p *Param) SetEmbedFiles(list []string) {
// Func holds Node fields used only with function-like nodes.
type Func struct {
Shortname *types.Sym
Enter Nodes // for example, allocate and initialize memory for escaping parameters
Exit Nodes
Cvars Nodes // closure params
Dcl []*Node // autodcl for this func/closure
// Extra entry code for the function. For example, allocate and initialize
// memory for escaping parameters. However, just for OCLOSURE, Enter is a
// list of ONAME nodes of captured variables
Enter Nodes
Exit Nodes
// ONAME nodes for closure params, each should have closurevar set
Cvars Nodes
// ONAME nodes for all params/locals for this func/closure, does NOT
// include closurevars until transformclosure runs.
Dcl []*Node
// Parents records the parent scope of each scope within a
// function. The root scope (0) has no parent, so the i'th
@ -630,7 +644,7 @@ type Func struct {
DebugInfo *ssa.FuncDebug
Ntype *Node // signature
Top int // top context (ctxCallee, etc)
Closure *Node // OCLOSURE <-> ODCLFUNC
Closure *Node // OCLOSURE <-> ODCLFUNC (see header comment above)
Nname *Node // The ONAME node associated with an ODCLFUNC (both have same Type)
lsym *obj.LSym
@ -680,6 +694,8 @@ const (
funcWrapper // is method wrapper
funcNeedctxt // function uses context register (has closure variables)
funcReflectMethod // function calls reflect.Type.Method or MethodByName
// true if closure inside a function; false if a simple function or a
// closure in a global variable initialization
funcIsHiddenClosure
funcHasDefer // contains a defer statement
funcNilCheckDisabled // disable nil checks when compiling this function
@ -731,8 +747,10 @@ const (
OXXX Op = iota
// names
ONAME // var or func name
ONONAME // unnamed arg or return value: f(int, string) (int, error) { etc }
ONAME // var or func name
// Unnamed arg or return value: f(int, string) (int, error) { etc }
// Also used for a qualified package identifier that hasn't been resolved yet.
ONONAME
OTYPE // type name
OPACK // import
OLITERAL // literal
@ -752,14 +770,18 @@ const (
OSTR2BYTES // Type(Left) (Type is []byte, Left is a string)
OSTR2BYTESTMP // Type(Left) (Type is []byte, Left is a string, ephemeral)
OSTR2RUNES // Type(Left) (Type is []rune, Left is a string)
OAS // Left = Right or (if Colas=true) Left := Right
OAS2 // List = Rlist (x, y, z = a, b, c)
OAS2DOTTYPE // List = Right (x, ok = I.(int))
OAS2FUNC // List = Right (x, y = f())
OAS2MAPR // List = Right (x, ok = m["foo"])
OAS2RECV // List = Right (x, ok = <-c)
OASOP // Left Etype= Right (x += y)
OCALL // Left(List) (function call, method call or type conversion)
// Left = Right or (if Colas=true) Left := Right
// If Colas, then Ninit includes a DCL node for Left.
OAS
// List = Rlist (x, y, z = a, b, c) or (if Colas=true) List := Rlist
// If Colas, then Ninit includes DCL nodes for List
OAS2
OAS2DOTTYPE // List = Right (x, ok = I.(int))
OAS2FUNC // List = Right (x, y = f())
OAS2MAPR // List = Right (x, ok = m["foo"])
OAS2RECV // List = Right (x, ok = <-c)
OASOP // Left Etype= Right (x += y)
OCALL // Left(List) (function call, method call or type conversion)
// OCALLFUNC, OCALLMETH, and OCALLINTER have the same structure.
// Prior to walk, they are: Left(List), where List is all regular arguments.

20
src/cmd/compile/internal/gc/typecheck.go
View file

@ -6,7 +6,6 @@ package gc
import (
"cmd/compile/internal/types"
"cmd/internal/objabi"
"fmt"
"strings"
)
@ -2065,11 +2064,6 @@ func typecheck1(n *Node, top int) (res *Node) {
n.Type = nil
return n
case OCASE:
ok |= ctxStmt
typecheckslice(n.List.Slice(), ctxExpr)
typecheckslice(n.Nbody.Slice(), ctxStmt)
case ODCLFUNC:
ok |= ctxStmt
typecheckfunc(n)
@ -2447,15 +2441,6 @@ func derefall(t *types.Type) *types.Type {
return t
}
type typeSymKey struct {
t *types.Type
s *types.Sym
}
// dotField maps (*types.Type, *types.Sym) pairs to the corresponding struct field (*types.Type with Etype==TFIELD).
// It is a cache for use during usefield in walk.go, only enabled when field tracking.
var dotField = map[typeSymKey]*types.Field{}
func lookdot(n *Node, t *types.Type, dostrcmp int) *types.Field {
s := n.Sym
@ -2486,9 +2471,6 @@ func lookdot(n *Node, t *types.Type, dostrcmp int) *types.Field {
}
n.Xoffset = f1.Offset
n.Type = f1.Type
if objabi.Fieldtrack_enabled > 0 {
dotField[typeSymKey{t.Orig, s}] = f1
}
if t.IsInterface() {
if n.Left.Type.IsPtr() {
n.Left = nod(ODEREF, n.Left, nil) // implicitstar
@ -2497,6 +2479,8 @@ func lookdot(n *Node, t *types.Type, dostrcmp int) *types.Field {
}
n.Op = ODOTINTER
} else {
n.SetOpt(f1)
}
return f1

5
src/cmd/compile/internal/gc/walk.go
View file

@ -3734,10 +3734,13 @@ func usefield(n *Node) {
if t.IsPtr() {
t = t.Elem()
}
field := dotField[typeSymKey{t.Orig, n.Sym}]
field := n.Opt().(*types.Field)
if field == nil {
Fatalf("usefield %v %v without paramfld", n.Left.Type, n.Sym)
}
if field.Sym != n.Sym || field.Offset != n.Xoffset {
Fatalf("field inconsistency: %v,%v != %v,%v", field.Sym, field.Offset, n.Sym, n.Xoffset)
}
if !strings.Contains(field.Note, "go:\"track\"") {
return
}

3
src/cmd/compile/internal/ppc64/ssa.go
View file

@ -1781,6 +1781,9 @@ func ssaGenValue(s *gc.SSAGenState, v *ssa.Value) {
pp := s.Call(v)
pp.To.Reg = ppc64.REG_LR
// Insert a hint this is not a subroutine return.
pp.SetFrom3(obj.Addr{Type: obj.TYPE_CONST, Offset: 1})
if gc.Ctxt.Flag_shared {
// When compiling Go into PIC, the function we just
// called via pointer might have been implemented in

14
src/cmd/compile/internal/s390x/ssa.go
View file

@ -188,6 +188,18 @@ func ssaGenValue(s *gc.SSAGenState, v *ssa.Value) {
{Type: obj.TYPE_REG, Reg: r2},
})
p.To = obj.Addr{Type: obj.TYPE_REG, Reg: r1}
case ssa.OpS390XRISBGZ:
r1 := v.Reg()
r2 := v.Args[0].Reg()
i := v.Aux.(s390x.RotateParams)
p := s.Prog(v.Op.Asm())
p.From = obj.Addr{Type: obj.TYPE_CONST, Offset: int64(i.Start)}
p.SetRestArgs([]obj.Addr{
{Type: obj.TYPE_CONST, Offset: int64(i.End)},
{Type: obj.TYPE_CONST, Offset: int64(i.Amount)},
{Type: obj.TYPE_REG, Reg: r2},
})
p.To = obj.Addr{Type: obj.TYPE_REG, Reg: r1}
case ssa.OpS390XADD, ssa.OpS390XADDW,
ssa.OpS390XSUB, ssa.OpS390XSUBW,
ssa.OpS390XAND, ssa.OpS390XANDW,
@ -360,7 +372,7 @@ func ssaGenValue(s *gc.SSAGenState, v *ssa.Value) {
case ssa.OpS390XSLDconst, ssa.OpS390XSLWconst,
ssa.OpS390XSRDconst, ssa.OpS390XSRWconst,
ssa.OpS390XSRADconst, ssa.OpS390XSRAWconst,
ssa.OpS390XRLLGconst, ssa.OpS390XRLLconst:
ssa.OpS390XRLLconst:
p := s.Prog(v.Op.Asm())
p.From.Type = obj.TYPE_CONST
p.From.Offset = v.AuxInt

3
src/cmd/compile/internal/ssa/expand_calls.go
View file

@ -196,6 +196,9 @@ func expandCalls(f *Func) {
}
if leaf.Op == OpIData {
leafType = removeTrivialWrapperTypes(leaf.Type)
if leafType.IsEmptyInterface() {
leafType = typ.BytePtr
}
}
aux := selector.Aux
auxInt := selector.AuxInt + offset

13
src/cmd/compile/internal/ssa/gen/ARM64.rules
View file

@ -543,17 +543,24 @@
(AtomicStore64 ...) => (STLR ...)
(AtomicStorePtrNoWB ...) => (STLR ...)
(AtomicExchange(32|64) ...) => (LoweredAtomicExchange(32|64) ...)
(AtomicAdd(32|64) ...) => (LoweredAtomicAdd(32|64) ...)
(AtomicExchange(32|64) ...) => (LoweredAtomicExchange(32|64) ...)
(AtomicAdd(32|64) ...) => (LoweredAtomicAdd(32|64) ...)
(AtomicCompareAndSwap(32|64) ...) => (LoweredAtomicCas(32|64) ...)
(AtomicAdd(32|64)Variant ...) => (LoweredAtomicAdd(32|64)Variant ...)
(AtomicExchange(32|64)Variant ...) => (LoweredAtomicExchange(32|64)Variant ...)
(AtomicCompareAndSwap(32|64)Variant ...) => (LoweredAtomicCas(32|64)Variant ...)
// Currently the updated value is not used, but we need a register to temporarily hold it.
(AtomicAnd8 ptr val mem) => (Select1 (LoweredAtomicAnd8 ptr val mem))
(AtomicAnd32 ptr val mem) => (Select1 (LoweredAtomicAnd32 ptr val mem))
(AtomicOr8 ptr val mem) => (Select1 (LoweredAtomicOr8 ptr val mem))
(AtomicOr32 ptr val mem) => (Select1 (LoweredAtomicOr32 ptr val mem))
(AtomicAdd(32|64)Variant ...) => (LoweredAtomicAdd(32|64)Variant ...)
(AtomicAnd8Variant ptr val mem) => (Select1 (LoweredAtomicAnd8Variant ptr val mem))
(AtomicAnd32Variant ptr val mem) => (Select1 (LoweredAtomicAnd32Variant ptr val mem))
(AtomicOr8Variant ptr val mem) => (Select1 (LoweredAtomicOr8Variant ptr val mem))
(AtomicOr32Variant ptr val mem) => (Select1 (LoweredAtomicOr32Variant ptr val mem))
// Write barrier.
(WB ...) => (LoweredWB ...)

35
src/cmd/compile/internal/ssa/gen/ARM64Ops.go
View file

@ -621,6 +621,12 @@ func init() {
{name: "LoweredAtomicExchange64", argLength: 3, reg: gpxchg, resultNotInArgs: true, faultOnNilArg0: true, hasSideEffects: true, unsafePoint: true},
{name: "LoweredAtomicExchange32", argLength: 3, reg: gpxchg, resultNotInArgs: true, faultOnNilArg0: true, hasSideEffects: true, unsafePoint: true},
// atomic exchange variant.
// store arg1 to arg0. arg2=mem. returns <old content of *arg0, memory>. auxint must be zero.
// SWPALD Rarg1, (Rarg0), Rout
{name: "LoweredAtomicExchange64Variant", argLength: 3, reg: gpxchg, resultNotInArgs: true, faultOnNilArg0: true, hasSideEffects: true},
{name: "LoweredAtomicExchange32Variant", argLength: 3, reg: gpxchg, resultNotInArgs: true, faultOnNilArg0: true, hasSideEffects: true},
// atomic add.
// *arg0 += arg1. arg2=mem. returns <new content of *arg0, memory>. auxint must be zero.
// LDAXR (Rarg0), Rout
@ -654,6 +660,21 @@ func init() {
{name: "LoweredAtomicCas64", argLength: 4, reg: gpcas, resultNotInArgs: true, clobberFlags: true, faultOnNilArg0: true, hasSideEffects: true, unsafePoint: true},
{name: "LoweredAtomicCas32", argLength: 4, reg: gpcas, resultNotInArgs: true, clobberFlags: true, faultOnNilArg0: true, hasSideEffects: true, unsafePoint: true},
// atomic compare and swap variant.
// arg0 = pointer, arg1 = old value, arg2 = new value, arg3 = memory. auxint must be zero.
// if *arg0 == arg1 {
// *arg0 = arg2
// return (true, memory)
// } else {
// return (false, memory)
// }
// MOV Rarg1, Rtmp
// CASAL Rtmp, (Rarg0), Rarg2
// CMP Rarg1, Rtmp
// CSET EQ, Rout
{name: "LoweredAtomicCas64Variant", argLength: 4, reg: gpcas, resultNotInArgs: true, clobberFlags: true, faultOnNilArg0: true, hasSideEffects: true, unsafePoint: true},
{name: "LoweredAtomicCas32Variant", argLength: 4, reg: gpcas, resultNotInArgs: true, clobberFlags: true, faultOnNilArg0: true, hasSideEffects: true, unsafePoint: true},
// atomic and/or.
// *arg0 &= (|=) arg1. arg2=mem. returns <new content of *arg0, memory>. auxint must be zero.
// LDAXR (Rarg0), Rout
@ -665,6 +686,20 @@ func init() {
{name: "LoweredAtomicOr8", argLength: 3, reg: gpxchg, resultNotInArgs: true, asm: "ORR", typ: "(UInt8,Mem)", faultOnNilArg0: true, hasSideEffects: true, unsafePoint: true},
{name: "LoweredAtomicOr32", argLength: 3, reg: gpxchg, resultNotInArgs: true, asm: "ORR", typ: "(UInt32,Mem)", faultOnNilArg0: true, hasSideEffects: true, unsafePoint: true},
// atomic and/or variant.
// *arg0 &= (|=) arg1. arg2=mem. returns <new content of *arg0, memory>. auxint must be zero.
// AND:
// MNV Rarg1, Rtemp
// LDANDALB Rtemp, (Rarg0), Rout
// AND Rarg1, Rout
// OR:
// LDORALB Rarg1, (Rarg0), Rout
// ORR Rarg1, Rout
{name: "LoweredAtomicAnd8Variant", argLength: 3, reg: gpxchg, resultNotInArgs: true, typ: "(UInt8,Mem)", faultOnNilArg0: true, hasSideEffects: true, unsafePoint: true},
{name: "LoweredAtomicAnd32Variant", argLength: 3, reg: gpxchg, resultNotInArgs: true, typ: "(UInt32,Mem)", faultOnNilArg0: true, hasSideEffects: true, unsafePoint: true},
{name: "LoweredAtomicOr8Variant", argLength: 3, reg: gpxchg, resultNotInArgs: true, typ: "(UInt8,Mem)", faultOnNilArg0: true, hasSideEffects: true},
{name: "LoweredAtomicOr32Variant", argLength: 3, reg: gpxchg, resultNotInArgs: true, typ: "(UInt32,Mem)", faultOnNilArg0: true, hasSideEffects: true},
// LoweredWB invokes runtime.gcWriteBarrier. arg0=destptr, arg1=srcptr, arg2=mem, aux=runtime.gcWriteBarrier
// It saves all GP registers if necessary,
// but clobbers R30 (LR) because it's a call.

25
src/cmd/compile/internal/ssa/gen/MIPS.rules
View file

@ -96,17 +96,17 @@
(Rsh8Ux16 <t> x y) => (CMOVZ (SRL <t> (ZeroExt8to32 x) (ZeroExt16to32 y) ) (MOVWconst [0]) (SGTUconst [32] (ZeroExt16to32 y)))
(Rsh8Ux8 <t> x y) => (CMOVZ (SRL <t> (ZeroExt8to32 x) (ZeroExt8to32 y) ) (MOVWconst [0]) (SGTUconst [32] (ZeroExt8to32 y)))
(Rsh32x32 x y) => (SRA x ( CMOVZ <typ.UInt32> y (MOVWconst [-1]) (SGTUconst [32] y)))
(Rsh32x16 x y) => (SRA x ( CMOVZ <typ.UInt32> (ZeroExt16to32 y) (MOVWconst [-1]) (SGTUconst [32] (ZeroExt16to32 y))))
(Rsh32x8 x y) => (SRA x ( CMOVZ <typ.UInt32> (ZeroExt8to32 y) (MOVWconst [-1]) (SGTUconst [32] (ZeroExt8to32 y))))
(Rsh32x32 x y) => (SRA x ( CMOVZ <typ.UInt32> y (MOVWconst [31]) (SGTUconst [32] y)))
(Rsh32x16 x y) => (SRA x ( CMOVZ <typ.UInt32> (ZeroExt16to32 y) (MOVWconst [31]) (SGTUconst [32] (ZeroExt16to32 y))))
(Rsh32x8 x y) => (SRA x ( CMOVZ <typ.UInt32> (ZeroExt8to32 y) (MOVWconst [31]) (SGTUconst [32] (ZeroExt8to32 y))))
(Rsh16x32 x y) => (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> y (MOVWconst [-1]) (SGTUconst [32] y)))
(Rsh16x16 x y) => (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt16to32 y) (MOVWconst [-1]) (SGTUconst [32] (ZeroExt16to32 y))))
(Rsh16x8 x y) => (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt8to32 y) (MOVWconst [-1]) (SGTUconst [32] (ZeroExt8to32 y))))
(Rsh16x32 x y) => (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> y (MOVWconst [31]) (SGTUconst [32] y)))
(Rsh16x16 x y) => (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt16to32 y) (MOVWconst [31]) (SGTUconst [32] (ZeroExt16to32 y))))
(Rsh16x8 x y) => (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt8to32 y) (MOVWconst [31]) (SGTUconst [32] (ZeroExt8to32 y))))
(Rsh8x32 x y) => (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> y (MOVWconst [-1]) (SGTUconst [32] y)))
(Rsh8x16 x y) => (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt16to32 y) (MOVWconst [-1]) (SGTUconst [32] (ZeroExt16to32 y))))
(Rsh8x8 x y) => (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt8to32 y) (MOVWconst [-1]) (SGTUconst [32] (ZeroExt8to32 y))))
(Rsh8x32 x y) => (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> y (MOVWconst [31]) (SGTUconst [32] y)))
(Rsh8x16 x y) => (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt16to32 y) (MOVWconst [31]) (SGTUconst [32] (ZeroExt16to32 y))))
(Rsh8x8 x y) => (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt8to32 y) (MOVWconst [31]) (SGTUconst [32] (ZeroExt8to32 y))))
// rotates
(RotateLeft8 <t> x (MOVWconst [c])) => (Or8 (Lsh8x32 <t> x (MOVWconst [c&7])) (Rsh8Ux32 <t> x (MOVWconst [-c&7])))
@ -567,10 +567,9 @@
(XOR x (MOVWconst [c])) => (XORconst [c] x)
(NOR x (MOVWconst [c])) => (NORconst [c] x)
(SRA x (MOVWconst [c])) && c >= 32 => (SRAconst x [31])
(SLL x (MOVWconst [c])) => (SLLconst x [c])
(SRL x (MOVWconst [c])) => (SRLconst x [c])
(SRA x (MOVWconst [c])) => (SRAconst x [c])
(SLL x (MOVWconst [c])) => (SLLconst x [c&31])
(SRL x (MOVWconst [c])) => (SRLconst x [c&31])
(SRA x (MOVWconst [c])) => (SRAconst x [c&31])
(SGT (MOVWconst [c]) x) => (SGTconst [c] x)
(SGTU (MOVWconst [c]) x) => (SGTUconst [c] x)

6
src/cmd/compile/internal/ssa/gen/MIPSOps.go
View file

@ -185,11 +185,11 @@ func init() {
// shifts
{name: "SLL", argLength: 2, reg: gp21, asm: "SLL"}, // arg0 << arg1, shift amount is mod 32
{name: "SLLconst", argLength: 1, reg: gp11, asm: "SLL", aux: "Int32"}, // arg0 << auxInt
{name: "SLLconst", argLength: 1, reg: gp11, asm: "SLL", aux: "Int32"}, // arg0 << auxInt, shift amount must be 0 through 31 inclusive
{name: "SRL", argLength: 2, reg: gp21, asm: "SRL"}, // arg0 >> arg1, unsigned, shift amount is mod 32
{name: "SRLconst", argLength: 1, reg: gp11, asm: "SRL", aux: "Int32"}, // arg0 >> auxInt, unsigned
{name: "SRLconst", argLength: 1, reg: gp11, asm: "SRL", aux: "Int32"}, // arg0 >> auxInt, shift amount must be 0 through 31 inclusive
{name: "SRA", argLength: 2, reg: gp21, asm: "SRA"}, // arg0 >> arg1, signed, shift amount is mod 32
{name: "SRAconst", argLength: 1, reg: gp11, asm: "SRA", aux: "Int32"}, // arg0 >> auxInt, signed
{name: "SRAconst", argLength: 1, reg: gp11, asm: "SRA", aux: "Int32"}, // arg0 >> auxInt, signed, shift amount must be 0 through 31 inclusive
{name: "CLZ", argLength: 1, reg: gp11, asm: "CLZ"},

162
src/cmd/compile/internal/ssa/gen/S390X.rules
View file

@ -643,8 +643,18 @@
// equivalent to the leftmost 32 bits being set.
// TODO(mundaym): modify the assembler to accept 64-bit values
// and use isU32Bit(^c).
(AND x (MOVDconst [c])) && is32Bit(c) && c < 0 => (ANDconst [c] x)
(AND x (MOVDconst [c])) && is32Bit(c) && c >= 0 => (MOVWZreg (ANDWconst <typ.UInt32> [int32(c)] x))
(AND x (MOVDconst [c]))
&& s390x.NewRotateParams(0, 63, 0).OutMerge(uint64(c)) != nil
=> (RISBGZ x {*s390x.NewRotateParams(0, 63, 0).OutMerge(uint64(c))})
(AND x (MOVDconst [c]))
&& is32Bit(c)
&& c < 0
=> (ANDconst [c] x)
(AND x (MOVDconst [c]))
&& is32Bit(c)
&& c >= 0
=> (MOVWZreg (ANDWconst <typ.UInt32> [int32(c)] x))
(ANDW x (MOVDconst [c])) => (ANDWconst [int32(c)] x)
((AND|ANDW)const [c] ((AND|ANDW)const [d] x)) => ((AND|ANDW)const [c&d] x)
@ -653,14 +663,20 @@
((OR|XOR)W x (MOVDconst [c])) => ((OR|XOR)Wconst [int32(c)] x)
// Constant shifts.
(S(LD|RD|RAD|LW|RW|RAW) x (MOVDconst [c]))
=> (S(LD|RD|RAD|LW|RW|RAW)const x [int8(c&63)])
(S(LD|RD|RAD) x (MOVDconst [c])) => (S(LD|RD|RAD)const x [int8(c&63)])
(S(LW|RW|RAW) x (MOVDconst [c])) && c&32 == 0 => (S(LW|RW|RAW)const x [int8(c&31)])
(S(LW|RW) _ (MOVDconst [c])) && c&32 != 0 => (MOVDconst [0])
(SRAW x (MOVDconst [c])) && c&32 != 0 => (SRAWconst x [31])
// Shifts only use the rightmost 6 bits of the shift value.
(S(LD|RD|RAD|LW|RW|RAW) x (RISBGZ y {r}))
&& r.Amount == 0
&& r.OutMask()&63 == 63
=> (S(LD|RD|RAD|LW|RW|RAW) x y)
(S(LD|RD|RAD|LW|RW|RAW) x (AND (MOVDconst [c]) y))
=> (S(LD|RD|RAD|LW|RW|RAW) x (ANDWconst <typ.UInt32> [int32(c&63)] y))
=> (S(LD|RD|RAD|LW|RW|RAW) x (ANDWconst <typ.UInt32> [int32(c&63)] y))
(S(LD|RD|RAD|LW|RW|RAW) x (ANDWconst [c] y)) && c&63 == 63
=> (S(LD|RD|RAD|LW|RW|RAW) x y)
=> (S(LD|RD|RAD|LW|RW|RAW) x y)
(SLD x (MOV(W|H|B|WZ|HZ|BZ)reg y)) => (SLD x y)
(SRD x (MOV(W|H|B|WZ|HZ|BZ)reg y)) => (SRD x y)
(SRAD x (MOV(W|H|B|WZ|HZ|BZ)reg y)) => (SRAD x y)
@ -668,17 +684,13 @@
(SRW x (MOV(W|H|B|WZ|HZ|BZ)reg y)) => (SRW x y)
(SRAW x (MOV(W|H|B|WZ|HZ|BZ)reg y)) => (SRAW x y)
// Constant rotate generation
(RLL x (MOVDconst [c])) => (RLLconst x [int8(c&31)])
(RLLG x (MOVDconst [c])) => (RLLGconst x [int8(c&63)])
// Match rotate by constant.
(RLLG x (MOVDconst [c])) => (RISBGZ x {s390x.NewRotateParams(0, 63, int8(c&63))})
(RLL x (MOVDconst [c])) => (RLLconst x [int8(c&31)])
(ADD (SLDconst x [c]) (SRDconst x [d])) && d == 64-c => (RLLGconst [c] x)
( OR (SLDconst x [c]) (SRDconst x [d])) && d == 64-c => (RLLGconst [c] x)
(XOR (SLDconst x [c]) (SRDconst x [d])) && d == 64-c => (RLLGconst [c] x)
(ADDW (SLWconst x [c]) (SRWconst x [d])) && d == 32-c => (RLLconst [c] x)
( ORW (SLWconst x [c]) (SRWconst x [d])) && d == 32-c => (RLLconst [c] x)
(XORW (SLWconst x [c]) (SRWconst x [d])) && d == 32-c => (RLLconst [c] x)
// Match rotate by constant pattern.
((ADD|OR|XOR) (SLDconst x [c]) (SRDconst x [64-c])) => (RISBGZ x {s390x.NewRotateParams(0, 63, c)})
((ADD|OR|XOR)W (SLWconst x [c]) (SRWconst x [32-c])) => (RLLconst x [c])
// Signed 64-bit comparison with immediate.
(CMP x (MOVDconst [c])) && is32Bit(c) => (CMPconst x [int32(c)])
@ -692,15 +704,97 @@
(CMP(W|WU) x (MOVDconst [c])) => (CMP(W|WU)const x [int32(c)])
(CMP(W|WU) (MOVDconst [c]) x) => (InvertFlags (CMP(W|WU)const x [int32(c)]))
// Match (x >> c) << d to 'rotate then insert selected bits [into zero]'.
(SLDconst (SRDconst x [c]) [d]) => (RISBGZ x {s390x.NewRotateParams(max8(0, c-d), 63-d, (d-c)&63)})
// Match (x << c) >> d to 'rotate then insert selected bits [into zero]'.
(SRDconst (SLDconst x [c]) [d]) => (RISBGZ x {s390x.NewRotateParams(d, min8(63, 63-c+d), (c-d)&63)})
// Absorb input zero extension into 'rotate then insert selected bits [into zero]'.
(RISBGZ (MOVWZreg x) {r}) && r.InMerge(0xffffffff) != nil => (RISBGZ x {*r.InMerge(0xffffffff)})
(RISBGZ (MOVHZreg x) {r}) && r.InMerge(0x0000ffff) != nil => (RISBGZ x {*r.InMerge(0x0000ffff)})
(RISBGZ (MOVBZreg x) {r}) && r.InMerge(0x000000ff) != nil => (RISBGZ x {*r.InMerge(0x000000ff)})
// Absorb 'rotate then insert selected bits [into zero]' into zero extension.
(MOVWZreg (RISBGZ x {r})) && r.OutMerge(0xffffffff) != nil => (RISBGZ x {*r.OutMerge(0xffffffff)})
(MOVHZreg (RISBGZ x {r})) && r.OutMerge(0x0000ffff) != nil => (RISBGZ x {*r.OutMerge(0x0000ffff)})
(MOVBZreg (RISBGZ x {r})) && r.OutMerge(0x000000ff) != nil => (RISBGZ x {*r.OutMerge(0x000000ff)})
// Absorb shift into 'rotate then insert selected bits [into zero]'.
//
// Any unsigned shift can be represented as a rotate and mask operation:
//
// x << c => RotateLeft64(x, c) & (^uint64(0) << c)
// x >> c => RotateLeft64(x, -c) & (^uint64(0) >> c)
//
// Therefore when a shift is used as the input to a rotate then insert
// selected bits instruction we can merge the two together. We just have
// to be careful that the resultant mask is representable (non-zero and
// contiguous). For example, assuming that x is variable and c, y and m
// are constants, a shift followed by a rotate then insert selected bits
// could be represented as:
//
// RotateLeft64(RotateLeft64(x, c) & (^uint64(0) << c), y) & m
//
// We can split the rotation by y into two, one rotate for x and one for
// the mask:
//
// RotateLeft64(RotateLeft64(x, c), y) & (RotateLeft64(^uint64(0) << c, y)) & m
//
// The rotations of x by c followed by y can then be combined:
//
// RotateLeft64(x, c+y) & (RotateLeft64(^uint64(0) << c, y)) & m
// ^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
// rotate mask
//
// To perform this optimization we therefore just need to check that it
// is valid to merge the shift mask (^(uint64(0)<<c)) into the selected
// bits mask (i.e. that the resultant mask is non-zero and contiguous).
//
(RISBGZ (SLDconst x [c]) {r}) && r.InMerge(^uint64(0)<<c) != nil => (RISBGZ x {(*r.InMerge(^uint64(0)<<c)).RotateLeft(c)})
(RISBGZ (SRDconst x [c]) {r}) && r.InMerge(^uint64(0)>>c) != nil => (RISBGZ x {(*r.InMerge(^uint64(0)>>c)).RotateLeft(-c)})
// Absorb 'rotate then insert selected bits [into zero]' into left shift.
(SLDconst (RISBGZ x {r}) [c])
&& s390x.NewRotateParams(0, 63-c, c).InMerge(r.OutMask()) != nil
=> (RISBGZ x {(*s390x.NewRotateParams(0, 63-c, c).InMerge(r.OutMask())).RotateLeft(r.Amount)})
// Absorb 'rotate then insert selected bits [into zero]' into right shift.
(SRDconst (RISBGZ x {r}) [c])
&& s390x.NewRotateParams(c, 63, -c&63).InMerge(r.OutMask()) != nil
=> (RISBGZ x {(*s390x.NewRotateParams(c, 63, -c&63).InMerge(r.OutMask())).RotateLeft(r.Amount)})
// Merge 'rotate then insert selected bits [into zero]' instructions together.
(RISBGZ (RISBGZ x {y}) {z})
&& z.InMerge(y.OutMask()) != nil
=> (RISBGZ x {(*z.InMerge(y.OutMask())).RotateLeft(y.Amount)})
// Convert RISBGZ into 64-bit shift (helps CSE).
(RISBGZ x {r}) && r.End == 63 && r.Start == -r.Amount&63 => (SRDconst x [-r.Amount&63])
(RISBGZ x {r}) && r.Start == 0 && r.End == 63-r.Amount => (SLDconst x [r.Amount])
// Optimize single bit isolation when it is known to be equivalent to
// the most significant bit due to mask produced by arithmetic shift.
// Simply isolate the most significant bit itself and place it in the
// correct position.
//
// Example: (int64(x) >> 63) & 0x8 -> RISBGZ $60, $60, $4, Rsrc, Rdst
(RISBGZ (SRADconst x [c]) {r})
&& r.Start == r.End // single bit selected
&& (r.Start+r.Amount)&63 <= c // equivalent to most significant bit of x
=> (RISBGZ x {s390x.NewRotateParams(r.Start, r.Start, -r.Start&63)})
// Canonicalize the order of arguments to comparisons - helps with CSE.
((CMP|CMPW|CMPU|CMPWU) x y) && x.ID > y.ID => (InvertFlags ((CMP|CMPW|CMPU|CMPWU) y x))
// Using MOV{W,H,B}Zreg instead of AND is cheaper.
(AND x (MOVDconst [0xFF])) => (MOVBZreg x)
(AND x (MOVDconst [0xFFFF])) => (MOVHZreg x)
(AND x (MOVDconst [0xFFFFFFFF])) => (MOVWZreg x)
(ANDWconst [0xFF] x) => (MOVBZreg x)
(ANDWconst [0xFFFF] x) => (MOVHZreg x)
// Use sign/zero extend instead of RISBGZ.
(RISBGZ x {r}) && r == s390x.NewRotateParams(56, 63, 0) => (MOVBZreg x)
(RISBGZ x {r}) && r == s390x.NewRotateParams(48, 63, 0) => (MOVHZreg x)
(RISBGZ x {r}) && r == s390x.NewRotateParams(32, 63, 0) => (MOVWZreg x)
// Use sign/zero extend instead of ANDW.
(ANDWconst [0x00ff] x) => (MOVBZreg x)
(ANDWconst [0xffff] x) => (MOVHZreg x)
// Strength reduce multiplication to the sum (or difference) of two powers of two.
//
@ -773,21 +867,22 @@
// detect attempts to set/clear the sign bit
// may need to be reworked when NIHH/OIHH are added
(SRDconst [1] (SLDconst [1] (LGDR <t> x))) => (LGDR <t> (LPDFR <x.Type> x))
(LDGR <t> (SRDconst [1] (SLDconst [1] x))) => (LPDFR (LDGR <t> x))
(AND (MOVDconst [^(-1<<63)]) (LGDR <t> x)) => (LGDR <t> (LPDFR <x.Type> x))
(LDGR <t> (AND (MOVDconst [^(-1<<63)]) x)) => (LPDFR (LDGR <t> x))
(OR (MOVDconst [-1<<63]) (LGDR <t> x)) => (LGDR <t> (LNDFR <x.Type> x))
(LDGR <t> (OR (MOVDconst [-1<<63]) x)) => (LNDFR (LDGR <t> x))
(RISBGZ (LGDR <t> x) {r}) && r == s390x.NewRotateParams(1, 63, 0) => (LGDR <t> (LPDFR <x.Type> x))
(LDGR <t> (RISBGZ x {r})) && r == s390x.NewRotateParams(1, 63, 0) => (LPDFR (LDGR <t> x))
(OR (MOVDconst [-1<<63]) (LGDR <t> x)) => (LGDR <t> (LNDFR <x.Type> x))
(LDGR <t> (OR (MOVDconst [-1<<63]) x)) => (LNDFR (LDGR <t> x))
// detect attempts to set the sign bit with load
(LDGR <t> x:(ORload <t1> [off] {sym} (MOVDconst [-1<<63]) ptr mem)) && x.Uses == 1 && clobber(x) => @x.Block (LNDFR <t> (LDGR <t> (MOVDload <t1> [off] {sym} ptr mem)))
// detect copysign
(OR (SLDconst [63] (SRDconst [63] (LGDR x))) (LGDR (LPDFR <t> y))) => (LGDR (CPSDR <t> y x))
(OR (SLDconst [63] (SRDconst [63] (LGDR x))) (MOVDconst [c])) && c & -1<<63 == 0 => (LGDR (CPSDR <x.Type> (FMOVDconst <x.Type> [math.Float64frombits(uint64(c))]) x))
(OR (AND (MOVDconst [-1<<63]) (LGDR x)) (LGDR (LPDFR <t> y))) => (LGDR (CPSDR <t> y x))
(OR (AND (MOVDconst [-1<<63]) (LGDR x)) (MOVDconst [c])) && c & -1<<63 == 0 => (LGDR (CPSDR <x.Type> (FMOVDconst <x.Type> [math.Float64frombits(uint64(c))]) x))
(OR (RISBGZ (LGDR x) {r}) (LGDR (LPDFR <t> y)))
&& r == s390x.NewRotateParams(0, 0, 0)
=> (LGDR (CPSDR <t> y x))
(OR (RISBGZ (LGDR x) {r}) (MOVDconst [c]))
&& c >= 0
&& r == s390x.NewRotateParams(0, 0, 0)
=> (LGDR (CPSDR <x.Type> (FMOVDconst <x.Type> [math.Float64frombits(uint64(c))]) x))
(CPSDR y (FMOVDconst [c])) && !math.Signbit(c) => (LPDFR y)
(CPSDR y (FMOVDconst [c])) && math.Signbit(c) => (LNDFR y)
@ -966,6 +1061,9 @@
(CMPWconst (ANDWconst _ [m]) [n]) && int32(m) >= 0 && int32(m) < int32(n) => (FlagLT)
(CMPWUconst (ANDWconst _ [m]) [n]) && uint32(m) < uint32(n) => (FlagLT)
(CMPconst (RISBGZ x {r}) [c]) && c > 0 && r.OutMask() < uint64(c) => (FlagLT)
(CMPUconst (RISBGZ x {r}) [c]) && r.OutMask() < uint64(uint32(c)) => (FlagLT)
// Constant compare-and-branch with immediate.
(CGIJ {c} (MOVDconst [x]) [y] yes no) && c&s390x.Equal != 0 && int64(x) == int64(y) => (First yes no)
(CGIJ {c} (MOVDconst [x]) [y] yes no) && c&s390x.Less != 0 && int64(x) < int64(y) => (First yes no)

20
src/cmd/compile/internal/ssa/gen/S390XOps.go
View file

@ -331,25 +331,26 @@ func init() {
{name: "LTEBR", argLength: 1, reg: fp1flags, asm: "LTEBR", typ: "Flags"}, // arg0 compare to 0, f32
{name: "SLD", argLength: 2, reg: sh21, asm: "SLD"}, // arg0 << arg1, shift amount is mod 64
{name: "SLW", argLength: 2, reg: sh21, asm: "SLW"}, // arg0 << arg1, shift amount is mod 32
{name: "SLW", argLength: 2, reg: sh21, asm: "SLW"}, // arg0 << arg1, shift amount is mod 64
{name: "SLDconst", argLength: 1, reg: gp11, asm: "SLD", aux: "Int8"}, // arg0 << auxint, shift amount 0-63
{name: "SLWconst", argLength: 1, reg: gp11, asm: "SLW", aux: "Int8"}, // arg0 << auxint, shift amount 0-31
{name: "SRD", argLength: 2, reg: sh21, asm: "SRD"}, // unsigned arg0 >> arg1, shift amount is mod 64
{name: "SRW", argLength: 2, reg: sh21, asm: "SRW"}, // unsigned uint32(arg0) >> arg1, shift amount is mod 32
{name: "SRW", argLength: 2, reg: sh21, asm: "SRW"}, // unsigned uint32(arg0) >> arg1, shift amount is mod 64
{name: "SRDconst", argLength: 1, reg: gp11, asm: "SRD", aux: "Int8"}, // unsigned arg0 >> auxint, shift amount 0-63
{name: "SRWconst", argLength: 1, reg: gp11, asm: "SRW", aux: "Int8"}, // unsigned uint32(arg0) >> auxint, shift amount 0-31
// Arithmetic shifts clobber flags.
{name: "SRAD", argLength: 2, reg: sh21, asm: "SRAD", clobberFlags: true}, // signed arg0 >> arg1, shift amount is mod 64
{name: "SRAW", argLength: 2, reg: sh21, asm: "SRAW", clobberFlags: true}, // signed int32(arg0) >> arg1, shift amount is mod 32
{name: "SRAW", argLength: 2, reg: sh21, asm: "SRAW", clobberFlags: true}, // signed int32(arg0) >> arg1, shift amount is mod 64
{name: "SRADconst", argLength: 1, reg: gp11, asm: "SRAD", aux: "Int8", clobberFlags: true}, // signed arg0 >> auxint, shift amount 0-63
{name: "SRAWconst", argLength: 1, reg: gp11, asm: "SRAW", aux: "Int8", clobberFlags: true}, // signed int32(arg0) >> auxint, shift amount 0-31
{name: "RLLG", argLength: 2, reg: sh21, asm: "RLLG"}, // arg0 rotate left arg1, rotate amount 0-63
{name: "RLL", argLength: 2, reg: sh21, asm: "RLL"}, // arg0 rotate left arg1, rotate amount 0-31
{name: "RLLGconst", argLength: 1, reg: gp11, asm: "RLLG", aux: "Int8"}, // arg0 rotate left auxint, rotate amount 0-63
{name: "RLLconst", argLength: 1, reg: gp11, asm: "RLL", aux: "Int8"}, // arg0 rotate left auxint, rotate amount 0-31
// Rotate instructions.
// Note: no RLLGconst - use RISBGZ instead.
{name: "RLLG", argLength: 2, reg: sh21, asm: "RLLG"}, // arg0 rotate left arg1, rotate amount 0-63
{name: "RLL", argLength: 2, reg: sh21, asm: "RLL"}, // arg0 rotate left arg1, rotate amount 0-31
{name: "RLLconst", argLength: 1, reg: gp11, asm: "RLL", aux: "Int8"}, // arg0 rotate left auxint, rotate amount 0-31
// Rotate then (and|or|xor|insert) selected bits instructions.
//
@ -371,6 +372,7 @@ func init() {
// +-------------+-------+-----+--------+-----------------------+-----------------------+-----------------------+
//
{name: "RXSBG", argLength: 2, reg: gp21, asm: "RXSBG", resultInArg0: true, aux: "S390XRotateParams", clobberFlags: true}, // rotate then xor selected bits
{name: "RISBGZ", argLength: 1, reg: gp11, asm: "RISBGZ", aux: "S390XRotateParams", clobberFlags: true}, // rotate then insert selected bits [into zero]
// unary ops
{name: "NEG", argLength: 1, reg: gp11, asm: "NEG", clobberFlags: true}, // -arg0
@ -547,9 +549,9 @@ func init() {
// Atomic bitwise operations.
// Note: 'floor' operations round the pointer down to the nearest word boundary
// which reflects how they are used in the runtime.
{name: "LAN", argLength: 3, reg: gpstore, asm: "LAN", typ: "Mem", clobberFlags: true, hasSideEffects: true}, // *arg0 &= arg1. arg2 = mem.
{name: "LAN", argLength: 3, reg: gpstore, asm: "LAN", typ: "Mem", clobberFlags: true, hasSideEffects: true}, // *arg0 &= arg1. arg2 = mem.
{name: "LANfloor", argLength: 3, reg: gpstorelab, asm: "LAN", typ: "Mem", clobberFlags: true, hasSideEffects: true}, // *(floor(arg0, 4)) &= arg1. arg2 = mem.
{name: "LAO", argLength: 3, reg: gpstore, asm: "LAO", typ: "Mem", clobberFlags: true, hasSideEffects: true}, // *arg0 |= arg1. arg2 = mem.
{name: "LAO", argLength: 3, reg: gpstore, asm: "LAO", typ: "Mem", clobberFlags: true, hasSideEffects: true}, // *arg0 |= arg1. arg2 = mem.
{name: "LAOfloor", argLength: 3, reg: gpstorelab, asm: "LAO", typ: "Mem", clobberFlags: true, hasSideEffects: true}, // *(floor(arg0, 4)) |= arg1. arg2 = mem.
// Compare and swap.

12
src/cmd/compile/internal/ssa/gen/genericOps.go
View file

@ -574,8 +574,16 @@ var genericOps = []opData{
// These variants have the same semantics as above atomic operations.
// But they are used for generating more efficient code on certain modern machines, with run-time CPU feature detection.
// Currently, they are used on ARM64 only.
{name: "AtomicAdd32Variant", argLength: 3, typ: "(UInt32,Mem)", hasSideEffects: true}, // Do *arg0 += arg1. arg2=memory. Returns sum and new memory.
{name: "AtomicAdd64Variant", argLength: 3, typ: "(UInt64,Mem)", hasSideEffects: true}, // Do *arg0 += arg1. arg2=memory. Returns sum and new memory.
{name: "AtomicAdd32Variant", argLength: 3, typ: "(UInt32,Mem)", hasSideEffects: true}, // Do *arg0 += arg1. arg2=memory. Returns sum and new memory.
{name: "AtomicAdd64Variant", argLength: 3, typ: "(UInt64,Mem)", hasSideEffects: true}, // Do *arg0 += arg1. arg2=memory. Returns sum and new memory.
{name: "AtomicExchange32Variant", argLength: 3, typ: "(UInt32,Mem)", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicExchange64Variant", argLength: 3, typ: "(UInt64,Mem)", hasSideEffects: true}, // Store arg1 to *arg0. arg2=memory. Returns old contents of *arg0 and new memory.
{name: "AtomicCompareAndSwap32Variant", argLength: 4, typ: "(Bool,Mem)", hasSideEffects: true}, // if *arg0==arg1, then set *arg0=arg2. Returns true if store happens and new memory.
{name: "AtomicCompareAndSwap64Variant", argLength: 4, typ: "(Bool,Mem)", hasSideEffects: true}, // if *arg0==arg1, then set *arg0=arg2. Returns true if store happens and new memory.
{name: "AtomicAnd8Variant", argLength: 3, typ: "Mem", hasSideEffects: true}, // *arg0 &= arg1. arg2=memory. Returns memory.
{name: "AtomicAnd32Variant", argLength: 3, typ: "Mem", hasSideEffects: true}, // *arg0 &= arg1. arg2=memory. Returns memory.
{name: "AtomicOr8Variant", argLength: 3, typ: "Mem", hasSideEffects: true}, // *arg0 |= arg1. arg2=memory. Returns memory.
{name: "AtomicOr32Variant", argLength: 3, typ: "Mem", hasSideEffects: true}, // *arg0 |= arg1. arg2=memory. Returns memory.
// Clobber experiment op
{name: "Clobber", argLength: 0, typ: "Void", aux: "SymOff", symEffect: "None"}, // write an invalid pointer value to the given pointer slot of a stack variable

231
src/cmd/compile/internal/ssa/opGen.go
View file

@ -1581,16 +1581,24 @@ const (
OpARM64STLRW
OpARM64LoweredAtomicExchange64
OpARM64LoweredAtomicExchange32
OpARM64LoweredAtomicExchange64Variant
OpARM64LoweredAtomicExchange32Variant
OpARM64LoweredAtomicAdd64
OpARM64LoweredAtomicAdd32
OpARM64LoweredAtomicAdd64Variant
OpARM64LoweredAtomicAdd32Variant
OpARM64LoweredAtomicCas64
OpARM64LoweredAtomicCas32
OpARM64LoweredAtomicCas64Variant
OpARM64LoweredAtomicCas32Variant
OpARM64LoweredAtomicAnd8
OpARM64LoweredAtomicAnd32
OpARM64LoweredAtomicOr8
OpARM64LoweredAtomicOr32
OpARM64LoweredAtomicAnd8Variant
OpARM64LoweredAtomicAnd32Variant
OpARM64LoweredAtomicOr8Variant
OpARM64LoweredAtomicOr32Variant
OpARM64LoweredWB
OpARM64LoweredPanicBoundsA
OpARM64LoweredPanicBoundsB
@ -2277,9 +2285,9 @@ const (
OpS390XSRAWconst
OpS390XRLLG
OpS390XRLL
OpS390XRLLGconst
OpS390XRLLconst
OpS390XRXSBG
OpS390XRISBGZ
OpS390XNEG
OpS390XNEGW
OpS390XNOT
@ -2881,6 +2889,14 @@ const (
OpAtomicOr32
OpAtomicAdd32Variant
OpAtomicAdd64Variant
OpAtomicExchange32Variant
OpAtomicExchange64Variant
OpAtomicCompareAndSwap32Variant
OpAtomicCompareAndSwap64Variant
OpAtomicAnd8Variant
OpAtomicAnd32Variant
OpAtomicOr8Variant
OpAtomicOr32Variant
OpClobber
)
@ -20994,6 +21010,38 @@ var opcodeTable = [...]opInfo{
},
},
},
{
name: "LoweredAtomicExchange64Variant",
argLen: 3,
resultNotInArgs: true,
faultOnNilArg0: true,
hasSideEffects: true,
reg: regInfo{
inputs: []inputInfo{
{1, 805044223}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30
{0, 9223372038733561855}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30 SP SB
},
outputs: []outputInfo{
{0, 670826495}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 R30
},
},
},
{
name: "LoweredAtomicExchange32Variant",
argLen: 3,
resultNotInArgs: true,
faultOnNilArg0: true,
hasSideEffects: true,
reg: regInfo{
inputs: []inputInfo{
{1, 805044223}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30
{0, 9223372038733561855}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30 SP SB
},
outputs: []outputInfo{
{0, 670826495}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 R30
},
},
},
{
name: "LoweredAtomicAdd64",
argLen: 3,
@ -21098,6 +21146,44 @@ var opcodeTable = [...]opInfo{
},
},
},
{
name: "LoweredAtomicCas64Variant",
argLen: 4,
resultNotInArgs: true,
clobberFlags: true,
faultOnNilArg0: true,
hasSideEffects: true,
unsafePoint: true,
reg: regInfo{
inputs: []inputInfo{
{1, 805044223}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30
{2, 805044223}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30
{0, 9223372038733561855}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30 SP SB
},
outputs: []outputInfo{
{0, 670826495}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 R30
},
},
},
{
name: "LoweredAtomicCas32Variant",
argLen: 4,
resultNotInArgs: true,
clobberFlags: true,
faultOnNilArg0: true,
hasSideEffects: true,
unsafePoint: true,
reg: regInfo{
inputs: []inputInfo{
{1, 805044223}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30
{2, 805044223}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30
{0, 9223372038733561855}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30 SP SB
},
outputs: []outputInfo{
{0, 670826495}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 R30
},
},
},
{
name: "LoweredAtomicAnd8",
argLen: 3,
@ -21170,6 +21256,72 @@ var opcodeTable = [...]opInfo{
},
},
},
{
name: "LoweredAtomicAnd8Variant",
argLen: 3,
resultNotInArgs: true,
faultOnNilArg0: true,
hasSideEffects: true,
unsafePoint: true,
reg: regInfo{
inputs: []inputInfo{
{1, 805044223}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30
{0, 9223372038733561855}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30 SP SB
},
outputs: []outputInfo{
{0, 670826495}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 R30
},
},
},
{
name: "LoweredAtomicAnd32Variant",
argLen: 3,
resultNotInArgs: true,
faultOnNilArg0: true,
hasSideEffects: true,
unsafePoint: true,
reg: regInfo{
inputs: []inputInfo{
{1, 805044223}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30
{0, 9223372038733561855}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30 SP SB
},
outputs: []outputInfo{
{0, 670826495}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 R30
},
},
},
{
name: "LoweredAtomicOr8Variant",
argLen: 3,
resultNotInArgs: true,
faultOnNilArg0: true,
hasSideEffects: true,
reg: regInfo{
inputs: []inputInfo{
{1, 805044223}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30
{0, 9223372038733561855}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30 SP SB
},
outputs: []outputInfo{
{0, 670826495}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 R30
},
},
},
{
name: "LoweredAtomicOr32Variant",
argLen: 3,
resultNotInArgs: true,
faultOnNilArg0: true,
hasSideEffects: true,
reg: regInfo{
inputs: []inputInfo{
{1, 805044223}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30
{0, 9223372038733561855}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 g R30 SP SB
},
outputs: []outputInfo{
{0, 670826495}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14 R15 R16 R17 R19 R20 R21 R22 R23 R24 R25 R26 R30
},
},
},
{
name: "LoweredWB",
auxType: auxSym,
@ -30587,20 +30739,6 @@ var opcodeTable = [...]opInfo{
},
},
},
{
name: "RLLGconst",
auxType: auxInt8,
argLen: 1,
asm: s390x.ARLLG,
reg: regInfo{
inputs: []inputInfo{
{0, 23551}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R11 R12 R14
},
outputs: []outputInfo{
{0, 23551}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R11 R12 R14
},
},
},
{
name: "RLLconst",
auxType: auxInt8,
@ -30632,6 +30770,21 @@ var opcodeTable = [...]opInfo{
},
},
},
{
name: "RISBGZ",
auxType: auxS390XRotateParams,
argLen: 1,
clobberFlags: true,
asm: s390x.ARISBGZ,
reg: regInfo{
inputs: []inputInfo{
{0, 23551}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R11 R12 R14
},
outputs: []outputInfo{
{0, 23551}, // R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 R11 R12 R14
},
},
},
{
name: "NEG",
argLen: 1,
@ -35874,6 +36027,54 @@ var opcodeTable = [...]opInfo{
hasSideEffects: true,
generic: true,
},
{
name: "AtomicExchange32Variant",
argLen: 3,
hasSideEffects: true,
generic: true,
},
{
name: "AtomicExchange64Variant",
argLen: 3,
hasSideEffects: true,
generic: true,
},
{
name: "AtomicCompareAndSwap32Variant",
argLen: 4,
hasSideEffects: true,
generic: true,
},
{
name: "AtomicCompareAndSwap64Variant",
argLen: 4,
hasSideEffects: true,
generic: true,
},
{
name: "AtomicAnd8Variant",
argLen: 3,
hasSideEffects: true,
generic: true,
},
{
name: "AtomicAnd32Variant",
argLen: 3,
hasSideEffects: true,
generic: true,
},
{
name: "AtomicOr8Variant",
argLen: 3,
hasSideEffects: true,
generic: true,
},
{
name: "AtomicOr32Variant",
argLen: 3,
hasSideEffects: true,
generic: true,
},
{
name: "Clobber",
auxType: auxSymOff,

2
src/cmd/compile/internal/ssa/prove.go
View file

@ -1082,7 +1082,7 @@ func addLocalInductiveFacts(ft *factsTable, b *Block) {
return nil
}
pred, child := b.Preds[1].b, b
for ; pred != nil; pred = uniquePred(pred) {
for ; pred != nil; pred, child = uniquePred(pred), pred {
if pred.Kind != BlockIf {
continue
}

7
src/cmd/compile/internal/ssa/rewrite.go
View file

@ -1427,10 +1427,11 @@ func DecodePPC64RotateMask(sauxint int64) (rotate, mb, me int64, mask uint64) {
return
}
// This verifies that the mask occupies the
// rightmost bits.
// This verifies that the mask is a set of
// consecutive bits including the least
// significant bit.
func isPPC64ValidShiftMask(v int64) bool {
if ((v + 1) & v) == 0 {
if (v != 0) && ((v+1)&v) == 0 {
return true
}
return false

96
src/cmd/compile/internal/ssa/rewriteARM64.go
View file

@ -426,20 +426,36 @@ func rewriteValueARM64(v *Value) bool {
return true
case OpAtomicAnd32:
return rewriteValueARM64_OpAtomicAnd32(v)
case OpAtomicAnd32Variant:
return rewriteValueARM64_OpAtomicAnd32Variant(v)
case OpAtomicAnd8:
return rewriteValueARM64_OpAtomicAnd8(v)
case OpAtomicAnd8Variant:
return rewriteValueARM64_OpAtomicAnd8Variant(v)
case OpAtomicCompareAndSwap32:
v.Op = OpARM64LoweredAtomicCas32
return true
case OpAtomicCompareAndSwap32Variant:
v.Op = OpARM64LoweredAtomicCas32Variant
return true
case OpAtomicCompareAndSwap64:
v.Op = OpARM64LoweredAtomicCas64
return true
case OpAtomicCompareAndSwap64Variant:
v.Op = OpARM64LoweredAtomicCas64Variant
return true
case OpAtomicExchange32:
v.Op = OpARM64LoweredAtomicExchange32
return true
case OpAtomicExchange32Variant:
v.Op = OpARM64LoweredAtomicExchange32Variant
return true
case OpAtomicExchange64:
v.Op = OpARM64LoweredAtomicExchange64
return true
case OpAtomicExchange64Variant:
v.Op = OpARM64LoweredAtomicExchange64Variant
return true
case OpAtomicLoad32:
v.Op = OpARM64LDARW
return true
@ -454,8 +470,12 @@ func rewriteValueARM64(v *Value) bool {
return true
case OpAtomicOr32:
return rewriteValueARM64_OpAtomicOr32(v)
case OpAtomicOr32Variant:
return rewriteValueARM64_OpAtomicOr32Variant(v)
case OpAtomicOr8:
return rewriteValueARM64_OpAtomicOr8(v)
case OpAtomicOr8Variant:
return rewriteValueARM64_OpAtomicOr8Variant(v)
case OpAtomicStore32:
v.Op = OpARM64STLRW
return true
@ -21363,6 +21383,25 @@ func rewriteValueARM64_OpAtomicAnd32(v *Value) bool {
return true
}
}
func rewriteValueARM64_OpAtomicAnd32Variant(v *Value) bool {
v_2 := v.Args[2]
v_1 := v.Args[1]
v_0 := v.Args[0]
b := v.Block
typ := &b.Func.Config.Types
// match: (AtomicAnd32Variant ptr val mem)
// result: (Select1 (LoweredAtomicAnd32Variant ptr val mem))
for {
ptr := v_0
val := v_1
mem := v_2
v.reset(OpSelect1)
v0 := b.NewValue0(v.Pos, OpARM64LoweredAtomicAnd32Variant, types.NewTuple(typ.UInt32, types.TypeMem))
v0.AddArg3(ptr, val, mem)
v.AddArg(v0)
return true
}
}
func rewriteValueARM64_OpAtomicAnd8(v *Value) bool {
v_2 := v.Args[2]
v_1 := v.Args[1]
@ -21382,6 +21421,25 @@ func rewriteValueARM64_OpAtomicAnd8(v *Value) bool {
return true
}
}
func rewriteValueARM64_OpAtomicAnd8Variant(v *Value) bool {
v_2 := v.Args[2]
v_1 := v.Args[1]
v_0 := v.Args[0]
b := v.Block
typ := &b.Func.Config.Types
// match: (AtomicAnd8Variant ptr val mem)
// result: (Select1 (LoweredAtomicAnd8Variant ptr val mem))
for {
ptr := v_0
val := v_1
mem := v_2
v.reset(OpSelect1)
v0 := b.NewValue0(v.Pos, OpARM64LoweredAtomicAnd8Variant, types.NewTuple(typ.UInt8, types.TypeMem))
v0.AddArg3(ptr, val, mem)
v.AddArg(v0)
return true
}
}
func rewriteValueARM64_OpAtomicOr32(v *Value) bool {
v_2 := v.Args[2]
v_1 := v.Args[1]
@ -21401,6 +21459,25 @@ func rewriteValueARM64_OpAtomicOr32(v *Value) bool {
return true
}
}
func rewriteValueARM64_OpAtomicOr32Variant(v *Value) bool {
v_2 := v.Args[2]
v_1 := v.Args[1]
v_0 := v.Args[0]
b := v.Block
typ := &b.Func.Config.Types
// match: (AtomicOr32Variant ptr val mem)
// result: (Select1 (LoweredAtomicOr32Variant ptr val mem))
for {
ptr := v_0
val := v_1
mem := v_2
v.reset(OpSelect1)
v0 := b.NewValue0(v.Pos, OpARM64LoweredAtomicOr32Variant, types.NewTuple(typ.UInt32, types.TypeMem))
v0.AddArg3(ptr, val, mem)
v.AddArg(v0)
return true
}
}
func rewriteValueARM64_OpAtomicOr8(v *Value) bool {
v_2 := v.Args[2]
v_1 := v.Args[1]
@ -21420,6 +21497,25 @@ func rewriteValueARM64_OpAtomicOr8(v *Value) bool {
return true
}
}
func rewriteValueARM64_OpAtomicOr8Variant(v *Value) bool {
v_2 := v.Args[2]
v_1 := v.Args[1]
v_0 := v.Args[0]
b := v.Block
typ := &b.Func.Config.Types
// match: (AtomicOr8Variant ptr val mem)
// result: (Select1 (LoweredAtomicOr8Variant ptr val mem))
for {
ptr := v_0
val := v_1
mem := v_2
v.reset(OpSelect1)
v0 := b.NewValue0(v.Pos, OpARM64LoweredAtomicOr8Variant, types.NewTuple(typ.UInt8, types.TypeMem))
v0.AddArg3(ptr, val, mem)
v.AddArg(v0)
return true
}
}
func rewriteValueARM64_OpAvg64u(v *Value) bool {
v_1 := v.Args[1]
v_0 := v.Args[0]

65
src/cmd/compile/internal/ssa/rewriteMIPS.go
View file

@ -4431,7 +4431,7 @@ func rewriteValueMIPS_OpMIPSSLL(v *Value) bool {
v_1 := v.Args[1]
v_0 := v.Args[0]
// match: (SLL x (MOVWconst [c]))
// result: (SLLconst x [c])
// result: (SLLconst x [c&31])
for {
x := v_0
if v_1.Op != OpMIPSMOVWconst {
@ -4439,7 +4439,7 @@ func rewriteValueMIPS_OpMIPSSLL(v *Value) bool {
}
c := auxIntToInt32(v_1.AuxInt)
v.reset(OpMIPSSLLconst)
v.AuxInt = int32ToAuxInt(c)
v.AuxInt = int32ToAuxInt(c & 31)
v.AddArg(x)
return true
}
@ -4465,24 +4465,7 @@ func rewriteValueMIPS_OpMIPSSRA(v *Value) bool {
v_1 := v.Args[1]
v_0 := v.Args[0]
// match: (SRA x (MOVWconst [c]))
// cond: c >= 32
// result: (SRAconst x [31])
for {
x := v_0
if v_1.Op != OpMIPSMOVWconst {
break
}
c := auxIntToInt32(v_1.AuxInt)
if !(c >= 32) {
break
}
v.reset(OpMIPSSRAconst)
v.AuxInt = int32ToAuxInt(31)
v.AddArg(x)
return true
}
// match: (SRA x (MOVWconst [c]))
// result: (SRAconst x [c])
// result: (SRAconst x [c&31])
for {
x := v_0
if v_1.Op != OpMIPSMOVWconst {
@ -4490,7 +4473,7 @@ func rewriteValueMIPS_OpMIPSSRA(v *Value) bool {
}
c := auxIntToInt32(v_1.AuxInt)
v.reset(OpMIPSSRAconst)
v.AuxInt = int32ToAuxInt(c)
v.AuxInt = int32ToAuxInt(c & 31)
v.AddArg(x)
return true
}
@ -4516,7 +4499,7 @@ func rewriteValueMIPS_OpMIPSSRL(v *Value) bool {
v_1 := v.Args[1]
v_0 := v.Args[0]
// match: (SRL x (MOVWconst [c]))
// result: (SRLconst x [c])
// result: (SRLconst x [c&31])
for {
x := v_0
if v_1.Op != OpMIPSMOVWconst {
@ -4524,7 +4507,7 @@ func rewriteValueMIPS_OpMIPSSRL(v *Value) bool {
}
c := auxIntToInt32(v_1.AuxInt)
v.reset(OpMIPSSRLconst)
v.AuxInt = int32ToAuxInt(c)
v.AuxInt = int32ToAuxInt(c & 31)
v.AddArg(x)
return true
}
@ -5714,7 +5697,7 @@ func rewriteValueMIPS_OpRsh16x16(v *Value) bool {
b := v.Block
typ := &b.Func.Config.Types
// match: (Rsh16x16 x y)
// result: (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt16to32 y) (MOVWconst [-1]) (SGTUconst [32] (ZeroExt16to32 y))))
// result: (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt16to32 y) (MOVWconst [31]) (SGTUconst [32] (ZeroExt16to32 y))))
for {
x := v_0
y := v_1
@ -5725,7 +5708,7 @@ func rewriteValueMIPS_OpRsh16x16(v *Value) bool {
v2 := b.NewValue0(v.Pos, OpZeroExt16to32, typ.UInt32)
v2.AddArg(y)
v3 := b.NewValue0(v.Pos, OpMIPSMOVWconst, typ.UInt32)
v3.AuxInt = int32ToAuxInt(-1)
v3.AuxInt = int32ToAuxInt(31)
v4 := b.NewValue0(v.Pos, OpMIPSSGTUconst, typ.Bool)
v4.AuxInt = int32ToAuxInt(32)
v4.AddArg(v2)
@ -5740,7 +5723,7 @@ func rewriteValueMIPS_OpRsh16x32(v *Value) bool {
b := v.Block
typ := &b.Func.Config.Types
// match: (Rsh16x32 x y)
// result: (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> y (MOVWconst [-1]) (SGTUconst [32] y)))
// result: (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> y (MOVWconst [31]) (SGTUconst [32] y)))
for {
x := v_0
y := v_1
@ -5749,7 +5732,7 @@ func rewriteValueMIPS_OpRsh16x32(v *Value) bool {
v0.AddArg(x)
v1 := b.NewValue0(v.Pos, OpMIPSCMOVZ, typ.UInt32)
v2 := b.NewValue0(v.Pos, OpMIPSMOVWconst, typ.UInt32)
v2.AuxInt = int32ToAuxInt(-1)
v2.AuxInt = int32ToAuxInt(31)
v3 := b.NewValue0(v.Pos, OpMIPSSGTUconst, typ.Bool)
v3.AuxInt = int32ToAuxInt(32)
v3.AddArg(y)
@ -5811,7 +5794,7 @@ func rewriteValueMIPS_OpRsh16x8(v *Value) bool {
b := v.Block
typ := &b.Func.Config.Types
// match: (Rsh16x8 x y)
// result: (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt8to32 y) (MOVWconst [-1]) (SGTUconst [32] (ZeroExt8to32 y))))
// result: (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt8to32 y) (MOVWconst [31]) (SGTUconst [32] (ZeroExt8to32 y))))
for {
x := v_0
y := v_1
@ -5822,7 +5805,7 @@ func rewriteValueMIPS_OpRsh16x8(v *Value) bool {
v2 := b.NewValue0(v.Pos, OpZeroExt8to32, typ.UInt32)
v2.AddArg(y)
v3 := b.NewValue0(v.Pos, OpMIPSMOVWconst, typ.UInt32)
v3.AuxInt = int32ToAuxInt(-1)
v3.AuxInt = int32ToAuxInt(31)
v4 := b.NewValue0(v.Pos, OpMIPSSGTUconst, typ.Bool)
v4.AuxInt = int32ToAuxInt(32)
v4.AddArg(v2)
@ -5947,7 +5930,7 @@ func rewriteValueMIPS_OpRsh32x16(v *Value) bool {
b := v.Block
typ := &b.Func.Config.Types
// match: (Rsh32x16 x y)
// result: (SRA x ( CMOVZ <typ.UInt32> (ZeroExt16to32 y) (MOVWconst [-1]) (SGTUconst [32] (ZeroExt16to32 y))))
// result: (SRA x ( CMOVZ <typ.UInt32> (ZeroExt16to32 y) (MOVWconst [31]) (SGTUconst [32] (ZeroExt16to32 y))))
for {
x := v_0
y := v_1
@ -5956,7 +5939,7 @@ func rewriteValueMIPS_OpRsh32x16(v *Value) bool {
v1 := b.NewValue0(v.Pos, OpZeroExt16to32, typ.UInt32)
v1.AddArg(y)
v2 := b.NewValue0(v.Pos, OpMIPSMOVWconst, typ.UInt32)
v2.AuxInt = int32ToAuxInt(-1)
v2.AuxInt = int32ToAuxInt(31)
v3 := b.NewValue0(v.Pos, OpMIPSSGTUconst, typ.Bool)
v3.AuxInt = int32ToAuxInt(32)
v3.AddArg(v1)
@ -5971,14 +5954,14 @@ func rewriteValueMIPS_OpRsh32x32(v *Value) bool {
b := v.Block
typ := &b.Func.Config.Types
// match: (Rsh32x32 x y)
// result: (SRA x ( CMOVZ <typ.UInt32> y (MOVWconst [-1]) (SGTUconst [32] y)))
// result: (SRA x ( CMOVZ <typ.UInt32> y (MOVWconst [31]) (SGTUconst [32] y)))
for {
x := v_0
y := v_1
v.reset(OpMIPSSRA)
v0 := b.NewValue0(v.Pos, OpMIPSCMOVZ, typ.UInt32)
v1 := b.NewValue0(v.Pos, OpMIPSMOVWconst, typ.UInt32)
v1.AuxInt = int32ToAuxInt(-1)
v1.AuxInt = int32ToAuxInt(31)
v2 := b.NewValue0(v.Pos, OpMIPSSGTUconst, typ.Bool)
v2.AuxInt = int32ToAuxInt(32)
v2.AddArg(y)
@ -6032,7 +6015,7 @@ func rewriteValueMIPS_OpRsh32x8(v *Value) bool {
b := v.Block
typ := &b.Func.Config.Types
// match: (Rsh32x8 x y)
// result: (SRA x ( CMOVZ <typ.UInt32> (ZeroExt8to32 y) (MOVWconst [-1]) (SGTUconst [32] (ZeroExt8to32 y))))
// result: (SRA x ( CMOVZ <typ.UInt32> (ZeroExt8to32 y) (MOVWconst [31]) (SGTUconst [32] (ZeroExt8to32 y))))
for {
x := v_0
y := v_1
@ -6041,7 +6024,7 @@ func rewriteValueMIPS_OpRsh32x8(v *Value) bool {
v1 := b.NewValue0(v.Pos, OpZeroExt8to32, typ.UInt32)
v1.AddArg(y)
v2 := b.NewValue0(v.Pos, OpMIPSMOVWconst, typ.UInt32)
v2.AuxInt = int32ToAuxInt(-1)
v2.AuxInt = int32ToAuxInt(31)
v3 := b.NewValue0(v.Pos, OpMIPSSGTUconst, typ.Bool)
v3.AuxInt = int32ToAuxInt(32)
v3.AddArg(v1)
@ -6177,7 +6160,7 @@ func rewriteValueMIPS_OpRsh8x16(v *Value) bool {
b := v.Block
typ := &b.Func.Config.Types
// match: (Rsh8x16 x y)
// result: (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt16to32 y) (MOVWconst [-1]) (SGTUconst [32] (ZeroExt16to32 y))))
// result: (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt16to32 y) (MOVWconst [31]) (SGTUconst [32] (ZeroExt16to32 y))))
for {
x := v_0
y := v_1
@ -6188,7 +6171,7 @@ func rewriteValueMIPS_OpRsh8x16(v *Value) bool {
v2 := b.NewValue0(v.Pos, OpZeroExt16to32, typ.UInt32)
v2.AddArg(y)
v3 := b.NewValue0(v.Pos, OpMIPSMOVWconst, typ.UInt32)
v3.AuxInt = int32ToAuxInt(-1)
v3.AuxInt = int32ToAuxInt(31)
v4 := b.NewValue0(v.Pos, OpMIPSSGTUconst, typ.Bool)
v4.AuxInt = int32ToAuxInt(32)
v4.AddArg(v2)
@ -6203,7 +6186,7 @@ func rewriteValueMIPS_OpRsh8x32(v *Value) bool {
b := v.Block
typ := &b.Func.Config.Types
// match: (Rsh8x32 x y)
// result: (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> y (MOVWconst [-1]) (SGTUconst [32] y)))
// result: (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> y (MOVWconst [31]) (SGTUconst [32] y)))
for {
x := v_0
y := v_1
@ -6212,7 +6195,7 @@ func rewriteValueMIPS_OpRsh8x32(v *Value) bool {
v0.AddArg(x)
v1 := b.NewValue0(v.Pos, OpMIPSCMOVZ, typ.UInt32)
v2 := b.NewValue0(v.Pos, OpMIPSMOVWconst, typ.UInt32)
v2.AuxInt = int32ToAuxInt(-1)
v2.AuxInt = int32ToAuxInt(31)
v3 := b.NewValue0(v.Pos, OpMIPSSGTUconst, typ.Bool)
v3.AuxInt = int32ToAuxInt(32)
v3.AddArg(y)
@ -6274,7 +6257,7 @@ func rewriteValueMIPS_OpRsh8x8(v *Value) bool {
b := v.Block
typ := &b.Func.Config.Types
// match: (Rsh8x8 x y)
// result: (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt8to32 y) (MOVWconst [-1]) (SGTUconst [32] (ZeroExt8to32 y))))
// result: (SRA (SignExt16to32 x) ( CMOVZ <typ.UInt32> (ZeroExt8to32 y) (MOVWconst [31]) (SGTUconst [32] (ZeroExt8to32 y))))
for {
x := v_0
y := v_1
@ -6285,7 +6268,7 @@ func rewriteValueMIPS_OpRsh8x8(v *Value) bool {
v2 := b.NewValue0(v.Pos, OpZeroExt8to32, typ.UInt32)
v2.AddArg(y)
v3 := b.NewValue0(v.Pos, OpMIPSMOVWconst, typ.UInt32)
v3.AuxInt = int32ToAuxInt(-1)
v3.AuxInt = int32ToAuxInt(31)
v4 := b.NewValue0(v.Pos, OpMIPSSGTUconst, typ.Bool)
v4.AuxInt = int32ToAuxInt(32)
v4.AddArg(v2)

848
src/cmd/compile/internal/ssa/rewriteS390X.go
View file

File diff suppressed because it is too large Load diff

3
src/cmd/dist/build.go vendored
View file

@ -1580,8 +1580,7 @@ var cgoEnabled = map[string]bool{
// List of platforms which are supported but not complete yet. These get
// filtered out of cgoEnabled for 'dist list'. See golang.org/issue/28944
var incomplete = map[string]bool{
"linux/sparc64": true,
"openbsd/mips64": true,
"linux/sparc64": true,
}
func needCC() bool {

7
src/cmd/dist/test.go vendored
View file

@ -1021,7 +1021,7 @@ func (t *tester) supportedBuildmode(mode string) bool {
case "pie":
switch pair {
case "aix/ppc64",
"linux-386", "linux-amd64", "linux-arm", "linux-arm64", "linux-ppc64le", "linux-s390x",
"linux-386", "linux-amd64", "linux-arm", "linux-arm64", "linux-ppc64le", "linux-riscv64", "linux-s390x",
"android-amd64", "android-arm", "android-arm64", "android-386":
return true
case "darwin-amd64", "darwin-arm64":
@ -1095,7 +1095,6 @@ func (t *tester) cgoTest(dt *distTest) error {
pair := gohostos + "-" + goarch
switch pair {
case "darwin-amd64", "darwin-arm64",
"openbsd-386", "openbsd-amd64",
"windows-386", "windows-amd64":
// test linkmode=external, but __thread not supported, so skip testtls.
if !t.extLink() {
@ -1118,7 +1117,8 @@ func (t *tester) cgoTest(dt *distTest) error {
"dragonfly-amd64",
"freebsd-386", "freebsd-amd64", "freebsd-arm",
"linux-386", "linux-amd64", "linux-arm", "linux-arm64", "linux-ppc64le", "linux-riscv64", "linux-s390x",
"netbsd-386", "netbsd-amd64":
"netbsd-386", "netbsd-amd64",
"openbsd-386", "openbsd-amd64", "openbsd-arm", "openbsd-arm64", "openbsd-mips64":
cmd := t.addCmd(dt, "misc/cgo/test", t.goTest())
cmd.Env = append(os.Environ(), "GOFLAGS=-ldflags=-linkmode=external")
@ -1444,7 +1444,6 @@ func (t *tester) testDirTest(dt *distTest, shard, shards int) error {
// cgoPackages is the standard packages that use cgo.
var cgoPackages = []string{
"crypto/x509",
"net",
"os/user",
}

6
src/cmd/go.mod
View file

@ -6,8 +6,8 @@ require (
github.com/google/pprof v0.0.0-20201007051231-1066cbb265c7
github.com/ianlancetaylor/demangle v0.0.0-20200414190113-039b1ae3a340 // indirect
golang.org/x/arch v0.0.0-20201008161808-52c3e6f60cff
golang.org/x/crypto v0.0.0-20200820211705-5c72a883971a
golang.org/x/crypto v0.0.0-20201016220609-9e8e0b390897
golang.org/x/mod v0.3.1-0.20200828183125-ce943fd02449
golang.org/x/sys v0.0.0-20201101102859-da207088b7d1 // indirect
golang.org/x/tools v0.0.0-20201014170642-d1624618ad65
golang.org/x/sys v0.0.0-20201110211018-35f3e6cf4a65 // indirect
golang.org/x/tools v0.0.0-20201110201400-7099162a900a
)

20
src/cmd/go.sum
View file

@ -12,26 +12,28 @@ golang.org/x/arch v0.0.0-20201008161808-52c3e6f60cff/go.mod h1:flIaEI6LNU6xOCD5P
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200820211705-5c72a883971a h1:vclmkQCjlDX5OydZ9wv8rBCcS0QyQY66Mpf/7BZbInM=
golang.org/x/crypto v0.0.0-20200820211705-5c72a883971a/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201016220609-9e8e0b390897 h1:pLI5jrR7OSLijeIDcmRxNmw2api+jEfxLoykJVice/E=
golang.org/x/crypto v0.0.0-20201016220609-9e8e0b390897/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.1-0.20200828183125-ce943fd02449 h1:xUIPaMhvROX9dhPvRCenIJtU78+lbEenGbgqB5hfHCQ=
golang.org/x/mod v0.3.1-0.20200828183125-ce943fd02449/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201101102859-da207088b7d1 h1:a/mKvvZr9Jcc8oKfcmgzyp7OwF73JPWsQLvH1z2Kxck=
golang.org/x/sys v0.0.0-20201101102859-da207088b7d1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201110211018-35f3e6cf4a65 h1:Qo9oJ566/Sq7N4hrGftVXs8GI2CXBCuOd4S2wHE/e0M=
golang.org/x/sys v0.0.0-20201110211018-35f3e6cf4a65/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20201014170642-d1624618ad65 h1:q80OtYaeeySe8Kqg0vjXehHwj5fUTqe3xOvnbi5w3Gg=
golang.org/x/tools v0.0.0-20201014170642-d1624618ad65/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/tools v0.0.0-20201110201400-7099162a900a h1:5E6TPwSBG74zT8xSrVc8W59K4ch4NFobVTnh2BYzHyU=
golang.org/x/tools v0.0.0-20201110201400-7099162a900a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=

230
src/cmd/go/alldocs.go
View file

@ -49,10 +49,11 @@
// modules modules, module versions, and more
// module-get module-aware go get
// module-auth module authentication using go.sum
// module-private module configuration for non-public modules
// packages package lists and patterns
// private configuration for downloading non-public code
// testflag testing flags
// testfunc testing functions
// vcs controlling version control with GOVCS
//
// Use "go help <topic>" for more information about that topic.
//
@ -71,7 +72,7 @@
//
// Usage:
//
// go build [-o output] [-i] [build flags] [packages]
// go build [-o output] [build flags] [packages]
//
// Build compiles the packages named by the import paths,
// along with their dependencies, but it does not install the results.
@ -98,6 +99,7 @@
// will be written to that directory.
//
// The -i flag installs the packages that are dependencies of the target.
// The -i flag is deprecated. Compiled packages are cached automatically.
//
// The build flags are shared by the build, clean, get, install, list, run,
// and test commands:
@ -617,15 +619,15 @@
// dependency should be removed entirely, downgrading or removing modules
// depending on it as needed.
//
// The version suffix @latest explicitly requests the latest minor release of the
// module named by the given path. The suffix @upgrade is like @latest but
// The version suffix @latest explicitly requests the latest minor release of
// the module named by the given path. The suffix @upgrade is like @latest but
// will not downgrade a module if it is already required at a revision or
// pre-release version newer than the latest released version. The suffix
// @patch requests the latest patch release: the latest released version
// with the same major and minor version numbers as the currently required
// version. Like @upgrade, @patch will not downgrade a module already required
// at a newer version. If the path is not already required, @upgrade and @patch
// are equivalent to @latest.
// at a newer version. If the path is not already required, @upgrade is
// equivalent to @latest, and @patch is disallowed.
//
// Although get defaults to using the latest version of the module containing
// a named package, it does not use the latest version of that module's
@ -672,6 +674,17 @@
// The second step is to download (if needed), build, and install
// the named packages.
//
// The -d flag instructs get to skip this step, downloading source code
// needed to build the named packages and their dependencies, but not
// building or installing.
//
// Building and installing packages with get is deprecated. In a future release,
// the -d flag will be enabled by default, and 'go get' will be only be used to
// adjust dependencies of the current module. To install a package using
// dependencies from the current module, use 'go install'. To install a package
// ignoring the current module, use 'go install' with an @version suffix like
// "@latest" after each argument.
//
// If an argument names a module but not a package (because there is no
// Go source code in the module's root directory), then the install step
// is skipped for that argument, instead of causing a build failure.
@ -683,10 +696,6 @@
// adds the latest golang.org/x/perf and then installs the commands in that
// latest version.
//
// The -d flag instructs get to download the source code needed to build
// the named packages, including downloading necessary dependencies,
// but not to build and install them.
//
// With no package arguments, 'go get' applies to Go package in the
// current directory, if any. In particular, 'go get -u' and
// 'go get -u=patch' update all the dependencies of that package.
@ -709,7 +718,7 @@
//
// Usage:
//
// go install [-i] [build flags] [packages]
// go install [build flags] [packages]
//
// Install compiles and installs the packages named by the import paths.
//
@ -750,6 +759,7 @@
// other packages are built and cached but not installed.
//
// The -i flag installs the dependencies of the named packages as well.
// The -i flag is deprecated. Compiled packages are cached automatically.
//
// For more about the build flags, see 'go help build'.
// For more about specifying packages, see 'go help packages'.
@ -1145,7 +1155,7 @@
//
// The -retract=version and -dropretract=version flags add and drop a
// retraction on the given version. The version may be a single version
// like "v1.2.3" or a closed interval like "[v1.1.0-v1.1.9]". Note that
// like "v1.2.3" or a closed interval like "[v1.1.0,v1.1.9]". Note that
// -retract=version is a no-op if that retraction already exists.
//
// The -require, -droprequire, -exclude, -dropexclude, -replace,
@ -1220,12 +1230,17 @@
//
// go mod init [module]
//
// Init initializes and writes a new go.mod to the current directory,
// in effect creating a new module rooted at the current directory.
// The file go.mod must not already exist.
// If possible, init will guess the module path from import comments
// (see 'go help importpath') or from version control configuration.
// To override this guess, supply the module path as an argument.
// Init initializes and writes a new go.mod file in the current directory, in
// effect creating a new module rooted at the current directory. The go.mod file
// must not already exist.
//
// Init accepts one optional argument, the module path for the new module. If the
// module path argument is omitted, init will attempt to infer the module path
// using import comments in .go files, vendoring tool configuration files (like
// Gopkg.lock), and the current directory (if in GOPATH).
//
// If a configuration file for a vendoring tool is present, init will attempt to
// import module requirements from it.
//
//
// Add missing and remove unused modules
@ -1451,6 +1466,7 @@
// -i
// Install packages that are dependencies of the test.
// Do not run the test.
// The -i flag is deprecated. Compiled packages are cached automatically.
//
// -json
// Convert test output to JSON suitable for automated processing.
@ -1799,7 +1815,7 @@
// Comma-separated list of glob patterns (in the syntax of Go's path.Match)
// of module path prefixes that should always be fetched directly
// or that should not be compared against the checksum database.
// See 'go help module-private'.
// See 'go help private'.
// GOROOT
// The root of the go tree.
// GOSUMDB
@ -1905,6 +1921,8 @@
// If module-aware mode is disabled, GOMOD will be the empty string.
// GOTOOLDIR
// The directory where the go tools (compile, cover, doc, etc...) are installed.
// GOVERSION
// The version of the installed Go tree, as reported by runtime.Version.
//
//
// File types
@ -1976,7 +1994,7 @@
// like in Go imports:
//
// require (
// new/thing v2.3.4
// new/thing/v2 v2.3.4
// old/thing v1.2.3
// )
//
@ -2869,7 +2887,7 @@
// followed by a pipe character, indicating it is safe to fall back on any error.
//
// The GOPRIVATE and GONOPROXY environment variables allow bypassing
// the proxy for selected modules. See 'go help module-private' for details.
// the proxy for selected modules. See 'go help private' for details.
//
// No matter the source of the modules, the go command checks downloads against
// known checksums, to detect unexpected changes in the content of any specific
@ -2989,52 +3007,7 @@
// accepted, at the cost of giving up the security guarantee of verified repeatable
// downloads for all modules. A better way to bypass the checksum database
// for specific modules is to use the GOPRIVATE or GONOSUMDB environment
// variables. See 'go help module-private' for details.
//
// The 'go env -w' command (see 'go help env') can be used to set these variables
// for future go command invocations.
//
//
// Module configuration for non-public modules
//
// The go command defaults to downloading modules from the public Go module
// mirror at proxy.golang.org. It also defaults to validating downloaded modules,
// regardless of source, against the public Go checksum database at sum.golang.org.
// These defaults work well for publicly available source code.
//
// The GOPRIVATE environment variable controls which modules the go command
// considers to be private (not available publicly) and should therefore not use the
// proxy or checksum database. The variable is a comma-separated list of
// glob patterns (in the syntax of Go's path.Match) of module path prefixes.
// For example,
//
// GOPRIVATE=*.corp.example.com,rsc.io/private
//
// causes the go command to treat as private any module with a path prefix
// matching either pattern, including git.corp.example.com/xyzzy, rsc.io/private,
// and rsc.io/private/quux.
//
// The GOPRIVATE environment variable may be used by other tools as well to
// identify non-public modules. For example, an editor could use GOPRIVATE
// to decide whether to hyperlink a package import to a godoc.org page.
//
// For fine-grained control over module download and validation, the GONOPROXY
// and GONOSUMDB environment variables accept the same kind of glob list
// and override GOPRIVATE for the specific decision of whether to use the proxy
// and checksum database, respectively.
//
// For example, if a company ran a module proxy serving private modules,
// users would configure go using:
//
// GOPRIVATE=*.corp.example.com
// GOPROXY=proxy.example.com
// GONOPROXY=none
//
// This would tell the go command and other tools that modules beginning with
// a corp.example.com subdomain are private but that the company proxy should
// be used for downloading both public and private modules, because
// GONOPROXY has been set to a pattern that won't match any modules,
// overriding GOPRIVATE.
// variables. See 'go help private' for details.
//
// The 'go env -w' command (see 'go help env') can be used to set these variables
// for future go command invocations.
@ -3124,6 +3097,56 @@
// by the go tool, as are directories named "testdata".
//
//
// Configuration for downloading non-public code
//
// The go command defaults to downloading modules from the public Go module
// mirror at proxy.golang.org. It also defaults to validating downloaded modules,
// regardless of source, against the public Go checksum database at sum.golang.org.
// These defaults work well for publicly available source code.
//
// The GOPRIVATE environment variable controls which modules the go command
// considers to be private (not available publicly) and should therefore not use the
// proxy or checksum database. The variable is a comma-separated list of
// glob patterns (in the syntax of Go's path.Match) of module path prefixes.
// For example,
//
// GOPRIVATE=*.corp.example.com,rsc.io/private
//
// causes the go command to treat as private any module with a path prefix
// matching either pattern, including git.corp.example.com/xyzzy, rsc.io/private,
// and rsc.io/private/quux.
//
// The GOPRIVATE environment variable may be used by other tools as well to
// identify non-public modules. For example, an editor could use GOPRIVATE
// to decide whether to hyperlink a package import to a godoc.org page.
//
// For fine-grained control over module download and validation, the GONOPROXY
// and GONOSUMDB environment variables accept the same kind of glob list
// and override GOPRIVATE for the specific decision of whether to use the proxy
// and checksum database, respectively.
//
// For example, if a company ran a module proxy serving private modules,
// users would configure go using:
//
// GOPRIVATE=*.corp.example.com
// GOPROXY=proxy.example.com
// GONOPROXY=none
//
// This would tell the go command and other tools that modules beginning with
// a corp.example.com subdomain are private but that the company proxy should
// be used for downloading both public and private modules, because
// GONOPROXY has been set to a pattern that won't match any modules,
// overriding GOPRIVATE.
//
// The GOPRIVATE variable is also used to define the "public" and "private"
// patterns for the GOVCS variable; see 'go help vcs'. For that usage,
// GOPRIVATE applies even in GOPATH mode. In that case, it matches import paths
// instead of module paths.
//
// The 'go env -w' command (see 'go help env') can be used to set these variables
// for future go command invocations.
//
//
// Testing flags
//
// The 'go test' command takes both flags that apply to 'go test' itself
@ -3416,4 +3439,77 @@
// See the documentation of the testing package for more information.
//
//
// Controlling version control with GOVCS
//
// The 'go get' command can run version control commands like git
// to download imported code. This functionality is critical to the decentralized
// Go package ecosystem, in which code can be imported from any server,
// but it is also a potential security problem, if a malicious server finds a
// way to cause the invoked version control command to run unintended code.
//
// To balance the functionality and security concerns, the 'go get' command
// by default will only use git and hg to download code from public servers.
// But it will use any known version control system (bzr, fossil, git, hg, svn)
// to download code from private servers, defined as those hosting packages
// matching the GOPRIVATE variable (see 'go help private'). The rationale behind
// allowing only Git and Mercurial is that these two systems have had the most
// attention to issues of being run as clients of untrusted servers. In contrast,
// Bazaar, Fossil, and Subversion have primarily been used in trusted,
// authenticated environments and are not as well scrutinized as attack surfaces.
//
// The version control command restrictions only apply when using direct version
// control access to download code. When downloading modules from a proxy,
// 'go get' uses the proxy protocol instead, which is always permitted.
// By default, the 'go get' command uses the Go module mirror (proxy.golang.org)
// for public packages and only falls back to version control for private
// packages or when the mirror refuses to serve a public package (typically for
// legal reasons). Therefore, clients can still access public code served from
// Bazaar, Fossil, or Subversion repositories by default, because those downloads
// use the Go module mirror, which takes on the security risk of running the
// version control commands, using a custom sandbox.
//
// The GOVCS variable can be used to change the allowed version control systems
// for specific packages (identified by a module or import path).
// The GOVCS variable applies both when using modules and when using GOPATH.
// When using modules, the patterns match against the module path.
// When using GOPATH, the patterns match against the import path
// corresponding to the root of the version control repository.
//
// The general form of the GOVCS setting is a comma-separated list of
// pattern:vcslist rules. The pattern is a glob pattern that must match
// one or more leading elements of the module or import path. The vcslist
// is a pipe-separated list of allowed version control commands, or "all"
// to allow use of any known command, or "off" to allow nothing.
// The earliest matching pattern in the list applies, even if later patterns
// might also match.
//
// For example, consider:
//
// GOVCS=github.com:git,evil.com:off,*:git|hg
//
// With this setting, code with an module or import path beginning with
// github.com/ can only use git; paths on evil.com cannot use any version
// control command, and all other paths (* matches everything) can use
// only git or hg.
//
// The special patterns "public" and "private" match public and private
// module or import paths. A path is private if it matches the GOPRIVATE
// variable; otherwise it is public.
//
// If no rules in the GOVCS variable match a particular module or import path,
// the 'go get' command applies its default rule, which can now be summarized
// in GOVCS notation as 'public:git|hg,private:all'.
//
// To allow unfettered use of any version control system for any package, use:
//
// GOVCS=*:all
//
// To disable all use of version control, use:
//
// GOVCS=*:off
//
// The 'go env -w' command (see 'go help env') can be used to set the GOVCS
// variable for future go command invocations.
//
//
package main

22
src/cmd/go/go_test.go
View file

@ -35,6 +35,14 @@ import (
"cmd/internal/sys"
)
func init() {
// GOVCS defaults to public:git|hg,private:all,
// which breaks many tests here - they can't use non-git, non-hg VCS at all!
// Change to fully permissive.
// The tests of the GOVCS setting itself are in ../../testdata/script/govcs.txt.
os.Setenv("GOVCS", "*:all")
}
var (
canRace = false // whether we can run the race detector
canCgo = false // whether we can use cgo
@ -1878,6 +1886,18 @@ func TestGoEnv(t *testing.T) {
tg.grepStdout("gcc", "CC not found")
tg.run("env", "GOGCCFLAGS")
tg.grepStdout("-ffaster", "CC arguments not found")
tg.run("env", "GOVERSION")
envVersion := strings.TrimSpace(tg.stdout.String())
tg.run("version")
cmdVersion := strings.TrimSpace(tg.stdout.String())
// If 'go version' is "go version <version> <goos>/<goarch>", then
// 'go env GOVERSION' is just "<version>".
if cmdVersion == envVersion || !strings.Contains(cmdVersion, envVersion) {
t.Fatalf("'go env GOVERSION' %q should be a shorter substring of 'go version' %q", envVersion, cmdVersion)
}
}
const (
@ -2022,7 +2042,7 @@ func TestBuildmodePIE(t *testing.T) {
platform := fmt.Sprintf("%s/%s", runtime.GOOS, runtime.GOARCH)
switch platform {
case "linux/386", "linux/amd64", "linux/arm", "linux/arm64", "linux/ppc64le", "linux/s390x",
case "linux/386", "linux/amd64", "linux/arm", "linux/arm64", "linux/ppc64le", "linux/riscv64", "linux/s390x",
"android/amd64", "android/arm", "android/arm64", "android/386",
"freebsd/amd64",
"windows/386", "windows/amd64", "windows/arm":

1
src/cmd/go/internal/cfg/cfg.go
View file

@ -268,6 +268,7 @@ var (
GONOPROXY = envOr("GONOPROXY", GOPRIVATE)
GONOSUMDB = envOr("GONOSUMDB", GOPRIVATE)
GOINSECURE = Getenv("GOINSECURE")
GOVCS = Getenv("GOVCS")
)
var SumdbDir = gopathDir("pkg/sumdb")

4
src/cmd/go/internal/envcmd/env.go
View file

@ -87,6 +87,8 @@ func MkEnv() []cfg.EnvVar {
{Name: "GOSUMDB", Value: cfg.GOSUMDB},
{Name: "GOTMPDIR", Value: cfg.Getenv("GOTMPDIR")},
{Name: "GOTOOLDIR", Value: base.ToolDir},
{Name: "GOVCS", Value: cfg.GOVCS},
{Name: "GOVERSION", Value: runtime.Version()},
}
if work.GccgoBin != "" {
@ -398,7 +400,7 @@ func getOrigEnv(key string) string {
func checkEnvWrite(key, val string) error {
switch key {
case "GOEXE", "GOGCCFLAGS", "GOHOSTARCH", "GOHOSTOS", "GOMOD", "GOTOOLDIR":
case "GOEXE", "GOGCCFLAGS", "GOHOSTARCH", "GOHOSTOS", "GOMOD", "GOTOOLDIR", "GOVERSION":
return fmt.Errorf("%s cannot be modified", key)
case "GOENV":
return fmt.Errorf("%s can only be set using the OS environment", key)

4
src/cmd/go/internal/help/helpdoc.go
View file

@ -526,7 +526,7 @@ General-purpose environment variables:
Comma-separated list of glob patterns (in the syntax of Go's path.Match)
of module path prefixes that should always be fetched directly
or that should not be compared against the checksum database.
See 'go help module-private'.
See 'go help private'.
GOROOT
The root of the go tree.
GOSUMDB
@ -632,6 +632,8 @@ Additional information available from 'go env' but not read from the environment
If module-aware mode is disabled, GOMOD will be the empty string.
GOTOOLDIR
The directory where the go tools (compile, cover, doc, etc...) are installed.
GOVERSION
The version of the installed Go tree, as reported by runtime.Version.
`,
}

7
src/cmd/go/internal/modcmd/download.go
View file

@ -88,12 +88,11 @@ func runDownload(ctx context.Context, cmd *base.Command, args []string) {
args = []string{"all"}
} else if modload.HasModRoot() {
modload.LoadModFile(ctx) // to fill Target
targetAtLatest := modload.Target.Path + "@latest"
targetAtUpgrade := modload.Target.Path + "@upgrade"
targetAtPatch := modload.Target.Path + "@patch"
for _, arg := range args {
switch arg {
case modload.Target.Path, targetAtLatest, targetAtUpgrade, targetAtPatch:
case modload.Target.Path, targetAtUpgrade, targetAtPatch:
os.Stderr.WriteString("go mod download: skipping argument " + arg + " that resolves to the main module\n")
}
}
@ -170,7 +169,7 @@ func runDownload(ctx context.Context, cmd *base.Command, args []string) {
for _, m := range mods {
b, err := json.MarshalIndent(m, "", "\t")
if err != nil {
base.Fatalf("%v", err)
base.Fatalf("go mod download: %v", err)
}
os.Stdout.Write(append(b, '\n'))
if m.Error != "" {
@ -180,7 +179,7 @@ func runDownload(ctx context.Context, cmd *base.Command, args []string) {
} else {
for _, m := range mods {
if m.Error != "" {
base.Errorf("%s", m.Error)
base.Errorf("go mod download: %v", m.Error)
}
}
base.ExitIfErrors()

2
src/cmd/go/internal/modcmd/edit.go
View file

@ -69,7 +69,7 @@ a version on the left side is dropped.
The -retract=version and -dropretract=version flags add and drop a
retraction on the given version. The version may be a single version
like "v1.2.3" or a closed interval like "[v1.1.0-v1.1.9]". Note that
like "v1.2.3" or a closed interval like "[v1.1.0,v1.1.9]". Note that
-retract=version is a no-op if that retraction already exists.
The -require, -droprequire, -exclude, -dropexclude, -replace,

19
src/cmd/go/internal/modcmd/init.go
View file

@ -16,13 +16,18 @@ var cmdInit = &base.Command{
UsageLine: "go mod init [module]",
Short: "initialize new module in current directory",
Long: `
Init initializes and writes a new go.mod to the current directory,
in effect creating a new module rooted at the current directory.
The file go.mod must not already exist.
If possible, init will guess the module path from import comments
(see 'go help importpath') or from version control configuration.
To override this guess, supply the module path as an argument.
`,
Init initializes and writes a new go.mod file in the current directory, in
effect creating a new module rooted at the current directory. The go.mod file
must not already exist.
Init accepts one optional argument, the module path for the new module. If the
module path argument is omitted, init will attempt to infer the module path
using import comments in .go files, vendoring tool configuration files (like
Gopkg.lock), and the current directory (if in GOPATH).
If a configuration file for a vendoring tool is present, init will attempt to
import module requirements from it.
`,
Run: runInit,
}

48
src/cmd/go/internal/modcmd/vendor.go
View file

@ -73,7 +73,7 @@ func runVendor(ctx context.Context, cmd *base.Command, args []string) {
modpkgs := make(map[module.Version][]string)
for _, pkg := range pkgs {
m := modload.PackageModule(pkg)
if m == modload.Target {
if m.Path == "" || m == modload.Target {
continue
}
modpkgs[m] = append(modpkgs[m], pkg)
@ -91,28 +91,38 @@ func runVendor(ctx context.Context, cmd *base.Command, args []string) {
includeAllReplacements = true
}
var vendorMods []module.Version
for m := range isExplicit {
vendorMods = append(vendorMods, m)
}
for m := range modpkgs {
if !isExplicit[m] {
vendorMods = append(vendorMods, m)
}
}
module.Sort(vendorMods)
var buf bytes.Buffer
for _, m := range modload.LoadedModules()[1:] {
if pkgs := modpkgs[m]; len(pkgs) > 0 || isExplicit[m] {
line := moduleLine(m, modload.Replacement(m))
buf.WriteString(line)
for _, m := range vendorMods {
line := moduleLine(m, modload.Replacement(m))
buf.WriteString(line)
if cfg.BuildV {
os.Stderr.WriteString(line)
}
if isExplicit[m] {
buf.WriteString("## explicit\n")
if cfg.BuildV {
os.Stderr.WriteString(line)
os.Stderr.WriteString("## explicit\n")
}
if isExplicit[m] {
buf.WriteString("## explicit\n")
if cfg.BuildV {
os.Stderr.WriteString("## explicit\n")
}
}
sort.Strings(pkgs)
for _, pkg := range pkgs {
fmt.Fprintf(&buf, "%s\n", pkg)
if cfg.BuildV {
fmt.Fprintf(os.Stderr, "%s\n", pkg)
}
vendorPkg(vdir, pkg)
}
pkgs := modpkgs[m]
sort.Strings(pkgs)
for _, pkg := range pkgs {
fmt.Fprintf(&buf, "%s\n", pkg)
if cfg.BuildV {
fmt.Fprintf(os.Stderr, "%s\n", pkg)
}
vendorPkg(vdir, pkg)
}
}

2
src/cmd/go/internal/modfetch/cache.go
View file

@ -483,7 +483,7 @@ func readDiskStatByHash(path, rev string) (file string, info *RevInfo, err error
for _, name := range names {
if strings.HasSuffix(name, suffix) {
v := strings.TrimSuffix(name, ".info")
if IsPseudoVersion(v) && semver.Max(maxVersion, v) == v {
if IsPseudoVersion(v) && semver.Compare(v, maxVersion) > 0 {
maxVersion = v
file, info, err = readDiskStat(path, strings.TrimSuffix(name, ".info"))
}

2
src/cmd/go/internal/modfetch/codehost/vcs.go
View file

@ -568,7 +568,7 @@ func bzrParseStat(rev, out string) (*RevInfo, error) {
func fossilParseStat(rev, out string) (*RevInfo, error) {
for _, line := range strings.Split(out, "\n") {
if strings.HasPrefix(line, "uuid:") {
if strings.HasPrefix(line, "uuid:") || strings.HasPrefix(line, "hash:") {
f := strings.Fields(line)
if len(f) != 5 || len(f[1]) != 40 || f[4] != "UTC" {
return nil, vcsErrorf("unexpected response from fossil info: %q", line)

13
src/cmd/go/internal/modfetch/fetch.go
View file

@ -848,16 +848,16 @@ the checksum database is not consulted, and all unrecognized modules are
accepted, at the cost of giving up the security guarantee of verified repeatable
downloads for all modules. A better way to bypass the checksum database
for specific modules is to use the GOPRIVATE or GONOSUMDB environment
variables. See 'go help module-private' for details.
variables. See 'go help private' for details.
The 'go env -w' command (see 'go help env') can be used to set these variables
for future go command invocations.
`,
}
var HelpModulePrivate = &base.Command{
UsageLine: "module-private",
Short: "module configuration for non-public modules",
var HelpPrivate = &base.Command{
UsageLine: "private",
Short: "configuration for downloading non-public code",
Long: `
The go command defaults to downloading modules from the public Go module
mirror at proxy.golang.org. It also defaults to validating downloaded modules,
@ -898,6 +898,11 @@ be used for downloading both public and private modules, because
GONOPROXY has been set to a pattern that won't match any modules,
overriding GOPRIVATE.
The GOPRIVATE variable is also used to define the "public" and "private"
patterns for the GOVCS variable; see 'go help vcs'. For that usage,
GOPRIVATE applies even in GOPATH mode. In that case, it matches import paths
instead of module paths.
The 'go env -w' command (see 'go help env') can be used to set these variables
for future go command invocations.
`,

2113
src/cmd/go/internal/modget/get.go
View file

File diff suppressed because it is too large Load diff

202
src/cmd/go/internal/modget/mvs.go
View file

@ -1,202 +0,0 @@
// Copyright 2020 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package modget
import (
"context"
"errors"
"cmd/go/internal/base"
"cmd/go/internal/modload"
"cmd/go/internal/mvs"
"golang.org/x/mod/module"
)
// An upgrader adapts an underlying mvs.Reqs to apply an
// upgrade policy to a list of targets and their dependencies.
type upgrader struct {
mvs.Reqs
// cmdline maps a module path to a query made for that module at a
// specific target version. Each query corresponds to a module
// matched by a command line argument.
cmdline map[string]*query
// upgrade is a set of modules providing dependencies of packages
// matched by command line arguments. If -u or -u=patch is set,
// these modules are upgraded accordingly.
upgrade map[string]bool
}
// newUpgrader creates an upgrader. cmdline contains queries made at
// specific versions for modules matched by command line arguments. pkgs
// is the set of packages matched by command line arguments. If -u or -u=patch
// is set, modules providing dependencies of pkgs are upgraded accordingly.
func newUpgrader(cmdline map[string]*query, pkgs map[string]bool) *upgrader {
u := &upgrader{
Reqs: modload.Reqs(),
cmdline: cmdline,
}
if getU != "" {
u.upgrade = make(map[string]bool)
// Traverse package import graph.
// Initialize work queue with root packages.
seen := make(map[string]bool)
var work []string
add := func(path string) {
if !seen[path] {
seen[path] = true
work = append(work, path)
}
}
for pkg := range pkgs {
add(pkg)
}
for len(work) > 0 {
pkg := work[0]
work = work[1:]
m := modload.PackageModule(pkg)
u.upgrade[m.Path] = true
// testImports is empty unless test imports were actually loaded,
// i.e., -t was set or "all" was one of the arguments.
imports, testImports := modload.PackageImports(pkg)
for _, imp := range imports {
add(imp)
}
for _, imp := range testImports {
add(imp)
}
}
}
return u
}
// Required returns the requirement list for m.
// For the main module, we override requirements with the modules named
// one the command line, and we include new requirements. Otherwise,
// we defer to u.Reqs.
func (u *upgrader) Required(m module.Version) ([]module.Version, error) {
rs, err := u.Reqs.Required(m)
if err != nil {
return nil, err
}
if m != modload.Target {
return rs, nil
}
overridden := make(map[string]bool)
for i, m := range rs {
if q := u.cmdline[m.Path]; q != nil && q.m.Version != "none" {
rs[i] = q.m
overridden[q.m.Path] = true
}
}
for _, q := range u.cmdline {
if !overridden[q.m.Path] && q.m.Path != modload.Target.Path && q.m.Version != "none" {
rs = append(rs, q.m)
}
}
return rs, nil
}
// Upgrade returns the desired upgrade for m.
//
// If m was requested at a specific version on the command line, then
// Upgrade returns that version.
//
// If -u is set and m provides a dependency of a package matched by
// command line arguments, then Upgrade may provider a newer tagged version.
// If m is a tagged version, then Upgrade will return the latest tagged
// version (with the same minor version number if -u=patch).
// If m is a pseudo-version, then Upgrade returns the latest tagged version
// only if that version has a time-stamp newer than m. This special case
// prevents accidental downgrades when already using a pseudo-version
// newer than the latest tagged version.
//
// If none of the above cases apply, then Upgrade returns m.
func (u *upgrader) Upgrade(m module.Version) (module.Version, error) {
// Allow pkg@vers on the command line to override the upgrade choice v.
// If q's version is < m.Version, then we're going to downgrade anyway,
// and it's cleaner to avoid moving back and forth and picking up
// extraneous other newer dependencies.
// If q's version is > m.Version, then we're going to upgrade past
// m.Version anyway, and again it's cleaner to avoid moving back and forth
// picking up extraneous other newer dependencies.
if q := u.cmdline[m.Path]; q != nil {
return q.m, nil
}
if !u.upgrade[m.Path] {
// Not involved in upgrade. Leave alone.
return m, nil
}
// Run query required by upgrade semantics.
// Note that Query "latest" is not the same as using repo.Latest,
// which may return a pseudoversion for the latest commit.
// Query "latest" returns the newest tagged version or the newest
// prerelease version if there are no non-prereleases, or repo.Latest
// if there aren't any tagged versions.
// If we're querying "upgrade" or "patch", Query will compare the current
// version against the chosen version and will return the current version
// if it is newer.
info, err := modload.Query(context.TODO(), m.Path, string(getU), m.Version, checkAllowedOrCurrent(m.Version))
if err != nil {
// Report error but return m, to let version selection continue.
// (Reporting the error will fail the command at the next base.ExitIfErrors.)
// Special case: if the error is for m.Version itself and m.Version has a
// replacement, then keep it and don't report the error: the fact that the
// version is invalid is likely the reason it was replaced to begin with.
var vErr *module.InvalidVersionError
if errors.As(err, &vErr) && vErr.Version == m.Version && modload.Replacement(m).Path != "" {
return m, nil
}
// Special case: if the error is "no matching versions" then don't
// even report the error. Because Query does not consider pseudo-versions,
// it may happen that we have a pseudo-version but during -u=patch
// the query v0.0 matches no versions (not even the one we're using).
var noMatch *modload.NoMatchingVersionError
if !errors.As(err, &noMatch) {
base.Errorf("go get: upgrading %s@%s: %v", m.Path, m.Version, err)
}
return m, nil
}
if info.Version != m.Version {
logOncef("go: %s %s => %s", m.Path, getU, info.Version)
}
return module.Version{Path: m.Path, Version: info.Version}, nil
}
// buildListForLostUpgrade returns the build list for the module graph
// rooted at lost. Unlike mvs.BuildList, the target module (lost) is not
// treated specially. The returned build list may contain a newer version
// of lost.
//
// buildListForLostUpgrade is used after a downgrade has removed a module
// requested at a specific version. This helps us understand the requirements
// implied by each downgrade.
func buildListForLostUpgrade(lost module.Version, reqs mvs.Reqs) ([]module.Version, error) {
return mvs.BuildList(lostUpgradeRoot, &lostUpgradeReqs{Reqs: reqs, lost: lost})
}
var lostUpgradeRoot = module.Version{Path: "lost-upgrade-root", Version: ""}
type lostUpgradeReqs struct {
mvs.Reqs
lost module.Version
}
func (r *lostUpgradeReqs) Required(mod module.Version) ([]module.Version, error) {
if mod == lostUpgradeRoot {
return []module.Version{r.lost}, nil
}
return r.Reqs.Required(mod)
}

357
src/cmd/go/internal/modget/query.go Normal file
View file

@ -0,0 +1,357 @@
// Copyright 2020 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package modget
import (
"fmt"
"path/filepath"
"regexp"
"strings"
"sync"
"cmd/go/internal/base"
"cmd/go/internal/modload"
"cmd/go/internal/search"
"cmd/go/internal/str"
"golang.org/x/mod/module"
)
// A query describes a command-line argument and the modules and/or packages
// to which that argument may resolve..
type query struct {
// raw is the original argument, to be printed in error messages.
raw string
// rawVersion is the portion of raw corresponding to version, if any
rawVersion string
// pattern is the part of the argument before "@" (or the whole argument
// if there is no "@"), which may match either packages (preferred) or
// modules (if no matching packages).
//
// The pattern may also be "-u", for the synthetic query representing the -u
// (“upgrade”)flag.
pattern string
// patternIsLocal indicates whether pattern is restricted to match only paths
// local to the main module, such as absolute filesystem paths or paths
// beginning with './'.
//
// A local pattern must resolve to one or more packages in the main module.
patternIsLocal bool
// version is the part of the argument after "@", or an implied
// "upgrade" or "patch" if there is no "@". version specifies the
// module version to get.
version string
// matchWildcard, if non-nil, reports whether pattern, which must be a
// wildcard (with the substring "..."), matches the given package or module
// path.
matchWildcard func(path string) bool
// canMatchWildcard, if non-nil, reports whether the module with the given
// path could lexically contain a package matching pattern, which must be a
// wildcard.
canMatchWildcardInModule func(mPath string) bool
// conflict is the first query identified as incompatible with this one.
// conflict forces one or more of the modules matching this query to a
// version that does not match version.
conflict *query
// candidates is a list of sets of alternatives for a path that matches (or
// contains packages that match) the pattern. The query can be resolved by
// choosing exactly one alternative from each set in the list.
//
// A path-literal query results in only one set: the path itself, which
// may resolve to either a package path or a module path.
//
// A wildcard query results in one set for each matching module path, each
// module for which the matching version contains at least one matching
// package, and (if no other modules match) one candidate set for the pattern
// overall if no existing match is identified in the build list.
//
// A query for pattern "all" results in one set for each package transitively
// imported by the main module.
//
// The special query for the "-u" flag results in one set for each
// otherwise-unconstrained package that has available upgrades.
candidates []pathSet
candidatesMu sync.Mutex
// pathSeen ensures that only one pathSet is added to the query per
// unique path.
pathSeen sync.Map
// resolved contains the set of modules whose versions have been determined by
// this query, in the order in which they were determined.
//
// The resolver examines the candidate sets for each query, resolving one
// module per candidate set in a way that attempts to avoid obvious conflicts
// between the versions resolved by different queries.
resolved []module.Version
// matchesPackages is true if the resolved modules provide at least one
// package mathcing q.pattern.
matchesPackages bool
}
// A pathSet describes the possible options for resolving a specific path
// to a package and/or module.
type pathSet struct {
// path is a package (if "all" or "-u" or a non-wildcard) or module (if
// wildcard) path that could be resolved by adding any of the modules in this
// set. For a wildcard pattern that so far matches no packages, the path is
// the wildcard pattern itself.
//
// Each path must occur only once in a query's candidate sets, and the path is
// added implicitly to each pathSet returned to pathOnce.
path string
// pkgMods is a set of zero or more modules, each of which contains the
// package with the indicated path. Due to the requirement that imports be
// unambiguous, only one such module can be in the build list, and all others
// must be excluded.
pkgMods []module.Version
// mod is either the zero Version, or a module that does not contain any
// packages matching the query but for which the module path itself
// matches the query pattern.
//
// We track this module separately from pkgMods because, all else equal, we
// prefer to match a query to a package rather than just a module. Also,
// unlike the modules in pkgMods, this module does not inherently exclude
// any other module in pkgMods.
mod module.Version
err error
}
// errSet returns a pathSet containing the given error.
func errSet(err error) pathSet { return pathSet{err: err} }
// newQuery returns a new query parsed from the raw argument,
// which must be either path or path@version.
func newQuery(raw string) (*query, error) {
pattern := raw
rawVers := ""
if i := strings.Index(raw, "@"); i >= 0 {
pattern, rawVers = raw[:i], raw[i+1:]
if strings.Contains(rawVers, "@") || rawVers == "" {
return nil, fmt.Errorf("invalid module version syntax %q", raw)
}
}
// If no version suffix is specified, assume @upgrade.
// If -u=patch was specified, assume @patch instead.
version := rawVers
if version == "" {
if getU.version == "" {
version = "upgrade"
} else {
version = getU.version
}
}
q := &query{
raw: raw,
rawVersion: rawVers,
pattern: pattern,
patternIsLocal: filepath.IsAbs(pattern) || search.IsRelativePath(pattern),
version: version,
}
if strings.Contains(q.pattern, "...") {
q.matchWildcard = search.MatchPattern(q.pattern)
q.canMatchWildcardInModule = search.TreeCanMatchPattern(q.pattern)
}
if err := q.validate(); err != nil {
return q, err
}
return q, nil
}
// validate reports a non-nil error if q is not sensible and well-formed.
func (q *query) validate() error {
if q.patternIsLocal {
if q.rawVersion != "" {
return fmt.Errorf("can't request explicit version %q of path %q in main module", q.rawVersion, q.pattern)
}
return nil
}
if q.pattern == "all" {
// If there is no main module, "all" is not meaningful.
if !modload.HasModRoot() {
return fmt.Errorf(`cannot match "all": working directory is not part of a module`)
}
if !versionOkForMainModule(q.version) {
// TODO(bcmills): "all@none" seems like a totally reasonable way to
// request that we remove all module requirements, leaving only the main
// module and standard library. Perhaps we should implement that someday.
return &modload.QueryMatchesMainModuleError{
Pattern: q.pattern,
Query: q.version,
}
}
}
if search.IsMetaPackage(q.pattern) && q.pattern != "all" {
if q.pattern != q.raw {
return fmt.Errorf("can't request explicit version of standard-library pattern %q", q.pattern)
}
}
return nil
}
// String returns the original argument from which q was parsed.
func (q *query) String() string { return q.raw }
// ResolvedString returns a string describing m as a resolved match for q.
func (q *query) ResolvedString(m module.Version) string {
if m.Path != q.pattern {
if m.Version != q.version {
return fmt.Sprintf("%v (matching %s@%s)", m, q.pattern, q.version)
}
return fmt.Sprintf("%v (matching %v)", m, q)
}
if m.Version != q.version {
return fmt.Sprintf("%s@%s (%s)", q.pattern, q.version, m.Version)
}
return q.String()
}
// isWildcard reports whether q is a pattern that can match multiple paths.
func (q *query) isWildcard() bool {
return q.matchWildcard != nil || (q.patternIsLocal && strings.Contains(q.pattern, "..."))
}
// matchesPath reports whether the given path matches q.pattern.
func (q *query) matchesPath(path string) bool {
if q.matchWildcard != nil {
return q.matchWildcard(path)
}
return path == q.pattern
}
// canMatchInModule reports whether the given module path can potentially
// contain q.pattern.
func (q *query) canMatchInModule(mPath string) bool {
if q.canMatchWildcardInModule != nil {
return q.canMatchWildcardInModule(mPath)
}
return str.HasPathPrefix(q.pattern, mPath)
}
// pathOnce invokes f to generate the pathSet for the given path,
// if one is still needed.
//
// Note that, unlike sync.Once, pathOnce does not guarantee that a concurrent
// call to f for the given path has completed on return.
//
// pathOnce is safe for concurrent use by multiple goroutines, but note that
// multiple concurrent calls will result in the sets being added in
// nondeterministic order.
func (q *query) pathOnce(path string, f func() pathSet) {
if _, dup := q.pathSeen.LoadOrStore(path, nil); dup {
return
}
cs := f()
if len(cs.pkgMods) > 0 || cs.mod != (module.Version{}) || cs.err != nil {
cs.path = path
q.candidatesMu.Lock()
q.candidates = append(q.candidates, cs)
q.candidatesMu.Unlock()
}
}
// reportError logs err concisely using base.Errorf.
func reportError(q *query, err error) {
errStr := err.Error()
// If err already mentions all of the relevant parts of q, just log err to
// reduce stutter. Otherwise, log both q and err.
//
// TODO(bcmills): Use errors.As to unpack these errors instead of parsing
// strings with regular expressions.
patternRE := regexp.MustCompile("(?m)(?:[ \t(\"`]|^)" + regexp.QuoteMeta(q.pattern) + "(?:[ @:)\"`]|$)")
if patternRE.MatchString(errStr) {
if q.rawVersion == "" {
base.Errorf("go get: %s", errStr)
return
}
versionRE := regexp.MustCompile("(?m)(?:[ @(\"`]|^)" + regexp.QuoteMeta(q.version) + "(?:[ :)\"`]|$)")
if versionRE.MatchString(errStr) {
base.Errorf("go get: %s", errStr)
return
}
}
if qs := q.String(); qs != "" {
base.Errorf("go get %s: %s", qs, errStr)
} else {
base.Errorf("go get: %s", errStr)
}
}
func reportConflict(pq *query, m module.Version, conflict versionReason) {
if pq.conflict != nil {
// We've already reported a conflict for the proposed query.
// Don't report it again, even if it has other conflicts.
return
}
pq.conflict = conflict.reason
proposed := versionReason{
version: m.Version,
reason: pq,
}
if pq.isWildcard() && !conflict.reason.isWildcard() {
// Prefer to report the specific path first and the wildcard second.
proposed, conflict = conflict, proposed
}
reportError(pq, &conflictError{
mPath: m.Path,
proposed: proposed,
conflict: conflict,
})
}
type conflictError struct {
mPath string
proposed versionReason
conflict versionReason
}
func (e *conflictError) Error() string {
argStr := func(q *query, v string) string {
if v != q.version {
return fmt.Sprintf("%s@%s (%s)", q.pattern, q.version, v)
}
return q.String()
}
pq := e.proposed.reason
rq := e.conflict.reason
modDetail := ""
if e.mPath != pq.pattern {
modDetail = fmt.Sprintf("for module %s, ", e.mPath)
}
return fmt.Sprintf("%s%s conflicts with %s",
modDetail,
argStr(pq, e.proposed.version),
argStr(rq, e.conflict.version))
}
func versionOkForMainModule(version string) bool {
return version == "upgrade" || version == "patch"
}

8
src/cmd/go/internal/modload/build.go
View file

@ -123,13 +123,13 @@ func addRetraction(ctx context.Context, m *modinfo.ModulePublic) {
return
}
err := checkRetractions(ctx, module.Version{Path: m.Path, Version: m.Version})
var rerr *retractedError
err := CheckRetractions(ctx, module.Version{Path: m.Path, Version: m.Version})
var rerr *ModuleRetractedError
if errors.As(err, &rerr) {
if len(rerr.rationale) == 0 {
if len(rerr.Rationale) == 0 {
m.Retracted = []string{"retracted by module author"}
} else {
m.Retracted = rerr.rationale
m.Retracted = rerr.Rationale
}
} else if err != nil && m.Error == nil {
m.Error = &modinfo.ModuleError{Err: err.Error()}

162
src/cmd/go/internal/modload/buildlist.go
View file

@ -12,6 +12,7 @@ import (
"context"
"fmt"
"os"
"strings"
"golang.org/x/mod/module"
)
@ -27,6 +28,11 @@ import (
//
var buildList []module.Version
// capVersionSlice returns s with its cap reduced to its length.
func capVersionSlice(s []module.Version) []module.Version {
return s[:len(s):len(s)]
}
// LoadAllModules loads and returns the list of modules matching the "all"
// module pattern, starting with the Target module and in a deterministic
// (stable) order, without loading any packages.
@ -35,21 +41,21 @@ var buildList []module.Version
// LoadAllModules need only be called if LoadPackages is not,
// typically in commands that care about modules but no particular package.
//
// The caller must not modify the returned list.
// The caller must not modify the returned list, but may append to it.
func LoadAllModules(ctx context.Context) []module.Version {
LoadModFile(ctx)
ReloadBuildList()
WriteGoMod()
return buildList
return capVersionSlice(buildList)
}
// LoadedModules returns the list of module requirements loaded or set by a
// previous call (typically LoadAllModules or LoadPackages), starting with the
// Target module and in a deterministic (stable) order.
//
// The caller must not modify the returned list.
// The caller must not modify the returned list, but may append to it.
func LoadedModules() []module.Version {
return buildList
return capVersionSlice(buildList)
}
// Selected returns the selected version of the module with the given path, or
@ -67,15 +73,149 @@ func Selected(path string) (version string) {
return ""
}
// SetBuildList sets the module build list.
// The caller is responsible for ensuring that the list is valid.
// SetBuildList does not retain a reference to the original list.
func SetBuildList(list []module.Version) {
buildList = append([]module.Version{}, list...)
// EditBuildList edits the global build list by first adding every module in add
// to the existing build list, then adjusting versions (and adding or removing
// requirements as needed) until every module in mustSelect is selected at the
// given version.
//
// (Note that the newly-added modules might not be selected in the resulting
// build list: they could be lower than existing requirements or conflict with
// versions in mustSelect.)
//
// After performing the requested edits, EditBuildList returns the updated build
// list.
//
// If the versions listed in mustSelect are mutually incompatible (due to one of
// the listed modules requiring a higher version of another), EditBuildList
// returns a *ConstraintError and leaves the build list in its previous state.
func EditBuildList(ctx context.Context, add, mustSelect []module.Version) error {
var upgraded = capVersionSlice(buildList)
if len(add) > 0 {
// First, upgrade the build list with any additions.
// In theory we could just append the additions to the build list and let
// mvs.Downgrade take care of resolving the upgrades too, but the
// diagnostics from Upgrade are currently much better in case of errors.
var err error
upgraded, err = mvs.Upgrade(Target, &mvsReqs{buildList: upgraded}, add...)
if err != nil {
return err
}
}
downgraded, err := mvs.Downgrade(Target, &mvsReqs{buildList: append(upgraded, mustSelect...)}, mustSelect...)
if err != nil {
return err
}
final, err := mvs.Upgrade(Target, &mvsReqs{buildList: downgraded}, mustSelect...)
if err != nil {
return err
}
selected := make(map[string]module.Version, len(final))
for _, m := range final {
selected[m.Path] = m
}
inconsistent := false
for _, m := range mustSelect {
s, ok := selected[m.Path]
if !ok {
if m.Version != "none" {
panic(fmt.Sprintf("internal error: mvs.BuildList lost %v", m))
}
continue
}
if s.Version != m.Version {
inconsistent = true
break
}
}
if !inconsistent {
buildList = final
return nil
}
// We overshot one or more of the modules in mustSelected, which means that
// Downgrade removed something in mustSelect because it conflicted with
// something else in mustSelect.
//
// Walk the requirement graph to find the conflict.
//
// TODO(bcmills): Ideally, mvs.Downgrade (or a replacement for it) would do
// this directly.
reqs := &mvsReqs{buildList: final}
reason := map[module.Version]module.Version{}
for _, m := range mustSelect {
reason[m] = m
}
queue := mustSelect[:len(mustSelect):len(mustSelect)]
for len(queue) > 0 {
var m module.Version
m, queue = queue[0], queue[1:]
required, err := reqs.Required(m)
if err != nil {
return err
}
for _, r := range required {
if _, ok := reason[r]; !ok {
reason[r] = reason[m]
queue = append(queue, r)
}
}
}
var conflicts []Conflict
for _, m := range mustSelect {
s, ok := selected[m.Path]
if !ok {
if m.Version != "none" {
panic(fmt.Sprintf("internal error: mvs.BuildList lost %v", m))
}
continue
}
if s.Version != m.Version {
conflicts = append(conflicts, Conflict{
Source: reason[s],
Dep: s,
Constraint: m,
})
}
}
return &ConstraintError{
Conflicts: conflicts,
}
}
// A ConstraintError describes inconsistent constraints in EditBuildList
type ConstraintError struct {
// Conflict lists the source of the conflict for each version in mustSelect
// that could not be selected due to the requirements of some other version in
// mustSelect.
Conflicts []Conflict
}
func (e *ConstraintError) Error() string {
b := new(strings.Builder)
b.WriteString("version constraints conflict:")
for _, c := range e.Conflicts {
fmt.Fprintf(b, "\n\t%v requires %v, but %v is requested", c.Source, c.Dep, c.Constraint)
}
return b.String()
}
// A Conflict documents that Source requires Dep, which conflicts with Constraint.
// (That is, Dep has the same module path as Constraint but a higher version.)
type Conflict struct {
Source module.Version
Dep module.Version
Constraint module.Version
}
// ReloadBuildList resets the state of loaded packages, then loads and returns
// the build list set in SetBuildList.
// the build list set by EditBuildList.
func ReloadBuildList() []module.Version {
loaded = loadFromRoots(loaderParams{
PackageOpts: PackageOpts{
@ -84,7 +224,7 @@ func ReloadBuildList() []module.Version {
listRoots: func() []string { return nil },
allClosesOverTests: index.allPatternClosesOverTests(), // but doesn't matter because the root list is empty.
})
return buildList
return capVersionSlice(buildList)
}
// TidyBuildList trims the build list to the minimal requirements needed to

4
src/cmd/go/internal/modload/help.go
View file

@ -365,7 +365,7 @@ list if the error is a 404 or 410 HTTP response or if the current proxy is
followed by a pipe character, indicating it is safe to fall back on any error.
The GOPRIVATE and GONOPROXY environment variables allow bypassing
the proxy for selected modules. See 'go help module-private' for details.
the proxy for selected modules. See 'go help private' for details.
No matter the source of the modules, the go command checks downloads against
known checksums, to detect unexpected changes in the content of any specific
@ -439,7 +439,7 @@ The leading verb can be factored out of adjacent lines to create a block,
like in Go imports:
require (
new/thing v2.3.4
new/thing/v2 v2.3.4
old/thing v1.2.3
)

6
src/cmd/go/internal/modload/import.go
View file

@ -188,9 +188,9 @@ func (e *invalidImportError) Unwrap() error {
// importFromBuildList can return an empty directory string, for fake packages
// like "C" and "unsafe".
//
// If the package cannot be found in the current build list,
// If the package cannot be found in buildList,
// importFromBuildList returns an *ImportMissingError.
func importFromBuildList(ctx context.Context, path string) (m module.Version, dir string, err error) {
func importFromBuildList(ctx context.Context, path string, buildList []module.Version) (m module.Version, dir string, err error) {
if strings.Contains(path, "@") {
return module.Version{}, "", fmt.Errorf("import path should not have @version")
}
@ -383,7 +383,7 @@ func queryImport(ctx context.Context, path string) (module.Version, error) {
// and return m, dir, ImpportMissingError.
fmt.Fprintf(os.Stderr, "go: finding module for package %s\n", path)
candidates, err := QueryPattern(ctx, path, "latest", Selected, CheckAllowed)
candidates, err := QueryPackages(ctx, path, "latest", Selected, CheckAllowed)
if err != nil {
if errors.Is(err, fs.ErrNotExist) {
// Return "cannot find module providing package […]" instead of whatever

17
src/cmd/go/internal/modload/load.go
View file

@ -141,6 +141,15 @@ type PackageOpts struct {
// if the flag is set to "readonly" (the default) or "vendor".
ResolveMissingImports bool
// AllowPackage, if non-nil, is called after identifying the module providing
// each package. If AllowPackage returns a non-nil error, that error is set
// for the package, and the imports and test of that package will not be
// loaded.
//
// AllowPackage may be invoked concurrently by multiple goroutines,
// and may be invoked multiple times for a given package path.
AllowPackage func(ctx context.Context, path string, mod module.Version) error
// LoadTests loads the test dependencies of each package matching a requested
// pattern. If ResolveMissingImports is also true, test dependencies will be
// resolved if missing.
@ -550,6 +559,7 @@ func DirImportPath(dir string) string {
func TargetPackages(ctx context.Context, pattern string) *search.Match {
// TargetPackages is relative to the main module, so ensure that the main
// module is a thing that can contain packages.
LoadModFile(ctx)
ModRoot()
m := search.NewMatch(pattern)
@ -1042,7 +1052,7 @@ func (ld *loader) load(pkg *loadPkg) {
return
}
pkg.mod, pkg.dir, pkg.err = importFromBuildList(context.TODO(), pkg.path)
pkg.mod, pkg.dir, pkg.err = importFromBuildList(context.TODO(), pkg.path, buildList)
if pkg.dir == "" {
return
}
@ -1058,6 +1068,11 @@ func (ld *loader) load(pkg *loadPkg) {
// to scanning source code for imports).
ld.applyPkgFlags(pkg, pkgInAll)
}
if ld.AllowPackage != nil {
if err := ld.AllowPackage(context.TODO(), pkg.path, pkg.mod); err != nil {
pkg.err = err
}
}
imports, testImports, err := scanDir(pkg.dir, ld.Tags)
if err != nil {

20
src/cmd/go/internal/modload/modfile.go
View file

@ -59,7 +59,7 @@ func CheckAllowed(ctx context.Context, m module.Version) error {
if err := CheckExclusions(ctx, m); err != nil {
return err
}
if err := checkRetractions(ctx, m); err != nil {
if err := CheckRetractions(ctx, m); err != nil {
return err
}
return nil
@ -85,9 +85,9 @@ type excludedError struct{}
func (e *excludedError) Error() string { return "excluded by go.mod" }
func (e *excludedError) Is(err error) bool { return err == ErrDisallowed }
// checkRetractions returns an error if module m has been retracted by
// CheckRetractions returns an error if module m has been retracted by
// its author.
func checkRetractions(ctx context.Context, m module.Version) error {
func CheckRetractions(ctx context.Context, m module.Version) error {
if m.Version == "" {
// Main module, standard library, or file replacement module.
// Cannot be retracted.
@ -165,28 +165,28 @@ func checkRetractions(ctx context.Context, m module.Version) error {
}
}
if isRetracted {
return module.VersionError(m, &retractedError{rationale: rationale})
return module.VersionError(m, &ModuleRetractedError{Rationale: rationale})
}
return nil
}
var retractCache par.Cache
type retractedError struct {
rationale []string
type ModuleRetractedError struct {
Rationale []string
}
func (e *retractedError) Error() string {
func (e *ModuleRetractedError) Error() string {
msg := "retracted by module author"
if len(e.rationale) > 0 {
if len(e.Rationale) > 0 {
// This is meant to be a short error printed on a terminal, so just
// print the first rationale.
msg += ": " + ShortRetractionRationale(e.rationale[0])
msg += ": " + ShortRetractionRationale(e.Rationale[0])
}
return msg
}
func (e *retractedError) Is(err error) bool {
func (e *ModuleRetractedError) Is(err error) bool {
return err == ErrDisallowed
}

12
src/cmd/go/internal/modload/mvs.go
View file

@ -24,7 +24,7 @@ type mvsReqs struct {
}
// Reqs returns the current module requirement graph.
// Future calls to SetBuildList do not affect the operation
// Future calls to EditBuildList do not affect the operation
// of the returned Reqs.
func Reqs() mvs.Reqs {
r := &mvsReqs{
@ -58,7 +58,7 @@ func (r *mvsReqs) Required(mod module.Version) ([]module.Version, error) {
// be chosen over other versions of the same module in the module dependency
// graph.
func (*mvsReqs) Max(v1, v2 string) string {
if v1 != "" && semver.Compare(v1, v2) == -1 {
if v1 != "" && (v2 == "" || semver.Compare(v1, v2) == -1) {
return v2
}
return v1
@ -99,8 +99,16 @@ func versions(ctx context.Context, path string, allowed AllowedFunc) ([]string,
// Previous returns the tagged version of m.Path immediately prior to
// m.Version, or version "none" if no prior version is tagged.
//
// Since the version of Target is not found in the version list,
// it has no previous version.
func (*mvsReqs) Previous(m module.Version) (module.Version, error) {
// TODO(golang.org/issue/38714): thread tracing context through MVS.
if m == Target {
return module.Version{Path: m.Path, Version: "none"}, nil
}
list, err := versions(context.TODO(), m.Path, CheckAllowed)
if err != nil {
if errors.Is(err, os.ErrNotExist) {

33
src/cmd/go/internal/modload/mvs_test.go Normal file
View file

@ -0,0 +1,33 @@
// Copyright 2020 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package modload_test
import (
"testing"
"cmd/go/internal/modload"
)
func TestReqsMax(t *testing.T) {
type testCase struct {
a, b, want string
}
reqs := modload.Reqs()
for _, tc := range []testCase{
{a: "v0.1.0", b: "v0.2.0", want: "v0.2.0"},
{a: "v0.2.0", b: "v0.1.0", want: "v0.2.0"},
{a: "", b: "v0.1.0", want: ""}, // "" is Target.Version
{a: "v0.1.0", b: "", want: ""},
{a: "none", b: "v0.1.0", want: "v0.1.0"},
{a: "v0.1.0", b: "none", want: "v0.1.0"},
{a: "none", b: "", want: ""},
{a: "", b: "none", want: ""},
} {
max := reqs.Max(tc.a, tc.b)
if max != tc.want {
t.Errorf("Reqs().Max(%q, %q) = %q; want %q", tc.a, tc.b, max, tc.want)
}
}
}

196
src/cmd/go/internal/modload/query.go
View file

@ -47,8 +47,9 @@ import (
// with non-prereleases preferred over prereleases.
// - a repository commit identifier or tag, denoting that commit.
//
// current denotes the current version of the module; it may be "" if the
// current version is unknown or should not be considered. If query is
// current denotes the currently-selected version of the module; it may be
// "none" if no version is currently selected, or "" if the currently-selected
// version is unknown or should not be considered. If query is
// "upgrade" or "patch", current will be returned if it is a newer
// semantic version or a chronologically later pseudo-version than the
// version that would otherwise be chosen. This prevents accidental downgrades
@ -98,7 +99,7 @@ func queryProxy(ctx context.Context, proxy, path, query, current string, allowed
ctx, span := trace.StartSpan(ctx, "modload.queryProxy "+path+" "+query)
defer span.Done()
if current != "" && !semver.IsValid(current) {
if current != "" && current != "none" && !semver.IsValid(current) {
return nil, fmt.Errorf("invalid previous version %q", current)
}
if cfg.BuildMod == "vendor" {
@ -108,10 +109,7 @@ func queryProxy(ctx context.Context, proxy, path, query, current string, allowed
allowed = func(context.Context, module.Version) error { return nil }
}
if path == Target.Path {
if query != "latest" {
return nil, fmt.Errorf("can't query specific version (%q) for the main module (%s)", query, path)
}
if path == Target.Path && (query == "upgrade" || query == "patch") {
if err := allowed(ctx, Target); err != nil {
return nil, fmt.Errorf("internal error: main module version is not allowed: %w", err)
}
@ -236,7 +234,7 @@ func queryProxy(ctx context.Context, proxy, path, query, current string, allowed
}
}
if (query == "upgrade" || query == "patch") && current != "" {
if (query == "upgrade" || query == "patch") && current != "" && current != "none" {
// "upgrade" and "patch" may stay on the current version if allowed.
if err := allowed(ctx, module.Version{Path: path, Version: current}); errors.Is(err, ErrDisallowed) {
return nil, err
@ -323,7 +321,7 @@ func newQueryMatcher(path string, query, current string, allowed AllowedFunc) (*
qm.mayUseLatest = true
case query == "upgrade":
if current == "" {
if current == "" || current == "none" {
qm.mayUseLatest = true
} else {
qm.mayUseLatest = modfetch.IsPseudoVersion(current)
@ -331,6 +329,9 @@ func newQueryMatcher(path string, query, current string, allowed AllowedFunc) (*
}
case query == "patch":
if current == "none" {
return nil, &NoPatchBaseError{path}
}
if current == "" {
qm.mayUseLatest = true
} else {
@ -504,20 +505,39 @@ type QueryResult struct {
Packages []string
}
// QueryPackages is like QueryPattern, but requires that the pattern match at
// least one package and omits the non-package result (if any).
func QueryPackages(ctx context.Context, pattern, query string, current func(string) string, allowed AllowedFunc) ([]QueryResult, error) {
pkgMods, modOnly, err := QueryPattern(ctx, pattern, query, current, allowed)
if len(pkgMods) == 0 && err == nil {
return nil, &PackageNotInModuleError{
Mod: modOnly.Mod,
Replacement: Replacement(modOnly.Mod),
Query: query,
Pattern: pattern,
}
}
return pkgMods, err
}
// QueryPattern looks up the module(s) containing at least one package matching
// the given pattern at the given version. The results are sorted by module path
// length in descending order.
// length in descending order. If any proxy provides a non-empty set of candidate
// modules, no further proxies are tried.
//
// QueryPattern queries modules with package paths up to the first "..."
// in the pattern. For the pattern "example.com/a/b.../c", QueryPattern would
// consider prefixes of "example.com/a". If multiple modules have versions
// that match the query and packages that match the pattern, QueryPattern
// picks the one with the longest module path.
// For wildcard patterns, QueryPattern looks in modules with package paths up to
// the first "..." in the pattern. For the pattern "example.com/a/b.../c",
// QueryPattern would consider prefixes of "example.com/a".
//
// If any matching package is in the main module, QueryPattern considers only
// the main module and only the version "latest", without checking for other
// possible modules.
func QueryPattern(ctx context.Context, pattern, query string, current func(string) string, allowed AllowedFunc) ([]QueryResult, error) {
//
// QueryPattern always returns at least one QueryResult (which may be only
// modOnly) or a non-nil error.
func QueryPattern(ctx context.Context, pattern, query string, current func(string) string, allowed AllowedFunc) (pkgMods []QueryResult, modOnly *QueryResult, err error) {
ctx, span := trace.StartSpan(ctx, "modload.QueryPattern "+pattern+" "+query)
defer span.Done()
@ -531,9 +551,13 @@ func QueryPattern(ctx context.Context, pattern, query string, current func(strin
}
var match func(mod module.Version, root string, isLocal bool) *search.Match
matchPattern := search.MatchPattern(pattern)
if i := strings.Index(pattern, "..."); i >= 0 {
base = pathpkg.Dir(pattern[:i+3])
if base == "." {
return nil, nil, &WildcardInFirstElementError{Pattern: pattern, Query: query}
}
match = func(mod module.Version, root string, isLocal bool) *search.Match {
m := search.NewMatch(pattern)
matchPackages(ctx, m, imports.AnyTags(), omitStd, []module.Version{mod})
@ -555,23 +579,41 @@ func QueryPattern(ctx context.Context, pattern, query string, current func(strin
}
}
var queryMatchesMainModule bool
if HasModRoot() {
m := match(Target, modRoot, true)
if len(m.Pkgs) > 0 {
if query != "latest" {
return nil, fmt.Errorf("can't query specific version for package %s in the main module (%s)", pattern, Target.Path)
if query != "upgrade" && query != "patch" {
return nil, nil, &QueryMatchesPackagesInMainModuleError{
Pattern: pattern,
Query: query,
Packages: m.Pkgs,
}
}
if err := allowed(ctx, Target); err != nil {
return nil, fmt.Errorf("internal error: package %s is in the main module (%s), but version is not allowed: %w", pattern, Target.Path, err)
return nil, nil, fmt.Errorf("internal error: package %s is in the main module (%s), but version is not allowed: %w", pattern, Target.Path, err)
}
return []QueryResult{{
Mod: Target,
Rev: &modfetch.RevInfo{Version: Target.Version},
Packages: m.Pkgs,
}}, nil
}}, nil, nil
}
if err := firstError(m); err != nil {
return nil, err
return nil, nil, err
}
if matchPattern(Target.Path) {
queryMatchesMainModule = true
}
if (query == "upgrade" || query == "patch") && queryMatchesMainModule {
if err := allowed(ctx, Target); err == nil {
modOnly = &QueryResult{
Mod: Target,
Rev: &modfetch.RevInfo{Version: Target.Version},
}
}
}
}
@ -580,14 +622,23 @@ func QueryPattern(ctx context.Context, pattern, query string, current func(strin
candidateModules = modulePrefixesExcludingTarget(base)
)
if len(candidateModules) == 0 {
return nil, &PackageNotInModuleError{
Mod: Target,
Query: query,
Pattern: pattern,
if modOnly != nil {
return nil, modOnly, nil
} else if queryMatchesMainModule {
return nil, nil, &QueryMatchesMainModuleError{
Pattern: pattern,
Query: query,
}
} else {
return nil, nil, &PackageNotInModuleError{
Mod: Target,
Query: query,
Pattern: pattern,
}
}
}
err := modfetch.TryProxies(func(proxy string) error {
err = modfetch.TryProxies(func(proxy string) error {
queryModule := func(ctx context.Context, path string) (r QueryResult, err error) {
ctx, span := trace.StartSpan(ctx, "modload.QueryPattern.queryModule ["+proxy+"] "+path)
defer span.Done()
@ -606,7 +657,7 @@ func QueryPattern(ctx context.Context, pattern, query string, current func(strin
}
m := match(r.Mod, root, isLocal)
r.Packages = m.Pkgs
if len(r.Packages) == 0 {
if len(r.Packages) == 0 && !matchPattern(path) {
if err := firstError(m); err != nil {
return r, err
}
@ -620,12 +671,25 @@ func QueryPattern(ctx context.Context, pattern, query string, current func(strin
return r, nil
}
var err error
results, err = queryPrefixModules(ctx, candidateModules, queryModule)
allResults, err := queryPrefixModules(ctx, candidateModules, queryModule)
results = allResults[:0]
for _, r := range allResults {
if len(r.Packages) == 0 {
modOnly = &r
} else {
results = append(results, r)
}
}
return err
})
return results, err
if queryMatchesMainModule && len(results) == 0 && modOnly == nil && errors.Is(err, fs.ErrNotExist) {
return nil, nil, &QueryMatchesMainModuleError{
Pattern: pattern,
Query: query,
}
}
return results[:len(results):len(results)], modOnly, err
}
// modulePrefixesExcludingTarget returns all prefixes of path that may plausibly
@ -651,11 +715,6 @@ func modulePrefixesExcludingTarget(path string) []string {
return prefixes
}
type prefixResult struct {
QueryResult
err error
}
func queryPrefixModules(ctx context.Context, candidateModules []string, queryModule func(ctx context.Context, path string) (QueryResult, error)) (found []QueryResult, err error) {
ctx, span := trace.StartSpan(ctx, "modload.queryPrefixModules")
defer span.Done()
@ -686,6 +745,7 @@ func queryPrefixModules(ctx context.Context, candidateModules []string, queryMod
var (
noPackage *PackageNotInModuleError
noVersion *NoMatchingVersionError
noPatchBase *NoPatchBaseError
notExistErr error
)
for _, r := range results {
@ -702,6 +762,10 @@ func queryPrefixModules(ctx context.Context, candidateModules []string, queryMod
if noVersion == nil {
noVersion = rErr
}
case *NoPatchBaseError:
if noPatchBase == nil {
noPatchBase = rErr
}
default:
if errors.Is(rErr, fs.ErrNotExist) {
if notExistErr == nil {
@ -733,6 +797,8 @@ func queryPrefixModules(ctx context.Context, candidateModules []string, queryMod
err = noPackage
case noVersion != nil:
err = noVersion
case noPatchBase != nil:
err = noPatchBase
case notExistErr != nil:
err = notExistErr
default:
@ -757,12 +823,34 @@ type NoMatchingVersionError struct {
func (e *NoMatchingVersionError) Error() string {
currentSuffix := ""
if (e.query == "upgrade" || e.query == "patch") && e.current != "" {
if (e.query == "upgrade" || e.query == "patch") && e.current != "" && e.current != "none" {
currentSuffix = fmt.Sprintf(" (current version is %s)", e.current)
}
return fmt.Sprintf("no matching versions for query %q", e.query) + currentSuffix
}
// A NoPatchBaseError indicates that Query was called with the query "patch"
// but with a current version of "" or "none".
type NoPatchBaseError struct {
path string
}
func (e *NoPatchBaseError) Error() string {
return fmt.Sprintf(`can't query version "patch" of module %s: no existing version is required`, e.path)
}
// A WildcardInFirstElementError indicates that a pattern passed to QueryPattern
// had a wildcard in its first path element, and therefore had no pattern-prefix
// modules to search in.
type WildcardInFirstElementError struct {
Pattern string
Query string
}
func (e *WildcardInFirstElementError) Error() string {
return fmt.Sprintf("no modules to query for %s@%s because first path element contains a wildcard", e.Pattern, e.Query)
}
// A PackageNotInModuleError indicates that QueryPattern found a candidate
// module at the requested version, but that module did not contain any packages
// matching the requested pattern.
@ -989,3 +1077,39 @@ func (rr *replacementRepo) replacementStat(v string) (*modfetch.RevInfo, error)
}
return rev, nil
}
// A QueryMatchesMainModuleError indicates that a query requests
// a version of the main module that cannot be satisfied.
// (The main module's version cannot be changed.)
type QueryMatchesMainModuleError struct {
Pattern string
Query string
}
func (e *QueryMatchesMainModuleError) Error() string {
if e.Pattern == Target.Path {
return fmt.Sprintf("can't request version %q of the main module (%s)", e.Query, e.Pattern)
}
return fmt.Sprintf("can't request version %q of pattern %q that includes the main module (%s)", e.Query, e.Pattern, Target.Path)
}
// A QueryMatchesPackagesInMainModuleError indicates that a query cannot be
// satisfied because it matches one or more packages found in the main module.
type QueryMatchesPackagesInMainModuleError struct {
Pattern string
Query string
Packages []string
}
func (e *QueryMatchesPackagesInMainModuleError) Error() string {
if len(e.Packages) > 1 {
return fmt.Sprintf("pattern %s matches %d packages in the main module, so can't request version %s", e.Pattern, len(e.Packages), e.Query)
}
if search.IsMetaPackage(e.Pattern) || strings.Contains(e.Pattern, "...") {
return fmt.Sprintf("pattern %s matches package %s in the main module, so can't request version %s", e.Pattern, e.Packages[0], e.Query)
}
return fmt.Sprintf("package %s is in the main module, so can't request version %s", e.Packages[0], e.Query)
}

45
src/cmd/go/internal/modload/search.go
View file

@ -156,7 +156,7 @@ func matchPackages(ctx context.Context, m *search.Match, tags map[string]bool, f
isLocal = true
} else {
var err error
needSum := true
const needSum = true
root, isLocal, err = fetch(ctx, mod, needSum)
if err != nil {
m.AddError(err)
@ -174,3 +174,46 @@ func matchPackages(ctx context.Context, m *search.Match, tags map[string]bool, f
return
}
// MatchInModule identifies the packages matching the given pattern within the
// given module version, which does not need to be in the build list or module
// requirement graph.
//
// If m is the zero module.Version, MatchInModule matches the pattern
// against the standard library (std and cmd) in GOROOT/src.
func MatchInModule(ctx context.Context, pattern string, m module.Version, tags map[string]bool) *search.Match {
match := search.NewMatch(pattern)
if m == (module.Version{}) {
matchPackages(ctx, match, tags, includeStd, nil)
}
LoadModFile(ctx)
if !match.IsLiteral() {
matchPackages(ctx, match, tags, omitStd, []module.Version{m})
return match
}
const needSum = true
root, isLocal, err := fetch(ctx, m, needSum)
if err != nil {
match.Errs = []error{err}
return match
}
dir, haveGoFiles, err := dirInModule(pattern, m.Path, root, isLocal)
if err != nil {
match.Errs = []error{err}
return match
}
if haveGoFiles {
if _, _, err := scanDir(dir, tags); err != imports.ErrNoGo {
// ErrNoGo indicates that the directory is not actually a Go package,
// perhaps due to the tags in use. Any other non-nil error indicates a
// problem with one or more of the Go source files, but such an error does
// not stop the package from existing, so it has no impact on matching.
match.Pkgs = []string{pattern}
}
}
return match
}

54
src/cmd/go/internal/mvs/mvs.go
View file

@ -108,19 +108,21 @@ func buildList(target module.Version, reqs Reqs, upgrade func(module.Version) (m
node := &modGraphNode{m: m}
mu.Lock()
modGraph[m] = node
if v, ok := min[m.Path]; !ok || reqs.Max(v, m.Version) != v {
min[m.Path] = m.Version
if m.Version != "none" {
if v, ok := min[m.Path]; !ok || reqs.Max(v, m.Version) != v {
min[m.Path] = m.Version
}
}
mu.Unlock()
required, err := reqs.Required(m)
if err != nil {
setErr(node, err)
return
}
node.required = required
for _, r := range node.required {
if r.Version != "none" {
if m.Version != "none" {
required, err := reqs.Required(m)
if err != nil {
setErr(node, err)
return
}
node.required = required
for _, r := range node.required {
work.Add(r)
}
}
@ -333,16 +335,36 @@ func Upgrade(target module.Version, reqs Reqs, upgrade ...module.Version) ([]mod
if err != nil {
return nil, err
}
// TODO: Maybe if an error is given,
// rerun with BuildList(upgrade[0], reqs) etc
// to find which ones are the buggy ones.
pathInList := make(map[string]bool, len(list))
for _, m := range list {
pathInList[m.Path] = true
}
list = append([]module.Version(nil), list...)
list = append(list, upgrade...)
return BuildList(target, &override{target, list, reqs})
upgradeTo := make(map[string]string, len(upgrade))
for _, u := range upgrade {
if !pathInList[u.Path] {
list = append(list, module.Version{Path: u.Path, Version: "none"})
}
if prev, dup := upgradeTo[u.Path]; dup {
upgradeTo[u.Path] = reqs.Max(prev, u.Version)
} else {
upgradeTo[u.Path] = u.Version
}
}
return buildList(target, &override{target, list, reqs}, func(m module.Version) (module.Version, error) {
if v, ok := upgradeTo[m.Path]; ok {
return module.Version{Path: m.Path, Version: v}, nil
}
return m, nil
})
}
// Downgrade returns a build list for the target module
// in which the given additional modules are downgraded.
// in which the given additional modules are downgraded,
// potentially overriding the requirements of the target.
//
// The versions to be downgraded may be unreachable from reqs.Latest and
// reqs.Previous, but the methods of reqs must otherwise handle such versions

40
src/cmd/go/internal/mvs/mvs_test.go
View file

@ -54,7 +54,7 @@ build A: A B C D2 E2
name: cross1V
A: B2 C D2 E1
B1:
B1:
B2: D1
C: D2
D1: E2
@ -63,7 +63,7 @@ build A: A B2 C D2 E2
name: cross1U
A: B1 C
B1:
B1:
B2: D1
C: D2
D1: E2
@ -72,7 +72,7 @@ build A: A B1 C D2 E1
upgrade A B2: A B2 C D2 E2
name: cross1R
A: B C
A: B C
B: D2
C: D1
D1: E2
@ -165,7 +165,7 @@ M: A1 B1
A1: X1
B1: X2
X1: I1
X2:
X2:
build M: M A1 B1 I1 X2
# Upgrade from B1 to B2 should not drop the transitive dep on D.
@ -229,28 +229,31 @@ E1:
F1:
downgrade A F1: A B1 E1
name: down3
A:
name: downcycle
A: A B2
B2: A
B1:
downgrade A B1: A B1
# golang.org/issue/25542.
name: noprev1
A: B4 C2
B2.hidden:
C2:
B2.hidden:
C2:
downgrade A B2.hidden: A B2.hidden C2
name: noprev2
A: B4 C2
B2.hidden:
B1:
C2:
B2.hidden:
B1:
C2:
downgrade A B2.hidden: A B2.hidden C2
name: noprev3
A: B4 C2
B3:
B2.hidden:
C2:
B3:
B2.hidden:
C2:
downgrade A B2.hidden: A B2.hidden C2
# Cycles involving the target.
@ -315,7 +318,7 @@ M: A1 B1
A1: X1
B1: X2
X1: I1
X2:
X2:
req M: A1 B1
name: reqnone
@ -338,6 +341,9 @@ func Test(t *testing.T) {
for _, fn := range fns {
fn(t)
}
if len(fns) == 0 {
t.Errorf("no functions tested")
}
})
}
}
@ -485,9 +491,9 @@ func (r reqsMap) Max(v1, v2 string) string {
}
func (r reqsMap) Upgrade(m module.Version) (module.Version, error) {
var u module.Version
u := module.Version{Version: "none"}
for k := range r {
if k.Path == m.Path && u.Version < k.Version && !strings.HasSuffix(k.Version, ".hidden") {
if k.Path == m.Path && r.Max(u.Version, k.Version) == k.Version && !strings.HasSuffix(k.Version, ".hidden") {
u = k
}
}

2
src/cmd/go/internal/test/test.go
View file

@ -150,6 +150,7 @@ In addition to the build flags, the flags handled by 'go test' itself are:
-i
Install packages that are dependencies of the test.
Do not run the test.
The -i flag is deprecated. Compiled packages are cached automatically.
-json
Convert test output to JSON suitable for automated processing.
@ -640,6 +641,7 @@ func runTest(ctx context.Context, cmd *base.Command, args []string) {
b.Init()
if cfg.BuildI {
fmt.Fprint(os.Stderr, "go test: -i flag is deprecated\n")
cfg.BuildV = testV
deps := make(map[string]bool)

144
src/cmd/go/internal/vcs/vcs.go
View file

@ -22,7 +22,10 @@ import (
"cmd/go/internal/base"
"cmd/go/internal/cfg"
"cmd/go/internal/search"
"cmd/go/internal/web"
"golang.org/x/mod/module"
)
// A vcsCmd describes how to use a version control system
@ -591,12 +594,146 @@ func FromDir(dir, srcRoot string) (vcs *Cmd, root string, err error) {
}
if vcsRet != nil {
if err := checkGOVCS(vcsRet, rootRet); err != nil {
return nil, "", err
}
return vcsRet, rootRet, nil
}
return nil, "", fmt.Errorf("directory %q is not using a known version control system", origDir)
}
// A govcsRule is a single GOVCS rule like private:hg|svn.
type govcsRule struct {
pattern string
allowed []string
}
// A govcsConfig is a full GOVCS configuration.
type govcsConfig []govcsRule
func parseGOVCS(s string) (govcsConfig, error) {
s = strings.TrimSpace(s)
if s == "" {
return nil, nil
}
var cfg govcsConfig
have := make(map[string]string)
for _, item := range strings.Split(s, ",") {
item = strings.TrimSpace(item)
if item == "" {
return nil, fmt.Errorf("empty entry in GOVCS")
}
i := strings.Index(item, ":")
if i < 0 {
return nil, fmt.Errorf("malformed entry in GOVCS (missing colon): %q", item)
}
pattern, list := strings.TrimSpace(item[:i]), strings.TrimSpace(item[i+1:])
if pattern == "" {
return nil, fmt.Errorf("empty pattern in GOVCS: %q", item)
}
if list == "" {
return nil, fmt.Errorf("empty VCS list in GOVCS: %q", item)
}
if search.IsRelativePath(pattern) {
return nil, fmt.Errorf("relative pattern not allowed in GOVCS: %q", pattern)
}
if old := have[pattern]; old != "" {
return nil, fmt.Errorf("unreachable pattern in GOVCS: %q after %q", item, old)
}
have[pattern] = item
allowed := strings.Split(list, "|")
for i, a := range allowed {
a = strings.TrimSpace(a)
if a == "" {
return nil, fmt.Errorf("empty VCS name in GOVCS: %q", item)
}
allowed[i] = a
}
cfg = append(cfg, govcsRule{pattern, allowed})
}
return cfg, nil
}
func (c *govcsConfig) allow(path string, private bool, vcs string) bool {
for _, rule := range *c {
match := false
switch rule.pattern {
case "private":
match = private
case "public":
match = !private
default:
// Note: rule.pattern is known to be comma-free,
// so MatchPrefixPatterns is only matching a single pattern for us.
match = module.MatchPrefixPatterns(rule.pattern, path)
}
if !match {
continue
}
for _, allow := range rule.allowed {
if allow == vcs || allow == "all" {
return true
}
}
return false
}
// By default, nothing is allowed.
return false
}
var (
govcs govcsConfig
govcsErr error
govcsOnce sync.Once
)
// defaultGOVCS is the default setting for GOVCS.
// Setting GOVCS adds entries ahead of these but does not remove them.
// (They are appended to the parsed GOVCS setting.)
//
// The rationale behind allowing only Git and Mercurial is that
// these two systems have had the most attention to issues
// of being run as clients of untrusted servers. In contrast,
// Bazaar, Fossil, and Subversion have primarily been used
// in trusted, authenticated environments and are not as well
// scrutinized as attack surfaces.
//
// See golang.org/issue/41730 for details.
var defaultGOVCS = govcsConfig{
{"private", []string{"all"}},
{"public", []string{"git", "hg"}},
}
func checkGOVCS(vcs *Cmd, root string) error {
if vcs == vcsMod {
// Direct module (proxy protocol) fetches don't
// involve an external version control system
// and are always allowed.
return nil
}
govcsOnce.Do(func() {
govcs, govcsErr = parseGOVCS(os.Getenv("GOVCS"))
govcs = append(govcs, defaultGOVCS...)
})
if govcsErr != nil {
return govcsErr
}
private := module.MatchPrefixPatterns(cfg.GOPRIVATE, root)
if !govcs.allow(root, private, vcs.Cmd) {
what := "public"
if private {
what = "private"
}
return fmt.Errorf("GOVCS disallows using %s for %s %s", vcs.Cmd, what, root)
}
return nil
}
// CheckNested checks for an incorrectly-nested VCS-inside-VCS
// situation for dir, checking parents up until srcRoot.
func CheckNested(vcs *Cmd, dir, srcRoot string) error {
@ -733,6 +870,9 @@ func repoRootFromVCSPaths(importPath string, security web.SecurityMode, vcsPaths
if vcs == nil {
return nil, fmt.Errorf("unknown version control system %q", match["vcs"])
}
if err := checkGOVCS(vcs, match["root"]); err != nil {
return nil, err
}
var repoURL string
if !srv.schemelessRepo {
repoURL = match["repo"]
@ -857,6 +997,10 @@ func repoRootForImportDynamic(importPath string, mod ModuleMode, security web.Se
}
}
if err := checkGOVCS(vcs, mmi.Prefix); err != nil {
return nil, err
}
rr := &RepoRoot{
Repo: mmi.RepoRoot,
Root: mmi.Prefix,

104
src/cmd/go/internal/vcs/vcs_test.go
View file

@ -11,11 +11,20 @@ import (
"os"
"path"
"path/filepath"
"strings"
"testing"
"cmd/go/internal/web"
)
func init() {
// GOVCS defaults to public:git|hg,private:all,
// which breaks many tests here - they can't use non-git, non-hg VCS at all!
// Change to fully permissive.
// The tests of the GOVCS setting itself are in ../../testdata/script/govcs.txt.
os.Setenv("GOVCS", "*:all")
}
// Test that RepoRootForImportPath determines the correct RepoRoot for a given importPath.
// TODO(cmang): Add tests for SVN and BZR.
func TestRepoRootForImportPath(t *testing.T) {
@ -473,3 +482,98 @@ func TestValidateRepoRoot(t *testing.T) {
}
}
}
var govcsTests = []struct {
govcs string
path string
vcs string
ok bool
}{
{"private:all", "is-public.com/foo", "zzz", false},
{"private:all", "is-private.com/foo", "zzz", true},
{"public:all", "is-public.com/foo", "zzz", true},
{"public:all", "is-private.com/foo", "zzz", false},
{"public:all,private:none", "is-public.com/foo", "zzz", true},
{"public:all,private:none", "is-private.com/foo", "zzz", false},
{"*:all", "is-public.com/foo", "zzz", true},
{"golang.org:git", "golang.org/x/text", "zzz", false},
{"golang.org:git", "golang.org/x/text", "git", true},
{"golang.org:zzz", "golang.org/x/text", "zzz", true},
{"golang.org:zzz", "golang.org/x/text", "git", false},
{"golang.org:zzz", "golang.org/x/text", "zzz", true},
{"golang.org:zzz", "golang.org/x/text", "git", false},
{"golang.org:git|hg", "golang.org/x/text", "hg", true},
{"golang.org:git|hg", "golang.org/x/text", "git", true},
{"golang.org:git|hg", "golang.org/x/text", "zzz", false},
{"golang.org:all", "golang.org/x/text", "hg", true},
{"golang.org:all", "golang.org/x/text", "git", true},
{"golang.org:all", "golang.org/x/text", "zzz", true},
{"other.xyz/p:none,golang.org/x:git", "other.xyz/p/x", "git", false},
{"other.xyz/p:none,golang.org/x:git", "unexpected.com", "git", false},
{"other.xyz/p:none,golang.org/x:git", "golang.org/x/text", "zzz", false},
{"other.xyz/p:none,golang.org/x:git", "golang.org/x/text", "git", true},
{"other.xyz/p:none,golang.org/x:zzz", "golang.org/x/text", "zzz", true},
{"other.xyz/p:none,golang.org/x:zzz", "golang.org/x/text", "git", false},
{"other.xyz/p:none,golang.org/x:git|hg", "golang.org/x/text", "hg", true},
{"other.xyz/p:none,golang.org/x:git|hg", "golang.org/x/text", "git", true},
{"other.xyz/p:none,golang.org/x:git|hg", "golang.org/x/text", "zzz", false},
{"other.xyz/p:none,golang.org/x:all", "golang.org/x/text", "hg", true},
{"other.xyz/p:none,golang.org/x:all", "golang.org/x/text", "git", true},
{"other.xyz/p:none,golang.org/x:all", "golang.org/x/text", "zzz", true},
{"other.xyz/p:none,golang.org/x:git", "golang.org/y/text", "zzz", false},
{"other.xyz/p:none,golang.org/x:git", "golang.org/y/text", "git", false},
{"other.xyz/p:none,golang.org/x:zzz", "golang.org/y/text", "zzz", false},
{"other.xyz/p:none,golang.org/x:zzz", "golang.org/y/text", "git", false},
{"other.xyz/p:none,golang.org/x:git|hg", "golang.org/y/text", "hg", false},
{"other.xyz/p:none,golang.org/x:git|hg", "golang.org/y/text", "git", false},
{"other.xyz/p:none,golang.org/x:git|hg", "golang.org/y/text", "zzz", false},
{"other.xyz/p:none,golang.org/x:all", "golang.org/y/text", "hg", false},
{"other.xyz/p:none,golang.org/x:all", "golang.org/y/text", "git", false},
{"other.xyz/p:none,golang.org/x:all", "golang.org/y/text", "zzz", false},
}
func TestGOVCS(t *testing.T) {
for _, tt := range govcsTests {
cfg, err := parseGOVCS(tt.govcs)
if err != nil {
t.Errorf("parseGOVCS(%q): %v", tt.govcs, err)
continue
}
private := strings.HasPrefix(tt.path, "is-private")
ok := cfg.allow(tt.path, private, tt.vcs)
if ok != tt.ok {
t.Errorf("parseGOVCS(%q).allow(%q, %v, %q) = %v, want %v",
tt.govcs, tt.path, private, tt.vcs, ok, tt.ok)
}
}
}
var govcsErrors = []struct {
s string
err string
}{
{`,`, `empty entry in GOVCS`},
{`,x`, `empty entry in GOVCS`},
{`x,`, `malformed entry in GOVCS (missing colon): "x"`},
{`x:y,`, `empty entry in GOVCS`},
{`x`, `malformed entry in GOVCS (missing colon): "x"`},
{`x:`, `empty VCS list in GOVCS: "x:"`},
{`x:|`, `empty VCS name in GOVCS: "x:|"`},
{`x:y|`, `empty VCS name in GOVCS: "x:y|"`},
{`x:|y`, `empty VCS name in GOVCS: "x:|y"`},
{`x:y,z:`, `empty VCS list in GOVCS: "z:"`},
{`x:y,z:|`, `empty VCS name in GOVCS: "z:|"`},
{`x:y,z:|w`, `empty VCS name in GOVCS: "z:|w"`},
{`x:y,z:w|`, `empty VCS name in GOVCS: "z:w|"`},
{`x:y,z:w||v`, `empty VCS name in GOVCS: "z:w||v"`},
{`x:y,x:z`, `unreachable pattern in GOVCS: "x:z" after "x:y"`},
}
func TestGOVCSErrors(t *testing.T) {
for _, tt := range govcsErrors {
_, err := parseGOVCS(tt.s)
if err == nil || !strings.Contains(err.Error(), tt.err) {
t.Errorf("parseGOVCS(%s): err=%v, want %v", tt.s, err, tt.err)
}
}
}

11
src/cmd/go/internal/work/action.go
View file

@ -93,11 +93,12 @@ type Action struct {
output []byte // output redirect buffer (nil means use b.Print)
// Execution state.
pending int // number of deps yet to complete
priority int // relative execution priority
Failed bool // whether the action failed
json *actionJSON // action graph information
traceSpan *trace.Span
pending int // number of deps yet to complete
priority int // relative execution priority
Failed bool // whether the action failed
json *actionJSON // action graph information
nonGoOverlay map[string]string // map from non-.go source files to copied files in objdir. Nil if no overlay is used.
traceSpan *trace.Span
}
// BuildActionID returns the action ID section of a's build ID.

43
src/cmd/go/internal/work/build.go
View file

@ -31,7 +31,7 @@ import (
)
var CmdBuild = &base.Command{
UsageLine: "go build [-o output] [-i] [build flags] [packages]",
UsageLine: "go build [-o output] [build flags] [packages]",
Short: "compile packages and dependencies",
Long: `
Build compiles the packages named by the import paths,
@ -59,6 +59,7 @@ ends with a slash or backslash, then any resulting executables
will be written to that directory.
The -i flag installs the packages that are dependencies of the target.
The -i flag is deprecated. Compiled packages are cached automatically.
The build flags are shared by the build, clean, get, install, list, run,
and test commands:
@ -381,6 +382,7 @@ func runBuild(ctx context.Context, cmd *base.Command, args []string) {
depMode := ModeBuild
if cfg.BuildI {
depMode = ModeInstall
fmt.Fprint(os.Stderr, "go build: -i flag is deprecated\n")
}
pkgs = omitTestOnly(pkgsFilter(load.Packages(ctx, args)))
@ -444,7 +446,7 @@ func runBuild(ctx context.Context, cmd *base.Command, args []string) {
}
var CmdInstall = &base.Command{
UsageLine: "go install [-i] [build flags] [packages]",
UsageLine: "go install [build flags] [packages]",
Short: "compile and install packages and dependencies",
Long: `
Install compiles and installs the packages named by the import paths.
@ -486,6 +488,7 @@ directory $GOPATH/pkg/$GOOS_$GOARCH. When module-aware mode is enabled,
other packages are built and cached but not installed.
The -i flag installs the dependencies of the named packages as well.
The -i flag is deprecated. Compiled packages are cached automatically.
For more about the build flags, see 'go help build'.
For more about specifying packages, see 'go help packages'.
@ -551,14 +554,35 @@ func libname(args []string, pkgs []*load.Package) (string, error) {
}
func runInstall(ctx context.Context, cmd *base.Command, args []string) {
// TODO(golang.org/issue/41696): print a deprecation message for the -i flag
// whenever it's set (or just remove it). For now, we don't print a message
// if all named packages are in GOROOT. cmd/dist (run by make.bash) uses
// 'go install -i' when bootstrapping, and we don't want to show deprecation
// messages in that case.
for _, arg := range args {
if strings.Contains(arg, "@") && !build.IsLocalImport(arg) && !filepath.IsAbs(arg) {
if cfg.BuildI {
fmt.Fprint(os.Stderr, "go install: -i flag is deprecated\n")
}
installOutsideModule(ctx, args)
return
}
}
BuildInit()
InstallPackages(ctx, args, load.PackagesForBuild(ctx, args))
pkgs := load.PackagesForBuild(ctx, args)
if cfg.BuildI {
allGoroot := true
for _, pkg := range pkgs {
if !pkg.Goroot {
allGoroot = false
break
}
}
if !allGoroot {
fmt.Fprint(os.Stderr, "go install: -i flag is deprecated\n")
}
}
InstallPackages(ctx, args, pkgs)
}
// omitTestOnly returns pkgs with test-only packages removed.
@ -741,7 +765,8 @@ func installOutsideModule(ctx context.Context, args []string) {
// Don't check for retractions if a specific revision is requested.
allowed = nil
}
qrs, err := modload.QueryPattern(ctx, patterns[0], version, modload.Selected, allowed)
noneSelected := func(path string) (version string) { return "none" }
qrs, err := modload.QueryPackages(ctx, patterns[0], version, noneSelected, allowed)
if err != nil {
base.Fatalf("go install %s: %v", args[0], err)
}
@ -765,10 +790,12 @@ func installOutsideModule(ctx context.Context, args []string) {
base.Fatalf(directiveFmt, args[0], installMod, "exclude")
}
// Initialize the build list using a dummy main module that requires the
// module providing the packages on the command line.
target := module.Version{Path: "go-install-target"}
modload.SetBuildList([]module.Version{target, installMod})
// Since we are in NoRoot mode, the build list initially contains only
// the dummy command-line-arguments module. Add a requirement on the
// module that provides the packages named on the command line.
if err := modload.EditBuildList(ctx, nil, []module.Version{installMod}); err != nil {
base.Fatalf("go install %s: %v", args[0], err)
}
// Load packages for all arguments. Ignore non-main packages.
// Print a warning if an argument contains "..." and matches no main packages.

2
src/cmd/go/internal/work/buildid.go
View file

@ -15,6 +15,7 @@ import (
"cmd/go/internal/base"
"cmd/go/internal/cache"
"cmd/go/internal/cfg"
"cmd/go/internal/fsys"
"cmd/go/internal/str"
"cmd/internal/buildid"
)
@ -375,6 +376,7 @@ func (b *Builder) buildID(file string) string {
// fileHash returns the content hash of the named file.
func (b *Builder) fileHash(file string) string {
file, _ = fsys.OverlayPath(file)
sum, err := cache.FileHash(file)
if err != nil {
return ""

142
src/cmd/go/internal/work/exec.go
View file

@ -8,6 +8,7 @@ package work
import (
"bytes"
"cmd/go/internal/fsys"
"context"
"encoding/json"
"errors"
@ -537,6 +538,34 @@ func (b *Builder) build(ctx context.Context, a *Action) (err error) {
}
}
// Compute overlays for .c/.cc/.h/etc. and if there are any overlays
// put correct contents of all those files in the objdir, to ensure
// the correct headers are included. nonGoOverlay is the overlay that
// points from nongo files to the copied files in objdir.
nonGoFileLists := [][]string{a.Package.CFiles, a.Package.SFiles, a.Package.CXXFiles, a.Package.HFiles, a.Package.FFiles}
OverlayLoop:
for _, fs := range nonGoFileLists {
for _, f := range fs {
if _, ok := fsys.OverlayPath(mkAbs(p.Dir, f)); ok {
a.nonGoOverlay = make(map[string]string)
break OverlayLoop
}
}
}
if a.nonGoOverlay != nil {
for _, fs := range nonGoFileLists {
for i := range fs {
from := mkAbs(p.Dir, fs[i])
opath, _ := fsys.OverlayPath(from)
dst := objdir + filepath.Base(fs[i])
if err := b.copyFile(dst, opath, 0666, false); err != nil {
return err
}
a.nonGoOverlay[from] = dst
}
}
}
// Run SWIG on each .swig and .swigcxx file.
// Each run will generate two files, a .go file and a .c or .cxx file.
// The .go file will use import "C" and is to be processed by cgo.
@ -737,7 +766,7 @@ func (b *Builder) build(ctx context.Context, a *Action) (err error) {
}
if err != nil {
if p.Module != nil && !allowedVersion(p.Module.GoVersion) {
b.showOutput(a, a.Package.Dir, a.Package.Desc(), "note: module requires Go "+p.Module.GoVersion)
b.showOutput(a, a.Package.Dir, a.Package.Desc(), "note: module requires Go "+p.Module.GoVersion+"\n")
}
return err
}
@ -2242,8 +2271,6 @@ func (b *Builder) ccompile(a *Action, p *load.Package, outfile string, flags []s
// when -trimpath is enabled.
if b.gccSupportsFlag(compiler, "-fdebug-prefix-map=a=b") {
if cfg.BuildTrimpath {
// TODO(#39958): handle overlays
// Keep in sync with Action.trimpath.
// The trimmed paths are a little different, but we need to trim in the
// same situations.
@ -2270,7 +2297,11 @@ func (b *Builder) ccompile(a *Action, p *load.Package, outfile string, flags []s
}
}
output, err := b.runOut(a, filepath.Dir(file), b.cCompilerEnv(), compiler, flags, "-o", outfile, "-c", filepath.Base(file))
overlayPath := file
if p, ok := a.nonGoOverlay[overlayPath]; ok {
overlayPath = p
}
output, err := b.runOut(a, filepath.Dir(overlayPath), b.cCompilerEnv(), compiler, flags, "-o", outfile, "-c", filepath.Base(overlayPath))
if len(output) > 0 {
// On FreeBSD 11, when we pass -g to clang 3.8 it
// invokes its internal assembler with -dwarf-version=2.
@ -2313,7 +2344,8 @@ func (b *Builder) gccld(a *Action, p *load.Package, objdir, outfile string, flag
cmdargs := []interface{}{cmd, "-o", outfile, objs, flags}
dir := p.Dir
out, err := b.runOut(a, dir, b.cCompilerEnv(), cmdargs...)
out, err := b.runOut(a, base.Cwd, b.cCompilerEnv(), cmdargs...)
if len(out) > 0 {
// Filter out useless linker warnings caused by bugs outside Go.
// See also cmd/link/internal/ld's hostlink method.
@ -2641,7 +2673,8 @@ func (b *Builder) cgo(a *Action, cgoExe, objdir string, pcCFLAGS, pcLDFLAGS, cgo
cgoLDFLAGS = append([]string{"-fsanitize=memory"}, cgoLDFLAGS...)
}
// Allows including _cgo_export.h from .[ch] files in the package.
// Allows including _cgo_export.h, as well as the user's .h files,
// from .[ch] files in the package.
cgoCPPFLAGS = append(cgoCPPFLAGS, "-I", objdir)
// cgo
@ -2698,7 +2731,23 @@ func (b *Builder) cgo(a *Action, cgoExe, objdir string, pcCFLAGS, pcLDFLAGS, cgo
cgoflags = append(cgoflags, "-exportheader="+objdir+"_cgo_install.h")
}
if err := b.run(a, p.Dir, p.ImportPath, cgoenv, cfg.BuildToolexec, cgoExe, "-objdir", objdir, "-importpath", p.ImportPath, cgoflags, "--", cgoCPPFLAGS, cgoCFLAGS, cgofiles); err != nil {
execdir := p.Dir
// Rewrite overlaid paths in cgo files.
// cgo adds //line and #line pragmas in generated files with these paths.
var trimpath []string
for i := range cgofiles {
path := mkAbs(p.Dir, cgofiles[i])
if opath, ok := fsys.OverlayPath(path); ok {
cgofiles[i] = opath
trimpath = append(trimpath, opath+"=>"+path)
}
}
if len(trimpath) > 0 {
cgoflags = append(cgoflags, "-trimpath", strings.Join(trimpath, ";"))
}
if err := b.run(a, execdir, p.ImportPath, cgoenv, cfg.BuildToolexec, cgoExe, "-objdir", objdir, "-importpath", p.ImportPath, cgoflags, "--", cgoCPPFLAGS, cgoCFLAGS, cgofiles); err != nil {
return nil, nil, err
}
outGo = append(outGo, gofiles...)
@ -2779,6 +2828,81 @@ func (b *Builder) cgo(a *Action, cgoExe, objdir string, pcCFLAGS, pcLDFLAGS, cgo
noCompiler()
}
// Double check the //go:cgo_ldflag comments in the generated files.
// The compiler only permits such comments in files whose base name
// starts with "_cgo_". Make sure that the comments in those files
// are safe. This is a backstop against people somehow smuggling
// such a comment into a file generated by cgo.
if cfg.BuildToolchainName == "gc" && !cfg.BuildN {
var flags []string
for _, f := range outGo {
if !strings.HasPrefix(filepath.Base(f), "_cgo_") {
continue
}
src, err := ioutil.ReadFile(f)
if err != nil {
return nil, nil, err
}
const cgoLdflag = "//go:cgo_ldflag"
idx := bytes.Index(src, []byte(cgoLdflag))
for idx >= 0 {
// We are looking at //go:cgo_ldflag.
// Find start of line.
start := bytes.LastIndex(src[:idx], []byte("\n"))
if start == -1 {
start = 0
}
// Find end of line.
end := bytes.Index(src[idx:], []byte("\n"))
if end == -1 {
end = len(src)
} else {
end += idx
}
// Check for first line comment in line.
// We don't worry about /* */ comments,
// which normally won't appear in files
// generated by cgo.
commentStart := bytes.Index(src[start:], []byte("//"))
commentStart += start
// If that line comment is //go:cgo_ldflag,
// it's a match.
if bytes.HasPrefix(src[commentStart:], []byte(cgoLdflag)) {
// Pull out the flag, and unquote it.
// This is what the compiler does.
flag := string(src[idx+len(cgoLdflag) : end])
flag = strings.TrimSpace(flag)
flag = strings.Trim(flag, `"`)
flags = append(flags, flag)
}
src = src[end:]
idx = bytes.Index(src, []byte(cgoLdflag))
}
}
// We expect to find the contents of cgoLDFLAGS in flags.
if len(cgoLDFLAGS) > 0 {
outer:
for i := range flags {
for j, f := range cgoLDFLAGS {
if f != flags[i+j] {
continue outer
}
}
flags = append(flags[:i], flags[i+len(cgoLDFLAGS):]...)
break
}
}
if err := checkLinkerFlags("LDFLAGS", "go:cgo_ldflag", flags); err != nil {
return nil, nil, err
}
}
return outGo, outObj, nil
}
@ -2792,7 +2916,7 @@ func (b *Builder) dynimport(a *Action, p *load.Package, objdir, importGo, cgoExe
return err
}
linkobj := str.StringList(ofile, outObj, p.SysoFiles)
linkobj := str.StringList(ofile, outObj, mkAbsFiles(p.Dir, p.SysoFiles))
dynobj := objdir + "_cgo_.o"
// we need to use -pie for Linux/ARM to get accurate imported sym
@ -2817,7 +2941,7 @@ func (b *Builder) dynimport(a *Action, p *load.Package, objdir, importGo, cgoExe
if p.Standard && p.ImportPath == "runtime/cgo" {
cgoflags = []string{"-dynlinker"} // record path to dynamic linker
}
return b.run(a, p.Dir, p.ImportPath, b.cCompilerEnv(), cfg.BuildToolexec, cgoExe, "-dynpackage", p.Name, "-dynimport", dynobj, "-dynout", importGo, cgoflags)
return b.run(a, base.Cwd, p.ImportPath, b.cCompilerEnv(), cfg.BuildToolexec, cgoExe, "-dynpackage", p.Name, "-dynimport", dynobj, "-dynout", importGo, cgoflags)
}
// Run SWIG on all SWIG input files.

51
src/cmd/go/internal/work/gc.go
View file

@ -262,7 +262,7 @@ func (a *Action) trimpath() string {
if len(objdir) > 1 && objdir[len(objdir)-1] == filepath.Separator {
objdir = objdir[:len(objdir)-1]
}
rewrite := objdir + "=>"
rewrite := ""
rewriteDir := a.Package.Dir
if cfg.BuildTrimpath {
@ -271,21 +271,54 @@ func (a *Action) trimpath() string {
} else {
rewriteDir = a.Package.ImportPath
}
rewrite += ";" + a.Package.Dir + "=>" + rewriteDir
rewrite += a.Package.Dir + "=>" + rewriteDir + ";"
}
// Add rewrites for overlays. The 'from' and 'to' paths in overlays don't need to have
// same basename, so go from the overlay contents file path (passed to the compiler)
// to the path the disk path would be rewritten to.
cgoFiles := make(map[string]bool)
for _, f := range a.Package.CgoFiles {
cgoFiles[f] = true
}
// TODO(matloob): Higher up in the stack, when the logic for deciding when to make copies
// of c/c++/m/f/hfiles is consolidated, use the same logic that Build uses to determine
// whether to create the copies in objdir to decide whether to rewrite objdir to the
// package directory here.
var overlayNonGoRewrites string // rewrites for non-go files
hasCgoOverlay := false
if fsys.OverlayFile != "" {
for _, filename := range a.Package.AllFiles() {
overlayPath, ok := fsys.OverlayPath(filepath.Join(a.Package.Dir, filename))
if !ok {
continue
path := filename
if !filepath.IsAbs(path) {
path = filepath.Join(a.Package.Dir, path)
}
base := filepath.Base(path)
isGo := strings.HasSuffix(filename, ".go") || strings.HasSuffix(filename, ".s")
isCgo := cgoFiles[filename] || !isGo
overlayPath, isOverlay := fsys.OverlayPath(path)
if isCgo && isOverlay {
hasCgoOverlay = true
}
if !isCgo && isOverlay {
rewrite += overlayPath + "=>" + filepath.Join(rewriteDir, base) + ";"
} else if isCgo {
// Generate rewrites for non-Go files copied to files in objdir.
if filepath.Dir(path) == a.Package.Dir {
// This is a file copied to objdir.
overlayNonGoRewrites += filepath.Join(objdir, base) + "=>" + filepath.Join(rewriteDir, base) + ";"
}
} else {
// Non-overlay Go files are covered by the a.Package.Dir rewrite rule above.
}
rewrite += ";" + overlayPath + "=>" + filepath.Join(rewriteDir, filename)
}
}
if hasCgoOverlay {
rewrite += overlayNonGoRewrites
}
rewrite += objdir + "=>"
return rewrite
}
@ -337,9 +370,10 @@ func (gcToolchain) asm(b *Builder, a *Action, sfiles []string) ([]string, error)
var ofiles []string
for _, sfile := range sfiles {
overlayPath, _ := fsys.OverlayPath(mkAbs(p.Dir, sfile))
ofile := a.Objdir + sfile[:len(sfile)-len(".s")] + ".o"
ofiles = append(ofiles, ofile)
args1 := append(args, "-o", ofile, mkAbs(p.Dir, sfile))
args1 := append(args, "-o", ofile, overlayPath)
if err := b.run(a, p.Dir, p.ImportPath, nil, args1...); err != nil {
return nil, err
}
@ -355,7 +389,8 @@ func (gcToolchain) symabis(b *Builder, a *Action, sfiles []string) (string, erro
if p.ImportPath == "runtime/cgo" && strings.HasPrefix(sfile, "gcc_") {
continue
}
args = append(args, mkAbs(p.Dir, sfile))
op, _ := fsys.OverlayPath(mkAbs(p.Dir, sfile))
args = append(args, op)
}
// Supply an empty go_asm.h as if the compiler had been run.

2
src/cmd/go/internal/work/gccgo.go
View file

@ -199,7 +199,7 @@ func (tools gccgoToolchain) asm(b *Builder, a *Action, sfiles []string) ([]strin
base := filepath.Base(sfile)
ofile := a.Objdir + base[:len(base)-len(".s")] + ".o"
ofiles = append(ofiles, ofile)
sfile = mkAbs(p.Dir, sfile)
sfile, _ = fsys.OverlayPath(mkAbs(p.Dir, sfile))
defs := []string{"-D", "GOOS_" + cfg.Goos, "-D", "GOARCH_" + cfg.Goarch}
if pkgpath := tools.gccgoCleanPkgpath(b, p); pkgpath != "" {
defs = append(defs, `-D`, `GOPKGPATH=`+pkgpath)

10
src/cmd/go/internal/work/security.go
View file

@ -42,8 +42,8 @@ import (
var re = lazyregexp.New
var validCompilerFlags = []*lazyregexp.Regexp{
re(`-D([A-Za-z_].*)`),
re(`-U([A-Za-z_]*)`),
re(`-D([A-Za-z_][A-Za-z0-9_]*)(=[^@\-]*)?`),
re(`-U([A-Za-z_][A-Za-z0-9_]*)`),
re(`-F([^@\-].*)`),
re(`-I([^@\-].*)`),
re(`-O`),
@ -51,8 +51,8 @@ var validCompilerFlags = []*lazyregexp.Regexp{
re(`-W`),
re(`-W([^@,]+)`), // -Wall but not -Wa,-foo.
re(`-Wa,-mbig-obj`),
re(`-Wp,-D([A-Za-z_].*)`),
re(`-Wp,-U([A-Za-z_]*)`),
re(`-Wp,-D([A-Za-z_][A-Za-z0-9_]*)(=[^@,\-]*)?`),
re(`-Wp,-U([A-Za-z_][A-Za-z0-9_]*)`),
re(`-ansi`),
re(`-f(no-)?asynchronous-unwind-tables`),
re(`-f(no-)?blocks`),
@ -179,7 +179,7 @@ var validLinkerFlags = []*lazyregexp.Regexp{
re(`-Wl,-berok`),
re(`-Wl,-Bstatic`),
re(`-Wl,-Bsymbolic-functions`),
re(`-WL,-O([^@,\-][^,]*)?`),
re(`-Wl,-O([^@,\-][^,]*)?`),
re(`-Wl,-d[ny]`),
re(`-Wl,--disable-new-dtags`),
re(`-Wl,-e[=,][a-zA-Z0-9]*`),

5
src/cmd/go/internal/work/security_test.go
View file

@ -13,6 +13,7 @@ var goodCompilerFlags = [][]string{
{"-DFOO"},
{"-Dfoo=bar"},
{"-Ufoo"},
{"-Ufoo1"},
{"-F/Qt"},
{"-I/"},
{"-I/etc/passwd"},
@ -24,6 +25,8 @@ var goodCompilerFlags = [][]string{
{"-Wall"},
{"-Wp,-Dfoo=bar"},
{"-Wp,-Ufoo"},
{"-Wp,-Dfoo1"},
{"-Wp,-Ufoo1"},
{"-fobjc-arc"},
{"-fno-objc-arc"},
{"-fomit-frame-pointer"},
@ -80,6 +83,8 @@ var badCompilerFlags = [][]string{
{"-O@1"},
{"-Wa,-foo"},
{"-W@foo"},
{"-Wp,-DX,-D@X"},
{"-Wp,-UX,-U@X"},
{"-g@gdb"},
{"-g-gdb"},
{"-march=@dawn"},

3
src/cmd/go/main.go
View file

@ -75,10 +75,11 @@ func init() {
modload.HelpModules,
modget.HelpModuleGet,
modfetch.HelpModuleAuth,
modfetch.HelpModulePrivate,
help.HelpPackages,
modfetch.HelpPrivate,
test.HelpTestflag,
test.HelpTestfunc,
modget.HelpVCS,
}
}

8
src/cmd/go/script_test.go
View file

@ -135,6 +135,7 @@ func (ts *testScript) setup() {
"GOSUMDB=" + testSumDBVerifierKey,
"GONOPROXY=",
"GONOSUMDB=",
"GOVCS=*:all",
"PWD=" + ts.cd,
tempEnvName() + "=" + filepath.Join(ts.workdir, "tmp"),
"devnull=" + os.DevNull,
@ -1255,7 +1256,12 @@ func (ts *testScript) parse(line string) command {
if cmd.name != "" {
cmd.args = append(cmd.args, arg)
isRegexp = false // Commands take only one regexp argument, so no subsequent args are regexps.
// Commands take only one regexp argument (after the optional flags),
// so no subsequent args are regexps. Liberally assume an argument that
// starts with a '-' is a flag.
if len(arg) == 0 || arg[0] != '-' {
isRegexp = false
}
return
}

10
src/cmd/go/testdata/mod/example.com_retract_ambiguous_nested_v1.9.0-bad.txt vendored Normal file
View file

@ -0,0 +1,10 @@
-- .mod --
module example.com/retract/ambiguous/nested
go 1.16
retract v1.9.0-bad // nested modules are bad
-- .info --
{"Version":"v1.9.0-bad"}
-- nested.go --
package nested

12
src/cmd/go/testdata/mod/example.com_retract_ambiguous_other_v1.0.0.txt vendored Normal file
View file

@ -0,0 +1,12 @@
-- .mod --
module example.com/retract/ambiguous/other
go 1.16
require example.com/retract/ambiguous v1.0.0
-- .info --
{"Version":"v1.0.0"}
-- other.go --
package other
import _ "example.com/retract/ambiguous/nested"

9
src/cmd/go/testdata/mod/example.com_retract_ambiguous_v1.0.0.txt vendored Normal file
View file

@ -0,0 +1,9 @@
-- .mod --
module example.com/retract/ambiguous
go 1.16
-- .info --
{"Version":"v1.0.0"}
-- nested/nested.go --
package nested

24
src/cmd/go/testdata/script/build_i_deprecate.txt vendored Normal file
View file

@ -0,0 +1,24 @@
# Check that deprecation warnings are printed when the -i flag is used.
# TODO(golang.org/issue/41696): remove the -i flag after Go 1.16, and this test.
go build -n -i
stderr '^go build: -i flag is deprecated$'
go install -n -i
stderr '^go install: -i flag is deprecated$'
go test -n -i
stderr '^go test: -i flag is deprecated$'
# 'go clean -i' should not print a deprecation warning.
# It will continue working.
go clean -i .
! stderr .
-- go.mod --
module m
go 1.16
-- m.go --
package m

182
src/cmd/go/testdata/script/build_overlay.txt vendored
View file

@ -1,9 +1,11 @@
[short] skip
# Test building in overlays.
# TODO(matloob): add a test case where the destination file in the replace map
# TODO(#39958): add a test case where the destination file in the replace map
# isn't a go file. Either completely exclude that case in fs.IsDirWithGoFiles
# if the compiler doesn't allow it, or test that it works all the way.
# TODO(#39958): add a test that both gc and gccgo assembly files can include .h
# files.
# The main package (m) is contained in an overlay. It imports m/dir2 which has one
# file in an overlay and one file outside the overlay, which in turn imports m/dir,
@ -29,6 +31,36 @@ exec ./print_trimpath_two_files$GOEXE
stdout $WORK[/\\]gopath[/\\]src[/\\]m[/\\]printpath[/\\]main.go
stdout $WORK[/\\]gopath[/\\]src[/\\]m[/\\]printpath[/\\]other.go
go build -overlay overlay.json -o main_cgo_replace$GOEXE ./cgo_hello_replace
exec ./main_cgo_replace$GOEXE
stdout '^hello cgo\r?\n'
go build -overlay overlay.json -o main_cgo_quote$GOEXE ./cgo_hello_quote
exec ./main_cgo_quote$GOEXE
stdout '^hello cgo\r?\n'
go build -overlay overlay.json -o main_cgo_angle$GOEXE ./cgo_hello_angle
exec ./main_cgo_angle$GOEXE
stdout '^hello cgo\r?\n'
go build -overlay overlay.json -o main_call_asm$GOEXE ./call_asm
exec ./main_call_asm$GOEXE
! stdout .
# Change the contents of a file in the overlay and ensure that makes the target stale
go install -overlay overlay.json ./test_cache
go list -overlay overlay.json -f '{{.Stale}}' ./test_cache
stdout '^false$'
cp overlay/test_cache_different.go overlay/test_cache.go
go list -overlay overlay.json -f '{{.Stale}}' ./test_cache
stdout '^true$'
go list -compiled -overlay overlay.json -f '{{range .CompiledGoFiles}}{{. | printf "%s\n"}}{{end}}' ./cgo_hello_replace
cp stdout compiled_cgo_sources.txt
go run ../print_line_comments.go compiled_cgo_sources.txt
stdout $GOPATH[/\\]src[/\\]m[/\\]cgo_hello_replace[/\\]cgo_hello_replace.go
! stdout $GOPATH[/\\]src[/\\]m[/\\]overlay[/\\]hello.c
# Run same tests but with gccgo.
env GO111MODULE=off
[!exec:gccgo] stop
@ -46,6 +78,24 @@ go build -compiler=gccgo -overlay overlay.json -o print_trimpath_gccgo$GOEXE -tr
exec ./print_trimpath_gccgo$GOEXE
stdout ^\.[/\\]printpath[/\\]main.go
go build -compiler=gccgo -overlay overlay.json -o main_cgo_replace_gccgo$GOEXE ./cgo_hello_replace
exec ./main_cgo_replace_gccgo$GOEXE
stdout '^hello cgo\r?\n'
go build -compiler=gccgo -overlay overlay.json -o main_cgo_quote_gccgo$GOEXE ./cgo_hello_quote
exec ./main_cgo_quote_gccgo$GOEXE
stdout '^hello cgo\r?\n'
go build -compiler=gccgo -overlay overlay.json -o main_cgo_angle_gccgo$GOEXE ./cgo_hello_angle
exec ./main_cgo_angle_gccgo$GOEXE
stdout '^hello cgo\r?\n'
go build -compiler=gccgo -overlay overlay.json -o main_call_asm_gccgo$GOEXE ./call_asm
exec ./main_call_asm_gccgo$GOEXE
! stdout .
-- m/go.mod --
// TODO(matloob): how do overlays work with go.mod (especially if mod=readonly)
module m
@ -71,9 +121,36 @@ the actual code is in the overlay
"dir/g.go": "overlay/dir_g.go",
"dir2/i.go": "overlay/dir2_i.go",
"printpath/main.go": "overlay/printpath.go",
"printpath/other.go": "overlay2/printpath2.go"
"printpath/other.go": "overlay2/printpath2.go",
"call_asm/asm_gc.s": "overlay/asm_gc.s",
"call_asm/asm_gccgo.s": "overlay/asm_gccgo.s",
"test_cache/main.go": "overlay/test_cache.go",
"cgo_hello_replace/cgo_header.h": "overlay/cgo_head.h",
"cgo_hello_replace/hello.c": "overlay/hello.c",
"cgo_hello_quote/cgo_hello.go": "overlay/cgo_hello_quote.go",
"cgo_hello_quote/cgo_header.h": "overlay/cgo_head.h",
"cgo_hello_angle/cgo_hello.go": "overlay/cgo_hello_angle.go",
"cgo_hello_angle/cgo_header.h": "overlay/cgo_head.h"
}
}
-- m/cgo_hello_replace/cgo_hello_replace.go --
package main
// #include "cgo_header.h"
import "C"
func main() {
C.say_hello()
}
-- m/cgo_hello_replace/cgo_header.h --
// Test that this header is replaced with one that has the proper declaration.
void say_goodbye();
-- m/cgo_hello_replace/hello.c --
#include <stdio.h>
void say_goodbye() { puts("goodbye cgo\n"); fflush(stdout); }
-- m/overlay/f.go --
package main
@ -82,6 +159,14 @@ import "m/dir2"
func main() {
dir2.PrintMessage()
}
-- m/call_asm/main.go --
package main
func foo() // There will be a "missing function body" error if the assembly file isn't found.
func main() {
foo()
}
-- m/overlay/dir_g.go --
package dir
@ -128,3 +213,96 @@ import "m/dir"
func printMessage() {
dir.PrintMessage()
}
-- m/overlay/cgo_hello_quote.go --
package main
// #include "cgo_header.h"
import "C"
func main() {
C.say_hello()
}
-- m/overlay/cgo_hello_angle.go --
package main
// #include <cgo_header.h>
import "C"
func main() {
C.say_hello()
}
-- m/overlay/cgo_head.h --
void say_hello();
-- m/overlay/hello.c --
#include <stdio.h>
void say_hello() { puts("hello cgo\n"); fflush(stdout); }
-- m/overlay/asm_gc.s --
// +build !gccgo
TEXT ·foo(SB),0,$0
RET
-- m/overlay/asm_gccgo.s --
// +build gccgo
.globl main.foo
.text
main.foo:
ret
-- m/overlay/test_cache.go --
package foo
import "fmt"
func bar() {
fmt.Println("something")
}
-- m/overlay/test_cache_different.go --
package foo
import "fmt"
func bar() {
fmt.Println("different")
}
-- m/cgo_hello_quote/hello.c --
#include <stdio.h>
void say_hello() { puts("hello cgo\n"); fflush(stdout); }
-- m/cgo_hello_angle/hello.c --
#include <stdio.h>
void say_hello() { puts("hello cgo\n"); fflush(stdout); }
-- print_line_comments.go --
package main
import (
"fmt"
"io/ioutil"
"log"
"os"
"strings"
)
func main() {
compiledGoFilesArg := os.Args[1]
b, err := ioutil.ReadFile(compiledGoFilesArg)
if err != nil {
log.Fatal(err)
}
compiledGoFiles := strings.Split(strings.TrimSpace(string(b)), "\n")
for _, f := range compiledGoFiles {
b, err := ioutil.ReadFile(f)
if err != nil {
log.Fatal(err)
}
for _, line := range strings.Split(string(b), "\n") {
if strings.HasPrefix(line, "#line") || strings.HasPrefix(line, "//line") {
fmt.Println(line)
}
}
}
}

6
src/cmd/go/testdata/script/build_plugin_non_main.txt vendored
View file

@ -1,7 +1,5 @@
# Plugins are only supported on linux,cgo (!riscv64) and darwin,cgo.
[!linux] [!darwin] skip
[linux] [riscv64] skip
[!cgo] skip
# Plugins are not supported on all platforms.
[!buildmode:plugin] skip
go build -n testdep
! go build -buildmode=plugin testdep

28
src/cmd/go/testdata/script/build_trimpath_cgo.txt vendored
View file

@ -20,10 +20,38 @@ go build -trimpath -o hello.exe .
go run ./list-dwarf hello.exe
! stdout gopath/src
# Do the above, with the cgo (but not .c) sources in an overlay
# Check that the source path appears when -trimpath is not used.
mkdir $WORK/overlay
cp hello.go $WORK/overlay/hello.go
mkdir hello_overlay
cp hello.c hello_overlay/hello.c
go build -overlay overlay.json -o hello_overlay.exe ./hello_overlay
grep -q gopath[/\\]src hello_overlay.exe
! grep -q $WORK[/\\]overlay hello_overlay.exe
go run ./list-dwarf hello_overlay.exe
stdout gopath[/\\]src
! stdout $WORK[/\\]overlay
# Check that the source path does not appear when -trimpath is used.
go build -overlay overlay.json -trimpath -o hello_overlay.exe ./hello_overlay
! grep -q gopath[/\\]src hello_overlay.exe
! grep -q $WORK[/\\]overlay hello_overlay.exe
go run ./list-dwarf hello_overlay.exe
! stdout gopath/src
! stdout $WORK[/\\]overlay
-- go.mod --
module m
go 1.14
-- overlay.json --
{
"Replace": {
"hello_overlay/hello.go": "../../overlay/hello.go"
}
}
-- hello.c --
#include <stdio.h>

2
src/cmd/go/testdata/script/env_write.txt vendored
View file

@ -69,6 +69,8 @@ go env -u GOPATH
stderr 'unknown go command variable GODEBUG'
! go env -w GOEXE=.bat
stderr 'GOEXE cannot be modified'
! go env -w GOVERSION=customversion
stderr 'GOVERSION cannot be modified'
! go env -w GOENV=/env
stderr 'GOENV can only be set using the OS environment'

18
src/cmd/go/testdata/script/get_go_file.txt vendored
View file

@ -5,46 +5,46 @@
env GO111MODULE=off
# argument doesn't have .go suffix
go get test
go get -d test
# argument has .go suffix, is a file and exists
! go get test.go
! go get -d test.go
stderr 'go get test.go: arguments must be package or module paths'
# argument has .go suffix, doesn't exist and has no slashes
! go get test_missing.go
! go get -d test_missing.go
stderr 'go get test_missing.go: arguments must be package or module paths'
# argument has .go suffix, is a file and exists in sub-directory
! go get test/test.go
! go get -d test/test.go
stderr 'go get: test/test.go exists as a file, but ''go get'' requires package arguments'
# argument has .go suffix, doesn't exist and has slashes
! go get test/test_missing.go
! go get -d test/test_missing.go
! stderr 'arguments must be package or module paths'
! stderr 'exists as a file, but ''go get'' requires package arguments'
# argument has .go suffix, is a symlink and exists
[symlink] symlink test_sym.go -> test.go
[symlink] ! go get test_sym.go
[symlink] ! go get -d test_sym.go
[symlink] stderr 'go get test_sym.go: arguments must be package or module paths'
[symlink] rm test_sym.go
# argument has .go suffix, is a symlink and exists in sub-directory
[symlink] symlink test/test_sym.go -> test.go
[symlink] ! go get test/test_sym.go
[symlink] ! go get -d test/test_sym.go
[symlink] stderr 'go get: test/test_sym.go exists as a file, but ''go get'' requires package arguments'
[symlink] rm test_sym.go
# argument has .go suffix, is a directory and exists
mkdir test_dir.go
! go get test_dir.go
! go get -d test_dir.go
stderr 'go get test_dir.go: arguments must be package or module paths'
rm test_dir.go
# argument has .go suffix, is a directory and exists in sub-directory
mkdir test/test_dir.go
! go get test/test_dir.go
! go get -d test/test_dir.go
! stderr 'arguments must be package or module paths'
! stderr 'exists as a file, but ''go get'' requires package arguments'
rm test/test_dir.go

Some files were not shown because too many files have changed in this diff Show more