languages/go
languages/go
1
0
Fork 0
mirror of https://github.com/golang/go synced 2024-10-02 22:25:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

cmd/go: add -reuse flag to make proxy invocations more efficient

Browse source 
The go list -m and go mod download commands now have a -reuse flag,
which is passed the name of a file containing the JSON output from a
previous run of the same command. (It is up to the caller to ensure
that flags such as -versions or -retracted, which affect the output,
are consistent between the old and new run.)

The new run uses the old JSON to evaluate whether the answer is
unchanged since the old run. If so, it reuses that information,
avoiding a costly 'git fetch', and sets a new Reuse: true field in its
own JSON output.

This dance with saving the JSON output and passing it back to -reuse
is not necessary on most systems, because the go command caches
version control checkouts in the module cache. That cache means that a
new 'git fetch' would only download the commits that are new since the
previous one (often none at all).

The dance becomes important only on systems that do not preserve the
module cache, for example by running 'go clean -modcache' aggressively
or by running in some environment that starts with an empty file
system.

For #53644.

Change-Id: I447960abf8055f83cc6dbc699a9fde9931130004
Reviewed-on: https://go-review.googlesource.com/c/go/+/411398
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
This commit is contained in:
Russ Cox 2022-06-10 12:03:06 -04:00
parent 84e091eef0
commit 5f305ae8e5
15 changed files with 711 additions and 65 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

26
src/cmd/go/alldocs.go
View file

@ -930,6 +930,7 @@
//
// type Module struct {
// Path string // module path
// Query string // version query corresponding to this version
// Version string // module version
// Versions []string // available module versions
// Replace *Module // replaced by this module
@ -943,6 +944,8 @@
// Retracted []string // retraction information, if any (with -retracted or -u)
// Deprecated string // deprecation message, if any (with -u)
// Error *ModuleError // error loading module
// Origin any // provenance of module
// Reuse bool // reuse of old module info is safe
// }
//
// type ModuleError struct {
@ -1019,6 +1022,16 @@
// module as a Module struct. If an error occurs, the result will
// be a Module struct with a non-nil Error field.
//
// When using -m, the -reuse=old.json flag accepts the name of file containing
// the JSON output of a previous 'go list -m -json' invocation with the
// same set of modifier flags (such as -u, -retracted, and -versions).
// The go command may use this file to determine that a module is unchanged
// since the previous invocation and avoid redownloading information about it.
// Modules that are not redownloaded will be marked in the new output by
// setting the Reuse field to true. Normally the module cache provides this
// kind of reuse automatically; the -reuse flag can be useful on systems that
// do not preserve the module cache.
//
// For more about build flags, see 'go help build'.
//
// For more about specifying packages, see 'go help packages'.
@ -1055,7 +1068,7 @@
//
// Usage:
//
// go mod download [-x] [-json] [modules]
// go mod download [-x] [-json] [-reuse=old.json] [modules]
//
// Download downloads the named modules, which can be module patterns selecting
// dependencies of the main module or module queries of the form path@version.
@ -1078,6 +1091,7 @@
//
// type Module struct {
// Path string // module path
// Query string // version query corresponding to this version
// Version string // module version
// Error string // error loading module
// Info string // absolute path to cached .info file
@ -1086,8 +1100,18 @@
// Dir string // absolute path to cached source root directory
// Sum string // checksum for path, version (as in go.sum)
// GoModSum string // checksum for go.mod (as in go.sum)
// Origin any // provenance of module
// Reuse bool // reuse of old module info is safe
// }
//
// The -reuse flag accepts the name of file containing the JSON output of a
// previous 'go mod download -json' invocation. The go command may use this
// file to determine that a module is unchanged since the previous invocation
// and avoid redownloading it. Modules that are not redownloaded will be marked
// in the new output by setting the Reuse field to true. Normally the module
// cache provides this kind of reuse automatically; the -reuse flag can be
// useful on systems that do not preserve the module cache.
//
// The -x flag causes download to print the commands download executes.
//
// See https://golang.org/ref/mod#go-mod-download for more about 'go mod download'.

27
src/cmd/go/internal/list/list.go
View file

@ -223,6 +223,7 @@ applied to a Go struct, but now a Module struct:
type Module struct {
Path string // module path
Query string // version query corresponding to this version
Version string // module version
Versions []string // available module versions
Replace *Module // replaced by this module
@ -236,6 +237,8 @@ applied to a Go struct, but now a Module struct:
Retracted []string // retraction information, if any (with -retracted or -u)
Deprecated string // deprecation message, if any (with -u)
Error *ModuleError // error loading module
Origin any // provenance of module
Reuse bool // reuse of old module info is safe
}
type ModuleError struct {
@ -312,6 +315,16 @@ that must be a module path or query and returns the specified
module as a Module struct. If an error occurs, the result will
be a Module struct with a non-nil Error field.
When using -m, the -reuse=old.json flag accepts the name of file containing
the JSON output of a previous 'go list -m -json' invocation with the
same set of modifier flags (such as -u, -retracted, and -versions).
The go command may use this file to determine that a module is unchanged
since the previous invocation and avoid redownloading information about it.
Modules that are not redownloaded will be marked in the new output by
setting the Reuse field to true. Normally the module cache provides this
kind of reuse automatically; the -reuse flag can be useful on systems that
do not preserve the module cache.
For more about build flags, see 'go help build'.
For more about specifying packages, see 'go help packages'.
@ -337,6 +350,7 @@ var (
listJsonFields jsonFlag // If not empty, only output these fields.
listM = CmdList.Flag.Bool("m", false, "")
listRetracted = CmdList.Flag.Bool("retracted", false, "")
listReuse = CmdList.Flag.String("reuse", "", "")
listTest = CmdList.Flag.Bool("test", false, "")
listU = CmdList.Flag.Bool("u", false, "")
listVersions = CmdList.Flag.Bool("versions", false, "")
@ -398,6 +412,12 @@ func runList(ctx context.Context, cmd *base.Command, args []string) {
if *listFmt != "" && listJson == true {
base.Fatalf("go list -f cannot be used with -json")
}
if *listReuse != "" && !*listM {
base.Fatalf("go list -reuse cannot be used without -m")
}
if *listReuse != "" && modload.HasModRoot() {
base.Fatalf("go list -reuse cannot be used inside a module")
}
work.BuildInit()
out := newTrackingWriter(os.Stdout)
@ -532,7 +552,10 @@ func runList(ctx context.Context, cmd *base.Command, args []string) {
mode |= modload.ListRetractedVersions
}
}
mods, err := modload.ListModules(ctx, args, mode)
if *listReuse != "" && len(args) == 0 {
base.Fatalf("go: list -m -reuse only has an effect with module@version arguments")
}
mods, err := modload.ListModules(ctx, args, mode, *listReuse)
if !*listE {
for _, m := range mods {
if m.Error != nil {
@ -783,7 +806,7 @@ func runList(ctx context.Context, cmd *base.Command, args []string) {
if *listRetracted {
mode |= modload.ListRetracted
}
rmods, err := modload.ListModules(ctx, args, mode)
rmods, err := modload.ListModules(ctx, args, mode, *listReuse)
if err != nil && !*listE {
base.Errorf("go: %v", err)
}

40
src/cmd/go/internal/modcmd/download.go
View file

@ -13,6 +13,7 @@ import (
"cmd/go/internal/base"
"cmd/go/internal/cfg"
"cmd/go/internal/modfetch"
"cmd/go/internal/modfetch/codehost"
"cmd/go/internal/modload"
"golang.org/x/mod/module"
@ -20,7 +21,7 @@ import (
)
var cmdDownload = &base.Command{
UsageLine: "go mod download [-x] [-json] [modules]",
UsageLine: "go mod download [-x] [-json] [-reuse=old.json] [modules]",
Short: "download modules to local cache",
Long: `
Download downloads the named modules, which can be module patterns selecting
@ -44,6 +45,7 @@ corresponding to this Go struct:
type Module struct {
Path string // module path
Query string // version query corresponding to this version
Version string // module version
Error string // error loading module
Info string // absolute path to cached .info file
@ -52,8 +54,18 @@ corresponding to this Go struct:
Dir string // absolute path to cached source root directory
Sum string // checksum for path, version (as in go.sum)
GoModSum string // checksum for go.mod (as in go.sum)
Origin any // provenance of module
Reuse bool // reuse of old module info is safe
}
The -reuse flag accepts the name of file containing the JSON output of a
previous 'go mod download -json' invocation. The go command may use this
file to determine that a module is unchanged since the previous invocation
and avoid redownloading it. Modules that are not redownloaded will be marked
in the new output by setting the Reuse field to true. Normally the module
cache provides this kind of reuse automatically; the -reuse flag can be
useful on systems that do not preserve the module cache.
The -x flag causes download to print the commands download executes.
See https://golang.org/ref/mod#go-mod-download for more about 'go mod download'.
@ -62,7 +74,10 @@ See https://golang.org/ref/mod#version-queries for more about version queries.
`,
}
var downloadJSON = cmdDownload.Flag.Bool("json", false, "")
var (
downloadJSON = cmdDownload.Flag.Bool("json", false, "")
downloadReuse = cmdDownload.Flag.String("reuse", "", "")
)
func init() {
cmdDownload.Run = runDownload // break init cycle
@ -75,6 +90,7 @@ func init() {
type moduleJSON struct {
Path string `json:",omitempty"`
Version string `json:",omitempty"`
Query string `json:",omitempty"`
Error string `json:",omitempty"`
Info string `json:",omitempty"`
GoMod string `json:",omitempty"`
@ -82,6 +98,9 @@ type moduleJSON struct {
Dir string `json:",omitempty"`
Sum string `json:",omitempty"`
GoModSum string `json:",omitempty"`
Origin *codehost.Origin `json:",omitempty"`
Reuse bool `json:",omitempty"`
}
func runDownload(ctx context.Context, cmd *base.Command, args []string) {
@ -148,12 +167,12 @@ func runDownload(ctx context.Context, cmd *base.Command, args []string) {
}
downloadModule := func(m *moduleJSON) {
var err error
_, m.Info, err = modfetch.InfoFile(m.Path, m.Version)
_, file, err := modfetch.InfoFile(m.Path, m.Version)
if err != nil {
m.Error = err.Error()
return
}
m.Info = file
m.GoMod, err = modfetch.GoModFile(m.Path, m.Version)
if err != nil {
m.Error = err.Error()
@ -179,9 +198,14 @@ func runDownload(ctx context.Context, cmd *base.Command, args []string) {
}
var mods []*moduleJSON
if *downloadReuse != "" && modload.HasModRoot() {
base.Fatalf("go mod download -reuse cannot be used inside a module")
}
type token struct{}
sem := make(chan token, runtime.GOMAXPROCS(0))
infos, infosErr := modload.ListModules(ctx, args, 0)
infos, infosErr := modload.ListModules(ctx, args, 0, *downloadReuse)
if !haveExplicitArgs {
// 'go mod download' is sometimes run without arguments to pre-populate the
// module cache. It may fetch modules that aren't needed to build packages
@ -209,12 +233,18 @@ func runDownload(ctx context.Context, cmd *base.Command, args []string) {
m := &moduleJSON{
Path: info.Path,
Version: info.Version,
Query: info.Query,
Reuse: info.Reuse,
Origin: info.Origin,
}
mods = append(mods, m)
if info.Error != nil {
m.Error = info.Error.Err
continue
}
if m.Reuse {
continue
}
sem <- token{}
go func() {
downloadModule(m)

2
src/cmd/go/internal/modcmd/why.go
View file

@ -82,7 +82,7 @@ func runWhy(ctx context.Context, cmd *base.Command, args []string) {
}
}
mods, err := modload.ListModules(ctx, args, 0)
mods, err := modload.ListModules(ctx, args, 0, "")
if err != nil {
base.Fatalf("go: %v", err)
}

20
src/cmd/go/internal/modfetch/cache.go
View file

@ -573,6 +573,26 @@ func writeDiskStat(file string, info *RevInfo) error {
if file == "" {
return nil
}
if info.Origin != nil {
// Clean the origin information, which might have too many
// validation criteria, for example if we are saving the result of
// m@master as m@pseudo-version.
clean := *info
info = &clean
o := *info.Origin
info.Origin = &o
// Tags never matter if you are starting with a semver version,
// as we would be when finding this cache entry.
o.TagSum = ""
o.TagPrefix = ""
// Ref doesn't matter if you have a pseudoversion.
if module.IsPseudoVersion(info.Version) {
o.Ref = ""
}
}
js, err := json.Marshal(info)
if err != nil {
return err

17
src/cmd/go/internal/modfetch/codehost/codehost.go
View file

@ -113,20 +113,17 @@ type Origin struct {
}
// Checkable reports whether the Origin contains anything that can be checked.
// If not, it's purely informational and should fail a CheckReuse call.
// If not, the Origin is purely informational and should fail a CheckReuse call.
func (o *Origin) Checkable() bool {
return o.TagSum != "" || o.Ref != "" || o.Hash != ""
}
func (o *Origin) Merge(other *Origin) {
if o.TagSum == "" {
o.TagPrefix = other.TagPrefix
o.TagSum = other.TagSum
}
if o.Ref == "" {
o.Ref = other.Ref
o.Hash = other.Hash
}
// ClearCheckable clears the Origin enough to make Checkable return false.
func (o *Origin) ClearCheckable() {
o.TagSum = ""
o.TagPrefix = ""
o.Ref = ""
o.Hash = ""
}
// A Tags describes the available tags in a code repository.

5
src/cmd/go/internal/modfetch/codehost/git.go
View file

@ -423,8 +423,11 @@ func (r *gitRepo) stat(rev string) (info *RevInfo, err error) {
defer func() {
if info != nil {
info.Origin.Ref = ref
info.Origin.Hash = info.Name
// There's a ref = hash below; don't write that hash down as Origin.Ref.
if ref != info.Origin.Hash {
info.Origin.Ref = ref
}
}
}()

36
src/cmd/go/internal/modfetch/coderepo.go
View file

@ -153,6 +153,9 @@ func (r *codeRepo) Versions(prefix string) (*Versions, error) {
Err: err,
}
}
if tags.Origin != nil {
tags.Origin.Subdir = r.codeDir
}
var list, incompatible []string
for _, tag := range tags.List {
@ -450,23 +453,26 @@ func (r *codeRepo) convert(info *codehost.RevInfo, statVers string) (*RevInfo, e
}
origin := info.Origin
if module.IsPseudoVersion(v) {
// Add tags that are relevant to pseudo-version calculation to origin.
prefix := ""
if r.codeDir != "" {
prefix = r.codeDir + "/"
}
if r.pathMajor != "" { // "/v2" or "/.v2"
prefix += r.pathMajor[1:] + "." // += "v2."
}
tags, err := r.code.Tags(prefix)
if err != nil {
return nil, err
}
if origin != nil {
o := *origin
origin = &o
origin.TagPrefix = tags.Origin.TagPrefix
origin.TagSum = tags.Origin.TagSum
origin.Subdir = r.codeDir
if module.IsPseudoVersion(v) && (v != statVers || !strings.HasPrefix(v, "v0.0.0-")) {
// Add tags that are relevant to pseudo-version calculation to origin.
prefix := r.codeDir
if prefix != "" {
prefix += "/"
}
if r.pathMajor != "" { // "/v2" or "/.v2"
prefix += r.pathMajor[1:] + "." // += "v2."
}
tags, err := r.code.Tags(prefix)
if err != nil {
return nil, err
}
origin.TagPrefix = tags.Origin.TagPrefix
origin.TagSum = tags.Origin.TagSum
}
}
return &RevInfo{

22
src/cmd/go/internal/modinfo/info.go
View file

@ -4,7 +4,11 @@
package modinfo
import "time"
import (
"cmd/go/internal/modfetch/codehost"
"encoding/json"
"time"
)
// Note that these structs are publicly visible (part of go list's API)
// and the fields are documented in the help text in ../list/list.go
@ -12,6 +16,7 @@ import "time"
type ModulePublic struct {
Path string `json:",omitempty"` // module path
Version string `json:",omitempty"` // module version
Query string `json:",omitempty"` // version query corresponding to this version
Versions []string `json:",omitempty"` // available module versions
Replace *ModulePublic `json:",omitempty"` // replaced by this module
Time *time.Time `json:",omitempty"` // time version was created
@ -24,12 +29,27 @@ type ModulePublic struct {
Retracted []string `json:",omitempty"` // retraction information, if any (with -retracted or -u)
Deprecated string `json:",omitempty"` // deprecation message, if any (with -u)
Error *ModuleError `json:",omitempty"` // error loading module
Origin *codehost.Origin `json:",omitempty"` // provenance of module
Reuse bool `json:",omitempty"` // reuse of old module info is safe
}
type ModuleError struct {
Err string // error text
}
type moduleErrorNoMethods ModuleError
// UnmarshalJSON accepts both {"Err":"text"} and "text",
// so that the output of go mod download -json can still
// be unmarshalled into a ModulePublic during -reuse processing.
func (e *ModuleError) UnmarshalJSON(data []byte) error {
if len(data) > 0 && data[0] == '"' {
return json.Unmarshal(data, &e.Err)
}
return json.Unmarshal(data, (*moduleErrorNoMethods)(e))
}
func (m *ModulePublic) String() string {
s := m.Path
versionString := func(mm *ModulePublic) string {

62
src/cmd/go/internal/modload/build.go
View file

@ -17,6 +17,7 @@ import (
"cmd/go/internal/base"
"cmd/go/internal/cfg"
"cmd/go/internal/modfetch"
"cmd/go/internal/modfetch/codehost"
"cmd/go/internal/modindex"
"cmd/go/internal/modinfo"
"cmd/go/internal/search"
@ -60,7 +61,7 @@ func PackageModuleInfo(ctx context.Context, pkgpath string) *modinfo.ModulePubli
}
rs := LoadModFile(ctx)
return moduleInfo(ctx, rs, m, 0)
return moduleInfo(ctx, rs, m, 0, nil)
}
// PackageModRoot returns the module root directory for the module that provides
@ -90,7 +91,7 @@ func ModuleInfo(ctx context.Context, path string) *modinfo.ModulePublic {
if i := strings.Index(path, "@"); i >= 0 {
m := module.Version{Path: path[:i], Version: path[i+1:]}
return moduleInfo(ctx, nil, m, 0)
return moduleInfo(ctx, nil, m, 0, nil)
}
rs := LoadModFile(ctx)
@ -119,7 +120,7 @@ func ModuleInfo(ctx context.Context, path string) *modinfo.ModulePublic {
}
}
return moduleInfo(ctx, rs, module.Version{Path: path, Version: v}, 0)
return moduleInfo(ctx, rs, module.Version{Path: path, Version: v}, 0, nil)
}
// addUpdate fills in m.Update if an updated version is available.
@ -156,6 +157,45 @@ func addUpdate(ctx context.Context, m *modinfo.ModulePublic) {
}
}
// mergeOrigin merges two origins,
// returning and possibly modifying one of its arguments.
// If the two origins conflict, mergeOrigin returns a non-specific one
// that will not pass CheckReuse.
// If m1 or m2 is nil, the other is returned unmodified.
// But if m1 or m2 is non-nil and uncheckable, the result is also uncheckable,
// to preserve uncheckability.
func mergeOrigin(m1, m2 *codehost.Origin) *codehost.Origin {
if m1 == nil {
return m2
}
if m2 == nil {
return m1
}
if !m1.Checkable() {
return m1
}
if !m2.Checkable() {
return m2
}
if m2.TagSum != "" {
if m1.TagSum != "" && (m1.TagSum != m2.TagSum || m1.TagPrefix != m2.TagPrefix) {
m1.ClearCheckable()
return m1
}
m1.TagSum = m2.TagSum
m1.TagPrefix = m2.TagPrefix
}
if m2.Hash != "" {
if m1.Hash != "" && (m1.Hash != m2.Hash || m1.Ref != m2.Ref) {
m1.ClearCheckable()
return m1
}
m1.Hash = m2.Hash
m1.Ref = m2.Ref
}
return m1
}
// addVersions fills in m.Versions with the list of known versions.
// Excluded versions will be omitted. If listRetracted is false, retracted
// versions will also be omitted.
@ -164,11 +204,12 @@ func addVersions(ctx context.Context, m *modinfo.ModulePublic, listRetracted boo
if listRetracted {
allowed = CheckExclusions
}
var err error
m.Versions, err = versions(ctx, m.Path, allowed)
v, origin, err := versions(ctx, m.Path, allowed)
if err != nil && m.Error == nil {
m.Error = &modinfo.ModuleError{Err: err.Error()}
}
m.Versions = v
m.Origin = mergeOrigin(m.Origin, origin)
}
// addRetraction fills in m.Retracted if the module was retracted by its author.
@ -230,7 +271,7 @@ func addDeprecation(ctx context.Context, m *modinfo.ModulePublic) {
// moduleInfo returns information about module m, loaded from the requirements
// in rs (which may be nil to indicate that m was not loaded from a requirement
// graph).
func moduleInfo(ctx context.Context, rs *Requirements, m module.Version, mode ListMode) *modinfo.ModulePublic {
func moduleInfo(ctx context.Context, rs *Requirements, m module.Version, mode ListMode, reuse map[module.Version]*modinfo.ModulePublic) *modinfo.ModulePublic {
if m.Version == "" && MainModules.Contains(m.Path) {
info := &modinfo.ModulePublic{
Path: m.Path,
@ -260,6 +301,15 @@ func moduleInfo(ctx context.Context, rs *Requirements, m module.Version, mode Li
// completeFromModCache fills in the extra fields in m using the module cache.
completeFromModCache := func(m *modinfo.ModulePublic) {
if old := reuse[module.Version{Path: m.Path, Version: m.Version}]; old != nil {
if err := checkReuse(ctx, m.Path, old.Origin); err == nil {
*m = *old
m.Query = ""
m.Dir = ""
return
}
}
checksumOk := func(suffix string) bool {
return rs == nil || m.Version == "" || cfg.BuildMod == "mod" ||
modfetch.HaveSum(module.Version{Path: m.Path, Version: m.Version + suffix})

2
src/cmd/go/internal/modload/edit.go
View file

@ -509,7 +509,7 @@ func (l *versionLimiter) UpgradeToward(ctx context.Context, m module.Version) er
}
if l.check(m, l.pruning).isDisqualified() {
candidates, err := versions(ctx, m.Path, CheckAllowed)
candidates, _, err := versions(ctx, m.Path, CheckAllowed)
if err != nil {
// This is likely a transient error reaching the repository,
// rather than a permanent error with the retrieved version.

60
src/cmd/go/internal/modload/list.go
View file

@ -5,15 +5,19 @@
package modload
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"runtime"
"strings"
"cmd/go/internal/base"
"cmd/go/internal/cfg"
"cmd/go/internal/modfetch/codehost"
"cmd/go/internal/modinfo"
"cmd/go/internal/search"
@ -34,13 +38,44 @@ const (
// along with any error preventing additional matches from being identified.
//
// The returned slice can be nonempty even if the error is non-nil.
func ListModules(ctx context.Context, args []string, mode ListMode) ([]*modinfo.ModulePublic, error) {
rs, mods, err := listModules(ctx, LoadModFile(ctx), args, mode)
func ListModules(ctx context.Context, args []string, mode ListMode, reuseFile string) ([]*modinfo.ModulePublic, error) {
var reuse map[module.Version]*modinfo.ModulePublic
if reuseFile != "" {
data, err := os.ReadFile(reuseFile)
if err != nil {
return nil, err
}
dec := json.NewDecoder(bytes.NewReader(data))
reuse = make(map[module.Version]*modinfo.ModulePublic)
for {
var m modinfo.ModulePublic
if err := dec.Decode(&m); err != nil {
if err == io.EOF {
break
}
return nil, fmt.Errorf("parsing %s: %v", reuseFile, err)
}
if m.Origin == nil || !m.Origin.Checkable() {
// Nothing to check to validate reuse.
continue
}
m.Reuse = true
reuse[module.Version{Path: m.Path, Version: m.Version}] = &m
if m.Query != "" {
reuse[module.Version{Path: m.Path, Version: m.Query}] = &m
}
}
}
rs, mods, err := listModules(ctx, LoadModFile(ctx), args, mode, reuse)
type token struct{}
sem := make(chan token, runtime.GOMAXPROCS(0))
if mode != 0 {
for _, m := range mods {
if m.Reuse {
continue
}
add := func(m *modinfo.ModulePublic) {
sem <- token{}
go func() {
@ -80,11 +115,11 @@ func ListModules(ctx context.Context, args []string, mode ListMode) ([]*modinfo.
return mods, err
}
func listModules(ctx context.Context, rs *Requirements, args []string, mode ListMode) (_ *Requirements, mods []*modinfo.ModulePublic, mgErr error) {
func listModules(ctx context.Context, rs *Requirements, args []string, mode ListMode, reuse map[module.Version]*modinfo.ModulePublic) (_ *Requirements, mods []*modinfo.ModulePublic, mgErr error) {
if len(args) == 0 {
var ms []*modinfo.ModulePublic
for _, m := range MainModules.Versions() {
ms = append(ms, moduleInfo(ctx, rs, m, mode))
ms = append(ms, moduleInfo(ctx, rs, m, mode, reuse))
}
return rs, ms, nil
}
@ -157,12 +192,17 @@ func listModules(ctx context.Context, rs *Requirements, args []string, mode List
// specific revision or used 'go list -retracted'.
allowed = nil
}
info, err := Query(ctx, path, vers, current, allowed)
info, err := queryReuse(ctx, path, vers, current, allowed, reuse)
if err != nil {
var origin *codehost.Origin
if info != nil {
origin = info.Origin
}
mods = append(mods, &modinfo.ModulePublic{
Path: path,
Version: vers,
Error: modinfoError(path, vers, err),
Origin: origin,
})
continue
}
@ -171,7 +211,11 @@ func listModules(ctx context.Context, rs *Requirements, args []string, mode List
// *Requirements instead.
var noRS *Requirements
mod := moduleInfo(ctx, noRS, module.Version{Path: path, Version: info.Version}, mode)
mod := moduleInfo(ctx, noRS, module.Version{Path: path, Version: info.Version}, mode, reuse)
if vers != mod.Version {
mod.Query = vers
}
mod.Origin = info.Origin
mods = append(mods, mod)
continue
}
@ -200,7 +244,7 @@ func listModules(ctx context.Context, rs *Requirements, args []string, mode List
continue
}
if v != "none" {
mods = append(mods, moduleInfo(ctx, rs, module.Version{Path: arg, Version: v}, mode))
mods = append(mods, moduleInfo(ctx, rs, module.Version{Path: arg, Version: v}, mode, reuse))
} else if cfg.BuildMod == "vendor" {
// In vendor mode, we can't determine whether a missing module is “a
// known dependency” because the module graph is incomplete.
@ -229,7 +273,7 @@ func listModules(ctx context.Context, rs *Requirements, args []string, mode List
matched = true
if !matchedModule[m] {
matchedModule[m] = true
mods = append(mods, moduleInfo(ctx, rs, m, mode))
mods = append(mods, moduleInfo(ctx, rs, m, mode, reuse))
}
}
}

11
src/cmd/go/internal/modload/mvs.go
View file

@ -11,6 +11,7 @@ import (
"sort"
"cmd/go/internal/modfetch"
"cmd/go/internal/modfetch/codehost"
"golang.org/x/mod/module"
"golang.org/x/mod/semver"
@ -78,11 +79,10 @@ func (*mvsReqs) Upgrade(m module.Version) (module.Version, error) {
return m, nil
}
func versions(ctx context.Context, path string, allowed AllowedFunc) ([]string, error) {
func versions(ctx context.Context, path string, allowed AllowedFunc) (versions []string, origin *codehost.Origin, err error) {
// Note: modfetch.Lookup and repo.Versions are cached,
// so there's no need for us to add extra caching here.
var versions []string
err := modfetch.TryProxies(func(proxy string) error {
err = modfetch.TryProxies(func(proxy string) error {
repo, err := lookupRepo(proxy, path)
if err != nil {
return err
@ -100,9 +100,10 @@ func versions(ctx context.Context, path string, allowed AllowedFunc) ([]string,
}
}
versions = allowedVersions
origin = allVersions.Origin
return nil
})
return versions, err
return versions, origin, err
}
// previousVersion returns the tagged version of m.Path immediately prior to
@ -117,7 +118,7 @@ func previousVersion(m module.Version) (module.Version, error) {
return module.Version{Path: m.Path, Version: "none"}, nil
}
list, err := versions(context.TODO(), m.Path, CheckAllowed)
list, _, err := versions(context.TODO(), m.Path, CheckAllowed)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return module.Version{Path: m.Path, Version: "none"}, nil

75
src/cmd/go/internal/modload/query.go
View file

@ -20,6 +20,8 @@ import (
"cmd/go/internal/cfg"
"cmd/go/internal/imports"
"cmd/go/internal/modfetch"
"cmd/go/internal/modfetch/codehost"
"cmd/go/internal/modinfo"
"cmd/go/internal/search"
"cmd/go/internal/str"
"cmd/go/internal/trace"
@ -72,18 +74,39 @@ import (
//
// If path is the path of the main module and the query is "latest",
// Query returns Target.Version as the version.
//
// Query often returns a non-nil *RevInfo with a non-nil error,
// to provide an info.Origin that can allow the error to be cached.
func Query(ctx context.Context, path, query, current string, allowed AllowedFunc) (*modfetch.RevInfo, error) {
ctx, span := trace.StartSpan(ctx, "modload.Query "+path)
defer span.Done()
return queryReuse(ctx, path, query, current, allowed, nil)
}
// queryReuse is like Query but also takes a map of module info that can be reused
// if the validation criteria in Origin are met.
func queryReuse(ctx context.Context, path, query, current string, allowed AllowedFunc, reuse map[module.Version]*modinfo.ModulePublic) (*modfetch.RevInfo, error) {
var info *modfetch.RevInfo
err := modfetch.TryProxies(func(proxy string) (err error) {
info, err = queryProxy(ctx, proxy, path, query, current, allowed)
info, err = queryProxy(ctx, proxy, path, query, current, allowed, reuse)
return err
})
return info, err
}
// checkReuse checks whether a revision of a given module or a version list
// for a given module may be reused, according to the information in origin.
func checkReuse(ctx context.Context, path string, old *codehost.Origin) error {
return modfetch.TryProxies(func(proxy string) error {
repo, err := lookupRepo(proxy, path)
if err != nil {
return err
}
return repo.CheckReuse(old)
})
}
// AllowedFunc is used by Query and other functions to filter out unsuitable
// versions, for example, those listed in exclude directives in the main
// module's go.mod file.
@ -106,7 +129,7 @@ func (queryDisabledError) Error() string {
return fmt.Sprintf("cannot query module due to -mod=%s\n\t(%s)", cfg.BuildMod, cfg.BuildModReason)
}
func queryProxy(ctx context.Context, proxy, path, query, current string, allowed AllowedFunc) (*modfetch.RevInfo, error) {
func queryProxy(ctx context.Context, proxy, path, query, current string, allowed AllowedFunc, reuse map[module.Version]*modinfo.ModulePublic) (*modfetch.RevInfo, error) {
ctx, span := trace.StartSpan(ctx, "modload.queryProxy "+path+" "+query)
defer span.Done()
@ -137,6 +160,19 @@ func queryProxy(ctx context.Context, proxy, path, query, current string, allowed
return nil, err
}
if old := reuse[module.Version{Path: path, Version: query}]; old != nil {
if err := repo.CheckReuse(old.Origin); err == nil {
info := &modfetch.RevInfo{
Version: old.Version,
Origin: old.Origin,
}
if old.Time != nil {
info.Time = *old.Time
}
return info, nil
}
}
// Parse query to detect parse errors (and possibly handle query)
// before any network I/O.
qm, err := newQueryMatcher(path, query, current, allowed)
@ -177,15 +213,23 @@ func queryProxy(ctx context.Context, proxy, path, query, current string, allowed
if err != nil {
return nil, err
}
revErr := &modfetch.RevInfo{Origin: versions.Origin} // RevInfo to return with error
releases, prereleases, err := qm.filterVersions(ctx, versions.List)
if err != nil {
return nil, err
return revErr, err
}
lookup := func(v string) (*modfetch.RevInfo, error) {
rev, err := repo.Stat(v)
// Stat can return a non-nil rev and a non-nil err,
// in order to provide origin information to make the error cacheable.
if rev == nil && err != nil {
return revErr, err
}
rev.Origin = mergeOrigin(rev.Origin, versions.Origin)
if err != nil {
return nil, err
return rev, err
}
if (query == "upgrade" || query == "patch") && module.IsPseudoVersion(current) && !rev.Time.IsZero() {
@ -210,9 +254,14 @@ func queryProxy(ctx context.Context, proxy, path, query, current string, allowed
currentTime, err := module.PseudoVersionTime(current)
if err == nil && rev.Time.Before(currentTime) {
if err := allowed(ctx, module.Version{Path: path, Version: current}); errors.Is(err, ErrDisallowed) {
return nil, err
return revErr, err
}
return repo.Stat(current)
info, err := repo.Stat(current)
if info == nil && err != nil {
return revErr, err
}
info.Origin = mergeOrigin(info.Origin, versions.Origin)
return info, err
}
}
@ -242,7 +291,7 @@ func queryProxy(ctx context.Context, proxy, path, query, current string, allowed
return lookup(latest.Version)
}
} else if !errors.Is(err, fs.ErrNotExist) {
return nil, err
return revErr, err
}
}
@ -254,7 +303,7 @@ func queryProxy(ctx context.Context, proxy, path, query, current string, allowed
return lookup(current)
}
return nil, &NoMatchingVersionError{query: query, current: current}
return revErr, &NoMatchingVersionError{query: query, current: current}
}
// IsRevisionQuery returns true if vers is a version query that may refer to
@ -663,7 +712,7 @@ func QueryPattern(ctx context.Context, pattern, query string, current func(strin
pathCurrent := current(path)
r.Mod.Path = path
r.Rev, err = queryProxy(ctx, proxy, path, query, pathCurrent, allowed)
r.Rev, err = queryProxy(ctx, proxy, path, query, pathCurrent, allowed, nil)
if err != nil {
return r, err
}
@ -991,6 +1040,7 @@ func versionHasGoMod(_ context.Context, m module.Version) (bool, error) {
// available versions, but cannot fetch specific source files.
type versionRepo interface {
ModulePath() string
CheckReuse(*codehost.Origin) error
Versions(prefix string) (*modfetch.Versions, error)
Stat(rev string) (*modfetch.RevInfo, error)
Latest() (*modfetch.RevInfo, error)
@ -1024,6 +1074,9 @@ type emptyRepo struct {
var _ versionRepo = emptyRepo{}
func (er emptyRepo) ModulePath() string { return er.path }
func (er emptyRepo) CheckReuse(old *codehost.Origin) error {
return fmt.Errorf("empty repo")
}
func (er emptyRepo) Versions(prefix string) (*modfetch.Versions, error) {
return &modfetch.Versions{}, nil
}
@ -1044,6 +1097,10 @@ var _ versionRepo = (*replacementRepo)(nil)
func (rr *replacementRepo) ModulePath() string { return rr.repo.ModulePath() }
func (rr *replacementRepo) CheckReuse(old *codehost.Origin) error {
return fmt.Errorf("replacement repo")
}
// Versions returns the versions from rr.repo augmented with any matching
// replacement versions.
func (rr *replacementRepo) Versions(prefix string) (*modfetch.Versions, error) {

371
src/cmd/go/testdata/script/reuse_git.txt vendored Normal file
View file

@ -0,0 +1,371 @@
[short] skip
[!exec:git] skip
[!net] skip
env GO111MODULE=on
env GOPROXY=direct
env GOSUMDB=off
# go mod download with the pseudo-version should invoke git but not have a TagSum or Ref.
go mod download -x -json vcs-test.golang.org/git/hello.git@v0.0.0-20170922010558-fc3a09f3dc5c
stderr 'git fetch'
cp stdout hellopseudo.json
! stdout '"(Query|TagPrefix|TagSum|Ref)"'
stdout '"Version": "v0.0.0-20170922010558-fc3a09f3dc5c"'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/hello"'
stdout '"Hash": "fc3a09f3dc5cfe0d7a743ea18f1f5226e68b3777"'
go clean -modcache
# go mod download vcstest/hello should invoke git, print origin info
go mod download -x -json vcs-test.golang.org/git/hello.git@latest
stderr 'git fetch'
cp stdout hello.json
stdout '"Version": "v0.0.0-20170922010558-fc3a09f3dc5c"'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/hello"'
stdout '"Query": "latest"'
! stdout '"TagPrefix"'
stdout '"TagSum": "t1:47DEQpj8HBSa[+]/TImW[+]5JCeuQeRkm5NMpJWZG3hSuFU="'
stdout '"Ref": "HEAD"'
stdout '"Hash": "fc3a09f3dc5cfe0d7a743ea18f1f5226e68b3777"'
# pseudo-version again should not invoke git fetch (it has the version from the @latest query)
# but still be careful not to include a TagSum or a Ref, especially not Ref set to HEAD,
# which is easy to do when reusing the cached version from the @latest query.
go mod download -x -json vcs-test.golang.org/git/hello.git@v0.0.0-20170922010558-fc3a09f3dc5c
! stderr 'git fetch'
cp stdout hellopseudo2.json
cmp hellopseudo.json hellopseudo2.json
# go mod download vcstest/hello@hash needs to check TagSum to find pseudoversion base.
go mod download -x -json vcs-test.golang.org/git/hello.git@fc3a09f3dc5c
! stderr 'git fetch'
cp stdout hellohash.json
stdout '"Version": "v0.0.0-20170922010558-fc3a09f3dc5c"'
stdout '"Query": "fc3a09f3dc5c"'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/hello"'
stdout '"TagSum": "t1:47DEQpj8HBSa[+]/TImW[+]5JCeuQeRkm5NMpJWZG3hSuFU="'
stdout '"Hash": "fc3a09f3dc5cfe0d7a743ea18f1f5226e68b3777"'
# go mod download vcstest/hello/v9 should fail, still print origin info
! go mod download -x -json vcs-test.golang.org/git/hello.git/v9@latest
cp stdout hellov9.json
stdout '"Version": "latest"'
stdout '"Error":.*no matching versions'
! stdout '"TagPrefix"'
stdout '"TagSum": "t1:47DEQpj8HBSa[+]/TImW[+]5JCeuQeRkm5NMpJWZG3hSuFU="'
! stdout '"Ref":'
! stdout '"Hash":'
# go mod download vcstest/hello/sub/v9 should also fail, print origin info with TagPrefix
! go mod download -x -json vcs-test.golang.org/git/hello.git/sub/v9@latest
cp stdout hellosubv9.json
stdout '"Version": "latest"'
stdout '"Error":.*no matching versions'
stdout '"TagPrefix": "sub/"'
stdout '"TagSum": "t1:47DEQpj8HBSa[+]/TImW[+]5JCeuQeRkm5NMpJWZG3hSuFU="'
! stdout '"Ref":'
! stdout '"Hash":'
# go mod download vcstest/tagtests should invoke git, print origin info
go mod download -x -json vcs-test.golang.org/git/tagtests.git@latest
stderr 'git fetch'
cp stdout tagtests.json
stdout '"Version": "v0.2.2"'
stdout '"Query": "latest"'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/tagtests"'
! stdout '"TagPrefix"'
stdout '"TagSum": "t1:Dp7yRKDuE8WjG0429PN9hYWjqhy2te7P9Oki/sMEOGo="'
stdout '"Ref": "refs/tags/v0.2.2"'
stdout '"Hash": "59356c8cd18c5fe9a598167d98a6843e52d57952"'
# go mod download vcstest/tagtests@v0.2.2 should print origin info, no TagSum needed
go mod download -x -json vcs-test.golang.org/git/tagtests.git@v0.2.2
cp stdout tagtestsv022.json
stdout '"Version": "v0.2.2"'
! stdout '"Query":'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/tagtests"'
! stdout '"TagPrefix"'
! stdout '"TagSum"'
stdout '"Ref": "refs/tags/v0.2.2"'
stdout '"Hash": "59356c8cd18c5fe9a598167d98a6843e52d57952"'
# go mod download vcstest/tagtests@master needs a TagSum again
go mod download -x -json vcs-test.golang.org/git/tagtests.git@master
cp stdout tagtestsmaster.json
stdout '"Version": "v0.2.3-0.20190509225625-c7818c24fa2f"'
stdout '"Query": "master"'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/tagtests"'
! stdout '"TagPrefix"'
stdout '"TagSum": "t1:Dp7yRKDuE8WjG0429PN9hYWjqhy2te7P9Oki/sMEOGo="'
stdout '"Ref": "refs/heads/master"'
stdout '"Hash": "c7818c24fa2f3f714c67d0a6d3e411c85a518d1f"'
# go mod download vcstest/prefixtagtests should invoke git, print origin info
go mod download -x -json vcs-test.golang.org/git/prefixtagtests.git/sub@latest
stderr 'git fetch'
cp stdout prefixtagtests.json
stdout '"Version": "v0.0.10"'
stdout '"Query": "latest"'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/prefixtagtests"'
stdout '"Subdir": "sub"'
stdout '"TagPrefix": "sub/"'
stdout '"TagSum": "t1:YGSbWkJ8dn9ORAr[+]BlKHFK/2ZhXLb9hVuYfTZ9D8C7g="'
stdout '"Ref": "refs/tags/sub/v0.0.10"'
stdout '"Hash": "2b7c4692e12c109263cab51b416fcc835ddd7eae"'
# go mod download of a bunch of these should fail (some are invalid) but write good JSON for later
! go mod download -json vcs-test.golang.org/git/hello.git@latest vcs-test.golang.org/git/hello.git/v9@latest vcs-test.golang.org/git/hello.git/sub/v9@latest vcs-test.golang.org/git/tagtests.git@latest vcs-test.golang.org/git/tagtests.git@v0.2.2 vcs-test.golang.org/git/tagtests.git@master
cp stdout all.json
# clean the module cache, make sure that makes go mod download re-run git fetch, clean again
go clean -modcache
go mod download -x -json vcs-test.golang.org/git/hello.git@latest
stderr 'git fetch'
go clean -modcache
# reuse go mod download vcstest/hello result
go mod download -reuse=hello.json -x -json vcs-test.golang.org/git/hello.git@latest
! stderr 'git fetch'
stdout '"Reuse": true'
stdout '"Version": "v0.0.0-20170922010558-fc3a09f3dc5c"'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/hello"'
! stdout '"TagPrefix"'
stdout '"TagSum": "t1:47DEQpj8HBSa[+]/TImW[+]5JCeuQeRkm5NMpJWZG3hSuFU="'
stdout '"Ref": "HEAD"'
stdout '"Hash": "fc3a09f3dc5cfe0d7a743ea18f1f5226e68b3777"'
! stdout '"Dir"'
! stdout '"Info"'
! stdout '"GoMod"'
! stdout '"Zip"'
# reuse go mod download vcstest/hello pseudoversion result
go mod download -reuse=hellopseudo.json -x -json vcs-test.golang.org/git/hello.git@v0.0.0-20170922010558-fc3a09f3dc5c
! stderr 'git fetch'
stdout '"Reuse": true'
stdout '"Version": "v0.0.0-20170922010558-fc3a09f3dc5c"'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/hello"'
! stdout '"(Query|TagPrefix|TagSum|Ref)"'
stdout '"Hash": "fc3a09f3dc5cfe0d7a743ea18f1f5226e68b3777"'
! stdout '"(Dir|Info|GoMod|Zip)"'
# reuse go mod download vcstest/hello@hash
go mod download -reuse=hellohash.json -x -json vcs-test.golang.org/git/hello.git@fc3a09f3dc5c
! stderr 'git fetch'
stdout '"Reuse": true'
stdout '"Query": "fc3a09f3dc5c"'
stdout '"Version": "v0.0.0-20170922010558-fc3a09f3dc5c"'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/hello"'
! stdout '"(TagPrefix|Ref)"'
stdout '"TagSum": "t1:47DEQpj8HBSa[+]/TImW[+]5JCeuQeRkm5NMpJWZG3hSuFU="'
stdout '"Hash": "fc3a09f3dc5cfe0d7a743ea18f1f5226e68b3777"'
! stdout '"(Dir|Info|GoMod|Zip)"'
# reuse go mod download vcstest/hello/v9 error result
! go mod download -reuse=hellov9.json -x -json vcs-test.golang.org/git/hello.git/v9@latest
! stderr 'git fetch'
stdout '"Reuse": true'
stdout '"Error":.*no matching versions'
! stdout '"TagPrefix"'
stdout '"TagSum": "t1:47DEQpj8HBSa[+]/TImW[+]5JCeuQeRkm5NMpJWZG3hSuFU="'
! stdout '"(Ref|Hash)":'
! stdout '"(Dir|Info|GoMod|Zip)"'
# reuse go mod download vcstest/hello/sub/v9 error result
! go mod download -reuse=hellosubv9.json -x -json vcs-test.golang.org/git/hello.git/sub/v9@latest
! stderr 'git fetch'
stdout '"Reuse": true'
stdout '"Error":.*no matching versions'
stdout '"TagPrefix": "sub/"'
stdout '"TagSum": "t1:47DEQpj8HBSa[+]/TImW[+]5JCeuQeRkm5NMpJWZG3hSuFU="'
! stdout '"(Ref|Hash)":'
! stdout '"(Dir|Info|GoMod|Zip)"'
# reuse go mod download vcstest/tagtests result
go mod download -reuse=tagtests.json -x -json vcs-test.golang.org/git/tagtests.git@latest
! stderr 'git fetch'
stdout '"Reuse": true'
stdout '"Version": "v0.2.2"'
stdout '"Query": "latest"'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/tagtests"'
! stdout '"TagPrefix"'
stdout '"TagSum": "t1:Dp7yRKDuE8WjG0429PN9hYWjqhy2te7P9Oki/sMEOGo="'
stdout '"Ref": "refs/tags/v0.2.2"'
stdout '"Hash": "59356c8cd18c5fe9a598167d98a6843e52d57952"'
! stdout '"(Dir|Info|GoMod|Zip)"'
# reuse go mod download vcstest/tagtests@v0.2.2 result
go mod download -reuse=tagtestsv022.json -x -json vcs-test.golang.org/git/tagtests.git@v0.2.2
! stderr 'git fetch'
stdout '"Reuse": true'
stdout '"Version": "v0.2.2"'
! stdout '"Query":'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/tagtests"'
! stdout '"TagPrefix"'
! stdout '"TagSum"'
stdout '"Ref": "refs/tags/v0.2.2"'
stdout '"Hash": "59356c8cd18c5fe9a598167d98a6843e52d57952"'
! stdout '"(Dir|Info|GoMod|Zip)"'
# reuse go mod download vcstest/tagtests@master result
go mod download -reuse=tagtestsmaster.json -x -json vcs-test.golang.org/git/tagtests.git@master
! stderr 'git fetch'
stdout '"Reuse": true'
stdout '"Version": "v0.2.3-0.20190509225625-c7818c24fa2f"'
stdout '"Query": "master"'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/tagtests"'
! stdout '"TagPrefix"'
stdout '"TagSum": "t1:Dp7yRKDuE8WjG0429PN9hYWjqhy2te7P9Oki/sMEOGo="'
stdout '"Ref": "refs/heads/master"'
stdout '"Hash": "c7818c24fa2f3f714c67d0a6d3e411c85a518d1f"'
! stdout '"(Dir|Info|GoMod|Zip)"'
# reuse go mod download vcstest/tagtests@master result again with all.json
go mod download -reuse=all.json -x -json vcs-test.golang.org/git/tagtests.git@master
! stderr 'git fetch'
stdout '"Reuse": true'
stdout '"Version": "v0.2.3-0.20190509225625-c7818c24fa2f"'
stdout '"Query": "master"'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/tagtests"'
! stdout '"TagPrefix"'
stdout '"TagSum": "t1:Dp7yRKDuE8WjG0429PN9hYWjqhy2te7P9Oki/sMEOGo="'
stdout '"Ref": "refs/heads/master"'
stdout '"Hash": "c7818c24fa2f3f714c67d0a6d3e411c85a518d1f"'
! stdout '"(Dir|Info|GoMod|Zip)"'
# go mod download vcstest/prefixtagtests result with json
go mod download -reuse=prefixtagtests.json -x -json vcs-test.golang.org/git/prefixtagtests.git/sub@latest
! stderr 'git fetch'
stdout '"Version": "v0.0.10"'
stdout '"Query": "latest"'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/prefixtagtests"'
stdout '"Subdir": "sub"'
stdout '"TagPrefix": "sub/"'
stdout '"TagSum": "t1:YGSbWkJ8dn9ORAr[+]BlKHFK/2ZhXLb9hVuYfTZ9D8C7g="'
stdout '"Ref": "refs/tags/sub/v0.0.10"'
stdout '"Hash": "2b7c4692e12c109263cab51b416fcc835ddd7eae"'
! stdout '"(Dir|Info|GoMod|Zip)"'
# reuse the bulk results with all.json
! go mod download -reuse=all.json -json vcs-test.golang.org/git/hello.git@latest vcs-test.golang.org/git/hello.git/v9@latest vcs-test.golang.org/git/hello.git/sub/v9@latest vcs-test.golang.org/git/tagtests.git@latest vcs-test.golang.org/git/tagtests.git@v0.2.2 vcs-test.golang.org/git/tagtests.git@master
! stderr 'git fetch'
stdout '"Reuse": true'
! stdout '"(Dir|Info|GoMod|Zip)"'
# reuse attempt with stale hash should reinvoke git, not report reuse
go mod download -reuse=tagtestsv022badhash.json -x -json vcs-test.golang.org/git/tagtests.git@v0.2.2
stderr 'git fetch'
! stdout '"Reuse": true'
stdout '"Version": "v0.2.2"'
! stdout '"Query"'
stdout '"VCS": "git"'
stdout '"URL": "https://vcs-test.golang.org/git/tagtests"'
! stdout '"(TagPrefix|TagSum)"'
stdout '"Ref": "refs/tags/v0.2.2"'
stdout '"Hash": "59356c8cd18c5fe9a598167d98a6843e52d57952"'
stdout '"Dir"'
stdout '"Info"'
stdout '"GoMod"'
stdout '"Zip"'
# reuse with stale repo URL
go mod download -reuse=tagtestsv022badurl.json -x -json vcs-test.golang.org/git/tagtests.git@v0.2.2
! stdout '"Reuse": true'
stdout '"URL": "https://vcs-test.golang.org/git/tagtests"'
stdout '"Dir"'
stdout '"Info"'
stdout '"GoMod"'
stdout '"Zip"'
# reuse with stale VCS
go mod download -reuse=tagtestsv022badvcs.json -x -json vcs-test.golang.org/git/tagtests.git@v0.2.2
! stdout '"Reuse": true'
stdout '"URL": "https://vcs-test.golang.org/git/tagtests"'
# reuse with stale Dir
go mod download -reuse=tagtestsv022baddir.json -x -json vcs-test.golang.org/git/tagtests.git@v0.2.2
! stdout '"Reuse": true'
stdout '"URL": "https://vcs-test.golang.org/git/tagtests"'
# reuse with stale TagSum
go mod download -reuse=tagtestsbadtagsum.json -x -json vcs-test.golang.org/git/tagtests.git@latest
! stdout '"Reuse": true'
stdout '"TagSum": "t1:Dp7yRKDuE8WjG0429PN9hYWjqhy2te7P9Oki/sMEOGo="'
-- tagtestsv022badhash.json --
{
"Path": "vcs-test.golang.org/git/tagtests.git",
"Version": "v0.2.2",
"Origin": {
"VCS": "git",
"URL": "https://vcs-test.golang.org/git/tagtests",
"Ref": "refs/tags/v0.2.2",
"Hash": "59356c8cd18c5fe9a598167d98a6843e52d57952XXX"
}
}
-- tagtestsbadtagsum.json --
{
"Path": "vcs-test.golang.org/git/tagtests.git",
"Version": "v0.2.2",
"Query": "latest",
"Origin": {
"VCS": "git",
"URL": "https://vcs-test.golang.org/git/tagtests",
"TagSum": "t1:Dp7yRKDuE8WjG0429PN9hYWjqhy2te7P9Oki/sMEOGo=XXX",
"Ref": "refs/tags/v0.2.2",
"Hash": "59356c8cd18c5fe9a598167d98a6843e52d57952"
},
"Reuse": true
}
-- tagtestsv022badvcs.json --
{
"Path": "vcs-test.golang.org/git/tagtests.git",
"Version": "v0.2.2",
"Origin": {
"VCS": "gitXXX",
"URL": "https://vcs-test.golang.org/git/tagtests",
"Ref": "refs/tags/v0.2.2",
"Hash": "59356c8cd18c5fe9a598167d98a6843e52d57952"
}
}
-- tagtestsv022baddir.json --
{
"Path": "vcs-test.golang.org/git/tagtests.git",
"Version": "v0.2.2",
"Origin": {
"VCS": "git",
"URL": "https://vcs-test.golang.org/git/tagtests",
"Subdir": "subdir",
"Ref": "refs/tags/v0.2.2",
"Hash": "59356c8cd18c5fe9a598167d98a6843e52d57952"
}
}
-- tagtestsv022badurl.json --
{
"Path": "vcs-test.golang.org/git/tagtests.git",
"Version": "v0.2.2",
"Origin": {
"VCS": "git",
"URL": "https://vcs-test.golang.org/git/tagtestsXXX",
"Ref": "refs/tags/v0.2.2",
"Hash": "59356c8cd18c5fe9a598167d98a6843e52d57952"
}
}