cmd/compile: fix deadlock in (*Named).load

For lazy import resolution, there's reentrancy issue with (*Named).load
method, when "t.resolve(t)" can lead us to the same named type, thus
(*Named).load is called recursively, causing the deadlock.

The main problem is that when instantinate a type, we calculate the type
hashing, including TParams. Calling t.TParams().Len() triggers the
reentrancy call to "(*Named).load".

To fix this, just not checking TParams().Len() if we are hashing.

Updates #48185

Change-Id: Ie34842d7b10fad5d11fbcf75bb1c64a89deac6b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/347534
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>

src/cmd/compile/internal/types2/typestring.go

@@ -234,7 +234,7 @@ func (w *typeWriter) typ(typ Type) {
 		if t.targs != nil {
 			// instantiated type
 			w.typeList(t.targs.list())
-		} else if t.TParams().Len() != 0 {
+		} else if !w.hash && t.TParams().Len() != 0 { // For type hashing, don't need to format the TParams
 			// parameterized type
 			w.tParamList(t.TParams().list())
 		}

test/typeparam/issue48185a.dir/p.go

@@ -0,0 +1,19 @@
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package p
+
+type MarshalOptions struct {
+	Marshalers *Marshalers
+}
+
+type Encoder struct {}
+
+type Marshalers = marshalers[MarshalOptions, Encoder]
+
+type marshalers[Options, Coder any] struct{}
+
+func MarshalFuncV1[T any](fn func(T) ([]byte, error)) *Marshalers {
+	return &Marshalers{}
+}

test/typeparam/issue48185a.dir/p_test.go

@@ -0,0 +1,11 @@
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package main
+
+import "p"
+
+func main() {
+	_ = p.MarshalFuncV1[int](func(int) ([]byte, error) { return nil, nil })
+}

test/typeparam/issue48185a.go

@@ -0,0 +1,7 @@
+// rundir
+
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package ignored