Commit graph

27532 commits

Author SHA1 Message Date
Alexandru Moșoi
7a4211bc1f cmd/compile: constant fold ANDs.
ANDQConst show up occassionally because of right shifting lowering.
ORs and XORs are already folded properly during generic.

Change-Id: I2f9134679555029c641264ce5333d70e167c65f7
Reviewed-on: https://go-review.googlesource.com/21375
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Alexandru Moșoi <alexandru@mosoi.ro>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-31 23:32:11 +00:00
Alexandru Moșoi
ec5083e49b cmd/compile: combine SHLQ into loads and stores
Very common, cuts about 70k from pkg/tools/linux_amd64/* binaries.

Change-Id: Ied0c049e56e56a56810c781435d79027fbcaf274
Reviewed-on: https://go-review.googlesource.com/21374
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Alexandru Moșoi <alexandru@mosoi.ro>
2016-03-31 23:18:24 +00:00
Christopher Nelson
8677cad199 cmd/link: Replace fmt.Sprintf with filepath.Join
In a number of places the code was joining filepaths explicitly with
"/", instead of using filepath.Join. This may cause problems on Windows
(or other) platforms.

This is in support of https://go-review.googlesource.com/#/c/18057

Change-Id: Ieb1334f35ddb2e125be690afcdadff8d7b0ace10
Reviewed-on: https://go-review.googlesource.com/21369
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-31 22:32:28 +00:00
Joe Tsai
c27efce66b compress/flate: make Reader.Read return io.EOF eagerly
Rather than checking the block final bit on the next invocation
of nextBlock, we check it at the termination of the current block.
This ensures that we return (n, io.EOF) instead of (0, io.EOF)
more frequently for most streams.

However, there are certain situations where an eager io.EOF is not done:
1) We previously returned from Read because the write buffer of the internal
dictionary was full, and it just so happens that there is no more data
remaining in the stream.
2) There exists a [non-final, empty, raw block] after all blocks that
actually contain uncompressed data. We cannot return io.EOF eagerly here
since it would break flushing semantics.

Both situations happen infrequently, but it is still important to note that
this change does *not* guarantee that flate will *always* return (n, io.EOF).

Furthermore, this CL makes no changes to the pattern of ReadByte calls
to the underlying io.ByteReader.

Below is the motivation for this change, pulling the text from
@bradfitz's CL/21290:

net/http and other things work better when io.Reader implementations
return (n, io.EOF) at the end, instead of (n, nil) followed by (0,
io.EOF). Both are legal, but the standard library has been moving
towards n+io.EOF.

An investigation of net/http connection re-use in
https://github.com/google/go-github/pull/317 revealed that with gzip
compression + http/1.1 chunking, the net/http package was not
automatically reusing the underlying TCP connections when the final
EOF bytes were already read off the wire. The net/http package only
reuses the connection if the underlying Readers (many of them nested
in this case) all eagerly return io.EOF.

Previous related CLs:
    https://golang.org/cl/76400046 - tls.Reader
    https://golang.org/cl/58240043 - http chunked reader

In addition to net/http, this behavior also helps things like
ioutil.ReadAll (see comments about performance improvements in
https://codereview.appspot.com/49570044)

Updates #14867
Updates google/go-github#317

Change-Id: I637c45552efb561d34b13ed918b73c660f668378
Reviewed-on: https://go-review.googlesource.com/21302
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-31 22:19:12 +00:00
Keith Randall
4a7aba775d cmd/compile: better job of naming compound types
Compound AUTO types weren't named previously.  That was because live
variable analysis (plive.go) doesn't handle spilling to compound types.
It can't handle them because there is no valid place to put VARDEFs when
regalloc is spilling compound types.

compound types = multiword builtin types: complex, string, slice, and
interface.

Instead, we split named AUTOs into individual one-word variables.  For
example, a string s gets split into a byte ptr s.ptr and an integer
s.len.  Those two variables can be spilled to / restored from
independently.  As a result, live variable analysis can handle them
because they are one-word objects.

This CL will change how AUTOs are described in DWARF information.
Consider the code:

func f(s string, i int) int {
    x := s[i:i+5]
    g()
    return lookup(x)
}

The old compiler would spill x to two consecutive slots on the stack,
both named x (at offsets 0 and 8).  The new compiler spills the pointer
of x to a slot named x.ptr.  It doesn't spill x.len at all, as it is a
constant (5) and can be rematerialized for the call to lookup.

So compound objects may not be spilled in their entirety, and even if
they are they won't necessarily be contiguous.  Such is the price of
optimization.

Re-enable live variable analysis tests.  One test remains disabled, it
fails because of #14904.

Change-Id: I8ef2b5ab91e43a0d2136bfc231c05d100ec0b801
Reviewed-on: https://go-review.googlesource.com/21233
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2016-03-31 22:18:26 +00:00
Brad Fitzpatrick
e55896b9f4 net/http, net/http/httputil: rename lk to mu
The conventional name for a sync.Mutex is "mu".

These "lk" names date back to a time before conventions.

Change-Id: Iee57f9f4423d04269e1125b5d82455c453aac26f
Reviewed-on: https://go-review.googlesource.com/21361
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-31 22:18:00 +00:00
Keith Randall
9f66636c93 cmd/compile: don't put SP in index slot
For idx1 ops, SP can appear in the index slot.
Swap SP into the base register slot so we can encode
the instruction.

Fixes #15053

Change-Id: I19000cc9d6c86c7611743481e6e2cb78b1ef04eb
Reviewed-on: https://go-review.googlesource.com/21384
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Alexandru Moșoi <alexandru@mosoi.ro>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-31 22:10:47 +00:00
Keith Randall
47c9e139ae cmd/compile: extend prove pass to handle constant comparisons
Find comparisons to constants and propagate that information
down the dominator tree.  Use it to resolve other constant
comparisons on the same variable.

So if we know x >= 7, then a x > 4 condition must return true.

This change allows us to use "_ = b[7]" hints to eliminate bounds checks.

Fixes #14900

Change-Id: Idbf230bd5b7da43de3ecb48706e21cf01bf812f7
Reviewed-on: https://go-review.googlesource.com/21008
Reviewed-by: Alexandru Moșoi <alexandru@mosoi.ro>
2016-03-31 21:16:23 +00:00
Ilya Tocar
f5bd3556f5 hash/crc64: Add tests for ECMA polynomial
Currently we test crc64 only with ISO polynomial.

Change-Id: Ibc5e202db3b960369cbbb18e31eb0fea07b54dba
Reviewed-on: https://go-review.googlesource.com/21309
Run-TryBot: Ilya Tocar <ilya.tocar@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-31 20:42:02 +00:00
Matthew Dempsky
11d916b157 cmd/compile: remove Node.Nointerface field
We already keep the entire pragma bitset in n.Func.Pragma, so there's
no need to track Nointerface separately.

Passes toolstash -cmp.

Change-Id: Ic027ece477fcf63b0c1df128a08b89ef0f34fd58
Reviewed-on: https://go-review.googlesource.com/21381
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-31 20:31:22 +00:00
Matthew Dempsky
7c4d53c2c8 cmd/compile: stop generating garbage when checking map key types
Change-Id: Ib500ee92ae1a3d15f7c9f3f46d238b75184b4304
Reviewed-on: https://go-review.googlesource.com/21382
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-31 20:30:32 +00:00
Josh Bleecher Snyder
00289c296c cmd/compile: dump stack trace in Fatalf during development
See discussion in #15029.

Change-Id: I5cc8be5737ddb7c1f5e4a6cd92cf557af45e961d
Reviewed-on: https://go-review.googlesource.com/21347
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-31 20:17:09 +00:00
David Crawshaw
43a274fd67 cmd/compile: include pkgPath on all struct types
Fixes #15026.

Change-Id: I61ed71152b99973270d79264d1e8f466f7343c02
Reviewed-on: https://go-review.googlesource.com/21286
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-03-31 20:11:39 +00:00
Josh Bleecher Snyder
e775b8df7a cmd/compile: add sliceBound
Add a constant for the magic -1 for slice bounds.
Use it.
Enforce more aggressively that bounds must be
slice, ddd, or non-negative.
Remove ad hoc check in plive.go.
Check bounds before constructing an array type
when typechecking.

All changes are manual.

Passes toolstash -cmp.

Change-Id: I9fd9cc789d7d4b4eea3b30b24037a254d3788add
Reviewed-on: https://go-review.googlesource.com/21348
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-03-31 18:41:30 +00:00
Keith Randall
4b95575bd4 cmd/compile: fix build
Pushed from an old client by mistake.  These are the
missing changes.

Change-Id: Ia8d61c5c0bde907369366ea9ea98711823342803
Reviewed-on: https://go-review.googlesource.com/21349
Reviewed-by: Todd Neal <todd@tneal.org>
2016-03-31 18:00:36 +00:00
Keith Randall
af517da2f9 cmd/compile: Add more idx1 load/store instructions
Helpful for indexed loads and stores when the stride is not equal to
the size being loaded/stored.

Update #7927

Change-Id: I8714dd4c7b18a96a611bf5647ee21f753d723945
Reviewed-on: https://go-review.googlesource.com/21346
Run-TryBot: Todd Neal <todd@tneal.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>
2016-03-31 17:30:40 +00:00
Keith Randall
b81f2f106f cmd/compile: place combined loads at the location of the last byte load
We need to make sure all the bounds checks pass before issuing
a load which combines several others.  We do this by issuing the
combined load at the last load's block, where "last" = closest to
the leaf of the dominator tree.

Fixes #15002

Change-Id: I7358116db1e039a072c12c0a73d861f3815d72af
Reviewed-on: https://go-review.googlesource.com/21246
Reviewed-by: Todd Neal <todd@tneal.org>
2016-03-31 17:21:33 +00:00
Josh Bleecher Snyder
b83618f964 cmd/compile: encapsulate Type.Nname
Generated by eg, manually fixed up.

I’m not thrilled about having a setter,
but given the variety of contexts in which this
gets fiddled with, it is the cleanest
available alternative.

Change-Id: Ibdf23e638fe0bdabded014c9e59d557fab8c955f
Reviewed-on: https://go-review.googlesource.com/21341
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-03-31 15:38:34 +00:00
Brad Fitzpatrick
139891e815 net/http/httptest: clean up unnecessary goroutine
Finishes cleanup which was too late to do when discovered during the
Go 1.6 cycle.

Fixes #14291

Change-Id: Idc69fadbba10baf246318a22b366709eff088a75
Reviewed-on: https://go-review.googlesource.com/21360
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
2016-03-31 10:48:47 +00:00
Matthew Dempsky
e6066711a0 cmd/compile, runtime: fix pedantic int->string conversions
Previously, cmd/compile rejected constant int->string conversions if
the integer value did not fit into an "int" value. Also, runtime
incorrectly truncated 64-bit values to 32-bit before checking if
they're a valid Unicode code point. According to the Go spec, both of
these cases should instead yield "\uFFFD".

Fixes #15039.

Change-Id: I3c8a3ad9a0780c0a8dc1911386a523800fec9764
Reviewed-on: https://go-review.googlesource.com/21344
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-31 10:28:23 +00:00
Hiroshi Ioka
e7538df701 net/mail: throw error when multiple addresses are given to ParseAddress
Fixes #14610

Change-Id: I3e57dd60b531c1495ea3bc37ef707a1e4e644baa
Reviewed-on: https://go-review.googlesource.com/20180
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-31 10:09:55 +00:00
Brad Fitzpatrick
a62ae9f62f crypto/x509: add SystemCertPool, refactor system cert pool loading
This exports the system cert pool.

The system cert loading was refactored to let it be run multiple times
(so callers get a copy, and can't mutate global state), and also to
not discard errors.

SystemCertPool returns an error on Windows. Maybe it's fixable later,
but so far we haven't used it, since the system verifies TLS.

Fixes #13335

Change-Id: I3dfb4656a373f241bae8529076d24c5f532f113c
Reviewed-on: https://go-review.googlesource.com/21293
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
2016-03-31 07:52:10 +00:00
Matthew Dempsky
71ab3c1ccf cmd/link: remove -H elf flag
We create appropriate ELF files automatically based on GOOS. There's
no point in supporting -H elf flag, particularly since we need to emit
different flavors of ELF depending on GOOS anyway.

If that weren't reason enough, -H elf appears to be broken since at
least Go 1.4. At least I wasn't able to find a way to make use of it.

As best I can tell digging through commit history, -H elf is just an
artifact leftover from Plan 9's 6l linker.

Change-Id: I7393caaadbc60107bbd6bc99b976a4f4fe6b5451
Reviewed-on: https://go-review.googlesource.com/21343
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-31 06:33:01 +00:00
Brad Fitzpatrick
758447cdc9 test/fixedbugs: add test for divide by zero being optimized away
This only tests amd64 because it's currently broken on non-SSA
backends.

Fixes #8613

Change-Id: I6bc501c81c395e533bb9c7335789750e0c6b7a8f
Reviewed-on: https://go-review.googlesource.com/21325
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-31 06:12:38 +00:00
Brad Fitzpatrick
a6557a05a0 net/http: allow Handlers to handle http2 upgrade PRI requests
The http2 spec defines a magic string which initates an http2 session:

    "PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n"

It was intentionally chosen to kinda look like an HTTP request, but
just different enough to break things not ready for it. This change
makes Go ready for it.

Notably: Go now accepts the request header (the prefix "PRI *
HTTP/2.0\r\n\r\n") as a valid request, even though it doesn't have a
Host header. But we now mark it as "Connection: close" and teach the
Server to never read a second request from the connection once that's
seen. If the http.Handler wants to deal with the upgrade, it has to
hijack the request, read out the "body", compare it against
"SM\r\n\r\n", and then speak http2. One of the new tests demonstrates
that hijacking.

Fixes #14451
Updates #14141 (h2c)

Change-Id: Ib46142f31c55be7d00c56fa2624ec8a232e00c43
Reviewed-on: https://go-review.googlesource.com/21327
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-31 06:12:36 +00:00
Brad Fitzpatrick
0026cb788b net/http: validate transmitted header fields
This makes sure the net/http package never attempts to transmit a
bogus header field key or value and instead fails fast with an error
to the user, rather than relying on the server to maybe return an
error.

It's still possible to use x/net/http2.Transport directly to send
bogus stuff. This change only stops h1 & h2 usage via the net/http
package. A future change will update x/net/http2.

This change also moves some code from request.go to lex.go, which in a
separate future change should be moved so it can be shared with http2
to reduce code bloat.

Updates #14048

Change-Id: I0a44ae1ab357fbfcbe037aa4b5d50669a87f2856
Reviewed-on: https://go-review.googlesource.com/21326
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-31 04:55:58 +00:00
Ian Lance Taylor
b4117995e3 cmd/pprof: use DWARF info to lookup unknown PC addresses
Test to follow in a separate CL that arranges for the runtime package to
store non-Go addresses in a CPU profile.

Change-Id: I33ce1d66b77340b1e62b54505fc9b1abcec108a9
Reviewed-on: https://go-review.googlesource.com/21055
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
2016-03-31 04:05:06 +00:00
Keith Randall
4b209dbf0b runtime: don't use REP;MOVSB if CPUID doesn't say it is fast
Only use REP;MOVSB if:
 1) The CPUID flag says it is fast, and
 2) The pointers are unaligned
Otherwise, use REP;MOVSQ.

Update #14630

Change-Id: I946b28b87880c08e5eed1ce2945016466c89db66
Reviewed-on: https://go-review.googlesource.com/21300
Reviewed-by: Nigel Tao <nigeltao@golang.org>
2016-03-31 02:54:10 +00:00
Dave Cheney
1a9373bc57 cmd/compile/internal/gc: avoid append when building Type fields
As a followup to CL 21296, avoid append operations when constructing the
fields of a Type if the length is known beforehand

This also includes some small scoping driveby cleanups, and a change to
tointerface0 to avoid iterating over the field list twice.

compilebench shows a very small reduction in allocations.

 name      old time/op    new time/op    delta
Template     364ms ± 5%     363ms ± 4%    ~     (p=0.945 n=20+19)
Unicode      182ms ±11%     185ms ±12%    ~     (p=0.445 n=20+20)
GoTypes      1.14s ± 2%     1.14s ± 3%    ~     (p=0.221 n=20+20)
Compiler     5.85s ± 2%     5.84s ± 2%    ~     (p=0.369 n=20+20)

name      old alloc/op   new alloc/op   delta
Template    56.7MB ± 0%    56.7MB ± 0%  -0.04%  (p=0.000 n=20+20)
Unicode     38.3MB ± 0%    38.3MB ± 0%    ~     (p=0.728 n=20+19)
GoTypes      180MB ± 0%     180MB ± 0%  -0.02%  (p=0.000 n=20+20)
Compiler     812MB ± 0%     812MB ± 0%  -0.02%  (p=0.000 n=19+20)

name      old allocs/op  new allocs/op  delta
Template      482k ± 0%      480k ± 0%  -0.34%  (p=0.000 n=20+20)
Unicode       377k ± 0%      377k ± 0%  -0.04%  (p=0.010 n=20+20)
GoTypes      1.36M ± 0%     1.35M ± 0%  -0.24%  (p=0.000 n=20+20)
Compiler     5.47M ± 0%     5.46M ± 0%  -0.11%  (p=0.000 n=20+18)

Change-Id: Ibb4c40229fa3816acd8de98ba41d1571a2aabacf
Reviewed-on: https://go-review.googlesource.com/21352
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Dave Cheney <dave@cheney.net>
2016-03-31 02:22:18 +00:00
Dave Cheney
ea5091fcd7 cmd/compile/internal/gc: don't let the argument to Fields.Set escape
Apply Robert's optimisation from CL 21241 to Type.Fields. The results
are less impressive, possibly because of the makeup of the test data.

name      old time/op    new time/op    delta
Template     365ms ± 5%     365ms ± 3%    ~     (p=0.888 n=20+16)
Unicode      182ms ±10%     180ms ± 9%    ~     (p=0.883 n=20+20)
GoTypes      1.14s ± 2%     1.13s ± 3%    ~     (p=0.096 n=20+20)
Compiler     5.74s ± 1%     5.76s ± 2%    ~     (p=0.369 n=20+20)

name      old alloc/op   new alloc/op   delta
Template    56.8MB ± 0%    56.7MB ± 0%  -0.15%  (p=0.000 n=19+20)
Unicode     38.3MB ± 0%    38.3MB ± 0%  -0.02%  (p=0.006 n=20+19)
GoTypes      180MB ± 0%     180MB ± 0%  -0.13%  (p=0.000 n=20+20)
Compiler     805MB ± 0%     804MB ± 0%  -0.05%  (p=0.000 n=20+20)

name      old allocs/op  new allocs/op  delta
Template      485k ± 0%      482k ± 0%  -0.54%  (p=0.000 n=19+20)
Unicode       377k ± 0%      377k ± 0%  -0.05%  (p=0.005 n=20+20)
GoTypes      1.37M ± 0%     1.36M ± 0%  -0.53%  (p=0.000 n=20+19)
Compiler     5.42M ± 0%     5.41M ± 0%  -0.21%  (p=0.000 n=20+20)

Change-Id: I6782659fadd605ce9931bf5c737c7058b96a29eb
Reviewed-on: https://go-review.googlesource.com/21296
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-31 00:40:11 +00:00
Nigel Tao
225b223e47 image/jpeg: reconstruct progressive images even if incomplete.
Fixes #14522.

As I said on that issue:

----
This is a progressive JPEG image. There are two dimensions of
progressivity: spectral selection (variables zs and ze in scan.go,
ranging in [0, 63]) and successive approximation (variables ah and al in
scan.go, ranging in [0, 8), from LSB to MSB, although ah=0 implicitly
means ah=8).

For this particular image, there are three components, and the SOS
markers contain this progression:

zs, ze, ah, al:  0  0 0 0	components: 0, 1, 2
zs, ze, ah, al:  1 63 0 0	components: 1
zs, ze, ah, al:  1 63 0 0	components: 2
zs, ze, ah, al:  1 63 0 2	components: 0
zs, ze, ah, al:  1 10 2 1	components: 0
zs, ze, ah, al: 11 63 2 1	components: 0
zs, ze, ah, al:  1 10 1 0	components: 0

The combination of all of these is complete (i.e. spectra 0 to 63 and
bits 8 exclusive to 0) for components 1 and 2, but it is incomplete for
component 0 (the luma component). In particular, there is no data for
component 0, spectra 11 to 63 and bits 1 exclusive to 0.

The image/jpeg code, as of Go 1.6, waits until both dimensions are
complete before performing the de-quantization, IDCT and copy to an
*image.YCbCr. This is the "if zigEnd != blockSize-1 || al != 0 { ...
continue }" code and associated commentary in scan.go.

Almost all progressive JPEG images end up complete in both dimensions
for all components, but this particular image is incomplete for
component 0, so the Go code never writes anything to the Y values of the
resultant *image.YCbCr, which is why the broken output is so dark (but
still looks recognizable in terms of red and blue hues).

My reading of the ITU T.81 JPEG specification (Annex G) doesn't
explicitly say that this is a valid image, but it also doesn't rule it
out.

In any case, the fix is, for progressive JPEG images, to always
reconstruct the decoded blocks (by performing the de-quantization, IDCT
and copy to an *image.YCbCr), regardless of whether or not they end up
complete. Note that, in Go, the jpeg.Decode function does not return
until the entire image is decoded, so we still only want to reconstruct
each block once, not once per SOS (Start Of Scan) marker.
----

A test image was also added, based on video-001.progressive.jpeg. When
decoding that image, inserting a

println("nComp, zs, ze, ah, al:", nComp, zigStart, zigEnd, ah, al)

into decoder.processSOS in scan.go prints:

nComp, zs, ze, ah, al: 3 0 0 0 1
nComp, zs, ze, ah, al: 1 1 5 0 2
nComp, zs, ze, ah, al: 1 1 63 0 1
nComp, zs, ze, ah, al: 1 1 63 0 1
nComp, zs, ze, ah, al: 1 6 63 0 2
nComp, zs, ze, ah, al: 1 1 63 2 1
nComp, zs, ze, ah, al: 3 0 0 1 0
nComp, zs, ze, ah, al: 1 1 63 1 0
nComp, zs, ze, ah, al: 1 1 63 1 0
nComp, zs, ze, ah, al: 1 1 63 1 0

In other words, video-001.progressive.jpeg contains 10 different scans.
This little program below drops half of them (remembering to keep the
"\xff\xd9" End of Image marker):

----
package main

import (
	"bytes"
	"io/ioutil"
	"log"
)

func main() {
	sos := []byte{0xff, 0xda}
	eoi := []byte{0xff, 0xd9}

	src, err := ioutil.ReadFile("video-001.progressive.jpeg")
	if err != nil {
		log.Fatal(err)
	}
	b := bytes.Split(src, sos)
	println(len(b)) // Prints 11.
	dst := bytes.Join(b[:5], sos)
	dst = append(dst, eoi...)
	if err := ioutil.WriteFile("video-001.progressive.truncated.jpeg", dst, 0666); err != nil {
		log.Fatal(err)
	}
}
----

The video-001.progressive.truncated.jpeg was converted to png via
libjpeg and ImageMagick:

djpeg -nosmooth video-001.progressive.truncated.jpeg > tmp.tga
convert tmp.tga video-001.progressive.truncated.png
rm tmp.tga

Change-Id: I72b20cd4fb6746d36d8d4d587f891fb3bc641f84
Reviewed-on: https://go-review.googlesource.com/21062
Reviewed-by: Rob Pike <r@golang.org>
2016-03-31 00:33:24 +00:00
Dave Cheney
0373128318 cmd/compile/internal/gc: don't iterate over field list twice
In tostruct0 and tofunargs we take a list of nodes, transform them into
a slice of Fields, set the fields on a type, then use the IterFields
iterator to iterate over the list again to see if any of them are
broken.

As we know the slice of fielde-we just created it-we can combine these two
interations into one pass over the fields.

Change-Id: I8b04c90fb32fd6c3b1752cfc607128a634ee06c5
Reviewed-on: https://go-review.googlesource.com/21350
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-31 00:21:37 +00:00
Matthew Dempsky
e76fc1b921 cmd/compile: use t.IsFoo() instead of Isfoo[t.Etype]
This allows us to get rid of Isptr and Issigned. Still some code to
clean up for Isint, Isfloat, and Iscomplex.

CL produced mechanically using gofmt -w -r.

Passes toolstash -cmp.

Change-Id: If4f807bb7f2b357288d2547be2380eb511875786
Reviewed-on: https://go-review.googlesource.com/21339
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2016-03-30 22:48:34 +00:00
Matthew Dempsky
3efefd9395 cmd/compile: use t.IsFoo() instead of t.Etype == TFOO
CL produced mechanically using gofmt -w -r.

Passes toolstash -cmp.

Change-Id: Ib2e8710ebd844e2149125b41c335b71a02fcab53
Reviewed-on: https://go-review.googlesource.com/21338
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-30 22:31:07 +00:00
Alexandru Moșoi
d8f1f8d856 cmd/compile: generalize strength reduction of mulq
* This is an improved version of an earlier patch.
* Verified with gcc up to 100.
* Limited to two instructions based on costs from
https://gmplib.org/~tege/x86-timing.pdf

Change-Id: Ib7c37de6fd8e0ba554459b15c7409508cbcf6728
Reviewed-on: https://go-review.googlesource.com/21103
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Alexandru Moșoi <alexandru@mosoi.ro>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-30 22:27:13 +00:00
Matthew Dempsky
1624a9c9e7 cmd/compile: get rid of redundant Type helper functions
Replace Isfixedarray, Isslice, and Isinter with the IsArray, IsSlice,
and IsInterface methods added for SSA. Rewrite performed mechanically
using gofmt -w -r "Isfoo(t) -> t.IsFoo()".

Because the IsFoo methods panic when given a nil pointer, a handful of
call sites had to be modified to check for nil Type values. These
aren't strictly necessary, because nil Type values should only occur
in invalid Go source programs, so it would be okay if we panicked on
them and gave up type checking the rest of the package. However, there
are a couple regress tests that expect we continue, so add checks to
keep those tests passing. (See #15029.)

Passes toolstash -cmp.

Change-Id: I511c6ac4cfdf3f9cbdb3e52a5fa91b6d09d82f80
Reviewed-on: https://go-review.googlesource.com/21336
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-30 21:58:18 +00:00
Josh Bleecher Snyder
2592e0999e cmd/compile: s/typeX/typX/
Apparently I’m having a hard time following my
own naming scheme.

Change-Id: I99c801bef09fa65c1f0e8ecc2fba154a495e9c17
Reviewed-on: https://go-review.googlesource.com/21332
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
2016-03-30 21:22:09 +00:00
Josh Bleecher Snyder
8640b51df8 cmd/compile: add Type.Elem
This removes almost all direct access to
Type’s heavily overloaded Type field.

Mostly generated by eg, manually checked.

Significant manual changes:

* reflect.go's typPkg used Type indiscriminately.
  Use it only for specific etypes.
* gen.go's visitComponents contained a usage of Type
  with structs. Using Type for structs no longer
  occurs, and the Fatal contained therein has not triggered,
  so it has been axed.
* Scary code in cgen.go's cgen_slice is now explicitly scary.

Passes toolstash -cmp.

Change-Id: I2dbfb3c959da7ae239f964d83898c204affcabc6
Reviewed-on: https://go-review.googlesource.com/21331
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-30 21:21:55 +00:00
Josh Bleecher Snyder
76e72691a0 cmd/compile: add typMap
Also, add two uses of Key and Val that I missed earlier.
As before, direct writes to Down and Type remain in bimport.

Change-Id: I487aa975926b30092db1ad74ace17994697117c1
Reviewed-on: https://go-review.googlesource.com/21330
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-03-30 21:05:29 +00:00
Alexandru Moșoi
dc5a7682f0 cmd/compile: use inc/dec for bytes, too
Change-Id: Ib2890ab1983cbef7c1c1ee5a10204ba3ace19b53
Reviewed-on: https://go-review.googlesource.com/21312
Run-TryBot: Alexandru Moșoi <alexandru@mosoi.ro>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2016-03-30 20:18:16 +00:00
Brad Fitzpatrick
ca72f5f5df internal/testenv: prefer to find go binary in GOROOT
Partial revert of https://golang.org/cl/20967 which
I can't reproduce and actually breaks me more.

Fixes #14901

Change-Id: I8cce443fbd95f5f6f2a5b6a4b9f2faab36167a12
Reviewed-on: https://go-review.googlesource.com/21292
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2016-03-30 19:12:50 +00:00
Matthew Dempsky
788f11263a cmd/compile: rename Type.IsPtr to Type.IsPtrShaped
Previously, t.IsPtr() reported whether t was represented with a
pointer, but some of its callers expected it to report whether t is an
actual Go pointer. Resolve this by renaming t.IsPtr to t.IsPtrShaped
and adding a new t.IsPtr method to report Go pointer types.

Updated a couple callers in gc/ssa.go to use IsPtr instead of
IsPtrShaped.

Passes toolstash -cmp.

Updates #15028.

Change-Id: I0a8154b5822ad8a6ad296419126ad01a3d2a5dc5
Reviewed-on: https://go-review.googlesource.com/21232
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2016-03-30 19:11:16 +00:00
Josh Bleecher Snyder
fdf6761e01 cmd/compile: add typPtr
Passes toolstash -cmp.

Change-Id: I721348ed2122b6a9cd87ad2041b6ee3bf6b2bbb5
Reviewed-on: https://go-review.googlesource.com/21306
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-30 17:19:59 +00:00
Josh Bleecher Snyder
e3c7497327 cmd/compile: add typWrapper and Type.Wrapped
Passes toolstash -cmp.

Change-Id: I7dffd9bc5bab323590df6fb591bf1e73edf2e465
Reviewed-on: https://go-review.googlesource.com/21305
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-30 17:19:50 +00:00
Josh Bleecher Snyder
09c672d50a cmd/compile: add typChan
Passes toolstash -cmp.

Change-Id: I2c71882f957c44047c7ac83c78236dcc3dfa15a1
Reviewed-on: https://go-review.googlesource.com/21304
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-03-30 17:19:42 +00:00
Josh Bleecher Snyder
331f962508 cmd/compile: use IsSlice and IsArray instead of checking Bound
Changes generated by eg and manually checked.

Isfixedarray, Isslice, and many other
Type-related functions in subr.go should
either be deleted or moved to type.go.
Later, though; the game now is cleanup via encapsulation.

Passes toolstash -cmp.

Change-Id: I83dd8816f6263b74367d23c2719a08c362e330f9
Reviewed-on: https://go-review.googlesource.com/21303
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-30 17:18:29 +00:00
Alexandru Moșoi
d7f7ea6ea8 cmd/compile: ignore dead phis in fuse
Happens occasionally for boolean phis was used as a control.

Change-Id: Ie0f2483e9004c1706751d8dfb25ee2e5106d917e
Reviewed-on: https://go-review.googlesource.com/21310
Run-TryBot: Alexandru Moșoi <alexandru@mosoi.ro>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2016-03-30 15:37:18 +00:00
Joe Tsai
152a08c531 compress/gzip: fix error handling in Read
The Read logic should not assume that only (0, io.EOF) is returned
instead of (n, io.EOF) where n is positive.

The fix done here is very similar to the fix to compress/zlib
in CL/20292.

Change-Id: Icb76258cdcf8cfa386a60bab330fefde46fc071d
Reviewed-on: https://go-review.googlesource.com/21308
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-30 11:37:20 +00:00
kortschak
6b97dbf848 cmd/dist: make fortran test conditional on bash existence
Fixes #14929.

Change-Id: I0391acf9f5f65389f73637533306a7c4240320b8
Reviewed-on: https://go-review.googlesource.com/21295
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-30 11:35:40 +00:00
Joe Tsai
897dcdb5ec debug/elf: deflake file_test.go
It is valid for io.Reader to return (n, io.EOF) where n is positive.
The unit test should not fail if io.EOF is returned when read until
the end.

Change-Id: I7b918e3cc03db8b90c8aa58f4c0f7806a1d4af7e
Reviewed-on: https://go-review.googlesource.com/21307
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-30 06:56:25 +00:00