Commit graph

9977 commits

Author SHA1 Message Date
Robert Griesemer f30719dc89 encoding/binary: support for varint encoding
R=rsc, r, nigeltao, r, dsymonds
CC=golang-dev
https://golang.org/cl/5146048
2011-09-28 22:36:52 -07:00
Russ Cox b74136984d go: skeleton implementation
R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/5141051
2011-09-28 20:07:21 -04:00
Mike Samuel f17e3d2288 exp/template/html: handle custom attrs and HTML5 embedded elements.
HTML5 allows embedded SVG and MathML.
Code searches show SVG is used for graphing.

This changes transition to deal with constructs like
   <svg xmlns:xlink="http://www.w3.org/1999/xlink">
It changes attr and clients to call a single function that combines
the name lookup and "on" prefix check to determine an attribute
value type given an attribute name.

That function uses heuristics to recognize that
     xlink:href and svg:href
have URL content, and that data-url is likely contains URL content,
since "javascript:" injection is such a problem.

I did a code search over a closure templates codebase to determine
patterns of custom attribute usage.  I did something like

$ find . -name \*.soy | \
    xargs egrep perl -ne 'while (s/\b((data-|\w+:)\w+)\s*=//) { print "$1\n"; }' | \
    sort | uniq

to produce the list at the bottom.

Filtering that by egrep -i 'src|url|uri' produces

data-docConsumptionUri
data-docIconUrl
data-launchUrl
data-lazySrc
data-pageUrl
data-shareurl
data-suggestServerUrl
data-tweetUrl
g:secondaryurls
g:url

which seem to match all the ones that are likely URL content.
There are some short words that match that heuristic, but I still think it decent since
any custom attribute that has a numeric or enumerated keyword value will be unaffected by
the URL assumption.
Counterexamples from /usr/share/dict:
during, hourly, maturity, nourish, purloin, security, surly

Custom attributes present in existing closure templates codebase:
buzz:aid
data-a
data-action
data-actor
data-allowEqualityOps
data-analyticsId
data-bid
data-c
data-cartId
data-categoryId
data-cid
data-command
data-count
data-country
data-creativeId
data-cssToken
data-dest
data-docAttribution
data-docConsumptionUri
data-docCurrencyCode
data-docIconUrl
data-docId
data-docPrice
data-docPriceMicros
data-docTitle
data-docType
data-docid
data-email
data-entityid
data-errorindex
data-f
data-feature
data-fgid
data-filter
data-fireEvent
data-followable
data-followed
data-hashChange
data-height
data-hover
data-href
data-id
data-index
data-invitable
data-isFree
data-isPurchased
data-jid
data-jumpid
data-launchUrl
data-lazySrc
data-listType
data-maxVisiblePages
data-name
data-nid
data-nodeid
data-numItems
data-numPerPage
data-offerType
data-oid
data-opUsesEquality
data-overflowclass
data-packageName
data-pageId
data-pageUrl
data-pos
data-priceBrief
data-profileIds
data-query
data-rating
data-ref
data-rentalGrantPeriodDays
data-rentalactivePeriodHours
data-reviewId
data-role
data-score
data-shareurl
data-showGeLe
data-showLineInclude
data-size
data-sortval
data-suggestServerType
data-suggestServerUrl
data-suggestionIndex
data-tabBarId
data-tabBarIndex
data-tags
data-target
data-textColor
data-theme
data-title
data-toggletarget
data-tooltip
data-trailerId
data-transactionId
data-transition
data-ts
data-tweetContent
data-tweetUrl
data-type
data-useAjax
data-value
data-width
data-x
dm:index
dm:type
g:aspects
g:decorateusingsecondary
g:em
g:entity
g:groups
g:id
g:istoplevel
g:li
g:numresults
g:oid
g:parentId
g:pl
g:pt
g:rating_override
g:secondaryurls
g:sortby
g:startindex
g:target
g:type
g:url
g:value
ga:barsize
ga:css
ga:expandAfterCharsExceed
ga:initialNumRows
ga:nocancelicon
ga:numRowsToExpandTo
ga:type
ga:unlockwhenrated
gw:address
gw:businessname
gw:comment
gw:phone
gw:source
ng:controller
xlink:href
xml:lang
xmlns:atom
xmlns:dc
xmlns:jstd
xmlns:ng
xmlns:og
xmlns:webstore
xmlns:xlink

R=nigeltao
CC=golang-dev
https://golang.org/cl/5119041
2011-09-28 14:07:48 -07:00
Mike Samuel 582bb30466 exp/template/html: don't normalize '<' in doctypes.
The normalization that prevents element name and comment injection in
  <{{.}}
by converting it to
  &lt;{{.}}
breaks
  <!DOCTYPE html>

Instead of splitting states to have a start of document state and a text
state, I whitelist <!DOCTYPE.

R=nigeltao
CC=golang-dev
https://golang.org/cl/5131051
2011-09-28 13:32:56 -07:00
Russ Cox 9aae6482f4 go: documentation for new command
*** This is a design review, not a code review. ***
Feel free to reply to the mail instead of picking out
individual lines to comment on in Rietveld.

This command, go, will replace both gomake/make and goinstall.
Make will stick around only for building our C commands
and perhaps package runtime.

In normal use while developing you'd run commands like

        go compile
        go test
        go clean
        go install

which apply to the package in the current directory.

To operate on code written by others, you add an explicit
package path:

        go get gopath.googlecode.com/hg/oauth
        go test gopath.googlecode.com/hg/oauth

The script.txt file is a script showing the output of
the various help commands that the command has.
(Right now, all the command can do is print help messages.)

R=golang-dev, bradfitz, kevlar, r, edsrzf, gri, adg, rogpeppe, r
CC=golang-dev
https://golang.org/cl/5019045
2011-09-28 16:00:12 -04:00
Brad Fitzpatrick 7b0f3caa26 strings: add Replacer, NewReplacer
This is just a new API to do many replacements at once.

While the point of this API is to be faster than doing replacements one
at a time, the implementation in this CL has the optimizations removed
and may actually be slower.

Future CLs will bring back & add optimizations.

R=r, rsc, rogpeppe
CC=golang-dev
https://golang.org/cl/5081042
2011-09-28 09:34:26 -07:00
Brad Fitzpatrick 58a5f1e84f http: don't send a 400 Bad Request after a client shutdown
Fixes #2312

R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/5143049
2011-09-28 09:27:11 -07:00
Russ Cox 32d1e46058 json: use strings.EqualFold instead of strings.ToLower.
R=golang-dev, bradfitz, iant
CC=golang-dev
https://golang.org/cl/5127043
2011-09-28 12:00:45 -04:00
Russ Cox 8f699a3fb9 regexp: speedups
MatchEasy0_1K        500000        4207 ns/op   243.35 MB/s
MatchEasy0_1K_Old    500000        4625 ns/op   221.40 MB/s
MatchEasy0_1M           500     3948932 ns/op   265.53 MB/s
MatchEasy0_1M_Old       500     3943926 ns/op   265.87 MB/s
MatchEasy0_32K        10000      122974 ns/op   266.46 MB/s
MatchEasy0_32K_Old    10000      123270 ns/op   265.82 MB/s
MatchEasy0_32M           10   127265400 ns/op   263.66 MB/s
MatchEasy0_32M_Old       10   127123500 ns/op   263.95 MB/s
MatchEasy1_1K        500000        5637 ns/op   181.63 MB/s
MatchEasy1_1K_Old     10000      100690 ns/op    10.17 MB/s
MatchEasy1_1M           200     7683150 ns/op   136.48 MB/s
MatchEasy1_1M_Old        10   145774000 ns/op     7.19 MB/s
MatchEasy1_32K        10000      239887 ns/op   136.60 MB/s
MatchEasy1_32K_Old      500     4508182 ns/op     7.27 MB/s
MatchEasy1_32M           10   247103500 ns/op   135.79 MB/s
MatchEasy1_32M_Old        1  4660191000 ns/op     7.20 MB/s
MatchMedium_1K        10000      160567 ns/op     6.38 MB/s
MatchMedium_1K_Old    10000      158367 ns/op     6.47 MB/s
MatchMedium_1M           10   162928000 ns/op     6.44 MB/s
MatchMedium_1M_Old       10   159699200 ns/op     6.57 MB/s
MatchMedium_32K         500     5090758 ns/op     6.44 MB/s
MatchMedium_32K_Old     500     5005800 ns/op     6.55 MB/s
MatchMedium_32M           1  5233973000 ns/op     6.41 MB/s
MatchMedium_32M_Old       1  5109676000 ns/op     6.57 MB/s
MatchHard_1K          10000      249087 ns/op     4.11 MB/s
MatchHard_1K_Old       5000      364569 ns/op     2.81 MB/s
MatchHard_1M              5   256050000 ns/op     4.10 MB/s
MatchHard_1M_Old          5   372446400 ns/op     2.82 MB/s
MatchHard_32K           200     7944525 ns/op     4.12 MB/s
MatchHard_32K_Old       100    11609380 ns/op     2.82 MB/s
MatchHard_32M             1  8144503000 ns/op     4.12 MB/s
MatchHard_32M_Old         1 11885434000 ns/op     2.82 MB/s

R=r, bradfitz
CC=golang-dev
https://golang.org/cl/5134049
2011-09-28 12:00:31 -04:00
Yasuhiro Matsumoto 76ea456e45 hgpatch: do not use hg exit status
Fixes #2243.

R=rsc
CC=golang-dev
https://golang.org/cl/5146041
2011-09-28 12:00:17 -04:00
Brad Fitzpatrick 394842e2a5 net: add shutdown: TCPConn.CloseWrite and CloseRead
R=golang-dev, rsc, iant
CC=golang-dev
https://golang.org/cl/5136052
2011-09-28 08:12:38 -07:00
Mike Samuel 260991ad5f exp/template/html: do not escape the RHS of assignments
In

  {{$x := . | foo}}
  {{$x}}

the first action is a variable assignment that contributes
nothing to the output while the first is a use that needs
to be escaped.

This CL fixes escapeAction to distinguish assignments from
interpolations and to only modify interpolations.

R=nigeltao, r
CC=golang-dev
https://golang.org/cl/5143048
2011-09-27 22:08:14 -07:00
Robert Griesemer 71557713b0 index/suffixarray: revert change from int -> int32
CL 5040041 (https://golang.org/cl/5040041)
changed the use of []int to []int32 internally so
that encoding/binary could be used. This is no
longer needed (gobs can encode ints), and using
[]int is more in sync w/ the semantics of the data
structure (the index elements are indices which are
ints). Changing it back.

R=r
CC=golang-dev
https://golang.org/cl/5141049
2011-09-27 16:21:28 -07:00
Mike Samuel 0287647b13 exp/template/html: add doc comments for undocumented Err... constants.
Does some TODOs and changes the term "div" in an error message
to "division" to avoid confusion with "<div>".

R=nigeltao, r
CC=golang-dev
https://golang.org/cl/5141047
2011-09-27 13:22:01 -07:00
Rob Pike f3050dbbb5 exp/ssh: update tag usage to new convention.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5136047
2011-09-27 10:22:42 -07:00
Rob Pike d4507d44a6 archive/zip: fix Fatal call
Error found by govet.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5142048
2011-09-27 09:33:26 -07:00
Russ Cox 7b1c1811a5 crypto/elliptic: use %x consistently in error print
Fixes #2308.

R=agl, gri
CC=golang-dev
https://golang.org/cl/5121052
2011-09-27 09:40:01 -04:00
Russ Cox a1a3acbd95 test: fix windows build
TBR=bradfitz
CC=golang-dev
https://golang.org/cl/5124049
2011-09-27 09:13:00 -04:00
Ian Lance Taylor 45301ba8c0 runtime: check for nil value pointer in select syncsend case
Fixes #2309.

R=rsc, bradfitz
CC=golang-dev
https://golang.org/cl/5128053
2011-09-26 20:46:37 -07:00
Robert Griesemer 9c643bb3fa exp/norm: fix benchmark bug
- don't use range over string to copy string bytes
- some code simplification

R=mpvl
CC=golang-dev
https://golang.org/cl/5144044
2011-09-26 18:23:21 -07:00
Russ Cox aeaa817140 websocket: remove use of container/vector
R=golang-dev, r
CC=golang-dev, ukai
https://golang.org/cl/5140046
2011-09-26 19:52:34 -04:00
Russ Cox c68ae9d467 bytes: add EqualFold
R=golang-dev, r, r
CC=golang-dev
https://golang.org/cl/5123047
2011-09-26 19:35:32 -04:00
Russ Cox 4bdf1fc02b test: silence/coalesce some tests
Add copyright notice to nilptr.go.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5139048
2011-09-26 19:35:21 -04:00
Rob Pike 4c462e6fd7 gob: protect against invalid message length
Fixes #2301.

R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/5134048
2011-09-26 15:58:01 -07:00
Russ Cox 6c230fbc67 regexp: move to old/regexp, replace with exp/regexp
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5127042
2011-09-26 18:33:13 -04:00
Russ Cox a8a18f6566 strings: add EqualFold
Case-insensitive strcmp without using ToLower.
(Using ToLower is not always correct, and it allocates.)

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5143044
2011-09-26 18:32:51 -04:00
Russ Cox 1fa87ada55 codereview: extra repo sanity check
Also work around Mercurial issue 3023.

If anyone has local changes in their repo (due to
patch queues or whatever) stop them from leaking
into the main repository.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5144043
2011-09-26 18:32:16 -04:00
Russ Cox ba444d8422 strconv: faster Unquote in common case
Also reject literal newline in " and ' quoted strings.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5139045
2011-09-26 13:59:12 -04:00
Russ Cox 67d48daae9 test: merge nilptr/* into one test
The 512 MB array causes load delays on some systems.
Now that we have recover, we can do all the tests in
one binary, so that the delay is incurred just once.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5142044
2011-09-26 13:59:01 -04:00
Russ Cox 0b09a56a64 test: disable sigchld test on Windows
Alex Brainman reports that this is the only test
that keeps us from running test/run.

R=alex.brainman, lucio.dere, bradfitz, hectorchu
CC=golang-dev
https://golang.org/cl/4777043
2011-09-26 13:11:22 -04:00
Russ Cox 92703ff605 unicode: fix make tables
R=r
CC=golang-dev
https://golang.org/cl/5131044
2011-09-26 13:10:16 -04:00
Russ Cox 44d13e3cfe doc: fix memory model read visibility bug
Fixes #2277.

R=dvyukov, r
CC=golang-dev
https://golang.org/cl/5083044
2011-09-26 12:54:30 -04:00
Dave Cheney aa2a31e6c4 exp/ssh: move common code to common.go
R=agl
CC=golang-dev
https://golang.org/cl/5132041
2011-09-26 10:25:13 -04:00
Mike Samuel 10bf744772 exp/template/html: make sure marshalled JSON can be parsed as JS.
This makes sure that all JS newlines are encoded in JSON.

It also moots a TODO about possibly escaping supplemental codepoints.
I served:

Content-Type: text/javascript;charset=UTF-8

var s = "%s";
document.write("<p>", s, "</p><ol>");
for (var i = 0; i < s.length; i++) {
  document.write("<li>", s.charCodeAt(i).toString(16), "</li>");
}
document.write("</l>");

where %s was replaced with bytes "\xf0\x9d\x84\x9e" to test
straight UTF-8 instead of encoding surrogates separately.

Recent Firefox, Chrome, and Safari all decoded it properly.
I have yet to try it on IE or older versions.

R=nigeltao
CC=golang-dev
https://golang.org/cl/5129042
2011-09-26 02:10:43 -07:00
Mike Samuel 3771415100 exp/template/html: fix infinite loop in escapeText on bad input
The template
    "<a="
caused an infinite loop in escape text.

The change to tTag fixes that and the change to escape.go causes
escapeText to panic on any infinite loop that does not involve
a state cycle.

R=nigeltao
CC=golang-dev
https://golang.org/cl/5115041
2011-09-26 00:56:49 -07:00
Mike Samuel 66cdd02038 exp/template/html: error out on ambiguous unquoted attributes
HTML parsers may differ on whether
<input id= onchange=f(             ends in id's or onchange's value,
<a class=`foo                      ends inside a value,
<input style=font:'Arial'          needs open-quote fixup.

Per
http://www.w3.org/TR/html5/tokenization.html#attribute-value-unquoted-state
this treats the error cases in 8.2.4.40 Attribute value (unquoted) state
as fatal errors.

\> U+0022 QUOTATION MARK (")
\> U+0027 APOSTROPHE (')
\> U+003C LESS-THAN SIGN (<)
\> U+003D EQUALS SIGN (=)
\> U+0060 GRAVE ACCENT (`)
        Parse error. Treat it as per the "anything else" entry below.

and emits ErrBadHTML.

R=nigeltao
CC=golang-dev
https://golang.org/cl/5085050
2011-09-26 00:42:38 -07:00
Mike Samuel b3d8e6d7f4 exp/template/html: remove TODO comments that have been done or mooted
R=nigeltao
CC=golang-dev
https://golang.org/cl/5128041
2011-09-26 00:10:21 -07:00
Gustavo Niemeyer ecda69e667 archive/zip: read and write unix file modes
R=golang-dev, rsc, adg
CC=golang-dev
https://golang.org/cl/5124044
2011-09-25 20:48:03 -03:00
David G. Andersen d53afb8d83 rpc: fix typo in documentation client example
The example incorrectly dereferenced an integer variable

R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/5129041
2011-09-25 14:19:08 +10:00
Ian Lance Taylor 8a06936ea1 test: match gccgo error messages
bug340.go:14:7: error: expected type
bug340.go:15:4: error: reference to undefined field or method ‘x’

bug350.go:12:1: error: redefinition of ‘m’
bug350.go:11:1: note: previous definition of ‘m’ was here
bug350.go:15:1: error: redefinition of ‘p’
bug350.go:14:1: note: previous definition of ‘p’ was here

bug351.go:12:6: error: non-name on left side of ‘:=’

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5127041
2011-09-23 21:23:40 -07:00
Jaroslavas Počepko 10b23e7fc9 syscall: mksyscall_windows.pl to produce packages other than syscall (for example pkg/exp/wingui/zwinapi.go)
R=golang-dev, alex.brainman, rsc
CC=golang-dev
https://golang.org/cl/4964074
2011-09-24 10:38:39 +10:00
Brad Fitzpatrick cf6d0175d9 http: add Location method to Response
Fixes #2300

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5117041
2011-09-23 10:57:31 -07:00
Mike Samuel 967d68c00a exp/template/html: tighten rules on dynamic attr names.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5076049
2011-09-23 09:25:10 -07:00
Eric Eisner 481e619c50 suffixarray: add benchmarks for construction
R=gri, jeff
CC=golang-dev
https://golang.org/cl/5040048
2011-09-23 09:18:10 -07:00
Mikio Hara 17410d75d0 syscall: add GetsockoptByte, SetsockoptByte for openbsd
R=golang-dev, fullung, dave, rsc
CC=golang-dev
https://golang.org/cl/5081044
2011-09-23 23:39:19 +09:00
Mikio Hara 4e2f2c7704 syscall: add IPv4 ancillary data for linux
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5039042
2011-09-23 23:38:52 +09:00
Mikio Hara 33a15eb6c0 net: make use of AF_UNSPEC instead of individual address family
R=fullung, rsc
CC=golang-dev
https://golang.org/cl/5034044
2011-09-23 23:37:42 +09:00
Luuk van Dijk 46ed89b7a3 runtime: gdb support: gracefully handle not being able to find types
The Dwarf info has the full typenames, the go *struct runtime.commonType
has the short name.  A more permanent fix would link the two together
but this way the user gets useable stack traces for now.

R=rsc
CC=golang-dev
https://golang.org/cl/5097046
2011-09-23 10:28:02 +02:00
Alex Brainman 7249fa773a syscall: mark stdin, stdout and stderr as non-inheritable by child process
Fixes #2250.

R=golang-dev, hectorchu
CC=golang-dev, vincent.vanackere
https://golang.org/cl/5086050
2011-09-23 18:16:13 +10:00
Fumitoshi Ukai e4790b5fa4 websocket: add mutex to make websocket full-duplex
One benefit of websocket is that it is full-duplex so that it could
send and receive at the same time.
This CL makes websocket goroutine safe, so user could use websocket
both on goroutine for read and on goroutine for write.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5058043
2011-09-22 21:49:24 -04:00