languages/dart-sdk
languages/dart-sdk
1
0
Fork 0
mirror of https://github.com/dart-lang/sdk synced 2024-10-14 15:09:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
dart-sdk/runtime/vm/token_position.cc

64 lines
2 KiB
C++
Raw Normal View History

Replace intptr_t with TokenDescriptor - Use TokenDescriptor instead of intptr_t for all token positions. - Use TokenDescriptor in raw_object instead of int32_t. - TokenDescriptor is a POD with an int32_t (this shrinks the size of AST and IR nodes by 32-bits on 64-bit architectures). There are some cleanups I plan on doing as a follow up CL: - Replace TokenDescriptor::value() with TokenDescriptor::TokenPos() R=iposva@google.com, rmacnak@google.com Review URL: https://codereview.chromium.org/1644793002 .
2016-02-02 18:15:44 +00:00
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
#include "vm/token_position.h"
#include "vm/object.h"
[vm] Remove non-private uses of TokenPosition::value(). Instead, split each old use into the following cases: * If the TokenPosition value is expected to be a real token position, then use TokenPosition::Pos(). * If the TokenPosition is being serialized in some way, then use TokenPosition::Serialize() and change the place where the TokenPosition is recreated to use TokenPosition::Deserialize(). * If the value of the TokenPosition is being printed for debugging purposes, then just use TokenPosition::ToCString() instead. That is, we try to pin down when token positions are expected to be real vs. when other types of token positions can be found. Another source of possible error when using token positions is to convert between synthetic and real token positions. In the past, synthetic token positions may have been based off real token positions, but that is no longer the case. Thus, all methods that allow that conversion have been removed, and instead there is a new static method for constructing synthetic tokens from valid nonces. This CL also makes it so that Pos() and relational operators on token positions are only defined on real token positions, to avoid any assumptions about what the value encoded in synthetic positions mean. To help with cases where non-real token positions may occur, four helper methods are added: * TokenPosition::Min(a, b): A static method that returns the smallest real token position provided. If neither `a` or `b` are real, returns `a`. * TokenPosition::Max(a, b): A static method that returns the largest real token position provided. If neither `a` or `b` are real, returns `a`. * TokenPosition::IsWithin(start, end): Determines whether `this` falls between `start` and `end` (inclusive). If `this` is non-real, then it must be either `start` or `end` if synthetic, otherwise false. Otherwise, we mimic the old style of range checking, which means that non-real starts and ends are treated as less than every real token. * TokenPosition::CompareForSorting(other): Unlike the relational operators, provides a comparison between any types of token positions for purposes such as sorting. Currently only used in the profiler. It also changes TokenPosition::ToCString() to tag synthetic token positions, so they can be distinguished from real ones at a glance. TEST=Existing test suite on trybots, especially the observatory tests which make heavy use of the debugger and the unit tests for the profiler/source report modules. Bug: https://github.com/dart-lang/sdk/issues/44436 Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try Change-Id: Ic06aa0bc7a1f0fbac7257ed22ca5e7e0ccd7f3f2 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174924 Commit-Queue: Tess Strickland <sstrickl@google.com> Reviewed-by: Martin Kustermann <kustermann@google.com> Reviewed-by: Daco Harkes <dacoharkes@google.com>
2020-12-16 08:27:32 +00:00
#include "vm/zone_text_buffer.h"
Replace intptr_t with TokenDescriptor - Use TokenDescriptor instead of intptr_t for all token positions. - Use TokenDescriptor in raw_object instead of int32_t. - TokenDescriptor is a POD with an int32_t (this shrinks the size of AST and IR nodes by 32-bits on 64-bit architectures). There are some cleanups I plan on doing as a follow up CL: - Replace TokenDescriptor::value() with TokenDescriptor::TokenPos() R=iposva@google.com, rmacnak@google.com Review URL: https://codereview.chromium.org/1644793002 .
2016-02-02 18:15:44 +00:00
namespace dart {
[vm] Standardize hashing instance methods in C++. In the codebase, we have several different interfaces for instance methods that return a hash: - uword Hash() const; - intptr_t Hash() const; - uint32_t Hash() const; - intptr_t Hashcode() const; This CL standardizes on `uword Hash() const` and adjusts any related functions to match. TEST=Existing test suites, as this is an internal refactoring. Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try Change-Id: If2cbce57f3fae0f0d24031b6e324f0323c965f41 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195067 Commit-Queue: Tess Strickland <sstrickl@google.com> Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-04-14 10:04:48 +00:00
uword TokenPosition::Hash() const {
return Utils::WordHash(value_);
Replace intptr_t with TokenDescriptor - Use TokenDescriptor instead of intptr_t for all token positions. - Use TokenDescriptor in raw_object instead of int32_t. - TokenDescriptor is a POD with an int32_t (this shrinks the size of AST and IR nodes by 32-bits on 64-bit architectures). There are some cleanups I plan on doing as a follow up CL: - Replace TokenDescriptor::value() with TokenDescriptor::TokenPos() R=iposva@google.com, rmacnak@google.com Review URL: https://codereview.chromium.org/1644793002 .
2016-02-02 18:15:44 +00:00
}
[vm] Remove non-private uses of TokenPosition::value(). Instead, split each old use into the following cases: * If the TokenPosition value is expected to be a real token position, then use TokenPosition::Pos(). * If the TokenPosition is being serialized in some way, then use TokenPosition::Serialize() and change the place where the TokenPosition is recreated to use TokenPosition::Deserialize(). * If the value of the TokenPosition is being printed for debugging purposes, then just use TokenPosition::ToCString() instead. That is, we try to pin down when token positions are expected to be real vs. when other types of token positions can be found. Another source of possible error when using token positions is to convert between synthetic and real token positions. In the past, synthetic token positions may have been based off real token positions, but that is no longer the case. Thus, all methods that allow that conversion have been removed, and instead there is a new static method for constructing synthetic tokens from valid nonces. This CL also makes it so that Pos() and relational operators on token positions are only defined on real token positions, to avoid any assumptions about what the value encoded in synthetic positions mean. To help with cases where non-real token positions may occur, four helper methods are added: * TokenPosition::Min(a, b): A static method that returns the smallest real token position provided. If neither `a` or `b` are real, returns `a`. * TokenPosition::Max(a, b): A static method that returns the largest real token position provided. If neither `a` or `b` are real, returns `a`. * TokenPosition::IsWithin(start, end): Determines whether `this` falls between `start` and `end` (inclusive). If `this` is non-real, then it must be either `start` or `end` if synthetic, otherwise false. Otherwise, we mimic the old style of range checking, which means that non-real starts and ends are treated as less than every real token. * TokenPosition::CompareForSorting(other): Unlike the relational operators, provides a comparison between any types of token positions for purposes such as sorting. Currently only used in the profiler. It also changes TokenPosition::ToCString() to tag synthetic token positions, so they can be distinguished from real ones at a glance. TEST=Existing test suite on trybots, especially the observatory tests which make heavy use of the debugger and the unit tests for the profiler/source report modules. Bug: https://github.com/dart-lang/sdk/issues/44436 Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try Change-Id: Ic06aa0bc7a1f0fbac7257ed22ca5e7e0ccd7f3f2 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174924 Commit-Queue: Tess Strickland <sstrickl@google.com> Reviewed-by: Martin Kustermann <kustermann@google.com> Reviewed-by: Daco Harkes <dacoharkes@google.com>
2020-12-16 08:27:32 +00:00
TokenPosition TokenPosition::Deserialize(int32_t value) {
return TokenPosition(value);
Replace intptr_t with TokenDescriptor - Use TokenDescriptor instead of intptr_t for all token positions. - Use TokenDescriptor in raw_object instead of int32_t. - TokenDescriptor is a POD with an int32_t (this shrinks the size of AST and IR nodes by 32-bits on 64-bit architectures). There are some cleanups I plan on doing as a follow up CL: - Replace TokenDescriptor::value() with TokenDescriptor::TokenPos() R=iposva@google.com, rmacnak@google.com Review URL: https://codereview.chromium.org/1644793002 .
2016-02-02 18:15:44 +00:00
}
[vm] Remove non-private uses of TokenPosition::value(). Instead, split each old use into the following cases: * If the TokenPosition value is expected to be a real token position, then use TokenPosition::Pos(). * If the TokenPosition is being serialized in some way, then use TokenPosition::Serialize() and change the place where the TokenPosition is recreated to use TokenPosition::Deserialize(). * If the value of the TokenPosition is being printed for debugging purposes, then just use TokenPosition::ToCString() instead. That is, we try to pin down when token positions are expected to be real vs. when other types of token positions can be found. Another source of possible error when using token positions is to convert between synthetic and real token positions. In the past, synthetic token positions may have been based off real token positions, but that is no longer the case. Thus, all methods that allow that conversion have been removed, and instead there is a new static method for constructing synthetic tokens from valid nonces. This CL also makes it so that Pos() and relational operators on token positions are only defined on real token positions, to avoid any assumptions about what the value encoded in synthetic positions mean. To help with cases where non-real token positions may occur, four helper methods are added: * TokenPosition::Min(a, b): A static method that returns the smallest real token position provided. If neither `a` or `b` are real, returns `a`. * TokenPosition::Max(a, b): A static method that returns the largest real token position provided. If neither `a` or `b` are real, returns `a`. * TokenPosition::IsWithin(start, end): Determines whether `this` falls between `start` and `end` (inclusive). If `this` is non-real, then it must be either `start` or `end` if synthetic, otherwise false. Otherwise, we mimic the old style of range checking, which means that non-real starts and ends are treated as less than every real token. * TokenPosition::CompareForSorting(other): Unlike the relational operators, provides a comparison between any types of token positions for purposes such as sorting. Currently only used in the profiler. It also changes TokenPosition::ToCString() to tag synthetic token positions, so they can be distinguished from real ones at a glance. TEST=Existing test suite on trybots, especially the observatory tests which make heavy use of the debugger and the unit tests for the profiler/source report modules. Bug: https://github.com/dart-lang/sdk/issues/44436 Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try Change-Id: Ic06aa0bc7a1f0fbac7257ed22ca5e7e0ccd7f3f2 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174924 Commit-Queue: Tess Strickland <sstrickl@google.com> Reviewed-by: Martin Kustermann <kustermann@google.com> Reviewed-by: Daco Harkes <dacoharkes@google.com>
2020-12-16 08:27:32 +00:00
int32_t TokenPosition::Serialize() const {
return static_cast<int32_t>(value_);
Replace intptr_t with TokenDescriptor - Use TokenDescriptor instead of intptr_t for all token positions. - Use TokenDescriptor in raw_object instead of int32_t. - TokenDescriptor is a POD with an int32_t (this shrinks the size of AST and IR nodes by 32-bits on 64-bit architectures). There are some cleanups I plan on doing as a follow up CL: - Replace TokenDescriptor::value() with TokenDescriptor::TokenPos() R=iposva@google.com, rmacnak@google.com Review URL: https://codereview.chromium.org/1644793002 .
2016-02-02 18:15:44 +00:00
}
[vm] Change the encoding of token positions in the coverage array The source report RPC needs to be able to distinguish branch coverage vs normal coverage token positions. So encode normal positions as 2 * pos and branch positions as 2 * pos + 1. TEST=CI Change-Id: I247796d1eb4c97947f9b38aa56bc318ecea09425 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/224320 Reviewed-by: Ryan Macnak <rmacnak@google.com> Commit-Queue: Liam Appelbe <liama@google.com>
2021-12-15 23:12:19 +00:00
intptr_t TokenPosition::EncodeCoveragePosition(bool is_branch_coverage) {
// Normal coverage positions are encoded as 2 * pos, and branch coverage are
// encoded as 2 * pos + 1.
intptr_t encoded_position = 2 * static_cast<intptr_t>(value_);
return is_branch_coverage ? encoded_position + 1 : encoded_position;
}
TokenPosition TokenPosition::DecodeCoveragePosition(intptr_t encoded_position,
bool* is_branch_coverage) {
*is_branch_coverage = ((encoded_position % 2) == 1);
return TokenPosition(encoded_position / 2);
}
Replace intptr_t with TokenDescriptor - Use TokenDescriptor instead of intptr_t for all token positions. - Use TokenDescriptor in raw_object instead of int32_t. - TokenDescriptor is a POD with an int32_t (this shrinks the size of AST and IR nodes by 32-bits on 64-bit architectures). There are some cleanups I plan on doing as a follow up CL: - Replace TokenDescriptor::value() with TokenDescriptor::TokenPos() R=iposva@google.com, rmacnak@google.com Review URL: https://codereview.chromium.org/1644793002 .
2016-02-02 18:15:44 +00:00
#define DEFINE_VALUES(name, value) \
[vm] Remove the only cases where TokenPositions are mutated. The Next() method on TokenPositions used to increment the internal value_ field, but now creates a new TokenPosition with value_ + 1. There are only three places this is used: * Getting the token position after the end token position for implicit closure functions in the scope builder. * Getting the token position after the end token position for variable declarations in the scope builder. * Iterating over the token positions on a single source line in the debugger. So the overhead of creating new TokenPositions vs. mutating the original should be limited, compared to the benefit of being able to assume TokenPosition values are constant once created. There's too much dependence on the copy and copy assignment operators due to the previous possibility of mutation to make the internal field constant at the moment without much more sweeping changes, unfortunately. Also remove the copy assignment in initializing TokenPosition constants. TEST=Tested using existing tests on trybots. Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try Change-Id: Ie18b7f45bd9c6a43647ecd189330d91a4dff373b Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174922 Commit-Queue: Tess Strickland <sstrickl@google.com> Reviewed-by: Clement Skau <cskau@google.com>
2020-12-03 12:40:28 +00:00
const TokenPosition TokenPosition::k##name(value);
clang-format runtime/vm R=johnmccutchan@google.com Review URL: https://codereview.chromium.org/2481873005 .
2016-11-08 21:54:47 +00:00
SENTINEL_TOKEN_DESCRIPTORS(DEFINE_VALUES);
Replace intptr_t with TokenDescriptor - Use TokenDescriptor instead of intptr_t for all token positions. - Use TokenDescriptor in raw_object instead of int32_t. - TokenDescriptor is a POD with an int32_t (this shrinks the size of AST and IR nodes by 32-bits on 64-bit architectures). There are some cleanups I plan on doing as a follow up CL: - Replace TokenDescriptor::value() with TokenDescriptor::TokenPos() R=iposva@google.com, rmacnak@google.com Review URL: https://codereview.chromium.org/1644793002 .
2016-02-02 18:15:44 +00:00
#undef DEFINE_VALUES
[vm] Remove the only cases where TokenPositions are mutated. The Next() method on TokenPositions used to increment the internal value_ field, but now creates a new TokenPosition with value_ + 1. There are only three places this is used: * Getting the token position after the end token position for implicit closure functions in the scope builder. * Getting the token position after the end token position for variable declarations in the scope builder. * Iterating over the token positions on a single source line in the debugger. So the overhead of creating new TokenPositions vs. mutating the original should be limited, compared to the benefit of being able to assume TokenPosition values are constant once created. There's too much dependence on the copy and copy assignment operators due to the previous possibility of mutation to make the internal field constant at the moment without much more sweeping changes, unfortunately. Also remove the copy assignment in initializing TokenPosition constants. TEST=Tested using existing tests on trybots. Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try Change-Id: Ie18b7f45bd9c6a43647ecd189330d91a4dff373b Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174922 Commit-Queue: Tess Strickland <sstrickl@google.com> Reviewed-by: Clement Skau <cskau@google.com>
2020-12-03 12:40:28 +00:00
const TokenPosition TokenPosition::kMinSource(kMinSourcePos);
const TokenPosition TokenPosition::kMaxSource(kMaxSourcePos);
Replace intptr_t with TokenDescriptor - Use TokenDescriptor instead of intptr_t for all token positions. - Use TokenDescriptor in raw_object instead of int32_t. - TokenDescriptor is a POD with an int32_t (this shrinks the size of AST and IR nodes by 32-bits on 64-bit architectures). There are some cleanups I plan on doing as a follow up CL: - Replace TokenDescriptor::value() with TokenDescriptor::TokenPos() R=iposva@google.com, rmacnak@google.com Review URL: https://codereview.chromium.org/1644793002 .
2016-02-02 18:15:44 +00:00
const char* TokenPosition::ToCString() const {
switch (value_) {
#define DEFINE_CASE(name, value) \
clang-format runtime/vm R=johnmccutchan@google.com Review URL: https://codereview.chromium.org/2481873005 .
2016-11-08 21:54:47 +00:00
case value: \
return #name;
Replace intptr_t with TokenDescriptor - Use TokenDescriptor instead of intptr_t for all token positions. - Use TokenDescriptor in raw_object instead of int32_t. - TokenDescriptor is a POD with an int32_t (this shrinks the size of AST and IR nodes by 32-bits on 64-bit architectures). There are some cleanups I plan on doing as a follow up CL: - Replace TokenDescriptor::value() with TokenDescriptor::TokenPos() R=iposva@google.com, rmacnak@google.com Review URL: https://codereview.chromium.org/1644793002 .
2016-02-02 18:15:44 +00:00
SENTINEL_TOKEN_DESCRIPTORS(DEFINE_CASE);
#undef DEFINE_CASE
[vm] Remove non-private uses of TokenPosition::value(). Instead, split each old use into the following cases: * If the TokenPosition value is expected to be a real token position, then use TokenPosition::Pos(). * If the TokenPosition is being serialized in some way, then use TokenPosition::Serialize() and change the place where the TokenPosition is recreated to use TokenPosition::Deserialize(). * If the value of the TokenPosition is being printed for debugging purposes, then just use TokenPosition::ToCString() instead. That is, we try to pin down when token positions are expected to be real vs. when other types of token positions can be found. Another source of possible error when using token positions is to convert between synthetic and real token positions. In the past, synthetic token positions may have been based off real token positions, but that is no longer the case. Thus, all methods that allow that conversion have been removed, and instead there is a new static method for constructing synthetic tokens from valid nonces. This CL also makes it so that Pos() and relational operators on token positions are only defined on real token positions, to avoid any assumptions about what the value encoded in synthetic positions mean. To help with cases where non-real token positions may occur, four helper methods are added: * TokenPosition::Min(a, b): A static method that returns the smallest real token position provided. If neither `a` or `b` are real, returns `a`. * TokenPosition::Max(a, b): A static method that returns the largest real token position provided. If neither `a` or `b` are real, returns `a`. * TokenPosition::IsWithin(start, end): Determines whether `this` falls between `start` and `end` (inclusive). If `this` is non-real, then it must be either `start` or `end` if synthetic, otherwise false. Otherwise, we mimic the old style of range checking, which means that non-real starts and ends are treated as less than every real token. * TokenPosition::CompareForSorting(other): Unlike the relational operators, provides a comparison between any types of token positions for purposes such as sorting. Currently only used in the profiler. It also changes TokenPosition::ToCString() to tag synthetic token positions, so they can be distinguished from real ones at a glance. TEST=Existing test suite on trybots, especially the observatory tests which make heavy use of the debugger and the unit tests for the profiler/source report modules. Bug: https://github.com/dart-lang/sdk/issues/44436 Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try Change-Id: Ic06aa0bc7a1f0fbac7257ed22ca5e7e0ccd7f3f2 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174924 Commit-Queue: Tess Strickland <sstrickl@google.com> Reviewed-by: Martin Kustermann <kustermann@google.com> Reviewed-by: Daco Harkes <dacoharkes@google.com>
2020-12-16 08:27:32 +00:00
default:
break;
}
ASSERT(IsReal() || IsSynthetic());
ZoneTextBuffer buffer(Thread::Current()->zone());
if (IsSynthetic()) {
buffer.AddString("syn:");
Replace intptr_t with TokenDescriptor - Use TokenDescriptor instead of intptr_t for all token positions. - Use TokenDescriptor in raw_object instead of int32_t. - TokenDescriptor is a POD with an int32_t (this shrinks the size of AST and IR nodes by 32-bits on 64-bit architectures). There are some cleanups I plan on doing as a follow up CL: - Replace TokenDescriptor::value() with TokenDescriptor::TokenPos() R=iposva@google.com, rmacnak@google.com Review URL: https://codereview.chromium.org/1644793002 .
2016-02-02 18:15:44 +00:00
}
[vm] Remove non-private uses of TokenPosition::value(). Instead, split each old use into the following cases: * If the TokenPosition value is expected to be a real token position, then use TokenPosition::Pos(). * If the TokenPosition is being serialized in some way, then use TokenPosition::Serialize() and change the place where the TokenPosition is recreated to use TokenPosition::Deserialize(). * If the value of the TokenPosition is being printed for debugging purposes, then just use TokenPosition::ToCString() instead. That is, we try to pin down when token positions are expected to be real vs. when other types of token positions can be found. Another source of possible error when using token positions is to convert between synthetic and real token positions. In the past, synthetic token positions may have been based off real token positions, but that is no longer the case. Thus, all methods that allow that conversion have been removed, and instead there is a new static method for constructing synthetic tokens from valid nonces. This CL also makes it so that Pos() and relational operators on token positions are only defined on real token positions, to avoid any assumptions about what the value encoded in synthetic positions mean. To help with cases where non-real token positions may occur, four helper methods are added: * TokenPosition::Min(a, b): A static method that returns the smallest real token position provided. If neither `a` or `b` are real, returns `a`. * TokenPosition::Max(a, b): A static method that returns the largest real token position provided. If neither `a` or `b` are real, returns `a`. * TokenPosition::IsWithin(start, end): Determines whether `this` falls between `start` and `end` (inclusive). If `this` is non-real, then it must be either `start` or `end` if synthetic, otherwise false. Otherwise, we mimic the old style of range checking, which means that non-real starts and ends are treated as less than every real token. * TokenPosition::CompareForSorting(other): Unlike the relational operators, provides a comparison between any types of token positions for purposes such as sorting. Currently only used in the profiler. It also changes TokenPosition::ToCString() to tag synthetic token positions, so they can be distinguished from real ones at a glance. TEST=Existing test suite on trybots, especially the observatory tests which make heavy use of the debugger and the unit tests for the profiler/source report modules. Bug: https://github.com/dart-lang/sdk/issues/44436 Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try Change-Id: Ic06aa0bc7a1f0fbac7257ed22ca5e7e0ccd7f3f2 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174924 Commit-Queue: Tess Strickland <sstrickl@google.com> Reviewed-by: Martin Kustermann <kustermann@google.com> Reviewed-by: Daco Harkes <dacoharkes@google.com>
2020-12-16 08:27:32 +00:00
buffer.Printf("%" Pd32 "", value_);
return buffer.buffer();
Replace intptr_t with TokenDescriptor - Use TokenDescriptor instead of intptr_t for all token positions. - Use TokenDescriptor in raw_object instead of int32_t. - TokenDescriptor is a POD with an int32_t (this shrinks the size of AST and IR nodes by 32-bits on 64-bit architectures). There are some cleanups I plan on doing as a follow up CL: - Replace TokenDescriptor::value() with TokenDescriptor::TokenPos() R=iposva@google.com, rmacnak@google.com Review URL: https://codereview.chromium.org/1644793002 .
2016-02-02 18:15:44 +00:00
}
} // namespace dart
Reference in a new issue Copy permalink