1
0
mirror of https://github.com/SerenityOS/serenity synced 2024-07-09 10:40:45 +00:00
Commit Graph

25 Commits

Author SHA1 Message Date
Andreas Kling
407d6cd9e4 AK: Rename Utf8CodepointIterator => Utf8CodePointIterator 2021-06-01 09:45:52 +02:00
Max Wipfli
14506e8f5e AK: Implement Utf8CodepointIterator::peek(size_t)
This adds a peek method for Utf8CodepointIterator, which enables it to
be used in some parsing cases where peeking is necessary.

peek(0) is equivalent to operator*, expect that peek() does not contain
any assertions and will just return an empty Optional<u32>.

This also implements a test case for iterating UTF-8.
2021-06-01 09:28:05 +02:00
Max Wipfli
a72bb34970 AK: Add Utf8View::iterator_at_byte_offset method
This implements a method to get a Utf8CodepointIterator at a specified
byte offset.
2021-05-21 21:57:03 +02:00
Max Wipfli
c1b452f754 AK: Add substring methods to Utf8View
This patch implements a Unicode-safe substring method, which can be used
when offset and length should be specified in actual characters instead
of bytes.

This can be used to mitigate issues where a string is split in the
middle of a UTF-8 multi-byte character, which leads to invalid UTF-8.

Furthermore, it implements to common shorthands for substring methods
which take only an offset and return the substring until the end of the
string.
2021-05-21 21:57:03 +02:00
Max Wipfli
8c19c2f296 AK: Change some argument and return types in Utf8View from int to size_t
This changes the return type of Utf8View::byte_length and the argument
types of substring_view from int to size_t.
2021-05-21 21:57:03 +02:00
Brian Gianforcaro
1682f0b760 Everything: Move to SPDX license identifiers in all files.
SPDX License Identifiers are a more compact / standardized
way of representing file license information.

See: https://spdx.dev/resources/use/#identifiers

This was done with the `ambr` search and replace tool.

 ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 11:22:27 +02:00
Idan Horowitz
edecf8f6a3 AK: Add starts_with to Utf8View
Unlike String/StringView::starts_with this compares utf8 code points
instead of "characters" (bytes), which is important when handling
aribtary utf-8 input that could include overlong characters.
2021-03-25 10:59:34 +01:00
Lenny Maiorani
e6f907a155 AK: Simplify constructors and conversions from nullptr_t
Problem:
- Many constructors are defined as `{}` rather than using the ` =
  default` compiler-provided constructor.
- Some types provide an implicit conversion operator from `nullptr_t`
  instead of requiring the caller to default construct. This violates
  the C++ Core Guidelines suggestion to declare single-argument
  constructors explicit
  (https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#c46-by-default-declare-single-argument-constructors-explicit).

Solution:
- Change default constructors to use the compiler-provided default
  constructor.
- Remove implicit conversion operators from `nullptr_t` and change
  usage to enforce type consistency without conversion.
2021-01-12 09:11:45 +01:00
asynts
2927656d85 AK: Use size_t in methods of Utf8View. 2021-01-02 01:37:22 +01:00
Andreas Kling
13594b7146 LibGfx+AK: Make text elision work with multi-byte characters
This was causing WindowServer and Taskbar to crash sometimes when the
stars aligned and we tried cutting off a string ending with "..." right
on top of an emoji. :^)
2020-12-28 23:54:10 +01:00
Lenny Maiorani
f5ced347e6 AK: Prefer using instead of typedef
Problem:
- `typedef` is a keyword which comes from C and carries with it old
  syntax that is hard to read.
- Creating type aliases with the `using` keyword allows for easier
  future maintenance because it supports template syntax.
- There is inconsistent use of `typedef` vs `using`.

Solution:
- Use `clang-tidy`'s checker called `modernize-use-using` to update
  the syntax to use the newer syntax.
- Remove unused functions to make `clang-tidy` happy.
- This results in consistency within the codebase.
2020-11-12 10:19:04 +01:00
Tom
6413acd78c AK: Make Utf8View and Utf32View more consistent
This enables use of these classes in templated code.
2020-10-22 15:23:45 +02:00
Nico Weber
ce95628b7f Unicode: Try s/codepoint/code_point/g again
This time, without trailing 's'. Ran:

    git grep -l 'codepoint' | xargs sed -ie 's/codepoint/code_point/g
2020-08-05 22:33:42 +02:00
Nico Weber
19ac1f6368 Revert "Unicode: s/codepoint/code_point/g"
This reverts commit ea9ac3155d.
It replaced "codepoint" with "code_points", not "code_point".
2020-08-05 22:33:42 +02:00
Andreas Kling
ea9ac3155d Unicode: s/codepoint/code_point/g
Unicode calls them "code points" so let's follow their style.
2020-08-03 19:06:41 +02:00
Matthew Olsson
c831fb17bf LibJS: Add StringIterator 2020-07-13 15:07:29 +02:00
Andreas Kling
23dad305e9 AK: Allow default-constructing Utf8View and Utf8CodepointIterator 2020-06-04 21:12:17 +02:00
AnotherTest
a4e0b585fe AK: Add a way to get the number of valid bytes in a Utf8View 2020-05-18 11:31:43 +02:00
Andreas Kling
6cde7e4d20 AK: Add Utf8View::length_in_codepoints() 2020-05-17 13:05:39 +02:00
Sergey Bugaev
c0b32f7b76 Meta: Claim copyright for files created by me
This changes copyright holder to myself for the source code files that I've
created or have (almost) completely rewritten. Not included are the files
that were significantly changed by others even though it was me who originally
created them (think HtmlView), or the many other files I've contributed code to.
2020-01-24 15:15:16 +01:00
Andreas Kling
94ca55cefd Meta: Add license header to source files
As suggested by Joshua, this commit adds the 2-clause BSD license as a
comment block to the top of every source file.

For the first pass, I've just added myself for simplicity. I encourage
everyone to add themselves as copyright holders of any file they've
added or modified in some significant way. If I've added myself in
error somewhere, feel free to replace it with the appropriate copyright
holder instead.

Going forward, all new source files should include a license header.
2020-01-18 09:45:54 +01:00
Andreas Kling
f4e6dae6fe UTF-8: Add Utf8CodepointIterator::codepoint_length_in_bytes()
This allows you to retrieve the length (in bytes) of the codepoint the
iterator is currently pointing at.
2019-10-18 22:49:23 +02:00
Andreas Kling
fb39e46d3d Utf8View: Try fixing the travis-ci build
There's some overload ambiguity when doing Utf8View("literal")
2019-09-05 19:06:39 +02:00
Sergey Bugaev
c379f43d2a AK: Add some more utility methods to Utf8View 2019-09-05 16:37:39 +02:00
Sergey Bugaev
5d3696174b AK: Add a Utf8View type for iterating over UTF-8 codepoints
Utf8View wraps a StringView and implements begin() and end() that
return a Utf8CodepointIterator, which parses UTF-8-encoded Unicode
codepoints and returns them as 32-bit integers.

This is the first step towards supporting emojis in Serenity ^)
https://github.com/SerenityOS/serenity/issues/490
2019-08-28 13:46:02 +02:00