system/serenity
system/serenity
1
0
Fork 0
mirror of https://github.com/SerenityOS/serenity synced 2024-07-22 18:46:18 +00:00
Code Issues Actions 5 Packages Projects Releases Wiki Activity
serenity/AK/URL.cpp

519 lines
18 KiB
C++
Raw Normal View History

Meta: Add license header to source files As suggested by Joshua, this commit adds the 2-clause BSD license as a comment block to the top of every source file. For the first pass, I've just added myself for simplicity. I encourage everyone to add themselves as copyright holders of any file they've added or modified in some significant way. If I've added myself in error somewhere, feel free to replace it with the appropriate copyright holder instead. Going forward, all new source files should include a license header.
2020-01-18 08:38:21 +00:00
/*
* Copyright (c) 2018-2020, Andreas Kling <kling@serenityos.org>
AK: Internally rename protocol to scheme in URL This renames all references to protocol to scheme, which is the name used by the URL standard (https://url.spec.whatwg.org/). Externally, all methods referencing "protocol" were duplicated with "scheme". The old methods still exist as compatibility.
2021-05-23 21:31:16 +00:00
* Copyright (c) 2021, Max Wipfli <mail@maxwipfli.ch>
Meta: Add license header to source files As suggested by Joshua, this commit adds the 2-clause BSD license as a comment block to the top of every source file. For the first pass, I've just added myself for simplicity. I encourage everyone to add themselves as copyright holders of any file they've added or modified in some significant way. If I've added myself in error somewhere, feel free to replace it with the appropriate copyright holder instead. Going forward, all new source files should include a license header.
2020-01-18 08:38:21 +00:00
*
Everything: Move to SPDX license identifiers in all files. SPDX License Identifiers are a more compact / standardized way of representing file license information. See: https://spdx.dev/resources/use/#identifiers This was done with the `ambr` search and replace tool. ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 08:24:48 +00:00
* SPDX-License-Identifier: BSD-2-Clause
Meta: Add license header to source files As suggested by Joshua, this commit adds the 2-clause BSD license as a comment block to the top of every source file. For the first pass, I've just added myself for simplicity. I encourage everyone to add themselves as copyright holders of any file they've added or modified in some significant way. If I've added myself in error somewhere, feel free to replace it with the appropriate copyright holder instead. Going forward, all new source files should include a license header.
2020-01-18 08:38:21 +00:00
*/
Everywhere: Replace ctype.h to avoid narrowing conversions This replaces ctype.h with CharacterType.h everywhere I could find issues with narrowing conversions. While using it will probably make sense almost everywhere in the future, the most critical places should have been addressed.
2021-06-01 19:18:08 +00:00
#include <AK/CharacterTypes.h>
AK: Replace old URL parser with new URLParser::parse() This replaces the old URL::parse() and URL::complete_url() parsing mechanisms with the new spec-compliant URLParser::parse().
2021-05-27 19:05:07 +00:00
#include <AK/Debug.h>
AK: Rename FileSystemPath -> LexicalPath And move canonicalized_path() to a static method on LexicalPath. This is to make it clear that FileSystemPath/canonicalized_path() only perform *lexical* canonicalization.
2020-05-26 11:52:44 +00:00
#include <AK/LexicalPath.h>
AK: Add a basic URL class to help us handle URL's We're gonna need these as we start to write more networking programs.
2019-08-10 15:27:56 +00:00
#include <AK/StringBuilder.h>
#include <AK/URL.h>
AK: Replace old URL parser with new URLParser::parse() This replaces the old URL::parse() and URL::complete_url() parsing mechanisms with the new spec-compliant URLParser::parse().
2021-05-27 19:05:07 +00:00
#include <AK/URLParser.h>
AK: Implement more conforming URL percent encode/decode mechanism This adds a few new functions to percent encode/decode strings according to the URL specification. The functions allow specifying a PercentEncodeSet, which is defined by the specification. It will be used to replace the current urlencode() and urldecode() functions in a further commit. This commit adds a few duplicate helper functions in the URL class, such as is_digit() and is_ascii_digit(). This will be cleaned up as soon as the upcoming new URL parser will replace the current one.
2021-05-25 11:50:03 +00:00
#include <AK/Utf8View.h>
AK: Add a basic URL class to help us handle URL's We're gonna need these as we start to write more networking programs.
2019-08-10 15:27:56 +00:00
namespace AK {
AK: Replace old URL parser with new URLParser::parse() This replaces the old URL::parse() and URL::complete_url() parsing mechanisms with the new spec-compliant URLParser::parse().
2021-05-27 19:05:07 +00:00
// FIXME: It could make sense to force users of URL to use URLParser::parse() explicitly instead of using a constructor.
Everywhere: Pass AK::StringView by value
2021-11-10 23:55:02 +00:00
URL::URL(StringView string)
AK: Accept optional url and state override parameters in URLParser These are required in the specification and used by the web's URL built-in, this commit also removes the Badge<AK::URL> from URLParser to allow other classes that need to call the parser directly like the web's URL built-in to do so.
2021-09-13 19:34:14 +00:00
: URL(URLParser::parse(string))
AK: Add a basic URL class to help us handle URL's We're gonna need these as we start to write more networking programs.
2019-08-10 15:27:56 +00:00
{
AK: Replace old URL parser with new URLParser::parse() This replaces the old URL::parse() and URL::complete_url() parsing mechanisms with the new spec-compliant URLParser::parse().
2021-05-27 19:05:07 +00:00
if constexpr (URL_PARSER_DEBUG) {
if (m_valid)
dbgln("URL constructor: Parsed URL to be '{}'.", serialize());
else
dbgln("URL constructor: Parsed URL to be invalid.");
}
AK: Add a basic URL class to help us handle URL's We're gonna need these as we start to write more networking programs.
2019-08-10 15:27:56 +00:00
}
AK+Tests+LibWeb: Make `URL::complete_url()` take a StringView All it does is pass this to `URLParser::parse()` which takes a StringView, so we might as well take one here too.
2023-02-13 17:42:27 +00:00
URL URL::complete_url(StringView relative_url) const
LibHTML+AK: Move URL completion from Document to AK::URL Completing a relative URL based on a base URL seems like generally useful functionality.
2019-11-18 21:04:39 +00:00
{
AK: Don't try to complete relative data: URLs
2020-06-07 16:23:33 +00:00
if (!is_valid())
return {};
AK+Everywhere: Use Optional for URLParser::parse's base_url parameter
2023-04-11 12:53:40 +00:00
return URLParser::parse(relative_url, *this);
LibHTML+AK: Move URL completion from Document to AK::URL Completing a relative URL based on a base URL seems like generally useful functionality.
2019-11-18 21:04:39 +00:00
}
AK+Everywhere: Add ApplyPercentDecoding option to URL getters The defaults selected for this are based on the behaviour of URL when it applied percent decoding during parsing. This does mean now in some cases the getters will allocate, but percent_decode() checks if there's anything to decode first, so in many cases still won't.
2023-04-13 22:06:58 +00:00
DeprecatedString URL::username(ApplyPercentDecoding apply_percent_decoding) const
{
return apply_percent_decoding == ApplyPercentDecoding::Yes ? percent_decode(m_username) : m_username;
}
DeprecatedString URL::password(ApplyPercentDecoding apply_percent_decoding) const
{
return apply_percent_decoding == ApplyPercentDecoding::Yes ? percent_decode(m_password) : m_password;
}
AK+Everywhere: Replace URL::paths() with path_segment_at_index() This allows accessing and looping over the path segments in a URL without necessarily allocating a new vector if you want them percent decoded too (which path_segment_at_index() has an option for).
2023-04-13 22:29:51 +00:00
DeprecatedString URL::path_segment_at_index(size_t index, ApplyPercentDecoding apply_percent_decoding) const
{
VERIFY(index < path_segment_count());
return apply_percent_decoding == ApplyPercentDecoding::Yes ? percent_decode(m_paths[index]) : m_paths[index];
}
AK+Everywhere: Add ApplyPercentDecoding option to URL getters The defaults selected for this are based on the behaviour of URL when it applied percent decoding during parsing. This does mean now in some cases the getters will allocate, but percent_decode() checks if there's anything to decode first, so in many cases still won't.
2023-04-13 22:06:58 +00:00
DeprecatedString URL::basename(ApplyPercentDecoding apply_percent_decoding) const
{
if (!m_valid)
return {};
if (m_paths.is_empty())
return {};
auto& last_segment = m_paths.last();
return apply_percent_decoding == ApplyPercentDecoding::Yes ? percent_decode(last_segment) : last_segment;
}
DeprecatedString URL::query(ApplyPercentDecoding apply_percent_decoding) const
{
return apply_percent_decoding == ApplyPercentDecoding::Yes ? percent_decode(m_query) : m_query;
}
DeprecatedString URL::fragment(ApplyPercentDecoding apply_percent_decoding) const
{
return apply_percent_decoding == ApplyPercentDecoding::Yes ? percent_decode(m_fragment) : m_fragment;
}
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
// NOTE: This only exists for compatibility with the existing URL tests which check for both .is_null() and .is_empty().
static DeprecatedString deprecated_string_percent_encode(DeprecatedString const& input, URL::PercentEncodeSet set = URL::PercentEncodeSet::Userinfo, URL::SpaceAsPlus space_as_plus = URL::SpaceAsPlus::No)
{
if (input.is_null() || input.is_empty())
return input;
return URL::percent_encode(input.view(), set, space_as_plus);
}
AK+Everywhere: Rename String to DeprecatedString We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-12-04 18:02:33 +00:00
void URL::set_scheme(DeprecatedString scheme)
AK: Recompute URL validity after changing protocol/host/path This allows you to build URLs by calling setters on an empty URL and actually get a valid URL at the end.
2020-04-11 21:07:23 +00:00
{
AK: Use correct constness in URL class methods This changes the URL class to use the correct constness for getters, setters and other methods. It also changes the entire class to use east const style.
2021-06-01 08:58:27 +00:00
m_scheme = move(scheme);
AK: Recompute URL validity after changing protocol/host/path This allows you to build URLs by calling setters on an empty URL and actually get a valid URL at the end.
2020-04-11 21:07:23 +00:00
m_valid = compute_validity();
}
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
void URL::set_username(DeprecatedString username, ApplyPercentEncoding apply_percent_encoding)
AK: Add member variables to the URL class This adds the m_username, m_password, m_paths and m_cannot_be_a_base_url member variables to the URL class. These are necessary for the upcoming new URL parser. The deprecated m_path variable shadows the m_paths variable if it is non-null. This behavior will be removed once the old URL parser has been removed.
2021-05-25 19:32:20 +00:00
{
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
if (apply_percent_encoding == ApplyPercentEncoding::Yes)
username = deprecated_string_percent_encode(username, PercentEncodeSet::Userinfo);
AK: Use correct constness in URL class methods This changes the URL class to use the correct constness for getters, setters and other methods. It also changes the entire class to use east const style.
2021-06-01 08:58:27 +00:00
m_username = move(username);
AK: Add member variables to the URL class This adds the m_username, m_password, m_paths and m_cannot_be_a_base_url member variables to the URL class. These are necessary for the upcoming new URL parser. The deprecated m_path variable shadows the m_paths variable if it is non-null. This behavior will be removed once the old URL parser has been removed.
2021-05-25 19:32:20 +00:00
m_valid = compute_validity();
}
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
void URL::set_password(DeprecatedString password, ApplyPercentEncoding apply_percent_encoding)
AK: Add member variables to the URL class This adds the m_username, m_password, m_paths and m_cannot_be_a_base_url member variables to the URL class. These are necessary for the upcoming new URL parser. The deprecated m_path variable shadows the m_paths variable if it is non-null. This behavior will be removed once the old URL parser has been removed.
2021-05-25 19:32:20 +00:00
{
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
if (apply_percent_encoding == ApplyPercentEncoding::Yes)
password = deprecated_string_percent_encode(password, PercentEncodeSet::Userinfo);
AK: Use correct constness in URL class methods This changes the URL class to use the correct constness for getters, setters and other methods. It also changes the entire class to use east const style.
2021-06-01 08:58:27 +00:00
m_password = move(password);
AK: Add member variables to the URL class This adds the m_username, m_password, m_paths and m_cannot_be_a_base_url member variables to the URL class. These are necessary for the upcoming new URL parser. The deprecated m_path variable shadows the m_paths variable if it is non-null. This behavior will be removed once the old URL parser has been removed.
2021-05-25 19:32:20 +00:00
m_valid = compute_validity();
}
AK+Everywhere: Rename String to DeprecatedString We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-12-04 18:02:33 +00:00
void URL::set_host(DeprecatedString host)
AK: Recompute URL validity after changing protocol/host/path This allows you to build URLs by calling setters on an empty URL and actually get a valid URL at the end.
2020-04-11 21:07:23 +00:00
{
AK: Use correct constness in URL class methods This changes the URL class to use the correct constness for getters, setters and other methods. It also changes the entire class to use east const style.
2021-06-01 08:58:27 +00:00
m_host = move(host);
AK: Recompute URL validity after changing protocol/host/path This allows you to build URLs by calling setters on an empty URL and actually get a valid URL at the end.
2020-04-11 21:07:23 +00:00
m_valid = compute_validity();
}
AK: Make URL::m_port an Optional<u16>, Expose raw port getter Our current way of signalling a missing port with m_port == 0 was lacking, as 0 is a valid port number in URLs.
2021-09-13 20:12:16 +00:00
void URL::set_port(Optional<u16> port)
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
{
AK: Add member variables to the URL class This adds the m_username, m_password, m_paths and m_cannot_be_a_base_url member variables to the URL class. These are necessary for the upcoming new URL parser. The deprecated m_path variable shadows the m_paths variable if it is non-null. This behavior will be removed once the old URL parser has been removed.
2021-05-25 19:32:20 +00:00
if (port == default_port_for_scheme(m_scheme)) {
AK: Make URL::m_port an Optional<u16>, Expose raw port getter Our current way of signalling a missing port with m_port == 0 was lacking, as 0 is a valid port number in URLs.
2021-09-13 20:12:16 +00:00
m_port = {};
AK: Add member variables to the URL class This adds the m_username, m_password, m_paths and m_cannot_be_a_base_url member variables to the URL class. These are necessary for the upcoming new URL parser. The deprecated m_path variable shadows the m_paths variable if it is non-null. This behavior will be removed once the old URL parser has been removed.
2021-05-25 19:32:20 +00:00
return;
}
AK: Make URL::m_port an Optional<u16>, Expose raw port getter Our current way of signalling a missing port with m_port == 0 was lacking, as 0 is a valid port number in URLs.
2021-09-13 20:12:16 +00:00
m_port = move(port);
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
m_valid = compute_validity();
}
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
void URL::set_paths(Vector<DeprecatedString> paths, ApplyPercentEncoding apply_percent_encoding)
AK: Add member variables to the URL class This adds the m_username, m_password, m_paths and m_cannot_be_a_base_url member variables to the URL class. These are necessary for the upcoming new URL parser. The deprecated m_path variable shadows the m_paths variable if it is non-null. This behavior will be removed once the old URL parser has been removed.
2021-05-25 19:32:20 +00:00
{
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
if (apply_percent_encoding == ApplyPercentEncoding::Yes) {
Vector<DeprecatedString> encoded_paths;
encoded_paths.ensure_capacity(paths.size());
for (auto& segment : paths)
encoded_paths.unchecked_append(deprecated_string_percent_encode(segment, PercentEncodeSet::Path));
m_paths = move(encoded_paths);
} else {
m_paths = move(paths);
}
AK: Add member variables to the URL class This adds the m_username, m_password, m_paths and m_cannot_be_a_base_url member variables to the URL class. These are necessary for the upcoming new URL parser. The deprecated m_path variable shadows the m_paths variable if it is non-null. This behavior will be removed once the old URL parser has been removed.
2021-05-25 19:32:20 +00:00
m_valid = compute_validity();
}
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
void URL::append_path(DeprecatedString path, ApplyPercentEncoding apply_percent_encoding)
{
if (apply_percent_encoding == ApplyPercentEncoding::Yes)
path = deprecated_string_percent_encode(path, PercentEncodeSet::Path);
m_paths.append(path);
}
void URL::set_query(DeprecatedString query, ApplyPercentEncoding apply_percent_encoding)
AK: Recompute URL validity after changing protocol/host/path This allows you to build URLs by calling setters on an empty URL and actually get a valid URL at the end.
2020-04-11 21:07:23 +00:00
{
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
if (apply_percent_encoding == ApplyPercentEncoding::Yes)
query = deprecated_string_percent_encode(query, is_special() ? PercentEncodeSet::SpecialQuery : PercentEncodeSet::Query);
AK: Use correct constness in URL class methods This changes the URL class to use the correct constness for getters, setters and other methods. It also changes the entire class to use east const style.
2021-06-01 08:58:27 +00:00
m_query = move(query);
AK: Recompute URL validity after changing protocol/host/path This allows you to build URLs by calling setters on an empty URL and actually get a valid URL at the end.
2020-04-11 21:07:23 +00:00
}
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
void URL::set_fragment(DeprecatedString fragment, ApplyPercentEncoding apply_percent_encoding)
AK: Support fragment in URL
2020-04-11 22:38:13 +00:00
{
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
if (apply_percent_encoding == ApplyPercentEncoding::Yes)
fragment = deprecated_string_percent_encode(fragment, PercentEncodeSet::Fragment);
AK: Use correct constness in URL class methods This changes the URL class to use the correct constness for getters, setters and other methods. It also changes the entire class to use east const style.
2021-06-01 08:58:27 +00:00
m_fragment = move(fragment);
AK: Support fragment in URL
2020-04-11 22:38:13 +00:00
}
AK: Rewrite URL::compute_validity() to conform to new parser This rewrites the URL validation check to be more specific, so it can more accurately detect if a user of URL class constructs invalid URLs by hand.
2021-05-29 18:46:49 +00:00
// FIXME: This is by no means complete.
// NOTE: This relies on some assumptions about how the spec-defined URL parser works that may turn out to be wrong.
AK: Recompute URL validity after changing protocol/host/path This allows you to build URLs by calling setters on an empty URL and actually get a valid URL at the end.
2020-04-11 21:07:23 +00:00
bool URL::compute_validity() const
{
AK: Internally rename protocol to scheme in URL This renames all references to protocol to scheme, which is the name used by the URL standard (https://url.spec.whatwg.org/). Externally, all methods referencing "protocol" were duplicated with "scheme". The old methods still exist as compatibility.
2021-05-23 21:31:16 +00:00
if (m_scheme.is_empty())
AK: Recompute URL validity after changing protocol/host/path This allows you to build URLs by calling setters on an empty URL and actually get a valid URL at the end.
2020-04-11 21:07:23 +00:00
return false;
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
AK: Internally rename protocol to scheme in URL This renames all references to protocol to scheme, which is the name used by the URL standard (https://url.spec.whatwg.org/). Externally, all methods referencing "protocol" were duplicated with "scheme". The old methods still exist as compatibility.
2021-05-23 21:31:16 +00:00
if (m_scheme == "data") {
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
if (m_data_mime_type.is_empty())
AK: Consider more URLs invalid Not just http or https. This fixes "foo" being recognized as a valid URL with protocol "foo", empty host and empty path.
2020-04-19 08:36:56 +00:00
return false;
AK: Rewrite URL::compute_validity() to conform to new parser This rewrites the URL validation check to be more specific, so it can more accurately detect if a user of URL class constructs invalid URLs by hand.
2021-05-29 18:46:49 +00:00
if (m_data_payload_is_base64) {
if (m_data_payload.length() % 4 != 0)
return false;
for (auto character : m_data_payload) {
if (!is_ascii_alphanumeric(character) || character == '+' || character == '/' || character == '=')
return false;
}
}
} else if (m_cannot_be_a_base_url) {
if (m_paths.size() != 1)
return false;
if (m_paths[0].is_empty())
return false;
} else {
if (m_scheme.is_one_of("about", "mailto"))
return false;
// NOTE: Maybe it is allowed to have a zero-segment path.
if (m_paths.size() == 0)
return false;
AK: Recompute URL validity after changing protocol/host/path This allows you to build URLs by calling setters on an empty URL and actually get a valid URL at the end.
2020-04-11 21:07:23 +00:00
}
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
AK: Rewrite URL::compute_validity() to conform to new parser This rewrites the URL validation check to be more specific, so it can more accurately detect if a user of URL class constructs invalid URLs by hand.
2021-05-29 18:46:49 +00:00
// NOTE: A file URL's host should be the empty string for localhost, not null.
if (m_scheme == "file" && m_host.is_null())
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
return false;
AK: Recompute URL validity after changing protocol/host/path This allows you to build URLs by calling setters on an empty URL and actually get a valid URL at the end.
2020-04-11 21:07:23 +00:00
return true;
}
Everywhere: Pass AK::StringView by value
2021-11-10 23:55:02 +00:00
bool URL::scheme_requires_port(StringView scheme)
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
{
AK: Internally rename protocol to scheme in URL This renames all references to protocol to scheme, which is the name used by the URL standard (https://url.spec.whatwg.org/). Externally, all methods referencing "protocol" were duplicated with "scheme". The old methods still exist as compatibility.
2021-05-23 21:31:16 +00:00
return (default_port_for_scheme(scheme) != 0);
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
}
Everywhere: Pass AK::StringView by value
2021-11-10 23:55:02 +00:00
u16 URL::default_port_for_scheme(StringView scheme)
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
{
AK: Internally rename protocol to scheme in URL This renames all references to protocol to scheme, which is the name used by the URL standard (https://url.spec.whatwg.org/). Externally, all methods referencing "protocol" were duplicated with "scheme". The old methods still exist as compatibility.
2021-05-23 21:31:16 +00:00
if (scheme == "http")
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
return 80;
AK: Internally rename protocol to scheme in URL This renames all references to protocol to scheme, which is the name used by the URL standard (https://url.spec.whatwg.org/). Externally, all methods referencing "protocol" were duplicated with "scheme". The old methods still exist as compatibility.
2021-05-23 21:31:16 +00:00
if (scheme == "https")
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
return 443;
AK: Internally rename protocol to scheme in URL This renames all references to protocol to scheme, which is the name used by the URL standard (https://url.spec.whatwg.org/). Externally, all methods referencing "protocol" were duplicated with "scheme". The old methods still exist as compatibility.
2021-05-23 21:31:16 +00:00
if (scheme == "gemini")
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
return 1965;
AK: Internally rename protocol to scheme in URL This renames all references to protocol to scheme, which is the name used by the URL standard (https://url.spec.whatwg.org/). Externally, all methods referencing "protocol" were duplicated with "scheme". The old methods still exist as compatibility.
2021-05-23 21:31:16 +00:00
if (scheme == "irc")
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
return 6667;
AK: Internally rename protocol to scheme in URL This renames all references to protocol to scheme, which is the name used by the URL standard (https://url.spec.whatwg.org/). Externally, all methods referencing "protocol" were duplicated with "scheme". The old methods still exist as compatibility.
2021-05-23 21:31:16 +00:00
if (scheme == "ircs")
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
return 6697;
AK: Internally rename protocol to scheme in URL This renames all references to protocol to scheme, which is the name used by the URL standard (https://url.spec.whatwg.org/). Externally, all methods referencing "protocol" were duplicated with "scheme". The old methods still exist as compatibility.
2021-05-23 21:31:16 +00:00
if (scheme == "ws")
AK: Add default ports for Websockets to the URL class
2021-04-16 13:21:03 +00:00
return 80;
AK: Internally rename protocol to scheme in URL This renames all references to protocol to scheme, which is the name used by the URL standard (https://url.spec.whatwg.org/). Externally, all methods referencing "protocol" were duplicated with "scheme". The old methods still exist as compatibility.
2021-05-23 21:31:16 +00:00
if (scheme == "wss")
AK: Add default ports for Websockets to the URL class
2021-04-16 13:21:03 +00:00
return 443;
AK::URL: Check if URL requires a port set to be considered a valid URL `AK::URL` will now check if the URL requires a port to be set using `AK::URL.protocol_requires_port(protocol)`. If the URL does not specify a port, and no default port for the URL protocol is found with `AK::URL.default_port_for_protocol(protocol)`, the URL is considered to be invalid.
2020-11-04 06:20:20 +00:00
return 0;
}
AK+Everywhere: Rename String to DeprecatedString We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-12-04 18:02:33 +00:00
URL URL::create_with_file_scheme(DeprecatedString const& path, DeprecatedString const& fragment, DeprecatedString const& hostname)
AK: Add URL::create_with_file_protocol(path) This is a convenience helper that allows you to easily construct a file:// URL from an absolute path.
2020-04-18 20:02:04 +00:00
{
AK: Remove deprecated m_path member variable from URL The m_path member variable has been superseded by m_paths. Thus, it has been removed. The path() getter will continue to exist as a convenience method for getting the path joined together as a string.
2021-05-27 19:40:02 +00:00
LexicalPath lexical_path(path);
AK: Remove the LexicalPath::is_valid() API Since this is always set to true on the non-default constructor and subsequently never modified, it is somewhat pointless. Furthermore, there are arguably no invalid relative paths.
2021-06-29 11:11:03 +00:00
if (!lexical_path.is_absolute())
AK: Remove deprecated m_path member variable from URL The m_path member variable has been superseded by m_paths. Thus, it has been removed. The path() getter will continue to exist as a convenience method for getting the path joined together as a string.
2021-05-27 19:40:02 +00:00
return {};
AK: Add hostname parameter to URL::create_with_file_scheme() This adds a hostname parameter as the third parameter to URL::create_with_file_scheme(). If the hostname is "localhost", it will be ignored (as per the URL specification). This can for example be used by ls(1) to create more conforming file URLs.
2021-05-29 19:57:20 +00:00
AK: Add URL::create_with_file_protocol(path) This is a convenience helper that allows you to easily construct a file:// URL from an absolute path.
2020-04-18 20:02:04 +00:00
URL url;
AK: Internally rename protocol to scheme in URL This renames all references to protocol to scheme, which is the name used by the URL standard (https://url.spec.whatwg.org/). Externally, all methods referencing "protocol" were duplicated with "scheme". The old methods still exist as compatibility.
2021-05-23 21:31:16 +00:00
url.set_scheme("file");
AK: Add hostname parameter to URL::create_with_file_scheme() This adds a hostname parameter as the third parameter to URL::create_with_file_scheme(). If the hostname is "localhost", it will be ignored (as per the URL specification). This can for example be used by ls(1) to create more conforming file URLs.
2021-05-29 19:57:20 +00:00
// NOTE: If the hostname is localhost (or null, which implies localhost), it should be set to the empty string.
AK: Add `URL::create_with_help_scheme` helper function
2022-03-24 02:46:52 +00:00
// This is because a file URL always needs a non-null hostname.
AK+Everywhere: Rename String to DeprecatedString We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-12-04 18:02:33 +00:00
url.set_host(hostname.is_null() || hostname == "localhost" ? DeprecatedString::empty() : hostname);
AK: Add `URL::create_with_help_scheme` helper function
2022-03-24 02:46:52 +00:00
url.set_paths(lexical_path.parts());
if (path.ends_with('/'))
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
url.append_slash();
AK: Add `URL::create_with_help_scheme` helper function
2022-03-24 02:46:52 +00:00
url.set_fragment(fragment);
return url;
}
AK+Everywhere: Rename String to DeprecatedString We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-12-04 18:02:33 +00:00
URL URL::create_with_help_scheme(DeprecatedString const& path, DeprecatedString const& fragment, DeprecatedString const& hostname)
AK: Add `URL::create_with_help_scheme` helper function
2022-03-24 02:46:52 +00:00
{
LexicalPath lexical_path(path);
URL url;
url.set_scheme("help");
// NOTE: If the hostname is localhost (or null, which implies localhost), it should be set to the empty string.
AK: Add hostname parameter to URL::create_with_file_scheme() This adds a hostname parameter as the third parameter to URL::create_with_file_scheme(). If the hostname is "localhost", it will be ignored (as per the URL specification). This can for example be used by ls(1) to create more conforming file URLs.
2021-05-29 19:57:20 +00:00
// This is because a file URL always needs a non-null hostname.
AK+Everywhere: Rename String to DeprecatedString We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-12-04 18:02:33 +00:00
url.set_host(hostname.is_null() || hostname == "localhost" ? DeprecatedString::empty() : hostname);
AK: Remove deprecated m_path member variable from URL The m_path member variable has been superseded by m_paths. Thus, it has been removed. The path() getter will continue to exist as a convenience method for getting the path joined together as a string.
2021-05-27 19:40:02 +00:00
url.set_paths(lexical_path.parts());
if (path.ends_with('/'))
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
url.append_slash();
AK: Add optional fragment parameter to create_with_file_protocol() Now that we use fragment for specifying starting selection in FileManager we would benefit from providing it as argument instead of setting it each time separately.
2021-03-01 21:24:34 +00:00
url.set_fragment(fragment);
AK: Add URL::create_with_file_protocol(path) This is a convenience helper that allows you to easily construct a file:// URL from an absolute path.
2020-04-18 20:02:04 +00:00
return url;
}
AK+Everywhere: Rename String to DeprecatedString We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-12-04 18:02:33 +00:00
URL URL::create_with_url_or_path(DeprecatedString const& url_or_path)
AK: Add URL::create_with_url_or_path() This is an utility to create a URL from a given string, which may be either a URL such as http://example.com (which will be used as-is), or a file path such as /etc/fstab (which will be transformed into file:///etc/fstab).
2020-04-19 08:55:59 +00:00
{
URL url = url_or_path;
if (url.is_valid())
return url;
AK+Everywhere: Rename String to DeprecatedString We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-12-04 18:02:33 +00:00
DeprecatedString path = LexicalPath::canonicalized_path(url_or_path);
AK: Internally rename protocol to scheme in URL This renames all references to protocol to scheme, which is the name used by the URL standard (https://url.spec.whatwg.org/). Externally, all methods referencing "protocol" were duplicated with "scheme". The old methods still exist as compatibility.
2021-05-23 21:31:16 +00:00
return URL::create_with_file_scheme(path);
AK: Add URL::create_with_url_or_path() This is an utility to create a URL from a given string, which may be either a URL such as http://example.com (which will be used as-is), or a file path such as /etc/fstab (which will be transformed into file:///etc/fstab).
2020-04-19 08:55:59 +00:00
}
AK: Add helper functions and private data URL constructor to URL This adds a few helper functions and a private constructor to instantiate a data URL to the URL class. These will be needed by the upcoming URL parser.
2021-05-25 20:05:01 +00:00
// https://url.spec.whatwg.org/#special-scheme
Everywhere: Pass AK::StringView by value
2021-11-10 23:55:02 +00:00
bool URL::is_special_scheme(StringView scheme)
AK: Add helper functions and private data URL constructor to URL This adds a few helper functions and a private constructor to instantiate a data URL to the URL class. These will be needed by the upcoming URL parser.
2021-05-25 20:05:01 +00:00
{
return scheme.is_one_of("ftp", "file", "http", "https", "ws", "wss");
}
AK+Everywhere: Change URL::path() to serialize_path() This now defaults to serializing the path with percent decoded segments (which is what all callers expect), but has an option not to. This fixes `file://` URLs with spaces in their paths. The name has been changed to serialize_path() path to make it more clear that this method will generate a new string each call (except for the cannot_be_a_base_url() case). A few callers have then been updated to avoid repeatedly calling this function.
2023-04-14 19:12:03 +00:00
DeprecatedString URL::serialize_path(ApplyPercentDecoding apply_percent_decoding) const
{
if (cannot_be_a_base_url())
return m_paths[0];
StringBuilder builder;
for (auto& path : m_paths) {
builder.append('/');
builder.append(apply_percent_decoding == ApplyPercentDecoding::Yes ? percent_decode(path) : path);
}
return builder.to_deprecated_string();
}
AK+Everywhere: Rename String to DeprecatedString We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-12-04 18:02:33 +00:00
DeprecatedString URL::serialize_data_url() const
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
{
VERIFY(m_scheme == "data");
VERIFY(!m_data_mime_type.is_null());
VERIFY(!m_data_payload.is_null());
StringBuilder builder;
builder.append(m_scheme);
builder.append(':');
builder.append(m_data_mime_type);
if (m_data_payload_is_base64)
Everywhere: Add sv suffix to strings relying on StringView(char const*) Each of these strings would previously rely on StringView's char const* constructor overload, which would call __builtin_strlen on the string. Since we now have operator ""sv, we can replace these with much simpler versions. This opens the door to being able to remove StringView(char const*). No functional changes.
2022-07-11 17:32:29 +00:00
builder.append(";base64"sv);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
builder.append(',');
// NOTE: The specification does not say anything about encoding this, but we should encode at least control and non-ASCII
// characters (since this is also a valid representation of the same data URL).
builder.append(URL::percent_encode(m_data_payload, PercentEncodeSet::C0Control));
Everywhere: Rename to_{string => deprecated_string}() where applicable This will make it easier to support both string types at the same time while we convert code, and tracking down remaining uses. One big exception is Value::to_string() in LibJS, where the name is dictated by the ToString AO.
2022-12-06 01:12:49 +00:00
return builder.to_deprecated_string();
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
}
// https://url.spec.whatwg.org/#concept-url-serializer
AK+Everywhere: Rename String to DeprecatedString We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-12-04 18:02:33 +00:00
DeprecatedString URL::serialize(ExcludeFragment exclude_fragment) const
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
{
if (m_scheme == "data")
return serialize_data_url();
StringBuilder builder;
builder.append(m_scheme);
builder.append(':');
if (!m_host.is_null()) {
Everywhere: Add sv suffix to strings relying on StringView(char const*) Each of these strings would previously rely on StringView's char const* constructor overload, which would call __builtin_strlen on the string. Since we now have operator ""sv, we can replace these with much simpler versions. This opens the door to being able to remove StringView(char const*). No functional changes.
2022-07-11 17:32:29 +00:00
builder.append("//"sv);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
if (includes_credentials()) {
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
builder.append(m_username);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
if (!m_password.is_empty()) {
builder.append(':');
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
builder.append(m_password);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
}
builder.append('@');
}
builder.append(m_host);
AK: Make URL::m_port an Optional<u16>, Expose raw port getter Our current way of signalling a missing port with m_port == 0 was lacking, as 0 is a valid port number in URLs.
2021-09-13 20:12:16 +00:00
if (m_port.has_value())
builder.appendff(":{}", *m_port);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
}
if (cannot_be_a_base_url()) {
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
builder.append(m_paths[0]);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
} else {
AK: Remove deprecated m_path member variable from URL The m_path member variable has been superseded by m_paths. Thus, it has been removed. The path() getter will continue to exist as a convenience method for getting the path joined together as a string.
2021-05-27 19:40:02 +00:00
if (m_host.is_null() && m_paths.size() > 1 && m_paths[0].is_empty())
Everywhere: Add sv suffix to strings relying on StringView(char const*) Each of these strings would previously rely on StringView's char const* constructor overload, which would call __builtin_strlen on the string. Since we now have operator ""sv, we can replace these with much simpler versions. This opens the door to being able to remove StringView(char const*). No functional changes.
2022-07-11 17:32:29 +00:00
builder.append("/."sv);
AK: Remove deprecated m_path member variable from URL The m_path member variable has been superseded by m_paths. Thus, it has been removed. The path() getter will continue to exist as a convenience method for getting the path joined together as a string.
2021-05-27 19:40:02 +00:00
for (auto& segment : m_paths) {
builder.append('/');
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
builder.append(segment);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
}
}
if (!m_query.is_null()) {
builder.append('?');
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
builder.append(m_query);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
}
if (exclude_fragment == ExcludeFragment::No && !m_fragment.is_null()) {
builder.append('#');
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
builder.append(m_fragment);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
}
Everywhere: Rename to_{string => deprecated_string}() where applicable This will make it easier to support both string types at the same time while we convert code, and tracking down remaining uses. One big exception is Value::to_string() in LibJS, where the name is dictated by the ToString AO.
2022-12-06 01:12:49 +00:00
return builder.to_deprecated_string();
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
}
// https://url.spec.whatwg.org/#url-rendering
// NOTE: This does e.g. not display credentials.
// FIXME: Parts of the URL other than the host should have their sequences of percent-encoded bytes replaced with code points
// resulting from percent-decoding those sequences converted to bytes, unless that renders those sequences invisible.
AK+Everywhere: Rename String to DeprecatedString We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-12-04 18:02:33 +00:00
DeprecatedString URL::serialize_for_display() const
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
{
VERIFY(m_valid);
if (m_scheme == "data")
return serialize_data_url();
StringBuilder builder;
builder.append(m_scheme);
builder.append(':');
if (!m_host.is_null()) {
Everywhere: Add sv suffix to strings relying on StringView(char const*) Each of these strings would previously rely on StringView's char const* constructor overload, which would call __builtin_strlen on the string. Since we now have operator ""sv, we can replace these with much simpler versions. This opens the door to being able to remove StringView(char const*). No functional changes.
2022-07-11 17:32:29 +00:00
builder.append("//"sv);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
builder.append(m_host);
AK: Make URL::m_port an Optional<u16>, Expose raw port getter Our current way of signalling a missing port with m_port == 0 was lacking, as 0 is a valid port number in URLs.
2021-09-13 20:12:16 +00:00
if (m_port.has_value())
builder.appendff(":{}", *m_port);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
}
if (cannot_be_a_base_url()) {
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
builder.append(m_paths[0]);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
} else {
AK: Remove deprecated m_path member variable from URL The m_path member variable has been superseded by m_paths. Thus, it has been removed. The path() getter will continue to exist as a convenience method for getting the path joined together as a string.
2021-05-27 19:40:02 +00:00
if (m_host.is_null() && m_paths.size() > 1 && m_paths[0].is_empty())
Everywhere: Add sv suffix to strings relying on StringView(char const*) Each of these strings would previously rely on StringView's char const* constructor overload, which would call __builtin_strlen on the string. Since we now have operator ""sv, we can replace these with much simpler versions. This opens the door to being able to remove StringView(char const*). No functional changes.
2022-07-11 17:32:29 +00:00
builder.append("/."sv);
AK: Remove deprecated m_path member variable from URL The m_path member variable has been superseded by m_paths. Thus, it has been removed. The path() getter will continue to exist as a convenience method for getting the path joined together as a string.
2021-05-27 19:40:02 +00:00
for (auto& segment : m_paths) {
builder.append('/');
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
builder.append(segment);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
}
}
if (!m_query.is_null()) {
builder.append('?');
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
builder.append(m_query);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
}
if (!m_fragment.is_null()) {
builder.append('#');
AK: Don't store parts of URLs percent decoded As noted in serval comments doing this goes against the WC3 spec, and breaks parsing then re-serializing URLs that contain percent encoded data, that was not encoded using the same character set as the serializer. For example, previously if you had a URL like: https:://foo.com/what%2F%2F (the path is what + '//' percent encoded) Creating URL("https:://foo.com/what%2F%2F").serialize() would return: https://foo.com/what// Which is incorrect and not the same as the URL we passed. This is because the re-serializing uses the PercentEncodeSet::Path which does not include '/'. Only doing the percent encoding in the setters fixes this, which is required to navigate to Google Street View (which includes a percent encoded URL in its URL). Seems to fix #13477 too
2023-04-09 13:21:00 +00:00
builder.append(m_fragment);
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
}
Everywhere: Rename to_{string => deprecated_string}() where applicable This will make it easier to support both string types at the same time while we convert code, and tracking down remaining uses. One big exception is Value::to_string() in LibJS, where the name is dictated by the ToString AO.
2022-12-06 01:12:49 +00:00
return builder.to_deprecated_string();
AK: Add spec-compliant URL serialization methods This adds URL serialization methods which are more in line with the specification. The serialize_for_display() method should be used e.g. in the browser address bar, and as per the spec should not display username and password. Furthermore, it could decode most percent-encoded code points, although that is not implemented yet.
2021-05-25 20:32:39 +00:00
}
AK: Add URL::to_string() returning a String For convenience wire up a to_string() method which returns a new String.
2023-06-17 07:15:40 +00:00
ErrorOr<String> URL::to_string() const
{
return String::from_deprecated_string(serialize());
}
AK: Add URL::serialize_origin based on HTML's origin definition
2021-09-13 19:18:14 +00:00
// https://html.spec.whatwg.org/multipage/origin.html#ascii-serialisation-of-an-origin
// https://url.spec.whatwg.org/#concept-url-origin
AK+Everywhere: Rename String to DeprecatedString We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-12-04 18:02:33 +00:00
DeprecatedString URL::serialize_origin() const
AK: Add URL::serialize_origin based on HTML's origin definition
2021-09-13 19:18:14 +00:00
{
VERIFY(m_valid);
if (m_scheme == "blob"sv) {
// TODO: 1. If URL’s blob URL entry is non-null, then return URL’s blob URL entry’s environment’s origin.
// 2. Let url be the result of parsing URL’s path[0].
VERIFY(!m_paths.is_empty());
URL url = m_paths[0];
// 3. Return a new opaque origin, if url is failure, and url’s origin otherwise.
if (!url.is_valid())
return "null";
return url.serialize_origin();
} else if (!m_scheme.is_one_of("ftp"sv, "http"sv, "https"sv, "ws"sv, "wss"sv)) { // file: "Unfortunate as it is, this is left as an exercise to the reader. When in doubt, return a new opaque origin."
return "null";
}
StringBuilder builder;
builder.append(m_scheme);
builder.append("://"sv);
builder.append(m_host);
AK: Make URL::m_port an Optional<u16>, Expose raw port getter Our current way of signalling a missing port with m_port == 0 was lacking, as 0 is a valid port number in URLs.
2021-09-13 20:12:16 +00:00
if (m_port.has_value())
AK: Append correct number of port characters when serializing a URL Instead of formatting a port string, it put bytes from stack, using the port number as a length (so for port 8000 it appended 8000 bytes).
2022-06-10 18:37:51 +00:00
builder.appendff(":{}", *m_port);
AK: Remove StringBuilder::build() in favor of to_deprecated_string() Having an alias function that only wraps another one is silly, and keeping the more obvious name should flush out more uses of deprecated strings. No behavior change.
2023-01-26 18:58:09 +00:00
return builder.to_deprecated_string();
AK: Add URL::serialize_origin based on HTML's origin definition
2021-09-13 19:18:14 +00:00
}
AK: Use correct constness in URL class methods This changes the URL class to use the correct constness for getters, setters and other methods. It also changes the entire class to use east const style.
2021-06-01 08:58:27 +00:00
bool URL::equals(URL const& other, ExcludeFragment exclude_fragments) const
AK: Replace URL::to_string() with new serialize() implementation
2021-05-27 19:38:16 +00:00
{
AK: Move identity check from URL::operator==() to equals()
2021-06-01 09:14:30 +00:00
if (this == &other)
return true;
AK: Replace URL::to_string() with new serialize() implementation
2021-05-27 19:38:16 +00:00
if (!m_valid || !other.m_valid)
return false;
return serialize(exclude_fragments) == other.serialize(exclude_fragments);
}
AK: Implement more conforming URL percent encode/decode mechanism This adds a few new functions to percent encode/decode strings according to the URL specification. The functions allow specifying a PercentEncodeSet, which is defined by the specification. It will be used to replace the current urlencode() and urldecode() functions in a further commit. This commit adds a few duplicate helper functions in the URL class, such as is_digit() and is_ascii_digit(). This will be cleaned up as soon as the upcoming new URL parser will replace the current one.
2021-05-25 11:50:03 +00:00
void URL::append_percent_encoded(StringBuilder& builder, u32 code_point)
{
if (code_point <= 0x7f)
builder.appendff("%{:02X}", code_point);
else if (code_point <= 0x07ff)
builder.appendff("%{:02X}%{:02X}", ((code_point >> 6) & 0x1f) | 0xc0, (code_point & 0x3f) | 0x80);
else if (code_point <= 0xffff)
builder.appendff("%{:02X}%{:02X}%{:02X}", ((code_point >> 12) & 0x0f) | 0xe0, ((code_point >> 6) & 0x3f) | 0x80, (code_point & 0x3f) | 0x80);
else if (code_point <= 0x10ffff)
builder.appendff("%{:02X}%{:02X}%{:02X}%{:02X}", ((code_point >> 18) & 0x07) | 0xf0, ((code_point >> 12) & 0x3f) | 0x80, ((code_point >> 6) & 0x3f) | 0x80, (code_point & 0x3f) | 0x80);
else
VERIFY_NOT_REACHED();
}
// https://url.spec.whatwg.org/#c0-control-percent-encode-set
AK: Don't destructively re-encode query strings in the URL parser We were decoding and then re-encoding the query string in URLs. This round-trip caused us to lose information about plus ('+') ASCII characters encoded as "%2B".
2022-04-09 22:48:15 +00:00
bool URL::code_point_is_in_percent_encode_set(u32 code_point, URL::PercentEncodeSet set)
AK: Implement more conforming URL percent encode/decode mechanism This adds a few new functions to percent encode/decode strings according to the URL specification. The functions allow specifying a PercentEncodeSet, which is defined by the specification. It will be used to replace the current urlencode() and urldecode() functions in a further commit. This commit adds a few duplicate helper functions in the URL class, such as is_digit() and is_ascii_digit(). This will be cleaned up as soon as the upcoming new URL parser will replace the current one.
2021-05-25 11:50:03 +00:00
{
switch (set) {
case URL::PercentEncodeSet::C0Control:
return code_point < 0x20 || code_point > 0x7E;
case URL::PercentEncodeSet::Fragment:
return code_point_is_in_percent_encode_set(code_point, URL::PercentEncodeSet::C0Control) || " \"<>`"sv.contains(code_point);
case URL::PercentEncodeSet::Query:
return code_point_is_in_percent_encode_set(code_point, URL::PercentEncodeSet::C0Control) || " \"#<>"sv.contains(code_point);
case URL::PercentEncodeSet::SpecialQuery:
return code_point_is_in_percent_encode_set(code_point, URL::PercentEncodeSet::Query) || code_point == '\'';
case URL::PercentEncodeSet::Path:
return code_point_is_in_percent_encode_set(code_point, URL::PercentEncodeSet::Query) || "?`{}"sv.contains(code_point);
case URL::PercentEncodeSet::Userinfo:
return code_point_is_in_percent_encode_set(code_point, URL::PercentEncodeSet::Path) || "/:;=@[\\]^|"sv.contains(code_point);
case URL::PercentEncodeSet::Component:
return code_point_is_in_percent_encode_set(code_point, URL::PercentEncodeSet::Userinfo) || "$%&+,"sv.contains(code_point);
case URL::PercentEncodeSet::ApplicationXWWWFormUrlencoded:
AK: Make URL ApplicationXWWWFormUrlencoded encoding closer to spec It was mostly implemented based on a spec note, that described only allowed characters, but instead of allowing some special characters not to be escaped, we escaped every special character except those 'new in this encode set' disallowed characters from the spec definition.
2022-06-10 18:39:08 +00:00
return code_point_is_in_percent_encode_set(code_point, URL::PercentEncodeSet::Component) || "!'()~"sv.contains(code_point);
AK: Implement more conforming URL percent encode/decode mechanism This adds a few new functions to percent encode/decode strings according to the URL specification. The functions allow specifying a PercentEncodeSet, which is defined by the specification. It will be used to replace the current urlencode() and urldecode() functions in a further commit. This commit adds a few duplicate helper functions in the URL class, such as is_digit() and is_ascii_digit(). This will be cleaned up as soon as the upcoming new URL parser will replace the current one.
2021-05-25 11:50:03 +00:00
case URL::PercentEncodeSet::EncodeURI:
// NOTE: This is the same percent encode set that JS encodeURI() uses.
// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURI
AK: Remove tilde from URL::PercentEncodeSet::EncodeURI
2022-12-25 19:25:34 +00:00
return code_point > 0x7E || (!is_ascii_alphanumeric(code_point) && !";,/?:@&=+$-_.!~*'()#"sv.contains(static_cast<char>(code_point)));
AK: Implement more conforming URL percent encode/decode mechanism This adds a few new functions to percent encode/decode strings according to the URL specification. The functions allow specifying a PercentEncodeSet, which is defined by the specification. It will be used to replace the current urlencode() and urldecode() functions in a further commit. This commit adds a few duplicate helper functions in the URL class, such as is_digit() and is_ascii_digit(). This will be cleaned up as soon as the upcoming new URL parser will replace the current one.
2021-05-25 11:50:03 +00:00
default:
VERIFY_NOT_REACHED();
}
}
AK+LibHTTP: Revert prior change to percent encode plus signs A change was made prior to percent encode plus signs in order to fix an issue with the Google cookie consent page. Unforunately, this was treating a symptom of a problem and not the root cause and is incorrect behavior.
2022-04-08 13:20:30 +00:00
void URL::append_percent_encoded_if_necessary(StringBuilder& builder, u32 code_point, URL::PercentEncodeSet set)
AK: Implement more conforming URL percent encode/decode mechanism This adds a few new functions to percent encode/decode strings according to the URL specification. The functions allow specifying a PercentEncodeSet, which is defined by the specification. It will be used to replace the current urlencode() and urldecode() functions in a further commit. This commit adds a few duplicate helper functions in the URL class, such as is_digit() and is_ascii_digit(). This will be cleaned up as soon as the upcoming new URL parser will replace the current one.
2021-05-25 11:50:03 +00:00
{
AK+LibHTTP: Revert prior change to percent encode plus signs A change was made prior to percent encode plus signs in order to fix an issue with the Google cookie consent page. Unforunately, this was treating a symptom of a problem and not the root cause and is incorrect behavior.
2022-04-08 13:20:30 +00:00
if (code_point_is_in_percent_encode_set(code_point, set))
AK: Implement more conforming URL percent encode/decode mechanism This adds a few new functions to percent encode/decode strings according to the URL specification. The functions allow specifying a PercentEncodeSet, which is defined by the specification. It will be used to replace the current urlencode() and urldecode() functions in a further commit. This commit adds a few duplicate helper functions in the URL class, such as is_digit() and is_ascii_digit(). This will be cleaned up as soon as the upcoming new URL parser will replace the current one.
2021-05-25 11:50:03 +00:00
append_percent_encoded(builder, code_point);
else
builder.append_code_point(code_point);
}
AK+Everywhere: Rename String to DeprecatedString We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-12-04 18:02:33 +00:00
DeprecatedString URL::percent_encode(StringView input, URL::PercentEncodeSet set, SpaceAsPlus space_as_plus)
AK: Implement more conforming URL percent encode/decode mechanism This adds a few new functions to percent encode/decode strings according to the URL specification. The functions allow specifying a PercentEncodeSet, which is defined by the specification. It will be used to replace the current urlencode() and urldecode() functions in a further commit. This commit adds a few duplicate helper functions in the URL class, such as is_digit() and is_ascii_digit(). This will be cleaned up as soon as the upcoming new URL parser will replace the current one.
2021-05-25 11:50:03 +00:00
{
StringBuilder builder;
for (auto code_point : Utf8View(input)) {
AK+LibWeb: Encode ' ' as '+' in application/x-www-form-urlencoded This matches what the URL and HTML specifications ask us to do.
2022-04-09 16:34:49 +00:00
if (space_as_plus == SpaceAsPlus::Yes && code_point == ' ')
builder.append('+');
else
append_percent_encoded_if_necessary(builder, code_point, set);
AK: Implement more conforming URL percent encode/decode mechanism This adds a few new functions to percent encode/decode strings according to the URL specification. The functions allow specifying a PercentEncodeSet, which is defined by the specification. It will be used to replace the current urlencode() and urldecode() functions in a further commit. This commit adds a few duplicate helper functions in the URL class, such as is_digit() and is_ascii_digit(). This will be cleaned up as soon as the upcoming new URL parser will replace the current one.
2021-05-25 11:50:03 +00:00
}
Everywhere: Rename to_{string => deprecated_string}() where applicable This will make it easier to support both string types at the same time while we convert code, and tracking down remaining uses. One big exception is Value::to_string() in LibJS, where the name is dictated by the ToString AO.
2022-12-06 01:12:49 +00:00
return builder.to_deprecated_string();
AK: Implement more conforming URL percent encode/decode mechanism This adds a few new functions to percent encode/decode strings according to the URL specification. The functions allow specifying a PercentEncodeSet, which is defined by the specification. It will be used to replace the current urlencode() and urldecode() functions in a further commit. This commit adds a few duplicate helper functions in the URL class, such as is_digit() and is_ascii_digit(). This will be cleaned up as soon as the upcoming new URL parser will replace the current one.
2021-05-25 11:50:03 +00:00
}
AK+Everywhere: Rename String to DeprecatedString We have a new, improved string type coming up in AK (OOM aware, no null state), and while it's going to use UTF-8, the name UTF8String is a mouthful - so let's free up the String name by renaming the existing class. Making the old one have an annoying name will hopefully also help with quick adoption :^)
2022-12-04 18:02:33 +00:00
DeprecatedString URL::percent_decode(StringView input)
AK: Implement more conforming URL percent encode/decode mechanism This adds a few new functions to percent encode/decode strings according to the URL specification. The functions allow specifying a PercentEncodeSet, which is defined by the specification. It will be used to replace the current urlencode() and urldecode() functions in a further commit. This commit adds a few duplicate helper functions in the URL class, such as is_digit() and is_ascii_digit(). This will be cleaned up as soon as the upcoming new URL parser will replace the current one.
2021-05-25 11:50:03 +00:00
{
if (!input.contains('%'))
return input;
StringBuilder builder;
Utf8View utf8_view(input);
for (auto it = utf8_view.begin(); !it.done(); ++it) {
if (*it != '%') {
builder.append_code_point(*it);
} else if (!is_ascii_hex_digit(it.peek(1).value_or(0)) || !is_ascii_hex_digit(it.peek(2).value_or(0))) {
builder.append_code_point(*it);
} else {
++it;
Everywhere: Replace ctype.h to avoid narrowing conversions This replaces ctype.h with CharacterType.h everywhere I could find issues with narrowing conversions. While using it will probably make sense almost everywhere in the future, the most critical places should have been addressed.
2021-06-01 19:18:08 +00:00
u8 byte = parse_ascii_hex_digit(*it) << 4;
AK: Implement more conforming URL percent encode/decode mechanism This adds a few new functions to percent encode/decode strings according to the URL specification. The functions allow specifying a PercentEncodeSet, which is defined by the specification. It will be used to replace the current urlencode() and urldecode() functions in a further commit. This commit adds a few duplicate helper functions in the URL class, such as is_digit() and is_ascii_digit(). This will be cleaned up as soon as the upcoming new URL parser will replace the current one.
2021-05-25 11:50:03 +00:00
++it;
Everywhere: Replace ctype.h to avoid narrowing conversions This replaces ctype.h with CharacterType.h everywhere I could find issues with narrowing conversions. While using it will probably make sense almost everywhere in the future, the most critical places should have been addressed.
2021-06-01 19:18:08 +00:00
byte += parse_ascii_hex_digit(*it);
AK: Implement more conforming URL percent encode/decode mechanism This adds a few new functions to percent encode/decode strings according to the URL specification. The functions allow specifying a PercentEncodeSet, which is defined by the specification. It will be used to replace the current urlencode() and urldecode() functions in a further commit. This commit adds a few duplicate helper functions in the URL class, such as is_digit() and is_ascii_digit(). This will be cleaned up as soon as the upcoming new URL parser will replace the current one.
2021-05-25 11:50:03 +00:00
builder.append(byte);
}
}
Everywhere: Rename to_{string => deprecated_string}() where applicable This will make it easier to support both string types at the same time while we convert code, and tracking down remaining uses. One big exception is Value::to_string() in LibJS, where the name is dictated by the ToString AO.
2022-12-06 01:12:49 +00:00
return builder.to_deprecated_string();
AK: Implement more conforming URL percent encode/decode mechanism This adds a few new functions to percent encode/decode strings according to the URL specification. The functions allow specifying a PercentEncodeSet, which is defined by the specification. It will be used to replace the current urlencode() and urldecode() functions in a further commit. This commit adds a few duplicate helper functions in the URL class, such as is_digit() and is_ascii_digit(). This will be cleaned up as soon as the upcoming new URL parser will replace the current one.
2021-05-25 11:50:03 +00:00
}
AK: Add a basic URL class to help us handle URL's We're gonna need these as we start to write more networking programs.
2019-08-10 15:27:56 +00:00
}
Reference in a new issue Copy permalink