Commit graph

1919 commits

Author SHA1 Message Date
Zaggy1024 074f771b59 LibVideo: Add VideoFrame class for decoded video frames
The class is virtual and has one subclass, SubsampledYUVFrame, which
is used by the VP9 decoder to return a single frame. The
output_to_bitmap(Bitmap&) function can be used to set pixels on an
existing bitmap of the correct size to the RGB values that
should be displayed. The to_bitmap() function will allocate a new bitmap
and fill it using output_to_bitmap.

This new class also implements bilinear scaling of the subsampled U and
V planes so that subsampled videos' colors will appear smoother.
2022-10-31 14:47:13 +01:00
Linus Groh 4db85493e8 AK+Meta: Add WEB_FETCH_DEBUG macro 2022-10-30 20:10:29 +00:00
Jan200101 4b305e8fe4 Meta: Convert build directory path to be completely absolute 2022-10-27 20:43:55 +01:00
Timothy Flynn bd592480e4 Meta: Replace Bash script for generating emoji.txt with C++ generator
We currently have two build-time parsers for the UCD's emoji-test.txt
file. To prepare for future changes, this removes the Bash parser and
moves its functionality to the newer C++ parser.
2022-10-27 12:59:56 +02:00
Timothy Flynn b5a876e606 Meta: Allow specifying additional dependencies with invoke_generator 2022-10-27 12:59:56 +02:00
Timothy Flynn 006118599c Meta: Disallow emoji images with invalid code points in their file name 2022-10-26 18:40:20 +01:00
Linus Groh bc2ebcadc0 Everywhere: Require version >= 12 for GCC host compiler
So far we've gotten away with using GCC 11 for Lagom and to compile the
toolchain, but via #15795 we discovered a compiler bug that has been
fixed in the latest version but would error the build with CI's GCC 11.
Time for an upgrade :^)

We already use ubuntu-22.04 images in most places, so this is pretty
straightforward. The only exception is Idan's self-hosted runner, which
uses Ubuntu Focal. LibJS should build fine with GCC 11, still.
2022-10-25 23:15:51 +01:00
Timothy Flynn b077fccd3d LibLocale+LibJS: Update to CLDR version 42.0.0
There were some notable changes to the CLDR JSON format and data in this
release.

The patterns for a date at a specific time, i.e. "{date} at {time}", now
appear under the "atTime" attribute of the "dateTimeFormats" object.

Locale specific changes that affected test-js:

All locales:

* In many patterns, the code points U+00A0 (NO-BREAK SPACE) and U+202F
  (NARROW NO-BREAK SPACE) are now used in place of an ASCII space. For
  example, before the "dayPeriod" fields AM and PM.

* Separators such as U+2013 (EN DASH) are now surrounded by U+2009 (THIN
  SPACE) in place of an ASCII space character.

Locale "en":

* Narrow localizations of time formats are even more narrow. For
  example, the abbreviation "wk." for "week" is now just "wk".

Locale "ar":

* The code point U+060C (ARABIC COMMA) is now used in place of an ASCII
  comma.

* The code point U+200F (RIGHT-TO-LEFT MARK) now appears at the
  beginning of many localizations.

* When the "latn" numbering system is used for currency formatting, the
  currency symbol more consistently is placed at the end of the pattern.

Locale "he":

* The "many" plural rules category has been removed.

Locales "zh" and "es-419":

* Several display-name localizations were changed.
2022-10-25 10:10:39 +01:00
demostanis 3e8b5ac920 AK+Everywhere: Turn bool keep_empty to an enum in split* functions 2022-10-24 23:29:18 +01:00
Tim Schumacher 4ee0737d10 Meta: Default to the SDL QEMU frontend on SerenityOS
We don't yet have anything else than SDL, and certainly not GTK.
2022-10-24 15:52:42 +02:00
Tim Schumacher 40165a1c40 Meta: Also set CC/CXX when selecting the host toolchain
This ensures that the toolchain building scripts will use the host
toolchain that has been found manually, and that they won't fall back to
`cc` (which in the worst case may not even be installed).
2022-10-24 15:52:42 +02:00
Tim Schumacher 7ecc50db57 Lagom: Link the system's LibSystem on SerenityOS 2022-10-24 15:52:42 +02:00
Daniel Bertalan f1f6c4c0b6 Meta: Detect Homebrew clang-format
Homebrew does not add upstream LLVM's install location to $PATH so as
not to conflict with XCode tools, so we need to run `brew --prefix llvm`
to figure out its install path.
2022-10-22 12:53:03 +02:00
Linus Groh 3b2e5b08d9 LibWeb: Remove dbgln FIXME spam from BindingsGenerator 2022-10-20 22:34:55 +02:00
Kenneth Myhra ebd93c8d57 LibWeb: Revert support for DOM::Document in XHR::send()
This is a manual revert of commit: 7831e62

Let's revert this until we got nested union support in our IDL
generator/parser.
2022-10-20 17:57:52 +02:00
Andreas Kling 940b455ed0 LibWeb: Teach IDL generator to go from WindowProxy to EventTarget
This case requires some special handling, since there's no direct C++
inheritance relationship between HTML::WindowProxy and DOM::EventTarget.
2022-10-20 15:52:20 +02:00
Andreas Kling 68452c749a LibWeb: Prevent GC from running during intrinsics allocation
Due to the way we lazily construct prototypes and constructors for web
platform interfaces, it's possible for nested GC allocation to occur
while GC objects have been allocated but not fully constructed.

If the garbage collector ends up running in this state, it may attempt
to call JS::Cell::visit_edges() on an object whose vtable pointer hasn't
been set up yet.

This patch works around the issue by deferring GC while intrinsics are
being brought up. Furthermore, we also create a dummy global object for
the internal realm, and populate it with intrinsics. This works around
the same issue happening when allocating something (like the default UA
stylesheets) in the internal realm.

These solutions are pretty hacky and sad, so I've left FIXMEs about
finding a nicer way.
2022-10-20 15:16:23 +02:00
Timothy Flynn 27737f613c LibTimeZone+LibJS: Update to TZDB version 2022e
https://mm.icann.org/pipermail/tz-announce/2022-October/000074.html

This version changes America/Chicago's transtion from LMT to CST from
1883 Nov 18 12:09:24 to 1883 Nov 18 18:00.
2022-10-18 16:01:44 +02:00
Andrew Kaster 5e2fe7e2bf Lagom: Enable compile_commands.json for the lagom build directory
This allows a developer who only wants to look at Lagom to have their
IDE find source files and headers properly.
2022-10-17 15:55:55 +02:00
Andrew Kaster b8e51425e9 Lagom+CMake: Propagate dependencies for generated custom targets
We have logic for serenity_generated_sources which works well for source
files that are specified in GENERATED_SOURCES prior to calling
serenity_lib or serenity_bin. However, code generated with
invoke_generator, and the LibWeb generators do not always follow the
pattern of the IDL and GML files.

For the LibWeb generators, we can just add_dependencies to LibWeb at the
time we declare the generate_Foo custom target. However for LibLocale,
LibTimeZone, and LibUnicode, we don't have the name of the target
available, so export the name in a variable to set into
GENERATED_SOURCES.

To make this work for Lagom, we need to make sure that lagom_lib and
serenity_bin in Lagom/CMakeLists.txt call serenity_generated_sources on
the target.

This enables the Xcode generator on macOS hosts, at least for Lagom.
2022-10-17 15:55:55 +02:00
Andrew Kaster 8160b53228 Lagom: Make lagom_lib take names prefixed with Lib
This matches serenity_lib, and consolidates the logic to strip Lib from
the front of the library name for the Lagom export name into one place
at the top of lagom_lib.
2022-10-17 15:55:55 +02:00
Tim Schumacher 2a4f81fc83 Meta: Start moving common shell definitions into a common file 2022-10-16 23:39:45 +02:00
Tim Schumacher 9d6c4c5137 Meta: Allow shellcheck to search sourced files in the script directory 2022-10-16 23:39:45 +02:00
Tim Schumacher 032ef1b461 Meta: Define SUDO for all the image building scripts
Back when adding support for `pls` as a `sudo` replacement,
`build-image-qemu` dynamically set the `SUDO` variable depending on what
system it was running on and used the contents of that variable to call
the correct elevation utility.

During recent changes to the elevation error message, that usage of the
variable was replicated across all of our scripts, but without also
replicating the logic to set that variable in the first place.

Add back the variable setting logic to all the other scripts to keep
them from running into unset variables.
2022-10-16 23:39:45 +02:00
Timothy Flynn 9fad23018a Meta: Remove unused "prefix" variable from invoke_generator() helper
This became unused after 1ae0cfd.
2022-10-16 21:16:48 +02:00
demostanis 1c979889b2 Meta: Print error if cmake does not exist 2022-10-16 17:49:18 +02:00
Idan Horowitz 9f3de0be6a Fuzzers: Add fuzzer for the Tar parser 2022-10-16 18:34:48 +03:00
Andrew Kaster aacc49bab5 Lagom: Remove unecessary _lagom suffix from test and executable names
We needed this back when we included Lagom into the main serenity build,
but it has been unecessary since the superbuild.
2022-10-16 16:36:39 +02:00
Andrew Kaster 1ae0cfd08b CMake+Userland: Use CMakeLists from Userland to build Lagom Libraries
Also do this for Shell.

This greatly simplifies the CMakeLists in Lagom, replacing many glob
patterns with a big list of libraries. There are still a few special
libraries that need some help to conform to the pattern, like LibELF and
LibWebView.

It also lets us remove essentially all of the Serenity or Lagom binary
directory detection logic from code generators, as now both projects
directories enter the generator logic from the same place.
2022-10-16 16:36:39 +02:00
Andrew Kaster a01c0e81f3 Meta: Use add_serenity_directory for LibTimeZone
This lets us eliminate the "meta prefix" for the LibTimeZone generators.
2022-10-16 16:36:39 +02:00
Andrew Kaster f7f92f104f Lagom+LibCore: Build LibCore, AK, and LibMain with add_subdirectory
By deferring to the CMakeLists in each of these libraries' directories,
we can get rid of a lot of curious GLOB patterns and list removals in
the Lagom CMakeLists.
2022-10-16 16:36:39 +02:00
Kenneth Myhra 7831e62955 LibWeb: Add support for DOM::Document to XHR::send()
This patch adds support for posting a DOM::Document using XHR::send().
2022-10-16 02:06:29 +03:30
Gunnar Beutner f70ec7582f Meta: Properly ignore targets which don't have a component name
Previously we'd add [component_name-NOTFOUND] sections to the
components.ini file for targets which weren't part of a component.
2022-10-15 13:12:42 +02:00
Undefine 135ca3fa1b Kernel: Add support for the FAT32 filesystem
This commit adds read-only support for the FAT32 filesystem. It also
includes support for long file names.
2022-10-14 18:36:40 -06:00
demostanis 48aea321c5 Meta: Don't tell to run script as root when that's not the actual error
Before, and that was the cause of many confusion in #build-problems
on Discord, the Meta/build-image-*.sh scripts would error out with
the message "this script needs to run as root" while the actual error
was unrelated.
2022-10-15 01:40:01 +02:00
Lucas CHOLLET c837a1a8de LibAudio: Factorize stream initialisation to base class LoaderPlugin
All actual plugins follow the same logic to initialize their stream,
this commit factorizes all of this to their base class: `LoaderPlugin`.
2022-10-14 23:11:23 +02:00
kleines Filmröllchen c439a34ff7 Meta: Get rid of /res/version.ini
There are no more users of this legacy version API. :yakgone:
2022-10-14 13:45:33 +02:00
ne0ndrag0n b4456ecdbb Escalator: Add new method to privilege escalate within GUI 2022-10-14 11:23:06 +02:00
Andrew Kaster b704f82487 Fuzzers: Add VP9Decoder and MatroskaReader fuzzers for LibVideo 2022-10-13 11:25:03 +02:00
Sam Atkins 80603f141a WebDriver: Add new WebDriver service
WebDriver aims to implement the WebDriver specification found at
https://w3c.github.io/webdriver/webdriver-spec.html . It's an HTTP
server that can create Browser sessions and control them.

Co-authored-by: Florent Castelli <florent.castelli@gmail.com>
2022-10-12 23:07:42 +02:00
Andrew Kaster 539fb08551 Userland: Remove unecessary uses of __serenity__ macro 2022-10-10 12:23:12 +02:00
Andrew Kaster 896d4e8dc1 LibRegex: Don't build LibRegex/C/Regex.cpp on Lagom
This file implements the POSIX APIs from <regex.h>, and is not suitable
for inclusion in a Lagom build. If we do include it, it will override
the host's regex functions and wreak havoc if it's resolved before the
host's implementation.
2022-10-10 12:23:12 +02:00
Zaggy1024 bbc053fa56 LibVideo: Add LibVideo test to Lagom 2022-10-09 20:32:40 -06:00
Andrew Kaster 2d5bee256e LibWeb: Re-implement HTML::Navigator using IDL
Get rid of the bespoke NavigatorObject class and use the modern IDL
strategies for creating platform objects to re-implement Navigator and
its associcated mixin interfaces. While we're here, implement it in a
way that brings WorkerNavigator up to spec :^)
2022-10-09 10:14:57 +02:00
Andrew Kaster 2acff8d853 LibWeb: Add new code generator for creating exposed interface helpers
This new code generator takes all the .idl files in LibWeb, looks for
each top level interface in there with an [Exposed=Foo] attribute, and
adds code to add the constructor and prototype for each of those exposed
interfaces to the realm of the relevant global object we're initialzing.

It will soon replace WindowObjectHelper as the way that web interfaces
are added to the Window object, and will be used in the future for
creating proper WorkerGlobalScope objects for dedicated and shared
workers.
2022-10-09 10:14:57 +02:00
Andrew Kaster 067a53b7e7 LibIDL: Remove static maps for interfaces and resolved imports
Instead, create a tree of Parsers all pointing to a top-level Parser.

All module imports and interfaces are stored at the top level, instead
of in a static map. This allows creating multiple IDL::Parsers in the
same process without them stepping on each others toes.
2022-10-09 10:14:57 +02:00
Timothy Flynn f08a979b96 LibUnicode: Remove GCC codegen workaround
Reverts commits:
ffbf5596cd
f190e394b3
2022-10-07 18:21:40 +01:00
Timothy Flynn f38c68177b LibUnicode: Update code point ideographic replacements for Unicode 15 2022-10-07 18:17:40 +01:00
Sam Atkins 0823a3c422 BindingsGenerator+LibWeb: Pass a VM to static IDL-based functions
This saves us from having to yoink the VM out of thin air.
2022-10-06 21:45:07 +01:00
Sam Atkins a6f0508f9f BindingsGenerator+LibIDL: Parse "inherit" attributes
An "inherit attribute" calls an ancestor's getter with the same name,
but defines its own setter. Since a parent class's public methods are
exposed to child classes, we don't have to do any special handling here
to call the parent's methods, it just works. :^)
2022-10-06 21:45:07 +01:00
Andreas Kling f190e394b3 LibUnicode: Let's use the GCC 11/12 workaround on all platforms
I seem to be getting some miscompiles on Linux as well, so let's make
the hitherto macOS-specific workaround universal.
2022-10-06 17:15:28 +02:00
Andreas Kling a4ccdce011 LibWeb: Generate WebContent IPC endpoints for Lagom's LibWebView
This is pretty hackish but should get us back to building.
2022-10-06 16:35:53 +02:00
matcool 70d0c1616f LibUnicode: Add decomposition mappings and Unicode normalization
The mappings are exposed via `Unicode::code_point_decomposition(u32)`
and `Unicode::code_point_decompositions()`, the latter being useful for
reverse searching a code point from its decomposition.

The normalization code does not make use of `Quick_Check` props (https://www.unicode.org/reports/tr44/#Decompositions_and_Normalization),
meaning no quick check optimizations.
2022-10-06 08:24:39 -04:00
Timothy Flynn e8410bc2ee Meta: Hide WebView behind ENABLE_LAGOM_LIBWEB CMake option
If ENABLE_LAGOM_LIBWEB is OFF, this will fail to build.
2022-10-06 14:12:54 +02:00
Andreas Kling 2ac385f4db CMake: Don't mess with absolute compile_ipc() source paths
If given an absolute path, compile_ipc() should just use it verbatim
instead of trying to be helpful.
2022-10-06 11:23:53 +02:00
Andreas Kling 4d762fd3a6 Lagom: Add Build WebContentClient as part of Lagom's LibWebView
This will allow us to use it in Ladybird. :^)
2022-10-06 09:51:04 +02:00
Nico Weber 2af028132a AK+Everywhere: Add AK_COMPILER_{GCC,CLANG} and use them most places
Doesn't use them in libc headers so that those don't have to pull in
AK/Platform.h.

AK_COMPILER_GCC is set _only_ for gcc, not for clang too. (__GNUC__ is
defined in clang builds as well.) Using AK_COMPILER_GCC simplifies
things some.

AK_COMPILER_CLANG isn't as much of a win, other than that it's
consistent with AK_COMPILER_GCC.
2022-10-04 23:35:07 +01:00
Andrew Kaster 636602a54e LibWeb: Implement <input type=file> behavior
This includes punting on the actual file picker implementation all the
way out to the PageClient. It's likely that some of the real details
should be implemented somewhere closer, like the BrowsingContext or the
Page, but we'll get there.

For now, this allows https://copy.sh/v86 to load the emulation of the
preselected images all the way until it hits a call to
URL.createObjectURL.
2022-10-04 22:05:14 +02:00
Andreas Kling 3ca44e2258 BindingsGenerator: Allow the word "inline" in IDL identifiers
This has to be treated carefully since it's a C++ keyword.
2022-10-04 21:30:58 +02:00
vl-ms d8b6f38748 Lagom: Change dl to ${CMAKE_DL_LIBS} for portability 2022-10-03 20:36:04 +01:00
Nico Weber ffbf5596cd Lagom: Work around gcc codegen bug
Without this, GenerateUnicodeData crashes when run during the build.
With this, `serenity.sh run` brings up a running SerenityOS.
Since GenerateUnicodeData doesn't take a lot of time to run, just
disable optimizations to work around the problem for now.

Works around #15449.
2022-10-03 15:30:51 +01:00
Liav A 74018be739 Meta: Make x86-64 target the default
This is a preparation to check if our users find noticeable bugs in the
x86-64 target, before we can decide if we want to remove the i686 target
for good.
2022-10-03 11:14:53 +02:00
Andrew Kaster 376425639d Meta: Force -machine pc-i440fx-7.0 when using QEMU >= 7.1
Until the root cause behind #14952 is found, this workaround should stem
the tide of users in discord asking why their system crashes on startup.
2022-10-03 09:57:45 +01:00
Andrew Kaster 56b381aac0 LibWeb: Cleanup unecessary uses and includes of HTML::Window
The big global refactor left some stragglers behind for atomicity.

Clean up the rest, and remove a ton of includes of LibWeb/HTML/Window.h
2022-10-01 21:05:32 +01:00
Andrew Kaster cc164dc1e2 Fuzzers: Convert FuzzCSSParser to use the MainThreadVM
Instead of trying to create a Window and a Document, and use those to
create a ParsingContext, just use the JS::Realm only constructor to make
sure that bindings are stashed on the main thread VM's realm.
2022-10-01 21:05:32 +01:00
Andrew Kaster 8407bf60c5 LibWeb: Introduce new construct_impl method for bound classes
In this generator change, we introduce a new factory method for bound
LibWeb objects that takes a JS::Realm instead of Web::HTML::Window.

The two methods are allowed to co-exist at this point, but the option to
take an HTML::Window will be removed once all clases are converted to
the new API.

We also start using the new Bindings::ensure_web_[prototype/constructor]
helpers from the Bindings/Intrinsics class so that we can eventually
remove the helpers from Window.h for the same.
2022-10-01 21:05:32 +01:00
Daniel Bertalan 41a8f3f40f Lagom: Link LibMain statically
This matches how LibMain is used within Serenity. This commit makes it
possible to build Lagom with LTO. Previously, `serenity_main` functions
would be dead-stripped away, as the linker could prove that nothing from
the executable ever called them.
2022-10-01 20:39:48 +02:00
Tim Schumacher 1faca15193 Toolchain: Move the binutils patch into its own folder
This makes binutils compatible with `./package.sh dev`, so that we can
regenerate patches more easily, and neatly stack (temporary) patches on
top.
2022-10-01 18:43:29 +01:00
Diego Iastrubni 18257604eb Lagom: Win32 support baby steps
This is the initial port of Lagom to win32. This will enable developers
to use Lagom as an alternative to vanilla STL/StandardC++Library - which
gives a much richer environment (think QtCore - but modern).

My main incentive - is to have a native Windows Ladybird working.

I am starting with AK, which does not yet fully compile (on mingw). When
AK is compiling (currently fails building StringBuffer.cpp) - I will
continue to LibCore and then the rest of the user space libraries
(excluding the GUI, which will be another different effort).

Most of the code is happily stollen from Andrew Kaster's fork - he
deserves the credit.

Co-authored-by: Andrew Kaster <akaster@serenityos.org>
2022-09-29 17:01:22 +01:00
Timothy Flynn 5f9d8f25c6 AK+Meta: Define a debug flag for LibTimeZone 2022-09-28 23:52:51 +01:00
Timothy Flynn 9a1b24d5be LibTimeZone: Use the last DST rule in the TZDB if a match isn't found
Some time zones, like "Asia/Shanghai", use a set of DST rules that end
before present day. In these cases, we should fall back to last possible
RULE entry from the TZDB. The time zone compiler published by IANA (zic)
performs the same fallback starting with version 2 of the time zone file
format.
2022-09-28 23:52:51 +01:00
Timothy Flynn 7178c31f1c LibTimeZone: Update to TZDB version 2022d
https://mm.icann.org/pipermail/tz-announce/2022-September/000073.html
2022-09-27 15:16:09 +01:00
Linus Groh 9fb672e981 LibWeb: Implement '5.4. Request class' from the Fetch API :^) 2022-09-27 14:56:17 +01:00
Linus Groh b9d626d6b2 LibWeb: Don't always assign values to nullable IDL dictionary members
IDL dictionary members are nullable by default (unless marked as
`required`) and should not get any value assigned unless one was
provided by the userland code that isn't undefined, or if the member has
a default value.

This is so that we can use Optional<T> in the internal representation
and check for "is present" via Optional::has_value().
2022-09-27 14:56:17 +01:00
Linus Groh d38f1fb5b2 LibWeb: Fix bindings codegen for IDL enums with only one member
The SourceGenerator's @else@ mapping is only set in the second iteration
of the loop, causing the generated return for unrecognized values to not
be guarded by an else statement.
We can simply use a hardcoded 'else' here, @else@ is only to create the
first comparison as a plain 'if' and subsequent ones as 'else if'.
2022-09-27 14:56:17 +01:00
Timothy Flynn eb0657c23e LibWeb: Add WebIDL to the IDL generator's list of web namespaces
Without this, the generated DOMExceptionConstructor does not refer to
the WebIDL::DOMException with its fully qualified name. This caused an
ambiguity error on my machine.
2022-09-26 14:00:31 +01:00
Linus Groh bbaa05fcf9 LibWeb: Move DOMException from DOM/ to WebIDL/ 2022-09-25 19:13:31 +01:00
Andreas Kling 34c232def7 Lagom: Build DOMTreeModel from LibWebView
This will be used in Ladybird.
2022-09-25 12:10:28 +02:00
Andreas Kling 5778da765d Lagom: Build a tiny slice of LibGUI
We now build Model, ModelIndex and Icon from LibGUI. These will be used
in Ladybird to view the DOM tree model.
2022-09-25 12:09:36 +02:00
Linus Groh 2b2ccb3815 LibWeb: Replace 'heycam.github.io' links with 'webidl.spec.whatwg.org' 2022-09-24 19:32:09 +01:00
Linus Groh 86f68eb3c3 LibWeb: Move IDLOverloadResolution from Bindings/ to WebIDL/ 2022-09-24 19:31:39 +01:00
Linus Groh 4f73851afc LibWeb: Move CallbackType from Bindings/ to WebIDL/
Let's stop putting generic types and AOs from the Web IDL spec into
the Bindings namespace and directory in LibWeb, and instead follow our
usual naming rules of 'directory = namespace = spec name'. The IDL
namespace is already used by LibIDL, so Web::WebIDL seems like a good
choice.
2022-09-24 19:31:39 +01:00
kleines Filmröllchen a5b72577e6 Meta: Enable full region dumping on CI 2022-09-24 14:22:09 +02:00
kleines Filmröllchen 1a7d6508e3 Kernel: By default, don't dump regions when a userspace crash happens
There is the DUMP_REGIONS_ON_CRASH debug macro which re-enables this
(old) behavior.
2022-09-24 14:22:09 +02:00
Andreas Kling 131c3f50de LibJS: Add JS::SafeFunction, like Function but protects captures from GC
SafeFunction automatically registers its closure memory area in a place
where the JS garbage collector can find it.

This means that you can capture JS::Value and arbitrary pointers into
the GC heap in closures, as long as you're using a SafeFunction, and the
GC will not zap those values!

There's probably some performance impact from this, and there's a lot of
things that could be nicer/smarter about it, but let's build something
that ensures safety first, and we can worry about performance later. :^)
2022-09-24 12:23:29 +02:00
Andrew Kaster d6a741af6c Meta: Don't build Core::LocalServer on Android
We won't be able to use local servers on Android without some serious
Android work to create background tasks, so just disable this for now,
as it currently relies on Core::Account to take over from SystemServer.
2022-09-22 11:07:21 -04:00
Andrew Kaster cca54e9535 CI: Remove Android NDK Cache
This actually never worked as a cache, the build has failed since it
tried to use it. This should restore the Android nightly build.
2022-09-22 11:07:21 -04:00
Ben Wiederhake 48062b3cca Meta: Remove 'time' invocation in lint-ci
Some systems don't have /usr/bin/time available, and during most runs
of lint-ci we don't actually care that much about the exact timing.
Therefore, let's just remove it. It's easy enough to add back in, if
someone wants to investigate an issue.
2022-09-22 00:45:10 +02:00
Linus Groh edfef8e2f5 Everywhere: Rename WrapperGenerator to BindingsGenerator
This code generator no longer creates JS wrappers for platform objects
in the old sense, instead they're JS objects internally themselves.
Most of what we generate now are prototypes - which can be seen as
bindings for the internal C++ methods implementing getters, setters, and
methods - as well as object constructors, i.e. bindings for the internal
create_with_global_object() method.

Also tweak the naming of various CMake glue code existing around this.
2022-09-21 23:06:08 +01:00
Linus Groh 6055b0e850 LibWeb: Remove no-op impl() methods from the WEB_PLATFORM_OBJECT macro
These are leftovers from when wrapper objects still had an internal
implementation, which is no longer the case.
2022-09-21 21:12:24 +01:00
Linus Groh 2cab2a8e8f LibWeb: Turn is_platform_object() ifs into an Array 2022-09-21 21:12:24 +01:00
Linus Groh 464368f94a LibWeb: Rename is_wrappable_type() to is_platform_object()
This name more accurately reflects what we are checking. Also add an
explanatory note that only a hand-curated subset of platform object
types is checked in the absence of a full generated list.
2022-09-21 21:12:24 +01:00
Linus Groh daf0a8dd91 LibWeb: Remove impl_is_wrapper() check from WrapperGenerator
This concept no longer applies to GC'd platform objects. The only place
where this was still used can use is_wrappable_type() instead.
2022-09-21 21:12:24 +01:00
Linus Groh 64a1992040 LibWeb: Remove support for RefPtr types in idl_type_name_to_cpp_type()
This is not used anymore.
2022-09-21 21:12:24 +01:00
Linus Groh 38b3ce8f03 LibIDL+LibWeb: Remove stale references to "wrapper class" concept
There are no wrappers for the platform object types anymore :^)
2022-09-21 21:12:24 +01:00
Timothy Flynn 51854e345a LibUnicode: Update to Unicode version 15.0.0
https://unicode.org/versions/Unicode15.0.0/
2022-09-21 14:04:22 +01:00
Timothy Flynn 739798e075 LibUnicode: Use recently added Core::Stream::read_all in code generators
The generators had a manual implementation when Core::Stream did not
have a read_all method.
2022-09-21 14:04:22 +01:00
Liav A a40f289ee5 Meta: Add isa-vga device to ISA-PC and microvm machine types
With this device being added, we can now boot into graphics mode on
these platforms too. For ISA-PC machine this is basically the only
viable option to use, but in the future, we should remove this device
for the microvm machine type as it should allow us to determine better
options and detect them by using a given device tree blob.
2022-09-20 19:05:13 +01:00
Federico Guerinoni 4f46a0e4aa StateMachineGenerator: Remove unused variable
This fixes build on MacOS m1 arch.
2022-09-20 18:54:16 +01:00
Sam Atkins 684c3efc04 StateMachineGenerator: Port to LibMain and Core::Stream 2022-09-20 07:48:45 -04:00
Sam Atkins cb45fa79ee LibEDID: Use Core::Stream API to generate PnpIDs 2022-09-20 07:48:45 -04:00
Sam Atkins 32810f6d1a IPCCompiler: Use Core::Stream API 2022-09-20 07:48:45 -04:00
Ben Wiederhake 230226028a Meta: Use time instead of /usr/bin/time in lint-ci.sh
I totally overlooked that /usr/bin/time is not universal, which broke
some systems. This commit instead calls 'time', allowing either a shell
built-in to kick in, or a (potentially different) binary be found
anywhere in the PATH.
2022-09-19 22:18:25 +04:30
Ben Wiederhake 761f079a22 Meta: Fix example invocations of the serenity.sh run command 2022-09-18 18:47:34 -07:00
Ben Wiederhake f8a42ef0b3 Meta: Only check changed files in check-style.py during pre-commit
This speeds up the script from about 90ms down to about 10ms, for
reasonably common changesets.

80ms may not feel like much, but it adds up quickly, especially since
we run a dozen scripts during pre-commit.
2022-09-18 18:45:25 -07:00
Ben Wiederhake 532786848b Meta: Only check changed files in lint-gml-format.sh
This speeds up the script from about 140ms down to <10ms, even for
changesets that touch a handful of different GML files.

130ms may not feel like much, but it adds up quickly, especially since
we run a dozen scripts during pre-commit.
2022-09-18 18:45:25 -07:00
Ben Wiederhake de581ca5fd Meta: Only check changed files in check-newlines-at-eof
This speeds up the script from about 120ms down to about 20ms for
reasonably common changesets.

100ms may not feel like much, but it adds up quickly, especially since
we run a dozen scripts during pre-commit.
2022-09-18 18:45:25 -07:00
Ben Wiederhake 4caaa78baf Meta: Only check changed files in check-debug-flags during pre-commit
This speeds up the script from about 170ms down to about 80ms for
changes in Debug.h.in or similarly "DEBUG"-rich files, down to <10ms for
more common changesets.

160ms may not feel like much, but it adds up quickly, especially since
we run a dozen scripts during pre-commit.
2022-09-18 18:45:25 -07:00
Ben Wiederhake 88c8ad840d Meta: Rewrite pre-commit script check-emoji in python
This reduces the scripts execution time from 0.57 seconds to 0.01
seconds, while also making the check a bit tighter, leaving fewer
possible problems.
2022-09-18 18:45:25 -07:00
Ben Wiederhake c4895b7d95 Meta: Always show the timing of commands in lint-ci.sh
This should make it easier to identify slow-running scripts.
While we're at it, unify the output a little bit.
2022-09-18 18:45:25 -07:00
Ben Wiederhake 30cca01e24 HeaderCheck: Also check AK for broken headers 2022-09-18 13:27:24 -04:00
Ben Wiederhake 8a1e406615 HeaderCheck: Disable warning due to LibSoftGPU
Once LibSoftGPU drops this disable, HeaderCheck should remove it, too.
Until then, it is necessary to build at all.
2022-09-18 13:27:24 -04:00
Ben Wiederhake 2f7c9915b5 HeaderCheck: Remove outdated debug code
This should never have been committed.
2022-09-18 13:27:24 -04:00
Andreas Kling 9743849ddb LibWeb: Allow passing null to a nullable IDL union type
This fixes an issue on Twitter where they were instantiating an
IntersectionObserver with a null root. The root IDL type is
`(Element or Document)?` so null needs to be allowed.
2022-09-18 12:53:06 +02:00
Andreas Kling 530675993b LibWeb: Rename Attribute to Attr
This name is not very good, but it's what the specification calls it.
2022-09-18 02:08:01 +02:00
Timothy Flynn 302b6e966f LibTimeZone: Generate DST rule timestamps at compile time
Rather than invoking AK::Time::from_timestamp at runtime, we can do so
at compile time. This reduces invoking TimeZone::get_time_zone_offset
100,000 times in a loop from about 7 seconds to 30 milliseconds.
2022-09-17 23:21:32 +02:00
Sam Atkins 07a0d9df30 WrapperGenerator: Use the new shiny Overload Resolution Algorithm :^)
We compute the effective overload sets for each argument count at build
time, to save us having to do it every time a function with overloads
is called.
2022-09-17 21:27:17 +02:00
Sam Atkins 8b4cc07a54 LibIDL+WrapperGenerator: Implement Type::is_distinguishable_from()
As part of this, I've moved a couple of methods for checking for
null/undefined from UnionType to Type, and filled in more of their
steps.

This now detects more, and so causes us to hit a `TODO()` which is too
big for me to go after right now, so I've replaced that assertion with
a log message.
2022-09-17 21:27:17 +02:00
Sam Atkins 7c8ef79898 LibIDL+WrapperGenerator: Make it easier to work with IDL::Type classes
Track the kind of Type it is, and use that to provide some convenient
`is_foo()` / `as_foo()` methods. While I was at it, made these all
classes instead of structs and made their data private.
2022-09-17 21:27:17 +02:00
Sam Atkins c4668053d1 LibIDL+WrapperGenerator: Move IDL code into a library
IDL function overload resolution requires knowing each IDL function's
parameters and their types at runtime. The simplest way to do that is
just to make the types the generator uses available to the runtime.

Parsing has moved to LibIDL, but code generation has not, since that is
very specific to WrapperGenerator.
2022-09-17 21:27:17 +02:00
Sam Atkins 634a52b589 WrapperGenerator: Clarify function-length getter name
These all return the shortest length of the function, so let's name them
as such.
2022-09-17 21:27:17 +02:00
Sam Atkins 4424a50bc4 WrapperGenerator: Move generation code out of IDLTypes.h 2022-09-17 21:27:17 +02:00
Tim Schumacher a4f5a5d783 Meta: Remove the obsolete linter check for LibM/math.h 2022-09-16 16:09:19 +00:00
Andrew Kaster b04de3090f Lagom: Skip IMPORTED targets in get_linked_lagom_libraries
This script is useful when wanting to install lagom libraries for
projects using Lagom via FetchContent, but trips over itself if the
project links other non-Lagom imported targets to itself. So, let's just
skip them.
2022-09-16 07:48:51 -04:00
MacDue d1b99282d8 LibWeb+Meta: Parse the backdrop-filter CSS property
Note: The parsing and style value completely ignores the SVG filters
part of the spec for now... That's a yak for another day :^)
2022-09-16 10:50:48 +01:00
MacDue 0829aa5df0 Meta: Use is_abstract_image() for the "image" CSS type 2022-09-16 10:50:48 +01:00
Tim Schumacher 671712cae6 CMake: Use -static-libstdc++ on Clang as well
This resolves an old FIXME that we added ages ago, and our LLVM driver
apparently gained support for `-static-libstdc++` somewhere along the
way.
2022-09-16 05:39:28 +00:00
Ben Wiederhake 42db468ef5 Meta: Revise lint-commit.sh regex for the title
This mainly changes two aspects:
- The category can now be a single letter, such as 'w' to indicate the
  file Utilities/w.cpp
- Spaces in the category (or list) are no longer allowed. This follows
  the lived practice of writing category lists as "Foo+Bar: Quux"

Closes #15243.
2022-09-15 13:55:32 +01:00
Timothy Flynn 6290a25224 Meta: Enable vptr sanitation by default for Lagom UBSAN builds 2022-09-15 09:45:13 +02:00
Timothy Flynn 3dd11a007f CI: Fix typo preventing the CLDR cache from being pulled 2022-09-14 08:21:03 -04:00
Brian Gianforcaro d0a1775369 Everywhere: Fix a variety of typos
Spelling fixes found by `codespell`.
2022-09-14 04:46:49 +00:00
networkException 40984d3355 Meta: Add confirmation for toolchain rebuilds if there's one already 2022-09-13 13:39:15 -04:00
Timothy Flynn 10727ade4c Meta: Add Lagom option to disable building LibWeb
Compiling LibWeb for Lagom can add quite a bit of compile time. Add a
CMake option to disable it locally.
2022-09-13 17:05:20 +02:00
Ben Wiederhake 005cd4e456 Meta: Remove unused and outdated lint-missing-resources.sh lint
This was apparently never used by anyone except me, and currently
fails silently.

The script originally allowed easy inspection of potentially missing
resources, but that seems no longer useful. Even after restoring the
script to a working state, I found nothing with it.

A somewhat usable version might be available at
https://github.com/BenWiederhake/serenity/tree/historic/lint-missing-resources.sh

However, there seems to be no interest in the script, so it is better to
remove it.
2022-09-13 08:29:09 +00:00
Ben Wiederhake 93b8e76133 Meta: Remove unused and outdated check-syscall-lists.sh lint
This was apparently never used by anyone except me, and currently
fails silently.

The script originally allowed easy inspection of the difference between:
1. The list of declared syscalls according to Kernel/API/Syscall.h
2. The list of syscalls implemented by the UserspaceEmulator according
   to Userland/DevTools/UserspaceEmulator/Emulator_syscalls.cpp
3. The list of syscalls documented in Base/usr/share/man/man2/.

Here's how the script could have been updated:

SYSCALLS_KERNEL="$(echo 'Kernel syscalls'; echo; \
    grep -Eo '^ +S\(.*, NeedsBigProcessLock::' Kernel/API/Syscall.h | \
    sed -Ee 's-^ +S\((.+), .*-\1-' | sort)"
SYSCALLS_UE="$(echo 'Implemented in UserspaceEmulator'; echo; \
    grep -Eo '^ +case SC_.*:$' \
    Userland/DevTools/UserspaceEmulator/Emulator_syscalls.cpp | \
    sed -Ee 's,^ +case SC_(.*):$,\1,' | sort)"
SYSCALLS_MAN2="$(echo 'Documented syscalls'; echo; \
    find Base/usr/share/man/man2/ ! -type d -printf '%f\n' | \
    sed -Ee 's,\.md,,' | sort)"
diff --color=always \
    <(echo "${SYSCALLS_KERNEL}") <(echo "${SYSCALLS_UE}")
diff --color=always \
    <(echo "${SYSCALLS_KERNEL}") <(echo "${SYSCALLS_MAN2}")

A more readable version might be available at
https://github.com/BenWiederhake/serenity/tree/historic/syscall-linting

However, there seems to be no interest in the script, so it is better to
remove it.
2022-09-13 08:29:09 +00:00
Ben Wiederhake 616b3dc718 Meta: Fix check for AK test inclusion
basename from GNU coreutils 8.32 (the default on Debian Bullseye, as
bash 5.2.0 does not include basename as a built-in command) does not
accept multiple arguments. This will cause the command to fail with no
output, and the error code not being propagated for some reason. This
means that the loop never gets executed, and thus the check never
actually does anything. This commit fixes that behavior by calling
'basename' multiple times.
2022-09-13 08:29:09 +00:00
Timothy Flynn be41b19146 Meta: Do not log CMake version on each CMake invocation
It is a tad verbose to print this each time CMake runs.
2022-09-11 21:40:15 +01:00
Timothy Flynn b7ef36aa36 LibUnicode: Parse and generate custom emoji added for SerenityOS
Parse emoji from emoji-serenity.txt to allow displaying their names and
grouping them together in the EmojiInputDialog.

This also adds an "Unknown" value to the EmojiGroup enum. This will be
useful for emoji that aren't found in the UCD, or for when UCD downloads
are disabled.
2022-09-11 20:33:57 +01:00
Timothy Flynn 0aadd4869d LibUnicode: Generate emoji data for non-fully-qualified emoji
This allows us to find emoji data for files such as /res/emoji/U+A9.png.
U+00A9 is not fully-qualified (its full form is U+00A9 U+FE0F). But the
UCD has unqualified data for this code point; generating it allows us to
categorize these emoji appropriately in the EmojiInputDialog.
2022-09-11 20:33:57 +01:00
davidot 7948897688 LibJS: Import the libjs-test262-runner from linusg/libjs-test262
For now this is a lagom only application as it is not compatible with
serenity in its current state.

The only change is that it is released under a different license with
permission from all the authors.
2022-09-11 20:25:51 +01:00
davidot d4736d17ae LibJS: Allow negative pointers in Value
Also ensure that all a nullptr input gives null object and you don't
accidentally dereference a nullptr.
2022-09-10 00:05:32 +01:00
Diego Iastrubni b66ceafaba Meta: Download WASM spec tests using CMake
If we have a new CMake version - download the files using CMake
API, instead of external tools.
2022-09-09 10:31:12 +01:00
Diego Iastrubni 8b30b69dac Meta: Use CMake functions to extract files
Newer cmake's have internal functions to un-compress files. These
functions will work on pure windows - as well as linux. This
eliminates the need to search for external tools (TAR,GZIP,ZIP) - and
helps fixing #9866.

In order to finally fix #9866 we need to decide to bump the cmake
version requirements and remove the checks. If we demand a newer cmake
version, we will loose Ubuntu 20.04 as a build target - as it ships
with CMake 3.16.

For now - we keep compatibility with CMake 3.16 - and only if CMake
3.18 as been found - we use its new functionality.
2022-09-09 10:31:12 +01:00
Andrew Kaster 5ab3fcf710 Meta: Update jakt build support for fully bootstrapped compiler
Remove the Corrosion dependency, and use the now-builtin
add_jakt_executable function from the Jakt install rules to build our
example application.

By using find_package(Jakt), we now have to set ENABLE_JAKT manually on
both serenity and Lagom at the same time, so the preferred method to do
this for now is:

    cmake -B Build/superbuild<arch><toolchain> \
          -S Meta/CMake/Superbuild \
          -DENABLE_JAKT=ON \
          -DJAKT_SOURCE_DIR=/path/to/jakt

Where omitting JAKT_SOURCE_DIR will still pull from the main branch of
SerenityOS/jakt. This can be done after runing Meta/serenity.sh run.
2022-09-09 11:23:42 +02:00
Timothy Flynn b61eca0a1e LibUncode: Parse and generate emoji code point data
According to TR #51, the "best definition of the full set [of emojis] is
in the emoji-test.txt file". This defines not only the emoji themselves,
but the order in which they should be displayed, and what "group" of
emojis they belong to.
2022-09-08 23:12:31 +01:00
Andreas Kling ddc018fb75 LibWeb: Use fully qualified type names in the IDL generator more
There are still some remaining cases where generated code depends on the
existence of FooWrapper => Web::NS::Foo mappings. Fixing those will
require figuring out the appropriate namespace for all IDL types, not
just the currently parsed interface.
2022-09-06 00:27:09 +02:00
Andreas Kling 8f110e0fb1 LibWeb: Remove the NoInstanceWrapper extended IDL attribute
No interfaces require wrappers anymore, so we can just make this the
default mode.
2022-09-06 00:27:09 +02:00
Andreas Kling 266136b32f LibWeb: Stop using Bindings::wrap() in generated code 2022-09-06 00:27:09 +02:00
Andreas Kling bc74909308 LibWeb: Remove many unnecessary #includes in generated code 2022-09-06 00:27:09 +02:00
Andreas Kling b1b5d75ce6 LibWeb: Remove now-unused wrapper and iterator code generators
These are no longer used, as all wrappers have been folded into their
corresponding C++ objects.
2022-09-06 00:27:09 +02:00
Andreas Kling 5c9441799f LibWeb: Remove CMake target for wrapper cpp and h files 2022-09-06 00:27:09 +02:00
Andreas Kling ac274eee72 LibWeb: Remove the NO_INSTANCE option now that all wrappers are gone 2022-09-06 00:27:09 +02:00
Andreas Kling 497ead37bc LibWeb: Make DOMException GC-allocated 2022-09-06 00:27:09 +02:00
Andreas Kling 2fe97fa8db LibWeb: Make WorkerNavigator GC-allocated 2022-09-06 00:27:09 +02:00
Andreas Kling 9da72cdaba LibWeb: Make WorkerLocation GC-allocated 2022-09-06 00:27:09 +02:00
Andreas Kling fe9c5395d4 LibWeb: Make URL, URLSearchParams & URLSearchParamsIterator GC-allocated 2022-09-06 00:27:09 +02:00
Andreas Kling a8a42b4432 LibWeb: Don't build iterator wrapper for NO_INSTANCE classes 2022-09-06 00:27:09 +02:00
Andreas Kling 647ac1bdba LibWeb: Make Blob and File GC-allocated 2022-09-06 00:27:09 +02:00
Andreas Kling 2ac8e3db3a LibWeb: Make Storage GC-allocated 2022-09-06 00:27:09 +02:00
Andreas Kling ffad902c07 LibWeb: Use cached_web_prototype() as much as possible
Unlike ensure_web_prototype<T>(), the cached version doesn't require the
prototype type to be fully formed, so we can use it without including
the FooPrototype.h header. It's also a bit less verbose. :^)
2022-09-06 00:27:09 +02:00
Andreas Kling 2704bcdaaa LibWeb: Make Path2D GC-allocated 2022-09-06 00:27:09 +02:00
Andreas Kling 4452b5ca09 LibWeb: Make 2D and 3D canvas rendering contexts GC-allocated 2022-09-06 00:27:09 +02:00
Andreas Kling 18ca15b2cc Fuzzers: Fix FuzzCSSParser build 2022-09-06 00:27:09 +02:00
Andreas Kling 43ec0f734f LibWeb: Make MutationRecord GC-allocated 2022-09-06 00:27:09 +02:00
Andreas Kling 6f433c8656 LibWeb+LibJS: Make the EventTarget hierarchy (incl. DOM) GC-allocated
This is a monster patch that turns all EventTargets into GC-allocated
PlatformObjects. Their C++ wrapper classes are removed, and the LibJS
garbage collector is now responsible for their lifetimes.

There's a fair amount of hacks and band-aids in this patch, and we'll
have a lot of cleanup to do after this.
2022-09-06 00:27:09 +02:00
Andreas Kling 7c3db526b0 LibWeb: Make DOM::Event and all its subclasses GC-allocated 2022-09-06 00:27:09 +02:00
Andreas Kling 8cda70c892 LibWeb: Move event listeners, handlers and callbacks to the GC heap
This patch moves the following things to being GC-allocated:
- Bindings::CallbackType
- HTML::EventHandler
- DOM::IDLEventListener
- DOM::DOMEventListener
- DOM::NodeFilter

Note that we only use PlatformObject for things that might be exposed
to web content. Anything that is only used internally inherits directly
from JS::Cell instead, making them a bit more lightweight.
2022-09-06 00:27:09 +02:00
Andreas Kling 12042f0757 LibWeb: Make CSSRule and all its subclasses GC-allocated 2022-09-06 00:27:09 +02:00
Andreas Kling 5d6cb9cbdb LibWeb: Make CSSRuleList GC-allocated 2022-09-06 00:27:09 +02:00
Andreas Kling 5366924f11 LibWeb: Make StyleSheetList GC-allocated 2022-09-06 00:27:09 +02:00
Andreas Kling 5d60212076 LibWeb: Make StyleSheet and CSSStyleSheet GC-allocated 2022-09-06 00:27:09 +02:00
Andreas Kling 4ae2390554 LibWeb: Support NoInstanceWrapper extended attribute on interfaces
This tells the wrapper generator that there is no separate wrapper class
for this interface, and it should refer directly to the C++ "Foo" object
instead of "FooWrapper".
2022-09-06 00:27:09 +02:00
Andreas Kling b33b426b2e LibWeb: Add a mechanism to opt out of generating instance wrappers
To enable incremental movement towards the removal of DOM object
instance wrappers, this patch adds a NO_INSTANCE argument that can be
passed to libweb_js_wrapper().
2022-09-06 00:27:09 +02:00
Andreas Kling f6c61940f6 LibWeb: Add PlatformObject class
This represents the "platform object" concept from the IDL spec, which
refers to an object that implements an IDL interface.
2022-09-06 00:27:09 +02:00
Timothy Flynn 9e860d973e LibLocale: Move locale source files to the LibLocale library
Everything is now setup to create the LibLocale library and link it
where needed.
2022-09-05 14:37:16 -04:00
Timothy Flynn f082b6ae48 LibUnicode: Generate a separate Locale enumeration for special casing
The UCD only cares about a few locales for special casing rules (az, lt,
and tr). Unfortunately, LibUnicode cannot use LibLocale once the
libraries are separate because LibLocale will need to use LibUnicode for
many more things; thus there would be a circular dependency. Instead,
just generate the small enum needed for this one use case.
2022-09-05 14:37:16 -04:00
Timothy Flynn b2d2bb43ce LibLocale: Move locale test files to the LibLocale folder 2022-09-05 14:37:16 -04:00
Timothy Flynn 43a3471298 LibLocale: Move locale source files to the LibLocale folder
These are still included in LibUnicode, but this updates their location
and the include paths of other files which include them.
2022-09-05 14:37:16 -04:00
Timothy Flynn ff48220dca Userland: Move files destined for LibLocale to the Locale namespace 2022-09-05 14:37:16 -04:00
Timothy Flynn 88504b89e1 LibLocale: Rename a couple CLDR generator structures
When LibLocale is placed in the Locale namespace, this will conflict
with the Locale structure in each CLDR generator. Rename this to
"LocaleData", and rename its parent UnicodeLocaleData to just "CLDR"
to avoid confusion between LocaleData and UnicodeLocaleData.
2022-09-05 14:37:16 -04:00
Timothy Flynn 1e0276f541 LibLocale+LibUnicode: Move generated CLDR data files to LibLocale folder
They are still included into LibUnicode, but this moves their generated
location to be under LibLocale.
2022-09-05 14:37:16 -04:00
Timothy Flynn fc8bf7ac3e LibUnicode+Userland: Migrate generated CLDR data to LibLocaleData
Currently, LibUnicodeData contains the generated UCD and CLDR data. Move
the UCD data to the main LibUnicode library, and rename LibUnicodeData
to LibLocaleData. This is another prepatory change to migrate to
LibLocale.
2022-09-05 14:37:16 -04:00
Timothy Flynn 89d1813b5d LibUnicode: Move CLDR data generators to a LibLocale subfolder
To prepare for placing all CLDR generated data in a new library,
LibLocale, this moves the code generators for the CLDR data to the
LibLocale subfolder.
2022-09-05 14:37:16 -04:00
Tim Schumacher 854792c340 Meta: Don't generate emoji.txt into the source tree 2022-09-05 09:50:31 -04:00
kleines Filmröllchen c91511b883 Meta+Tests: Allow running FLAC spec tests
The FLAC "spec tests", or rather the test suite by xiph that exercises
weird FLAC features and edge cases, can be found at
https://github.com/ietf-wg-cellar/flac-test-files and is a good
challenge for our FLAC decoder to become more spec compliant. Running
these tests is similar to LibWasm spec tests, you need to pass
INCLUDE_FLAC_SPEC_TESTS to CMake.

As of integrating these tests, 23 out of 63 fail. :yakplus:
2022-09-02 23:54:50 +01:00
Liav A 159f9688dc Meta: Use correct boot device selections when using GRUB images 2022-09-02 23:36:08 +01:00
Liav A 94e39308e3 Meta: Remove an unnecessary instance of AHCI controller in the Q35 setup
We don't need this AHCI controller to be present as we already have one
in the Q35 machine. This will help using the correct boot device in GRUB
setups later on.
2022-09-02 23:36:08 +01:00
Linus Groh cfa5885855 LibJS: Turn initialize_global_object() into a regular initialize()
There's nothing special about global object initialization anymore, this
can just work the same way as for any other object now.
2022-08-28 16:36:56 +01:00
Ryan Liptak 9d14ed6c14 Meta: Enforce no leading zeros for emoji filenames
The current lookup code and emoji.txt generator expects codepoints to
not include leading zeros. This may change in the future, but it's worth
enforcing the current convention until then.
2022-08-28 16:07:00 +01:00
davidot cd763de280 LibJS+LibUnicode: Move some constant arrays to a separate header
Since LibUnicode depends on this data it used to include
Intl/AbstractOperations which in turn includes a number of other LibJS
headers. By moving this to its own header with minimal includes we can
save on rebuilding LibUnicode for unrelated LibJS header changes.
2022-08-27 10:55:44 -04:00
Linus Groh 50428ea8d2 LibJS: Move intrinsics to the realm
Intrinsics, i.e. mostly constructor and prototype objects, but also
things like empty and new object shape now live on a new heap-allocated
JS::Intrinsics object, thus completing the long journey of taking all
the magic away from the global object.
This represents the Realm's [[Intrinsics]] slot in the spec and matches
its existing [[GlobalObject]] / [[GlobalEnv]] slots in terms of
architecture.

In the majority of cases it should now be possibly to fully allocate a
regular object without the global object existing, and in fact that's
what we do now - the realm is allocated before the global object, and
the intrinsics between both :^)
2022-08-27 11:29:10 +01:00
Andreas Kling f1cd6453ae Meta/CMake: Build Lagom with -g1 instead of -g
This *drastically* improves build performance by reducing the size of
object files by around 10x or more.
2022-08-27 12:19:37 +02:00
Brian Gianforcaro e9e8111f4a Toolchain: Update binutils to version 2.39 2022-08-24 22:24:34 +00:00
Ryan Liptak 032e06b762 Meta: Only include headings for populated groups/subgroups in emoji.txt
The primary motivation for this is to make `generate-emoji-txt.sh` more
useful for generating a compact list of new emoji being added (e.g.
for use in commit messages / PRs) if it's run with an emoji image
directory that contains only the new emojis.
2022-08-24 15:55:42 +01:00
Timothy Flynn b9fc9aeba1 CI: Bust the macOS Lagom ccache and reduce its size
It currently takes upwards of 40 minutes to download the ccache on macOS
and often errors-out near the end. Change the cache version to bust it
so we can start anew. Reduce its max size to 2 GB (a clean build is ~0.9
GB, so this allows just over 2 clean builds to be cached).
2022-08-23 19:03:56 +01:00
Timothy Flynn 2e0b20ef01 Meta: Only run the emoji generator for Serenity builds
It is not needed on Lagom, and was incidentally run twice.
2022-08-23 19:03:43 +01:00
Timothy Flynn 6dd8161002 Meta: Ensure the emoji generator depends on its own script
If the script changes, it better be re-run.
2022-08-23 19:03:43 +01:00
Timothy Flynn 00470ee59a LibTimeZone: Update to TZDB version 2022c
https://mm.icann.org/pipermail/tz-announce/2022-August/000072.html
2022-08-23 15:06:40 +01:00
Timon Kruiper 93cf7d42d9 Meta: Remove check for QEMU < 7.0 on aarch64 build
With the previous commit, we can now run our aarch64 kernel with QEMU
7.x :^)
2022-08-23 09:23:27 -04:00
Linus Groh b345a0acca LibJS+LibWeb: Reduce use of GlobalObject as an intermediary
- Prefer VM::current_realm() over GlobalObject::associated_realm()
- Prefer VM::heap() over GlobalObject::heap()
- Prefer Cell::vm() over Cell::global_object()
- Prefer Wrapper::vm() over Wrapper::global_object()
- Inline Realm::global_object() calls used to access intrinsics as they
  will later perform a direct lookup without going through the global
  object
2022-08-23 13:58:30 +01:00
Linus Groh e3895e6c80 LibJS: Pass Realm to define_native_{accessor,function}()
This is needed so that the allocated NativeFunction receives the correct
realm, usually forwarded from the Object's initialize() function, rather
than using the current realm.
2022-08-23 13:58:30 +01:00
Linus Groh 7c468b5a77 LibJS: Pass Realm to GlobalObject::initialize_global_object()
Global object initialization is tightly coupled to realm creation, so
simply pass it to the function instead of relying on the non-standard
'associated realm' concept, which I'd like to remove later.

This works essentially the same way as regular Object::initialize() now.

Additionally this allows us to forward the realm to GlobalObject's
add_constructor() / initialize_constructor() helpers, so they set the
correct realm on the allocated constructor function object.
2022-08-23 13:58:30 +01:00
Linus Groh b465f46e00 LibJS: Remove GlobalObject parameter from native functions 2022-08-23 13:58:30 +01:00
Linus Groh 7b990c27a1 LibWeb: Replace GlobalObject with VM in remaining AOs [Part 4/4] 2022-08-23 13:58:30 +01:00
Linus Groh 40a70461a0 LibWeb: Replace GlobalObject with Realm in wrapper functions
Similar to create() in LibJS, wrap() et al. are on a low enough level to
warrant passing a Realm directly instead of relying on the current realm
from the VM, as a wrapper may need to be allocated while no JS is being
executed.
2022-08-23 13:58:30 +01:00
Linus Groh 25849f8a6d LibJS: Replace GlobalObject with VM in common AOs [Part 18/19] 2022-08-23 13:58:30 +01:00
Linus Groh ccdfa2320c LibJS: Replace GlobalObject with VM in Iterator AOs [Part 7/19] 2022-08-23 13:58:30 +01:00
Linus Groh a022e548b8 LibJS: Replace GlobalObject with VM in Value AOs [Part 4/19]
This is where the fun begins. :^)
2022-08-23 13:58:30 +01:00
Linus Groh 999da617c5 LibJS: Remove GlobalObject from VM::this_value()
This is a continuation of the previous six commits.

The global object is only needed to return it if the execution context
stack is empty, but that doesn't seem like a useful thing to allow in
the first place - if you're not currently executing JS, and the
execution context stack is empty, there is no this value to retrieve.
2022-08-23 13:58:30 +01:00
Linus Groh f3117d46dc LibJS: Remove GlobalObject from VM::throw_completion()
This is a continuation of the previous five commits.

A first big step into the direction of no longer having to pass a realm
(or currently, a global object) trough layers upon layers of AOs!
Unlike the create() APIs we can safely assume that this is only ever
called when a running execution context and therefore current realm
exists. If not, you can always manually allocate the Error and put it in
a Completion :^)

In the spec, throw exceptions implicitly use the current realm's
intrinsics as well: https://tc39.es/ecma262/#sec-throw-an-exception
2022-08-23 13:58:30 +01:00
Linus Groh e992a9f469 LibJS+LibWeb: Replace GlobalObject with Realm in Heap::allocate<T>()
This is a continuation of the previous three commits.

Now that create() receives the allocating realm, we can simply forward
that to allocate(), which accounts for the majority of these changes.
Additionally, we can get rid of the realm_from_global_object() in one
place, with one more remaining in VM::throw_completion().
2022-08-23 13:58:30 +01:00
Linus Groh b99cc7d050 LibJS+LibWeb: Replace GlobalObject with Realm in create() functions
This is a continuation of the previous two commits.

As allocating a JS cell already primarily involves a realm instead of a
global object, and we'll need to pass one to the allocate() function
itself eventually (it's bridged via the global object right now), the
create() functions need to receive a realm as well.
The plan is for this to be the highest-level function that actually
receives a realm and passes it around, AOs on an even higher level will
use the "current realm" concept via VM::current_realm() as that's what
the spec assumes; passing around realms (or global objects, for that
matter) on higher AO levels is pointless and unlike for allocating
individual objects, which may happen outside of regular JS execution, we
don't need control over the specific realm that is being used there.
2022-08-23 13:58:30 +01:00
Linus Groh 5dd5896588 LibJS+LibWeb: Replace GlobalObject with Realm in initialize() functions
This is a continuation of the previous commit.

Calling initialize() is the first thing that's done after allocating a
cell on the JS heap - and in the common case of allocating an object,
that's where properties are assigned and intrinsics occasionally
accessed.
Since those are supposed to live on the realm eventually, this is
another step into that direction.
2022-08-23 13:58:30 +01:00
Linus Groh ecd163bdf1 LibJS+LibWeb: Replace GlobalObject with Realm in object constructors
No functional changes - we can still very easily get to the global
object via `Realm::global_object()`. This is in preparation of moving
the intrinsics to the realm and no longer having to pass a global
object when allocating any object.
In a few (now, and many more in subsequent commits) places we get a
realm using `GlobalObject::associated_realm()`, this is intended to be
temporary. For example, create() functions will later receive the same
treatment and are passed a realm instead of a global object.
2022-08-23 13:58:30 +01:00
Linus Groh ae264c9143 CI: Install newer Bash via homebrew on macOS
The recently added generate-emoji-txt.sh script uses a Bash 4
substitution feature, causing CI to fail as macOS ships an ancient Bash
3.x. We'll want to use a more recent version anyway, so let's do that
instead of updading the script to older syntax.
2022-08-23 13:25:39 +01:00
Timothy Flynn d86b25c460 Meta: Move downloading of emoji-test.txt to unicode_data.cmake
The current emoji_txt.cmake does not handle download errors (which were
a common source of issues in the build problems channel) or Unicode
versioning. These are both handled by unicode_data.cmake. Move the
download to unicode_data.cmake so that we can more easily handle next
month's Unicode 15 release.
2022-08-22 16:00:29 +01:00
Ryan Liptak 221d9089e9 Meta: Generate emoji.txt at build time from Unicode's emoji-test.txt
Instead of manually updating emoji.txt whenever new emoji are added,
we use Unicode's emoji-test.txt to generate emoji.txt on each build,
including only the emojis that Serenity supports at that time.

By using emoji-test.txt, we can also include all forms of each emoji
(fully-qualified, minimally-qualified, and unqualified) which can be
helpful when double-checking how certain forms are handled.
2022-08-22 08:42:54 +01:00
Ryan Liptak 68ff0a7d13 Meta: Add check-emoji script to validate emoji filenames
Verifies that emoji filenames:
- Contain only uppercase letters, numbers, +, and _
- Use _ and a separator between codepoints, not +
- Do not include the U+FE0F emoji presentation specifier
2022-08-20 23:50:41 +01:00
Timon Kruiper a4dadaeb00 Meta: Disallow running with QEMU >= 7.x on aarch64 build
This is currently broken upstream, and our aarch64 Kernel only runs with
QEMU 6.x.
2022-08-17 15:44:38 +01:00
Timothy Flynn ca92e37ae0 LibUnicode: Generate code point display names with run-length encoding
Similar to commit becec35, our code point display name data was a large
list of StringViews. RLE can be used here as well to remove about 32 MB
from the initialized data section to the read-only section.

Some of the refactoring to store strings as indices into an RLE array
also lets us clean up some of the code point name generators.
2022-08-17 15:42:12 +01:00
Timothy Flynn 2c2ede8581 LibUnicode: Mark UniqueStringStorage::generate as constant
This is just to allow it to be invoked from callers who hold a constant
UniqueStringStorage instance.
2022-08-17 15:42:12 +01:00
Timothy Flynn becec3578f LibTimeZone+LibUnicode: Generate string data with run-length encoding
Currently, the unique string lists are stored in the initialized data
sections of their shared libraries. In order to move the data to the
read-only section, generate the strings using RLE arrays.

We generate two arrays: the first is the RLE data itself, the second is
a list of indices into the RLE array for each string. We then generate a
decoding method to convert an RLE string to a StringView.
2022-08-16 16:56:17 +02:00
Tim Schumacher 0598f52fc0 Meta: Remove an outdated MAKE_DIRECTORY call for pnp IDs
We are downloading these directly into the build directory now, and
generating the source code from there, so we no longer need the
manually created directory.

While we are at it, remove two variables that seem to be no longer in
use, and at least one of which is confusing regarding a missing prefix.
2022-08-16 06:07:19 -04:00
Emily Trau d43640037e Meta: Remove TZDB duplication 2022-08-15 09:57:49 +01:00
Sam Atkins 7b61d16262 LibWeb: Correct variable name in get_shortest_function_length() 2022-08-14 11:30:40 +02:00
Sam Atkins 2ec52bbbd5 LibWeb: Implement Path2D class 2022-08-14 11:30:40 +02:00
Sam Atkins 504d221e4a LibWeb: Allow "unrestricted" floats and doubles in IDL
For now, we don't treat them any differently from regular floats and
doubles.
2022-08-14 11:30:40 +02:00
Liav A 6164729d06 Everywhere: Get rid of the fbdev kernel boot argument remainders 2022-08-14 01:03:23 +01:00
Timothy Flynn d54e3a1cc0 LibTimeZone: Update to TZDB version 2022b
https://mm.icann.org/pipermail/tz-announce/2022-August/000071.html
2022-08-11 15:13:11 -04:00
Timothy Flynn 407a88ebf6 CI: Enable downloading the Azure remote data cache for Fuzzer builds
This cache was disabled in 3127454 because it wasn't needed and there
was a race between the builders for this cache. Then commit 0c95d99
started fuzzing the generated Unicode / TZDB data. Since then, we've
been pulling this data from the live servers instead of Azure's cache.
2022-08-09 20:06:34 -04:00
Timothy Flynn 5c0f1d5b8a CI: Add a restoration key for Azure's remote data caches
We do a similar trick for the compiler cache. This allows each builder
to separately push their local data cache (if it changed) while pulling
a shared cache, without the race outlined in commit 3127454. This is
needed for a subsequent commit which will enable this cache for Fuzzer
builds.
2022-08-09 20:06:34 -04:00
Liav A cf33d0b5f7 Kernel/FileSystem: Use a new debug flag for SysFS debug messages 2022-08-08 02:33:25 +00:00
Tom b4dd477644 LibWeb: Parse rect style value
Add ability to parse a rect when it is used as the value of a style
property.
2022-08-07 22:40:11 +02:00
Andreas Kling 50d951aea2 LibJS: Let Shape store a Realm instead of a GlobalObject
This is a cautious first step towards being able to create JS objects
before a global object has been instantiated.
2022-08-05 12:42:46 +02:00
MacDue 8da0c36212 Meta: Set has_unscopable_member for interfaces include mixins with them 2022-07-30 13:20:47 +02:00
Sam Atkins abd359a921 LibWeb: Skip whitespace when parsing IDL non-interface entities
This stops the WrapperGenerator freaking out when an IDL file starts
with a comment or whitespace. :^)
2022-07-29 17:15:49 +01:00
Timothy Flynn ae2acc8cdf LibJS+LibUnicode: Generate a set of default DateTimeFormat patterns
This isn't called out in TR-35, but before ICU even looks at CLDR data,
it adds a hard-coded set of default patterns to each locale's calendar.
It has done this since 2006 when its DateTimeFormat feature was first
created. Several test262 tests depend on this, which under ECMA-402,
falls into "implementation defined" behavior. For compatibility, we
can do the same in LibUnicode.
2022-07-22 23:51:56 +01:00
Kenneth Myhra 114120852d LibWeb/IDL: Handle passing ArrayBuffer to an IDL union type
This fixes a bug where an ArrayBuffer passed to the Blob constructor
would just be stringified to: "[object ArrayBuffer]".
2022-07-22 10:18:53 +01:00
Timothy Flynn 32c07bc6c3 LibUnicode: Generate per-locale data for the "noon" fixed day period
Note that not all locales have this day period.
2022-07-21 20:36:03 +01:00
Timothy Flynn 16b673eaa9 LibUnicode: Check whether a calendar symbol for a locale actually exists
In the generated unique string list, index 0 is the empty string, and is
used to indicate a value doesn't exist in the CLDR. Check for this
before returning an empty calendar symbol.

For example, an upcoming commit will add the fixed day period "noon",
which not all locales support.
2022-07-21 20:36:03 +01:00
Timothy Flynn 0f26ab89ae LibJS+LibUnicode: Handle flexible day periods on both sides of midnight
Commit ec7d535 only partially handled the case of flexible day periods
rolling over midnight, in that it only worked for hours after midnight.
For example, the en locale defines a day period range of [21:00, 06:00).
The previous method of adding 24 hours to the given hour would change
e.g. 23:00 to 47:00, which isn't valid.
2022-07-21 20:36:03 +01:00
Andrew Kaster 2aaaee6744 CI+Lagom: Add Lagom Android CI for arm64-v8a on NDK 24 with API level 30
This will let us validate that we aren't breaking any library compile
steps for arm64.
2022-07-21 16:37:15 +02:00
Timothy Flynn b2709f161e LibUnicode: Generate per-locale approximately & range separator symbols 2022-07-20 22:30:16 +01:00
Linus Groh dfd62437c4 LibWeb: Make IDL-generated iterator prototype next() enumerable as well
https://webidl.spec.whatwg.org/#es-iterator-prototype-object

> An iterator prototype object must have a next data property with
> attributes { [[Writable]]: true, [[Enumerable]]: true,
> [[Configurable]]: true } and whose value is a built-in function object
> that behaves as follows:

This makes three more WPT tests pass here, and likely various others:
http://wpt.live/fetch/api/headers/headers-basic.any.html
2022-07-20 18:03:21 +01:00
kleines Filmröllchen 3f59356c79 LibAudio: Rename ConnectionFromClient to ConnectionToServer
The automatic nomenclature change for IPC sockets got this one wrong.
2022-07-19 11:17:45 +01:00
Liav A 0810c1b972 Kernel/Storage: Introduce basic abstraction layer for ATA components
This abstraction layer is mainly for ATA ports (AHCI ports, IDE ports).
The goal is to create a convenient and flexible framework so it's
possible to expand to support other types of controller (e.g. Intel PIIX
and ICH IDE controllers) and to abstract operations that are possible on
each component.

Currently only the ATA IDE code is affected by this, making it much
cleaner and readable - the ATA bus mastering code is moved to the
ATAPort code so more implementations in the near future can take
advantage of such functionality easily.

In addition to that, the hierarchy of the ATA IDE code resembles more of
the SATA AHCI code now, which means the IDEChannel class is solely
responsible for getting interrupts, passing them for further processing
in the ATAPort code to take care of the rest of the handling logic.
2022-07-19 11:07:34 +01:00
Tim Schumacher 28061cf94d Everywhere: Fully remove the separate LibPthread directory 2022-07-19 11:00:35 +01:00
Andrew Kaster 9c2211f246 Meta: Add Android cross-compile support to Lagom
This is a start to properly letting us cross-compile Lagom where both
the Tools and the BUILD_LAGOM=ON build are using Lagom CMakeLists.

The initial cut allows an Android build to succeed, more or less.
But there are issues with namespace clashes when using FetchContent with
this approach.
2022-07-19 10:44:02 +01:00
Linus Groh f33df0ee05 LibWeb: Split WrapperGenerator namespace check into an Array + contains
Also sort the entries alphabetically while we're here :^)
2022-07-19 00:46:04 +01:00
Linus Groh 6b64ca4bb8 LibWeb: Prepare WrapperGenerator for Fetch bindings 2022-07-19 00:27:35 +01:00