Commit graph

2246 commits

Author SHA1 Message Date
kleines Filmröllchen 904a4dd314 Meta: Use extract_tar_path for WASM spec test generation 2023-03-19 14:15:35 +00:00
kleines Filmröllchen d3d93ef1df Meta: Refactor FLAC spec test extraction with extract_tar_path 2023-03-19 14:15:35 +00:00
kleines Filmröllchen a6dd992ad3 Meta: Add a tar extraction CMake wrapper
This is a copy of the zip wrapper adopted for tar archives.
2023-03-19 14:15:35 +00:00
Liav A c66f7f2e7c Meta: Use non-VGA VirtIO GPU variants when running on macOS
It appears that QEMU on macOS doesn't have the VirtIO GPU variants that
support VGA functionality. Those variants are not especially important
to us, because we don't use any kind of VGA functionality in our kernel
anyway.

Therefore, for macOS, we could decide to use virtio-gpu-gl-pci and
virtio-gpu-pci devices instead.
2023-03-19 00:19:06 +00:00
Timothy Flynn 1b260ab1f8 Meta: Move global VM creation to fuzzer "global" structure
Turns out LLVMFuzzerTestOneInput may be called more than once per
process.
2023-03-19 00:15:05 +00:00
Andrew Kaster cabc99e953 Fuzzers: Skip trying to parse invalid UTF-8 in LibJS Fuzzers
Invalid UTF-8 crashes JS::Script::Parse.
2023-03-18 15:56:18 +01:00
Timothy Flynn 5d7a85bc37 Meta: Ensure the main thread VM is created before use in the CSS fuzzer 2023-03-17 16:48:43 +00:00
Timothy Flynn 13dfadba79 LibJS: Propagate errors from VM creation 2023-03-17 16:39:08 +00:00
Luke Wilde 782cdaeccf LibWeb: Give generated constructor functions a name
Required by code that brand checks native constructors.

For example, Wistia brand checks XMLHttpRequest by doing:
```
XMLHttpRequest.prototype.constructor.toString()
```

It then checks if it matches either one of:
```
function XMLHttpRequest() { [native code] }
```
```
[object XMLHttpRequestConstructor]
```

If neither matches, it disables HLS playback and prints:
"The XMLHttpRequest constructor has been tampered with. Because this
affects CORS/Range XHR requests, HLS playback has been disabled.
To enable HLS playback and other important features, please remove code
that changes the definition of window.XMLHttpRequest."

We hit this path due to not giving generated constructors a name, as
we would provide `function () { [native code] }`.
2023-03-17 16:37:21 +00:00
Andreas Kling d38a3ca9eb LibGfx/OpenType: Add some initial support for GPOS glyph positioning
This patch parses enough of GPOS tables to be able to support the
kerning information embedded in Inter.

Since that specific font only applies positioning offsets to the first
glyph in each pair, I was able to get away with not changing our API.
Once we start adding support for more sophisticated positioning, we'll
need to be able to communicate more than a simple "kerning offset" to
the clients of this code.
2023-03-17 09:36:20 +01:00
Timothy Flynn 2cd7159694 LibWeb: Add a custom extended attribute for namespace-level GC visitors
Currently, the WebAssemblyObject implements a visitor to keep its static
objects alive. This custom attribute will be used to hook the generated
namespace object's visitor to one that we define in non-generated code.
2023-03-16 19:39:17 +00:00
Timothy Flynn aa06dd7f21 LibWeb: More fully implement the LegacyNamespace IDL extended attribute
Interfaces with a LegacyNamespace extended attribute should have their
constructors defined on the namespace identified by the LegacyNamespace
attribute value.
2023-03-16 19:39:17 +00:00
Timothy Flynn a277e645d3 LibWeb: Add LibJS includes to generated IDL namespace files
These will be needed by any namespace having a BufferSource parameter,
such as WebAssembly.

Similar to 49e6cb7c3d.
2023-03-16 19:39:17 +00:00
Fabian Dellwing b07654c3c1 Meta: Download cacert.pem at build time 2023-03-16 18:54:20 +03:30
Timothy Flynn 2cfcbccdb5 LibWeb: Port WebAssembly.Table to IDL 2023-03-16 13:54:16 +01:00
Timothy Flynn ca96f8e364 LibWeb: Port WebAssembly.Memory to IDL 2023-03-16 13:54:16 +01:00
Timothy Flynn de32c44762 LibWeb: Port WebAssembly.Instance to IDL 2023-03-16 13:54:16 +01:00
Timothy Flynn fb1f15774f LibWeb: Port WebAssembly.Module to IDL 2023-03-16 13:54:16 +01:00
Timothy Flynn 5c97ffb94f LibWeb: Define WebAssembly namespace and import it in web bindings
This will contain concrete definitions of WebAssembly objects used by
generated WebAssembly IDL.
2023-03-16 13:54:16 +01:00
Timothy Flynn af119d92cb LibIDL+LibWeb: Begin supporting the LegacyNamespace extended attribute
This is used by WebAssembly IDL files. For now, we mostly use this for
error messages and cache keys (to ensure compatibility with existing
code as WebAssembly is ported to IDL).
2023-03-16 13:54:16 +01:00
Timothy Flynn 3a11f65451 LibWeb: Add crude support for dereferencing JavaScript built-ins in IDL
This works similarly to the is_platform_object() helper, in that we do
not know that a type like ArrayBuffer needs to be qualified with JS::.
2023-03-16 13:54:16 +01:00
Timothy Flynn 0d921720b4 LibWeb: Support generating optional object IDL parameters
These are used by WebAssembly IDL.
2023-03-16 13:54:16 +01:00
Timothy Flynn 49e6cb7c3d LibWeb: Add LibJS includes to generated IDL constructor files
These will be needed by any constructor having a BufferSource parameter,
such as WebAssembly.Module.
2023-03-16 13:54:16 +01:00
Andrew Kaster 49b07cda12 CI: Bump Android NDK version used in CI to 25.2.9519653
The previous version is no longer able to compile Lagom

Clang version 14.0.1 in NDK 24.0.8215888 crashes trying to build
Core::Process
2023-03-15 20:36:22 -06:00
Sam Atkins b18c1c7291 LibUnicode: Remove now-unused dir-iterator helper functions 2023-03-15 12:49:33 -04:00
Sam Atkins 414cafa7f8 LibLocale: Migrate code generators to Directory::for_each_entry() 2023-03-15 12:49:33 -04:00
Sam Atkins 8a8ad81aa1 LibUnicode: Migrate GenerateEmojiData to Directory::for_each_entry() 2023-03-15 12:49:33 -04:00
Sam Atkins 8672b380f6 LibUnicode: Read emoji file title from LexicalPath directly
... rather than taking the whole file name, and then manually trimming
the extension off.
2023-03-15 12:49:33 -04:00
Timothy Flynn 020c2b59c4 LibWeb: Support generating IDL namespaces
These are similar to prototypes and constructors in that they will now
be lazily instantiated when they are first requested.
2023-03-15 12:48:25 -04:00
Matthew Olsson 0c7eac8d9a Lagom: Remove debug line in LibJSGCVerifier 2023-03-15 08:48:49 +01:00
Matthew Olsson 7c0c1c8f49 LibJS+LibWeb: Wrap raw JS::Cell*/& fields in GCPtr/NonnullGCPtr 2023-03-15 08:48:49 +01:00
gustrb 5141c86587 AK: Rename CaseInsensitiveStringViewTraits to reflect intent
Now it is called `CaseInsensitiveASCIIStringViewTraits`, so we can be
more specific about what data structure does it operate onto. ;)
2023-03-14 21:34:32 +00:00
Timothy Flynn 9c569e8a0f LibWeb: Implement the [PutForwards] IDL extended attribute
For example, consider the attribute:

    interface Element {
        [PutForwards=value] readonly attribute DOMTokenList classList;
    }

When `classList` is set, we should instead set the attribute `value` on
the `classList` attribute of the Element interface.
2023-03-14 16:30:19 -04:00
Timothy Flynn e6dc3c291e LibWeb: Prevent variadic arguments from reserving heaps of memory
Don't try to reserve capacity for a variadic arguments list unless we
actually have enough arguments to fill it with anything. Otherwise we
may overflow to an extremely large size if, e.g., the argument count
is 0 and the start of the variadic arguments is index 1.
2023-03-14 16:41:35 +00:00
Timothy Flynn 6c5e79ce3a LibWeb: Begin adding support for Function IDL types in unions
For example, setTimeout/setInterval use a TimerHandler, which is a union
of (DOMString or Function).
2023-03-14 09:07:40 -04:00
Timothy Flynn 50717ce404 LibWeb: Support variadic "any..." parameters in IDL
For example, this is used by setTimeout/setInterval.
2023-03-14 09:07:40 -04:00
kleines Filmröllchen 0563048949 file: Read more metadata from audio files
We can always read the basic format information (sample rate, bit depth,
etc.), but we will also print artist, album, and title if available in
the metadata.
2023-03-13 12:35:17 -04:00
Daniel Bertalan b668000e44 Meta+Documentation: Allow cross-debugging x86_64 Serenity on M1 Macs
While there is no native GDB on Apple Silicon, a cross-debugger that
supports x86-64 does exist.
2023-03-13 17:02:56 +01:00
Tim Schumacher 8032724574 CodeGenerators: Ensure that we always print the entire generated output 2023-03-13 15:16:20 +00:00
Tim Schumacher a3f73e7d85 AK: Rename Stream::read_entire_buffer to Stream::read_until_filled
No functional changes.
2023-03-13 15:16:20 +00:00
Tim Schumacher d5871f5717 AK: Rename Stream::{read,write} to Stream::{read_some,write_some}
Similar to POSIX read, the basic read and write functions of AK::Stream
do not have a lower limit of how much data they read or write (apart
from "none at all").

Rename the functions to "read some [data]" and "write some [data]" (with
"data" being omitted, since everything here is reading and writing data)
to make them sufficiently distinct from the functions that ensure to
use the entire buffer (which should be the go-to function for most
usages).

No functional changes, just a lot of new FIXMEs.
2023-03-13 15:16:20 +00:00
kleines Filmröllchen 264cc76ab4 LibAudio: Move audio stream buffering into the loader
Before, some loader plugins implemented their own buffering (FLAC&MP3),
some didn't require any (WAV), and some didn't buffer at all (QOA). This
meant that in practice, while you could load arbitrary amounts of
samples from some loader plugins, you couldn't do that with some others.
Also, it was ill-defined how many samples you would actually get back
from a get_more_samples call.

This commit fixes that by introducing a layer of abstraction between the
loader and its plugins (because that's the whole point of having the
extra class!). The plugins now only implement a load_chunks() function,
which is much simpler to implement and allows plugins to play fast and
loose with what they actually return. Basically, they can return many
chunks of samples, where one chunk is simply a convenient block of
samples to load. In fact, some loaders such as FLAC and QOA have
separate internal functions for loading exactly one chunk. The loaders
*should* load as many chunks as necessary for the sample count to be
reached or surpassed (the latter simplifies loading loops in the
implementations, since you don't need to know how large your next chunk
is going to be; a problem for e.g. FLAC). If a plugin has no problems
returning data of arbitrary size (currently WAV), it can return a single
chunk that exactly (or roughly) matches the requested sample count. If a
plugin is at the stream end, it can also return less samples than was
requested! The loader can handle all of these cases and may call into
load_chunk multiple times. If the plugin returns an empty chunk list (or
only empty chunks; again, they can play fast and loose), the loader
takes that as a stream end signal. Otherwise, the loader will always
return exactly as many samples as the user requested. Buffering is
handled by the loader, allowing any underlying plugin to deal with any
weird sample count requirement the user throws at it (looking at you,
SoundPlayer!).

This (not accidentally!) makes QOA work in SoundPlayer.
2023-03-13 13:25:42 +01:00
Timothy Flynn 11fe34ce0f headless-browser: Re-implement headless-browser using an OOPWV
headless-browser currently uses its own PageClient to load web pages
in-process. Due to this, it also needs to set up a whole bunch of other
objects needed to run LibWeb, e.g. image decoders, request servers, etc.

This changes headless-browser to instead implement a WebView to launch
WebContent out-of-process. This implementation is almost entirely empty,
but can be filled in as-needed. For example, we may want to print
JavaScript console messages.
2023-03-13 07:21:00 +00:00
Nico Weber b10ec6743f Userland: Add an image utility
At the moment, all it can do is read all image formats that LibGfx can
read and save to any image format that LibGfx can write (currently bmp,
png, qoi).

Currently, it drops all image metadata (including color profiles).

Over time, this could learn tricks like keeping color profiles,
converting an image to a different color profile, cropping out a part of
an image, and so on.
2023-03-12 12:17:26 +00:00
Nico Weber 3cff36b7ab Meta+CMake: Remove "image" ninja target in favor of "qemu-image"
"image" was an alias for "qemu-image".

I want to add an `image` userland utility, which clashes with that
shortname.

So remove the existing "image" target. It was just an alias for
"qemu-image".

If you use serenity.sh to build, nothing changes. This only affects you
if you run ninja manually -- you now have to say `ninja qemu-image` to
build the disk image.
2023-03-12 01:48:56 +00:00
Pankaj Raghav a1f2f08764 Meta: Use the correct boot device addressing mode for NVMe devices
Commit: 2c84466ad8 ("Kernel/Storage: Introduce new boot device
addressing modes") changed the way we pass the boot device parameter.

That commit missed updating boot parameter in the run.sh script for NVMe
boot devices.
2023-03-11 13:15:00 +00:00
Andrew Kaster b996e15d9d CI: Remove stale dependence on Toolchain.yml from nightly job
We removed this file in 0a7d0362ea but
forgot to update the nightly job.
2023-03-11 05:36:14 -07:00
Andreas Kling af8d5207f6 Meta: Don't check for newlines at EOF in Tests/LibWeb/Layout/ 2023-03-11 13:05:35 +01:00
Andrew Kaster 3d33217d60 Ladybird+CI: Move layout_test.sh test runner from CI yml into CMake
We should be able to run this locally, as long as ENABLE_LAGOM_LADYBIRD
is true, or if building ladybird from the ladybird source directory.

This removes a special case from the Lagom CI yml file.
2023-03-10 23:01:55 +00:00
Andreas Kling a504ac3e2a Everywhere: Rename equals_ignoring_case => equals_ignoring_ascii_case
Let's make it clear that these functions deal with ASCII case only.
2023-03-10 13:15:44 +01:00
kleines Filmröllchen 0f7a5006d1 Meta/Lagom: Add a fuzzer for QOA 2023-03-10 04:07:14 -07:00
Andrew Kaster 8f8e04677d Documentation: Reorganize Lagom README and update fuzzer documentation
Add some prose to the introduction of Lagom about how we use it. Also,
move the section on including Lagom in other projects above the fuzzer
documentation.

Remove the explicit cmake commands from the Fuzzer documentation, as the
script should be the source of truth.
2023-03-10 03:48:41 -07:00
Linus Groh f068ddb79f LibIMAP: Propagate OOM errors from decode_quoted_printable() 2023-03-09 14:47:45 +00:00
Kenneth Myhra 680e970597 LibWeb: Move generated #include statement for iterators out of loop
When an IDL file has #imports and the IDL interface exposes an iterator,
the bindings generator would generate #include statements missing the
class name of the iterator in the form 'LibWeb/{namespace}/Iterator'.

This change only generates the iterator #include statement for the top
interface that is the iterator.
2023-03-08 18:57:36 +00:00
Linus Groh 129ab02470 LibWeb/HTML: Port Window.structuredClone() to IDL 2023-03-07 23:33:34 +00:00
Linus Groh 7de9179a6d LibWeb/HTML: Port Window.performance to IDL 2023-03-07 23:33:34 +00:00
Linus Groh 95ce5735ce LibWeb/HTML: Port Window.event to IDL 2023-03-07 23:33:34 +00:00
Linus Groh e13f89c9eb LibWeb: Generate setter for [Replaceable] IDL attributes
The code is directly stolen from the REPLACEABLE_PROPERTY_SETTER() macro
which will hopefully be removed soon.
2023-03-07 23:33:34 +00:00
Linus Groh 3316d247bf LibWeb: Support interfaces with the [Global] extended attribute
These are treated differently as the interface members are placed on the
object itself, not its prototype.
As the object itself still needs to be hand-written code, and we can no
longer fully hide the gnarly generated code in the prototype object,
these now generate a 'mixin' class that is added to the actual object
through inheritance.

https://webidl.spec.whatwg.org/#Global
2023-03-07 23:33:34 +00:00
Linus Groh de83f5422d LibWeb: Generate Window{Constructor,Prototype} from IDL
The Window object is massive, so let's do the conversion to IDL step
by step. First up: getting rid of the manual constructor and prototype
definitions, which can be generated from an empty `interface Window`.
2023-03-07 23:33:34 +00:00
Luke Wilde f7ff1fd985 LibWeb: Remove CSS::Parser::ParsingContext's default constructor
This relied on pulling the current realm from the main thread VM, which
requires an execution context to be on the VM's stack. This heavily
relied on the dummy execution context that is always on the stack, for
example, when parsing the UA style sheets where no JavaScript is
running.
2023-03-07 11:51:12 +00:00
Andreas Kling 8a48246ed1 Everywhere: Stop using NonnullRefPtrVector
This class had slightly confusing semantics and the added weirdness
doesn't seem worth it just so we can say "." instead of "->" when
iterating over a vector of NNRPs.

This patch replaces NonnullRefPtrVector<T> with Vector<NNRP<T>>.
2023-03-06 23:46:35 +01:00
Matthew Olsson b33b950e45 Lagom: Add a tool to verify correctness of the LibJS GC
This is implemented as a Clang frontend tool, and currently does two
things:
  - Ensure for all fields wrapped in {Nonnull,}GCPtr<T>, T inherits from
    JS::Cell
  - Ensure for all fields not wrapped in {Nonnull,}GCPtr, that the type
    does not inherit from JS::Cell (otherwise it should be wrapped in a
    Ptr class).

In the future, this tool could be extended further. For example, we may
consider validating all implementations of Cell::visit_impl.
2023-03-06 13:05:43 +00:00
Sam Atkins 774f328783 LibCore+Everywhere: Return an Error from DirIterator::error()
This also removes DirIterator::error_string(), since the same strerror()
string will be included when you print the Error itself. Except in `ls`
which is still using fprintf() for now.
2023-03-05 20:23:42 +01:00
Timothy Flynn ca2b030336 LibUnicode: Use binary search for lookups into the generated emoji data
This sorts the array of generated emoji data by code point (first by
code point length, then by code point value). This lets us use a binary
search to find emoji data, rather than the current linear search.

In a profile of scrolling around /home/anon/Documents/emoji.txt, this
reduces the runtime of Gfx::Emoji::emoji_for_code_points from 69.03% to
28.42%. Within that, Unicode::find_emoji_for_code_points reduces from
28.42% to just 1.95%.
2023-03-05 16:44:20 +01:00
Linus Groh 2bc54ae56a LibWeb: Remove unused SourceGenerator mappings 2023-03-05 15:37:33 +00:00
Linus Groh 65b1229dae LibWeb: Don't use C++ name for error messages in generated bindings 2023-03-05 15:37:33 +00:00
Linus Groh a430d78843 LibWeb: Use 'Base::initialize()' in generated bindings 2023-03-05 15:37:33 +00:00
nipos d6fd97e8fb Meta: Link to libnsl and libsocket on Solaris in Lagom CMakeLists 2023-03-04 21:34:54 -07:00
Andrew Kaster 9fb781d7db Base+Meta: Remove invalid symlinks from Base for more and env
These symlinks' only purpose was to be copied into the rootfs along with
the rest of Base. Instead of storing symlinks to files that either
don't exist in the Base directory, or point to an absolute path outside
of the serenity folder, move these symlinks into the
build-root-filesystem.sh script.
2023-03-04 15:18:57 +01:00
Sam Atkins 3d8cc2257f LibCMake: Introduce a CMake lexer 2023-03-04 05:43:43 -07:00
Timothy Flynn 03f32bdf86 LibUnicode: Validate that all emoji images in /res/emoji actually exist
This will raise a compile error if an emoji image was neglected to be
added to e.g. emoji-serenity.txt, or if the code points are not correct.
2023-03-03 17:09:58 +00:00
Linus Groh f561940e54 LibWeb: Fix duplicate enum-to-string conversion in generated code
This regressed in e3a9ed0.
2023-03-03 09:25:34 +00:00
Kenneth Myhra c0559e8a10 LibWeb: Handle optional return values for getters returning new String 2023-03-01 22:44:20 +00:00
Kenneth Myhra 98705ecf71 LibWeb: Add new String support for parameters with empty string defaults
This adds new string support for parameters with an optional default
value of empty string ("").
2023-03-01 22:44:20 +00:00
Timothy Flynn fd1fbad1d2 LibGfx+LibUnicode: Support specifying the path to search for emoji
Similar to the FontDatabase, this will be needed for Ladybird to find
emoji images. We now generate just the file name of emoji image in
LibUnicode, and look for that file in the specified path (defaulting to
/res/emoji) at runtime.
2023-03-01 14:54:16 +00:00
Linus Groh 51c3967516 Everywhere: Use '_{short_,}string' literals more
This mostly updates code what was written before but merged after these
were added.
2023-02-28 15:15:36 +00:00
Sam Atkins c06f4ac6f5 AK+Everywhere: Make GenericLexer::ignore_until() stop before the value
`consume_until(foo)` stops before foo, and so does
`ignore_until(Predicate)`, so let's make the other `ignore_until()`
overloads consistent with that so they're less confusing.
2023-02-28 12:55:10 +00:00
Ali Mohammad Pur db886fe18b Userland+AK: Stop using getopt() for ArgsParser
This commit moves the implementation of getopt into AK, and converts its
API to understand and use StringView instead of char*.
Everything else is caught in the crossfire of making
Option::accept_value() take a StringView instead of a char const*.

With this, we must now pass a Span<StringView> to ArgsParser::parse(),
applications using LibMain are unaffected, but anything not using that
or taking its own argc/argv has to construct a Vector<StringView> for
this method.
2023-02-28 15:52:24 +03:30
Luke Wilde e3a9ed0c26 LibWeb: Make [UseNewAKString] work for enums and stringifiers 2023-02-27 20:55:09 +01:00
kleines Filmröllchen bd1cecb991 Meta: Lower QEMU DirectSound driver timer period to 2ms
10ms (the default) is ridiculous and causes all kinds of glitches if we
actually want to have a low-latency queue.
<https://gitlab.com/qemu-project/qemu/-/issues/1076#note_996636777>
suggests 2ms (and no lower than 1ms). This improves audio glitch
resistance at our current 512 sample buffer size, but going lower is
still not possible.
2023-02-26 22:27:42 +01:00
Andrew Kaster ce500ed170 Lagom: Look for clang the same way serenity.sh does in BuildFuzzers.sh
Also note that Xcode does not ship libfuzzer and is not usable for
a fuzzer build.
2023-02-26 08:11:08 +00:00
Ali Mohammad Pur 6b50f23242 LibWasm+LibWeb: Sneak a JS::Completion into Wasm::Result
Imported functions in Wasm may throw JS exceptions, and we need to
preserve these exceptions so we can pass them to the calling JS code.

This also adds a `assert_wasm_result()` API to Result for cases where
only Wasm traps or values are expected (e.g. internal uses) to avoid
making LibWasm (pointlessly) handle JS exceptions that will never show
up in reality.
2023-02-26 10:54:23 +03:30
Kenneth Myhra 2facb0e8ed BindingsGenerator: Use JS::Value::to_string() when new String
Make sure to use JS::Value::to_string() when the IDL interface is marked
with the extended attribute UseNewAKString.
2023-02-26 00:27:22 +01:00
Kenneth Myhra 99f094905e BindingsGenerator: Conditionally check dictionary member is string
This adds the condition member.type->is_string() to the if statement, so
that we now conditionally check the dictionary member is a new string
and associated with an optional constructor parameter.
2023-02-26 00:27:22 +01:00
Linus Groh 09d40bfbb2 Everywhere: Use _{short_,}string to create Strings from literals 2023-02-25 20:51:49 +01:00
kleines Filmröllchen 956450df00 Meta: Document how to manipulate CMake caches for the fuzzing build 2023-02-25 19:39:04 +01:00
kleines Filmröllchen 5ec5c68ed4 Meta: Allow disabling LibWeb fuzzers
The LibWeb fuzzer build is really slow, so for local builds it is useful
to disable it when you're not interested in running that fuzzer.

Co-authored-by: Andrew Kaster <akaster@serenityos.org>
2023-02-25 19:39:04 +01:00
Nico Weber 41980675f2 Lagom: Build file in lagom builds 2023-02-25 15:00:23 +01:00
Nico Weber b07e9d4c8a Lagom: Build LibELF on arm64
This seems to build fine now on Arm Macs.
2023-02-25 15:00:23 +01:00
MacDue 727218ff4a LibTimeZone: Propagate try_append() errors while building timezone data 2023-02-24 22:18:25 +01:00
MacDue 01fa3bb788 LibUnicode: Propagate try_append() errors when building emoji data 2023-02-24 22:18:25 +01:00
Timothy Flynn 8c38d46c1a LibUnicode: Generate the path to emoji images alongside emoji data
This will provide for quicker emoji lookups, rather than having to
discover and allocate these paths at runtime before we find out if they
even exist.
2023-02-24 19:48:47 +01:00
Nico Weber f7e152d049 LibGfx: Add scaffolding for a webp decoder
At the moment, this processes the RIFF chunk structure and extracts
the ICCP chunk, so that `icc` can now print ICC profiles embedded
in webp files. (And are image files really more than containers
of icc profiles?)

It doesn't even decode image dimensions yet.

The lossy format is a VP8 video frame. Once we get to that, we
might want to move all the image decoders into a new LibImageDecoders
that depends on both LibGfx and LibVideo. (Other newer image formats
like heic and av1f also use video frames for image data.)
2023-02-24 19:44:20 +01:00
Andrew Kaster 35b07048ef CI: Make sure to brew update in macOS Azure setup
This picks up any upstream brew changes that haven't been pulled into
the default Azure VMs. For example, the ffmpeg dependency of qt is
currently broken in the 8 day old brew commit they are using.
2023-02-24 02:39:26 -07:00
Andrew Kaster 1b24fa5829 CI+Lagom: Use the same options for running test-wasm in ctest and CI
Make sure that we set SERENITY_SOURCE_DIR in ctest, and make sure to
pass the test root to the CI job.

More overhaul of test-js 'test root' finding is needed however.
2023-02-24 00:02:25 -07:00
Kenneth Myhra 836cb73d29 BindingsGenerator: Release value setting members of dictionary
When a constructor has an optional dictionary as argument, and those
members are of type new string, make sure that we release_value()
setting the dictionary members.
2023-02-23 15:48:38 +00:00
Kenneth Myhra 98ef76b144 BindingsGenerator: Pass optional values as 'sv' to String::from_utf8()
Pass optional values as StringView to String::from_utf8(), otherwise
this would not compile.
2023-02-23 15:48:38 +00:00
Kenneth Myhra 511ce240f2 BindingsGenerator: Allow stringifier to return DOMException if needed 2023-02-23 15:48:38 +00:00
MacDue 56d8e36698 Lagom: Include AccessibilityTreeModel in LibWebView
This will be needed for the accessibility inspector in Ladybird.
2023-02-22 22:00:40 +00:00
Kenneth Myhra e776171d8f LibWeb: Allow prototype.{entries keys,value} to propagate errors
This allows the prototype.{entries keys,value} native functions
generated by the BindingsGenerator to propagate errors if needed.
2023-02-22 09:55:33 +01:00
Kenneth Myhra 3689d58c64 LibWeb: Allow EventListener/NodeFilter path to propagate errors
This allows the EventListener/NodeFilter path of the BindingsGenerator
to propagate errors if needed.
2023-02-22 09:55:33 +01:00
Andreas Kling f11899f885 LibWeb+LibIDL: Fix (or paper over) various const-correctness issues
There's definitely stuff to iterate on here, but this takes care of
making the libraries compile with stricter RP and NNRP.
2023-02-21 00:54:04 +01:00
nipos f700d553ab LibCore: Add support for NetBSD in anon_create 2023-02-20 18:46:51 +01:00
Sam Atkins 2db168acc1 LibTextCodec+Everywhere: Port Decoders to new Strings 2023-02-19 17:15:47 +01:00
Sam Atkins f2a9426885 LibTextCodec+Everywhere: Return Optional<Decoder&> from decoder_for() 2023-02-19 17:15:47 +01:00
Sam Atkins f0b72b819e LibWeb: Make property/media-feature name matching ASCII case-insensitive
https://www.w3.org/TR/css-conditional-3/#dom-css-supports specifically
asks for this when calling `CSS::property_id_from_string()`, but in
general, CSS property and media-feature names can only contain ASCII.
2023-02-19 00:46:47 +01:00
Lucas CHOLLET 856d0202f2 LibGfx: Rename JPGLoader to JPEGLoader
The patch also contains modifications on several classes, functions or
files that are related to the `JPGLoader`.

Renaming include:
 - JPGLoader{.h, .cpp}
 - JPGImageDecoderPlugin
 - JPGLoadingContext
 - JPG_DEBUG
 - decode_jpg
 - FuzzJPGLoader.cpp
 - Few string literals or texts
2023-02-18 23:56:24 +01:00
Luke Wilde 3275d659bf LibWeb: Convert FormData to String and Vector storage
This makes use of the new [UseNewAKString] extended attribute. Using
Vector storage will make it easier to make this interface into an IDL
iterable. It seems the reason it didn't use Vector originally was due
to awkward DeprecatedString -> String conversions.
2023-02-18 01:23:36 +01:00
Luke Wilde c0f22065ab LibWeb: Add an extended attribute that makes interfaces use AK::String
Adding the [UseNewAKString] extended attribute to an interface will
cause all IDL string types to use String instead of DeprecatedString.
This is done on an per interface level instead of per type/parameter
because:

- It's much simpler to implement, as the generators can already access
  the interface's extended attributes. Doing it per type/parameter
  would mean parsing and piping extended attributes for each type that
  doesn't already take extended attributes, such as unions.

- Allows more incremental adoption of AK::String. For example, adding
  [UseNewAKString] to BodyInit would require refactoring Request,
  Response and XMLHttpRequest to AK::String in one swoop. Doing it on
  the interface allows you to convert just XHR and its dependencies at
  once, for example.

- Simple string return types (i.e. not parameterised or not in a union)
  already accept any of the string types JS::PrimitiveString::create
  accepts. For example, you can add [UseNewAKString] to DOMStringMap to
  convert Element attributes to AK::String and still return AK::String
  from get_attribute, without adding [UseNewAKString] to Element.

- Adding [UseNewAKString] to one function typically means adding it to
  a bunch of other functions, if not the rest of them. For example,
  adding [UseNewAKString] to the parameters FormData.append would
  either mean converting AK::String to AK::DeprecatedString or storing
  the AK::String as-is, making the other functions of FormData have to
  convert back from AK::String or also support AK::String.
2023-02-18 01:23:36 +01:00
Timothy Flynn 88814acbd3 LibJS+Everywhere: Convert JS::Error to String
This includes an Error::create overload to create an Error from a UTF-8
StringView. If creating a String from that view fails, the factory will
return an OOM InternalError instead. VM::throw_completion can also make
use of this overload via its perfect forwarding.
2023-02-17 09:14:23 -05:00
Timothy Flynn 527edb55ad LibWeb: Convert usages of Value::TDSWOSE to Value::TSWOSE 2023-02-16 14:32:22 +01:00
Timothy Flynn b245300ba1 LibJS+Everywhere: Deprecate Value::to_string_without_side_effects 2023-02-16 14:32:22 +01:00
Timon Kruiper dfc6555fec CMake: Add -mstrict-align flag to compile options for aarch64 build
The aarch64 processor is set up to trap on unaligned memory accesses, so
to enforce that the compiler correctly generates aligned accesses, the
-mstrict-align flag is needed. We also need the -Wno-cast-align as there
are some files in AK that don't build without the flag.
2023-02-15 22:53:19 +01:00
Sam Atkins d6075ef5b5 LibTextCodec+Everywhere: Make TextCodec::decoder_for() take a StringView
We don't need a full String/DeprecatedString inside this function, so we
might as well not force users to create one.
2023-02-15 12:48:26 -05:00
Ali Mohammad Pur 617d112780 Meta: Add a Shell --posix parser fuzzer 2023-02-13 23:00:15 +03:30
Ali Mohammad Pur 2a276c86d4 Shell: Start implementing a POSIX-compliant parser
The parser is still very much a work-in-progress, but it can currently
parse most of the basic bits, the only *completely* unimplemented things
in the parser are:
- heredocs (io_here)
- alias expansion
- arithmetic expansion

There are a whole suite of bugs, and syntax highlighting is unreliable
at best.
For now, this is not attached anywhere, a future commit will enable it
for /bin/sh or a `Shell --posix` invocation.
2023-02-13 23:00:15 +03:30
Tim Schumacher 43f98ac6e1 Everywhere: Remove the AK:: qualifier from Stream usages 2023-02-13 00:50:07 +00:00
Tim Schumacher 874c7bba28 LibCore: Remove Stream.h 2023-02-13 00:50:07 +00:00
Tim Schumacher 606a3982f3 LibCore: Move Stream-based file into the Core namespace 2023-02-13 00:50:07 +00:00
Tim Schumacher a96339b72b LibCore: Move Stream-based sockets into the Core namespace 2023-02-13 00:50:07 +00:00
Tim Schumacher d43a7eae54 LibCore: Rename File to DeprecatedFile
As usual, this removes many unused includes and moves used includes
further down the chain.
2023-02-13 00:50:07 +00:00
Kenneth Myhra d5b5b94a35 LibWeb: Introduce the FormData interface from the XHR specification 2023-02-12 00:18:09 +00:00
networkException 143f28b735 CI: Add script to post mastodon toots for commits on master
This patch adds the toot-commits script (mirroring tweet-commits),
posting new commits on the master branch to
https://serenityos.social/@commits.
2023-02-10 13:54:07 +00:00
Timothy Flynn 604d5f5bca AK+Everywhere: Do not implicitly copy variables in TRY macros
For example, consider cases where we want to propagate errors only in
specific instances:

    auto result = read_data(); // something like ErrorOr<ByteBuffer>
    if (result.is_error() && result.error().code() != EINTR)
        continue;
    auto bytes = TRY(result);

The TRY invocation will currently copy the byte buffer when the
expression (in this case, just a local variable) is stored into
_temporary_result.

This patch binds the expression to a reference to prevent such copies.
In less trival invocations (such as TRY(some_function()), this will
incur only temporary lifetime extensions, i.e. no functional change.
2023-02-10 09:08:52 +00:00
Timothy Flynn c3abb1396c LibJS+LibWeb: Convert string view PrimitiveString instances to String
First, this adds an overload of PrimitiveString::create for StringView.
This overload will throw an OOM completion if creating a String fails.
This is not only a bit more convenient, but it also ensures at compile
time that all PrimitiveString::create(string_view) invocations will be
handled as String and OOM-aware.

Next, this wraps all invocations to PrimitiveString::create(string_view)
with MUST_OR_THROW_OOM.

A small PrimitiveString::create(DeprecatedFlyString) overload also had
to be added to disambiguate between the StringView and DeprecatedString
overloads.
2023-02-09 17:13:33 +00:00
Timothy Flynn 49474d8718 Revert "Meta: Automatically select best apt mirror"
This reverts commit b0606d90f0.

This seems to prevent libegl-mesa0 from being installed (which for some
reason isn't failing the Azure jobs - the failure seen later is that
ccache is not installed).
2023-02-09 14:57:27 +00:00
MacDue 63b11030f0 Everywhere: Use ReadonlySpan<T> instead of Span<T const> 2023-02-08 19:15:45 +00:00
Timothy Flynn 8670526f2a LibJS+LibLocale: Propagate OOM from CLDR RelativeTime Vector operations 2023-02-08 18:32:37 +00:00
Timothy Flynn 340434ce09 LibLocale: Propagate OOM from CLDR DateTime Vector and String operations 2023-02-08 18:32:37 +00:00
Timothy Flynn 89da8de4ca LibJS+LibLocale: Propagate OOM from CLDR NumberFormat Vector operations 2023-02-08 18:32:37 +00:00
Timon Kruiper a9f7267cbd Meta: Allow creating disk image without installing kernel into it
This change allows for creating smaller disk images, which is useful for
creating a disk image that will be put into ram.
2023-02-08 18:19:48 +00:00
Tim Schumacher 220fbcaa7e AK: Remove the fallible constructor from FixedMemoryStream 2023-02-08 17:44:32 +00:00
MacDue b0606d90f0 Meta: Automatically select best apt mirror
This is an attempt to avoid issues with the azure ubuntu apt mirror
by using apt-spy2 to select the best mirror before installing
dependencies.
2023-02-07 13:22:17 +00:00
Karol Kosek bc587b89d8 CMake: Don't require to install glapi.h to system
This failed when a current build configuration hadn't got any program
depending on LibGL.
2023-02-04 15:53:07 -07:00
Karol Kosek 22a99e39d8 Lagom/ConfigureComponents: Run cmake command last
Previously the tool was removing the Root directory after configuring
cmake which was breaking the build, as some cmake rules put some
necessary files there.

Moving the cmake command to be the last one makes it regenerate those
files automatically. :^)
2023-02-04 15:53:07 -07:00
MacDue 0c313c586b Meta: Register CanvasPattern as a platform object 2023-02-03 20:36:21 +01:00
Tim Schumacher ef7f343e95 CMake: Set DWARF version before the debug information level
Setting the DWARF version after having selected which level of debug
information to generate apparently undoes some settings again.

Doing the reverse apparently keeps both the version and the debug level
setting, resulting in a significantly smaller disk image size.
2023-02-02 16:31:09 +01:00
Andrew Kaster 8f70e365f0 Meta+CI: Disable Ladybird for fuzzer, compiler explorer and Android
And move it after the declaration of headless-browser, since WebDriver
depends on headless-browser.
2023-02-02 05:35:44 -07:00
Timothy Flynn 38b0b32bb4 Meta: Set LD_LIBRARY_PATH to Build/lagom for Lagom test targets
Some tests, such as those under LibGL, try to load shared libraries at
runtime which are stored in Build/lagom. Because all tests' working
directory is that of their CMakeLists.txt file, they fail to find those
shared libraries at runtime.

Set LD_LIBRARY_PATH to Build/lagom so the tests may find the shared
libraries. This fixes, for example, both of these commands:

    ./Meta/serenity.sh test lagom TestAPI
    ./Meta/serenity.sh run lagom TestAPI
2023-02-01 12:28:02 -05:00
Timothy Flynn 1bbd889f67 Meta: Set the Lagom test working directory for run-lagom-target
When we call add_test() from each test's CMakeLists.txt, we specify the
working directory to be that of the CMakeList.txt file itself. Create a
property to store that directory and reference it when run-lagom-target
is invoked by serenity.sh.

Note that for non-test Lagom targets which do not set this property, the
working directory will be Build/lagom (because the property will be the
empty string, which CMake resolves relative to the current build path).
2023-02-01 12:28:02 -05:00
Nico Weber 0a97206a25 Meta: Derust fuzzer build docs
The document wasn't updated in 9c2211f246.
2023-02-01 14:44:58 +00:00
Timothy Flynn 9f9b8e7273 CI: Move running LibWeb layout tests to Azure
The current config on GitHub Actions does not use ccache, so it takes
quite a while to build. Instead, let's just run these tests on Azure
where we already build Ladybird and have ccache enabled. This also lets
us sanitize LibWeb on both Linux and macOS.

The script changes here are to A) handle differences between Azure and
GitHub Actions and B) to support running on macOS.
2023-02-01 14:04:44 +00:00
Nico Weber 9bb469f324 Meta: Build and run Tests/LibGfx in lagom builds
They can be run using

    Meta/serenity.sh test lagom TestFontHandling

which will build _all_ test binaries and then run TestFontHandling.

There's also

    Meta/serenity.sh run lagom TestFontHandling

which will build just TestFontHandling and run it, but currently
in a way that it doesn't pass.

Finally, manually doing

    Meta/serenity.sh build lagom TestFontHandling

followed by

    (cd Tests/LibGfx; ../../Build/lagom/Tests/LibGfx/TestFontHandling)

will build and then run just that one test in a way that makes the test
pass.
2023-02-01 08:56:56 -05:00
Tim Schumacher 093cf428a3 AK: Move memory streams from LibCore 2023-01-29 19:16:44 -07:00
Tim Schumacher 8464da1439 AK: Move Stream and SeekableStream from LibCore
`Stream` will be qualified as `AK::Stream` until we remove the
`Core::Stream` namespace. `IODevice` now reuses the `SeekMode` that is
defined by `SeekableStream`, since defining its own would require us to
qualify it with `AK::SeekMode` everywhere.
2023-01-29 19:16:44 -07:00
Timothy Flynn b75b7f0c0d LibJS+Everywhere: Propagate Cell::initialize errors from Heap::allocate
Callers that are already in a fallible context will now TRY to allocate
cells. Callers in infallible contexts get a FIXME.
2023-01-29 00:02:45 +00:00
Timothy Flynn 2692db8699 LibJS+Everywhere: Allow Cell::initialize overrides to throw OOM errors
Note that as of this commit, there aren't any such throwers, and the
call site in Heap::allocate will drop exceptions on the floor. This
commit only serves to change the declaration of the overrides, make sure
they return an empty value, and to propagate OOM errors frm their base
initialize invocations.
2023-01-29 00:02:45 +00:00
Linus Groh ee0297d9ec LibAudio: Remove try_ prefix from fallible LoaderPlugin methods 2023-01-28 22:41:36 +01:00
Nico Weber 721b280849 LibGfx: Move ICCProfile.{h,cpp} to ICC/Profile.{h,cpp} 2023-01-28 00:27:07 +00:00
Linus Groh 6e7459322d AK: Remove StringBuilder::build() in favor of to_deprecated_string()
Having an alias function that only wraps another one is silly, and
keeping the more obvious name should flush out more uses of deprecated
strings.
No behavior change.
2023-01-27 20:38:49 +00:00
Timothy Flynn 5e29e04122 LibJS+LibLocale: Propagate errors from find_regional_values_for_locale
This had quite the footprint.
2023-01-27 18:00:17 +00:00
Timothy Flynn b2097f4059 LibJS+LibLocale: Port Intl.DateTimeFormat to String 2023-01-27 18:00:17 +00:00
Timothy Flynn ad8183beb4 Meta: Support running ladybird with arguments from serenity.sh
You may now, for example, run:

    ./Meta/serenity.sh run lagom ladybird https://serenityos.org
2023-01-27 08:14:24 -05:00
Timothy Flynn 9743445e44 Meta: Allow running Lagom tests with "serenity.sh run lagom <target>"
After 5ac57f9, we could no longer run "serenity.sh run lagom TestString"
because the TestString binary now lives in a subdirectory under
Build/lagom. Thus the existing method of running "$BUILD_DIR/TestString"
could not work.

This adds a "run-lagom-target" custom target to Lagom, to run a command
and pass arguments to that invocation. It turns out there really isn't a
"pretty" way of doing this with CMake or Ninja. But we can pass these as
environment variables for CMake to interpret. We just must be careful to
massage arguments into a CMake list.
2023-01-27 08:14:24 -05:00
Sam Atkins ad9b2043dd CodeGenerators: Replace uses of JsonObject::get_deprecated()/get_ptr() 2023-01-26 09:57:14 -05:00
Tim Schumacher 982ebbc304 LibWasm: Port the parser to Core::Stream 2023-01-25 17:10:05 +03:30
Timon Kruiper 3a802e365c Meta: Unify installing GCC dependencies in disk image
This makes sure that the aarch64 disk image also contains the correct
dynamic shared objects, specifically libgcc_s.so.1, as that one is a
dynamic dependency of every aarch64 executable.

To unify the x86_64 and aarch64 code paths, this commit just installs
everything from the compilers lib directory into the disk image lib
directory. This also happens for the Clang toolchain. This copies a few
extra files related to libsupc++ and libstdc++, increasing the size of
the disk image by 1.6MB. However, we were already copying libstdc++.a
manually anyway.
2023-01-25 14:28:53 +01:00
MacDue 2be4142138 Meta: Register CanvasGradient as a platform object 2023-01-22 18:15:52 +01:00
Timothy Flynn 20536897e4 LibJS+LibLocale: Port remaining locale APIs to String 2023-01-22 01:03:13 +00:00
Timothy Flynn f1de4f8872 Everywhere: Convert known short-strings to the infallible String factory
For now, this is limited to strings that are 3 bytes or less. We can use
7 bytes on 64-bit platforms, but we do not yet assume 64-bit for Lagom
hosts (e.g. wasm).
2023-01-20 14:24:12 -05:00
Liav A 57e19a7e56 LibGfx: Re-structure the whole initialization pattern for image decoders
When trying to figure out the correct implementation, we now have a very
strong distinction on plugins that are well suited for sniffing, and
plugins that need a MIME type to be chosen.

Instead of having multiple calls to non-static virtual sniff methods for
each Image decoding plugin, we have 2 static methods for each
implementation:
1. The sniff method, which in contrast to the old method, gets a
    ReadonlyBytes parameter and ensures we can figure out the result
    with zero heap allocations for most implementations.
2. The create method, which just creates a new instance so we don't
    expose the constructor to everyone anymore.

In addition to that, we have a new virtual method called initialize,
which has a per-implementation initialization pattern to actually ensure
each implementation can construct a decoder object, and then have a
correct context being applied to it for the actual decoding.
2023-01-20 15:13:31 +00:00
Timothy Flynn ca62aeb6bd LibLocale+LibJS: Port locale parsing and processing to String
In order to prevent this commit from having to refactor almost all of
Intl, the goal here is to update the internal parsing/canonicalization
of locales within LibLocale only. Call sites which are already equiped
to handle String and OOM errors do so, however.
2023-01-19 20:57:30 +00:00
Sam Atkins 5ac57f9d41 Meta: Use existing test-sources lists for Lagom instead of globbing
...mostly.

This creates and uses an override for the `serenity_test()` function, so
that Lagom can make use of the existing `Tests/LibFoo/CMakeLists.txt`
files instead of having to GLOB for test source files and manually copy
any data files.

Some GLOBs remain but this is most of them.

Co-authored-by: Andrew Kaster <akaster@serenityos.org>
2023-01-19 11:50:10 +00:00
Sam Atkins 71af0c296f Meta/Lagom: Make sure that libraries are always linked as PRIVATE 2023-01-19 11:50:10 +00:00
Timothy Flynn 0dce7b72f9 LibCore: Implement FileWatcher for macOS
The macOS FileWatcher depends on macOS dispatch queues, which run on a
different thread than the Core::EventLoop. This implementation handles
filesystem events on its dispatch queue, then forwards the event back to
the main Core::EventLoop for notifying the FileWatcher owner.
2023-01-19 11:29:48 +00:00
Linus Groh afc055c088 LibWeb: Convert the Location object to IDL
This includes:

- Moving it from Bindings/ to HTML/
- Renaming it from LocationObject to Location
- Removing the manual definitions of the constructor and prototype
- Removing special handling of the Location interface from the bindings
  generator
- Converting the JS_DEFINE_NATIVE_FUNCTIONs to regular functions
  returning DeprecatedString instead of PrimitiveString
- Adding missing (no-op) setters for the various attributes, which are
  expected to exist by the bindings generator
2023-01-18 17:36:39 -05:00
Timothy Flynn 8f2589b3b0 LibUnicode: Parse and generate case folding code point data
Case folding rules have a similar mapping style as special casing rules,
where one code point may map to zero or more case folding rules. These
will be used for case-insensitive string comparisons. To see how case
folding can differ from other casing rules, consider "ß" (U+00DF):

    >>> "ß".lower()
    'ß'

    >>> "ß".upper()
    'SS'

    >>> "ß".title()
    'Ss'

    >>> "ß".casefold()
    'ss'
2023-01-18 14:43:40 +00:00
Timothy Flynn 9226cf7272 LibUnicode: Rename a special casing variable name in the UCD generator
This name will soon be a bit ambiguous with a similar case folding
variable name.
2023-01-18 14:43:40 +00:00
Timothy Flynn 8d9fb898d7 LibUnicode: Update out-of-date spec links
And remove links that aren't adding much value but will often get out of
date (i.e. links to UCD files, which are already all listed in
unicode_data.cmake).
2023-01-18 14:43:40 +00:00
Timothy Flynn 5a1ac851aa FuzzilliJs: Print the result of JS::Value::to_string using AK::outln
JS::Value::to_string now returns a String, which does not have a null-
terminated characters() accessor.
2023-01-18 12:52:25 +00:00
Timothy Flynn 8ca528217c LibCore: Implement FileWatcher for Linux
This implements FileWatcher using inotify filesystem events. Serenity's
InodeWatcher is remarkably similar to inotify, so this is almost an
identical implementation.

The existing TestLibCoreFileWatcher test is added to Lagom (currently
just for Linux).

This does not implement BlockingFileWatcher as that is currently not
used anywhere but on Serenity.
2023-01-18 06:46:12 -05:00
Nico Weber cbb6f8de65 Lagom/Fuzzers: Add fuzzer for ICCProfile 2023-01-17 22:25:33 -05:00
Sam Atkins 1dd6b7f5b7 AK+Everywhere: Rename JsonObject::get() to ::get_deprecated()
This is a preparatory step to making `get()` return `ErrorOr`.
2023-01-17 19:52:52 -05:00
Timothy Flynn b562348d31 LibUnicode: Generate simple case folding mappings for titlecase
Note we already generate the special case foldings for titlecase.
2023-01-16 18:33:44 -05:00
Tim Schumacher 3ed9627f4e LibIPC: Decode messages using Core::Stream internally 2023-01-15 23:06:31 -05:00
Liav A b2626d3bc1 Lagom/Fuzzers: Add fuzzer for the TGALoader code 2023-01-15 12:43:03 +01:00
Timothy Flynn afeb7273cc LibJS+Everywhere: Rename Value::to_string to to_deprecated_string 2023-01-15 01:00:20 +00:00
Andrew Kaster f5d253dcfa Everywhere: Fully qualify IsLvalueReference in TRY() macros
If USING_AK_GLOBALLY is not defined, the name IsLvalueReference might
not be available in the global namespace. Follow the pattern established
in LibTest to fully qualify AK types in macros to avoid this problem.
2023-01-15 00:56:31 +00:00
Ben Wiederhake 23d111a459 LibWeb: Implement value_id_from_string by hash lookup
The old approach was basically a linear scan, which is slower than a
hash map for the currently 303 elements, as evidenced by the new
benchmark in TestCSSIDSpeed.

Before: Completed benchmark 'value_id_from_string' in 3238ms
After: Completed benchmark 'value_id_from_string' in 193ms
2023-01-14 15:43:27 -07:00
Ben Wiederhake 0687a75eaa LibWeb: Run tests in lagom if ENABLE_LAGOM_LIBWEB is set 2023-01-14 15:43:27 -07:00
Timothy Flynn afc0e461e1 AK+Everywhere: Disallow returning a reference from a fallible expression
This will silently make a copy. Rather than masking this behavior, let's
explicitly disallow it.
2023-01-13 18:50:47 -05:00
Linus Groh 5411adca22 LibWebView+Ladybird: Begin de-duplicate WebView implementations
This starts moving code equally shared between the OOPWV and Ladybird
WebContentView implementations to WebView::ViewImplementation, beginning
with the client state.
2023-01-12 23:39:36 +01:00
MacDue 1cf45ee930 Utilities: Add ttfdisasm for disassembling OpenType instructions
This utility when given a .tff font provides options for disassembling:

 - The 'fpgm' table, this a program that's run once when the font is
   loaded. It's used to define instructions and functions used by used
   by other programs.

 - The 'prep' table, this is a general program that's run when ever
   the font size (or other properties) changes.

 - And the programs associated with any individual glyph.

The disassembly is printed in a format that matches the examples from:
https://learn.microsoft.com/en-us/typography/opentype/spec/tt_instructions

I'm mainly adding this because I think it's neat to be able to look
at these things, and think it'll be helpful for debugging an
interpreter.

With this you can see that all of the LiberationSerif-XXX.tff fonts in
Serenity have these programs ready to go.
2023-01-12 11:27:57 +01:00
Timothy Flynn 491edaffc7 LibWeb: Ensure legacy constructors are defined on the global object
This regressed in 6e93d89ee3.
2023-01-10 17:55:23 +00:00
Timothy Flynn 6e93d89ee3 LibWeb: Generate exposed Window/Worker interfaces as lazy objects
We now lazily create these instances on-demand rather than all at once
when a Window/Worker object is created.
2023-01-10 16:08:14 +01:00
Timothy Flynn 7bd8fd000f LibWeb: Generate dedicated methods to create Web constructors/prototypes
Currently, for each exposed interface, we generate one massive function
to create every Web constructor and prototype. In an effort to lazily
create these instead, this first step is to extract the creation of each
of these into its own method.

First, this generates a forwarding header for all IDL types. This is to
allow callers to remain unchanged without forcing them to include the
(very heavy) generated IDL headers. This header is included by LibWeb's
forwarding header.

Next, this defines a base template method on Web::Bindings::Intrinsics
to create a prototype/constructor pair. Specializations of this template
are now generated in a new .cpp file, IntrinsicDefinitions.cpp. The base
Intrinsics class is updated to use this new method, and will continue to
cache the result.

Last, some WebAssembly classes are updated to use this new mechanism.
They were using some ad hoc cache keys that are now in line with the
generated specializations.

That one massive function is still used to invoke these specializations,
so they are not lazy as of this commit.
2023-01-10 16:08:14 +01:00
Timothy Flynn 6fcc1c7426 AK+LibUnicode: Provide Unicode-aware String case transformations
Since AK can't refer to LibUnicode directly, the strategy here is that
if you need case transformations, you can link LibUnicode and receive
them. If you try to use either of these methods without linking it, then
you'll of course get a linker error (note we don't do any fallbacks to
e.g. ASCII case transformations). If you don't need these methods, you
don't have to link LibUnicode.
2023-01-09 19:23:46 -07:00
Timothy Flynn 12f6793223 LibUnicode: Move Unicode-aware case transformations to a helper file
These will be needed by AK::String as well, so move them to a helper
file where they can be re-used.
2023-01-09 19:23:46 -07:00
martinfalisse ce0f41b9fb LibWeb+WebContent: Use new String class in CSS::StyleValue
Converts uses of DeprecatedString to String in StyleValue, and patches
surrounding files that depend on these functions.
2023-01-09 11:09:31 +01:00
Liav A c876412b1b Kernel: Remove the NE2000 PCI network adapter driver
Nobody tests this network card as the person who added it, Jean-Baptiste
Boric (known as boricj) is not an active contributor in the project now.
After a discussion with him on the Discord server, we agreed it's for
the best to remove the driver, as for two reasons:
- The original author (boricj) agreed to do this, stating that he will
  not be able to test the driver anymore after his Athlon XP machine is
  no longer supported after the removal of the i686 port.
- It was agreed that the NE2000 network card family is far from the
  ideal hardware we would want to support, similarly to the RTL8139 that
  got removed recently for almost the same reason.
2023-01-08 21:51:59 +01:00
kleines Filmröllchen e430667923 Meta/build-manpages-website: Use absolute paths for all links
Previously, we had some broken cross-manpage links on the website after
the introduction of subsections. This is fixed by simply always using an
absolute path (leading '/') for links, making all images, icons and page
links work in all subsections.

Unfortunately, this change means that navigating the website build while
opening the files in the browser directly will no longer work. However,
a local static server such as `python -m http.server 8080` in the
output/ directory will work just fine for testing.
2023-01-08 13:35:29 +01:00
kleines Filmröllchen 76e601d758 Meta/build-manpages-website: Copy all manpage PNGs
This will allow using images in manpages elsewhere in the future without
adjusting the PNG copying command.

rsync unfortunately cannot place all files into the root folder when
receiving a list of files via --files-from=-.
2023-01-08 13:35:29 +01:00
implicitfield 28c99e7a1f LibArchive+Utilities: Stop using DeprecatedString
This also slightly improves error propagation in tar, unzip and zip.
2023-01-08 09:54:08 +00:00
Liav A 5c97c6d874 Kernel: Remove the RTL8139 PCI network adapter driver
Nobody tests this network card, and the driver has bugs (see the issue
https://github.com/SerenityOS/serenity/issues/10198 for more details),
so it's almost certain that this happened due to code being rotting when
there's simply no testing of it.

Essentially this has been determined to be dead-code so this is the most
important reason to drop this code. Another good reason to do so is
because the RTL8139 only supports Fast Ethernet connections (10/100
Megabits per second), and is considered obsolete even for bare metal
setups.
2023-01-07 11:37:57 -07:00
Ben Wiederhake 3281050359 Everywhere: Remove "LibC/" includes, add lint-rule against it 2023-01-07 10:01:37 -07:00
MacDue b0ac88cf89 Meta/Lagom: Link Threads::Threads
This is required for me to be able to build both Serenity and
Ladybird from the same repo. Without this the two builds seem to
stomp on each other, then fail to link.
2023-01-06 08:04:23 -07:00
Lucas CHOLLET 4e164c9de7 Meta: Pass Core::Stream::OpenMode by value in the IPCCompiler 2023-01-06 14:52:29 +00:00
Luke Wilde a6716e694d CI: Automatically apply pull request labels for generic PR actions
Generic PR actions include opening a PR, submit review comments, adding
new commits, etc. This prevents the reviewer and PR submitter from
having to manually bounce the labels back and forth in the general
case. The reviewer also may not have permission to set labels, meaning
the reviewer won't be able to update the labels accordingly themselves.

This does not handle more subjective labels such as pr-is-blocked and
pr-unclear. Unfortunately, there does not seem to be a GitHub Actions
trigger for when a PR has merge conflicts, so the pr-has-conflicts
label cannot be automatically applied.

Co-authored-by: kleines Filmröllchen <filmroellchen@serenityos.org>
2023-01-06 15:25:30 +01:00
kleines Filmröllchen 8fb7c32ec3 Meta: Create nested manpage section indices for the website
This is probably committing some sed crimes, but it looks very nice!
2023-01-05 15:03:52 +01:00
kleines Filmröllchen 20f44e095d Meta: Use parallel rsync for file coyping for online manpages
The main point is to bring this in line with all the other file copying,
which is always done via rsync.

On my machine this leads to very little speedup, but I'm also on WSL, so
🤷

Timings:
Before:
  Time (abs ≡): 20.440 s [User: 133.928 s, System: 12.290 s]
After:
  Time (abs ≡): 20.346 s [User: 135.534 s, System: 11.207 s]
2023-01-05 15:03:52 +01:00
kleines Filmröllchen a205efd900 Meta: Generate manpages for website in parallel
pandoc is a single-threaded and pretty slow application, so we can run
it in the background and "synchronize" before generating section
indices.

Timing results:
Before:
  Time (abs ≡):        59.833 s [User: 49.541 s, System: 6.943 s]
After:
  Time (abs ≡):        20.440 s [User: 133.928 s, System: 12.290 s]

(both generated with hyperfine -p "rm -r output || true" -r 1
	Meta/build-manpages-website.sh )
2023-01-05 15:03:52 +01:00
kleines Filmröllchen a196fafd0f Meta: Print progress during the website manpage build
This especially helps when building locally.
2023-01-05 15:03:52 +01:00
kleines Filmröllchen 3affa922e3 Meta: Make the manpage website build work with manpage subsection
For now, we create simple but complete indices and actually generate the
HTML for the manpages in subsections. For these purposes, switch to
using "find" as a file finding tool everywhere, instead of the very
limited globs from before.
2023-01-05 15:03:52 +01:00
kleines Filmröllchen 69e5645955 Meta: Remove duplicate manpage section generation for the website 2023-01-05 15:03:52 +01:00
Timothy Flynn 8b7b03b369 IPCCompiler+LibIPC: Propagate IPC encoder errors
This propagates errors from user-defined encoders up to IPC::Connection.
There, we currently just log the error, as we aren't in a position to
propagate it further (i.e. we are inside a deferred invocation).
2023-01-04 11:49:15 +01:00
Tim Schumacher 77ea609cd3 Lagom: Fix up a missed usage of Compress::Zlib in the fuzzers 2023-01-03 23:26:45 +01:00
Ben Wiederhake 6fd478b6ce Everywhere: Remove unused includes of AK/Format.h
These instances were detected by searching for files that include
AK/Format.h, but don't match the regex:

\\b(CheckedFormatString|critical_dmesgln|dbgln|dbgln_if|dmesgln|FormatBu
ilder|__FormatIfSupported|FormatIfSupported|FormatParser|FormatString|Fo
rmattable|Formatter|__format_value|HasFormatter|max_format_arguments|out
|outln|set_debug_enabled|StandardFormatter|TypeErasedFormatParams|TypeEr
asedParameter|VariadicFormatParams|v_critical_dmesgln|vdbgln|vdmesgln|vf
ormat|vout|warn|warnln|warnln_if)\\b

(Without the linebreaks.)

This regex is pessimistic, so there might be more files that don't
actually use any formatting functions.

Observe that this revealed that Userland/Libraries/LibC/signal.cpp is
missing an include.

In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
2023-01-02 20:27:20 -05:00
Ben Wiederhake 8a331d4fa0 Everywhere: Move AK/Debug.h include to using files or remove 2023-01-02 20:27:20 -05:00
Timothy Flynn 0a7d0362ea CI: Remove extraneous toolchain job from Azure CI
This was useful when building both i686 and x86_64 SerenityOS targets as
we could use a single toolchain build for both targets. But now all this
extra job does is create the opportunity for the toolchain to need to be
built twice (i.e. if the pipelines are backed up and the toolchain cache
is busted between these jobs while the x86_64 step is waiting for a VM).
2022-12-28 15:26:12 -05:00
Liav A fb0dee5a54 Meta: Force compilation with DWARF revision 4
We currently don't support DWARF revision 5 and LLVM/Clang might create
such debug info into our binaries in x86_64, which will lead to a crash
in CrashReporter that is unable to parse that information correctly.
2022-12-28 11:53:41 +01:00
Liav A 55f17fff36 Meta: Remove i686 target 2022-12-28 11:53:41 +01:00
Nico Weber 7f52f45e9d LibGfx: Start adding a utility for handling ICC color profiles
For now, this dumps file version and device class.

https://github.com/saucecontrol/compact-icc-profiles has good
test inputs.
2022-12-27 07:44:37 -07:00
Timothy Flynn 9b483625e6 LibIPC+Everywhere: Change IPC decoders to construct values in-place
Currently, the generated IPC decoders will default-construct the type to
be decoded, then pass that value by reference to the concrete decoder.
This, of course, requires that the type is default-constructible. This
was an issue for decoding Variants, which had to require the first type
in the Variant list is Empty, to ensure it is default constructible.

Further, this made it possible for values to become uninitialized in
user-defined decoders.

This patch makes the decoder interface such that the concrete decoders
themselves contruct the decoded type upon return from the decoder. To do
so, the default decoders in IPC::Decoder had to be moved to the IPC
namespace scope, as these decoders are now specializations instead of
overloaded methods (C++ requires specializations to be in a namespace
scope).
2022-12-26 09:36:16 +01:00
Timothy Flynn 765c5b416f IPCCompiler+LibIPC: Generate message decoders with better TRY semantics
Instead of a bunch of manual error checking and returning a null OwnPtr,
we can propagate the errors up and return NonnullOwnPtr on success.
2022-12-26 09:36:16 +01:00
Timothy Flynn dc77ec733f IPCCompiler: Fix build when GENERATE_DEBUG is enabled
1. We don't need to escape the quotes in the dbgln() statements because
   these are raw strings.
2. The dbgln() statements must end with a semicolon.
2022-12-26 09:36:16 +01:00
Andrew Kaster 4ae2a54f3d CI: Bump macOS Azure runners to macos-12 tag 2022-12-25 07:58:58 -07:00
Andrew Kaster e8c2f67d47 Meta: Enable starting ladybird from serenity.sh
This solution feels really hacky and should be refined later.
2022-12-25 07:58:58 -07:00
Andrew Kaster 03edff1f39 CI: Enable ladybird builds in Lagom CI 2022-12-25 07:58:58 -07:00
Andrew Kaster 0384513779 Lagom: Add option to build Ladybird as part of a Lagom build
This means that Ladybird can be built with either Meta/Lagom or Ladybird
as the top-level source directory. This setup is a bit awkward, but will
preserve the packaging story for Ladybird until we come up with a more
permanent solution.
2022-12-25 07:58:58 -07:00
Timothy Flynn 2334b4cebd Meta: Move UCD/CLDR/TZDB downloaded artifacts to Build/caches
They currently reside under Build/<arch>, meaning that they would be
redownloaded for each architecture/toolchain build combo. Move them to a
location that can be re-used for all builds.
2022-12-24 09:46:28 -05:00
Tim Schumacher 1c630316ec CMake: Fall back to the Web Archive when downloading files 2022-12-21 09:13:50 +00:00
Andreas Kling f982400063 LibGfx: Rename TTF/TrueType to OpenType
OpenType is the backwards-compatible successor to TrueType, and the
format we're actually parsing in LibGfx. So let's call it that.
2022-12-21 08:44:22 +01:00
Timothy Flynn ed84a6f6ee LibUnicode: Use www.unicode.org domain to download emoji-test.txt
The non-www domain does not appear to be available now. We use the www
domain for UCD.zip already.

Co-authored-by: Stephan Unverwerth <s.unverwerth@serenityos.org>
2022-12-20 12:09:16 -05:00
Sam Atkins 3507679659 Lagom: Remove test apps
These are no longer useful or necessary now that a lot of different
applications run on Lagom.
2022-12-20 12:09:01 -05:00
Nico Weber c6f81b5b83 Meta: Tweak shell_include.sh
* `chmod -x` as it's for sourcing, not for executing
* Remove run line, for the same reason
* Rename it from .shell_include.sh to shell_include.sh, since e.g.
  `rg` doesn't search in hidden files by default

No behavior change.
2022-12-20 09:23:32 -05:00
Andrew Kaster c0542ed40b Meta+test262-runner: Check for signature of __assert_fail in CMake
Rather than trying to assume the only two C libraries on Linux are musl
and glibc, this solution fixes musl builds by explicitly checking for
the one C library function we are overwriting.

That being said, we should find another solution to retrieving this
error information from crashing tests. Possibly just overriding the
SIGABRT handler would work. The full solution might require checking
stderr as well as stdout in the test driver though.
2022-12-20 11:04:05 +01:00
Jelle Raaijmakers 2da1c1c10e LibGL: Generate the API wrappers
We now generate all LibGL API wrappers from a single API method
definition list stored in `GLAPI.json`. Since a significant portion of
the OpenGL API methods are relatively consistent variants, we take
advantage of this to generate a lot of these variants at once.

The autogenerated methods check for the non-nullness of the current
`GLContext`, and only perform an action if a `GLContext` is present.
This prevents a crash in ports like GLTron, who assume you can still
call the OpenGL API without an active context.

This increases our API wrapper method count from 211 to 356.

Fixes #15814.
2022-12-20 10:42:31 +01:00
MacDue 9dea61e095 Meta: Include StylePropertiesModel.cpp in lagom LibWebView 2022-12-19 11:37:56 -05:00
0xxFF c4bc889346 Documentation: Adjust default TARGET to x86_64 in the documentation 2022-12-19 09:00:34 -05:00
Nico Weber 1a85539741 Meta: Find mke2fs on macOS even if HOMEBREW_PREFIX is not set
...as long as `brew` is on the path and `brew --prefix e2fsprogs`
tells us where it is.

This is for people who don't have `"$(brew shellenv)"` in their .zshrc.
2022-12-18 17:49:49 -07:00
Nico Weber 15ce757f12 Meta: Disable gdbstub when running under HVF on macOS
Without this, `serenity.sh run` fails with

    gdbstub: current accelerator doesn't support guest debugging

on an intel mac.
2022-12-18 17:46:29 -07:00
Stephan Unverwerth 67b2f8d68d LibGLSL: Add LibGLSL
This adds a new library, LibGLSL for parsing and compiling GLSL programs
to LibGPU IR. Currently the compiler consists only of stubs.
2022-12-17 22:39:09 -07:00
Andrew Kaster 37517ba8bb Meta: Prefer clang as a host compiler to gcc 2022-12-17 22:21:43 -07:00
Andrew Kaster d2e4565f72 Meta: Allow building with Apple clang 14.x
We previously disregarded Apple clang entirely, since no released
version was able to succesfully build Lagom. Xcode 14 seems to have
all the features we need, as we haven't added any code that needs
trunk clang features in quite a while.
2022-12-17 22:21:43 -07:00
EWouters 74927ac76d Meta: Fix mke2fs on MacOS
Use `Meta/.shell_include.sh` to find the `mke2fs` executable.
2022-12-17 16:12:48 -07:00
Tim Schumacher c6d71ca727 LibCore: Rename MemoryStream to FixedMemoryStream
This is to differentiate between the upcoming `AllocatingMemoryStream`,
which automatically allocates memory as needed instead of operating on a
static memory area.
2022-12-15 13:28:29 +00:00
Linus Groh 6ae79a84df LibJS: Convert Object::construct() to NonnullGCPtr 2022-12-15 06:56:37 -05:00
Linus Groh 22089436ed LibJS: Convert Heap::allocate{,_without_realm}() to NonnullGCPtr 2022-12-15 06:56:37 -05:00
implicitfield 9665f41979 Kernel: Ignore an invalid QEMU multiboot entry
This was introduced in the QEMU commit 8504f12 and was causing the
kernel to fail to boot on the q35 machine.

Fixes #14952.
2022-12-14 17:05:06 +00:00
Andrew Kaster 733a318709 Meta+CMake: Extract Wasm spec tests into the binary directory
Clean up the Wasm spec tests CMake rules to extract and compile the wat
files into wasm files in the LibWasm binary directory instead of its
source directory. Also make the rules more robust to missing host tools,
and use more CMake install rules for the test files rather than relying
on build-root-filesystem.sh. Add some FIXMEs for later, we really
shouldn't be doing installation of test files into /home/anon at the
build-root-filesystem stage in $CURRENT_YEAR. Tests go in /usr/Tests
2022-12-14 20:29:43 +03:30
Lucas CHOLLET d3cdf151a4 Meta: Rename E2FSCK => E2FSCK_PATH
Done to increase consistency with other binaries' path.
2022-12-14 16:43:44 +00:00
Lucas CHOLLET 0fde7fe3c5 Meta: Factorize path resolution
This patch adds the `find_executable()` function that will hopefully
find executables in a distro-agnostic way and that is (hopefully as
well) easily upgradable.

The function uses some bash functionalities. So, we now require bash
for each script that includes `.shell_include.sh`.
2022-12-14 16:43:44 +00:00
Timothy Flynn d382e77d38 LibUnicode: Fix compilation when the UCD download is disabled 2022-12-14 15:24:48 +00:00
Tim Schumacher 9ec2cb3741 ConfigureComponents: Use Core::Stream to read output from whiptail 2022-12-14 15:19:35 +00:00
Baitinq 88c9e4f3b0 Meta: Don't depend on sudo for privileged operations
We previously depended on sudo's specific -E flag to keep all the
environment variables when performing a privilege escalation. We now
incorporate the -E flag into the $SUDO variable, allowing for other
privilege escalation binaries (such as doas) to be used (as long as
they preserve the current environment variables).
2022-12-14 14:38:13 +00:00
Andreas Kling 4abdb68655 LibJS: Remove Object(Object& prototype) footgun
This constructor was easily confused with a copy constructor, and it was
possible to accidentally copy-construct Objects in at least one way that
we dicovered (via generic ThrowCompletionOr construction).

This patch adds a mandatory ConstructWithPrototypeTag parameter to the
constructor to disambiguate it.
2022-12-14 15:11:57 +01:00
Andrew Kaster 16d4791212 CMake: Pass NO_POLICY_SCOPE to options cmake helpers
Tell CMake to not create a new policy scope for the
(lagom|serenity|common)_options.cmake helpers, and lets us set common
policies for both projects in common_options.cmake that actually apply
to the rest of the project, instead of just common_options.cmake itself.
2022-12-14 12:43:12 +01:00
Linus Groh a4d85cd522 LibJS: Convert Promise::create() to NonnullGCPtr 2022-12-14 09:59:45 +00:00
Linus Groh ddc6e139a6 LibJS: Convert Object::create() to NonnullGCPtr 2022-12-14 09:59:45 +00:00