Commit graph

842 commits

Author SHA1 Message Date
Andreas Kling ab549cf942 LibGUI+TextEditor+HackStudio: Add GML syntax highlighter :^) 2020-12-21 18:26:18 +01:00
Brendan Coles 9ca34c3047 UserspaceEmulator: Implement profiling, disown, purge syscalls 2020-12-21 09:57:51 +01:00
William Marlow 39364bdda4 Build: Embed application icons directly in the executables.
New serenity_app() targets can be defined which allows application
icons to be emedded directly into the executable. The embedded
icons will then be used when creating an icon for that file in
LibGUI.
2020-12-21 00:12:59 +01:00
Lenny Maiorani 765936ebae
Everywhere: Switch from (void) to [[maybe_unused]] (#4473)
Problem:
- `(void)` simply casts the expression to void. This is understood to
  indicate that it is ignored, but this is really a compiler trick to
  get the compiler to not generate a warning.

Solution:
- Use the `[[maybe_unused]]` attribute to indicate the value is unused.

Note:
- Functions taking a `(void)` argument list have also been changed to
  `()` because this is not needed and shows up in the same grep
  command.
2020-12-21 00:09:48 +01:00
Andreas Kling 71d92cef17 UserspaceEmulator: Add lazy caching of debug info for shared libraries
Keep the debug symbols for shared libraries in memory after we opened
them the first time. This dramatically speeds up symbolication of
backtraces when running dynamically linked programs in UE.
2020-12-20 15:45:39 +01:00
Andreas Kling 0e4ecca336 LibIPC: Remove use of ByteBuffer::wrap()
ByteBuffer::wrap() was useful before we had Span. Let's see if we can't
get rid of some more ByteBuffer wrapping.
2020-12-19 11:30:02 +01:00
Sahan Fernando 354a1a7891 HackStudio: Remove duplicate auto-indentation in HackStudio Language Server 2020-12-18 16:25:42 +01:00
Brendan Coles c8fb00fe4d UserspaceEmulator: Implement geteuid, getegid, ptsname syscalls 2020-12-17 00:02:03 +01:00
Brendan Coles a46e48089d UserspaceEmulator: Implement beep syscall 2020-12-16 17:27:20 +01:00
Andreas Kling d410449d87 HackStudio: Don't take LexicalPath as argument
Also make use of LexicalPath::has_extension() in one place.
2020-12-15 11:58:47 +01:00
Itamar 72ca45e300 UserspaceEmulator: Support dynamically loaded programs
When loading dynamic objects, the emulator loads the interpreter,
generates an auxiliary vector and starts executing the loader.

Additionally, this commits also makes the MallocTracer and backtrace
symbolication work for dynamically loaded programs.
2020-12-14 23:05:53 +01:00
Itamar 28cda567c1 UserspaceEmulator: Support the name argument in mmap for file backed mappings 2020-12-14 23:05:53 +01:00
Itamar d2262b8f6d UserspaceEmulator: Update memory protection of underlying pages
If a MmapRegion is file backed, we need to call mprotect on its
underlying pages.
2020-12-14 23:05:53 +01:00
Itamar d89858f42a Profiler: Symbolicate samples using coredump file 2020-12-14 23:05:53 +01:00
Andreas Kling dd3e6451ac HackStudio: Rethink the "project" concept to be about a directory
Instead of having .hsp files that determine which files are members
of a project, a project is now an entire directory tree instead.

This feels a lot less cumbersome to work with, and removes a fair
amount of busywork that would otherwise be expected from the user.

This patch refactors large parts of HackStudio to implement the new
way of thinking. I've probably missed some details here and there,
but generally I think it's pretty OK.
2020-12-10 20:42:11 +01:00
Andreas Kling e9280cba13 HackStudio: Don't vend invalid indices from SearchResultsModel
This fixes an assertion when clicking in the "find in files" search
results table when there were no matches.
2020-12-10 20:42:11 +01:00
Ben Wiederhake ebbae015ef UserspaceEmulator: Fix page determination
Note that this *only* affects the last few bytes of the ELF headers, since all
other allocations are PAGE_SIZE aligned.
2020-12-10 20:41:12 +01:00
Ben Wiederhake 809a8ee693 UserspaceEmulator: Implement readlink syscall 2020-12-08 09:37:30 +01:00
Ben Wiederhake 88b090a808 UserspaceEmulator: Handle overflow in virt$realpath 2020-12-08 09:37:30 +01:00
Emanuel Sprung 3b7884ee8a TextEditor: Add button to match regular expression during search 2020-11-27 21:32:41 +01:00
Andreas Kling a43aa82d69 Profiler: Fix assertion when all function samples hit one instruction
If the percentage is 100, we were trying to get the heat gradient pixel
at (100, 0), which was one pixel past the end. Fix this by making the
heat gradient 101 pixels wide :^)
2020-11-24 22:02:34 +01:00
Simon Danner 751e759be2 UserspaceEmulator: Implement clock_nanosleep 2020-11-23 18:41:42 +01:00
Sergey Bugaev e7e179212c HackStudio: Send an open file to language servers
Language servers will now receive an open file instead of just its path. This
means the language servers no longer need to access the filesystem to open the
file themselves.

The C++ language server now has no filesystem access whatsoever (although we
might need to relax this in the future if it learns to complete #include paths),
while the Shell language server can read /etc/passwd (it wants that in order to
get the user's home directory) and browse (but not read!) the whole file system
tree for completing paths.
2020-11-23 18:37:40 +01:00
Sergey Bugaev 23dc3ff0c2 LibIPC: Support sending file descriptors :^)
It is now possible to use the special IPC::File type in message arguments. In
C++, the type is nothing more than a wrapper over a file descriptor. But when
serializing/deserializing IPC::File arguments, LibIPC will use the sendfd/recvfd
kernel APIs instead of sending the integer inline.

This makes it quite convenient to pass files over IPC, and will allow us to
significantly tighten sandboxes in the future :^)

Closes https://github.com/SerenityOS/serenity/issues/3643
2020-11-23 18:37:40 +01:00
Sergey Bugaev fa2e3e2be4 LibIPC: Prepend each message with its size
This makes it much simpler to determine when we've read a complete message, and
will make it possible to integrate recvfd() in the future commit.
2020-11-23 18:37:40 +01:00
Simon Danner 09b095e62a UserspaceEmulator: Add support for watch_file 2020-11-22 10:53:58 +01:00
Andreas Kling da413a464a UserspaceEmulator: Inline some very hot functions
This improves the browser's load time on welcome.html by ~2%.
2020-11-19 21:46:01 +01:00
Andreas Kling 1965fc5b98 UserspaceEmulator: Keep Emulator& closer to the action in some places
This avoids the cost of calling Emulator::the() in some very hot paths.
2020-11-16 15:11:02 +01:00
Andreas Kling d14695f823 UserspaceEmulator: Cache the region we're executing code from
Instead of caching a raw pointer to the next instruction, cache the
region we're fetching instructions from, and a pointer to its base.
This way we don't need to keep invalidating and reloading the cache
whenever the CPU jumps.
2020-11-16 15:11:02 +01:00
Andreas Kling b4ff85f138 UserspaceEmulator: Reduce malloc thrashing in backtrace capture 2020-11-16 15:11:02 +01:00
Andreas Kling e1f617950e UserspaceEmulator: Make big malloc block lookup O(1) as well
By passing the Region& to the auditing functions, we know exactly which
block we are hitting. This allows us to track big mallocations the same
way we already do chunked ones.

This gets rid of the O(n) scan in find_mallocation() for allocations
larger than the maximum malloc chunk size. :^)
2020-11-16 15:11:02 +01:00
Andreas Kling 8d9dd4c518 UserspaceEmulator: Make Region a top-level class 2020-11-16 15:11:02 +01:00
Andreas Kling 3c64cec4d7 UserspaceEmulator: Devirtualize read/write/execute region permissions
These are getting quite hot (~4% of general emulation profile combined)
so let's just devirtualize them and turn the function calls into simple
boolean checks.
2020-11-16 09:44:30 +01:00
Andreas Kling f41b9946e2 UserspaceEmulator: Hang malloc metadata on malloc block MmapRegions
Instead of tracking known malloc blocks in a separate hash table,
add an optional malloc metadata pointer to MmapRegion.

This makes finding the malloc metadata for a given pointer extremely
fast since it can piggyback on the page table array. :^)
2020-11-16 09:33:30 +01:00
Andreas Kling fe7036d8f4 UserspaceEmulator: Improve error text on jump to non-executable memory
The memory is non-executable, not non-readable. :^)
2020-11-16 09:10:49 +01:00
Nico Weber 6252e5b1f6 UserspaceEmulator: Implement 32-bit FIDIV, 16-bit FIADD/FISUB/FISUBR/FIDIV/FIDIVR/FISTP, and fix 32-bit FIADD
Not motivated by anything in particular, they just looked easy to fill
in. With this, all arithmetic FI* FPU instructions are implemented.

Switch to the mXXint style in a few more functions, this part is no-op.
2020-11-15 20:16:55 +01:00
Andreas Kling 6dab0af9af UserspaceEmulator: Add a fast path for forward REP STOSD
Same as REP STOSB, except for 32-bit fills instead.
2020-11-15 18:09:15 +01:00
Andreas Kling 102e1d330c UserspaceEmulator: Add a fast path for forward REP STOSB
This is used by memset() so we get a lot of mileage out of optimizing
this instruction.

Note that we currently audit every individual byte accessed separately.
This could be greatly improved by adding a range auditing mechanism to
MallocTracer.
2020-11-15 18:09:08 +01:00
Andreas Kling 92e152f11d UserspaceEmulator: Add virtual data()/shadow_data() accessors to Region 2020-11-15 18:09:04 +01:00
Andreas Kling 59b4874443 UserspaceEmulator: Add a page-address-to-MMU-region lookup map
To make SoftMMU::find_region() O(1), this patch invests 3MiB into a
lookup table where we track each possible page base address and map
them to the SoftMMU::Region corresponding to that address.

This is another large improvement to general emulation performance. :^)
2020-11-15 18:08:59 +01:00
Andreas Kling a4a389156d UserspaceEmulator: Make sure the (crappy) VM allocator is page-aligned
We don't want the next_address pointer losing its alignment somehow.
This whole thing should be replaced at some point, since UE hosted
programs won't be able to run forever with this allocation strategy.
2020-11-15 18:08:56 +01:00
Andreas Kling adabcf24ec Everywhere: Add missing <AK/ByteBuffer.h> includes
All of these files were getting ByteBuffer.h from someone else and then
using it. Let's include it explicitly.
2020-11-15 13:11:21 +01:00
Nico Weber be73f9f544 UserspaceEmulator: Implement FISUBR_RM32
Fixes crash when playing `ue Breakout` with the mouse.
2020-11-15 09:36:37 +01:00
Andreas Kling 12d923bb7e UserspaceEmulator: Fix some FPU instructions' handling of RM32/RM64
m32int is a 32-bit integer stored in memory, and should not be mistaken
for a floating point number. :^)

Also add missing handling of 64-bit FPU register operands to some of
the RM64 instructions.
2020-11-14 23:47:50 +01:00
Andreas Kling 647e92b74f UserspaceEmulator: Skip destroying the Emulator object on shutdown
There are some destruction order races that can cause hangs while
shutting down UE. Since there's no particular value right now in
destroying the Emulator object properly, just avoid destruction and
add a FIXME about looking into it later.
2020-11-14 23:39:44 +01:00
Andreas Kling b5b535aa81 UserspaceEmulator: Untaint flags in FCOMI and FUCOMI for now
This makes UE logging bearable until we can get proper shadow data
support for the FPU stack.
2020-11-14 23:11:07 +01:00
Andreas Kling 8fd97bee7f UserspaceEmulator: Forget ChunkedBlocks after they are munmap()'ed
This is not ideal since we lose free() backtraces, but it will require
some thinking to get this right.
2020-11-14 23:07:07 +01:00
Andreas Kling 2066f48b87 UserspaceEmulator: Avoid one hash lookup in target_did_malloc() 2020-11-14 22:52:07 +01:00
Andreas Kling d88b36448b UserspaceEmulator: Add some helpers to tidy up TrackedChunkedBlock 2020-11-14 22:52:04 +01:00
Andreas Kling 2fceffff6f UserspaceEmulator: Track malloc ChunkedBlocks for faster auditing
Instead of doing an O(n) scan over all the mallocations whenever we're
doing a read/write audit, UE now keeps track of ChunkedBlocks and their
chunks. Both the block lookup and the chunk lookup is O(1).

We know what ChunkedBlocks look like via mallocdefs.h from LibC.

Note that the old linear scan is still in use for big mallocations,
but the vast majority of mallocations are chunked, so this helps a lot.

This makes malloc auditing significantly faster! :^)
2020-11-14 22:51:58 +01:00