Commit graph

3540 commits

Author SHA1 Message Date
Andreas Kling edac1d6748 Make PageDirectory store physical pages in a HashMap.
This container is really just there to keep a retain on the individual
PhysicalPages for each page table. A HashMap does the job with far greater
space efficiency.
2018-12-31 15:10:12 +01:00
Andreas Kling ab72666f48 Plug leaks in SynthFS::remove_file().
The process spawn stress test can now run forever. :^)
2018-12-28 03:09:45 +01:00
Andreas Kling 3f3535213b Fix some issues uncovered by the spawn stress test. 2018-12-26 22:02:24 +01:00
Andreas Kling 7bc41532be Remove FS::read_entire_inode() in favor of Inode::read_entire(). 2018-12-21 17:45:42 +01:00
Andreas Kling ec1c487dcd Yet another pass of style fixes. 2018-12-21 02:10:45 +01:00
Andreas Kling d0f06e5f3f Automatically call Inode::flush_metadata() before an Inode is destroyed.
Use a little template magic to have Retainable::release() call out to
T::will_be_destroyed() if such a function exists before actually calling
the destructor. This gives us full access to virtual functions in the
pre-destruction code.
2018-12-19 22:28:09 +01:00
Andreas Kling d2bb139c46 Support inserting a newline. 2018-12-05 01:43:07 +01:00
Andreas Kling ca6847b5bb Import a simple text editor I started working on. 2018-12-04 00:27:16 +01:00
Andreas Kling aff89d2fd7 Yet more coding style fixes. 2018-12-03 01:38:22 +01:00
Andreas Kling f31e1ceab0 Move InlineLinkedList to AK. 2018-12-03 01:18:54 +01:00
Andreas Kling e7cc08226f Implement basic support for times().
The kernel now bills processes for time spent in kernelspace and userspace
separately. The accounting is forwarded to the parent process in reap().

This makes the "time" builtin in bash work.
2018-12-03 01:14:19 +01:00
Andreas Kling f6e27c2abe More coding style changes. 2018-12-03 00:39:25 +01:00
Andreas Kling fb8a1186f6 Use decltype(sizeof(void*)) as a facsimile for std::size_t.
Clang demands that the size argument to the various operator new()'s
to be exactly whatever it thinks std::size_t is.

Since we can't include STL headers, this little trick will do.
2018-12-02 23:48:19 +01:00
Andreas Kling 85b886c2e0 Make it possible to build the Kernel on a macOS host.
It still requires an ELF compiler and linker, but at least it builds.
I need to get rid of the "Unix" namespace. This does a lot of that.
2018-12-02 23:34:50 +01:00
Andreas Kling d90104f9e0 Let reap() communicate the dead process's exit status to the caller.
This way the scheduler doesn't need to plumb the exit status into the waiter.
We still plumb the waitee pid though, I don't love it but it can be fixed.
2018-11-28 22:01:24 +01:00
Andreas Kling f5a0b6961c Fix mkdir with relative paths. 2018-11-18 23:28:43 +01:00
Andreas Kling de4604ac95 Finally hook up the mkdir code to a syscall.
Added a /bin/mkdir that makes directories. How very neat :^)
There are various limitations because of missing functionality.
2018-11-18 15:02:16 +01:00
Andreas Kling d2046e79cf Add a DoubleBuffer thingy to allow TTY read/write to be interleaved.
I feel like this concept might be useful in more places. It's very naive
right now and uses dynamically growing buffers. It should really use static
size buffers and never kmalloc().
2018-11-16 17:57:00 +01:00
Andreas Kling 10c470e95f Make page_in_from_vnode 2x faster.
...by adding a new class called Ext2Inode that inherits CoreInode.
The idea is that a vnode will wrap a CoreInode rather than InodeIdentifier.
Each CoreInode subclass can keep whatever caches they like.

Right now, Ext2Inode caches the list of block indices since it can be very
expensive to retrieve.
2018-11-13 13:02:39 +01:00
Andreas Kling 97c799576a Add close-on-exec flag for file descriptors.
I was surprised to find that dup()'ed fds don't share the close-on-exec flag.
That means it has to be stored separately from the FileDescriptor object.
2018-11-13 01:36:31 +01:00
Andreas Kling 19b9401487 Reduce kmalloc() traffic in directory iteration.
Pass the file name in a stack-allocated buffer instead of using an AK::String
when iterating directories. This dramatically reduces the amount of cycles
spent traversing the filesystem.
2018-11-13 00:17:30 +01:00
Andreas Kling f1404aa948 Add primitive FIFO and hook it up to sys$pipe().
It's now possible to do this in bash:

cat kernel.map | fgrep List

This is very cool! :^)
2018-11-12 01:28:46 +01:00
Andreas Kling d5d45d1088 Rage hacking to get bash to run. It finally runs. So cool! :^) 2018-11-11 15:38:07 +01:00
Andreas Kling c653bb09b3 Don't include empty segments in canonicalized paths.
This makes "/foo/" and "/foo" both resolve to "/foo".
2018-11-10 15:46:39 +01:00
Andreas Kling b8264e7d47 Merge Disk namespace into the IDEDiskDevice class. 2018-11-10 15:15:31 +01:00
Andreas Kling 4914f3b837 Build LibC and Userland with clang as well. 2018-11-09 14:29:00 +01:00
Andreas Kling ebf308d413 Make kernel build with clang.
It's a bit faster than g++ and seems to generate perfectly fine code.
The kernel is also roughly 10% smaller(!)
2018-11-09 12:22:31 +01:00
Andreas Kling e9cdb6bb9b Fix all current build warnings in LibC. 2018-11-09 10:09:46 +01:00
Andreas Kling e71cb1c56b Fix some paging related bugs exposed by the spawn stress test.
- Process::exec() needs to restore the original paging scope when called
  on a non-current process.
- Add missing InterruptDisabler guards around g_processes access.
- Only flush the TLB when modifying the active page tables.
2018-11-09 01:25:31 +01:00
Andreas Kling 3b2dcd5929 Add a VMO pointer to VNode.
This way, if anyone tries to map an already mapped file, we share the VMO.
2018-11-08 15:39:26 +01:00
Andreas Kling fdbd9f1e27 Start working on memory-mapped files.
First of all, change sys$mmap to take a struct SC_mmap_params since our
sycsall calling convention can't handle more than 3 arguments.

This exposed a bug in Syscall::invoke() needing to use clobber lists.
It was a bit confusing to debug. :^)
2018-11-08 11:40:58 +01:00
Andreas Kling 71bffa9864 Fix whiny build. 2018-11-07 16:38:45 +01:00
Andreas Kling 981a3ae4b3 Make VFS test environment build again. 2018-11-07 12:05:51 +01:00
Andreas Kling a7f1d892a9 Add some basic setgroups(), getgroups() and initgroups().
Also teach /bin/id to print the user's supplemental groups.
2018-11-07 01:38:51 +01:00
Andreas Kling 8135952832 Add a Chomp feature to String construction that removes a trailing newline.
This will be useful in many situations.
2018-11-07 00:19:35 +01:00
Andreas Kling 2d045d2a64 Implement COW pages! :^)
sys$fork() now clones all writable regions with per-page COW bits.
The pages are then mapped read-only and we handle a PF by COWing the pages.

This is quite delightful. Obviously there's lots of work to do still,
and it needs better data structures, but the general concept works.
2018-11-05 13:48:07 +01:00
Andreas Kling 72cdc62155 Replace zones with individually tracked physical pages.
It's just a simple struct { ref_count, paddr }.
This will allow me to implement lazy zeroing and COW pages.
2018-11-05 10:23:00 +01:00
Andreas Kling b59ce22fc5 Fix dumb-but-hard-to-find bug in paging.
This was the fix:

-process.m_page_directory[0] = m_kernel_page_directory[0];
-process.m_page_directory[1] = m_kernel_page_directory[1];
+process.m_page_directory->entries[0] = m_kernel_page_directory->entries[0];
+process.m_page_directory->entries[1] = m_kernel_page_directory->entries[1];

I spent a good two hours scratching my head, not being able to figure out why
user process page directories felt they had ownership of page tables in the
kernel page directory.

It was because I was copying the entire damn kernel page directory into
the process instead of only sharing the two first PDE's. Dang!
2018-11-03 00:35:57 +01:00
Andreas Kling 065f0aee35 Preallocate the maximum number of FileHandle pointers (fds) in every process.
This could even use a more specific data structure since it doesn't need the
grow/shrink capabilities of a vector.
2018-11-01 13:39:28 +01:00
Andreas Kling 9a086b2d35 Add a kmalloc_eternal() for things that will never be destroyed. 2018-10-31 23:19:15 +01:00
Andreas Kling a8f36f72a8 printfing a number or string bigger than the field width should not crash. 2018-10-31 22:40:10 +01:00
Andreas Kling 9886b27d9c Add getpwent() family of functions to LibC.
Also add a little /etc/passwd database. There's just me in there.
2018-10-31 19:54:25 +01:00
Andreas Kling 8e640539ef Add an inode metadata cache to the ext2fs implementation. 2018-10-29 23:45:34 +01:00
Andreas Kling e6284a8774 Fix broken SpinLock.
The SpinLock was all backwards and didn't actually work. Fixing it exposed
how wrong most of the locking here is.

I need to come up with a better granularity here.
2018-10-29 22:04:26 +01:00
Andreas Kling c76dc9a047 Add /proc/mm and a /bin/mm utility that just dumps it.
This shows some info about the MM. Right now it's just the zone count
and the number of free physical pages. Lots more can be added.

Also added "exit" to sh so we can nest shells and exit from them.

I also noticed that we were leaking all the physical pages, so fixed that.
2018-10-28 10:28:21 +01:00
Andreas Kling e904f193c1 Canonicalize the path used by sh.
With a bunch of LibC work to support the feature. LibC now initializes
AK::StringImpl by default. It's now fine to use AK in LibC/Userland! :^)
2018-10-28 09:36:21 +01:00
Andreas Kling 88ad59bfb1 Add a simple FileSystemPath class that can canonicalize paths.
Also a simple StringBuilder to help him out.
2018-10-28 08:54:20 +01:00
Andreas Kling 8289a5c93c Implement 'H' and 'J' escape sequences. 2018-10-27 23:42:20 +02:00
Andreas Kling cc7e3519a6 Add a /bin/clear that prints the clear terminal escape sequence.
It doesn't work yet, but it will!
2018-10-27 17:39:08 +02:00
Andreas Kling 8f91a47aeb Add some basic field width support to printf().
Use it to make "ls" output a bit better. Also sys$spawn now fails with EACCES
if the path is not a file that's executable by the current uid/gid.
2018-10-27 16:43:03 +02:00
Andreas Kling 601d0d1739 Better int hashing. This was going to bite me sooner or later. 2018-10-27 09:33:24 +02:00
Andreas Kling 2716a9e2d7 Greatly improve /proc/PID/stack by tracing the ebp frame chain.
I also added a generator cache to FileHandle. This way, multiple
reads to a generated file (i.e in a synthfs) can transparently
handle multiple calls to read() without the contents changing
between calls.

The cache is discarded at EOF (or when the FileHandle is destroyed.)
2018-10-27 00:14:24 +02:00
Andreas Kling a32b3a3ddf Implement /proc/PID/vm.
Refactored SyntheticFileSystem to maintain an arbitrary directory structure.
ProcFileSystem creates a directory entry in /proc for each new process.
2018-10-26 17:44:19 +02:00
Andreas Kling 53abfa7ea1 Add sys$gethostname and /bin/hostname 2018-10-26 09:54:29 +02:00
Andreas Kling fdc782c1d1 Add a very naive block cache to the DiskBackedFileSystem.
This would be a lot better as an LRU. Right now it's a 32-slot
hash table with random eviction.
2018-10-25 12:36:50 +02:00
Andreas Kling 16f318408d ELFLoader should fail with an error message for unresolved symbols. 2018-10-25 10:02:04 +02:00
Andreas Kling ec1d16b307 Add a "pwd" utility to userland.
It's implemented as a separate process. How cute is that.
Tasks now have a current working directory. Spawned tasks inherit their
parent task's working directory.
Currently everyone just uses "/" as there's no way to chdir().
2018-10-24 14:28:22 +02:00
Andreas Kling bca4b71bfa Lots of hacking to make a very simple "ls" utility.
I added a dead-simple malloc that only allows allocations < 4096 bytes.
It just forwards the request to mmap() every time.

I also added simplified versions of opendir() and readdir().
2018-10-24 12:50:07 +02:00
Andreas Kling 82dae8fc90 Add a kmalloc lock. This definitely reduces flakiness. 2018-10-24 00:51:19 +02:00
Andreas Kling 018da1be11 Generalize the SpinLock and move it to AK.
Add a separate lock to protect the VFS. I think this might be a good idea.
I'm not sure it's a good approach though. I'll fiddle with it as I go along.

It's really fun to figure out all these things on my own.
2018-10-23 23:34:05 +02:00
Andreas Kling ed2422d7af Start adding a basic /proc filesystem and a "ps" utility. 2018-10-23 12:04:03 +02:00
Andreas Kling fe237ee215 Lots of hacking:
- Turn Keyboard into a CharacterDevice (85,1) at /dev/keyboard.
- Implement MM::unmapRegionsForTask() and MM::unmapRegion()
- Save SS correctly on interrupt.
- Add a simple Spawn syscall for launching another process.
- Move a bunch of IO syscall debug output behind DEBUG_IO.
- Have ASSERT do a "cli" immediately when failing.
  This makes the output look proper every time.
- Implement a bunch of syscalls in LibC.
- Add a simple shell ("sh"). All it can do now is read a line
  of text from /dev/keyboard and then try launching the specified
  executable by calling spawn().

There are definitely bugs in here, but we're moving on forward.
2018-10-23 10:12:50 +02:00
Andreas Kling c8b7173aa8 Add a CircularQueue template class to AK. 2018-10-22 22:46:02 +02:00
Andreas Kling b824f15619 Launching an arbitrary ELF executable from disk works! :^)
This is so cool! It's a bit messy now with two Task constructors,
but eventually they should fold into a single constructor somehow.
2018-10-22 15:43:02 +02:00
Andreas Kling 63764b3a65 Import very modest Userland. 2018-10-22 14:06:22 +02:00
Andreas Kling 85bcf2ed0f Add getgid() and getpid() syscalls. Prep for LibC. 2018-10-22 13:55:11 +02:00
Andreas Kling 702d308e67 Oops, StringImpl's "the empty string" global was not always initialized.
These "oops forgot to initialize" bugs are getting annoying...
2018-10-22 13:10:08 +02:00
Andreas Kling c149d2a8f0 Build ELFLoader into Kernel. 2018-10-18 15:03:10 +02:00
Andreas Kling 347e202f7f Add size_t and ssize_t to kernel builds. 2018-10-17 16:47:19 +02:00
Andreas Kling 7580ac576f Make VFS host build work again. 2018-10-17 12:23:19 +02:00
Andreas Kling d2425495ca VirtualFileSystem class builds inside Kernel. 2018-10-17 11:40:58 +02:00
Andreas Kling 9171521752 Integrate ext2 from VFS into Kernel. 2018-10-17 10:57:23 +02:00
Andreas Kling aec8ab0a60 Add the basic character devices to kernel. 2018-10-16 14:33:16 +02:00
Andreas Kling 5d465582a3 Start fixing up AK to work inside the kernel. 2018-10-16 13:59:28 +02:00
Andreas Kling 1203c327c7 Merge some features from gerbert into OwnPtr and RetainPtr. 2018-10-16 12:21:06 +02:00
Andreas Kling fd708a4cb1 Reduce dependence on STL. 2018-10-16 12:11:27 +02:00
Andreas Kling 0c1a4e8de3 Add String::substring(). 2018-10-16 12:07:17 +02:00
Andreas Kling 39444c5916 Fix HashTable::find() return iterator for items found in non-0 buckets. 2018-10-14 22:08:36 +02:00
Andreas Kling 560405667e Start using WeakPtr for some of the WindowManager window pointers. 2018-10-13 17:00:45 +02:00
Andreas Kling 3e9a45d7f4 Add WeakPtr/Weakable templates. 2018-10-13 15:41:24 +02:00
Andreas Kling b7efd92937 Make SinglyLinkedList destruction actually work. 2018-10-13 14:29:00 +02:00
Andreas Kling 969334505d Add HashMap::remove(). 2018-10-13 14:23:47 +02:00
Andreas Kling c2ef54c044 Add HashTable::remove() and fix a bug where ConstIterator would skip the first. 2018-10-13 14:22:09 +02:00
Andreas Kling f794190de0 Add a DoublyLinkedList template and start using it for HashTable. 2018-10-13 13:50:44 +02:00
Andreas Kling 7777c8844b Add Vector::remove(). 2018-10-13 01:17:36 +02:00
Andreas Kling 750b27cb07 Ext2FileSystem::readInode() should return an empty buffer for 0-length files. 2018-10-13 00:50:35 +02:00
Andreas Kling c37ded0ae4 Support arbitrary pointer traits. 2018-10-11 16:52:30 +02:00
Andreas Kling f282df6617 AK::Buffer should not free unowned backing stores, duh. 2018-10-11 12:30:35 +02:00
Andreas Kling a181a8f6e7 Run without SimpleMalloc locally for now. 2018-10-10 15:11:43 +02:00
Andreas Kling 5a30055157 Import all this stuff into a single repo called Serenity. 2018-10-10 11:53:07 +02:00