mirror of
https://github.com/systemd/systemd
synced 2024-10-15 04:24:19 +00:00
docs/JOURNAL_FILE_FORMAT: format text
This commit is contained in:
parent
e73a7d03a9
commit
a738c6d9b8
|
@ -9,7 +9,8 @@ SPDX-License-Identifier: LGPL-2.1-or-later
|
|||
|
||||
_Note that this document describes the binary on-disk format of journals only.
|
||||
For interfacing with web technologies there's the [Journal JSON Format](JOURNAL_EXPORT_FORMATS#journal-json-format).
|
||||
For transfer of journal data across the network there's the [Journal Export Format](JOURNAL_EXPORT_FORMATS#journal-export-format)._
|
||||
For transfer of journal data across the network there's the
|
||||
[Journal Export Format](JOURNAL_EXPORT_FORMATS#journal-export-format)._
|
||||
|
||||
The systemd journal stores log data in a binary format with several features:
|
||||
|
||||
|
@ -20,65 +21,62 @@ The systemd journal stores log data in a binary format with several features:
|
|||
* Support for in-line compression
|
||||
* Support for in-line Forward Secure Sealing
|
||||
|
||||
This document explains the basic structure of the file format on disk. We are
|
||||
making this available primarily to allow review and provide documentation. Note
|
||||
that the actual implementation in the [systemd
|
||||
codebase](https://github.com/systemd/systemd/blob/main/src/libsystemd/sd-journal/) is the
|
||||
only ultimately authoritative description of the format, so if this document
|
||||
and the code disagree, the code is right. That said we'll of course try hard to
|
||||
keep this document up-to-date and accurate.
|
||||
This document explains the basic structure of the file format on disk.
|
||||
We are making this available primarily to allow review and provide documentation.
|
||||
Note that the actual implementation in the
|
||||
[systemd codebase](https://github.com/systemd/systemd/blob/main/src/libsystemd/sd-journal/)
|
||||
is the only ultimately authoritative description of the format,
|
||||
so if this document and the code disagree, the code is right.
|
||||
That said we'll of course try hard to keep this document up-to-date and accurate.
|
||||
|
||||
Instead of implementing your own reader or writer for journal files we ask you
|
||||
to use the [Journal's native C
|
||||
API](https://www.freedesktop.org/software/systemd/man/sd-journal.html) to access
|
||||
these files. It provides you with full access to the files, and will not
|
||||
withhold any data. If you find a limitation, please ping us and we might add
|
||||
some additional interfaces for you.
|
||||
Instead of implementing your own reader or writer for journal files we ask you to use the
|
||||
[Journal's native CAPI](https://www.freedesktop.org/software/systemd/man/sd-journal.html)
|
||||
to access these files.
|
||||
It provides you with full access to the files, and will not withhold any data.
|
||||
If you find a limitation, please ping us and we might add some additional interfaces for you.
|
||||
|
||||
If you need access to the raw journal data in serialized stream form without C
|
||||
API our recommendation is to make use of the [Journal Export
|
||||
Format](https://systemd.io/JOURNAL_EXPORT_FORMATS#journal-export-format), which you can
|
||||
get via `journalctl -o export` or via `systemd-journal-gatewayd`. The export
|
||||
format is much simpler to parse, but complete and accurate. Due to its
|
||||
stream-based nature it is not indexed.
|
||||
If you need access to the raw journal data in serialized stream form without C API our recommendation is to make use of the
|
||||
[Journal Export Format](JOURNAL_EXPORT_FORMATS#journal-export-format),
|
||||
which you can get via `journalctl -o export` or via `systemd-journal-gatewayd`.
|
||||
The export format is much simpler to parse, but complete and accurate.
|
||||
Due to its stream-based nature it is not indexed.
|
||||
|
||||
_Or, to put this in other words: this low-level document is probably not what
|
||||
you want to use as base of your project. You want our [C
|
||||
API](https://www.freedesktop.org/software/systemd/man/sd-journal.html) instead!
|
||||
_Or, to put this in other words: this low-level document is probably not what you want to use as base of your project.
|
||||
You want our [C API](https://www.freedesktop.org/software/systemd/man/sd-journal.html) instead!
|
||||
And if you really don't want the C API, then you want the
|
||||
[Journal Export Format or Journal JSON Format](JOURNAL_EXPORT_FORMATS) instead!
|
||||
This document is primarily for your entertainment and education.
|
||||
Thank you!_
|
||||
|
||||
This document assumes you have a basic understanding of the journal concepts,
|
||||
the properties of a journal entry and so on. If not, please go and read up,
|
||||
then come back! This is a good opportunity to read about the [basic properties
|
||||
of journal
|
||||
entries](https://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html),
|
||||
in particular realize that they may include binary non-text data (though
|
||||
usually don't), and the same field might have multiple values assigned within
|
||||
the same entry.
|
||||
This document assumes you have a basic understanding of the journal concepts, the properties of a journal entry and so on.
|
||||
If not, please go and read up, then come back!
|
||||
This is a good opportunity to read about the
|
||||
[basic properties of journal entries](https://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html),
|
||||
in particular realize that they may include binary non-text data (though usually don't),
|
||||
and the same field might have multiple values assigned within the same entry.
|
||||
|
||||
This document describes the current format of systemd 246. The documented
|
||||
format is compatible with the format used in the first versions of the journal,
|
||||
This document describes the current format of systemd 246.
|
||||
The documented format is compatible with the format used in the first versions of the journal,
|
||||
but received various compatible and incompatible additions since.
|
||||
|
||||
If you are wondering why the journal file format has been created in the first
|
||||
place instead of adopting an existing database implementation, please have a
|
||||
look [at this
|
||||
thread](https://lists.freedesktop.org/archives/systemd-devel/2012-October/007054.html).
|
||||
If you are wondering why the journal file format has been created in the first place instead of adopting an existing database implementation,
|
||||
please have a look [at this thread](https://lists.freedesktop.org/archives/systemd-devel/2012-October/007054.html).
|
||||
|
||||
|
||||
## Basics
|
||||
|
||||
* All offsets, sizes, time values, hashes (and most other numeric values) are 32-bit/64-bit unsigned integers in LE format.
|
||||
* Offsets are always relative to the beginning of the file.
|
||||
* The 64-bit hash function siphash24 is used for newer journal files. For older files [Jenkins lookup3](https://en.wikipedia.org/wiki/Jenkins_hash_function) is used, more specifically `jenkins_hashlittle2()` with the first 32-bit integer it returns as higher 32-bit part of the 64-bit value, and the second one uses as lower 32-bit part.
|
||||
* The 64-bit hash function siphash24 is used for newer journal files.
|
||||
For older files [Jenkins lookup3](https://en.wikipedia.org/wiki/Jenkins_hash_function) is used,
|
||||
more specifically `jenkins_hashlittle2()` with the first 32-bit integer it returns as higher 32-bit part of the 64-bit value,
|
||||
and the second one uses as lower 32-bit part.
|
||||
* All structures are aligned to 64-bit boundaries and padded to multiples of 64-bit
|
||||
* The format is designed to be read and written via memory mapping using multiple mapped windows.
|
||||
* All time values are stored in usec since the respective epoch.
|
||||
* Wall clock time values are relative to the Unix time epoch, i.e. January 1st, 1970. (`CLOCK_REALTIME`)
|
||||
* Monotonic time values are always stored jointly with the kernel boot ID value (i.e. `/proc/sys/kernel/random/boot_id`) they belong to. They tend to be relative to the start of the boot, but aren't for containers. (`CLOCK_MONOTONIC`)
|
||||
* Monotonic time values are always stored jointly with the kernel boot ID value (i.e. `/proc/sys/kernel/random/boot_id`) they belong to.
|
||||
They tend to be relative to the start of the boot, but aren't for containers. (`CLOCK_MONOTONIC`)
|
||||
* Randomized, unique 128-bit IDs are used in various locations. These are generally UUID v4 compatible, but this is not a requirement.
|
||||
|
||||
## General Rules
|
||||
|
|
Loading…
Reference in a new issue