jmarya/knowledge
jmarya/knowledge
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

add log

Browse source
This commit is contained in:
JMARyA 2024-02-12 21:55:34 +01:00
parent e7a85e5e00
commit b015a3e8e2
Signed by: jmarya
GPG key ID: 901B2ADDF27C2263
12 changed files with 116 additions and 63 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
technology/applications/backup/borg.md
View file

@ -16,7 +16,7 @@ All Borg commands share these options:
| Option | Description |
| ----------------------- | ------------------------------------------------------------------------------------- |
| `--info, -v, --verbose` | work on log level INFO |
| `--info, -v, --verbose` | work on [log](../../dev/Log.md) level INFO |
| `-p, --progress` | show progress information |
| `--log-json` | Output one [JSON](../../files/JSON.md) object per log line instead of formatted text. |
| `--bypass-lock` | Bypass locking mechanism |

42
technology/applications/cli/network/aria2.md
View file

@ -15,25 +15,25 @@ aria2c [<OPTIONS>] [<URI>|<MAGNET>|<TORRENT_FILE>|<METALINK_FILE>]
```
### Options
| Option | Description |
| -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `-d, --dir=<DIR>` | The directory to store the downloaded file. |
| `-i, --input-file=<FILE>` | Downloads the URIs listed in **FILE**. |
| `-l, --log=<LOG>` | The file name of the log file. If - is specified, log is written to stdout. If empty string("") is specified, or this option is omitted, no log is written to disk at all. |
| `-j, --max-concurrent-downloads=<N>` | Set the maximum number of parallel downloads for every queue item. |
| Option | Description |
| -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `-d, --dir=<DIR>` | The directory to store the downloaded file. |
| `-i, --input-file=<FILE>` | Downloads the URIs listed in **FILE**. |
| `-l, --log=<LOG>` | The file name of the [log file](../../../dev/Log.md). If - is specified, log is written to stdout. If empty string("") is specified, or this option is omitted, no log is written to disk at all. |
| `-j, --max-concurrent-downloads=<N>` | Set the maximum number of parallel downloads for every queue item. |
| `-V, --check-integrity [true/false]` | Check file integrity by validating piece hashes or a hash of entire file. This option has effect only in [BitTorrent](../../../internet/BitTorrent.md), Metalink downloads with checksums or [HTTP](../../../internet/HTTP.md)(S)/[FTP](../../../internet/FTP.md) downloads with --checksum option. If piece hashes are provided, this option can detect damaged portions of a file and re-download them. If a hash of entire file is provided, hash check is only done when file has been already download. This is determined by file length. If hash check fails, file is re-downloaded from scratch. If both piece hashes and a hash of entire file are provided, only piece hashes are used. Default: false |
| `-c, --continue [true/false]` | Continue downloading a partially downloaded file. Use this option to resume a download started by a web browser or another program which downloads files sequentially from the beginning. Currently this option is only applicable to [HTTP](../../../internet/HTTP.md)(S)/[FTP](../../../internet/FTP.md) downloads. |
| `--checksum=<TYPE>=<DIGEST>` | Set checksum. TYPE is hash type. The supported hash type is listed in Hash Algorithms in aria2c -v. DIGEST is hex digest. For example, setting sha-1 digest looks like this: `sha-1=0192ba11326fe2298c8cb4de616f4d4140213838` This option applies only to [HTTP](../../../internet/HTTP.md)(S)/[FTP](../../../internet/FTP.md) downloads. |
| `-x, --max-connection-per-server=<NUM>` | The maximum number of connections to one server for each download. Default: **1** |
| `-k, --min-split-size=<SIZE>` | aria2 does not split less than 2\*SIZE byte range. For example, let's consider downloading 20MiB file. If SIZE is 10M, aria2 can split file into 2 range (0-10MiB) and (10MiB-20MiB) and download it using 2 sources(if --split >= 2, of course). If SIZE is 15M, since 2\*15M > 20MiB, aria2 does not split file and download it using 1 source. You can append K or M (1K = 1024, 1M = 1024K). Possible Values: 1M -1024M Default: 20M |
| `-o, --out=<FILE>` | The file name of the downloaded file. It is always relative to the directory given in `--dir` option. |
| `-s, --split=<N>` | Download a file using N connections. If more than N URIs are given, first N URIs are used and remaining URIs are used for backup. If less than N URIs are given, those URIs are used more than once so that N connections total are made simultaneously. The number of connections to the same host is restricted by the `--max-connection-per-server` option. See also the `--min-split-size` option. Default: 5 |
| `-t, --timeout=<SEC>` | Set timeout in seconds. Default: 60 |
| `--check-certificate [true/false]` | Verify the peer using certificates specified in `--ca-certificate` option. Default: true |
| `--http-user=<USER>, --http-passwd=<PASSWD>` | Credentials for [HTTP](../../../internet/HTTP.md) Auth |
| `--header=<HEADER>` | Append HEADER to [HTTP](../../../internet/HTTP.md) request header. You can use this option repeatedly to specify more than one header:<br>`aria2c --header="X-A: b78" --header="X-B: 9J1" "http://host/file"` |
| `--load-cookies=<FILE>` | Load Cookies from FILE using the Firefox3 format (SQLite3), Chromium/Google Chrome (SQLite3) and the Mozilla/[Firefox](../../network/browsers/Firefox.md)(1.x/2.x)/Netscape format. |
| `--save-cookies=<FILE>` | Save Cookies to FILE in Mozilla/[Firefox](../../network/browsers/Firefox.md)(1.x/2.x)/ Netscape format. If FILE already exists, it is overwritten. Session Cookies are also saved and their expiry values are treated as 0. |
| `-U, --user-agent=<USER_AGENT>` | Set user agent for [HTTP](../../../internet/HTTP.md)(S) downloads. Default: `aria2/$VERSION`, $VERSION is replaced by package version. |
| `-S, --show-files [true/false]` | Print file listing of ".torrent", ".meta4" and ".metalink" file and exit. In case of ".torrent" file, additional information (infohash, piece length, etc) is also printed. |
| `--select-file=<INDEX>...` | (Torrent) Set file to download by specifying its index. You can find the file index using the `--show-files` option. Multiple indexes can be specified by using `,`, for example: `3,6`. You can also use `-` to specify a range: `1-5`. `,` and `-` can be used together: `1-5,8,9`. |
| `-c, --continue [true/false]` | Continue downloading a partially downloaded file. Use this option to resume a download started by a web browser or another program which downloads files sequentially from the beginning. Currently this option is only applicable to [HTTP](../../../internet/HTTP.md)(S)/[FTP](../../../internet/FTP.md) downloads. |
| `--checksum=<TYPE>=<DIGEST>` | Set checksum. TYPE is hash type. The supported hash type is listed in Hash Algorithms in aria2c -v. DIGEST is hex digest. For example, setting sha-1 digest looks like this: `sha-1=0192ba11326fe2298c8cb4de616f4d4140213838` This option applies only to [HTTP](../../../internet/HTTP.md)(S)/[FTP](../../../internet/FTP.md) downloads. |
| `-x, --max-connection-per-server=<NUM>` | The maximum number of connections to one server for each download. Default: **1** |
| `-k, --min-split-size=<SIZE>` | aria2 does not split less than 2\*SIZE byte range. For example, let's consider downloading 20MiB file. If SIZE is 10M, aria2 can split file into 2 range (0-10MiB) and (10MiB-20MiB) and download it using 2 sources(if --split >= 2, of course). If SIZE is 15M, since 2\*15M > 20MiB, aria2 does not split file and download it using 1 source. You can append K or M (1K = 1024, 1M = 1024K). Possible Values: 1M -1024M Default: 20M |
| `-o, --out=<FILE>` | The file name of the downloaded file. It is always relative to the directory given in `--dir` option. |
| `-s, --split=<N>` | Download a file using N connections. If more than N URIs are given, first N URIs are used and remaining URIs are used for backup. If less than N URIs are given, those URIs are used more than once so that N connections total are made simultaneously. The number of connections to the same host is restricted by the `--max-connection-per-server` option. See also the `--min-split-size` option. Default: 5 |
| `-t, --timeout=<SEC>` | Set timeout in seconds. Default: 60 |
| `--check-certificate [true/false]` | Verify the peer using certificates specified in `--ca-certificate` option. Default: true |
| `--http-user=<USER>, --http-passwd=<PASSWD>` | Credentials for [HTTP](../../../internet/HTTP.md) Auth |
| `--header=<HEADER>` | Append HEADER to [HTTP](../../../internet/HTTP.md) request header. You can use this option repeatedly to specify more than one header:<br>`aria2c --header="X-A: b78" --header="X-B: 9J1" "http://host/file"` |
| `--load-cookies=<FILE>` | Load Cookies from FILE using the Firefox3 format (SQLite3), Chromium/Google Chrome (SQLite3) and the Mozilla/[Firefox](../../network/browsers/Firefox.md)(1.x/2.x)/Netscape format. |
| `--save-cookies=<FILE>` | Save Cookies to FILE in Mozilla/[Firefox](../../network/browsers/Firefox.md)(1.x/2.x)/ Netscape format. If FILE already exists, it is overwritten. Session Cookies are also saved and their expiry values are treated as 0. |
| `-U, --user-agent=<USER_AGENT>` | Set user agent for [HTTP](../../../internet/HTTP.md)(S) downloads. Default: `aria2/$VERSION`, $VERSION is replaced by package version. |
| `-S, --show-files [true/false]` | Print file listing of ".torrent", ".meta4" and ".metalink" file and exit. In case of ".torrent" file, additional information (infohash, piece length, etc) is also printed. |
| `--select-file=<INDEX>...` | (Torrent) Set file to download by specifying its index. You can find the file index using the `--show-files` option. Multiple indexes can be specified by using `,`, for example: `3,6`. You can also use `-` to specify a range: `1-5`. `,` and `-` can be used together: `1-5,8,9`. |

24
technology/applications/cli/network/wget.md
View file

@ -20,19 +20,19 @@ Wget has been designed for robustness over slow or unstable network connections;
| `-e, --execute command` | Execute command as if it were a part of .wgetrc. A command thus invoked will be executed after the commands in .wgetrc, thus taking precedence over them. If you need to specify more than one wgetrc command, use multiple instances of -e. |
### Logging Options
| Option | Description |
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `-o, --output-file=logfile` | Log all messages to logfile. The messages are normally reported to standard error. |
| `-a, --append-output=logfile` | Append to logfile. This is the same as -o, only it appends to logfile instead of overwriting the old log file. |
| `-q, --quiet` | Turn off Wget's output. |
| Option | Description |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `-o, --output-file=logfile` | Log all messages to logfile. The messages are normally reported to standard error. |
| `-a, --append-output=logfile` | Append to logfile. This is the same as -o, only it appends to logfile instead of overwriting the old [log file](../../../dev/Log.md). |
| `-q, --quiet` | Turn off Wget's output. |
| `-i, --input-file=file` | Read URLs from a local or external file. If - is specified as file, URLs are read from the standard input. (Use ./- to read from a file literally named -.). If this function is used, no URLs need be present on the command line. If there are URLs both on the command line and in an input file, those on the command lines will be the first ones to be retrieved. If --force-html is not specified, then file should consist of a series of URLs, one per line. However, if you specify --force-html, the document will be regarded as [html](../../../internet/HTML.md). In that case you may have problems with relative links, which you can solve either by adding "\<base href="url">" to the documents or by specifying --base=url on the command line. If the file is an external one, the document will be automatically treated as [html](../../../internet/HTML.md) if the Content-Type matches text/html. Furthermore, the file's location will be implicitly used as base href if none was specified. |
| `-B, --base=URL` | Resolves relative links using URL as the point of reference, when reading links from an [HTML](../../../internet/HTML.md) file specified via the -i/--input-file option (together with --force-html, or when the input file was fetched remotely from a server describing it as [HTML](../../../internet/HTML.md)). This is equivalent to the presence of a "BASE" tag in the [HTML](../../../internet/HTML.md) input file, with URL as the value for the "href" attribute. |
| `-B, --base=URL` | Resolves relative links using URL as the point of reference, when reading links from an [HTML](../../../internet/HTML.md) file specified via the -i/--input-file option (together with --force-html, or when the input file was fetched remotely from a server describing it as [HTML](../../../internet/HTML.md)). This is equivalent to the presence of a "BASE" tag in the [HTML](../../../internet/HTML.md) input file, with URL as the value for the "href" attribute. |
### Download Options
| Option | Description |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `-t, --tries=number` | Set number of tries to number. Specify 0 or inf for infinite retrying. The default is to retry 20 times, with the exception of fatal errors like "connection refused" or "not found" (404), which are not retried. |
| `-O, --output-document=file` | The documents will not be written to the appropriate files, but all will be concatenated together and written to file. If - is used as file, documents will be printed to standard output, disabling link conversion. (Use ./- to print to a file literally named -.) |
| `-O, --output-document=file` | The documents will not be written to the appropriate files, but all will be concatenated together and written to file. If - is used as file, documents will be printed to standard output, disabling link conversion. (Use ./- to print to a file literally named -.) |
| `--backups=backups` | Before (over)writing a file, back up an existing file by adding a .1 suffix (\_1 on VMS) to the file name. Such backup files are rotated to .2, .3, and so on, up to `backups` (and lost beyond that) |
| `-c, --continue` | Continue getting a partially-downloaded file. This is useful when you want to finish up a download started by a previous instance of Wget, or by another program. |
| `--show-progress` | Force wget to display the progress bar in any verbosity. |
@ -41,7 +41,7 @@ Wget has been designed for robustness over slow or unstable network connections;
| `-w, --wait=seconds` | Wait the specified number of seconds between the retrievals. Use of this option is recommended, as it lightens the server load by making the requests less frequent. Instead of in seconds, the time can be specified in minutes using the "m" suffix, in hours using "h" suffix, or in days using "d" suffix. |
| `--waitretry=seconds` | If you don't want Wget to wait between every retrieval, but only between retries of failed downloads, you can use this option. Wget will use linear backoff, waiting 1 second after the first failure on a given file, then waiting 2 seconds after the second failure on that file, up to the maximum number of seconds you specify. |
| `--random-wait` | Some web sites may perform log analysis to identify retrieval programs such as Wget by looking for statistically significant similarities in the time between requests. This option causes the time between requests to vary between 0.5 and 1.5 * wait seconds, where wait was specified using the --wait option, in order to mask Wget's presence from such analysis. |
| `--user=user, --password=password` | Specify the username and password for both [FTP](../../../internet/FTP.md) and [HTTP](../../../internet/HTTP.md) file retrieval. |
| `--user=user, --password=password` | Specify the username and password for both [FTP](../../../internet/FTP.md) and [HTTP](../../../internet/HTTP.md) file retrieval. |
| `--ask-password` | Prompt for a password for each connection established. |
### Directory Options
@ -55,13 +55,13 @@ Wget has been designed for robustness over slow or unstable network connections;
| Option | Description |
| ---------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--no-cookies` | Disable the use of cookies. |
| `--load-cookies file` | Load cookies from file before the first [HTTP](../../../internet/HTTP.md) retrieval. file is a textual file in the format originally used by Netscape's cookies.txt file. |
| `--load-cookies file` | Load cookies from file before the first [HTTP](../../../internet/HTTP.md) retrieval. file is a textual file in the format originally used by Netscape's cookies.txt file. |
| `--save-cookies file` | Save cookies to file before exiting. This will not save cookies that have expired or that have no expiry time (so-called "session cookies"), but also see --keep-session-cookies. |
| `--keep-session-cookies` | When specified, causes --save-cookies to also save session cookies. Session cookies are normally not saved because they are meant to be kept in memory and forgotten when you exit the browser. Saving them is useful on sites that require you to log in or to visit the home page before you can access some pages. With this option, multiple Wget runs are considered a single browser session as far as the site is concerned. |
| `--header=header-line` | Send header-line along with the rest of the headers in each [HTTP](../../../internet/HTTP.md) request. The supplied header is sent as-is, which means it must contain name and value separated by colon, and must not contain newlines. |
| `--header=header-line` | Send header-line along with the rest of the headers in each [HTTP](../../../internet/HTTP.md) request. The supplied header is sent as-is, which means it must contain name and value separated by colon, and must not contain newlines. |
| `--proxy-user=user, --proxy-password=password` | Specify the username user and password password for authentication on a proxy server. Wget will encode them using the "basic" authentication scheme. |
| `--referer=url` | Include 'Referer: url' header in [HTTP](../../../internet/HTTP.md) request. Useful for retrieving documents with server-side processing that assume they are always being retrieved by interactive web browsers and only come out properly when Referer is set to one of the pages that point to them. |
| `-U, --user-agent=agent-string` | Identify as `agent-string` to the [HTTP](../../../internet/HTTP.md) server. |
| `--referer=url` | Include 'Referer: url' header in [HTTP](../../../internet/HTTP.md) request. Useful for retrieving documents with server-side processing that assume they are always being retrieved by interactive web browsers and only come out properly when Referer is set to one of the pages that point to them. |
| `-U, --user-agent=agent-string` | Identify as `agent-string` to the [HTTP](../../../internet/HTTP.md) server. |
### HTTPS Options
| Option | Description |

4
technology/applications/cli/tailspin.md
View file

@ -6,9 +6,9 @@ repo: https://github.com/bensadeh/tailspin
# tailspin
A log file highlighter
tailspin works by reading through a log file line by line, running a series of regexes against each line. The regexes recognize patterns like dates, numbers, severity keywords and more.
tailspin works by reading through a [log file](../../dev/Log.md) line by line, running a series of regexes against each line. The regexes recognize patterns like dates, numbers, severity keywords and more.
tailspin does not make any assumptions on the format or position of the items it wants to highlight. For this reason, it requires no configuration or setup and will work predictably regardless of the format the log file is in.
tailspin does not make any assumptions on the format or position of the items it wants to highlight. For this reason, it requires no configuration or setup and will work predictably regardless of the format the [log file](../../dev/Log.md) is in.
![Screenshot][Screenshot]

2
technology/applications/web/Authelia.md
View file

@ -82,7 +82,7 @@ server:
enable_expvars: false
## Log Configuration
## [Log](../../dev/Log.md) Configuration
log:
## Level of verbosity for logs: info, debug, trace.
level: debug

12
technology/applications/web/dufs.md
View file

@ -33,15 +33,15 @@ Usage: `dufs [OPTIONS] [serve-path]`
| `-a, --auth <rules>` | Add auth roles, e.g. `user:pass@/dir1:rw,/dir2` |
| `-A, --allow-all` | Allow all operations |
| `--allow-upload` | Allow upload files/folders |
| `--allow-delete` | Allow delete files/folders |
| `--allow-delete` | Allow delete files/folders |
| `--allow-search` | Allow search files/folders |
| `--allow-symlink` | Allow symlink to files/folders outside root directory |
| `--allow-archive` | Allow [zip](../../files/ZIP.md) archive generation |
| `--allow-archive` | Allow [zip](../../files/ZIP.md) archive generation |
| `--enable-cors` | Enable CORS, sets `Access-Control-Allow-Origin: *` |
| `--render-index` | Serve `index.html` when requesting a directory, returns 404 if not found `index.html` |
| `--render-try-index` | Serve `index.html` when requesting a directory, returns directory listing if not found `index.html` |
| `--assets <path>` | Set the path to the assets directory for overriding the built-in assets |
| `--log-format <format>` | Customize [http](../../internet/HTTP.md) log format |
| `--log-format <format>` | Customize [http](../../internet/HTTP.md) log format |
| `--tls-cert <path>` | Path to an SSL/TLS certificate to serve with HTTPS |
| `--tls-key <path>` | Path to the SSL/TLS certificate's private key |
@ -87,11 +87,11 @@ The log format can use following variables.
| -------------- | ----------------------------------------------------------------------------- |
| `$remote_addr` | client address |
| `$remote_user` | user name supplied with authentication |
| `$request` | full original request line |
| `$status` | response status |
| `$request` | full original request line |
| `$status` | response status |
| `$http_` | arbitrary request header field. examples: `$http_user_agent`, `$http_referer` |
The default log format is `'$remote_addr "$request" $status'`.
The default [log](../../dev/Log.md) format is `'$remote_addr "$request" $status'`.
```
2022-08-06T06:59:31+08:00 INFO - 127.0.0.1 "GET /" 200
```

2
technology/applications/web/traefik.md
View file

@ -723,7 +723,7 @@ accessLog: {}
```
- `filePath`:
By default access logs are written to the standard output. To write the logs into a log file, use the `filePath` option.
By default access logs are written to the standard output. To write the logs into a [log file](../../dev/Log.md), use the `filePath` option.
```yaml
accessLog:
filePath: "/path/to/access.log"

53
technology/dev/Log.md Normal file
View file

@ -0,0 +1,53 @@
---
obj: concept
aliases: ["Log", "Logging", "Log File"]
wiki: https://en.wikipedia.org/wiki/Logging_(computing)
---
# Logging
Logging is an essential aspect of software development that involves recording information about the execution of a program or system. It provides valuable insights into the behavior of the software, helps diagnose issues, and facilitates monitoring and debugging.
## Importance of Logging
Logging serves several important purposes in software development:
- **Debugging and Troubleshooting**: Logs capture information about the execution flow, errors, warnings, and exceptions, aiding developers in identifying and resolving issues.
- **Monitoring and Performance Analysis**: Logs can be used to monitor system performance, track resource usage, and analyze trends over time.
- **Auditing and Compliance**: Logs provide a record of system activities, which can be useful for auditing, compliance, and forensic analysis.
- **Security**: Logging security-related events helps detect and respond to potential security threats, such as unauthorized access attempts or suspicious activities.
## Common Logging Practices
### Logging Levels
Logging frameworks typically support different log levels, such as `DEBUG`, `INFO`, `WARN`, `ERROR`, and `FATAL`. Each level indicates the severity of the logged message, allowing developers to filter and prioritize messages based on their importance.
### Log Formatting
Logs should be formatted consistently to facilitate readability and analysis. Common log formats include plaintext, [JSON](../files/JSON.md), [XML](../files/XML.md), and key-value pairs. Formatting may include timestamps, log levels, source locations, and additional contextual information.
### Log Rotation
To manage log file size and prevent disk space exhaustion, log rotation strategies are employed. This involves periodically archiving or purging old log files and creating new ones. Log rotation may be based on size, time, or a combination of both.
### Centralized Logging
In distributed systems or large-scale applications, centralized logging solutions are used to aggregate logs from multiple sources into a central repository. This facilitates log analysis, monitoring, and troubleshooting across the entire system.
### Best Practices for Logging Implementation
- **Use Descriptive Log Messages**: Log messages should be clear, concise, and descriptive, providing useful information about the context and nature of the event.
- **Avoid Excessive Logging**: Logging too much information can clutter logs and degrade performance. Focus on logging essential events and errors while avoiding unnecessary verbosity.
- **Handle Sensitive Data Securely**: Avoid logging sensitive information such as passwords, credit card numbers, or personal data. If necessary, ensure that sensitive data is redacted or encrypted in logs.
- **Include Contextual Information**: Provide relevant contextual information in log messages, such as user IDs, session IDs, request parameters, and stack traces, to aid in troubleshooting and debugging.
- **Monitor and Analyze Logs**: Regularly monitor and analyze logs to detect anomalies, identify performance bottlenecks, and proactively address issues before they escalate.
- **Test Logging Configuration**: Verify logging configuration and behavior through automated testing to ensure that logs are generated correctly and capture the desired information.
## Log Files
Log files typically consist of structured or unstructured data, organized in a chronological sequence. The structure of log files may vary depending on the logging framework or application generating them. Common components of log files include:
- **Timestamp**: The date and time when the event or activity occurred.
- **Severity Level**: The severity or importance of the event, often indicated by log levels such as INFO, WARN, ERROR, or DEBUG.
- **Message**: A descriptive message providing details about the event, error, or activity.
- **Source**: Information about the source or origin of the event, such as the application, module, or process responsible.
There are various log [file formats](../files/File%20Formats.md) used in computing environments, each with its own structure and characteristics. Here are some common log file formats:
- **Plain Text Logs**: Plain text logs are the simplest and most widely used format. Each log entry typically consists of human-readable text, with fields separated by delimiters such as spaces, commas, or tabs. Plain text logs are easy to create, read, and parse, making them a popular choice for logging in many applications.
- **[JSON](../files/JSON.md) Logs**: Logs formatted in [JSON](../files/JSON.md) (JavaScript Object Notation) have a structured format where each log entry is represented as a [JSON](../files/JSON.md) object. [JSON](../files/JSON.md) logs include key-value pairs that provide detailed information about the event, such as timestamp, severity level, message, and additional metadata. [JSON](../files/JSON.md) logs are machine-readable and easily parsed by log analysis tools and frameworks.
- **[XML](../files/XML.md) Logs**: Logs formatted in [XML](../files/XML.md) (eXtensible Markup Language) follow a hierarchical structure where each log entry is represented as an [XML](../files/XML.md) element. [XML](../files/XML.md) logs contain tags and attributes that describe the event, along with any relevant metadata. While [XML](../files/XML.md) logs offer flexibility and extensibility, they tend to be more verbose and less human-readable compared to other formats.
- **[CSV](../files/CSV.md) Logs**: Logs formatted in [CSV](../files/CSV.md) (Comma-Separated Values) consist of rows and columns, with each log entry represented as a line of text where fields are separated by commas. [CSV](../files/CSV.md) logs are commonly used for tabular data and are suitable for logging events with a fixed set of attributes. They are easy to generate and can be opened and analyzed using spreadsheet software.
- **Syslog Format**: Syslog is a standard protocol for logging system messages, and it defines a specific format for log messages. Syslog-formatted logs consist of a header and a message body, where the header includes information such as timestamp, hostname, and severity level, while the message body contains the log message itself. Syslog is commonly used for system-level logging in Unix-like operating systems.
- **Apache Combined Log Format**: The Apache Combined Log Format is a specific log format used by the Apache HTTP Server for logging web server activity. It includes fields such as remote host, timestamp, request method, requested [URL](../internet/URL.md), [HTTP](../internet/HTTP.md) status code, and user agent. The Apache Combined Log Format is widely used for web server logging and analysis.
- **Custom Log Formats**: Many applications and logging frameworks allow users to define custom log formats tailored to their specific requirements. Custom log formats may include a combination of predefined fields, user-defined fields, and optional metadata, providing flexibility and adaptability to diverse logging needs.

28
technology/files/File Formats.md
View file

@ -5,26 +5,26 @@ obj: meta/collection
# File Formats
| Extension | Content |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `.7z` | [7z](../applications/cli/compression/p7zip.md) Archive |
| `.7z` | [7z](../applications/cli/compression/p7zip.md) Archive |
| `.apk` | - [Android](../systems/Android.md) package<br>- Package format of the [Alpine Linux](../linux/Alpine%20Linux.md) distribution |
| `.exe` | [Windows](../windows/Windows.md) Application |
| `.html` | [HTML](../internet/HTML.md) File |
| `.png` | [PNG](media/image/PNG.md) File |
| `.png` | [PNG](media/image/PNG.md) File |
| `.webp` | WebP Image |
| `.py` | [Python](../dev/programming/languages/Python.md) Code |
| `.rar` | RAR Archive |
| `.tar` | [TAR](../applications/cli/compression/tar.md) Archive |
| `.py` | [Python](../dev/programming/languages/Python.md) Code |
| `.rar` | RAR Archive |
| `.tar` | [TAR](../applications/cli/compression/tar.md) Archive |
| `.zip` | [ZIP](ZIP.md) Archive |
| `.iso` | DVD Image Format |
| `.img` | Raw Disk Image |
| `.db`<br>`.sqlite` | [SQLite](../dev/programming/SQLite.md) |
| `.sql` | [SQL](../dev/programming/languages/SQL.md) Queries |
| `.db`<br>`.sqlite` | [SQLite](../dev/programming/SQLite.md) |
| `.sql` | [SQL](../dev/programming/languages/SQL.md) Queries |
| `.pdf` | [PDF](PDF.md) File |
| `.psd` | Adobe Photoshop |
| `.xcf` | XCF File ([Gimp](../applications/media/images/GIMP.md)) |
| `.asc` | [ASCII](ASCII.md) Text |
| `.docx` | Office Open [XML](XML.md) document |
| `.log` | Log File |
| `.log` | [Log File](../dev/Log.md) |
| `.md` | [Markdown](Markdown.md) File |
| `.txt` | Text File |
| `.xml` | [XML](XML.md) File |
@ -35,20 +35,20 @@ obj: meta/collection
| `.blend` | [Blender](../applications/3d/Blender.md) File |
| `.obj` | Wavefront .obj file |
| `.desktop` | Linux [Desktop Entry](Desktop%20Entry.md) |
| `.rs` | [Rust](../dev/programming/languages/Rust.md) Code |
| `.rs` | [Rust](../dev/programming/languages/Rust.md) Code |
| `.kdbx` | [KeePass](../applications/utilities/KeePassXC.md) Database |
| `.wav` | [Wav](media/audio/WAV.md) File |
| `.flac` | [FLAC](media/audio/FLAC.md) Audio |
| `.wav` | [Wav](media/audio/WAV.md) File |
| `.flac` | [FLAC](media/audio/FLAC.md) Audio |
| `.xspf` | XML Shareable Playlist Format |
| `.m3u` | Playlist File |
| `.csv` | [Comma-seperated values](CSV.md) |
| `.mkv`<br>`.mka` | [Matroska](media/Matroska.md) |
| `.mkv`<br>`.mka` | [Matroska](media/Matroska.md) |
| `.qcow2` | [QEMU](../linux/qemu.md) copy-on-write – version 2 Qcow |
| `.json` | [JSON](JSON.md) File |
| `.yaml`<br>`.yml` | [YAML](YAML.md) File |
| `.diff` | text file differences created by [diff](../applications/cli/diff.md) and applied as updates by [patch](../applications/cli/patch.md) |
| `.part` | Partial File |
| `.torrent` | [Torrent](../internet/BitTorrent.md) File |
| `.torrent` | [Torrent](../internet/BitTorrent.md) File |
| `.lrc` | LRC (Lyrics) File |
| `.nfo` | Information Files |
| `.epub` | eBook Format |
@ -56,4 +56,4 @@ obj: meta/collection
| `.ass` | ASS (Advanced SubStation Alpha) Subtitle |
| `.cbz` | [Comic Book Archive](Comic%20Book%20Archive.md). A compressed archive format containing a collection of images or pages that make up a comic book or graphic novel. |
| `.eml` | [Email](../internet/eMail.md) Message File |
| `.toml` | [TOML](TOML.md) |
| `.toml` | [TOML](TOML.md) |

6
technology/linux/Linux.md
View file

@ -39,7 +39,7 @@ A typical Linux system has, among others, the following directories:
| `/usr/lib` | Object libraries, including dynamic libraries, plus some executables which usually are not invoked directly. More complicated programs may have whole subdirectories there. |
| `/usr/share` | This directory contains subdirectories with specific application data, that can be shared among different architectures of the same OS. |
| `/usr/share/doc` | Documentation about installed programs (optional). |
| `/usr/share/man` | [Manual pages](../applications/cli/system/man.md) go here in subdirectories according to the man page sections. |
| `/var` | This directory contains files which may change in size, such as spool and log files. |
| `/usr/share/man` | [Manual pages](../applications/cli/system/man.md) go here in subdirectories according to the man page sections. |
| `/var` | This directory contains files which may change in size, such as spool and [log](../dev/Log.md) files. |
| `/var/cache` | Data cached for programs. |
| `/var/log` | Miscellaneous log files. |
| `/var/log` | Miscellaneous [log](../dev/Log.md) files. |

2
technology/linux/UNIX Timestamp.md
View file

@ -7,7 +7,7 @@ A Unix timestamp is a way to represent a point in time, defined as the number of
## Usage
Unix timestamps are commonly used in computing systems to record and calculate time-based events. They are particularly useful for tasks such as:
- Logging events: Recording the time when an event occurs.
- [Logging](../dev/Log.md) events: Recording the time when an event occurs.
- Date arithmetic: Calculating time differences between two Unix timestamps.
- Scheduling tasks: Determining when a task should be executed based on a Unix timestamp.

2
technology/tools/Docker Compose.md
View file

@ -49,7 +49,7 @@ This is the equivalent of `docker exec` targeting a Compose service.
| `-w, --workdir` | Path to workdir directory for this command. |
### `docker-compose logs`
Displays log output from services.
Displays [log](../dev/Log.md) output from services.
#### Options
| Option | Description |