jmarya/knowledge
jmarya/knowledge
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

add log

Browse source
This commit is contained in:
JMARyA 2024-02-12 21:55:34 +01:00
parent e7a85e5e00
commit b015a3e8e2
Signed by: jmarya
GPG key ID: 901B2ADDF27C2263
12 changed files with 116 additions and 63 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
technology/applications/backup/borg.md
View file

@ -16,7 +16,7 @@ All Borg commands share these options:
| Option | Description |
| ----------------------- | ------------------------------------------------------------------------------------- |
| `--info, -v, --verbose` | work on log level INFO |
| `--info, -v, --verbose` | work on [log](../../dev/Log.md) level INFO |
| `-p, --progress` | show progress information |
| `--log-json` | Output one [JSON](../../files/JSON.md) object per log line instead of formatted text. |
| `--bypass-lock` | Bypass locking mechanism |

4
technology/applications/cli/network/aria2.md
View file

@ -16,10 +16,10 @@ aria2c [<OPTIONS>] [<URI>|<MAGNET>|<TORRENT_FILE>|<METALINK_FILE>]
### Options
| Option | Description |
| -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `-d, --dir=<DIR>` | The directory to store the downloaded file. |
| `-i, --input-file=<FILE>` | Downloads the URIs listed in **FILE**. |
| `-l, --log=<LOG>` | The file name of the log file. If - is specified, log is written to stdout. If empty string("") is specified, or this option is omitted, no log is written to disk at all. |
| `-l, --log=<LOG>` | The file name of the [log file](../../../dev/Log.md). If - is specified, log is written to stdout. If empty string("") is specified, or this option is omitted, no log is written to disk at all. |
| `-j, --max-concurrent-downloads=<N>` | Set the maximum number of parallel downloads for every queue item. |
| `-V, --check-integrity [true/false]` | Check file integrity by validating piece hashes or a hash of entire file. This option has effect only in [BitTorrent](../../../internet/BitTorrent.md), Metalink downloads with checksums or [HTTP](../../../internet/HTTP.md)(S)/[FTP](../../../internet/FTP.md) downloads with --checksum option. If piece hashes are provided, this option can detect damaged portions of a file and re-download them. If a hash of entire file is provided, hash check is only done when file has been already download. This is determined by file length. If hash check fails, file is re-downloaded from scratch. If both piece hashes and a hash of entire file are provided, only piece hashes are used. Default: false |
| `-c, --continue [true/false]` | Continue downloading a partially downloaded file. Use this option to resume a download started by a web browser or another program which downloads files sequentially from the beginning. Currently this option is only applicable to [HTTP](../../../internet/HTTP.md)(S)/[FTP](../../../internet/FTP.md) downloads. |

4
technology/applications/cli/network/wget.md
View file

@ -21,9 +21,9 @@ Wget has been designed for robustness over slow or unstable network connections;
### Logging Options
| Option | Description |
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `-o, --output-file=logfile` | Log all messages to logfile. The messages are normally reported to standard error. |
| `-a, --append-output=logfile` | Append to logfile. This is the same as -o, only it appends to logfile instead of overwriting the old log file. |
| `-a, --append-output=logfile` | Append to logfile. This is the same as -o, only it appends to logfile instead of overwriting the old [log file](../../../dev/Log.md). |
| `-q, --quiet` | Turn off Wget's output. |
| `-i, --input-file=file` | Read URLs from a local or external file. If - is specified as file, URLs are read from the standard input. (Use ./- to read from a file literally named -.). If this function is used, no URLs need be present on the command line. If there are URLs both on the command line and in an input file, those on the command lines will be the first ones to be retrieved. If --force-html is not specified, then file should consist of a series of URLs, one per line. However, if you specify --force-html, the document will be regarded as [html](../../../internet/HTML.md). In that case you may have problems with relative links, which you can solve either by adding "\<base href="url">" to the documents or by specifying --base=url on the command line. If the file is an external one, the document will be automatically treated as [html](../../../internet/HTML.md) if the Content-Type matches text/html. Furthermore, the file's location will be implicitly used as base href if none was specified. |
| `-B, --base=URL` | Resolves relative links using URL as the point of reference, when reading links from an [HTML](../../../internet/HTML.md) file specified via the -i/--input-file option (together with --force-html, or when the input file was fetched remotely from a server describing it as [HTML](../../../internet/HTML.md)). This is equivalent to the presence of a "BASE" tag in the [HTML](../../../internet/HTML.md) input file, with URL as the value for the "href" attribute. |

4
technology/applications/cli/tailspin.md
View file

@ -6,9 +6,9 @@ repo: https://github.com/bensadeh/tailspin
# tailspin
A log file highlighter
tailspin works by reading through a log file line by line, running a series of regexes against each line. The regexes recognize patterns like dates, numbers, severity keywords and more.
tailspin works by reading through a [log file](../../dev/Log.md) line by line, running a series of regexes against each line. The regexes recognize patterns like dates, numbers, severity keywords and more.
tailspin does not make any assumptions on the format or position of the items it wants to highlight. For this reason, it requires no configuration or setup and will work predictably regardless of the format the log file is in.
tailspin does not make any assumptions on the format or position of the items it wants to highlight. For this reason, it requires no configuration or setup and will work predictably regardless of the format the [log file](../../dev/Log.md) is in.
![Screenshot][Screenshot]

2
technology/applications/web/Authelia.md
View file

@ -82,7 +82,7 @@ server:
enable_expvars: false
## Log Configuration
## [Log](../../dev/Log.md) Configuration
log:
## Level of verbosity for logs: info, debug, trace.
level: debug

2
technology/applications/web/dufs.md
View file

@ -91,7 +91,7 @@ The log format can use following variables.
| `$status` | response status |
| `$http_` | arbitrary request header field. examples: `$http_user_agent`, `$http_referer` |
The default log format is `'$remote_addr "$request" $status'`.
The default [log](../../dev/Log.md) format is `'$remote_addr "$request" $status'`.
```
2022-08-06T06:59:31+08:00 INFO - 127.0.0.1 "GET /" 200
```

2
technology/applications/web/traefik.md
View file

@ -723,7 +723,7 @@ accessLog: {}
```
- `filePath`:
By default access logs are written to the standard output. To write the logs into a log file, use the `filePath` option.
By default access logs are written to the standard output. To write the logs into a [log file](../../dev/Log.md), use the `filePath` option.
```yaml
accessLog:
filePath: "/path/to/access.log"

53
technology/dev/Log.md Normal file
View file

@ -0,0 +1,53 @@
---
obj: concept
aliases: ["Log", "Logging", "Log File"]
wiki: https://en.wikipedia.org/wiki/Logging_(computing)
---
# Logging
Logging is an essential aspect of software development that involves recording information about the execution of a program or system. It provides valuable insights into the behavior of the software, helps diagnose issues, and facilitates monitoring and debugging.
## Importance of Logging
Logging serves several important purposes in software development:
- **Debugging and Troubleshooting**: Logs capture information about the execution flow, errors, warnings, and exceptions, aiding developers in identifying and resolving issues.
- **Monitoring and Performance Analysis**: Logs can be used to monitor system performance, track resource usage, and analyze trends over time.
- **Auditing and Compliance**: Logs provide a record of system activities, which can be useful for auditing, compliance, and forensic analysis.
- **Security**: Logging security-related events helps detect and respond to potential security threats, such as unauthorized access attempts or suspicious activities.
## Common Logging Practices
### Logging Levels
Logging frameworks typically support different log levels, such as `DEBUG`, `INFO`, `WARN`, `ERROR`, and `FATAL`. Each level indicates the severity of the logged message, allowing developers to filter and prioritize messages based on their importance.
### Log Formatting
Logs should be formatted consistently to facilitate readability and analysis. Common log formats include plaintext, [JSON](../files/JSON.md), [XML](../files/XML.md), and key-value pairs. Formatting may include timestamps, log levels, source locations, and additional contextual information.
### Log Rotation
To manage log file size and prevent disk space exhaustion, log rotation strategies are employed. This involves periodically archiving or purging old log files and creating new ones. Log rotation may be based on size, time, or a combination of both.
### Centralized Logging
In distributed systems or large-scale applications, centralized logging solutions are used to aggregate logs from multiple sources into a central repository. This facilitates log analysis, monitoring, and troubleshooting across the entire system.
### Best Practices for Logging Implementation
- **Use Descriptive Log Messages**: Log messages should be clear, concise, and descriptive, providing useful information about the context and nature of the event.
- **Avoid Excessive Logging**: Logging too much information can clutter logs and degrade performance. Focus on logging essential events and errors while avoiding unnecessary verbosity.
- **Handle Sensitive Data Securely**: Avoid logging sensitive information such as passwords, credit card numbers, or personal data. If necessary, ensure that sensitive data is redacted or encrypted in logs.
- **Include Contextual Information**: Provide relevant contextual information in log messages, such as user IDs, session IDs, request parameters, and stack traces, to aid in troubleshooting and debugging.
- **Monitor and Analyze Logs**: Regularly monitor and analyze logs to detect anomalies, identify performance bottlenecks, and proactively address issues before they escalate.
- **Test Logging Configuration**: Verify logging configuration and behavior through automated testing to ensure that logs are generated correctly and capture the desired information.
## Log Files
Log files typically consist of structured or unstructured data, organized in a chronological sequence. The structure of log files may vary depending on the logging framework or application generating them. Common components of log files include:
- **Timestamp**: The date and time when the event or activity occurred.
- **Severity Level**: The severity or importance of the event, often indicated by log levels such as INFO, WARN, ERROR, or DEBUG.
- **Message**: A descriptive message providing details about the event, error, or activity.
- **Source**: Information about the source or origin of the event, such as the application, module, or process responsible.
There are various log [file formats](../files/File%20Formats.md) used in computing environments, each with its own structure and characteristics. Here are some common log file formats:
- **Plain Text Logs**: Plain text logs are the simplest and most widely used format. Each log entry typically consists of human-readable text, with fields separated by delimiters such as spaces, commas, or tabs. Plain text logs are easy to create, read, and parse, making them a popular choice for logging in many applications.
- **[JSON](../files/JSON.md) Logs**: Logs formatted in [JSON](../files/JSON.md) (JavaScript Object Notation) have a structured format where each log entry is represented as a [JSON](../files/JSON.md) object. [JSON](../files/JSON.md) logs include key-value pairs that provide detailed information about the event, such as timestamp, severity level, message, and additional metadata. [JSON](../files/JSON.md) logs are machine-readable and easily parsed by log analysis tools and frameworks.
- **[XML](../files/XML.md) Logs**: Logs formatted in [XML](../files/XML.md) (eXtensible Markup Language) follow a hierarchical structure where each log entry is represented as an [XML](../files/XML.md) element. [XML](../files/XML.md) logs contain tags and attributes that describe the event, along with any relevant metadata. While [XML](../files/XML.md) logs offer flexibility and extensibility, they tend to be more verbose and less human-readable compared to other formats.
- **[CSV](../files/CSV.md) Logs**: Logs formatted in [CSV](../files/CSV.md) (Comma-Separated Values) consist of rows and columns, with each log entry represented as a line of text where fields are separated by commas. [CSV](../files/CSV.md) logs are commonly used for tabular data and are suitable for logging events with a fixed set of attributes. They are easy to generate and can be opened and analyzed using spreadsheet software.
- **Syslog Format**: Syslog is a standard protocol for logging system messages, and it defines a specific format for log messages. Syslog-formatted logs consist of a header and a message body, where the header includes information such as timestamp, hostname, and severity level, while the message body contains the log message itself. Syslog is commonly used for system-level logging in Unix-like operating systems.
- **Apache Combined Log Format**: The Apache Combined Log Format is a specific log format used by the Apache HTTP Server for logging web server activity. It includes fields such as remote host, timestamp, request method, requested [URL](../internet/URL.md), [HTTP](../internet/HTTP.md) status code, and user agent. The Apache Combined Log Format is widely used for web server logging and analysis.
- **Custom Log Formats**: Many applications and logging frameworks allow users to define custom log formats tailored to their specific requirements. Custom log formats may include a combination of predefined fields, user-defined fields, and optional metadata, providing flexibility and adaptability to diverse logging needs.

2
technology/files/File Formats.md
View file

@ -24,7 +24,7 @@ obj: meta/collection
| `.xcf` | XCF File ([Gimp](../applications/media/images/GIMP.md)) |
| `.asc` | [ASCII](ASCII.md) Text |
| `.docx` | Office Open [XML](XML.md) document |
| `.log` | Log File |
| `.log` | [Log File](../dev/Log.md) |
| `.md` | [Markdown](Markdown.md) File |
| `.txt` | Text File |
| `.xml` | [XML](XML.md) File |

4
technology/linux/Linux.md
View file

@ -40,6 +40,6 @@ A typical Linux system has, among others, the following directories:
| `/usr/share` | This directory contains subdirectories with specific application data, that can be shared among different architectures of the same OS. |
| `/usr/share/doc` | Documentation about installed programs (optional). |
| `/usr/share/man` | [Manual pages](../applications/cli/system/man.md) go here in subdirectories according to the man page sections. |
| `/var` | This directory contains files which may change in size, such as spool and log files. |
| `/var` | This directory contains files which may change in size, such as spool and [log](../dev/Log.md) files. |
| `/var/cache` | Data cached for programs. |
| `/var/log` | Miscellaneous log files. |
| `/var/log` | Miscellaneous [log](../dev/Log.md) files. |

2
technology/linux/UNIX Timestamp.md
View file

@ -7,7 +7,7 @@ A Unix timestamp is a way to represent a point in time, defined as the number of
## Usage
Unix timestamps are commonly used in computing systems to record and calculate time-based events. They are particularly useful for tasks such as:
- Logging events: Recording the time when an event occurs.
- [Logging](../dev/Log.md) events: Recording the time when an event occurs.
- Date arithmetic: Calculating time differences between two Unix timestamps.
- Scheduling tasks: Determining when a task should be executed based on a Unix timestamp.

2
technology/tools/Docker Compose.md
View file

@ -49,7 +49,7 @@ This is the equivalent of `docker exec` targeting a Compose service.
| `-w, --workdir` | Path to workdir directory for this command. |
### `docker-compose logs`
Displays log output from services.
Displays [log](../dev/Log.md) output from services.
#### Options
| Option | Description |