jmarya/knowledge
jmarya/knowledge
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

remove non ascii whitespaces

Browse source
This commit is contained in:
JMARyA 2024-01-17 09:44:04 +01:00
parent 598a10bc28
commit 5a6d6c4d13
Signed by: jmarya
GPG key ID: 901B2ADDF27C2263
117 changed files with 1928 additions and 1928 deletions

4
technology/linux/filesystems/MergerFS Tools.md
View file

@ -59,7 +59,7 @@ $ mergerfs.fsck -v -f manual /path/to/dir
```
## mergerfs.dup
Duplicates files & directories across branches in a pool. The file selected for duplication is picked by the `dup` option. Files will be copied to drives with the most free space. Deleted from others if `prune` is enabled.
Duplicates files & directories across branches in a pool. The file selected for duplication is picked by the `dup` option. Files will be copied to drives with the most free space. Deleted from others if `prune` is enabled.
```shell
usage: mergerfs.dup [<options>] <dir>
@ -91,7 +91,7 @@ optional arguments:
```
## mergerfs.dedup
Finds and removes duplicate files across mergerfs pool's branches. Use the `ignore`, `dedup`, and `strict` options to target specific use cases.
Finds and removes duplicate files across mergerfs pool's branches. Use the `ignore`, `dedup`, and `strict` options to target specific use cases.
```shell
usage: mergerfs.dedup [<options>] <dir>

36
technology/linux/filesystems/MergerFS.md
View file

@ -3,7 +3,7 @@ repo: https://github.com/trapexit/mergerfs
obj: filesystem
---
# MergerFS
**mergerfs** is a union filesystem geared towards simplifying storage and management of files across numerous commodity storage devices. It is similar to **mhddfs**, **unionfs**, and **aufs**.
**mergerfs** is a union filesystem geared towards simplifying storage and management of files across numerous commodity storage devices. It is similar to **mhddfs**, **unionfs**, and **aufs**.
Usage: `mergerfs -o<options> <branches> <mountpoint>`
@ -54,7 +54,7 @@ WantedBy=default.target
```
## Options
These options are the same regardless of whether you use them with the `mergerfs` commandline program, in fstab, or in a config file.
These options are the same regardless of whether you use them with the `mergerfs` commandline program, in fstab, or in a config file.
| mount option | description |
| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
@ -132,29 +132,29 @@ These options are the same regardless of whether you use them with the `mergerf
### branches
The 'branches' argument is a colon (':') delimited list of paths to be pooled together. It does not matter if the paths are on the same or different filesystems nor does it matter the filesystem type (within reason). Used and available space will not be duplicated for paths on the same filesystem and any features which aren't supported by the underlying filesystem (such as file attributes or extended attributes) will return the appropriate errors.
Branches currently have two options which can be set. A type which impacts whether or not the branch is included in a policy calculation and a individual minfreespace value. The values are set by prepending an `=` at the end of a branch designation and using commas as delimiters. Example: `/mnt/drive=RW,1234`
Branches currently have two options which can be set. A type which impacts whether or not the branch is included in a policy calculation and a individual minfreespace value. The values are set by prepending an `=` at the end of a branch designation and using commas as delimiters. Example: `/mnt/drive=RW,1234`
#### branch mode
- RW: (read/write) - Default behavior. Will be eligible in all policy categories.
- RO: (read-only) - Will be excluded from `create` and `action` policies. Same as a read-only mounted filesystem would be (though faster to process).
- NC: (no-create) - Will be excluded from `create` policies. You can't create on that branch but you can change or delete.
- RO: (read-only) - Will be excluded from `create` and `action` policies. Same as a read-only mounted filesystem would be (though faster to process).
- NC: (no-create) - Will be excluded from `create` policies. You can't create on that branch but you can change or delete.
#### globbing
To make it easier to include multiple branches mergerfs supports [globbing](http://linux.die.net/man/7/glob). **The globbing tokens MUST be escaped when using via the shell else the shell itself will apply the glob itself.**
To make it easier to include multiple branches mergerfs supports [globbing](http://linux.die.net/man/7/glob). **The globbing tokens MUST be escaped when using via the shell else the shell itself will apply the glob itself.**
```
# mergerfs /mnt/hdd\*:/mnt/ssd /media
```
The above line will use all mount points in /mnt prefixed with **hdd** and **ssd**.
The above line will use all mount points in /mnt prefixed with **hdd** and **ssd**.
To have the pool mounted at boot or otherwise accessible from related tools use **/etc/fstab**.
To have the pool mounted at boot or otherwise accessible from related tools use **/etc/fstab**.
```
# <file system> <mount point> <type> <options> <dump> <pass>
/mnt/hdd*:/mnt/ssd /media fuse.mergerfs minfreespace=16G 0 0
```
## Functions, Categories and Policies
The POSIX filesystem API is made up of a number of functions. **creat**, **stat**, **chown**, etc. For ease of configuration in mergerfs most of the core functions are grouped into 3 categories: **action**, **create**, and **search**. These functions and categories can be assigned a policy which dictates which branch is chosen when performing that function.
The POSIX filesystem API is made up of a number of functions. **creat**, **stat**, **chown**, etc. For ease of configuration in mergerfs most of the core functions are grouped into 3 categories: **action**, **create**, and **search**. These functions and categories can be assigned a policy which dictates which branch is chosen when performing that function.
### Functions and their Category classifications
| Category | FUSE Functions |
@ -171,22 +171,22 @@ A policy's behavior differs, as mentioned above, based on the function it is use
| Policy | Description |
| --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| all | Search: For **mkdir**, **mknod**, and **symlink** it will apply to all branches. **create** works like **ff**. |
| epall (existing path, all) | For **mkdir**, **mknod**, and **symlink** it will apply to all found. **create** works like **epff** (but more expensive because it doesn't stop after finding a valid branch). |
| all | Search: For **mkdir**, **mknod**, and **symlink** it will apply to all branches. **create** works like **ff**. |
| epall (existing path, all) | For **mkdir**, **mknod**, and **symlink** it will apply to all found. **create** works like **epff** (but more expensive because it doesn't stop after finding a valid branch). |
| epff (existing path, first found) | Given the order of the branches, as defined at mount time or configured at runtime, act on the first one found where the relative path exists. |
| eplfs (existing path, least free space) | Of all the branches on which the relative path exists choose the branch with the least free space. |
| eplus (existing path, least used space) | Of all the branches on which the relative path exists choose the branch with the least used space. |
| epmfs (existing path, most free space) | Of all the branches on which the relative path exists choose the branch with the most free space. |
| eppfrd (existing path, percentage free random distribution) | Like **pfrd** but limited to existing paths. |
| eprand (existing path, random) | Calls **epall** and then randomizes. Returns 1. |
| eppfrd (existing path, percentage free random distribution) | Like **pfrd** but limited to existing paths. |
| eprand (existing path, random) | Calls **epall** and then randomizes. Returns 1. |
| ff (first found) | Given the order of the branches, as defined at mount time or configured at runtime, act on the first one found. |
| lfs (least free space) | Pick the branch with the least available free space. |
| lus (least used space) | Pick the branch with the least used space. |
| mfs (most free space) | Pick the branch with the most available free space. |
| msplfs (most shared path, least free space) | Like **eplfs** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
| msplus (most shared path, least used space) | Like **eplus** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
| mspmfs (most shared path, most free space) | Like **epmfs** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
| msppfrd (most shared path, percentage free random distribution) | Like **eppfrd** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
| msplfs (most shared path, least free space) | Like **eplfs** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
| msplus (most shared path, least used space) | Like **eplus** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
| mspmfs (most shared path, most free space) | Like **epmfs** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
| msppfrd (most shared path, percentage free random distribution) | Like **eppfrd** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
| newest | Pick the file / directory with the largest mtime. |
| pfrd (percentage free random distribution) | Chooses a branch at random with the likelihood of selection based on a branch's available space relative to the total. |
| rand (random) | Calls **all** and then randomizes. Returns 1 branch. |
| rand (random) | Calls **all** and then randomizes. Returns 1 branch. |

18
technology/linux/filesystems/RAID.md
View file

@ -3,30 +3,30 @@ arch-wiki: https://wiki.archlinux.org/title/RAID
obj: concept
---
# RAID
Redundant Array of Independent Disks (RAID) is a storage technology that combines multiple disk drive components (typically disk drives or partitions thereof) into a logical unit. Depending on the RAID implementation, this logical unit can be a file system or an additional transparent layer that can hold several partitions. Data is distributed across the drives in one of several ways called RAID levels, depending on the level of redundancy and performance required. The RAID level chosen can thus prevent data loss in the event of a hard disk failure, increase performance or be a combination of both.
Redundant Array of Independent Disks (RAID) is a storage technology that combines multiple disk drive components (typically disk drives or partitions thereof) into a logical unit. Depending on the RAID implementation, this logical unit can be a file system or an additional transparent layer that can hold several partitions. Data is distributed across the drives in one of several ways called RAID levels, depending on the level of redundancy and performance required. The RAID level chosen can thus prevent data loss in the event of a hard disk failure, increase performance or be a combination of both.
## RAID Levels
### RAID 0
Uses striping to combine disks. Even though it _does not provide redundancy_, it is still considered RAID. It does, however, _provide a big speed benefit_. If the speed increase is worth the possibility of data loss, choose this RAID level. On a server, RAID 1 and RAID 5 arrays are more appropriate. The size of a RAID 0 array block device is the size of the smallest component partition times the number of component partitions.
Uses striping to combine disks. Even though it _does not provide redundancy_, it is still considered RAID. It does, however, _provide a big speed benefit_. If the speed increase is worth the possibility of data loss, choose this RAID level. On a server, RAID 1 and RAID 5 arrays are more appropriate. The size of a RAID 0 array block device is the size of the smallest component partition times the number of component partitions.
### RAID 1
The most straightforward RAID level: straight mirroring. As with other RAID levels, it only makes sense if the partitions are on different physical disk drives. If one of those drives fails, the block device provided by the RAID array will continue to function as normal. The example will be using RAID 1 for everything except swap and temporary data. Please note that with a software implementation, the RAID 1 level is the only option for the boot partition, because bootloaders reading the boot partition do not understand RAID, but a RAID 1 component partition can be read as a normal partition. The size of a RAID 1 array block device is the size of the smallest component partition.
The most straightforward RAID level: straight mirroring. As with other RAID levels, it only makes sense if the partitions are on different physical disk drives. If one of those drives fails, the block device provided by the RAID array will continue to function as normal. The example will be using RAID 1 for everything except swap and temporary data. Please note that with a software implementation, the RAID 1 level is the only option for the boot partition, because bootloaders reading the boot partition do not understand RAID, but a RAID 1 component partition can be read as a normal partition. The size of a RAID 1 array block device is the size of the smallest component partition.
### RAID 5
Requires 3 or more physical drives, and provides the redundancy of RAID 1 combined with the speed and size benefits of RAID 0. RAID 5 uses striping, like RAID 0, but also stores parity blocks _distributed across each member disk_. In the event of a failed disk, these parity blocks are used to reconstruct the data on a replacement disk. RAID 5 can withstand the loss of one member disk.
Requires 3 or more physical drives, and provides the redundancy of RAID 1 combined with the speed and size benefits of RAID 0. RAID 5 uses striping, like RAID 0, but also stores parity blocks _distributed across each member disk_. In the event of a failed disk, these parity blocks are used to reconstruct the data on a replacement disk. RAID 5 can withstand the loss of one member disk.
### RAID 6
Requires 4 or more physical drives, and provides the benefits of RAID 5 but with security against two drive failures. RAID 6 also uses striping, like RAID 5, but stores two distinct parity blocks _distributed across each member disk_. In the event of a failed disk, these parity blocks are used to reconstruct the data on a replacement disk. RAID 6 can withstand the loss of two member disks. The robustness against unrecoverable read errors is somewhat better, because the array still has parity blocks when rebuilding from a single failed drive. However, given the overhead, RAID 6 is costly and in most settings RAID 10 in far2 layout (see below) provides better speed benefits and robustness, and is therefore preferred.
Requires 4 or more physical drives, and provides the benefits of RAID 5 but with security against two drive failures. RAID 6 also uses striping, like RAID 5, but stores two distinct parity blocks _distributed across each member disk_. In the event of a failed disk, these parity blocks are used to reconstruct the data on a replacement disk. RAID 6 can withstand the loss of two member disks. The robustness against unrecoverable read errors is somewhat better, because the array still has parity blocks when rebuilding from a single failed drive. However, given the overhead, RAID 6 is costly and in most settings RAID 10 in far2 layout (see below) provides better speed benefits and robustness, and is therefore preferred.
## Using RAID
Software RAID is the easiest implementation as it does not rely on obscure proprietary firmware and software to be used. The array is managed by the operating system either by:
- an abstraction layer (e.g. mdadm);
- a logical volume manager (e.g. [LVM](LVM.md));
- a component of a file system (e.g. [ZFS](ZFS.md), [Btrfs](Btrfs.md))
- an abstraction layer (e.g. mdadm);
- a logical volume manager (e.g. [LVM](LVM.md));
- a component of a file system (e.g. [ZFS](ZFS.md), [Btrfs](Btrfs.md))
## Installation
_mdadm_ is used for administering pure software RAID using plain block devices: the underlying hardware does not provide any RAID logic, just a supply of disks. _mdadm_ will work with any collection of block devices. Even if unusual. For example, one can thus make a RAID array from a collection of thumb drives.
_mdadm_ is used for administering pure software RAID using plain block devices: the underlying hardware does not provide any RAID logic, just a supply of disks. _mdadm_ will work with any collection of block devices. Even if unusual. For example, one can thus make a RAID array from a collection of thumb drives.
**Erase disks**
```shell

2
technology/linux/filesystems/SquashFS.md
View file

@ -2,7 +2,7 @@
obj: filesystem
---
# SquashFS
**Squashfs** is a compressed read-only [file system](Filesystems.md) for [Linux](../Linux.md).
**Squashfs** is a compressed read-only [file system](Filesystems.md) for [Linux](../Linux.md).
## Creation
```shell

2
technology/linux/filesystems/ZFS.md
View file

@ -6,7 +6,7 @@ obj: filesystem
---
# ZFS
[ZFS](https://en.wikipedia.org/wiki/ZFS "wikipedia:ZFS") is an advanced filesystem.
[ZFS](https://en.wikipedia.org/wiki/ZFS "wikipedia:ZFS") is an advanced filesystem.
## ZPool
**Create a pool***