knowledge/technology/applications/network/rclone.md

150 lines
6.1 KiB
Markdown
Raw Normal View History

2023-12-04 10:02:23 +00:00
---
obj: application
website: https://rclone.org
repo: https://github.com/rclone/rclone
---
# rclone
Rclone is a command-line program to manage files on cloud storage. It is a feature-rich alternative to cloud vendors' web storage interfaces. Over 70 cloud storage products support rclone including S3 object stores, business & consumer file storage services, as well as standard transfer protocols. Virtual backends wrap local and cloud file systems to apply encryption, compression, chunking, hashing and joining.
## Usage
### `rclone config`
Enter an interactive configuration session.
### `rclone cat`
Concatenates any files and sends them to stdout.
```shell
rclone cat remote:path/to/file
```
#### Options
| Option | Description |
| -------------- | ----------------------------------------------- |
| `--count int` | Only print N characters (default -1) |
| `--head int` | Only print the first N characters |
| `--offset int` | Start printing at offset N (or from end if -ve) |
| `--tail int` | Only print the last N characters |
### `rclone copy`
Copy files from source to dest, skipping identical files.
```shell
rclone copy source:sourcepath dest:destpath
```
#### Options
| Option | Description |
| -------------- | -------------------------------------------- |
| `-u, --update` | Skip files that are newer on the destination |
### `rclone delete`
Remove the files in path.
```shell
rclone delete remote:path
```
### `rclone ls`
List the objects in the path with size and path.
```shell
rclone ls remote:path
```
### `rclone listremotes`
List all the remotes in the config file.
```shell
rclone listremotes
```
### `rclone mkdir`
Make the path if it doesn't already exist.
```shell
rclone mkdir remote:path
```
### `rclone move`
Move files from source to dest.
```shell
rclone move source:path dest:path
```
### `rclone sync`
Make source and dest identical, modifying destination only.
```shell
rclone sync --interactive SOURCE remote:DESTINATION
```
### `rclone serve`
Serve a remote over a protocol.
Protocols:
- [FTP](../../internet/FTP.md)
- [HTTP](../../internet/HTTP.md)
- SFTP
2024-01-17 08:00:45 +00:00
- [WebDAV](../../internet/WebDAV.md)
2023-12-04 10:02:23 +00:00
```shell
rclone serve <protocol> <remote>
```
### `rclone mount`
Mount the remote as file system on a mountpoint.
```shell
rclone mount remote:path/to/files /path/to/local/mount
```
## Storage Providers
### Alias
2024-01-17 08:44:04 +00:00
The `alias` remote provides a new name for another remote.
2023-12-04 10:02:23 +00:00
### Amazon S3 Storage
The S3 backend can be used with a number of compatible providers (including [Minio](../web/Minio.md)).
## Chunker
2024-01-17 08:44:04 +00:00
The `chunker` overlay transparently splits large files into smaller chunks during upload to wrapped remote and transparently assembles them back when the file is downloaded. This allows to effectively overcome size limits imposed by storage providers.
2023-12-04 10:02:23 +00:00
## Crypt
2024-01-17 08:44:04 +00:00
Rclone `crypt` remotes encrypt and decrypt other remotes.
2023-12-04 10:02:23 +00:00
2024-01-17 08:44:04 +00:00
A remote of type `crypt` does not access a storage system directly, but instead wraps another remote, which in turn accesses the storage system. This is similar to how alias, union, chunker and a few others work. It makes the usage very flexible, as you can add a layer, in this case an encryption layer, on top of any other backend, even in multiple layers. Rclone's functionality can be used as with any other remote, for example you can mount a crypt remote.
2023-12-04 10:02:23 +00:00
2024-01-17 08:44:04 +00:00
Accessing a storage system through a crypt remote realizes client-side encryption, which makes it safe to keep your data in a location you do not trust will not get compromised. When working against the `crypt` remote, rclone will automatically encrypt (before uploading) and decrypt (after downloading) on your local system as needed on the fly, leaving the data encrypted at rest in the wrapped remote. If you access the storage system using an application other than rclone, or access the wrapped remote directly using rclone, there will not be any encryption/decryption: Downloading existing content will just give you the encrypted (scrambled) format, and anything you upload will _not_ become encrypted.
2023-12-04 10:02:23 +00:00
## FTP
[FTP](../../internet/FTP.md) is the File Transfer Protocol.
## HTTP
The [HTTP](../../internet/HTTP.md) remote is a read only remote for reading files of a webserver. The webserver should provide file listings which rclone will read and turn into a remote.
## Mega
Mega is a cloud storage and file hosting service known for its security feature where all files are encrypted locally before they are uploaded. This prevents anyone (including employees of Mega) from accessing the files without knowledge of the key used for encryption.
This is an rclone backend for Mega which supports the file transfer features of Mega using the same client side encryption.
## Memory
The memory backend is an in RAM backend. It does not persist its data.
2024-01-17 08:44:04 +00:00
The memory backend behaves like a bucket-based remote (e.g. like s3). Because it has no parameters you can just use it with the `:memory:` remote name.
2023-12-04 10:02:23 +00:00
## SMB
SMB is a communication protocol to share files over network.
## Union
2024-01-17 08:44:04 +00:00
The `union` backend joins several remotes together to make a single unified view of them.
2023-12-04 10:02:23 +00:00
2024-01-17 08:44:04 +00:00
During the initial setup with `rclone config` you will specify the upstream remotes as a space separated list. The upstream remotes can either be a local paths or other remotes.
2023-12-04 10:02:23 +00:00
2024-01-17 08:44:04 +00:00
The attributes `:ro`, `:nc` and `:nc` can be attached to the end of the remote to tag the remote as **read only**, **no create** or **writeback**, e.g. `remote:directory/subdirectory:ro` or `remote:directory/subdirectory:nc`.
- `:ro` means files will only be read from here and never written
- `:nc` means new files or directories won't be created here
- `:writeback` means files found in different remotes will be written back here.
2023-12-04 10:02:23 +00:00
2024-01-17 08:44:04 +00:00
Subfolders can be used in upstream remotes. Assume a union remote named `backup` with the remotes `mydrive:private/backup`. Invoking `rclone mkdir backup:desktop` is exactly the same as invoking `rclone mkdir mydrive:private/backup/desktop`.
2023-12-04 10:02:23 +00:00
## WebDAV
2024-01-17 08:00:45 +00:00
To configure the [WebDAV](../../internet/WebDAV.md) remote you will need to have a [URL](../../internet/URL.md) for it, and a username and password