knowledge/technology/applications/network/rclone.md

obj	website	repo
application	https://rclone.org	https://github.com/rclone/rclone

rclone

Rclone is a command-line program to manage files on cloud storage. It is a feature-rich alternative to cloud vendors' web storage interfaces. Over 70 cloud storage products support rclone including S3 object stores, business & consumer file storage services, as well as standard transfer protocols. Virtual backends wrap local and cloud file systems to apply encryption, compression, chunking, hashing and joining.

Usage

rclone config

Enter an interactive configuration session.

rclone cat

Concatenates any files and sends them to stdout.

rclone cat remote:path/to/file

Options

Option	Description
--count int	Only print N characters (default -1)
--head int	Only print the first N characters
--offset int	Start printing at offset N (or from end if -ve)
--tail int	Only print the last N characters

rclone copy

Copy files from source to dest, skipping identical files.

rclone copy source:sourcepath dest:destpath

Options

Option	Description
-u, --update	Skip files that are newer on the destination

rclone delete

Remove the files in path.

rclone delete remote:path

rclone ls

List the objects in the path with size and path.

rclone ls remote:path

rclone listremotes

List all the remotes in the config file.

rclone listremotes

rclone mkdir

Make the path if it doesn't already exist.

rclone mkdir remote:path

rclone move

Move files from source to dest.

rclone move source:path dest:path

rclone sync

Make source and dest identical, modifying destination only.

rclone sync --interactive SOURCE remote:DESTINATION

rclone serve

Serve a remote over a protocol.

Protocols:

• FTP
• HTTP
• SFTP
• WebDAV

rclone serve <protocol> <remote>

rclone mount

Mount the remote as file system on a mountpoint.

rclone mount remote:path/to/files /path/to/local/mount

Storage Providers

Alias

The alias remote provides a new name for another remote.

Amazon S3 Storage

The S3 backend can be used with a number of compatible providers (including Minio).

Chunker

The chunker overlay transparently splits large files into smaller chunks during upload to wrapped remote and transparently assembles them back when the file is downloaded. This allows to effectively overcome size limits imposed by storage providers.

Crypt

Rclone crypt remotes encrypt and decrypt other remotes.

A remote of type crypt does not access a storage system directly, but instead wraps another remote, which in turn accesses the storage system. This is similar to how alias, union, chunker and a few others work. It makes the usage very flexible, as you can add a layer, in this case an encryption layer, on top of any other backend, even in multiple layers. Rclone's functionality can be used as with any other remote, for example you can mount a crypt remote.

Accessing a storage system through a crypt remote realizes client-side encryption, which makes it safe to keep your data in a location you do not trust will not get compromised. When working against the crypt remote, rclone will automatically encrypt (before uploading) and decrypt (after downloading) on your local system as needed on the fly, leaving the data encrypted at rest in the wrapped remote. If you access the storage system using an application other than rclone, or access the wrapped remote directly using rclone, there will not be any encryption/decryption: Downloading existing content will just give you the encrypted (scrambled) format, and anything you upload will not become encrypted.

FTP

FTP is the File Transfer Protocol.

HTTP

The HTTP remote is a read only remote for reading files of a webserver. The webserver should provide file listings which rclone will read and turn into a remote.

Mega

Mega is a cloud storage and file hosting service known for its security feature where all files are encrypted locally before they are uploaded. This prevents anyone (including employees of Mega) from accessing the files without knowledge of the key used for encryption.

This is an rclone backend for Mega which supports the file transfer features of Mega using the same client side encryption.

Memory

The memory backend is an in RAM backend. It does not persist its data.

The memory backend behaves like a bucket-based remote (e.g. like s3). Because it has no parameters you can just use it with the :memory: remote name.

SMB

SMB is a communication protocol to share files over network.

Union

The union backend joins several remotes together to make a single unified view of them.

During the initial setup with rclone config you will specify the upstream remotes as a space separated list. The upstream remotes can either be a local paths or other remotes.

The attributes :ro, :nc and :nc can be attached to the end of the remote to tag the remote as read only, no create or writeback, e.g. remote:directory/subdirectory:ro or remote:directory/subdirectory:nc.

• :ro means files will only be read from here and never written
• :nc means new files or directories won't be created here
• :writeback means files found in different remotes will be written back here.

Subfolders can be used in upstream remotes. Assume a union remote named backup with the remotes mydrive:private/backup. Invoking rclone mkdir backup:desktop is exactly the same as invoking rclone mkdir mydrive:private/backup/desktop.

WebDAV

To configure the WebDAV remote you will need to have a URL for it, and a username and password