self-hosted/minio
self-hosted/minio
1
0
Fork 0
mirror of https://github.com/minio/minio synced 2024-10-01 13:44:10 +00:00
Code Issues Packages Projects Releases Wiki Activity

doc:README.md/Updated to sync with docs.minio.io (#2210)

Browse source 
* doc:README.md/Updated to sync with docs.minio.io

* doc:README.me/Modified the minio server output terminal to reflect new release changes.

* docs:README.md/Modified and changed location of other markdown files.
This commit is contained in:
koolhead17 2016-07-15 15:03:59 -07:00 committed by Harshavardhana
parent 3bce5db6d1
commit 204ec2c6c0
8 changed files with 171 additions and 216 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

BIN
Minio_Browser.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 99 KiB

342
README.md
View file

@ -1,261 +1,171 @@
## Minio [![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/minio/minio?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
# Minio Quickstart Guide
Minio is an object storage server released under Apache License v2.0. It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB.
Minio is an object storage server compatible with Amazon S3 and licensed under [Apache license 2.0](./LICENSE).
## 1. Download
Minio server is light enough to be bundled with the application stack, similar to NodeJS, Redis and MySQL.
## Description
| Platform| Architecture | URL|
| ----------| -------- | ------|
|GNU/Linux|64-bit Intel|https://dl.minio.io/server/minio/release/linux-amd64/minio|
||32-bit Intel|https://dl.minio.io/server/minio/release/linux-386/minio|
||32-bit ARM|https://dl.minio.io/server/minio/release/linux-arm/minio|
|Apple OS X|64-bit Intel|https://dl.minio.io/server/minio/release/darwin-amd64/minio|
|Microsoft Windows|64-bit|https://dl.minio.io/server/minio/release/windows-amd64/minio.exe|
||32-bit|https://dl.minio.io/server/minio/release/windows-386/minio.exe|
|FreeBSD|64-bit|https://dl.minio.io/server/minio/release/freebsd-amd64/minio|
Minio is an open source object storage server released under Apache License V2. It is compatible with Amazon S3 cloud storage service. Minio follows a minimalist design philosophy.
Minio is light enough to be bundled with the application stack. It sits on the side of NodeJS, Redis, MySQL and the likes. Unlike databases, Minio stores objects such as photos, videos, log files, backups, container / VM images and so on. Minio is best suited for storing blobs of information ranging from KBs to 5 TBs each. In a simplistic sense, it is like a FTP server with a simple get / put API over HTTP.
Minio currently implements two backends
- Filesystem (FS) - is available and ready for general purpose use.
- ErasureCoded (XL) - is available, but it is not ready for general purpose use.
### Install from Source
## Minio Client
Source installation is only intended for developers and advanced users. If you do not have a working Golang environment, please follow [How to install Golang](https://docs.minio.io/docs/how-to-install-golang).
[Minio Client (mc)](https://github.com/minio/mc#minio-client-mc-) provides a modern alternative to Unix commands like ``ls``, ``cat``, ``cp``, ``sync``, and ``diff``. It supports POSIX compatible filesystems and Amazon S3 compatible cloud storage systems. It is entirely written in Golang.
## Amazon S3 Compatible Client Libraries
- [Golang Library](https://github.com/minio/minio-go)
- [Java Library](https://github.com/minio/minio-java)
- [Nodejs Library](https://github.com/minio/minio-js)
- [Python Library](https://github.com/minio/minio-py)
- [.Net Library](https://github.com/minio/minio-dotnet)
### Install [![Build Status](https://travis-ci.org/minio/minio.svg?branch=master)](https://travis-ci.org/minio/minio)[![Build status](https://ci.appveyor.com/api/projects/status/royh137dni8yevep/branch/master?svg=true)](https://ci.appveyor.com/project/harshavardhana/minio-qxbjq/branch/master)
#### GNU/Linux
Download ``minio`` for:
- ``64-bit Intel`` from https://dl.minio.io/server/minio/release/linux-amd64/minio
- ``32-bit Intel`` from https://dl.minio.io/server/minio/release/linux-386/minio
- ``32-bit ARM`` from https://dl.minio.io/server/minio/release/linux-arm/minio
~~~
$ chmod +x minio
$ ./minio --help
~~~
#### OS X
Download ``minio`` from https://dl.minio.io/server/minio/release/darwin-amd64/minio
~~~
$ chmod 755 minio
$ ./minio --help
~~~
#### Microsoft Windows
Download ``minio`` for:
- ``64-bit`` from https://dl.minio.io/server/minio/release/windows-amd64/minio.exe
- ``32-bit`` from https://dl.minio.io/server/minio/release/windows-386/minio.exe
~~~
C:\Users\Username\Downloads> minio.exe --help
~~~
#### FreeBSD
Download ``minio`` from https://dl.minio.io/server/minio/release/freebsd-amd64/minio
~~~
$ chmod 755 minio
$ ./minio --help
~~~
Read more here on [How to configure Minio on FreeBSD with ZFS backend.](./FreeBSD.md)
#### Docker container
Download ``minio`` for docker.
~~~
$ docker pull minio/minio
~~~
Read more here on [How to configure data volume containers for Minio?](./Docker.md)
#### Source
<blockquote>
NOTE: Source installation is intended for only developers and advanced users. For general use, please download official releases from https://minio.io/downloads.
</blockquote>
If you do not have a working Golang environment, please follow [Install Golang](./INSTALLGO.md).
~~~
```sh
$ go get -d github.com/minio/minio
$ cd $GOPATH/src/github.com/minio/minio
$ make
~~~
```
## 2. Run Minio Server
### 1. GNU/Linux
### How to use Minio?
Start minio server.
~~~
$ minio server ~/Photos
```sh
$ chmod +x minio
$ ./minio --help
$ ./minio server ~/Photos
Endpoint: http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
AccessKey: USWUXHGYZQYFYFFIT3RE
SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
AccessKey: USWUXHGYZQYFYFFIT3RE
SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
Region: us-east-1
Browser Access:
http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
Command-line Access: https://docs.minio.io/docs/minio-client-quick-start-guide
$ ./mc config host add myminio http://10.0.0.10:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
$ mc config host add myminio http://10.0.0.10:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
Object API (Amazon S3 compatible):
Go: https://docs.minio.io/docs/golang-client-quickstart-guide
Java: https://docs.minio.io/docs/java-client-quickstart-guide
Python: https://docs.minio.io/docs/python-client-quickstart-guide
JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
~~~
#### How to use AWS CLI with Minio?
<blockquote>
This section assumes that you have already installed aws-cli, if not please visit https://aws.amazon.com/cli/
</blockquote>
To configure `aws-cli`, type `aws configure` and follow below steps.
```
$ aws configure
AWS Access Key ID [None]: YOUR_ACCESS_KEY_HERE
AWS Secret Access Key [None]: YOUR_SECRET_KEY_HERE
Default region name [None]: us-east-1
Default output format [None]: ENTER
```
Additionally enable `aws-cli` to use AWS Signature Version '4' for Minio server.
### 2. OS X
```
$ aws configure set default.s3.signature_version s3v4
```sh
$ chmod 755 minio
$ ./minio --help
$ ./minio server ~/Photos
Endpoint: http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
AccessKey: USWUXHGYZQYFYFFIT3RE
SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
Region: us-east-1
Browser Access:
http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
Command-line Access: https://docs.minio.io/docs/minio-client-quick-start-guide
$ mc config host add myminio http://10.0.0.10:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
Object API (Amazon S3 compatible):
Go: https://docs.minio.io/docs/golang-client-quickstart-guide
Java: https://docs.minio.io/docs/java-client-quickstart-guide
Python: https://docs.minio.io/docs/python-client-quickstart-guide
JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
```
To list your buckets.
```
$ aws --endpoint-url http://localhost:9000 s3 ls
2016-03-27 02:06:30 deebucket
2016-03-28 21:53:49 guestbucket
2016-03-29 13:34:34 mbtest
2016-03-26 22:01:36 mybucket
2016-03-26 15:37:02 testbucket
```
### 3.Microsoft Windows
To list contents inside bucket.
```
$ aws --endpoint-url http://localhost:9000 s3 ls s3://mybucket
2016-03-30 00:26:53 69297 argparse-1.2.1.tar.gz
2016-03-30 00:35:37 67250 simplejson-3.3.0.tar.gz
```
```sh
C:\Users\Username\Downloads> minio.exe --help
C:\Users\Username\Downloads> minio.exe server D:\Photos
To make a bucket.
```
$ aws --endpoint-url http://localhost:9000 s3 mb s3://mybucket
make_bucket: s3://mybucket/
```
Endpoint: http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
AccessKey: USWUXHGYZQYFYFFIT3RE
SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
Region: us-east-1
To add an object to a bucket.
```
$ aws --endpoint-url http://localhost:9000 s3 cp simplejson-3.3.0.tar.gz s3://mybucket
upload: ./simplejson-3.3.0.tar.gz to s3://mybucket/simplejson-3.3.0.tar.gz
```
Browser Access:
http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
Delete an object from a bucket.
```
$ aws --endpoint-url http://localhost:9000 s3 rm s3://mybucket/argparse-1.2.1.tar.gzdelete: s3://mybucket/argparse-1.2.1.tar.gz
```
Command-line Access: https://docs.minio.io/docs/minio-client-quick-start-guide
$ mc.exe config host add myminio http://10.0.0.10:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
Remove a bucket.
```
$ aws --endpoint-url http://localhost:9000 s3 rb s3://mybucket
remove_bucket: s3://mybucket/
```
#### How to use AWS SDK with Minio?
Please follow the documentation here - [Using aws-sdk-go with Minio server](./AWS-SDK-GO.md)
#### How to use s3cmd with Minio?
<blockquote>
This section assumes that you have already installed s3cmd, if not please visit http://s3tools.org/s3cmd
</blockquote>
Edit the following fields in your s3cmd configuration file `~/.s3cfg` .
Object API (Amazon S3 compatible):
Go: https://docs.minio.io/docs/golang-client-quickstart-guide
Java: https://docs.minio.io/docs/java-client-quickstart-guide
Python: https://docs.minio.io/docs/python-client-quickstart-guide
JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
```
host_base = localhost:9000
host_bucket = localhost:9000
access_key = YOUR_ACCESS_KEY_HERE
secret_key = YOUR_SECRET_KEY_HERE
signature_v2 = False
bucket_location = us-east-1
### 4. Docker Container
```sh
$ docker pull minio/minio
$ docker run -p 9000:9000 minio/minio
```
To make a bucket.
```
$ s3cmd mb s3://mybucket
Bucket 's3://mybucket/' created
### 5. FreeBSD
```sh
$ chmod 755 minio
$ ./minio --help
$ ./minio server ~/Photos
Endpoint: http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
AccessKey: USWUXHGYZQYFYFFIT3RE
SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
Region: us-east-1
Browser Access:
http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
Command-line Access: https://docs.minio.io/docs/minio-client-quick-start-guide
$ mc config host add myminio http://10.0.0.10:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
Object API (Amazon S3 compatible):
Go: https://docs.minio.io/docs/golang-client-quickstart-guide
Java: https://docs.minio.io/docs/java-client-quickstart-guide
Python: https://docs.minio.io/docs/python-client-quickstart-guide
JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
```
To copy an object to bucket.
```
$ s3cmd put newfile s3://testbucket
upload: 'newfile' -> 's3://testbucket/newfile'
## 3. Test Minio Server using Minio Browser
Open a web browser and navigate to http://127.0.0.1:9000 to view your buckets on minio server.
![Screenshot](./Minio_Browser.png)
## 4. Test Minio Server using `mc`
Install mc from [here](https://docs.minio.io/docs/minio-client-quick-start-guide). Use `mc ls` command to list all the buckets on your minio server.
```sh
$ mc ls myminio/
[2015-08-05 08:13:22 IST] 0B andoria/
[2015-08-05 06:14:26 IST] 0B deflector/
[2015-08-05 08:13:11 IST] 0B ferenginar/
[2016-03-08 14:56:35 IST] 0B jarjarbing/
[2016-01-20 16:07:41 IST] 0B my.minio.io/
```
To copy an object to local system.
```
$ s3cmd get s3://testbucket/newfile
download: 's3://testbucket/newfile' -> './newfile'
```
For more examples please navigate to [Minio Client Complete Guide](https://docs.minio.io/docs/minio-client-complete-guide).
To sync local file/directory to a bucket.
```
$ s3cmd sync newdemo s3://testbucket
upload: 'newdemo/newdemofile.txt' -> 's3://testbucket/newdemo/newdemofile.txt'
```
To sync bucket or object with local filesystem.
```
$ s3cmd sync s3://otherbucket otherlocalbucket
download: 's3://otherbucket/cat.jpg' -> 'otherlocalbucket/cat.jpg'
```
To list buckets.
```
$ s3cmd ls s3://
2015-12-09 16:12 s3://testbbucket
```
To list contents inside bucket.
```
$ s3cmd ls s3://testbucket/
DIR s3://testbucket/test/
2015-12-09 16:05 138504 s3://testbucket/newfile
```
Delete an object from bucket.
```
$ s3cmd del s3://testbucket/newfile
delete: 's3://testbucket/newfile'
```
Delete a bucket.
```
$ s3cmd rb s3://testbucket
Bucket 's3://testbucket/' removed
```
## Contribute to Minio Project
## 5. Explore Further
- [Minio Erasure Code QuickStart Guide](https://docs.minio.io/docs/minio-erasure-code-quickstart-guide)
- [Minio Docker Quickstart Guide](https://docs.minio.io/docs/minio-docker-container)
- [Use `mc` with Minio Server](https://docs.minio.io/docs/minio-client-quick-start-guide)
- [Use `aws-cli` with Minio Server](https://docs.minio.io/docs/how-to-use-aws-cli-with-minio)
- [Use `s3cmd` with Minio Server](https://docs.minio.io/docs/s3cmd-with-minio-server)
- [Use `minio-go` SDK with Minio Server](https://docs/golang-client-quickstart-guide)
## 6. Contribute to Minio Project
Please follow Minio [Contributor's Guide](./CONTRIBUTING.md)
### Jobs
If you think in Lisp or Haskell and hack in go, you would blend right in. Send your github link to callhome@minio.io.

0
AWS-SDK-GO.md → docs/AWS-SDK-GO.md
View file

0
Caddy.md → docs/Caddy.md
View file

0
Docker.md → docs/Docker.md
View file

1
docs/Erasure_Code.svg Normal file
View file

File diff suppressed because one or more lines are too long
Side by side

After

Width:  |  Height:  |  Size: 19 KiB

0
INSTALLGO.md → docs/INSTALLGO.md
View file

44
docs/Minio-erasure-code-quickStart-guide.md Normal file
View file

@ -0,0 +1,44 @@
Minio protects data against hardware failures and silent data corruption using erasure code and checksums. You may lose roughly half the number ((N/2)-1) of drives and still be able to recover the data.
## 1. Prerequisites:
Install Minio - [Minio Quickstart Guide](https://docs.minio.io/docs/minio)
## What is Erasure Code?
Erasure code is a mathematical algorithm to reconstruct missing or corrupted data. Minio uses Reed-Solomon code to shard objects into N/2 data and N/2 parity blocks. This means that in a 12 drive setup, an object is sharded across as 6 data and 6 parity blocks. You can lose as many as 5 drives (be it parity or data) and still reconstruct the data reliably from the remaining drives.
## Why is Erasure Code useful?
Erasure code protects data from multiple drives failure unlike RAID or replication. Minio encodes each object individually with a high parity count. Storage servers once deployed should not require drive replacement or healing for the lifetime of the server. Minio's erasure coded backend is designed for operational efficiency and takes full advantage of hardware acceleration whenever available.
[![Screenshot](https://www.filepicker.io/api/file/hbaiKmmR9Fg3NZ1yEb8A)](https://docs.minio.io/)
## What is Bit Rot protection?
Bit Rot also known as Data Rot or Silent Data Corruption is a serious data loss issue faced by disk drives today. Data on the drive may silently get corrupted without signalling an error has occurred. This makes Bit Rot more dangerous than permanent hard drive failure.
Minio's erasure coded backend uses high speed [BLAKE2](https://blog.minio.io/accelerating-blake2b-by-4x-using-simd-in-go-assembly-33ef16c8a56b#.jrp1fdwer) hash based checksums to protect against Bit Rot.
## Deployment Scenarios
Minio server runs on a variety of hardware, operating systems and virtual/container environments.
Minio erasure code backend is limited by design to a minimum of 6 drives and a maximum of 16 drives. The hard limit of 16 drives comes from operational experience. Failure domain becomes too large beyond 16 drives. If you need to scale beyond 16 drives, you may run multiple instances of Minio server on different ports.
#### Reference Physical Hardware:
* [SMC 5018A-AR12L (Intel Atom)](http://www.supermicro.com/products/system/1U/5018/SSG-5018A-AR12L.cfm?parts=SHOW) - SMC 1U SoC Atom C2750 platform with 12x 3.5” drive bays
* [Quanta Grid D51B-2U (OCP Compliant) ](http://www.qct.io/Product/Servers/Rackmount-Servers/2U/QuantaGrid-D51B-2U-p256c77c70c83c118)- Quanta 2U DP E5-2600v3 platform with 12x 3.5” drive bays
* [Cisco UCS C240 M4 Rack Server](http://www.cisco.com/c/en/us/products/servers-unified-computing/ucs-c240-m4-rack-server/index.html) - Cisco 2U DP E5-2600v3 platform with 12x 3.5” drive bays
* [Intel® Server System R2312WTTYSR](http://ark.intel.com/products/88286) - Intel 2U DP E5-2600v3 platform with 12x 3.5” drive bays
#### Reference Cloud Hosting Providers:
* [Packet](https://www.packet.net): Packet is the baremetal cloud provider.
* [Digital Ocean](https://www.digitalocean.com): Deploy an SSD cloud server in 55 seconds.
* [OVH](https://www.ovh.com/us): Build your own infrastructure with OVH public cloud.
* [Onlinetech](http://www.onlinetech.com): Secure, compliant enterprise cloud.
* [SSD Nodes](https://www.ssdnodes.com): Simple, high performance cloud provider with truly personalized support.
## 2. Run Minio Server with Erasure Code.
Example: Start Minio server in a 12 drives setup.
```
$ minio server /mnt/export1/backend /mnt/export2/backend /mnt/export3/backend /mnt/export4/backend /mnt/export5/backend /mnt/export6/backend /mnt/export7/backend /mnt/export8/backend /mnt/export9/backend /mnt/export10/backend /mnt/export11/backend /mnt/export12/backend
```
## 3. Test your setup
You may unplug drives randomly and continue to perform I/O on the system.