Commit graph

8614 commits

Author SHA1 Message Date
Harshavardhana
495c55e6a5
fix: make sure to delete dangling objects during heal (#13138)
heal with --remove was not removing dangling versions
on versioned buckets, this PR fixes this properly.

this is a regression introduced in PR #12617
2021-09-02 17:45:30 -07:00
Poorna Krishnamoorthy
a366143c5b
Remove replication permission check (#13135)
Fixes #13105
2021-09-02 09:31:13 -07:00
Harshavardhana
b8f5a7db33 update console release to v0.9.6 2021-09-01 23:57:19 -07:00
Harshavardhana
f486cfae86 point all docs for containers to quay.io 2021-09-01 18:48:26 -07:00
Harshavardhana
0838732df3 point helm releases to quay.io 2021-09-01 18:47:43 -07:00
Harshavardhana
f422f09dff update console to latest master 2021-09-01 18:10:44 -07:00
Harshavardhana
aff196ae57 add helm chart support for gateway 2021-09-01 16:27:12 -07:00
Harshavardhana
27c9f8be7a ignore MINIO_CERT_PASSWD to be same on all instances 2021-09-01 16:02:35 -07:00
Harshavardhana
67b6c945e2 whitelist certain ENVs from bootstrap check 2021-09-01 15:10:17 -07:00
Harshavardhana
c89aee37b9
fix: log errors for incorrect environment inputs (#13121)
Invalid MINIO_ARGS, MINIO_ENDPOINTS would be silently
ignored when using remoteEnv style, make sure to log
errors to indicate invalid configuration.
2021-09-01 11:34:07 -07:00
Harshavardhana
03b7bebc96
fix: invalid quorum calculation in TransitionObject (#13125)
Quorum calculation should be based on the
existing metadata, custom quorum calculation
can lead to unreadable content.
2021-09-01 08:57:42 -07:00
Harshavardhana
f89d0f68d0
fix: missing cleanup of tmp folders in NAS gateway setup (#13124)
console service should be shutdown last once all shutdown
sequences are complete, this is to ensure that we do not
prematurely kill the server before it cleans up the

`.minio.sys/tmp/uuid` folder.

NOTE: this only applies to NAS gateway setup.
2021-08-31 18:52:48 -07:00
Harshavardhana
72a288f73f
fix: add x-amz-server-side-encryption header in ListObjectsM (#13122)
fixes #13046
2021-08-31 17:18:13 -07:00
Harshavardhana
0073aee1ed update to console release v0.9.5 2021-08-31 15:18:43 -07:00
Harshavardhana
0f7a51f461
fix: speedTest between peers keep the connection alive (#13120)
for longer durations keep the speedTest alive instead
of timing them out based on ResponseHeaderTimeout.
2021-08-31 14:08:23 -07:00
Klaus Post
556552340a
listing: Don't log errFileNotFound and friends (#13119) 2021-08-31 09:46:42 -07:00
Harshavardhana
50bce0130a release helm/minio v2.0.1 upgrade MinIO 2021-08-31 02:09:09 -07:00
Minio Trusted
5c6dc63577 Update yaml files to latest version RELEASE.2021-08-31T05-46-54Z 2021-08-31 07:27:43 +00:00
Harshavardhana
2077d27053 init() globalAPIConfig only once 2021-08-30 22:46:54 -07:00
Klaus Post
76b3d3c559
fix: missing close on error for keepAlive connections (#13109)
Add missing close when error is reported
before body is done.
2021-08-30 08:46:46 -07:00
Dominik Hassler
514b2d6f12
fix: build on illumos (Solaris) (#13097) 2021-08-30 08:40:16 -07:00
Klaus Post
470553ff5d
Tweak readall allocation and renameData buffer reuse (#13108)
Use a single allocation for reading the file, not the growing buffer of `io.ReadAll`.

Reuse the write buffer if we can when writing metadata in RenameData.
2021-08-30 08:38:11 -07:00
Harshavardhana
88e0aa1cb2
verify all nodes have same ENVs in bootstrap (#13096) 2021-08-30 08:27:39 -07:00
Harshavardhana
35f2552fc5
reduce extra getObjectInfo() calls during ILM transition (#13091)
* reduce extra getObjectInfo() calls during ILM transition

This PR also changes expiration logic to be non-blocking,
scanner is now free from additional costs incurred due
to slower object layer calls and hitting the drives.

* move verifying expiration inside locks
2021-08-27 17:06:47 -07:00
Anis Elleuch
e05886561d
lock: Fix Refresh logic with multi resources lock (#13092)
A multi resources lock is a single lock UID with multiple associated
resources. This is created for example by multi objects delete
operation. This commit changes the behavior of Refresh() to iterate over
all locks having the same UID and refresh them.

Bonus: Fix showing top locks for multi delete objects
2021-08-27 13:07:55 -07:00
Klaus Post
2451b9a75a
fix: hanging operations on PUT with slow IO (#13087)
#11878 added "keepHTTPResponseAlive" to CreateFile requests. 
The problem is that it will begin writing to the response before the 
body is read after 10 seconds. This will abort the writes on the 
client-side, since it assumes the server has received what it wants.

The proposed solution here is to monitor the completion of the body 
before beginning to send keepalive pings.

Fixes observed high number of goroutines stuck in `io.Copy` in 
`github.com/minio/minio/cmd.(*xlStorage).CreateFile` and 
`(*storageRESTClient).CreateFile` stuck in `http.DrainBody`.
2021-08-27 09:16:36 -07:00
Anis Elleuch
06b71c99ee
locks: Ensure local lock removal after a failed refresh (#12979)
In the event when a lock is not refreshed in the cluster, this latter
will be automatically removed in the subsequent cleanup of non 
refreshed locks routine, but it forgot to clean the local server, 
hence having the same weird stale locks present.

This commit will remove the lock locally also in remote nodes, if
removing a lock from a remote node will fail, it will be anyway 
removed later in the locks cleanup routine.
2021-08-27 08:59:36 -07:00
Harshavardhana
ae8f7f11d5
fix: svc accounts cannot have same name as parent/targetUser (#13082)
Currently in master this can cause existing
parent users to stop working and lead to
credentials getting overwritten.

```
~ mc admin user add alias/ minio123 minio123456
```

```
~ mc admin user svcacct add alias/ minio123 \
    --access-key minio123 --secret-key minio123456
```

This PR rejects all such scenarios.
2021-08-26 21:57:30 -07:00
Harshavardhana
ed16ce9b73
add healing workers support to parallelize healing (#13081)
Faster healing as well as making healing more
responsive for faster scanner times.

also fixes a bug introduced in #13079, newly replaced
disks were not healing automatically.
2021-08-26 20:32:58 -07:00
Poorna Krishnamoorthy
27f895cf2c
Check pathlength before reading metadata (#13080)
fixes bug where the server returns 503 instead of 400 if 
objectName is longer than 255 characters

Fixes regression introduced in #12942
2021-08-26 16:23:12 -07:00
Harshavardhana
c11a2ac396
refactor healing to remove certain structs (#13079)
- remove sourceCh usage from healing
  we already have tasks and resp channel

- use read locks to lookup globalHealConfig

- fix healing resolver to pick candidates quickly
  that need healing, without this resolver was
  unexpectedly skipping.
2021-08-26 14:06:04 -07:00
Harshavardhana
2f9ab26372
rename zones to pools in helm-chart (#13073) 2021-08-26 00:36:46 -07:00
Harshavardhana
0559f46bbb
fix: make healObject() make non-blocking (#13071)
healObject() should be non-blocking to ensure
that scanner is not blocked for a long time,
this adversely affects performance of the scanner
and also affects the way usage is updated
subsequently.

This PR allows for a non-blocking behavior for
healing, dropping operations that cannot be queued
anymore.
2021-08-25 17:46:20 -07:00
Harshavardhana
6e5f83c45b fix: helm service release and update to v1.0.5 2021-08-25 12:53:57 -07:00
Klaus Post
e1b0582859
fsOpenFile: Close on error (#13064)
Close files on error.
2021-08-25 09:43:01 -07:00
Klaus Post
88d719689c
Synchronize bucket cycle numbers (#13058)
Synchronize bucket cycles so it is much more
likely that the same prefixes will be picked up
for scanning.

Use the global bloom filter cycle for that. 
Bump bloom filter versions to clear those.
2021-08-25 08:25:26 -07:00
Harshavardhana
200eb8dc0e
fix: remove any internal metadata keys from notification (#13062) 2021-08-24 21:13:37 -07:00
Harshavardhana
082650bea3 update helm version to v1.0.4 2021-08-24 19:12:51 -07:00
Minio Trusted
abf079135e Update yaml files to latest version RELEASE.2021-08-25T00-41-18Z 2021-08-25 02:01:05 +00:00
Harshavardhana
f00e8bc107 update console v0.9.4 2021-08-24 17:41:18 -07:00
Shireesh Anjal
ce05e67a0c
Add admin api to return sys config info (#12988)
The intention is to list values of sys config that can potentially
impact the performance of minio.

At present, it will return max value configured for rlimit

Signed-off-by: Shireesh Anjal <shireesh@minio.io>

Co-authored-by: Harshavardhana <harsha@minio.io>
2021-08-24 17:09:37 -07:00
Harshavardhana
fecb1b0489 fix: look to exact return returned from initServer() in tests 2021-08-24 15:27:02 -07:00
Poorna Krishnamoorthy
6a7e22386e
Use part sizes correctly in multipart replication (#13061)
fixes #13057
2021-08-24 14:41:05 -07:00
Harshavardhana
85dfb4351c
fix: allow an entire set to be dropped (#13060)
proceed to heal the cluster when all the
drives in a set have failed, this is extremely
rare occurrence but even if it happens we allow
the cluster to be functional.
2021-08-24 12:43:57 -07:00
Harshavardhana
addf15f61f update console version to v0.9.3 2021-08-24 12:40:16 -07:00
Harshavardhana
bbf3576f70
remove unecessary metadata structs in applyTransitionAction() (#13059) 2021-08-24 12:24:00 -07:00
Harshavardhana
da3f4bd452 update helm release to v1.0.3 2021-08-24 12:04:07 -07:00
Harshavardhana
5eb6f903f2 fix: helm missing console ingress apiVersion
fixes #13049
2021-08-24 11:58:05 -07:00
Nitish Tiwari
60394ddf83
Add support for changing job name in Grafana dashboard (#13050) 2021-08-24 09:51:09 -07:00
Harshavardhana
293d261cf9
use available memory to restrict API calls (#13047)
also choose 90% of the available memory
to calculate maximum API calls.
2021-08-24 09:14:46 -07:00