Commit graph

9555 commits

Author SHA1 Message Date
Harshavardhana 401958938d
add load balance properly restClientFromHash() bucket/prefix (#14621)
spread out resuming further to other nodes
2022-03-25 03:41:31 -07:00
Poorna 566cffe53d
save format.json by default for inspect API (#14620) 2022-03-25 02:02:17 -07:00
Minio Trusted 028bc2f9be update console release to v0.15.6 2022-03-24 19:59:15 -07:00
Minio Trusted 813d9bc316 update helm release 2022-03-23 21:07:15 -07:00
Aditya Manthramurthy 79ba458051
fix: free up reader resources in S3Select properly (#14600) 2022-03-23 20:58:53 -07:00
Minio Trusted cf220be9b5 Update yaml files to latest version RELEASE.2022-03-24T00-43-44Z 2022-03-24 01:28:05 +00:00
Harshavardhana c433572585
update go mod to go1.16 deps (#14614) 2022-03-23 17:43:44 -07:00
Minio Trusted a42b576382 keep maximum concurrent operations to 512 (to sustain upto 1024 open fds) 2022-03-23 17:02:04 -07:00
Avimitin fb9b53026d
Add riscv64 support (#14601)
In riscv64, the `syscall.Uname` function will return a uint8 slice.

    func main() {
      var buf syscall.Utsname
      fmt.Printf("Buffer Type: %T\n", buf.Release)
    }

    output:
      Buffer Type: [65]uint8

This is tested in the Arch Linux RISC-V 64 QEMU environment.

Signed-off-by: Avimitin <avimitin@gmail.com>
2022-03-22 20:36:59 -07:00
Klaus Post 2ac54e5a7b
ListObjects: Filter lifecycle expired objects (#14606)
For ListObjects and ListObjectsV2 perform lifecycle checks on 
all objects before returning. This will filter out objects that are 
pending lifecycle expiration.

Bonus: Cheaper server pool conflict resolution by not converting to FileInfo.
2022-03-22 12:39:45 -07:00
Harshavardhana 8eecdc6d1f
odd stripe sizes should choose (odd+1)/2 to get correct quorum (#14610) 2022-03-22 12:21:14 -07:00
Klaus Post 50577e2bd2
Allow adjusting request pool both ways (#14609)
When reloading a dynamic config allow the request pool to scale both ways.

Existing requests hold on to the previous pool, so they will pop the elements from that.
2022-03-22 11:28:54 -07:00
Klaus Post 7bc1f986e8
Do not wait for results when canceled (#14607)
When canceled nobody may be listening for the results.

Prevents memory buildup from cancelled requests.
2022-03-22 09:37:01 -07:00
Harshavardhana d796621ccc
choose smaller default deadline for diagnostics without --full (#14599) 2022-03-21 23:25:24 -07:00
Minio Trusted 751e9fb7be Update yaml files to latest version RELEASE.2022-03-22T02-05-10Z 2022-03-22 02:45:24 +00:00
Harshavardhana f6113264f4 add detection for GOMAXPROCS < NumCPU 2022-03-21 19:05:10 -07:00
Harshavardhana a3534a730b
fallback quorum should be "strict" globally if config is not loaded (#14589) 2022-03-20 17:39:06 -07:00
Minio Trusted 7f8b8a0e43 update console to v0.15.4 2022-03-20 15:35:20 -07:00
Harshavardhana bd6f7b6d83
fix: make decommission restart non-blocking (#14591)
currently an on-going decommission, during a server
restart might block the startup sequence for relatively
longer periods, instead start the decommission in
background lazily.
2022-03-20 14:46:43 -07:00
Andreas Auernhammer b0a4beb66a
PutObjectPart: set SSE-KMS headers and truncate ETags. (#14578)
This commit fixes two bugs in the `PutObjectPartHandler`.
First, `PutObjectPart` should return SSE-KMS headers
when the object is encrypted using SSE-KMS.
Before, this was not the case.

Second, the ETag should always be a 16 byte hex string,
perhaps followed by a `-X` (where `X` is the number of parts).
However, `PutObjectPart` used to return the encrypted ETag
in case of SSE-KMS. This leaks MinIO internal etag details
through the S3 API.

The combination of both bugs causes clients that use SSE-KMS
to fail when trying to validate the ETag. Since `PutObjectPart`
did not send the SSE-KMS response headers, the response looked
like a plaintext `PutObjectPart` response. Hence, the client
tries to verify that the ETag is the content-md5 of the part.
This could never be the case, since MinIO used to return the
encrypted ETag.

Therefore, clients behaving as specified by the S3 protocol
tried to verify the ETag in a situation they should not.

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-03-19 10:15:12 -07:00
Klaus Post 472c2d828c
Fix waitgroup add after wait on config reload (#14584)
Fix `panic: "POST /minio/peer/v21/signalservice?signal=2": sync: WaitGroup is reused before previous Wait has returned`

Log entries already on the channel would cause `logEntry` to increment the
 waitgroup when sending messages, after Cancel has been called.

Instead of tracking every single message, just check the send goroutine. Faster 
and safe, since it will not decrement until the channel is closed.

Regression from #14289
2022-03-19 09:15:45 -07:00
Harshavardhana 01ee49045e
fix: handle race in server setup global CI/CD variable (#14579) 2022-03-18 18:21:09 -07:00
Harshavardhana 7bd9f821dd
return correct context errors for locking operations (#14569)
if a context is canceled do not need to return a timeout error
instead, return the appropriate error for context canceled.
2022-03-18 15:32:45 -07:00
Anis Elleuch b20ecc7b54
Add support of TLS session tickets with KES server (#14577)
Reduce overhead for communication between MinIO server and KES server.
2022-03-18 15:14:10 -07:00
Klaus Post 61eb9d4e29
Fix listing fallback re-using disks (#14576)
When more than 2 disks are unavailable for listing, the same disk will be used for fallback.

This makes quorum calculations incorrect since the same disk will have multiple entries.

This PR keeps track of which fallback disks have been handed out and only every returns a disk once.
2022-03-18 11:35:27 -07:00
Harshavardhana 43eb5a001c
re-use transport for AdminInfo() call (#14571)
avoids creating new transport for each `isServerResolvable`
request, instead re-use the available global transport and do
not try to forcibly close connections to avoid TIME_WAIT
build upon large clusters.

Never use httpClient.CloseIdleConnections() since that can have
a drastic effect on existing connections on the transport pool.

Remove it everywhere.
2022-03-17 16:20:10 -07:00
Minio Trusted f58692abb7 update helm to v3.6.2 2022-03-17 11:30:55 -07:00
Klaus Post c1760fb764
Move apiCalls to front for field alignment (#14568)
Fixes #14565
2022-03-17 10:57:52 -07:00
Minio Trusted e9bc0e7e98 Update yaml files to latest version RELEASE.2022-03-17T06-34-49Z 2022-03-17 00:11:59 -07:00
Minio Trusted ffcadcd99e Revert "Use S3 client for uplooads/downloads during perf test (#14553)"
This reverts commit ff811f594b.

Speedtest is broken need to fix this more cleanly.
2022-03-16 23:34:49 -07:00
Minio Trusted 7a733a8d54 Update yaml files to latest version RELEASE.2022-03-17T02-57-36Z 2022-03-16 22:27:48 -07:00
Aditya Manthramurthy ce97313fda
Add extra LDAP configuration validation (#14535)
- The result now contains suggestions on fixing common configuration issues.
- These suggestions will subsequently be exposed in console/mc
2022-03-16 19:57:36 -07:00
Krishnan Parthasarathi 7b81967a3c
Fix handling of object versions pending purge (#14555)
- GetObject() with vid should return 405
- GetObject() without vid should return 404
- ListObjects() should ignore this object if this is the "latest" version of the object
- ListObjectVersions() should list this object as "DELETE marker"
- Remove data parts before sync'ing the version pending purge
2022-03-16 16:59:43 -07:00
Krishna Srinivas ff811f594b
Use S3 client for uplooads/downloads during perf test (#14553) 2022-03-16 16:58:46 -07:00
Harshavardhana 0bf80b3c89 update console v0.15.3 2022-03-16 01:19:00 -07:00
Harshavardhana ae3b369fe1
logger webhook failure can overrun the queue_size (#14556)
PR introduced in #13819 was incorrect and was not
handling the situation where a buffer is full can
cause incessant amount of logs that would keep the
logger webhook overrun by the requests.

To avoid this only log failures to console logger
instead of all targets as it can cause self reference,
leading to an infinite loop.
2022-03-15 17:45:51 -07:00
Kourosh Tafreshi 77b15e7194
Add Console Service port to the NetworkPolicy (#14545) 2022-03-14 17:13:42 -07:00
Harshavardhana 20537f974e add missing v3.6.1 tarball 2022-03-14 17:13:17 -07:00
Harshavardhana 4476a64bdf update helm to v3.6.1 2022-03-14 14:40:24 -07:00
Steven Meyer d4b701576e
Fix helm chart k8s version comparison (#14552) 2022-03-14 14:39:32 -07:00
Minio Trusted 721c053712 Update yaml files to latest version RELEASE.2022-03-14T18-25-24Z 2022-03-14 19:32:22 +00:00
Harshavardhana e3071157f0
allow MakeBucketLocation to work for metaBucket (#14548)
decommission would fail to start due to failure
in MakeBucketLocation() error on .minio.sys/ bucket
creation.

Allow these special buckets.
2022-03-14 11:25:24 -07:00
Klaus Post c07af89e48
select: Add ScanRange to CSV&JSON (#14546)
Implements https://docs.aws.amazon.com/AmazonS3/latest/API/API_SelectObjectContent.html#AmazonS3-SelectObjectContent-request-ScanRange

Fixes #14539
2022-03-14 09:48:36 -07:00
Harshavardhana 9c846106fa
decouple service accounts from root credentials (#14534)
changing root credentials makes service accounts
in-operable, this PR changes the way sessionToken
is generated for service accounts.

It changes service account behavior to generate
sessionToken claims from its own secret instead
of using global root credential.

Existing credentials will be supported by
falling back to verify using root credential.

fixes #14530
2022-03-14 09:09:22 -07:00
Harshavardhana cf94d1f1f1
do not crash readXLMetaNoData - if the xl.meta has incorrect content (#14538)
```
tmp = buf[want:]
```

Would potentially crash when `buf` is truncated for some reason
and does not have the expected bytes, this is of course considered
not normal and is an odd situation. But we do not need to crash
here instead allow for errors to be returned and let callers handle
the errors.
2022-03-14 09:07:46 -07:00
Harshavardhana 6187440f35 update helm release v3.6.0 2022-03-13 15:44:21 -07:00
Minio Trusted 57b7c3494f Update yaml files to latest version RELEASE.2022-03-11T23-57-45Z 2022-03-13 08:47:27 +00:00
Harshavardhana dda18c28c5 Bump github.com/nats-io/nats-server/v2 from 2.7.2 to 2.7.4 2022-03-11 15:57:45 -08:00
Poorna f8d6eaaa96
fix: regression from range GET proxy on replicated buckets #14345 (#14532)
Fixes: #14531
2022-03-11 15:56:49 -08:00
Vijay Dharap 47d4fabb58
add filesystem group change policy for large minio deployments (#14528)
* add group change policy for large MinIO deployments
* Added Kubernetes version > 1.20 check for applying the proposed change
2022-03-11 14:21:58 -08:00