Commit graph

11558 commits

Author SHA1 Message Date
Minio Trusted b1a109a611 Update yaml files to latest version RELEASE.2023-12-20T01-00-02Z 2023-12-20 08:43:46 +00:00
Cesar N 7a311a3b66
Update Console version to v0.43.1 (#18683) 2023-12-19 17:00:02 -08:00
Poorna d55b6b9909
Fix quota config replication for SR (#18684)
Fixing regression introduced by PR #17988
2023-12-19 13:22:47 -08:00
Harshavardhana f4389fb322
update dependencies for https://pkg.go.dev/vuln/GO-2023-2402 (#18676) 2023-12-18 18:35:20 -08:00
Daniel Jakots 331208bec1
Enable cross compile for openbsd/amd64 (#18666) 2023-12-18 17:08:22 -08:00
Shireesh Anjal 7680e5f81d
Read new key license_v2 from SUBNET response (#18669)
SUBNET now has a v2 of license that is returned in the new key
`license_v2`. mc will start reading and storing the same. (The old key
`license` is deprecated but is still available in SUBNET response to
ensure that the current released version of minio doesn't break)
2023-12-18 08:21:44 -08:00
Anugrah Vijay 6acf038a84
docs: fix bucket metrics API\ path in docs (#18661) 2023-12-18 08:21:08 -08:00
Harshavardhana bdf4e386cf
update go dependencies (#18662) 2023-12-15 21:35:07 -08:00
Taran Pelkey ad8a34858f
Add APIs to create and list access keys for LDAP (#18402) 2023-12-15 13:00:43 -08:00
Krishnan Parthasarathi 162eced7d2
Fix incorrect metric desc for bucketRequestsDuration (#18657) 2023-12-14 19:02:11 -08:00
Krishnan Parthasarathi bec1f7c26a
metrics: Refactor handling of histogram vectors (#18632) 2023-12-14 14:02:52 -08:00
Anis Eleuch 8771617199
tier: Add support of AWS S3 tiering with web identity token file (#18648) 2023-12-14 14:01:49 -08:00
Minio Trusted 54bc995f0a Update yaml files to latest version RELEASE.2023-12-14T18-51-57Z 2023-12-14 19:54:40 +00:00
Klaus Post 6c89a81af4
Fix CreateFile shared buffer corruption. (#18652)
`(*xlStorageDiskIDCheck).CreateFile` wraps the incoming reader in `xioutil.NewDeadlineReader`.

The wrapped reader is handed to `(*xlStorage).CreateFile`. This performs a Read call via `writeAllDirect`, 
which reads into an `ODirectPool` buffer.

`(*DeadlineReader).Read` spawns an async read into the buffer. If a timeout is hit while reading, 
the read operation returns to `writeAllDirect`. The operation returns an error and the buffer is reused.

However, if the async `Read` call unblocks, it will write to the now recycled buffer.

Fix: Remove the `DeadlineReader` - it is inherently unsafe. Instead, rely on the network timeouts. 
This is not a disk timeout, anyway.

Regression in https://github.com/minio/minio/pull/17745
2023-12-14 10:51:57 -08:00
Harshavardhana 8fa2898ff1
update console UI to v0.43.0 release (#18653) 2023-12-14 10:50:54 -08:00
Praveen raj Mani 10ca0a6936
Label the notification target metrics by their target IDs (#18633)
This patch adds the targetID to the existing notification target metrics
and deprecates the current target metrics which points to the overall
event notification subsystem
2023-12-14 09:09:26 -08:00
Harshavardhana b3314e97a6
re-use the same local drive used by remote-peer (#18645)
historically, we have always kept storage-rest-server
and a local storage API separate without much trouble,
since they both can independently operate due to no
special state() between them.

however, over some time, we have added state()
such as

- drive monitoring threads now there will be "2" of
  them per drive instead of just 1.

- concurrent tokens available per drive are now twice
  instead of just single shared, allowing unexpectedly
  high amount of I/O to go through.

- applying serialization by using walkMutexes can now
  be adequately honored for both remote callers and local
  callers.
2023-12-13 19:27:55 -08:00
Minio Trusted 3b9a948045 Update yaml files to latest version RELEASE.2023-12-13T23-28-55Z 2023-12-14 03:04:03 +00:00
Poorna 3781a0f9ad
replication: Pass metadata timestamps in CopyObject call (#18647)
Regression from #18285. CopyObject options were inheriting source MTime
for metadata timestamps if unspecified, removing this prevented metadata
updates from being applied on target.
2023-12-13 15:28:55 -08:00
Poorna e79b289325
fix datadir missing check on HeadObject (#18646)
versions pending purge in replication were seeing a errFileCorrupt
that prevents permanent deletion after replication.

Regression from PR#18477
2023-12-13 14:54:01 -08:00
Shubhendu 6d4c1156d6
Changed the expression to render the value (#18627)
The metrics `minio_bucket_replication_received_bytes` and
`minio_bucket_replication_sent_bytes` are additive in nature
and rendering the value as is looks fine.

Also added sort order for few graphs for better reading of tool
tips as keeping ones with highest value at top helps.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2023-12-13 10:05:47 -08:00
Harshavardhana 3f72c7fcc7
healthcheck requests with user-agent mozilla do not need redirects (#18642)
apparently, windows powershell curl has this abhorrent behavior
2023-12-12 16:16:26 -08:00
Harshavardhana d521c84d55
reduce logging during permission denied errors (#18641)
log them if any only once
2023-12-12 16:11:17 -08:00
opencmit2 946b070744
resolve the absence of the sed command in ubi-micro image (#18459)
fix #18315
2023-12-12 15:43:32 -08:00
Anis Eleuch 4a21dce2b5
tier: Add support of SP credentials with Azure (#18630)
Co-authored-by: Anis Elleuch <anis@min.io>
2023-12-11 21:51:53 -08:00
Minio Trusted 5fe7f9fa93 Update yaml files to latest version RELEASE.2023-12-09T18-17-51Z 2023-12-09 21:03:28 +00:00
Harshavardhana 65f34cd823
fix: remove ODirectReader entirely since we do not need it anymore (#18619) 2023-12-09 10:17:51 -08:00
Harshavardhana 196e7e072b
allow bitrot files to be healed in MRF (#18618)
bitrot scanMode was ignored in MRF,
allow it to heal relevant content if
needed when seen as an error.
2023-12-08 12:26:01 -08:00
Anis Eleuch 6f97663174
yml-config: Add support of rootUser and rootPassword (#18615)
Users can define the root user and password in the yaml configuration
file; Root credentials defined in the environment variable still take
precedence
2023-12-08 12:04:54 -08:00
Anis Eleuch aed7a1818a
info: Populate pool/set/disk indexes for offline disks (#18613)
This can be calculated from the disk layout and some external
applications would like to know the location of the offline
disks.
2023-12-08 08:13:04 -08:00
Harshavardhana b50d90183e
use latest minisign release (#18614) 2023-12-08 00:22:22 -08:00
Poorna 6b06da76cb
add configuration to limit replication workers (#18601) 2023-12-07 16:22:00 -08:00
jiuker 6ca6788bb7
feat: add events_errors_total metric (#18610) 2023-12-07 16:21:17 -08:00
Anis Eleuch 2e23e61a45
Add support of conf file to pass arguments and options (#18592) 2023-12-07 01:33:56 -08:00
Minio Trusted 9cdf490bc5 Update yaml files to latest version RELEASE.2023-12-07T04-16-00Z 2023-12-07 06:28:15 +00:00
Alex cfed671ea3
Update Console version to v0.42.1 (#18606)
Signed-off-by: Benjamin Perez <benjamin@bexsoft.net>
2023-12-06 20:16:00 -08:00
Harshavardhana 53ce92b9ca
fix: use the right channel to feed the data in (#18605)
this PR fixes a regression in batch replication
where we weren't sending any data from the Walk()
results due to incorrect channels being used.
2023-12-06 18:17:03 -08:00
Shireesh Anjal 7350a29fec
Capture percentage of cpu load and memory used (#18596)
By default the cpu load is the cumulative of all cores. Capture the
percentage load (load * 100 / cpu-count)

Also capture the percentage memory used (used * 100 / total)
2023-12-06 13:19:59 -08:00
jiuker 5cc2c62c66
fix: GetFreePort() will get the same port (#18604) 2023-12-06 10:36:42 -08:00
Harshavardhana 4bc5ed6c76
support LDAP service accounts via SFTP, FTP logins (#18599) 2023-12-06 04:31:35 -08:00
Minio Trusted e99a597899 Update yaml files to latest version RELEASE.2023-12-06T09-09-22Z 2023-12-06 10:16:37 +00:00
Harshavardhana 73dde66dbe
stick to go1.19 go.mod (#18600) 2023-12-06 01:09:22 -08:00
Harshavardhana e30c0e7ca3 Revert "Heal buckets at node level (#18504)"
This reverts commit 708296ae1b.
2023-12-05 22:34:46 -08:00
Klaus Post 8fc200c0cc
Truncate long traces for internode communication (#18593)
Prevent excessively long request traces.
2023-12-05 12:16:48 -08:00
Shubhendu 708296ae1b
Heal buckets at node level (#18504) 2023-12-05 02:17:35 -08:00
Harshavardhana fbb5e75e01
avoid run-away goroutine build-up in notification send, use channels (#18533)
use memory for async events when necessary and dequeue them as
needed, for all synchronous events customers must enable

```
MINIO_API_SYNC_EVENTS=on
```

Async events can be lost but is upto to the admin to
decide what they want, we will not create run-away number
of goroutines per event instead we will queue them properly.

Currently the max async workers is set to runtime.GOMAXPROCS(0)
which is more than sufficient in general, but it can be made
configurable in future but may not be needed.
2023-12-05 02:16:33 -08:00
Harshavardhana f327b21557
handle crashes with ILM expiry changes (#18590) 2023-12-05 01:14:36 -08:00
Harshavardhana 45b7253f39
parallelize renameData() cleanup upon error (#18591) 2023-12-04 14:54:34 -08:00
Harshavardhana 05bb655efc
avoid caching metrics for timeout errors per drive (#18584)
Bonus: combine the loop for drive/REST registration.
2023-12-04 11:54:13 -08:00
Harshavardhana 8fdfcfb562
upon RenameData() quorum error delete any partial success (#18586)
there is potential for danglingWrites when quorum failed, where
only some drives took a successful write, generally this is left
to the healing routine to pick it up. However it is better that
we delete it right away to avoid potential for quorum issues on
version signature when there are many versions of an object.
2023-12-04 11:33:39 -08:00