Commit graph

118 commits

Author SHA1 Message Date
Harshavardhana a566bcf613
treat 0-byte objects to honor same quorum as delete marker (#17633)
on unversioned buckets its possible that 0-byte objects
might lose quorum on flaky systems, allow them to be same
as DELETE markers. Since practically speak they have no
content.
2023-07-11 21:53:49 -07:00
Harshavardhana 82075e8e3a
use strconv variants to improve on performance per 'op' (#17626)
```
BenchmarkItoa
BenchmarkItoa-8         	673628088	         1.946 ns/op	       0 B/op	       0 allocs/op
BenchmarkFormatInt
BenchmarkFormatInt-8    	592919769	         2.012 ns/op	       0 B/op	       0 allocs/op
BenchmarkSprint
BenchmarkSprint-8       	26149144	        49.06 ns/op	       2 B/op	       1 allocs/op
BenchmarkSprintBool
BenchmarkSprintBool-8   	26440180	        45.92 ns/op	       4 B/op	       1 allocs/op
BenchmarkFormatBool
BenchmarkFormatBool-8   	1000000000	         0.2558 ns/op	       0 B/op	       0 allocs/op
```
2023-07-11 07:46:58 -07:00
Harshavardhana 8522905d97 update helm linting workflow 2023-07-10 20:11:51 -07:00
Ian Martin 90c2129f44
Add helm chart linting to CI workflow (#17606) 2023-07-08 15:41:12 -07:00
Anis Eleuch 66bea3942a
CI/CD to stop one node per pool in the two pools mint test (#17518)
This is to make sure that all S3 ops work when there is enough quorum
2023-07-07 00:10:13 -07:00
Harshavardhana 6b7c98bd0f
make sure we pick up the right Go version in vulncheck (#17388) 2023-06-09 00:32:12 -07:00
Harshavardhana 75c6fc4f02
only allow decryption of etag for only sse-s3 (#17335) 2023-06-05 13:08:51 -07:00
Harshavardhana 4a425cbac1
cleanup scripts and apply shfmt (#17284) 2023-05-25 22:07:25 -07:00
Harshavardhana 876f51a708 remove minio-js from mint tests until next minio-js release 2023-05-17 09:09:54 -07:00
Harshavardhana 06557fe8be
allow decommissioned pools to be removed while others are finishing (#17221) 2023-05-16 16:00:57 -07:00
Harshavardhana b53376a3a4
change directory objects to never create new versions (#17109) 2023-05-02 16:09:33 -07:00
Harshavardhana 7ae69accc0
allow root user to be disabled via config settings (#17089) 2023-04-28 12:24:14 -07:00
Klaus Post 62c3df0ca3
fix: directory listing on Go 1.20 windows (#16976) 2023-04-05 14:36:49 -07:00
Klaus Post ae011663e8
mint: Ignore teardown errors (#16979) 2023-04-05 11:10:24 -07:00
Harshavardhana f65cce4317
move mint tests to separate folders to not confuse GitHub (#16940) 2023-03-31 14:38:10 -07:00
Harshavardhana 13151cbb2b
[testing] add mint runner test (#16868) 2023-03-29 11:38:43 -07:00
Aditya Manthramurthy c95ede35c1
Switch windows CI back to go 1.19.x (#16755) 2023-03-03 15:19:28 -08:00
Aditya Manthramurthy 7415e1aa56
Switch to go1.20 in CI (#16743) 2023-03-03 10:15:03 -08:00
Cesar Celis Hernandez 422c396d73
Removing old action that is no longer needed (#16550) 2023-02-07 07:06:29 +05:30
Cesar Celis Hernandez 8ca14e6267
Updating enterprise action (#16518) 2023-02-02 19:23:31 +05:30
Alex 1a17fc17bb
GitHub Workflows security hardening (#15708) 2023-01-21 09:55:17 -08:00
Harshavardhana 75faef888e
disable builds for go1.18 (#16332) 2022-12-30 11:37:07 -08:00
Harshavardhana dfe73629a3
fix: delete marker discrepancies via DeleteObject() API (#16195) 2022-12-08 18:15:16 -08:00
Anis Elleuch 52f4124678
Remove go1.18 from Github workflow tests (#16180) 2022-12-06 09:11:20 -08:00
Cesar Celis Hernandez 90e37a8745
Start PR on Enterprise when there is new MinIO Version (#16121) 2022-12-04 02:29:25 -08:00
yanggang d2c9a9e395
add windows port allot by "netsh dynamicport" (#15986) 2022-11-02 09:10:26 -07:00
Harshavardhana 07b6dce1a5
remove nancy, we rely on vulncheck from now on (#15893) 2022-10-18 10:45:44 -07:00
Minio Trusted 5c034e26bd fix: govulncheck must use go1.19.2 2022-10-06 12:42:59 -07:00
Harshavardhana cef0fb1434
remove build asset caching (#15807) 2022-10-06 11:36:07 -07:00
Harshavardhana b04c0697e1
validate correct ETag for the parts sent during CompleteMultipart (#15751) 2022-09-23 21:17:08 -07:00
Klaus Post 5c61c3ccdc
Fix flaky TestGetObjectWithOutdatedDisks (#15687)
On occasion this test fails:

```
2022-09-12T17:22:44.6562737Z === RUN   TestGetObjectWithOutdatedDisks
2022-09-12T17:22:44.6563751Z     erasure-object_test.go:1214: Test 2: Expected data to have md5sum = `c946b71bb69c07daf25470742c967e7c`, found `7d16d23f07072af1a809707ba101ae07`
2
```

Theory: Both objects are written with the same timestamp due to lower timer resolution on Windows. This results in secondary resolution, which is deterministic, but random.

Solution: Instead of hacking in a wait we request the specific version we want. Should still keep the test relevant.

Bonus: Remote action dependency for vulncheck
2022-09-14 08:17:39 -07:00
Harshavardhana a71629d4dd
use go1.19.1 for vulncheck (#15680) 2022-09-12 20:16:39 -07:00
Andreas Auernhammer d6132b854f
add vulncheck action to CI (#15676)
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-09-12 10:10:30 -07:00
Anis Elleuch bb855499e1
Update PR template to make it easy to request for doc changes (#15665)
A link can be clicked to open a new request in minio/docs if something
in the documentation needs to be changed.
2022-09-07 15:25:02 -07:00
Harshavardhana 433b6fa8fe
upgrade golang-lint to the latest (#15600) 2022-08-26 12:52:29 -07:00
Harshavardhana d350b666ff
feat: add idempotent delete marker support (#15521)
The bottom line is delete markers are a nuisance,
most applications are not version aware and this
has simply complicated the version management.

AWS S3 gave an unnecessary complication overhead
for customers, they need to now manage these
markers by applying ILM settings and clean
them up on a regular basis.

To make matters worse all these delete markers
get replicated as well in a replicated setup,
requiring two ILM settings on each site.

This PR is an attempt to address this inferior
implementation by deviating MinIO towards an
idempotent delete marker implementation i.e
MinIO will never create any more than single
consecutive delete markers.

This significantly reduces operational overhead
by making versioning more useful for real data.

This is an S3 spec deviation for pragmatic reasons.
2022-08-18 16:41:59 -07:00
Harshavardhana a6e0ec4e6f
Add support converting non-inlined to inlined (#15444)
This is a feature to allow for inode compaction on
large clusters that use a lot of small files spread
across a large heirarchy.
2022-08-02 23:10:22 -07:00
Naveen d46cf50760
chore(deps): Included dependency review (#14958)
> Dependency Review GitHub Action in your repository to enforce dependency
> reviews on your pull requests.
> The action scans for vulnerable versions of dependencies introduced by package version
> changes in pull requests,
> and warns you about the associated security vulnerabilities.
> This gives you better visibility of what's changing in a pull request,
> and helps prevent vulnerabilities from being added to your repository.

https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-dependency-review#dependency-review-enforcement

Signed-off-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
2022-07-27 20:53:26 -07:00
Harshavardhana 4c6498d726
move all CI/CD to go1.18 (#15401) 2022-07-25 15:27:20 -07:00
Harshavardhana b6eb8dff64
Add decommission compression+encryption enabled tests (#15322)
update compression environment variables to follow
the expected sub-system style, however support fallback
mode.
2022-07-17 08:43:14 -07:00
Anis Elleuch 1cfa2e04bc
Add a github workflow test for root disk detection (#15267)
Use losetup to create fake disks, start a MinIO cluster, umount 
one disk, and fails if the mount point directory will have format.json
recreated. It should fail because the mount point directory will belong
to the root disk after unmount.
2022-07-13 16:29:44 -07:00
Harshavardhana b311abed31
decom IAM, Bucket metadata properly (#15220)
Current code incorrectly passed the
config asset object name while decommissioning,
make sure that we pass the right object name
to be hashed on the newer set of pools.

This PR fixes situations after a successful
decommission, the users and policies might go
missing due to wrong hashed set.
2022-07-04 14:02:54 -07:00
Aditya Manthramurthy addfa35d93
Add FIPS build to CI and add README.fips.md (#15038) 2022-06-04 18:25:37 -07:00
Aditya Manthramurthy 4629abd5a2
Add tests for Access Management Plugin (#14909) 2022-05-12 15:24:19 -07:00
Aditya Manthramurthy 0e502899a8
Add support for multiple OpenID providers with role policies (#14223)
- When using multiple providers, claim-based providers are not allowed. All
providers must use role policies.

- Update markdown config to allow `details` HTML element
2022-04-28 18:27:09 -07:00
Harshavardhana 507f993075
attempt to real resolve when there is a quorum failure on reads (#14613) 2022-04-20 12:49:05 -07:00
Harshavardhana 2a6a40e93b
enable go1.18.x builds (#14746) 2022-04-13 14:21:55 -07:00
Naveen 879de20edf
Set permissions for GitHub actions (#14693)
- Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions

https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs

[Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)

> Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much.

https://www.legitsecurity.com/blog/github-privilege-escalation-vulnerability

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
2022-04-11 02:45:59 -07:00
Naveen c547a4d835
Pin actions to a full length commit SHA (#14590)
- Pinned actions by SHA https://github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies
- Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions

https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#using-third-party-actions

Also, dependabot supports upgrades based on SHA.
2022-03-31 10:12:53 -07:00
Sergey Zhuk 3970204009
ci: Check for new go-version. Bump setup-go to v3 (#14598) 2022-03-25 08:56:04 -07:00