Commit graph

72 commits

Author SHA1 Message Date
Harshavardhana
28f9c477a8
fix: assume parentUser correctly for serviceAccounts (#9504)
ListServiceAccounts/DeleteServiceAccount didn't work properly
with STS credentials yet due to incorrect Parent user.
2020-05-01 08:05:14 -07:00
Harshavardhana
09571d03a5
avoid unnecessary logging in IAM (#9502) 2020-05-01 18:11:17 +05:30
Harshavardhana
1b122526aa
fix: add service account support for AssumeRole/LDAPIdentity creds (#9451)
allow generating service accounts for temporary credentials
which have a designated parent, currently OpenID is not yet
supported.

added checks to ensure that service account cannot generate
further service accounts for itself, service accounts can
never be a parent to any credential.
2020-04-28 12:49:56 -07:00
Anis Elleuch
20766069a8
add list/delete API service accounts admin API (#9402) 2020-04-24 12:10:09 -07:00
Harshavardhana
c82fa2c829
fix: load LDAP users appropriately (#9360)
This PR also fixes issues when

deletePolicy, deleteUser is idempotent so can lead to
issues when client can prematurely timeout, so a retry
call error response should be ignored when call returns
http.StatusNotFound

Fixes #9347
2020-04-16 16:22:34 -07:00
Harshavardhana
37d066b563
fix: deprecate requirement of session token for service accounts (#9320)
This PR fixes couple of behaviors with service accounts

- not need to have session token for service accounts
- service accounts can be generated by any user for themselves
  implicitly, with a valid signature.
- policy input for AddNewServiceAccount API is not fully typed
  allowing for validation before it is sent to the server.
- also bring in additional context for admin API errors if any
  when replying back to client.
- deprecate GetServiceAccount API as we do not need to reply
  back session tokens
2020-04-14 11:28:56 -07:00
Aditya Manthramurthy
6bb693488c
Fix policy setting error in LDAP setups (#9303)
Fixes #8667

In addition to the above, if the user is mapped to a policy or 
belongs in a group, the user-info API returns this information, 
but otherwise, the API will now return a non-existent user error.
2020-04-09 01:04:08 -07:00
Harshavardhana
ac07df2985
start watcher after all creds have been loaded (#9301)
start watcher after all creds have been loaded
to avoid any conflicting locks that might get
deadlocked.

Deprecate unused peer calls for LoadUsers()
2020-04-08 19:00:39 -07:00
César Nieto
3ea1be3c52
allow delete of a group with no policy set (#9288) 2020-04-08 06:03:57 -07:00
Harshavardhana
2642e12d14
fix: change policies API to return and take struct (#9181)
This allows for order guarantees in returned values
can be consumed safely by the caller to avoid any
additional parsing and validation.

Fixes #9171
2020-04-07 19:30:59 -07:00
Harshavardhana
e7276b7b9b
fix: make single locks for both IAM and object-store (#9279)
Additionally add context support for IAM sub-system
2020-04-07 14:26:39 -07:00
Harshavardhana
d8af244708
Add numeric/date policy conditions (#9233)
add new policy conditions

- NumericEquals
- NumericNotEquals
- NumericLessThan
- NumericLessThanEquals
- NumericGreaterThan
- NumericGreaterThanEquals
- DateEquals
- DateNotEquals
- DateLessThan
- DateLessThanEquals
- DateGreaterThan
- DateGreaterThanEquals
2020-04-01 00:04:25 -07:00
Anis Elleuch
791821d590
sa: Allow empty policy to indicate parent user's policy is inherited (#9185) 2020-03-23 14:17:18 -07:00
Harshavardhana
ea18e51f4d
Support multiple LDAP OU's, smAccountName support (#9139)
Fixes #8532
2020-03-21 22:47:26 -07:00
Anis Elleuch
496f4a7dc7
Add service account type in IAM (#9029) 2020-03-17 10:36:13 -07:00
Harshavardhana
ab7d3cd508
fix: Speed up multi-object delete by taking bulk locks (#8974)
Change distributed locking to allow taking bulk locks
across objects, reduces usually 1000 calls to 1.

Also allows for situations where multiple clients sends
delete requests to objects with following names

```
{1,2,3,4,5}
```

```
{5,4,3,2,1}
```

will block and ensure that we do not fail the request
on each other.
2020-02-21 11:29:57 +05:30
Harshavardhana
16a6e68d7b
fix: indicate PutBucketEncryption as a valid policy action (#9009) 2020-02-18 10:32:53 -08:00
Anis Elleuch
52bdbcd046
Add new admin API to return Accounting Usage (#8689) 2020-02-04 18:20:39 -08:00
poornas
301c50b721
Add canned diagnostics policy for admin users (#8937) 2020-02-04 17:58:38 -08:00
Harshavardhana
d76160c245
Initialize only one retry timer for all sub-systems (#8913)
Also make sure that we create buckets on all zones
successfully, do not run quick heal buckets if not
running with expansion.
2020-02-02 06:37:43 +05:30
Harshavardhana
fe5d599802 fix: STS creds without "aud" should be honored with STS checks (#8868)
Fixes #8865
2020-01-22 15:09:46 +05:30
Aditya Manthramurthy
55063906b5 Fix group add/remove membership bug (#8877) 2020-01-21 19:00:41 -08:00
Harshavardhana
abc1c1070a Add custom policy claim name (#8764)
In certain organizations policy claim names
can be not just 'policy' but also things like
'roles', the value of this field might also
be *string* or *[]string* support this as well

In this PR we are still not supporting multiple
policies per STS account which will require a
more comprehensive change.
2020-01-08 17:21:58 -08:00
Harshavardhana
99ad445260
Avoid double for loops in notification init (#8691) 2019-12-24 13:49:48 -08:00
Aditya Manthramurthy
01468d5a75 Fix user and policy deletion IAM commands (#8683) 2019-12-20 14:42:08 -08:00
Harshavardhana
586614c73f fix: temp credentials shouldn't allow policy/group changes (#8675)
This PR fixes the issue where we might allow policy changes
for temporary credentials out of band, this situation allows
privilege escalation for those temporary credentials. We
should disallow any external actions on temporary creds
as a practice and we should clearly differentiate which
are static and which are temporary credentials.

Refer #8667
2019-12-19 14:21:21 -08:00
Harshavardhana
3e9ab5f4a9
Fix k8s replica set deployment (#8629)
In replica sets, hosts resolve to localhost
IP automatically until the deployment fully
comes up. To avoid this issue we need to
wait for such resolution.
2019-12-10 20:28:22 -08:00
Harshavardhana
c9940d8c3f Final changes to config sub-system (#8600)
- Introduces changes such as certain types of
  errors that can be ignored or which need to 
  go into safe mode.
- Update help text as per the review
2019-12-04 15:32:37 -08:00
Harshavardhana
26a866a202
Fix review comments and new changes in config (#8515)
- Migrate and save only settings which are enabled
- Rename logger_http to logger_webhook and
  logger_http_audit to audit_webhook
- No more pretty printing comments, comment
  is a key=value pair now.
- Avoid quotes on values which do not have space in them
- `state="on"` is implicit for all SetConfigKV unless
  specified explicitly as `state="off"`
- Disabled IAM users should be disabled always
2019-11-13 17:38:05 -08:00
Harshavardhana
e9b2bf00ad Support MinIO to be deployed on more than 32 nodes (#8492)
This PR implements locking from a global entity into
a more localized set level entity, allowing for locks
to be held only on the resources which are writing
to a collection of disks rather than a global level.

In this process this PR also removes the top-level
limit of 32 nodes to an unlimited number of nodes. This
is a precursor change before bring in bucket expansion.
2019-11-13 12:17:45 -08:00
Harshavardhana
d97d53bddc
Honor etcd legacy v/s new config settings properly (#8510)
This PR also fixes issues related to

- Add proper newline for `mc admin config get` output
  for more than one targets
- Fixes issue of temporary user credentials to have
  consistent output
- Fixes a crash when setting a key with empty values
- Fixes a parsing issue with `mc admin config history`
- Fixes gateway ENV handling for etcd server and gateway
2019-11-12 03:16:25 -08:00
Harshavardhana
822eb5ddc7 Bring in safe mode support (#8478)
This PR refactors object layer handling such
that upon failure in sub-system initialization
server reaches a stage of safe-mode operation
wherein only certain API operations are enabled
and available.

This allows for fixing many scenarios such as

 - incorrect configuration in vault, etcd,
   notification targets
 - missing files, incomplete config migrations
   unable to read encrypted content etc
 - any other issues related to notification,
   policies, lifecycle etc
2019-11-09 09:27:23 -08:00
Harshavardhana
9e7a3e6adc Extend further validation of config values (#8469)
- This PR allows config KVS to be validated properly
  without being affected by ENV overrides, rejects
  invalid values during set operation

- Expands unit tests and refactors the error handling
  for notification targets, returns error instead of
  ignoring targets for invalid KVS

- Does all the prep-work for implementing safe-mode
  style operation for MinIO server, introduces a new
  global variable to toggle safe mode based operations
  NOTE: this PR itself doesn't provide safe mode operations
2019-10-30 23:39:09 -07:00
Harshavardhana
ee4a6a823d Migrate config to KV data format (#8392)
- adding oauth support to MinIO browser (#8400) by @kanagaraj
- supports multi-line get/set/del for all config fields
- add support for comments, allow toggle
- add extensive validation of config before saving
- support MinIO browser to support proper claims, using STS tokens
- env support for all config parameters, legacy envs are also
  supported with all documentation now pointing to latest ENVs
- preserve accessKey/secretKey from FS mode setups
- add history support implements three APIs
  - ClearHistory
  - RestoreHistory
  - ListHistory
- add help command support for each config parameters
- all the bug fixes after migration to KV, and other bug
  fixes encountered during testing.
2019-10-22 22:59:13 -07:00
Harshavardhana
d48fd6fde9
Remove unusued params and functions (#8399) 2019-10-15 18:35:41 -07:00
Harshavardhana
ff5bf51952 admin/heal: Fix deep healing to heal objects under more conditions (#8321)
- Heal if the part.1 is truncated from its original size
- Heal if the part.1 fails while being verified in between
- Heal if the part.1 fails while being at a certain offset

Other cleanups include make sure to flush the HTTP responses
properly from storage-rest-server, avoid using 'defer' to
improve call latency. 'defer' incurs latency avoid them
in our hot-paths such as storage-rest handlers.

Fixes #8319
2019-10-02 01:42:15 +05:30
Harshavardhana
fd53057654 Add InfoCannedPolicy API to fetch only necessary policy (#8307)
This PR adds
- InfoCannedPolicy() API for efficiency in fetching policies
- Send group memberships for LDAPUser if available
2019-09-26 23:53:13 +05:30
Aditya Manthramurthy
a0456ce940 LDAP STS API (#8091)
Add LDAP based users-groups system

This change adds support to integrate an LDAP server for user
authentication. This works via a custom STS API for LDAP. Each user
accessing the MinIO who can be authenticated via LDAP receives
temporary credentials to access the MinIO server.

LDAP is enabled only over TLS.

User groups are also supported via LDAP. The administrator may
configure an LDAP search query to find the group attribute of a user -
this may correspond to any attribute in the LDAP tree (that the user
has access to view). One or more groups may be returned by such a
query.

A group is mapped to an IAM policy in the usual way, and the server
enforces a policy corresponding to all the groups and the user's own
mapped policy.

When LDAP is configured, the internal MinIO users system is disabled.
2019-09-10 04:42:29 +05:30
Aditya Manthramurthy
825e29f301 Check if user or group is disabled when evaluating policy (#8078) 2019-08-14 16:59:16 -07:00
Aditya Manthramurthy
bf9b619d86 Set the policy mapping for a user or group (#8036)
Add API to set policy mapping for a user or group

Contains a breaking Admin APIs change.

- Also enforce all applicable policies
- Removes the previous /set-user-policy API

 Bump up peerRESTVersion

Add get user info API to show groups of a user
2019-08-13 13:41:06 -07:00
Aditya Manthramurthy
5d2b5ee6a9 Refactor IAM to use new IAMStorageAPI (#7999) 2019-08-08 15:10:04 -07:00
kannappanr
930943f058
Fix IAM users migration regression in etcd (#8029)
PR #8008 did not migrate user data stored in etcd.
This PR fixes that.
2019-08-06 17:06:31 -07:00
Harshavardhana
e6d8e272ce
Use const slashSeparator instead of "/" everywhere (#8028) 2019-08-06 12:08:58 -07:00
Harshavardhana
b52b90412b Avoid data-transfer in distributed locking (#8004) 2019-08-05 11:45:30 -07:00
Aditya Manthramurthy
414a7eca83 Add IAM groups support (#7981)
This change adds admin APIs and IAM subsystem APIs to:

- add or remove members to a group (group addition and deletion is
  implicit on add and remove)

- enable/disable a group

- list and fetch group info
2019-08-02 14:25:00 -07:00
Aditya Manthramurthy
4101d4917c Fix IAM users migration regression (#8008) 2019-08-01 12:31:04 -07:00
Aditya Manthramurthy
c71895f225 Listen for PolicyDB events from etcd and fix etcd watch handling (#7992) 2019-07-30 18:50:49 -07:00
Harshavardhana
94c88890b8 Add additional logging for OPA connections (#7982) 2019-07-28 08:33:25 +05:30
Harshavardhana
e871e27562 Refactor and simplify etcd helpers used in IAM subsystem (#7980) 2019-07-26 13:42:54 -07:00
Aditya Manthramurthy
7bdaf9bc50 Update on-disk storage format for users system (#7949)
- Policy mapping is now at `config/iam/policydb/users/myuser1.json`
  and includes version.

- User identity file is now versioned.

- Migrate old data to the new format.
2019-07-24 17:34:23 -07:00