152 commits

Author	SHA1	Message	Date
Harshavardhana	289e1d8b2a	fix: reduce crawler memory usage by orders of magnitude (#11556) ... currently crawler waits for an entire readdir call to return until it processes usage, lifecycle, replication and healing - instead we should pass the applicator all the way down to avoid building any special stack for all the contents in a single directory. This allows for - no need to remember the entire list of entries per directory before applying the required functions - no need to wait for entire readdir() call to finish before applying the required functions	2021-02-17 15:34:42 -08:00
Harshavardhana	ffea6fcf09	fix: rename crawler as scanner in config (#11549)	2021-02-17 12:04:11 -08:00

Renamed from cmd/data-crawler.go (Browse further)