mirror of
https://github.com/freebsd/freebsd-src
synced 2024-10-15 04:43:53 +00:00
Large sync writes perform worse with slog
For synchronous write workloads with large IO sizes, a pool configured
with a slog performs worse than one with an embedded zil:
sequential_writes 1m sync ios, 16 threads
Write IOPS: 1292 438 -66.10%
Write Bandwidth: 1323570 448910 -66.08%
Write Latency: 12128400 36330970 3.0x
sequential_writes 1m sync ios, 32 threads
Write IOPS: 1293 430 -66.74%
Write Bandwidth: 1324184 441188 -66.68%
Write Latency: 24486278 74028536 3.0x
The reason is the `zil_slog_bulk` variable. In `zil_lwb_write_open`,
if a zil block is greater than 768K, the priority of the write is
downgraded from sync to async. Increasing the value allows greater
throughput. To select a value for this PR, I ran an fio workload with
the following values for `zil_slog_bulk`:
zil_slog_bulk KiB/s
1048576 422132
2097152 478935
4194304 533645
8388608 623031
12582912 827158
16777216 1038359
25165824 1142210
33554432 1211472
50331648 1292847
67108864 1308506
100663296 1306821
134217728 1304998
At 64M, the results with a slog are now improved to parity with an
embedded zil:
sequential_writes 1m sync ios, 16 threads
Write IOPS: 438 1288 2.9x
Write Bandwidth: 448910 1319062 2.9x
Write Latency: 36330970 12163408 -66.52%
sequential_writes 1m sync ios, 32 threads
Write IOPS: 430 1290 3.0x
Write Bandwidth: 441188 1321693 3.0x
Write Latency: 74028536 24519698 -66.88%
None of the other tests in the performance suite (run with a zil or
slog) had a significant change, including the random_write_zil tests,
which use multiple datasets.
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: John Wren Kennedy <john.kennedy@delphix.com>
Closes #14378
This commit is contained in:
parent
380c25f640
commit
c0e58995e3
|
@ -2172,7 +2172,7 @@ if a volatile out-of-order write cache is enabled.
|
|||
Disable intent logging replay.
|
||||
Can be disabled for recovery from corrupted ZIL.
|
||||
.
|
||||
.It Sy zil_slog_bulk Ns = Ns Sy 786432 Ns B Po 768 KiB Pc Pq u64
|
||||
.It Sy zil_slog_bulk Ns = Ns Sy 67108864 Ns B Po 64 MiB Pc Pq u64
|
||||
Limit SLOG write size per commit executed with synchronous priority.
|
||||
Any writes above that will be executed with lower (asynchronous) priority
|
||||
to limit potential SLOG device abuse by single active ZIL writer.
|
||||
|
|
|
@ -145,7 +145,7 @@ static int zil_nocacheflush = 0;
|
|||
* Any writes above that will be executed with lower (asynchronous) priority
|
||||
* to limit potential SLOG device abuse by single active ZIL writer.
|
||||
*/
|
||||
static uint64_t zil_slog_bulk = 768 * 1024;
|
||||
static uint64_t zil_slog_bulk = 64 * 1024 * 1024;
|
||||
|
||||
static kmem_cache_t *zil_lwb_cache;
|
||||
static kmem_cache_t *zil_zcw_cache;
|
||||
|
|
Loading…
Reference in a new issue