linux/fs/fuse
Kirill Smelkov ad2ba64dd4 fuse: allow filesystems to have precise control over data cache
On networked filesystems file data can be changed externally.  FUSE
provides notification messages for filesystem to inform kernel that
metadata or data region of a file needs to be invalidated in local page
cache. That provides the basis for filesystem implementations to invalidate
kernel cache explicitly based on observed filesystem-specific events.

FUSE has also "automatic" invalidation mode(*) when the kernel
automatically invalidates data cache of a file if it sees mtime change.  It
also automatically invalidates whole data cache of a file if it sees file
size being changed.

The automatic mode has corresponding capability - FUSE_AUTO_INVAL_DATA.
However, due to probably historical reason, that capability controls only
whether mtime change should be resulting in automatic invalidation or
not. A change in file size always results in invalidating whole data cache
of a file irregardless of whether FUSE_AUTO_INVAL_DATA was negotiated(+).

The filesystem I write[1] represents data arrays stored in networked
database as local files suitable for mmap. It is read-only filesystem -
changes to data are committed externally via database interfaces and the
filesystem only glues data into contiguous file streams suitable for mmap
and traditional array processing. The files are big - starting from
hundreds gigabytes and more. The files change regularly, and frequently by
data being appended to their end. The size of files thus changes
frequently.

If a file was accessed locally and some part of its data got into page
cache, we want that data to stay cached unless there is memory pressure, or
unless corresponding part of the file was actually changed. However current
FUSE behaviour - when it sees file size change - is to invalidate the whole
file. The data cache of the file is thus completely lost even on small size
change, and despite that the filesystem server is careful to accurately
translate database changes into FUSE invalidation messages to kernel.

Let's fix it: if a filesystem, through new FUSE_EXPLICIT_INVAL_DATA
capability, indicates to kernel that it is fully responsible for data cache
invalidation, then the kernel won't invalidate files data cache on size
change and only truncate that cache to new size in case the size decreased.

(*) see 72d0d248ca "fuse: add FUSE_AUTO_INVAL_DATA init flag",
eed2179efe "fuse: invalidate inode mapping if mtime changes"

(+) in writeback mode the kernel does not invalidate data cache on file
size change, but neither it allows the filesystem to set the size due to
external event (see 8373200b12 "fuse: Trust kernel i_size only")

[1] https://lab.nexedi.com/kirr/wendelin.core/blob/a50f1d9f/wcfs/wcfs.go#L20

Signed-off-by: Kirill Smelkov <kirr@nexedi.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2019-04-24 17:05:06 +02:00
..
acl.c fuse: Support fuse filesystems outside of init_user_ns 2018-03-20 17:11:44 +01:00
control.c fuse: clean up aborted 2019-02-13 13:15:14 +01:00
cuse.c fuse: convert printk -> pr_* 2019-04-24 17:05:06 +02:00
dev.c fuse: convert printk -> pr_* 2019-04-24 17:05:06 +02:00
dir.c fuse: Protect fi->nlookup with fi->lock 2019-02-13 13:15:14 +01:00
file.c fuse: honor RLIMIT_FSIZE in fuse_file_fallocate 2019-04-24 17:05:06 +02:00
fuse_i.h fuse: allow filesystems to have precise control over data cache 2019-04-24 17:05:06 +02:00
inode.c fuse: allow filesystems to have precise control over data cache 2019-04-24 17:05:06 +02:00
Kconfig fuse: Add posix ACL support 2016-10-01 07:32:32 +02:00
Makefile fuse: split out readdir.c 2018-09-28 16:43:23 +02:00
readdir.c fuse: Protect fi->nlookup with fi->lock 2019-02-13 13:15:14 +01:00
xattr.c fuse: Ensure posix acls are translated outside of init_user_ns 2018-05-31 12:26:10 +02:00