Random file access methods, supporting application-level page cache.

Please see the README on GitHub at https://github.com/portnov/random-access-file#readme

random-access-file README

This package is aimed to provide some number of different implementations of random file access methods with the same interface.

It can be of use for implementing multithread read-write random access for large files, for example for DB engines.

The following implementations are provided:

• Simple: trivial wrapper around standard System.IO calls. Created mostly for demonstrative purposes. Specific thing about System.IO calls is that they work with one Handle per file, and that Handle contains a pointer to current position in the file. So it is not possible for several threads to read or write to different positions in the file. This implementation is made thread-safe by adding a global per-file lock: several threads can read and/or write to different positions of the file, but all access will be serialized.
• Threaded: file access using Posix pread(3), pwrite(3) calls. This implementation is thread-safe. It is using block-level locks; each block can be accessed for write by single thread, or for read by many threads. Size of blocks being locked is adjustable.
• MMaped: file access using mmap(2) call. This implementation is thread-safe. It is using block-level locks also.
• Cached: File access using application-level page cache, which can be used over Threaded or MMaped file access. Size of cache pages and capacity of the cache are adjustable. Performance of this implementation depends very seriously on your usage pattern and page size.

Benchmark results

Benchmark random-access-file-benchmark: RUNNING...
benchmarking main/simple
time                 691.1 ms   (-610.8 ms .. 1.689 s)
                     0.639 R²   (0.044 R² .. 1.000 R²)
mean                 1.781 s    (1.088 s .. 2.106 s)
std dev              543.2 ms   (192.5 ms .. 732.0 ms)
variance introduced by outliers: 73% (severely inflated)

benchmarking main/threaded
time                 228.5 ms   (196.1 ms .. 248.2 ms)
                     0.993 R²   (0.978 R² .. 1.000 R²)
mean                 250.5 ms   (239.5 ms .. 257.1 ms)
std dev              11.31 ms   (3.916 ms .. 16.04 ms)
variance introduced by outliers: 16% (moderately inflated)

benchmarking main/mmaped
time                 166.0 ms   (147.6 ms .. 182.9 ms)
                     0.989 R²   (0.963 R² .. 1.000 R²)
mean                 162.5 ms   (157.6 ms .. 168.0 ms)
std dev              7.960 ms   (4.764 ms .. 11.38 ms)
variance introduced by outliers: 12% (moderately inflated)

benchmarking main/cached/threaded
time                 1.272 s    (575.8 ms .. 1.865 s)
                     0.951 R²   (0.925 R² .. 1.000 R²)
mean                 1.517 s    (1.365 s .. 1.626 s)
std dev              150.9 ms   (64.02 ms .. 207.5 ms)
variance introduced by outliers: 23% (moderately inflated)

benchmarking main/cached/mmaped
time                 199.2 ms   (78.98 ms .. 332.2 ms)
                     0.863 R²   (0.708 R² .. 1.000 R²)
mean                 237.7 ms   (190.4 ms .. 285.1 ms)
std dev              62.14 ms   (31.83 ms .. 85.03 ms)
variance introduced by outliers: 58% (severely inflated)

Benchmark random-access-file-benchmark: FINISH