Explain better how LRU pre-eviction and cram work

and how they relate to cache usages.

Closes #5
parent 4b04a599
......@@ -506,7 +506,7 @@ STRING xfellow.tune([INT logbuffer_size], [DURATION logbuffer_flush_interval], [
[ENUM {panic, fail} allocerr_log]
)
Using the `xfellow.tune()` method, the following parameters of the
Using the `xfellow.tune()`_ method, the following parameters of the
fellow storage can be fine tuned:
* *logbuffer_size*
......@@ -801,7 +801,7 @@ STRING tune_fellow(STEVEDORE storage, [INT logbuffer_size], [DURATION logbuffer_
)
Tune the given globally defined fellow storage, for all other
parameters see `xfellow.tune()`.
parameters see `xfellow.tune()`_.
STATISTICS / COUNTERS
=====================
......@@ -815,6 +815,60 @@ and the :ref:`slash-counters(7)` man page.
The ``g_dsk_*`` and ``g_mem_*`` gauges are updated at regular
intervals of *logbuffer_flush_interval*.
Interpreting Gauges and Background on Cache Behavior
----------------------------------------------------
The gauges ``g_mem_space`` and ``g_mem_space`` give the number of free
bytes in memory and on disk, the ``*_bytes`` statistics give the
number of used bytes.
On a typical system which uses all of the available cache and evicts
objects mostly through LRU, these gauges should more or less stabilize
over time, which should become obvious when logging and graphing the
above values over longer time spans. But depending on how the cache is
used and tuned, that point might well be in the region of 70% and
below.
The fact that fellow does not, by default, attempt to use each and
every byte of the available cache is a deliberate decision:
To achieve optimal disk and network I/O throughput, object data should
be stored in contiguous regions. However, such a region might not
always be available, and `fellow` needs to make a decision if
returning a smaller region or waiting for LRU to make room is the
better option. Also, it might be better to return a smaller region
than to split a larger region, which could instead be used for a
larger object coming in later.
The *cram* parameter (see `xbuddy.tune()`_) controls this trade off:
If *cram* allows a smaller segment, it is returned, otherwise the
allocator needs to wait for LRU to make room.
While higher absolute *cram* values improve space usage, they lead to
higher fragmentation and might negatively impact performance. Positive
*cram* values avoid using larger free regions for smaller
requests. Negative *cram* values do not.
Another factor is that the LRU algorithm pre-evicts segments and
objects from cache until ``mem_reserve_chunks`` have been reserved
The important aspect here is that the reserved chunks are contiguous
in order to counteract fragmentation: LRU runs until there happens to
be enough contiguous space for each of the reserved chunks.
The smaller objects are compared to the chunk size, the more objects
need to be evicted for a contiguous chunk to become available.
This behavior can be controlled by adjusting ``chunk_exponent`` /
``chunk_bytes``. We recommend to set the chunk size larger than the
expected object size such that typical new objects will fit into
reserved chunks. However, if the goal is to maximize ram cache usage,
the chunk size can be reduced at the expense of somehow higher I/O
overhead and fragmentation.
The higher ``reserve_chunks`` is set, the more agressively LRU will
pre-evict objects in order to have space available for new requests.
FELLOW DIAGNOSTICS
==================
......
......@@ -444,7 +444,7 @@ $Method STRING .tune(
[ ENUM { panic, purge } allocerr_obj ],
[ ENUM { panic, fail } allocerr_log ])
Using the `xfellow.tune()` method, the following parameters of the
Using the `xfellow.tune()`_ method, the following parameters of the
fellow storage can be fine tuned:
* *logbuffer_size*
......@@ -722,7 +722,7 @@ $Function STRING tune_fellow(
[ ENUM { panic, fail } allocerr_log ])
Tune the given globally defined fellow storage, for all other
parameters see `xfellow.tune()`.
parameters see `xfellow.tune()`_.
STATISTICS / COUNTERS
=====================
......@@ -736,6 +736,60 @@ and the :ref:`slash-counters(7)` man page.
The ``g_dsk_*`` and ``g_mem_*`` gauges are updated at regular
intervals of *logbuffer_flush_interval*.
Interpreting Gauges and Background on Cache Behavior
----------------------------------------------------
The gauges ``g_mem_space`` and ``g_mem_space`` give the number of free
bytes in memory and on disk, the ``*_bytes`` statistics give the
number of used bytes.
On a typical system which uses all of the available cache and evicts
objects mostly through LRU, these gauges should more or less stabilize
over time, which should become obvious when logging and graphing the
above values over longer time spans. But depending on how the cache is
used and tuned, that point might well be in the region of 70% and
below.
The fact that fellow does not, by default, attempt to use each and
every byte of the available cache is a deliberate decision:
To achieve optimal disk and network I/O throughput, object data should
be stored in contiguous regions. However, such a region might not
always be available, and `fellow` needs to make a decision if
returning a smaller region or waiting for LRU to make room is the
better option. Also, it might be better to return a smaller region
than to split a larger region, which could instead be used for a
larger object coming in later.
The *cram* parameter (see `xbuddy.tune()`_) controls this trade off:
If *cram* allows a smaller segment, it is returned, otherwise the
allocator needs to wait for LRU to make room.
While higher absolute *cram* values improve space usage, they lead to
higher fragmentation and might negatively impact performance. Positive
*cram* values avoid using larger free regions for smaller
requests. Negative *cram* values do not.
Another factor is that the LRU algorithm pre-evicts segments and
objects from cache until ``mem_reserve_chunks`` have been reserved
The important aspect here is that the reserved chunks are contiguous
in order to counteract fragmentation: LRU runs until there happens to
be enough contiguous space for each of the reserved chunks.
The smaller objects are compared to the chunk size, the more objects
need to be evicted for a contiguous chunk to become available.
This behavior can be controlled by adjusting ``chunk_exponent`` /
``chunk_bytes``. We recommend to set the chunk size larger than the
expected object size such that typical new objects will fit into
reserved chunks. However, if the goal is to maximize ram cache usage,
the chunk size can be reduced at the expense of somehow higher I/O
overhead and fragmentation.
The higher ``reserve_chunks`` is set, the more agressively LRU will
pre-evict objects in order to have space available for new requests.
FELLOW DIAGNOSTICS
==================
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment