I have tried hard to make value tracking understand the code, but to
no avail. It seems, for example

	assert(n <= 56)

and later
	assert(n > 0)

will just lead to flexelint knowning 1? but not 56 as the limit.

it was never accessed, but triggered flexelint

First and foremost, fellow_log_prep_max_regions was defined wrong:

Except in fellow_cache_test, we call log submission with a maximum of
FELLOW_DISK_LOG_BLOCK_ENTRIES = 56 DLEs. The intention of the
fellow_log_prep_max_regions was was to allocate space to track return
of the maximum number of regions possibly contained. The exact maximum
would be (FELLOW_DISK_LOG_BLOCK_ENTRIES - 1) * DLE_REG_NREGION + 1 =
(55 * 4) + 1 = 221, which is higher than FELLOW_DISK_LOG_BLOCK_ENTRIES
* DLE_BAN_REG_NREGION = 56 * 3 = 168.

Yet it seems prudent to not reply on any fixed maximum, and also our
test cases call for a higher value, so we now define the maximum three
times the actually used value, and also ensure that we batch the code
to this size.

In addition, one assertion in fellow_log_entries_prep() was wrong (it
compared a number of DLEs with a number of regions).

We also tighten some assertions to help future analysis of possible
issues in this area:

- Ensure that the data path via fellow_log_entries_prep() only ever
  uses a region list on the stack.

- By using the regionlist_onlystk_add() macro, ensure that we hit an
  assertion on the array on stack, rather than one on the regionlist
  pointer.

Diff best viewed with -b

Fixes #18

Related to #18

We should do this right and not over-allocate, this is just confusing.

Motivated by #18, but does not fix the root cause yet

For the call path in the bug ticket, the stack regionlist is supposed
to be big enough and the root cause is that it is not. But at any
rate, for that call path, the regionlist is OK to be NULL and
regionlist_add() should never be called.

If, however, it _is_ called, the regionlist can't be NULL.

Avoids:

fellow_io_uring.c:234:1: error: ‘try_flag’ defined but not used [-Werror=unused-function]
  234 | try_flag(unsigned flag)
      | ^~~~~~~~

the lru_mtx is our most contended mtx.

As a first improvement, batch changes to LRU for multiple segments
and maintain the effective change locally outside the lru mtx (but
while holding the obj mtx).

is there a better way?

https://github.com/axboe/liburing/issues/906

during error paths, we might call it multiple times

varnish-cache does not touch objects for OA_VARY, but we need
to keep FCOs in memory which are frequently used during lookup.

Thoughts on why this should not race LRU:

- lru_list is owned by lru_mtx
- object can't go away, because
  - for call from hash, we hold the oh->mtx
  - otherwise, we hold a ref

... which happens potentially under the cache lock

upfront: This is not the segment allocation, which uses parts of the busy
obj region allocation, and is mostly motivated by how much data we need
to have in RAM at minimum.

For the region allocation, we have conflicting goals:

- To keep the log short, we want to use the least number of regions
- To reduce fragmentation, we want to use the largest possible
  allocations
- To use space efficiently, we want to split regions into power of
  two allocations.

Also, for chunked encoding, we do not have an upper limit of
how much space we are going to need, so we have to use the
estimate provided by fellow_busy_obj_getspace(). It can not
guess more than objsize_max.

The new region alloc algorithm takes this compromise:

- For the base case that we ran out of available regions (220), we
  allocate all we need without cramming.
- Otherwise if we need less than a chunk, we request it
- Otherwise if we know the size, we round down to a power of two
- Otherwise we round up

We then allow any cramming down to the chunk size, because that
is what our LRU reservation uses.

Ref #10

Ref #10

adjust dsk size if mem allocation was smaller than requested

Could have caused #5, related to #10

This is counter-intuitive and could lead to extreme values, for
example:

	default: chunk_exponent = 20, dsk_reserve_chunks 4

	adjusted to: 12, 4 << 8 = 1024

now user sets chunk_exponent = 21

	adjusted to: 12, 1024 << 9 = 524288

Could have caused #5, related to #10

tiny glitch

The main cause for #11 seems to be that the chunk size in relation to
the memory cache was too big.

We now clamp it at memsz >> 10 (less than 1/1024 of the memsz).

This can still lead to issues when the memory size is reduced and
the cache reloaded, but then at least new objects will not compete
for the available memory.

Fixes #11 I hope

With the previous commit, we restore the objcore's oa_present field
when we first read the object.

On top of that, it would be nice if we could restore the field or at
least the bit for OA_VARY when we read the log, because a cache lookup
checks it and could avoid reading objects if the OA_VARY bit is not
set.

On the other hand, if vary is used, the object needs to be read
anyway, so restoring oa_present during log read only payed off for no
vary.

The straight forward solution would be to add 2 bytes to struct
fellow_dle_obj. This would

- require a DLE version bump and compatibility code
- bump the dle size from 72 to 80 bytes (because alignment)
- consequently, reduce FELLOW_DISK_LOG_BLOCK_ENTRIES from 56 to 4032 /
  80 = 50

All this is possible and part of the fellow design, but at
this point I conclude that it is not worth the effort.

Another option would be to cram two bits into the existing
fellow_dle_obj, for example by exploiting the fact that the sign bit
of ban and t_origin is always zero. This could break backwards
compatibility, so we prepare for the option, but do not implement it.

All in all, we just add a TODO to add this when we need to extend the
DLE anyway.

Closes #17