- 21 Jul, 2023 4 commits
-
-
Nils Goroll authored
With the previous commit, we restore the objcore's oa_present field when we first read the object. On top of that, it would be nice if we could restore the field or at least the bit for OA_VARY when we read the log, because a cache lookup checks it and could avoid reading objects if the OA_VARY bit is not set. On the other hand, if vary is used, the object needs to be read anyway, so restoring oa_present during log read only payed off for no vary. The straight forward solution would be to add 2 bytes to struct fellow_dle_obj. This would - require a DLE version bump and compatibility code - bump the dle size from 72 to 80 bytes (because alignment) - consequently, reduce FELLOW_DISK_LOG_BLOCK_ENTRIES from 56 to 4032 / 80 = 50 All this is possible and part of the fellow design, but at this point I conclude that it is not worth the effort. Another option would be to cram two bits into the existing fellow_dle_obj, for example by exploiting the fact that the sign bit of ban and t_origin is always zero. This could break backwards compatibility, so we prepare for the option, but do not implement it. All in all, we just add a TODO to add this when we need to extend the DLE anyway. Closes #17
-
Nils Goroll authored
-
Nils Goroll authored
Implements most of #17, except that we would like to restore OA_VARY with the log read... This should be backwards and forwards compatible, because the default in both cases is 0, and varnish-cache will not use the bitfield if it is 0. Conflicts: src/fellow_cache.c because fellow_disk_obj_compat() in master but not in 7.3
-
Nils Goroll authored
-
- 15 Jul, 2023 2 commits
-
-
Nils Goroll authored
Closes #16
-
Nils Goroll authored
Ref #16
-
- 10 Jul, 2023 5 commits
-
-
Nils Goroll authored
-
Nils Goroll authored
In stvfe_dskoc_fco() we change the oc's stevedore to memstv only _after_ we have taken a refernce via fellow_cache_obj_get() and thus potentially put the fco onto the lru list. This order is important, because other requests racing on stvfe_dskoc_fco() need to also go through fellow_cache_obj_get() until the memstv object has been established for real. Yet this opens a window for a window where an fco is on LRU with oc->stobj->stevedore still set to dskstv, even without dirty reads, and even more so with dirty reads. Because any additional synchronization here could kill performance, we handle this outdated read in stvfe_mutate by not touching the object (yet). It will be hit the next time we iterate the LRU list. Fixes #13, hopefully.
-
Nils Goroll authored
-
Nils Goroll authored
Because of #14 I would like to understand if #13 is really a genuine bug or some odd consequence
-
Nils Goroll authored
This bug was also present in varnish-cache (will document) and found its way into fellow through the api-definition-by-example paradigm of varnish-cache: In varnish-cache vbf_stp_error(), an existing storage object is freed to be replaced with a new one, potentially from a different storage (usually -sTransient). In this case, ObjBocDone is not called, because the boc is to be preserved - from the client's perspective, the backend transaction is still in progress, and the client is waiting for the busy object to finish. At the storage level, however, the process of creation of a new object _is_ finished, and any busy object state which the storage might keep needs to be finalized. So, if oc->boc is not NULL for an ObjFreeObj() call, the stevedore needs to finish any busy object transaction. Fixes #14
-
- 17 Apr, 2023 3 commits
-
-
Nils Goroll authored
-
Nils Goroll authored
Ref b7ac17d9
-
Nils Goroll authored
-
- 10 Apr, 2023 2 commits
-
-
Nils Goroll authored
make distcheck works now
-
Nils Goroll authored
Ref aa35f2b3 Should fix #9
-
- 07 Apr, 2023 6 commits
-
-
Nils Goroll authored
-
Nils Goroll authored
-
Nils Goroll authored
and how they relate to cache usages. Closes #5
-
Nils Goroll authored
They also get updated during resurrection when the checkpoint logs are output. Implements #6
-
Nils Goroll authored
-
Nils Goroll authored
-
- 27 Mar, 2023 1 commit
-
-
Nils Goroll authored
closes #7
-
- 20 Mar, 2023 1 commit
-
-
Nils Goroll authored
Closes #4
-
- 02 Mar, 2023 5 commits
-
-
Nils Goroll authored
-
Nils Goroll authored
-
Nils Goroll authored
-
Nils Goroll authored
-
Nils Goroll authored
-
- 01 Mar, 2023 6 commits
-
-
Nils Goroll authored
-
Nils Goroll authored
-
Nils Goroll authored
-
Nils Goroll authored
It seems to have caused confusion regarding the relation between io_uring and the fellow memory cache, which exists only to the extent that io_uring manages IO in and out of it.
-
Nils Goroll authored
-
Nils Goroll authored
Previous code would only deliver fully received segments even if a busy object was being written to by the backend side (streaming). I guess at some point before the public release I must have thought about this and decided that streaming only completed segments should be acceptable at least to begin with, but I overlooked the fact that the previous implementation could lead to short body writes due to a lack of coordination between the writing and reading side. This commit introduces proper streaming support, also within busy segments. In particular, this should also solve an issue reported by @tomazz75 on gitlab where no body was sent at all. This problem could lead to the client stalling on a wait for body data which never came. The nature of this problem was a race condition, which I was only able to reproduce on my system with the following patch to favor the race: diff --git a/src/fellow_cache.c b/src/fellow_cache.c index 3073b35..c9d9a4e 100644 --- a/src/fellow_cache.c +++ b/src/fellow_cache.c @@ -3338,6 +3338,9 @@ fellow_busy_obj_getspace(struct fellow_busy *fbo, size_t *sz, uint8_t **ptr) assert(*sz > 0); AN(ptr); + // XXX DEBUG + usleep(10000); + CHECK_OBJ_NOTNULL(fbo->fc, FELLOW_CACHE_MAGIC); CHECK_OBJ_NOTNULL(fbo->fc->tune, STVFE_TUNE_MAGIC); max = (size_t)1 << fbo->fc->tune->chunk_exponent; This fix survived 1000 calls to the respective test case: /tmp/bin/varnishtest \ -Dlibvmod_slash=/tmp/lib/varnish/vmods/libvmod_slash.so \ src/vtc/fellow_c00093.vtc \ -n 1000 -j 20 Thank you to @tomazz75 for reporting the bug and help with additional information to track it down. Fixes #2
-
- 28 Feb, 2023 4 commits
-
-
Nils Goroll authored
-
Nils Goroll authored
-
Nils Goroll authored
The next segment might not be available yet when we try to read ahead.
-
Nils Goroll authored
Should fix this assertion failure during shutdown for the most part (draining a storage still is not guaranteed to succeed): Assert error in LRU_Free(), storage/storage_lru.c line 77: Condition((((&lru->lru_head)->vtqh_first == ((void*)0))) != 0) not true.
-
- 27 Feb, 2023 1 commit
-
-
Nils Goroll authored
Avoid users accidentally running into a file being created where block devices would normally live. Motivated by #2
-