slash:master commitshttps://code.uplex.de/uplex-varnish/slash/commits/master2024-03-25T17:13:40Zhttps://code.uplex.de/uplex-varnish/slash/commit/36e038c4fe05c79ca0fb596f4a86fcf05eb39d52Adjust to varnish-cache 7.5 / master 9aa45a9f290b3e43c3939622c55017075862965f2024-03-25T17:13:40ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/e25cafcdb87f9d3a89bedfd3d75997a063c97a11For global fellow storage, use the jail facility to fix ownership/mode2024-03-01T11:47:37ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/930492ad7861e68260cc3ed8d86e01a62607f250Improve open() flags and permissions2024-03-01T11:47:37ZNils Gorollnils.goroll@uplex.de
Add O_LARGEFILE just in case
Fix mode to 0600https://code.uplex.de/uplex-varnish/slash/commit/48c60a21d7161fd9e44f44f99634a74164c34cfbUpdate CHANGES.rst2024-03-01T10:38:17ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/ca4953e20be91f881a1bb4c8088a2c52d5531d82fellow cache: Use objsize_hint to estimate chunked size2024-02-19T12:58:19ZNils Gorollnils.goroll@uplex.de
Fixes the second part of #60https://code.uplex.de/uplex-varnish/slash/commit/73e1d6b702e8af29bb3129159337e2ef66c3abf6fellow storage: Improve comment regarding VC#40562024-02-19T12:48:00ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/6da939ea00fbdc9892b4ce74cb48cdfef5b80b9fbuddy storage: Adjust to changed STV API2024-02-19T12:47:21ZNils Gorollnils.goroll@uplex.de
Ref: <a href="https://github.com/varnishcache/varnish-cache/pull/4056" rel="nofollow noreferrer noopener" target="_blank">https://github.com/varnishcache/varnish-cache/pull/4056</a>https://code.uplex.de/uplex-varnish/slash/commit/c57d7879a4ec3509edadd967411757abede9fbf3Bring back cramming to fellow_busy_seg_memalloc()2024-02-14T18:08:36ZNils Gorollnils.goroll@uplex.de
Ref #60https://code.uplex.de/uplex-varnish/slash/commit/bef98385c0c4880de7c635beee70d1a43c0b4c7dAdd a cramlimit function on page sizes2024-02-14T18:00:27ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/3b9df28ef1e3f7ebade49fcf4631402f0d18fd2eRename for clarity2024-02-14T18:00:27ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/bee79167c7c1b9486799b550744ca0b4dccc9520Optimize segment memory allocation further2024-02-14T18:00:27ZNils Gorollnils.goroll@uplex.de
If the page from the segmem pool is too big, do not just trim it, but
rather trade it for a smaller page if that is sufficient.
Ref #60https://code.uplex.de/uplex-varnish/slash/commit/0dd59f2304f280c599175e3e0cf6d6f162cdbfcfNeuter fetch_chunksize from Varnish-Cache and allocate chunksize2024-02-14T17:22:51ZNils Gorollnils.goroll@uplex.de
... if it looks like we were handling chunked encoding.
fellow has (and needs to have) its own strategy for allocating growing
objects and is basically working around the fetch_chunksize coming
from varnish-cache by recording if subsequent allocation requests are
growing the object (for chunked encoding) or are converging onto a
maximum (for content-length).
Now this strategy had an undesired side effect, that the newly
introduced fbo_segmem pool is always allocating the chunk size, but
the disk segment allocation was using the size from varnish-cache,
which, for the example in the ticket, would lead to 1MB chunks being
allocated, but trimmed down to only 16kb - for each allocation.
We now explicitly test if varnish-cache is requesting fetch_chunksize
and, if so, allocate the chunk size.
This brings the disk segment allocation in line with the mempool.
On the other hand, for chunked encoding, we will still over-allocate
and trim when the actual object is smaller than the chunk size, but
this is by design.
Fixes #60https://code.uplex.de/uplex-varnish/slash/commit/f58dca47ff1fcd68e2991380d0ca390df064ca23Rename variable for clarity2024-02-14T15:43:21ZNils Gorollnils.goroll@uplex.de
while working on #60https://code.uplex.de/uplex-varnish/slash/commit/2fc3cf676a044fc4bb55ce54787ae73472e09fbeBack to trunk2024-02-09T09:29:02ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/9943690d4dcdcb5d9466aafbba98c91e1fc027e0Tag version 1.0.0-rc22024-02-09T09:28:09ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/c46709454786434b3d04c9271978c4c98cc0fd4adoc: Add an updated performance report2024-02-08T17:51:43ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/63b003ec461f756ef34bc310ae286030317b731dfellow_log: Bring back obj deref during FP_INIT2024-02-07T22:37:45ZNils Gorollnils.goroll@uplex.de
<a href="/uplex-varnish/slash/commit/8134e93be4c34cd57b7f0f44dff87e7d79210c83" data-original="8134e93be4c34cd57b7f0f44dff87e7d79210c83" data-link="false" data-link-reference="false" data-project="84" data-commit="8134e93be4c34cd57b7f0f44dff87e7d79210c83" data-reference-type="commit" data-container="body" data-placement="bottom" title="Refactor fellow_log API: Replace fellow_region_* with fellow_dskbuddy()" class="gfm gfm-commit has-tooltip">8134e93b</a> broke object deletions during
FP_INIT, because fellow_dskbuddy() waits for FP_OPEN:
#3 0x00007f3f51a85d2b in fellow_wait_open (ffd=0x7f3f445a8000) at fellow_log.c:847
#4 fellow_dskbuddy (ffd=0x7f3f445a8000) at fellow_log.c:6381
#5 0x00007f3f51aa31f7 in fellow_cache_obj_delete (fc=0x7f3f446d4000, fco=<optimized out>,
hash=hash@entry=0x7f3f404ce670 "ǹ\216N\032\217\230},p\245\205\361i \002\253\253Rn\372ز\303\307\355,\254\342\024\360M")
at fellow_cache.c:6032
#6 0x00007f3f51a597a9 in sfedsk_objfree (wrk=0x7f3f40dfc5d0, dskoc=0x7f3f404d5440) at fellow_storage.c:655
#7 0x0000564d8e23c14a in ObjFreeObj (wrk=wrk@entry=0x7f3f40dfc5d0, oc=0x7f3f404d5440) at cache/cache_obj.c:412
#8 0x0000564d8e232a9f in HSH_DerefObjCore (wrk=0x7f3f40dfc5d0, ocp=ocp@entry=0x7fffcd3303d0, rushmax=rushmax@entry=-1)
at cache/cache_hash.c:1065
#9 0x00007f3f51a5022f in festash_work_fes (fet=fet@entry=0x7fffcd33bbf0, fes=0x7f3e7a006640, ban=ban@entry=0x7ee790c56160)
at fellow_stash.h:195
#10 0x00007f3f51a54be2 in festash_top_work (fet=fet@entry=0x7fffcd33bbf0, has_bans=1) at fellow_stash.h:226
#11 0x00007f3f51a586b8 in sfe_resurrect_ban (e=0x7f3e401d7c98, sfer=0x7fffcd33bbb0) at fellow_storage.c:2078
#12 sfe_resurrect (priv=0x7fffcd33bbb0, e=0x7f3e401d7c98) at fellow_storage.c:2111
#13 0x00007f3f51a81163 in fellow_logs_iter_block (flics=flics@entry=0x7fffcd332b80, flivs=flivs@entry=0x7fffcd337050,
logblk=logblk@entry=0x7f3e401d7000) at fellow_log.c:4834
#14 0x00007f3f51a82864 in fellow_logs_iter (flics=0x7fffcd332b80, flivs=flivs@entry=0x7fffcd337050, active_logregion=0x7f3f445a8360,
empty_logregion=0x7f3f445a8370, off=594695172096, off@entry=656178581504) at fellow_log.c:5294
#15 0x00007f3f51a84886 in fellow_logs_rewrite (ffd=ffd@entry=0x7f3f445a8000, new_log_fdr=new_log_fdr@entry=0x0,
resur_f=resur_f@entry=0x7f3f51a57da0 <sfe_resurrect>, resur_priv=resur_priv@entry=0x7fffcd33bbb0) at fellow_log.c:5789
#16 0x00007f3f51a8763b in fellow_log_open (ffd=0x7f3f445a8000, resur_f=resur_f@entry=0x7f3f51a57da0 <sfe_resurrect>,
resur_priv=resur_priv@entry=0x7fffcd33bbb0) at fellow_log.c:6809
#17 0x00007f3f51a5516a in sfe_open_task (priv=0x7fffcd33bbb0, wrk=<optimized out>) at fellow_storage.c:2199
But rather than bringing this back, we postpone deletion work with a
thin delete.https://code.uplex.de/uplex-varnish/slash/commit/67e9f228cdf9a20f04d1418e73f9a207890e1541build: fence stack usage on ubuntu2024-02-07T21:03:22ZNils Gorollnils.goroll@uplex.de
for interesting detail, read the issue
Fixes #57https://code.uplex.de/uplex-varnish/slash/commit/cf999c4a1b266409a055d1b3f5e00e168d81f84ffellow_log: fix bad assertions in logbuffer_ref()2024-02-07T19:38:54ZNils Gorollnils.goroll@uplex.de
flush_active() can be called multiple times from logbuffer_flush(), and
so can logbuffer_ref()https://code.uplex.de/uplex-varnish/slash/commit/ecbe6ef026478b6916181eca77ca78238253257fbuild: fix unused function error2024-02-07T19:26:03ZNils Gorollnils.goroll@uplex.de
so far, this has only been seen with clang 10.0.0-4ubuntu1
Using different compilers never disappoints...https://code.uplex.de/uplex-varnish/slash/commit/b389fcc1495e5044aa44211d4e6862cb2df1662ddoc: Update generated rst2024-02-07T19:22:39ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/e4514f4e438b8bf010b8e2da4bdf4654221ff1b5doc: remove additional notes on error handling2024-02-07T16:50:30ZNils Gorollnils.goroll@uplex.de
ioerr_log and allocerr_log have appropriate NOTEs attached.https://code.uplex.de/uplex-varnish/slash/commit/cc261e3b8f5377ce34bb19444e45390a7c5b0b38doc: update planned features2024-02-07T16:49:04ZNils Gorollnils.goroll@uplex.de
I think statistics are good at this point and timeouts do not sound like
a good idea to me any more: What are we going to do when a timeout is
hit?https://code.uplex.de/uplex-varnish/slash/commit/d716e946452ad62d21539f90c6e2be80d9def0ebfellow_log: Fix a pthread_cond_wait (minor)2024-02-07T16:18:37ZNils Gorollnils.goroll@uplex.de
Spotted by Coverity, CID#486184https://code.uplex.de/uplex-varnish/slash/commit/1dfca806847d4f1f3e2fdfa2b2f26e84a005fbf7build: trunk version2024-02-07T16:04:57ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/fe09a042e55065c573dbfa875d674608d6856ec8doc: typos suck2024-02-07T16:04:57ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/66da22f69f0ac966347a63c5bb2da7c141ef5132doc: Mention version tag in CHANGES.rst2024-02-06T20:24:43ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/ef37068bcd3c31dbc87ee5f1248de015cdf52e47Tag version 1.0.0-rc12024-02-06T20:03:24ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/db9aacca5aef8e078b838d2ba554b25ffd9a9720doc: Add a details on sizing2024-02-06T19:59:50ZNils Gorollnils.goroll@uplex.de
Closes #46https://code.uplex.de/uplex-varnish/slash/commit/eb0d61cfe5dc9bd2935371fc247ccdc0c0ca28faUpdate documentation wrt build of VSC_main.h2024-02-06T18:43:06ZNils Gorollnils.goroll@uplex.de
<a href="https://github.com/varnishcache/varnish-cache/commit/dcaf616c66d93de69735237967cc091fa490bb93" rel="nofollow noreferrer noopener" target="_blank">https://github.com/varnishcache/varnish-cache/commit/dcaf616c66d93de69735237967cc091fa490bb93</a>
removed the option to specifically build a single VSC header.
Closes #56https://code.uplex.de/uplex-varnish/slash/commit/2263f93a9f7e40d5483df0b97823b40a6fb0b19eUpdate CHANGES.rst2024-02-06T18:36:08ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/958d16c12ff7123eeccc8e1bfb83ba2d07ff17b4fellow_log: automatically resize log regions2024-02-06T18:23:41ZNils Gorollnils.goroll@uplex.de
... controlled by limiting parameters, which can also be used, in the
worst case, to deactivate the feature.
Closes #54
Closes !3 (implemented somehow similarly)https://code.uplex.de/uplex-varnish/slash/commit/31c09e5045fdb315c762be4976c66f9068b86299fellow_tune: Remove objsize_hint tunable2024-02-06T18:23:41ZNils Gorollnils.goroll@uplex.de
The potential impact of changing this parameter to a wrong value is too
high, so we better rely on changes to automatically adjust it (coming
up).
Ref #54https://code.uplex.de/uplex-varnish/slash/commit/d1ef255f613837a06c0f76f7bab4ae10c020e78bfellow_log: Extend memsz check to shared memory cache2024-02-06T18:23:41ZNils Gorollnils.goroll@uplex.de
Each fellow storage reserves a minimum amount of memory.
Closes #51https://code.uplex.de/uplex-varnish/slash/commit/4144fd97b0d74651005b863b389839a023f95de9fellow_log: size log regions by rounded down log22024-02-06T18:23:41ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/5ad6768badc5c8e4694867201f6cef997fc85513fellow_log: emit shrink info via diag2024-02-06T18:23:41ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/15ea6057aa7d2b6f7ed484a43879cd21286c3dcefellow_cache: Fix seglist strategy2024-02-06T18:23:34ZNils Gorollnils.goroll@uplex.de
this wrong line counteracted <a href="/uplex-varnish/slash/commit/fdb260e273d95a19fe3f38458cd784bad190e241" data-original="fdb260e273d95a19fe3f38458cd784bad190e241" data-link="false" data-link-reference="false" data-project="84" data-commit="fdb260e273d95a19fe3f38458cd784bad190e241" data-reference-type="commit" data-container="body" data-placement="bottom" title="fellow_cache: Improve the seglist sizing strategy" class="gfm gfm-commit has-tooltip">fdb260e2</a> :(https://code.uplex.de/uplex-varnish/slash/commit/7a08cb09e234ba46f6ae59a26ea9a6ce27339bd2fellow_cache: fellow_busy_obj_trim_seglists and off==02024-02-06T18:23:11ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/333fed0dd2732f2d0feaf5c276e6b718fa8f6477fellow_cache: Support pointers not from fbo->region in _busy_region_free2024-02-06T18:23:11ZNils Gorollnils.goroll@uplex.dehttps://code.uplex.de/uplex-varnish/slash/commit/5042ec817004ebea592047425e4a0a0fb3f9d33afellow_log: Prevent starting with too little memory2024-02-05T23:35:59ZNils Gorollnils.goroll@uplex.de
Part of the solution to #54