• Nils Goroll's avatar
    Add missing bits for full streaming support · 97578b06
    Nils Goroll authored
    Previous code would only deliver fully received segments even if a
    busy object was being written to by the backend side (streaming).
    
    I guess at some point before the public release I must have thought
    about this and decided that streaming only completed segments should
    be acceptable at least to begin with, but I overlooked the fact that
    the previous implementation could lead to short body writes due to a
    lack of coordination between the writing and reading side.
    
    This commit introduces proper streaming support, also within busy
    segments.
    
    In particular, this should also solve an issue reported by @tomazz75
    on gitlab where no body was sent at all. This problem could lead to
    the client stalling on a wait for body data which never came.
    
    The nature of this problem was a race condition, which I was only able
    to reproduce on my system with the following patch to favor the race:
    
    diff --git a/src/fellow_cache.c b/src/fellow_cache.c
    index 3073b35..c9d9a4e 100644
    --- a/src/fellow_cache.c
    +++ b/src/fellow_cache.c
    @@ -3338,6 +3338,9 @@ fellow_busy_obj_getspace(struct fellow_busy *fbo, size_t *sz, uint8_t **ptr)
            assert(*sz > 0);
            AN(ptr);
    
    +       // XXX DEBUG
    +       usleep(10000);
    +
            CHECK_OBJ_NOTNULL(fbo->fc, FELLOW_CACHE_MAGIC);
            CHECK_OBJ_NOTNULL(fbo->fc->tune, STVFE_TUNE_MAGIC);
            max = (size_t)1 << fbo->fc->tune->chunk_exponent;
    
    This fix survived 1000 calls to the respective test case:
    
    	/tmp/bin/varnishtest \
    		-Dlibvmod_slash=/tmp/lib/varnish/vmods/libvmod_slash.so \
    		src/vtc/fellow_c00093.vtc \
    		-n 1000 -j 20
    
    Thank you to @tomazz75 for reporting the bug and help with additional
    information to track it down.
    
    Fixes #2
    97578b06
Name
Last commit
Last update
..
foreign Loading commit data...
tbl Loading commit data...
vtc Loading commit data...
Makefile.am Loading commit data...
VSC_buddy.vsc Loading commit data...
VSC_fellow.vsc Loading commit data...
bitf.h Loading commit data...
bitsof.h Loading commit data...
buddy.c Loading commit data...
buddy.h Loading commit data...
buddy_storage.c Loading commit data...
buddy_storage.h Loading commit data...
buddy_tune.c Loading commit data...
buddy_tune.h Loading commit data...
buddy_util.h Loading commit data...
buddy_witness.h Loading commit data...
compiler.h Loading commit data...
debug.h Loading commit data...
fcs_states.doth Loading commit data...
fellow_cache.c Loading commit data...
fellow_cache.h Loading commit data...
fellow_cache_storage.h Loading commit data...
fellow_debug.h Loading commit data...
fellow_diag.h Loading commit data...
fellow_errhandling.h Loading commit data...
fellow_hash.c Loading commit data...
fellow_hash.h Loading commit data...
fellow_hashes.h Loading commit data...
fellow_inject.h Loading commit data...
fellow_io.h Loading commit data...
fellow_io_aio.c Loading commit data...
fellow_io_backend.h Loading commit data...
fellow_io_ioctl.c Loading commit data...
fellow_io_ioctl.h Loading commit data...
fellow_io_threads.c Loading commit data...
fellow_io_uring.c Loading commit data...
fellow_log.c Loading commit data...
fellow_log.h Loading commit data...
fellow_log_dbg.c Loading commit data...
fellow_log_dle_chg.h Loading commit data...
fellow_log_iter_out.h Loading commit data...
fellow_log_storage.h Loading commit data...
fellow_log_test.c Loading commit data...
fellow_pri.h Loading commit data...
fellow_regionlist.h Loading commit data...
fellow_sha256.c Loading commit data...
fellow_sha256.h Loading commit data...
fellow_stash.h Loading commit data...
fellow_storage.c Loading commit data...
fellow_storage.h Loading commit data...
fellow_storage_deref.h Loading commit data...
fellow_task.h Loading commit data...
fellow_testenv.c Loading commit data...
fellow_testenv.h Loading commit data...
fellow_tune.c Loading commit data...
fellow_tune.h Loading commit data...
flint.h Loading commit data...
flint.lnt Loading commit data...
pow2_units.c Loading commit data...
pow2_units.h Loading commit data...
slash-counters.rst Loading commit data...
slashmap.c Loading commit data...
slashmap.h Loading commit data...
slashmap.rst.in Loading commit data...
slashmap_options.h Loading commit data...
vmod_slash.c Loading commit data...
vmod_slash.man.rst Loading commit data...
vmod_slash.vcc Loading commit data...
vmod_slash_loadmasters.c Loading commit data...