• Nils Goroll's avatar
    Avoid deadlock in VRT_AddDirector() when VCL is going cold · f3575e38
    Nils Goroll authored
    If VRT_AddDirector() was called from handling a VCL_COLD event or,
    indirectly, from another thread which the VCL_COLD event handler was
    waiting for, varnishd would deadlock and prevent any CLI or director
    changes, because VRT_AddDirector() requires the vcl_mtx, which is held
    during vcl_BackendEvent() to ensure a consistent view of the director
    list. Because of the early return from VRT_AddDirector() this likely
    only happened in VTC mode, but the underlying race existed
    nevertheless.
    
    This patch _almost_ fixes the issue with the intend of making it
    highly unlikely to occur without getting too involved with the vcl
    temperature controls: We now check the same conditions under which
    vcl_set_state() would transition the temperature to COOLING and, if
    they apply, use Lck_Trylock() in a try/wait loop instead of
    Lck_Lock(), avoiding the deadlock.
    
    The patch presumably still does not fix the problem entirely, because
    the reads of vcl->busy and vcl->temp before the Lck_Trylock() could
    still be outdated. With the temperature controls otherwise unchanged,
    the only alternative idea I could come up with was to always use a
    try/wait loop, which I dismissed due to the performance impact
    (overhead and added latency).
    
    Ref https://github.com/nigoroll/libvmod-dynamic/issues/110
    f3575e38
cache_vrt_vcl.c 15.8 KB