• Dridi Boukelmoune's avatar
    Easier regression tracking with vtc-bisect.sh · 657e31f2
    Dridi Boukelmoune authored
    With this script we can essentially use a VTC to track down a regression
    and see what broke after getting a cup of coffee, lunch, or whatever you
    wish to do while the script is running.
    
    It will search out of the box for a regression between the last tag and
    the current git head, and assumes the last tag passed the test case. It
    is otherwise the maintainer's responsibility to provide the good and/or
    bad commits beforehand.
    
    For more information:
    
        tools/vtc-bisect.sh -h
    
    I had this idea after setting up an automated bisect one time too many
    for #3003 and decided to generalize this in a handy script.
    
    For example:
    
        tools/vtc-bisect.sh -b d6d34160~ bin/varnishtest/tests/r03003.vtc
    
    The script works on a best-effort basis and tries to minimize rebuild
    time, or broken builds. It has minimal error handling and may leave you
    in a dirty git tree stuck in bisect mode as a result.
    
    I also noticed that 6.2.0 was not tagged in the master branch, so the
    example above starts with the varnish-6.1.0 tag. Tagging major releases
    (aka x.y.0 or dot-zero releases) from master is part of the release
    procedure and this facility should give one more reason to enforce the
    rule. Last time it happened before 6.2.0 was 5.2.0 so there's clearly
    something wrong with x.2.0 releases...
    
    This will of course not solve all bisect problems if for example a VTC
    relies on a feature that wasn't available in varnishtest when the
    regression was introduced and we still may need to perform manual git
    bisect operations but hopefully this should save us time from now on.
    657e31f2
vtc-bisect.sh 3.42 KB