• Jerome Martinez's avatar
    avcodec/dpx: fix check of minimal data size for unpadded content · 05e34523
    Jerome Martinez authored
    stride value is not relevant with unpadded content and the total count
    of pixels (width x height) must be used instead of the rounding based on
    width only then multiplied by height
    
    unpadded_10bit value computing is moved sooner in the code in order to
    be able to use it during computing of minimal content size. Also make sure to
    only set it for 10bit.
    
    Fix 'Overread buffer' error when the content is not lucky enough to have
    (enough) padding bytes at the end for not being rejected by the formula
    based on the stride value
    
    Fixes ticket #10259.
    Signed-off-by: 's avatarJerome Martinez <jerome@mediaarea.net>
    Signed-off-by: 's avatarMarton Balint <cus@passwd.hu>
    05e34523
dpx.c 23.2 KB