• Andreas Rheinhardt's avatar
    avutil/aes: Don't use misaligned pointers · 73930e4f
    Andreas Rheinhardt authored
    The AES code uses av_aes_block, a union consisting of
    uint64_t[2], uint32_t[4], uint8_t[4][4] and uint8_t[16].
    subshift() performs byte-wise manipulations of two av_aes_blocks,
    but when encrypting, it does so with a shift of two bytes;
    more precisely, it uses
    "av_aes_block *s1 = (av_aes_block *) (s0[0].u8 - s)"
    and lateron uses the uint8_t[16] member to access s0.
    Yet av_aes_block requires to be suitably aligned for
    the uint64_t[2] member, which s0[0].u8 - 2 is certainly
    not. This is in violation of 6.3.2.3 (7) of C11. UBSan
    reports this in the aes_ctr, mov-3elist-encrypted,
    mov-frag-encrypted, mov-tenc-only-encrypted and srtp
    tests.
    Furthermore, there is another issue here: The pointer points
    outside of s0; this works, because all the accesses lateron
    use an index >= 3. (Clang-)UBSan reports this as
    "runtime error: index -2 out of bounds for type 'uint8_t[16]'".
    
    This commit fixes both of these issues: The latter issue
    is fixed by applying an offset of "+ 3" during the cast
    and subtracting this from the indices used lateron.
    The former issue is solved by not casting to av_aes_block*
    at all; instead simply cast to unsigned char*.
    Signed-off-by: 's avatarAndreas Rheinhardt <andreas.rheinhardt@outlook.com>
    73930e4f
Name
Last commit
Last update
compat Loading commit data...
doc Loading commit data...
ffbuild Loading commit data...
fftools Loading commit data...
libavcodec Loading commit data...
libavdevice Loading commit data...
libavfilter Loading commit data...
libavformat Loading commit data...
libavutil Loading commit data...
libpostproc Loading commit data...
libswresample Loading commit data...
libswscale Loading commit data...
presets Loading commit data...
tests Loading commit data...
tools Loading commit data...
.gitattributes Loading commit data...
.gitignore Loading commit data...
.mailmap Loading commit data...
.travis.yml Loading commit data...
CONTRIBUTING.md Loading commit data...
COPYING.GPLv2 Loading commit data...
COPYING.GPLv3 Loading commit data...
COPYING.LGPLv2.1 Loading commit data...
COPYING.LGPLv3 Loading commit data...
CREDITS Loading commit data...
Changelog Loading commit data...
INSTALL.md Loading commit data...
LICENSE.md Loading commit data...
MAINTAINERS Loading commit data...
Makefile Loading commit data...
README.md Loading commit data...
RELEASE Loading commit data...
configure Loading commit data...