If the time code side data is overridden by the packet level, we also
make sure not to update `p->metadata` to a mismatched timecode.