We typically are only interesed in a single type of metadata set, so it is
better to keep them separated instead of always filtering for them.

Also use av_dynarray_add for increasing their array.
Signed-off-by: Marton Balint <cus@passwd.hu>