omx-player should be the default for hwaccelerated playback and can be selected in the menu. dvd-player is better in some applications like dvd-menus.
vlc is not hw-accelerated on the pi and as no x-server is used in osmc, it will not have a videotarget to play to.