Thank you very much for your confirmation and hint.
Since the kernel is compiled with CONFIG_MODVERSIONS enabled, I was looking for the kernel headers and Module.symvers only and considered to build only the module I need, or, use the media_build and rebuild only the media dirivers and not the whole kernel. This is why I was referring to perl - it is used by the media_build:
git://linuxtv.org/media_build.git
Compiling the whole kernel again with the patched driver will indeed take a lot of time on a pi0. I could use a spare Pi2/Pi3 but I’m afraid that the resulting code won’t be armv6 compatible. I have no Idea what these new Linux Distributions have hidden in their build scripts. I’m used to compile everything by hand (more like BSD style) and I think I’ll use the pi0 for the kernel compilation, at least I’m sure that these Debian build automation scripts will take the correct platform architecture (armv6) for the build.
Problem resolved. Consider this thread closed / resolved.