GPIO set to output by default on bootup?

No pins are set to output unless they are explicitly being used.
Unused pins are set to input (which may appear high if no pull-up is used).
See here for default pin usage: firmware/extra/dt-blob.dts at master · raspberrypi/firmware · GitHub

Which pin are you using?