Your USB receiver should be in the rear port. This should improve the remote performance.
Regarding 1) – if this after a period of inactivity, it’s probably due to the CPU governor and there being a slight delay bringing the CPU back up again. Switching CPU governor could be useful in checking this.
Nonetheless, first thing I would do is put the receiver in the right place.
Sam