SCP stalled most of the time

Most likely because the SD card is writing back (flushing buffer). If it oscillates in speed, then this would indicate it.

You should also use iperf to check your network connection.