Iperf3 results?


#1

Hi,

I’ve been struggling with ethernet transmission problems that I’ve been trying to fault find for weeks now. Anyway…

Is anyone able to please confirm or deny if the stats returned by iperf3 on a pi2 which has KODI running (but not actively playing/streaming anything) are meaningful? I think I’m seeing gaps in network transmission on pi’s running OSMC/Kodi but not vanilla raspbian, and it would be helpful to know if I’m chasing ghosts or not.

Many thanks.

Mark W.


#2

I am not aware of anything like that. OSMC and Raspbian differ in a number of ways and the issue might have nothing to do with Kodi itself.

If you can provide some more details, perhaps we can help. The more information, the better, including details of network configuration and how you ran the tests.


#3

Sure.

All networking is via gigabit ethernet, no (relevant) wifi. I have a pair of PI 2s configured with OSMC, and a pair of PI 1s configured with Raspbian. My distribution is as follows:-

Pair of QNAP NAS serving mostly a couple of laptops from one and mostly media from the other, connected to…
Netgear GS110TP (internet router connects here too)
Netgear GS728TP (WAP connects here too)
one connection goes to Netgear GS105PE and on to a PI (plus DVD player and TV)
One conneection goes to a 2nd GS110TP then on to a Netgear GS108PE and another PI (plus DVD player, amp and TV)

This setup was working faultlessly for about a year, until a few weeks ago.

My primary testing has been with the first pi with only 3 switches to traverse. I have iperf3 loaded on all PIs. I’ve been SSH into a pair of PIs then using iperf3. I have a Raspian PI plugged into the GS110TP closest to thre NAS’, which I’ve been using to initiate tests as e.g. iperf3 -t600 -c192.168.1.180 -u (or without -u). I’ve been swapping back an forth between the “normal” OSMC PI and the other Raspbian PI, and using that as a server with iperf3 -s.

Switches and cabling are therefore the same.

iperf3 is reporting essentially perfect transmission with the Raspian PI, however when I swap to the OSMC PI I’m seeing significant dropped packets.

The actual symptom that kicked this off is KODI replay on the OSMC PI had started fairly regularly stopping and returning to the KODI menu. MKV media on NAS being shared via OS level NFS v3 mount on the OSMC PIs. I was trying to ascertain if the issue was the NAS, the network or the PIs.

Regards,
Mark W.


#4

I suspect that not too many people who visit this forum are using 28-port smart switches. That’s quite a rig you have there!

Just to kick off proceedings, I see that the switches are all power-over-ethernet (PoE) devices. To clear up one point, are you using PoE to power the RPis?


#5

If I remember correctly there were others reporting issues with gigabit and managed switches. Try to use the search function maybe you find something


#6

Enable Flow Control if you can (should be able to on a Smart Switch); or try adding smsc95xx.turbo_mode=Y to the end of the line in /boot/cmdline.txt then rebooting.


#10

Sam, I’ve just tried both of these against iperf3 test scenario and can see no appreciable difference with the throughput drop outs with the osmc PI as the receiver.

With both sender and receiver as 100BaseT would it really make that much difference anyway?

Regards
Mark W.


#11

It would be good to actually see the iperf results.


#15

Please find below 60 second example of TCP/IP transmission for Raspbian PI to ODMS PI. This is pretty representative, and has flow control turned on on the switches and smsc95xx.turbo_mode=Y.

iperf3 -t60 -c192.168.1.176
Connecting to host 192.168.1.176, port 5201
[ 4] local 192.168.1.173 port 60970 connected to 192.168.1.176 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 1.09 MBytes 9.14 Mbits/sec 6 1.41 KBytes
[ 4] 1.00-2.04 sec 2.50 MBytes 20.2 Mbits/sec 26 110 KBytes
[ 4] 2.04-3.00 sec 6.25 MBytes 54.3 Mbits/sec 0 132 KBytes
[ 4] 3.00-4.16 sec 7.50 MBytes 54.3 Mbits/sec 0 132 KBytes
[ 4] 4.16-5.13 sec 6.25 MBytes 54.2 Mbits/sec 0 132 KBytes
[ 4] 5.13-6.09 sec 6.25 MBytes 54.7 Mbits/sec 0 132 KBytes
[ 4] 6.09-7.05 sec 6.25 MBytes 54.9 Mbits/sec 0 132 KBytes
[ 4] 7.05-8.00 sec 6.25 MBytes 54.8 Mbits/sec 0 132 KBytes
[ 4] 8.00-9.00 sec 175 KBytes 1.43 Mbits/sec 2 1.41 KBytes
[ 4] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes
[ 4] 10.00-11.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 4] 11.00-12.16 sec 6.25 MBytes 45.1 Mbits/sec 94 133 KBytes
[ 4] 12.16-13.12 sec 6.25 MBytes 54.8 Mbits/sec 0 133 KBytes
[ 4] 13.12-14.07 sec 6.25 MBytes 54.9 Mbits/sec 0 133 KBytes
[ 4] 14.07-15.03 sec 6.25 MBytes 55.0 Mbits/sec 0 133 KBytes
[ 4] 15.03-16.18 sec 7.50 MBytes 54.7 Mbits/sec 0 133 KBytes
[ 4] 16.18-17.13 sec 6.25 MBytes 54.7 Mbits/sec 0 133 KBytes
[ 4] 17.13-18.09 sec 6.25 MBytes 54.6 Mbits/sec 0 133 KBytes
[ 4] 18.09-19.05 sec 6.25 MBytes 54.8 Mbits/sec 0 202 KBytes
[ 4] 19.05-20.00 sec 6.25 MBytes 55.0 Mbits/sec 0 202 KBytes
[ 4] 20.00-21.15 sec 7.50 MBytes 54.7 Mbits/sec 0 202 KBytes
[ 4] 21.15-22.11 sec 6.25 MBytes 54.9 Mbits/sec 0 202 KBytes
[ 4] 22.11-23.07 sec 6.25 MBytes 54.7 Mbits/sec 0 202 KBytes
[ 4] 23.07-24.03 sec 6.25 MBytes 54.6 Mbits/sec 0 202 KBytes
[ 4] 24.03-25.17 sec 7.50 MBytes 54.8 Mbits/sec 0 202 KBytes
[ 4] 25.17-26.13 sec 6.25 MBytes 54.8 Mbits/sec 0 202 KBytes
[ 4] 26.13-27.08 sec 6.25 MBytes 54.8 Mbits/sec 0 202 KBytes
[ 4] 27.08-28.04 sec 6.25 MBytes 54.7 Mbits/sec 0 202 KBytes
[ 4] 28.04-29.00 sec 6.25 MBytes 54.7 Mbits/sec 0 202 KBytes
[ 4] 29.00-30.15 sec 7.50 MBytes 54.9 Mbits/sec 0 202 KBytes
[ 4] 30.15-31.10 sec 6.25 MBytes 54.7 Mbits/sec 0 202 KBytes
[ 4] 31.10-32.06 sec 6.25 MBytes 54.8 Mbits/sec 0 202 KBytes
[ 4] 32.06-33.02 sec 6.25 MBytes 54.9 Mbits/sec 0 202 KBytes
[ 4] 33.02-34.16 sec 7.50 MBytes 54.9 Mbits/sec 0 202 KBytes
[ 4] 34.16-35.12 sec 6.25 MBytes 54.8 Mbits/sec 0 202 KBytes
[ 4] 35.12-36.08 sec 6.25 MBytes 54.8 Mbits/sec 0 202 KBytes
[ 4] 36.08-37.03 sec 6.25 MBytes 54.7 Mbits/sec 0 202 KBytes
[ 4] 37.03-38.18 sec 7.50 MBytes 54.8 Mbits/sec 0 202 KBytes
[ 4] 38.18-39.14 sec 6.25 MBytes 54.8 Mbits/sec 0 202 KBytes
[ 4] 39.14-40.09 sec 6.25 MBytes 55.1 Mbits/sec 0 202 KBytes
[ 4] 40.09-41.04 sec 6.25 MBytes 55.0 Mbits/sec 0 202 KBytes
[ 4] 41.04-42.18 sec 7.50 MBytes 55.3 Mbits/sec 0 202 KBytes
[ 4] 42.18-43.14 sec 6.25 MBytes 55.0 Mbits/sec 0 202 KBytes
[ 4] 43.14-44.09 sec 6.25 MBytes 54.9 Mbits/sec 0 202 KBytes
[ 4] 44.09-45.04 sec 6.25 MBytes 55.2 Mbits/sec 0 202 KBytes
[ 4] 45.04-46.18 sec 7.50 MBytes 55.2 Mbits/sec 0 202 KBytes
[ 4] 46.18-47.13 sec 6.25 MBytes 55.0 Mbits/sec 0 202 KBytes
[ 4] 47.13-48.09 sec 6.25 MBytes 55.0 Mbits/sec 0 202 KBytes
[ 4] 48.09-49.04 sec 6.25 MBytes 54.9 Mbits/sec 0 202 KBytes
[ 4] 49.04-50.19 sec 7.50 MBytes 55.0 Mbits/sec 0 202 KBytes
[ 4] 50.19-51.14 sec 6.25 MBytes 55.2 Mbits/sec 0 202 KBytes
[ 4] 51.14-52.09 sec 6.25 MBytes 55.2 Mbits/sec 0 202 KBytes
[ 4] 52.09-53.00 sec 4.41 MBytes 40.5 Mbits/sec 1 1.41 KBytes
[ 4] 53.00-54.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes
[ 4] 54.00-55.08 sec 5.00 MBytes 38.7 Mbits/sec 130 148 KBytes
[ 4] 55.08-56.04 sec 6.25 MBytes 54.7 Mbits/sec 0 148 KBytes
[ 4] 56.04-57.00 sec 6.25 MBytes 54.6 Mbits/sec 0 148 KBytes
[ 4] 57.00-58.15 sec 7.50 MBytes 54.8 Mbits/sec 0 148 KBytes
[ 4] 58.15-59.11 sec 6.25 MBytes 54.7 Mbits/sec 0 148 KBytes
[ 4] 59.11-60.07 sec 6.25 MBytes 54.8 Mbits/sec 0 148 KBytes


[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-60.07 sec 352 MBytes 49.1 Mbits/sec 261 sender
[ 4] 0.00-60.07 sec 352 MBytes 49.1 Mbits/sec receiver

I’ll follow up with a Raspbian to Raspbian exampler soon.

Best regards.
Martk W.


#16

Does dmesg show any dropped kevents in this duration?
There’s a network issue here for sure.

Can you try avoiding the secondary switch?

There’s a stable 4.9 kernel (I believe Raspbian recently migrated to this version) coming for OSMC soon. It would be good if you could test iperf again on this kernel.

Can you also confirm you are running the same iperf commands, and same version of iperf on Raspian? I believe iperf2 is default in Debian Jessie (which OSMC and Raspbian are both based on), but assume you have chosen iperf3 for a reason.

Also interested in window size.


#17

Here’s a close mirror of your test: a Raspbian Pi 1 client to OSMC Pi 3 server, only this one goes through two crappy unmanaged switches and traverses subnets:

pi@raspberrypi:~ $ iperf3  -c 192.168.11.88 -t 60
Connecting to host 192.168.11.88, port 5201
[  4] local 192.168.8.31 port 41694 connected to 192.168.11.88 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.02   sec  6.08 MBytes  49.9 Mbits/sec    0    281 KBytes       
[  4]   1.02-2.12   sec  7.50 MBytes  57.2 Mbits/sec    0    337 KBytes       
[  4]   2.12-3.02   sec  6.25 MBytes  58.4 Mbits/sec    0    337 KBytes       
[  4]   3.02-4.11   sec  7.50 MBytes  57.7 Mbits/sec    0    358 KBytes       
[  4]   4.11-5.02   sec  6.25 MBytes  58.0 Mbits/sec    0    400 KBytes       
[  4]   5.02-6.11   sec  7.50 MBytes  57.6 Mbits/sec    0    400 KBytes       
[  4]   6.11-7.18   sec  7.50 MBytes  58.9 Mbits/sec    0    441 KBytes       
[  4]   7.18-8.04   sec  5.00 MBytes  48.4 Mbits/sec    0    441 KBytes       
[  4]   8.04-9.10   sec  7.50 MBytes  59.3 Mbits/sec    0    441 KBytes       
[  4]   9.10-10.17  sec  7.50 MBytes  58.9 Mbits/sec    0    441 KBytes       
[  4]  10.17-11.07  sec  6.25 MBytes  58.7 Mbits/sec    0    441 KBytes       
[  4]  11.07-12.14  sec  7.50 MBytes  58.8 Mbits/sec    0    441 KBytes       
[  4]  12.14-13.01  sec  6.25 MBytes  59.9 Mbits/sec    0    672 KBytes       
[  4]  13.01-14.09  sec  7.50 MBytes  58.6 Mbits/sec    0    672 KBytes       
[  4]  14.09-15.15  sec  7.50 MBytes  59.2 Mbits/sec    2    471 KBytes       
[  4]  15.15-16.03  sec  6.25 MBytes  59.2 Mbits/sec    0    471 KBytes       
[  4]  16.03-17.11  sec  7.50 MBytes  58.4 Mbits/sec    0    471 KBytes       
[  4]  17.11-18.03  sec  6.25 MBytes  57.4 Mbits/sec    0    471 KBytes       
[  4]  18.03-19.10  sec  7.50 MBytes  58.4 Mbits/sec    0    471 KBytes       
[  4]  19.10-20.15  sec  7.50 MBytes  60.0 Mbits/sec    0    471 KBytes       
[  4]  20.15-21.15  sec  6.45 MBytes  54.2 Mbits/sec    2    414 KBytes       
[  4]  21.15-22.04  sec  6.25 MBytes  58.8 Mbits/sec    0    458 KBytes       
[  4]  22.04-23.10  sec  7.50 MBytes  59.3 Mbits/sec    0    458 KBytes       
[  4]  23.10-24.17  sec  7.50 MBytes  59.0 Mbits/sec    0    458 KBytes       
[  4]  24.17-25.04  sec  6.25 MBytes  60.1 Mbits/sec    0    492 KBytes       
[  4]  25.04-26.11  sec  7.50 MBytes  58.8 Mbits/sec    0    492 KBytes       
[  4]  26.11-27.17  sec  7.50 MBytes  59.7 Mbits/sec    0    492 KBytes       
[  4]  27.17-28.00  sec  6.25 MBytes  62.8 Mbits/sec    2    331 KBytes       
[  4]  28.00-29.00  sec  7.13 MBytes  59.8 Mbits/sec    2    272 KBytes       
[  4]  29.00-30.08  sec  8.23 MBytes  64.1 Mbits/sec    0    296 KBytes       
[  4]  30.08-31.01  sec  7.45 MBytes  66.7 Mbits/sec    2    225 KBytes       
[  4]  31.01-32.03  sec  7.68 MBytes  63.6 Mbits/sec    0    245 KBytes       
[  4]  32.03-33.06  sec  8.14 MBytes  66.0 Mbits/sec    0    263 KBytes       
[  4]  33.06-34.10  sec  7.40 MBytes  59.9 Mbits/sec    0    283 KBytes       
[  4]  34.10-35.05  sec  7.50 MBytes  66.0 Mbits/sec    0    290 KBytes       
[  4]  35.05-36.08  sec  8.29 MBytes  67.6 Mbits/sec    2    233 KBytes       
[  4]  36.08-37.03  sec  7.25 MBytes  64.1 Mbits/sec    0    264 KBytes       
[  4]  37.03-38.00  sec  7.58 MBytes  65.1 Mbits/sec    0    284 KBytes       
[  4]  38.00-39.02  sec  7.89 MBytes  65.3 Mbits/sec    2    182 KBytes       
[  4]  39.02-40.01  sec  7.39 MBytes  62.2 Mbits/sec    0    230 KBytes       
[  4]  40.01-41.00  sec  7.31 MBytes  62.0 Mbits/sec    0    246 KBytes       
[  4]  41.00-42.01  sec  8.35 MBytes  69.5 Mbits/sec    0    266 KBytes       
[  4]  42.01-43.11  sec  8.74 MBytes  66.5 Mbits/sec    0    287 KBytes       
[  4]  43.11-44.00  sec  7.11 MBytes  67.0 Mbits/sec    0    303 KBytes       
[  4]  44.00-45.01  sec  7.91 MBytes  66.0 Mbits/sec    2    230 KBytes       
[  4]  45.01-46.01  sec  7.66 MBytes  64.5 Mbits/sec    0    270 KBytes       
[  4]  46.01-47.10  sec  8.65 MBytes  66.3 Mbits/sec    2    272 KBytes       
[  4]  47.10-48.05  sec  7.47 MBytes  65.7 Mbits/sec    0    230 KBytes       
[  4]  48.05-49.00  sec  7.15 MBytes  63.2 Mbits/sec    0    245 KBytes       
[  4]  49.00-50.01  sec  7.63 MBytes  63.7 Mbits/sec    0    264 KBytes       
[  4]  50.01-51.00  sec  7.84 MBytes  66.3 Mbits/sec    0    281 KBytes       
[  4]  51.00-52.00  sec  7.52 MBytes  63.0 Mbits/sec    2    219 KBytes       
[  4]  52.00-53.01  sec  7.94 MBytes  66.1 Mbits/sec    0    255 KBytes       
[  4]  53.01-54.00  sec  7.68 MBytes  64.8 Mbits/sec    0    277 KBytes       
[  4]  54.00-55.03  sec  8.17 MBytes  66.9 Mbits/sec    0    288 KBytes       
[  4]  55.03-56.01  sec  7.05 MBytes  60.4 Mbits/sec    1    214 KBytes       
[  4]  56.01-57.13  sec  8.73 MBytes  65.0 Mbits/sec    0    235 KBytes       
[  4]  57.13-58.01  sec  7.01 MBytes  66.6 Mbits/sec    0    253 KBytes       
[  4]  58.01-59.12  sec  8.44 MBytes  64.2 Mbits/sec    0    273 KBytes       
[  4]  59.12-60.01  sec  7.07 MBytes  66.8 Mbits/sec    0    290 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-60.01  sec   441 MBytes  61.7 Mbits/sec   21             sender
[  4]   0.00-60.01  sec   440 MBytes  61.6 Mbits/sec                  receiver

iperf Done.

This suggests, at least to me, that there’s nothing inherently wrong with either Raspbian or OSMC.


#18

You never experience a drop in your connection which is good.
Those speeds are fine for playing back HD video.


#21

For the record, mine’s still on 4.4. I guess I’ll upgrade it one day…

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.4.50+ #970 Mon Feb 20 19:12:50 GMT 2017 armv6l GNU/Linux

#22

Did you see the 4.9 thread?


#23

For the avoidance of doubt, I was simply pointing out that my Raspbian kernel is still on 4.4, though I doubt if it has a bearing on the iperf3 figures.

And, yes, I saw the Development thread, if that’s what you mean.


#24

Okay. I think Raspbian has since moved to 4.9

An update to 4.9 is now in the development thread.


#25

Yout iperf3 results have some of the characteristics of network buffer congestion (“bufferbloat”), though there are some anomalies:

  • there’s that strange retransmission episode right at the start;
  • the next episode occurs at around 8 seconds but then we need to wait until 52-53 seconds for the next one to occur.

It would be interesting if you ran it for a few minutes longer to see if it settles into some kind of cyclical pattern. If you do that, you might want to change the reporting interval to something like 3 seconds. (Please also use the </> symbol to format your data, as it’s difficult to read otherwise.)

I’d also be interested to see how UDP fares on your network with and without -b (bandwidth) being specified.


#26

Hmmm. iperf3 measurements seem particularly bad this evening. Wonder if that’s temperature related? Been quite warm this afternoon. This is all still Raspbian PI1 to OSMC PI2 as before.

UDP:-

Accepted connection from 192.168.1.173, port 60994
[  5] local 192.168.1.176 port 5201 connected to 192.168.1.173 port 37601
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec   120 KBytes   983 Kbits/sec  0.172 ms  0/15 (0%)
[  5]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec  0.175 ms  0/16 (0%)
[  5]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec  0.194 ms  0/16 (0%)
[  5]   3.00-4.00   sec   128 KBytes  1.05 Mbits/sec  0.204 ms  0/16 (0%)
[  5]   4.00-5.00   sec   120 KBytes   983 Kbits/sec  0.269 ms  1/16 (6.2%)
[  5]   5.00-6.00   sec   128 KBytes  1.05 Mbits/sec  0.241 ms  0/16 (0%)
[  5]   6.00-7.00   sec  80.0 KBytes   655 Kbits/sec  0.218 ms  0/10 (0%)
[  5]   7.00-8.00   sec  64.0 KBytes   524 Kbits/sec  0.206 ms  14/22 (64%)
[  5]   8.00-9.00   sec   128 KBytes  1.05 Mbits/sec  0.188 ms  0/16 (0%)
[  5]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  0.192 ms  0/16 (0%)
[  5]  10.00-11.00  sec   128 KBytes  1.05 Mbits/sec  0.215 ms  0/16 (0%)
[  5]  11.00-12.00  sec   128 KBytes  1.05 Mbits/sec  0.185 ms  0/16 (0%)
[  5]  12.00-13.00  sec   128 KBytes  1.05 Mbits/sec  0.188 ms  0/16 (0%)
[  5]  13.00-14.00  sec   128 KBytes  1.05 Mbits/sec  0.195 ms  0/16 (0%)
[  5]  14.00-15.00  sec   128 KBytes  1.05 Mbits/sec  0.187 ms  0/16 (0%)
[  5]  15.00-16.00  sec   120 KBytes   983 Kbits/sec  0.265 ms  1/16 (6.2%)
[  5]  16.00-17.00  sec   128 KBytes  1.05 Mbits/sec  0.218 ms  0/16 (0%)
[  5]  17.00-18.00  sec   120 KBytes   983 Kbits/sec  0.287 ms  1/16 (6.2%)
[  5]  18.00-19.00  sec   128 KBytes  1.05 Mbits/sec  0.232 ms  0/16 (0%)
[  5]  19.00-20.00  sec   128 KBytes  1.05 Mbits/sec  0.203 ms  0/16 (0%)
[  5]  20.00-21.00  sec   128 KBytes  1.05 Mbits/sec  0.207 ms  0/16 (0%)
[  5]  21.00-22.00  sec   128 KBytes  1.05 Mbits/sec  0.209 ms  0/16 (0%)
[  5]  22.00-23.00  sec   128 KBytes  1.05 Mbits/sec  0.211 ms  0/16 (0%)
[  5]  23.00-24.00  sec   128 KBytes  1.05 Mbits/sec  0.257 ms  0/16 (0%)
[  5]  24.00-25.00  sec   120 KBytes   983 Kbits/sec  0.270 ms  1/16 (6.2%)
[  5]  25.00-26.00  sec   112 KBytes   917 Kbits/sec  0.356 ms  1/15 (6.7%)
[  5]  26.00-27.00  sec   128 KBytes  1.05 Mbits/sec  0.266 ms  1/17 (5.9%)
[  5]  27.00-28.00  sec   128 KBytes  1.05 Mbits/sec  0.215 ms  0/16 (0%)
[  5]  28.00-29.00  sec   120 KBytes   983 Kbits/sec  0.244 ms  1/16 (6.2%)
[  5]  29.00-30.00  sec   128 KBytes  1.05 Mbits/sec  0.207 ms  0/16 (0%)
[  5]  30.00-30.04  sec  0.00 Bytes  0.00 bits/sec  0.207 ms  0/0 (nan%)

[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  5]   0.00-30.04  sec  3.74 MBytes  1.05 Mbits/sec  0.207 ms  21/479 (4.4%)

UDP with bandwidth limit raised to 100Mbs:-

Accepted connection from 192.168.1.173, port 60988
[  5] local 192.168.1.176 port 5201 connected to 192.168.1.173 port 40496
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datag                                                                                                             rams
[  5]   0.00-1.00   sec  8.01 MBytes  67.2 Mbits/sec  0.542 ms  161/1186 (14%)
[  5]   1.00-2.00   sec  9.86 MBytes  82.7 Mbits/sec  0.537 ms  108/1370 (7.9%)                                                                                                              
[  5]   2.00-3.00   sec  9.89 MBytes  83.0 Mbits/sec  0.449 ms  110/1376 (8%)
[  5]   3.00-4.00   sec  9.15 MBytes  76.7 Mbits/sec  0.643 ms  201/1372 (15%)
[  5]   4.00-5.00   sec  9.58 MBytes  80.3 Mbits/sec  0.525 ms  146/1372 (11%)
[  5]   5.00-6.00   sec  9.49 MBytes  79.6 Mbits/sec  0.434 ms  151/1366 (11%)
[  5]   6.00-7.00   sec  9.65 MBytes  80.9 Mbits/sec  0.598 ms  141/1376 (10%)
[  5]   7.00-8.00   sec  9.39 MBytes  78.8 Mbits/sec  0.529 ms  177/1379 (13%)
[  5]   8.00-9.00   sec  9.52 MBytes  79.9 Mbits/sec  0.658 ms  163/1382 (12%)
[  5]   9.00-10.00  sec  8.08 MBytes  67.8 Mbits/sec  0.504 ms  348/1382 (25%)
[  5]  10.00-11.00  sec  9.48 MBytes  79.6 Mbits/sec  0.408 ms  168/1382 (12%)
[  5]  11.00-12.00  sec  9.62 MBytes  80.7 Mbits/sec  0.597 ms  157/1388 (11%)
[  5]  12.00-13.00  sec  8.02 MBytes  67.3 Mbits/sec  0.408 ms  347/1374 (25%)
[  5]  13.00-14.00  sec  9.83 MBytes  82.4 Mbits/sec  0.609 ms  122/1380 (8.8%)                                                                                                              
[  5]  14.00-15.00  sec  8.28 MBytes  69.5 Mbits/sec  0.322 ms  321/1381 (23%)
[  5]  15.00-16.00  sec  6.35 MBytes  53.3 Mbits/sec  0.525 ms  566/1379 (41%)
[  5]  16.00-17.00  sec  6.90 MBytes  57.9 Mbits/sec  0.450 ms  496/1379 (36%)
[  5]  17.00-18.00  sec  7.74 MBytes  64.9 Mbits/sec  0.483 ms  387/1378 (28%)
[  5]  18.00-19.00  sec  9.82 MBytes  82.4 Mbits/sec  0.513 ms  115/1372 (8.4%)                                                                                                              
[  5]  19.00-20.00  sec  9.13 MBytes  76.6 Mbits/sec  0.408 ms  213/1382 (15%)
[  5]  20.00-21.00  sec  7.64 MBytes  64.1 Mbits/sec  0.384 ms  405/1383 (29%)
[  5]  21.00-22.00  sec  6.86 MBytes  57.5 Mbits/sec  0.267 ms  512/1390 (37%)
[  5]  22.00-23.00  sec  6.57 MBytes  55.1 Mbits/sec  0.419 ms  549/1390 (39%)
[  5]  23.00-24.00  sec  6.51 MBytes  54.6 Mbits/sec  0.438 ms  556/1389 (40%)
[  5]  24.00-25.00  sec  6.67 MBytes  56.0 Mbits/sec  0.516 ms  528/1382 (38%)
[  5]  25.00-26.00  sec  6.79 MBytes  56.9 Mbits/sec  0.396 ms  505/1374 (37%)
[  5]  26.00-27.00  sec  6.43 MBytes  53.9 Mbits/sec  0.380 ms  550/1373 (40%)
[  5]  27.00-28.00  sec  5.94 MBytes  49.8 Mbits/sec  0.399 ms  512/1272 (40%)
[  5]  28.00-29.00  sec  0.00 Bytes  0.00 bits/sec  0.399 ms  0/0 (nan%)
[  5]  29.00-30.00  sec  6.57 MBytes  55.2 Mbits/sec  0.605 ms  2018/2859 (71%)                                                                                                              
[  5]  30.00-30.04  sec   360 KBytes  82.1 Mbits/sec  0.491 ms  5/50 (10%)

[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datag                                                                                                             rams
[  5]   0.00-30.04  sec   322 MBytes  89.9 Mbits/sec  0.491 ms  10738/41218 (26%)                                                                                                             )

Nothing showing up in dmesg - no surpises as clearly looks like a network fault.

I’ll have to work at elimenating switches due to topology. Any suggestions as to whether this is switch or cable related?

PS. I will try with Raspbian to Raspbian though probably not 'til tomorrow.

Best regards,
Mark W.


#27

Tell a lie. Immediate repeat using Raspbian to Raspbian.

UDP:-

Accepted connection from 192.168.1.173, port 50120
[ 5] local 192.168.1.180 port 5201 connected to 192.168.1.173 port 39936
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 5] 0.00-1.00 sec 120 KBytes 983 Kbits/sec 0.413 ms 0/15 (0%)
[ 5] 1.00-2.00 sec 128 KBytes 1.05 Mbits/sec 0.500 ms 0/16 (0%)
[ 5] 2.00-3.00 sec 128 KBytes 1.05 Mbits/sec 0.513 ms 0/16 (0%)
[ 5] 3.00-4.00 sec 128 KBytes 1.05 Mbits/sec 0.525 ms 0/16 (0%)
[ 5] 4.00-5.00 sec 128 KBytes 1.05 Mbits/sec 0.528 ms 0/16 (0%)
[ 5] 5.00-6.00 sec 128 KBytes 1.05 Mbits/sec 0.515 ms 0/16 (0%)
[ 5] 6.00-7.00 sec 128 KBytes 1.05 Mbits/sec 0.522 ms 0/16 (0%)
[ 5] 7.00-8.00 sec 128 KBytes 1.05 Mbits/sec 0.514 ms 0/16 (0%)
[ 5] 8.00-9.00 sec 128 KBytes 1.05 Mbits/sec 0.522 ms 0/16 (0%)
[ 5] 9.00-10.00 sec 128 KBytes 1.05 Mbits/sec 0.514 ms 0/16 (0%)
[ 5] 10.00-11.00 sec 128 KBytes 1.05 Mbits/sec 0.505 ms 0/16 (0%)
[ 5] 11.00-12.00 sec 128 KBytes 1.05 Mbits/sec 0.507 ms 0/16 (0%)
[ 5] 12.00-13.00 sec 128 KBytes 1.05 Mbits/sec 0.516 ms 0/16 (0%)
[ 5] 13.00-14.00 sec 128 KBytes 1.05 Mbits/sec 0.515 ms 0/16 (0%)
[ 5] 14.00-15.00 sec 128 KBytes 1.05 Mbits/sec 0.516 ms 0/16 (0%)
[ 5] 15.00-16.00 sec 128 KBytes 1.05 Mbits/sec 0.513 ms 0/16 (0%)
[ 5] 16.00-17.00 sec 128 KBytes 1.05 Mbits/sec 0.514 ms 0/16 (0%)
[ 5] 17.00-18.00 sec 128 KBytes 1.05 Mbits/sec 0.512 ms 0/16 (0%)
[ 5] 18.00-19.00 sec 128 KBytes 1.05 Mbits/sec 0.509 ms 0/16 (0%)
[ 5] 19.00-20.00 sec 128 KBytes 1.05 Mbits/sec 0.507 ms 0/16 (0%)
[ 5] 20.00-21.00 sec 128 KBytes 1.05 Mbits/sec 0.515 ms 0/16 (0%)
[ 5] 21.00-22.00 sec 128 KBytes 1.05 Mbits/sec 0.511 ms 0/16 (0%)
[ 5] 22.00-23.00 sec 128 KBytes 1.05 Mbits/sec 0.515 ms 0/16 (0%)
[ 5] 23.00-24.00 sec 128 KBytes 1.05 Mbits/sec 0.525 ms 0/16 (0%)
[ 5] 24.00-25.00 sec 128 KBytes 1.05 Mbits/sec 0.531 ms 0/16 (0%)
[ 5] 25.00-26.00 sec 128 KBytes 1.05 Mbits/sec 0.524 ms 0/16 (0%)
[ 5] 26.00-27.00 sec 128 KBytes 1.05 Mbits/sec 0.518 ms 0/16 (0%)
[ 5] 27.00-28.00 sec 128 KBytes 1.05 Mbits/sec 0.526 ms 0/16 (0%)
[ 5] 28.00-29.00 sec 128 KBytes 1.05 Mbits/sec 0.527 ms 0/16 (0%)
[ 5] 29.00-30.00 sec 128 KBytes 1.05 Mbits/sec 0.516 ms 0/16 (0%)
[ 5] 30.00-30.03 sec 0.00 Bytes 0.00 bits/sec 0.516 ms 0/0 (nan%)


[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 5] 0.00-30.03 sec 3.74 MBytes 1.05 Mbits/sec 0.516 ms 0/479 (0%)

…and with bandwidth set to 100mMbs:-

Server listening on 5201

Accepted connection from 192.168.1.173, port 50122
[ 5] local 192.168.1.180 port 5201 connected to 192.168.1.173 port 56782
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 5] 0.00-1.00 sec 9.17 MBytes 76.8 Mbits/sec 0.995 ms 9/1183 (0.76%)
[ 5] 1.00-2.01 sec 10.5 MBytes 88.0 Mbits/sec 1.035 ms 21/1368 (1.5%)
[ 5] 2.01-3.00 sec 10.6 MBytes 89.1 Mbits/sec 0.877 ms 6/1358 (0.44%)
[ 5] 3.00-4.00 sec 10.4 MBytes 87.4 Mbits/sec 0.753 ms 37/1370 (2.7%)
[ 5] 4.00-5.00 sec 10.4 MBytes 87.6 Mbits/sec 0.873 ms 33/1370 (2.4%)
[ 5] 5.00-6.00 sec 10.3 MBytes 86.5 Mbits/sec 0.740 ms 62/1381 (4.5%)
[ 5] 6.00-7.00 sec 10.4 MBytes 87.3 Mbits/sec 0.816 ms 52/1384 (3.8%)
[ 5] 7.00-8.00 sec 10.2 MBytes 85.8 Mbits/sec 0.784 ms 77/1387 (5.6%)
[ 5] 8.00-9.00 sec 10.5 MBytes 88.3 Mbits/sec 0.811 ms 30/1377 (2.2%)
[ 5] 9.00-10.00 sec 10.4 MBytes 87.2 Mbits/sec 0.957 ms 53/1384 (3.8%)
[ 5] 10.00-11.00 sec 10.5 MBytes 88.1 Mbits/sec 0.818 ms 42/1387 (3%)
[ 5] 11.00-12.00 sec 10.5 MBytes 87.7 Mbits/sec 1.027 ms 42/1382 (3%)
[ 5] 12.00-13.00 sec 10.5 MBytes 88.0 Mbits/sec 0.765 ms 46/1385 (3.3%)
[ 5] 13.00-14.00 sec 10.6 MBytes 88.6 Mbits/sec 0.945 ms 20/1375 (1.5%)
[ 5] 14.00-15.00 sec 10.6 MBytes 88.4 Mbits/sec 0.990 ms 26/1377 (1.9%)
[ 5] 15.00-16.00 sec 10.5 MBytes 87.9 Mbits/sec 0.999 ms 32/1371 (2.3%)
[ 5] 16.00-17.00 sec 10.5 MBytes 88.0 Mbits/sec 0.922 ms 41/1383 (3%)
[ 5] 17.00-18.00 sec 10.6 MBytes 88.5 Mbits/sec 0.972 ms 36/1387 (2.6%)
[ 5] 18.00-19.01 sec 10.6 MBytes 88.5 Mbits/sec 1.040 ms 34/1389 (2.4%)
[ 5] 19.01-20.00 sec 10.2 MBytes 86.3 Mbits/sec 1.039 ms 70/1382 (5.1%)
[ 5] 20.00-21.00 sec 10.5 MBytes 88.4 Mbits/sec 0.770 ms 42/1388 (3%)
[ 5] 21.00-22.00 sec 10.5 MBytes 87.8 Mbits/sec 0.787 ms 44/1385 (3.2%)
[ 5] 22.00-23.00 sec 10.4 MBytes 87.4 Mbits/sec 0.795 ms 53/1385 (3.8%)
[ 5] 23.00-24.00 sec 10.6 MBytes 88.5 Mbits/sec 0.738 ms 37/1388 (2.7%)
[ 5] 24.00-25.00 sec 10.4 MBytes 86.9 Mbits/sec 0.791 ms 55/1381 (4%)
[ 5] 25.00-26.00 sec 10.5 MBytes 87.7 Mbits/sec 0.791 ms 36/1375 (2.6%)
[ 5] 26.00-27.00 sec 10.2 MBytes 85.4 Mbits/sec 0.850 ms 90/1394 (6.5%)
[ 5] 27.00-28.00 sec 10.5 MBytes 88.3 Mbits/sec 0.743 ms 47/1394 (3.4%)
[ 5] 28.00-29.00 sec 10.5 MBytes 87.7 Mbits/sec 0.970 ms 46/1390 (3.3%)
[ 5] 29.00-30.01 sec 10.5 MBytes 87.7 Mbits/sec 1.457 ms 44/1392 (3.2%)
[ 5] 30.01-30.03 sec 224 KBytes 82.2 Mbits/sec 0.876 ms 11/39 (28%)


[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 5] 0.00-30.03 sec 323 MBytes 90.1 Mbits/sec 0.876 ms 1274/41291 (3.1%)

Nothing in dmesg again.

Best regards,
Mark W.


#28

I’ve taken the liberty of formatting the figures in the first post using the </> symbol for ease of reading.

The first UDP figures at 1 Mbps are very bad. Even a Pi can handle that kind of traffic. They also seem to point away from buffer congestion being the issue, at least in this case. The 100 Mbps figures are very erratic but get worse as time goes by, so I guess could be buffer-related.

The Pi-to-Pi figures show that at least one segment of your LAN is working well. This should at least help you to narrow down the source of the problem.

Edit:

Impossible to say. Swapping cables methodically is an easy thing to do, though less convenient when they’re buried in walls. :wink: Maybe time to invest in a LONG piece of cat 5e and crimp some connectors to it.

Edit 2: There’s still the remaining question of whether you’re using PoE to power the RPi 2s. Even if the switches can supply sufficient peak-load current, the splitters would need to be up to spec, as well.