Vero 4K + network speed major disappointment

OK. so I have checked the packet analysis.


So, both devices are struggling with that. However I do see that too on my transfers (870Mbps), and as we are strying to reach max speed here, this is to be expected (especially if one device is not as fast as the other one).
One thing that can help, is to adapt the buffer sizes on the network cards. My NAS has it set to maximums. Note that this can also backfire if the CPU/System cannot free up the buffers. It however helps lower the interrupt requests.
Note2: This won’t work on the Vero, the driver has no such capability.

root@nas:~# ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX:             2047
RX Mini:        0
RX Jumbo:       0
TX:             511
Current hardware settings:
RX:             2047
RX Mini:        0
RX Jumbo:       0
TX:             511

The one thing I do not see on decent/clean setups, are the below error messages. means there is another switch in the network, and thereare some issue with these.
Note, in very simple terms, the spanning-tree protocol is usually inter-switch communication, elections etc. to determin who is master, and exchange informations etc.

So - I rather suspect there are issues on the traffic transfers at high speed where one side is not able to reception the data fast enough. This effect however can be induced by a switch (layer 2, it is transparent to both TCP/IP session communication parties), or by the layer 2 of the network card drivers.
We see these from time to time. Broadcom chips have made issues in the past, reason that for high-speed network interfaces we tend to only use Intel Chipsets based network cards, and no Dual or Quadruple cards.

@Wuffzack Have you tried an iperf3 server NOT running on a system with 4 port link aggregation with the Vero4k?

I am using a NETGEAR GS208-100UKS 8 Port Gigabit Switch to which my Vero 4k+ and Generic HTPC (file server) are directly connected and I get good speeds:

HTPC → Vero4k+

# iperf3 -c 192.168.0.147
Connecting to host 192.168.0.147, port 5201
[  4] local 192.168.0.6 port 40120 connected to 192.168.0.147 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   106 MBytes   883 Mbits/sec    0    428 KBytes
[  4]   1.00-2.01   sec   101 MBytes   846 Mbits/sec    0    428 KBytes
[  4]   2.01-3.00   sec   103 MBytes   868 Mbits/sec    0    479 KBytes
[  4]   3.00-4.00   sec   106 MBytes   889 Mbits/sec    0    570 KBytes
[  4]   4.00-5.01   sec   106 MBytes   883 Mbits/sec    0    601 KBytes
[  4]   5.01-6.00   sec   100 MBytes   844 Mbits/sec    0    601 KBytes
[  4]   6.00-7.01   sec   104 MBytes   862 Mbits/sec    0    601 KBytes
[  4]   7.01-8.01   sec   101 MBytes   853 Mbits/sec    0    601 KBytes
[  4]   8.01-9.00   sec   102 MBytes   861 Mbits/sec    0    601 KBytes
[  4]   9.00-10.01  sec   104 MBytes   869 Mbits/sec    0    631 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.01  sec  1.01 GBytes   866 Mbits/sec    0             sender
[  4]   0.00-10.01  sec  1.01 GBytes   864 Mbits/sec                  receiver

iperf Done.

Vero 4k+ → HTPC

$ iperf3 -c 192.168.0.6
Connecting to host 192.168.0.6, port 5201
[  4] local 192.168.0.147 port 32995 connected to 192.168.0.6 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   112 MBytes   941 Mbits/sec    0   2.27 MBytes
[  4]   1.00-2.00   sec   110 MBytes   921 Mbits/sec    0   2.18 MBytes
[  4]   2.00-3.00   sec   111 MBytes   934 Mbits/sec    0   2.38 MBytes
[  4]   3.00-4.00   sec   110 MBytes   927 Mbits/sec    0   2.55 MBytes
[  4]   4.00-5.00   sec   111 MBytes   926 Mbits/sec    0   2.68 MBytes
[  4]   5.00-6.00   sec   110 MBytes   927 Mbits/sec    0   1.96 MBytes
[  4]   6.00-7.00   sec   111 MBytes   928 Mbits/sec    0   2.08 MBytes
[  4]   7.00-8.00   sec   110 MBytes   927 Mbits/sec    0   2.17 MBytes
[  4]   8.00-9.00   sec   110 MBytes   926 Mbits/sec    0   2.24 MBytes
[  4]   9.00-10.00  sec   110 MBytes   920 Mbits/sec    0   2.29 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.08 GBytes   928 Mbits/sec    0             sender
[  4]   0.00-10.00  sec  1.08 GBytes   926 Mbits/sec                  receiver

iperf Done.

Yes. No difference.
To summarize my findings:
The problem only occurs, if the Vero is connected with the internal eth0 port to my TP-Link JetStream switch or an old DLink unmanaged Gigabit switch.

It does not occur, if I use USB ethernet adapter on the Vero or
if I insert something else (Mikrotik router) between Vero eth0 and the TP-Link.

I have a bunch of TP-Links lying around. I can test them, if you want me to.
They are usually nice, cheap and well working.
Only problem is, that TP-Link changes the hardware revision like other people change their underwear.
If you get reports from people having problems with TP-Link switches, you should ask them for the hardware revision. Some switches of the same type have 5 or 6 versions, each with different hardware.

BTW, my JetStream has hardware version 1.

Hi,

Please try the debug kernel:

wget "https://collab.osmc.tv/s/GOHJ6R7D3zyUDbT/download" -O kernel.deb
dpkg -i kernel.deb
reboot

Run iperf again. When you get bad results, type grab-logs -A and send me the URL.

Sam

1 Like

Very bad.
Here is the URL

Forgot to tell you, how bad:

root@osmckino:~# iperf3 -c whs.box
Connecting to host whs.box, port 5201
[  4] local 10.1.0.41 port 46744 connected to 10.1.0.10 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  13.1 MBytes   110 Mbits/sec   51   7.07 KBytes
[  4]   1.00-2.00   sec  17.5 MBytes   147 Mbits/sec   49   14.1 KBytes
[  4]   2.00-3.00   sec  15.6 MBytes   131 Mbits/sec   42   8.48 KBytes
[  4]   3.00-4.00   sec  15.1 MBytes   127 Mbits/sec   46   4.24 KBytes
[  4]   4.00-5.00   sec  14.3 MBytes   120 Mbits/sec   44   11.3 KBytes
[  4]   5.00-6.00   sec  16.1 MBytes   135 Mbits/sec   44   14.1 KBytes
[  4]   6.00-7.00   sec  14.8 MBytes   124 Mbits/sec   42   9.90 KBytes
[  4]   7.00-8.00   sec  12.8 MBytes   107 Mbits/sec   48   7.07 KBytes
[  4]   8.00-9.00   sec  14.9 MBytes   125 Mbits/sec   44   17.0 KBytes
[  4]   9.00-10.00  sec  15.9 MBytes   134 Mbits/sec   38   7.07 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   150 MBytes   126 Mbits/sec  448             sender
[  4]   0.00-10.00  sec   150 MBytes   126 Mbits/sec                  receiver

iperf Done.

This kernel isn’t intended to fix the issue, but to show me some clues as to what the issue is

Sam, I found another switch with the issue:
TP-Link TL-SG1005D (cheap unmanaged desktop switch)
Hardware Version 5

And another one:
TP-Link TL-SG108E (Older easy smart managed switch, I hate this thing)
Hardware version 1.0

And another one:
TP-Link TL-SG2008 8 port managed smart switch. I like this device very much.
Hardware version 1.0

root@osmckino:~# iperf3 -c whs.box -R
Connecting to host whs.box, port 5201
Reverse mode, remote host whs.box is sending
[  4] local 10.1.0.41 port 56098 connected to 10.1.0.10 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  5.90 MBytes  49.5 Mbits/sec
[  4]   1.00-2.00   sec  1.19 MBytes  10.0 Mbits/sec
[  4]   2.00-3.00   sec  1.49 MBytes  12.5 Mbits/sec
[  4]   3.00-4.00   sec  1.53 MBytes  12.8 Mbits/sec
[  4]   4.00-5.00   sec   829 KBytes  6.79 Mbits/sec
[  4]   5.00-6.00   sec  12.7 KBytes   104 Kbits/sec
[  4]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec
[  4]   7.00-8.00   sec  89.1 KBytes   730 Kbits/sec
[  4]   8.00-9.00   sec  89.1 KBytes   730 Kbits/sec
[  4]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  11.2 MBytes  9.44 Mbits/sec                  sender
[  4]   0.00-10.00  sec  11.1 MBytes  9.31 Mbits/sec                  receiver

iperf Done.

Just looked it up, the failing D-Link switch is a pretty old unmanaged DGS-1024D. With annoying fan.

I probably am the record holder for “having the most switches, the Vero 4K+ fails with”.
:wink:

I realized that the new computer I have provides better throughput that my mac mini 6,2 … especially less retries :slight_smile: Almost pristine.

@Wuffzack: do you have a switch that doesn’t have this issue? Can you run the debugging kernel on the good connection and the bad?

Sam

Yes, I have one. I have powered down the Vero in the meantime, do I have to install the debug kernel again, or is it persistant? Is a “good” log enough? (I already posted the bad one).

Yes, a good log will be fine. If you installed it earlier and didn’t downgrade it will be persistent

Here it is:

root@osmckino:~# iperf3 -c whs.box
Connecting to host whs.box, port 5201
[  4] local 10.1.0.41 port 56344 connected to 10.1.0.10 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   111 MBytes   930 Mbits/sec    0    272 KBytes
[  4]   1.00-2.00   sec   109 MBytes   914 Mbits/sec    0    272 KBytes
[  4]   2.00-3.00   sec   112 MBytes   936 Mbits/sec    0    272 KBytes
[  4]   3.00-4.00   sec   112 MBytes   940 Mbits/sec    0    272 KBytes
[  4]   4.00-5.00   sec   112 MBytes   939 Mbits/sec    0    272 KBytes
[  4]   5.00-6.00   sec   112 MBytes   940 Mbits/sec    0    272 KBytes
[  4]   6.00-7.00   sec   112 MBytes   938 Mbits/sec    0    272 KBytes
[  4]   7.00-8.00   sec   112 MBytes   940 Mbits/sec    0    272 KBytes
[  4]   8.00-9.00   sec   112 MBytes   940 Mbits/sec    0    272 KBytes
[  4]   9.00-10.00  sec   112 MBytes   939 Mbits/sec    0    272 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.09 GBytes   936 Mbits/sec    0             sender
[  4]   0.00-10.00  sec  1.09 GBytes   934 Mbits/sec                  receiver

iperf Done.
root@osmckino:~# grab-logs -A
Logs successfully uploaded.
Logs available at https://paste.osmc.tv/ukecazolol

Link

For what it’s worth, so far I have tested with the following switches and have not been able to reproduce slow speeds on my Vero4k+:

Virgin Media Superhub 2
TP-Link TL-SG108E Revision 1.0
Netgear GS608 v3
D-Link DGS-1224T
D-link DGS-3420-52P

These range from cheap and cheerful unmanaged to expensive managed enterprise switches.

This issue is proving to be challenging to solve due to neither Sam or myself being able to reproduce it yet… Hopefully Sam will have his hands on an affected switch shortly.

This is getting a little bit frustrating to keep reading this if i’m honest.

It is not only switches that are the problem, myself and user Matt.Capes still have the same problem when we remove our switches from our networks, probably other users as well but i have lost track of the multitude of threads/users with similar issues (and did these other users even test network performance without a switch anyway ?)

I have replied to Sam on at least 2 occasions on 2 threads to point this out, but he hasn’t acknowledged this.

Also why has Sam not got one of the affected Vero’s back in to actually test the unit ?
Surely that would have been one of the first things to happen wouldn’t it not ?
All I keep reading for several days now is that he will have one of the affected switches in ‘shortly’ to to test… It cant be that hard to buy a £20 switch off ebay/amazon and get it delivered next day can it?