Slow ethernet iperf3 speeds until reboot

no, nothing.

Are you able to log into the vero - not remotely?
In that case, you could disable the network subsystem, remove the network driver, and reload the network driver, to see if the initial speed is back.
Other possibility is to set the speed to 1Gbps hard (not on autodetect) of the network interface.
I use this in the past on some devices known to have issues with ethernet speeds:

sudo ethtool -s enp1s0 speed 1000 duplex full autoneg on

If it again falls back to 100Mbps, you can try forcing the speed by disabling the auto negotiation.
It does not always work as the network-manager (under ubuntu at least) disables the interface if it canā€™t interface with it itself. And I canā€™t test it on my vero right now (on the road).

sudo ethtool -s enp1s0 speed 1000 duplex full autoneg off

So make sure you can reach it without network before doing it.

No I dont have the ability to logon without the ethernet.

Itā€™s not dropping back to a 100MB link, thatā€™s the problem. Itā€™s dropping to somewhere around 100-200mb.

It sounds like something is causing a duplex mismatch somewhere.

This thread is extreme confusing to read since two folks @floatingkiwi and @Ainsley are talking about a similiar local issue in parallel. Simply to understand the raw infrastructure @floatingkiwi forced me to read the whole thread to divide things out. :unamused:

@floatingkiwi If only a reboot/disable+enable the ethernet interface of the Vero corrects the symptom the issue is either inside the Vero or the connected switch/switch port since the initial autonegotiation resets the situation when the interface comes up. So,

  • Did you replace the cable between Vero and switch (I think I read in the beginning but not sure)?
  • Did you change the switch 's port the Vero is connected to?
  • Did you change the switch for tests? (well, I know not everybody owns a spare switch)
  • When this happens, are there other devices connected to that switch which transfers high bandwidth in that moment? (Saturating the uplink to the router)
  • When this happens, are there other devices connected to that switch which are able to gain full bandwidth?

Yepā€¦

While I do agree that the thread is a little confusing, it can be divided into two very different problems, which Iā€™ll try to detail below. If anyone disagrees with what I say, or thinks that Iā€™ve forgotten something important, please feel free to chip in.

Iā€™ll deal with @floatingkiwiā€™s problem first. This issue seems to be a randomly occurring slowdown of iperf3 itself and, on the basis of evidence presented so far, is not related to a slowdown of the network itself, nor has it been suggested that the performance of video streaming has been degraded. After the iperf3 slowdown has occurred once, all subsequent runs of iperf3 are also slow until the Vero4k+ is rebooted.

iperf3 figures show that, when the problem occurs, data throughput drops from around 945 Mbps to around 100-200 Mbps. In this case, the ethernet interface has clearly not renegotiated to 100 Mbps.

floatingkiwi has already confirmed that the problem occurs even when the Vero4k+ is directly attached to the same router as the PC running the iperf3 server. From this we can exclude it being down to a network issue per se between the media room and the office PC and we can infer that itā€™s not an issue with a switch, cable or port on the switch.

I have confirmed, via ifstat, that there is nothing else consuming the ethernet bandwidth on the Vero4k+ when a slowdown occurs. If it had been a congestion issue elsewhere on the network, I donā€™t see how a reboot of the Vero4k+ would clear the problem.

The iperf3 figures (using the -V flag) suggest that iperf3 itself is the problem (or at least a part of it). When the slowdown occurs, iperf3 uses high CPU and the process seems to become CPU-bound. I have examined the iperf3 source code and the CPU figures shown by iperf3 are for the iperf3 process itself, and are not for the whole operating system.

There seems to be some kind of state change on the Vero4K+ that makes iperf3 run in low-throughput / high-CPU mode. I donā€™t have a Vero4k+, so canā€™t investigate this further.

One outstanding test that @floatingkiwi can run is to see if thie issue is confined to iperf3 or if the slowdown, once it occurs, affects other network traffic, eg NFS and/or SMB.

@Ainsleyā€™s problem seems simpler. The Vero4k+ is showing a relatively large number of dropped ethernet packets and the interface is renegotiating from 1 Gbps to 100 Mbps. Iā€™ve asked for a before and after ethtool report and full logs after the renegotiation has occurred. We can take it from there.

Iā€™ve asked ainsley to start another thread so I can continue with my own one and not confuse everyone.

  1. yes all cables were changed.
  2. yes tried diff ports
  3. not possible
    4)Iā€™m not able to test this - no, there shouldnā€™t be anything else using bandwidth at the time.
  4. unable to test this either.

Nothing else ever seems to be affected by/during this slowdown of the vero.

I know the vero is slowing down because even though streaming isnā€™t affected, (150 ish mb is enough to stream most files), I can see the ssh service slowing down, and the device takes longer when using things like the netflix app to view content.

In particular, I know exactly when it has slowed down as when sshā€™ing in, it takes a second or 2 to ask for teh password, rather than show that text instantly.

So in summary, (and relating to @dillthedog comment above) I know the vero network is indeed slowing down and this isnā€™t just an iperf3 thing.

Yes you asked for before and after ethtool report.
I did the ā€œbeforeā€ one, but since then the connection didnā€™t degrade to a 100 Mbit/s connection, so I did not make the ā€œafterā€ report. I have a guess that the connection only sometimes switches to 100 Mbit/s when I reboot the device. I used to reboot the Vero relatively often, so maybe there actually is no problem that causes the connection do degrade.

To avoid further confusion, just ignore my problem, that maybe isnā€™t actually one.

No worries. I was expecting you to post all the details after you experienced the problem.

  1. When a slowdown occurs, can you run a dd file copy over the network and compare it with the iperf3 speed?

  2. Sorry to be a stickler for detail, but how do you know that the network is slowing down, other than by using iperf3?

  3. Do you have anything else to add re. my write-up?

Edit: I see youā€™ve answered point 2, but 150 Mbps is way faster than you need for a responsive SSH session.

I think Iā€™ve seen this. Have you tried pinging the device while speed is affected? Can we see that output? Did you say you had Ubiquiti Unifi gear in the chain somewhere?

Iā€™ll try the dd file copy. Iā€™m very much a linux newbie!

I havenā€™t got anything to add really to your writeup - all I know is that the vero is slowing down as iperf3 seems to be saying. I can tell from network responsiveness and app response times. THe last of which is odd, because my internal network speed, even though dropping to 150-200 is way faster than my internet connection, which is 100mb.

It is slow right now and still good 1ms (consistent) ping times. <1 to the switch.
No ubiquity stuff in my home system.

Sorry, was a shot in the dark from me, but something rung familiar in my ears :frowning:

Iā€™m gonna need help with this dd file copy thing fellas. Iā€™m assuming of=/dev/null is the directory of my remote server? How do i find this.

I know dosā€¦

Could you elaborate?

sshā€™ing in is slower response times when asking for password and performing tasks.
netflix app takes longer to start streaming the same program. Longer to list and download subtitles from the internet.
Viewing local server files and directories takes longer, noticeably.

Nope. Itā€™s a device on your Vero box. Use ā€œas isā€.

where is the movie file located? I tried something on the smb file server and no worky.