"Vero V"... experiences

The Kodi web interface isn’t the best way of evaluating connectivity – I wouldn’t rely on that.

Honestly @sam_nazarko, at this point, I’m just trying to get any network connectivity to work, unsuccessfully it seems.

The WiFi interface is also dropping a large number of packets too (25%) which leads me to think there is something wrong with your network.

No. C.f. the Linux server figures for the interface (“enp5s0f1”) on the same LAN - 0.01% drop. I could probably drag out figures for other boxes on the same LAN - in fact, here’s the figures from an “rPi”, on the same LAN (look at “eth0”) with an uptime of 31 days:
image
Zero packet drops. Not one single packet. In 31 days. So what’s on the wire is absolutely fine and the ridiculous packet drop rate is nothing to do with our network. I’d look at the stats. for my workstation if I could remember how to drag them out of “Windoze”, but I suspect they’d be the same. There is nothing wrong with our networks.

If you could upload logs while connected to WiFi (with the Ethernet interface also active) that would be useful.

Sure, no problem. Just tell me what you’d like me to run.

To give network connectivity (at least via WiFi) a good test, I’d recommend checking for updates via My OSMC. You should upgrade to the April update anyway

Can’t see any way to do it. Update automatically is turned on (default) and I can’t see any way manually to check for updates - any hints?

We can swap the device over, …

I think that might be required, but let me run whatever tests you want first. Given that both NICs appear to have issues, and that I can’t plug in a USB3 disc and have it recognised, the highest level common system (I assume you’re using USB NICs?) would be the USB hardware - I suspect that might be… dodgy. But let’s see what we can find out first, eh?

Tell me what you want me to, very, very slowly, run, Sam.

Cheers,

Tris.

Oh sorry - my bad - you did say in “My OSMC” (a settings app. within a settings app…). Ran that, did nothing except report 40/40 (0bps) and then returned me to the “My OSMC” with no other information.

grablogs -A

Tris – let’s see a log, if possible.

You can check for updates via My OSMC under Updates and then Manual Controls → Check for Updates Now

We’ve shipped a lot of Vero V devices. WiFi and Ethernet are both tested before dispatch. WiFi is connected via the SDIO bus and not USB; Ethernet is connected to a pinmuxed PHY. USB is connected separately.

For all three interfaces to have failed and escaped testing at the manufacturing stage and the software imaging stage seem unlikely. I think you’ve a networking issue and a USB issue that we need to investigate, but one preempts the other.

There’s a simple way to test the USB ports. Try and reinstall OSMC. See Reinstalling OSMC - Vero V - OSMC for advice on how to do.

This will also update your system to the latest and greatest which always helps.

I have a Vero V in the corner compiling here via Ethernet and I’m getting:

osmc@osmc:~$ netstat -i
Kernel Interface table
Iface      MTU    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0      1500  2044732      0    147 0       1267823      0      0      0 BMdRU
lo       65536     6174      0      0 0          6174      0      0      0 LRU
osmc@osmc:~$ uptime
 23:55:40 up 1 day,  7:39,  2 users,  load average: 4.10, 4.18, 4.18

I’ve had networking issues before (even from CCNA certified users) and I hate this kind of network environment to troubleshoot. As I say – feel free to swap the unit over, but WiFi, Ethernet and USB sitting on all different buses is not going to be indicative of a hardware fault IMHO.

This is what I’d do that would take less than 10 minutes:

  • Reinstall OSMC from USB (verifies USB functionality)
  • Make a hotspot on a phone
  • Connect to hotspot via WiFi under My OSMC
  • Go to Add-ons and install an add-on like iPlayer and see if this works
  • Any issues encountered, upload a log (networking permitting) via My OSMC → Logs → Upload all Logs.

Cheers

Sam

1 Like

Hiya Sam,

As I say – feel free to swap the unit over, but WiFi, Ethernet and USB sitting on all different buses is not going to be indicative of a hardware fault IMHO.

OK, so no single common subsystem. Well, good to know.

Logs (cheers @ActionA, and/or presumably what @sam_nazarko said via “My OSMC”), try manual updates, reinstall OSMC, … - soon as I get a chance I’ll work through those and let’s see what we can find.
Meanwhile, switched on the office TV today… and there was the properly dimmed OSMC screen I’d left it on. So presumably whatever was causing the video to blank, irritatingly, was down to the CEC - probably being told the TV had been switched off? Whatever, that’s all now sorted so that’s good.

Pending working through that, here’s how other WiFi devices (in this case my 'phone) handle the local networks (WiFi and wired) - I’d be happier with a lower “ping” time, but I’d say the networks themselves are working fine…

Anyway, cheers for the help as always; I’ll be back to you when I’ve worked through some or all of those steps.

Tris.

Hi again Sam, all,

OK, current progress:
This morning, the wired Ethernet interface seemed to have limped back into getting an address, so both wired and WiFi connections were present.

There’s a simple way to test the USB ports. Try and reinstall OSMC.

Did that. I’m now running the April '24 version, entire process was smooth, fast and straightforward.
Since initial boot, I’ve gone through the initial setup, set up the WiFi, and that’s it - nothing else has been changed. The WiFi seems considerably improved - were there any driver changes between the Dec '23 (think that’s what it was running?) and Apr '24 versions? It now seems actually normally responsive, but without knowing the cause it’s equally possible it’s just randomly behaving better after a reboot.
Went to (manual) check for updates and it found 23.2MB of them (from memory of a message which flashed past quite quickly) - took about a second to download them, applied them, all apparently good.
Whilst the WiFi appears slightly less dead, has it fixed any wired connection issues? Apparently not:
image
(23 is the WiFi; 63 is the wired)

Alrighty, so how are the interfaces lookin’?
image
I think that’s most appropriately described as “Um…” - 22% packet drops on the WiFi, a somewhat better 3% on the wired I/F but still waaaaaaaaay too high.

While I’m on there, how’s that USB lookin’?
image
Well, the USB installation media is there, so that’s a good sign (it’s plugged into the USB3 port, BTW).
Replace it with the WD “Passport” drive, leave it a few seconds, and what do we get?
image
That’s looking somewhat healthier, and if I go into (for example) “Pictures” on the top level menu, the first (and only) item which appears is “931.5 GB Drive”. I’d say it’s found that then, even on a hot-plug (though, y’a know, I’d expect it to, but still…).

So hopefully we can tick USB off the issues as well, though I’ll actually try accessing things on that drive first (can’t even remember what’s on there :laughing:). Again - curiosity - were there any USB related updates between versions?

By my count, that leaves being able to access network resources and actually being able to access the network itself. Those drop rates are abysmal, and even with only 3% drop on the wired connection it still won’t establish an “ssh” session. Oh, and neither IP address will complete loading the web UI either, exactly the same as before (connection transfers no data then gets closed 2 minutes later). So no point even trying to access network resources without an actual network connection…
… which brings us to your suggestion about creating a 'phone hotspot? I’d really rather not do that if possible, as I have bugger-all data on my plan (my 'phone very rarely leaves the house), trying to download an app. will probably sting me with an unexpected and unpleasant bill at the end of the month. So I’d prefer not to try that if possible; meanwhile I’m about to try an “Upload all Logs”…

… 1 minute later and still “Uploading logs”…

… 3 minutes later (screen dim just kicked in) and at least there’s a progress bar now - at about 30%? …
… minute or so after that, “Could not retrieve URL. Copy logs to SD Card Instead(sic)?”. As there isn’t an SD card inserted, AFAIK, I’ll come back to that a bit later, grab the logs, and dump them on this case.

Cheers,

Tris.

I said “Yes” to “Copy logs…”… and that was that. Presumably they’ve been stashed on the local filesystem somewhere - save me digging through WiKis and give me a hint?

Cheers,

Tris.

When you got that message about saving locally I believe it stated the file was saved to /boot/uploadlog.txt. The not uploading and taking forever to generate is usually an indication that there is something generating a lot of log messages or a long uptime.

Could well have done, @darwindesign, but if so I didn’t notice it. However many thanks for saving me the trouble - we appear to have a winner:
image
I’m in the middle of half-a-dozen things, all of them irritating, but I’ll grab that off and dump it on here shortly.

Much appreciated!

Tris.

… or I could just yank the wired Ethernet and let it use the (awful, but better) WiFi instead :rofl:

https://paste.osmc.tv/ifefeyijiy

Incidentally, it shows that on screen with a space between the “/” and the “ife…”.

Cheers,

Tris.

Hi,

None whatsoever.

No – I haven’t changed anything here either.

One team member has commented:

By the send/receive type 20 you can see, there is a data exchange between the local system and the ssh target … but the communcation is stuck since the client waits for SSH2_MSG_KEX_ECDH_REPLY … which does not come in anymore.

This smells like a local network problem; The classic seems to be the MTU size. A simple ping test using ‘don’t fragment’ flag would at least help to troubleshoot this further.

ifconfig output shows that the dropped frames are only in the RX direction:

eth0: flags=-28669<UP,BROADCAST,MULTICAST,DYNAMIC>  mtu 1500
        ether 94:cc:04:60:01:7c  txqueuelen 1000  (Ethernet)
        RX packets 221740  bytes 112447503 (107.2 MiB)
        RX errors 0  dropped 8620  overruns 0  frame 0
        TX packets 18285  bytes 11402644 (10.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 16  

Nothing obvious in the logs, but I am not a networking expert.
You are in the UK if I recall, so it’s relatively quick (and straight forward) to swap the device over if you’d like.

Hiya Sam,

OK, I’m now officially completely and utterly confused. I’ll come back to that in a moment; let’s deal with your responses first:

I asked whether you’d updated the USB drivers? You said no. I asked whether you’d updated anything relating to the WiFi? You said no. And yet, here we are, with USB now working (when previously it apparently wasn’t, though I couldn’t poke at an OS level 'cause I had no “ssh” access) and WiFi apparently working somewhat less badly. Probably…
If there were no updates to the drivers which could explain the discrepancy, can you think of any other reason why an OS reinstall/upgrade on a device only a few days old (from my perspective anyway) would apparently resolve/improve those issues? I’m not trying to be an awkward git (not as in the software repository…), BTW, just trying to see whether it’s worth returning the unit to you for checks (yes, you remembered correctly, I am in the UK; yes, it should be relatively quick and straightforward to arrange, but it’s still a PiTA for me, and for your staff in processing any return - I’ll do that if we decide the hardware does need checking, but I’d rather avoid it otherwise). “Do I return it?” is a question which seems to be getting more and more complicated - see below…

One team member has commented:

By the send/receive type 20 you can see, there is a data exchange between the local system and the ssh target … but the communcation is stuck since the client waits for SH2_MSG_KEX_ECDH_REPLY … which does not come in anymore.
This smells like a local network problem; The classic seems to be the MTU size.

Yea… no. Don’t get confused by the IP addresses; this is a simple LAN connection, with everything using an MTU of 1500 and no fragmentation possible because this is a simple LAN connection (there’s no routing involved; nothing to fragment any packets):
image

ifconfig output shows that the dropped frames are only in the RX direction: …

That’s because, despite the similarity in names between the RX and TX drop counts, they’re not showing the same thing. Well, unless what they’re showing is a critical resource issue, but that’s not the problem here.
RX packets get dropped because of (critical resource issues or) getting rejected as junk by the link-layer. For example, a bad Ethernet frame checksum. It’s a sign of data corruption, possibly “on the wire” (but the stats. from all our other boxes on that LAN say “Computer says no” to that) but generally within the receiving hardware. TX packets get dropped because of critical resource issues. It’s slightly more complex than that, but that’s the general gist of it. If there were any TX drops at all I’d be at least mildly concerned…

Oh, as for the WiFi side of things, here is a picture of my 'phone quite literally sitting on the top of the “Vero V”, running a “SpeedTest”, and transferring at full wire-speed for that ISP:


(that’s also much more like the sort of “ping” time I’d expect…)
So I’m not sure why the “Vero V” is apparently dropping between a fifth and a quarter of the WiFi packets, because if, for example, my 'phone were doing the same there’s no way I could possibly get that sort of result.
However, I didn’t even know the “Vero V” had WiFi until this support thread, so I’m not actually even slightly bothered by that - I just find it… curious (and had there have been a common sub-system between the NICs then it would possibly have been diagnostically useful).

So, where are we at the moment? Well, I did a little bit of playing around earlier…
Unplug the “WD Passport” drive from the USB3 port and replace it with a random USB3 GBE NIC (I have these sorts of things lying around…). That gets the address “193.119.171.20”, and when I try to “ssh” to it, I get…
image
Helllllooooo… That appears to work, absolutely fine. Alright, let’s stress it a bit:


“Beer Mat Maths” - call that 13 minutes, ~57 megabytes per second, and while the RX drop count should be zero, it’ll do - it’s certainly about 10x the bandwidth which would be needed to watch a 3D Blu-Ray, so all good.

So that’s a clear indication that the onboard NIC is buggered, yes? Plug in an external NIC and all’s well; use the onboard one and it won’t even complete an “ssh” handshake - pretty clear indication that there’s a hardware fault, yes?..

… well…
… well actually… no.
That NIC was connected to a more readily accessible cable; straight from a different port on the office switch, via a PoE injector (it’s a grabbable cable I keep to hand on my workbench). I wonder what happens if I plug that cable into the “Vero V” onboard NIC? Well this is what happens…


Don’t even need “Beer Mat Math.s” here; that’s about 12 minutes, so it’s fine and faster.

Hm.

OK, so it has to be something like a knackered patch lead, doesn’t it (even though these are commercial pre-made leads from a supplier - “RhinoCables” - we’ve been using for years without a single issue)? Swap patch leads (so back to using the switch to which the “Vero 2” was connected without issue, and to which a “Dune Base 3D”, “LG 3D Blu-Ray player”, the TV, and a “SkyQ Mini-Box” are all connected, without a single issue)… buggered again. Different ports? No difference. Plug that, or any other patch lead from that switch, into the external NIC? Still buggered…

Hm.

I’ve been mucking around with networks for so long it’s within the last decade that I finally chucked out my “Vampire Tap” tools, and I wonder how many folks reading this even know what a vampire tap is without looking it up. It’s literally been part of the day job for over 3 decades, and I’ve even written router firmware. I have no idea how the “Vero V” can apparently be somehow incompatible with that Ethernet switch, whether using the onboard NIC (you said SDIO, IIRC?) or an external one - I cannot think of any way in which that’s possible…
… and yet here we are.

So…
So I’d appreciate your thoughts on the likelihood of whatever stopped the USB apparently working until the OS upgrade will recur (an intermittent hardware fault)? If so, it’s worth swapping the unit out. Meanwhile I’m going to arse around replacing that network switch, and doubtless considerable other faffing about, until I can at least work around whatever apparent issue there is on the wired networking side, and preferably reach at least some diagnostic conclusions along the way. I’ll let you know what I find…

I just wanted an item of consumer electronics I could plug in and use with minimal configuration or effort. I didn’t expect to have to be diving into obscure menus to disable bits of the hardware (and rebooting afterwards); neither did I expect to be having to reflash the entire OS. We may reach a point where I decide the game ain’t worth the candle, and drop this until I receive a “Sale!” e-mail from you for the “Vero VIII”, but we’re not there yet so let’s persist a little. If you have any thoughts on the USB and/or networking, I’d appreciate them; otherwise I’ll get back to you when I’ve tried faffing about replacing that gigabit switch.

Cheers,

Tris.

Hi Tris,

Honestly, no.

You can install the version you had before from Download - OSMC to see if there is indeed a regression.

How detailed was your investigation in to whether the USB was actually picked up or not? Did you add a source / check under media? Commands like lsusb would’ve been tricky to run without SSH access.

If it works now, it will stay working, but by all means do keep an eye on it. It will not be a hardware fault. Software and hardware are very different, and it won’t be an intermittent problem.

Did you only test with an external hard drive and if so, how was that drive powered? If it is self-powered, then perhaps when booting it didn’t have enough juice to spin up during boot. You have tested attaching it post-boot, does it show up when you boot the device with it attached?

The USB3 port will give you 0.9A output and if a drive needs more (and some do just to spin up), it can be problematic. The USB2 port will give you 0.5A – not really enough to power a modern spinning drive.

About networking. I had a Virgin Media SuperHub going in to a USG and it kept dropping down from 1Gbps to 100Mbps. I re-ran the cable twice, re-plastered, papered and painted. Still broken. Changed Ethernet reel and did it again. Problem was caused by a dirty port on the SuperHub. Not amused. I’ve also had a cable that’s been buried in the wall for 15 years decide to just fail. I recrimped both ends (having left some slack to mitigate potential wear and tear); no luck. So no – no clue about your network unfortunately. But I am confident USB is not an issue.

I can appreciate that. Sometimes things do go wrong but it’s how we deal with them.

You are welcome to return the device for a full refund and I don’t want you to have something in your hands you can’t use. But I can also appreciate you’d rather get this solved.

Cheers

Sam

Just because I’m curious: What kind of switch (make/model) is it you’re going to replace?
Is it managed or unmanaged?