Thanks, solved
1 Like
Got a new drive today and about to transfer the files over so i can run this stress test on the other drive, then it occurred to me would it be wise to test the new drive first then transfer the files over
1 Like
Good to see you got this worked out.
I have a dedicated 8TB drive just for archived REMUX files.
So if anything goes wrong and I have a full drive failure, atleast I have all my full REMUX files on an offline disk.
I have been testing the resiliency of the file system with a test drive.
I have found some rather interesting results when the drives are removed without ejecting/unmounting.
In one of my test I transfered ov er about a dozen files totalling 100GB.
After the transfer was complete, I removed the disk without unmounting it.
Then when I put it back on, the files were not there, as if they were never written to the disk.
Another test I did was I created a new folder and moved file (1) into the folder and deleted file (2).
After the task was complete, it didnāt take long, I unplugged the drive.
Then when I plugged it back in to check the files it was as if I didnāt do anything at all.
The folder I created was not there and file (1) and (2) were still in the root directory as if i never moved file (1) or deleted file (2).
I still have some reading up to do on how the ext4 file system works under the hood to better understand why I see such behaviour.
My guess is that those commands to move and delete were never actualy performed on the files but rather just the metadata so it looks done to the user but itās still pending behind the scenes.
But I donāt know that for sure.
Well if resilience is your goal can also take a look at btrfs
Moving and deleting on the same disk are done only in the file table on any kind of partition format that I am aware of (why move the actual data if you donāt have to). If you are going to follow these experiments you might look at the behavior with the cache disabled and see if that changes your results. It is possible that you pulled the plug before the operating system bothered flushing the cache.
The other thing you could look at is if the journal is actually enabled. I havenāt looked too deep into how the ext systems works yet but I know the journal can be disabled and I donāt really understand how in your file transfer test none of the files showed up. Maybe the process of rebuilding the file table from the journal entries is not automated like it is with ntfs in Windows?
Thanks for the link, that was a great read.
Explains alot, except for the file I deleted not actually getting deleted, unless the deletes are delayed too, which I guess weāll assume they are.
Honestly, I may just reformat back to NTFS.
I never had a single problem with NTFS.
The ext4 filesystem has already let me down and when it comes to data loss, I donāt know if I can live with that risk anymore.
Even though I have backups, ext4 has been consuming too much of my time.
I feel like I have to wear kiddy gloves and hold itās hands.
Iām jumping through hoops to make it work, just because itās the recommended file system.
I spent days, literally, and even purchased software to help with traansfer speeds from Windows.
For those who have ext4 and like it, Iām glad it works for you, but for me I will not be using it anymore moving forward.
My next HDD will just stay NTFS.
No more special command lines, no more third party software, no more lost data, just plug and play; drag and drop!