I believe that it would be nice if we have an option to create image of the sd card on external usb drive.
Lets say, ones per week during off hours, osmc could reboot, create image on external usb, and boot the osmc again.
If sd card ever breaks, we will have a very fast option for restore everything.
You could craft your own solution - perhaps use dd, after shutting the media centre down.
Derek
You cannot image a card that is currently mounted as the running operating system.
Interesting - I feel that I’ve done it in the past (but wouldn’t do it as normal practise)
Derek
We should be able to boot to let’s say recovery, and from there dd to the external USB drive.
If the “recovery” partition is at the end of the drive it would be beneficial as if the image is written to a slightly smaller sd card then the recovery partition would be truncated allowing the card to still boot and run OSMC
What is recovery though?
In theory, you can pipe wget to dd directly and erase the SD that is mounted, but you will probably see some ugly errors in the process.
Sam
When I said recovery, I meant the minimal os.
As we have two partitions on sd card, my idea is that the first partition should provide elementary os functions (additional to the all current functions) such as access to the usb drives and dd linux command.
I assume that the first sd partition would change rarely, so from time to time it could
be backed up while osmc is on. The second partition could be backed up when
minimal os from the first partition is on.
Backing up a raspberry pi is quit time demanding to me.
Usually, the procedure is as the following:
Turn off RPI, take out the cart and put it in the laptop
On the laptop do the backup
Turn on RPI
Copy image file from laptop to the RPI samba share (I need this in order to be able to mount the image on RPI in case I need something)
I am looking for a way to simplify the procedure.
I currently backup my live FreeBSD system partition through dd. Can this not translate to Linux?
dd is a block level transaction. Provided you dd back to a disk of at least the same size, that will work.
Sam
Thanks Sam.
Do we need to install any packages to run the dd command?
No, it’s part of the standard utilities that OSMC has
Sam
By live, I hope you don’t mean you are using dd to back up an in-use file system that is mounted read/write ?
If so please be aware that this will always result in an inconsistent backup which will at the least have some file system errors or at worse will contain serious corruption.
I just use cp -a to make backups of my Raspberry Pi installs and to copy them between cards. I have been doing this on another machine running linux but a quick test just now shows the backup seems to work even from a mounted system drive, just using a running OSMC. Another advantage over dd is you back up only the files, not the empty space.
For this method, you need a backup drive (partition) formatted with an extX filesystem, so that permissions are preserved. There may be an alternative of writing the files to a tar archive (on any filesystem) but I haven’t tried that.
I’m going to assume it’s a removeable drive, so mounted at /media/backup (say). If you back up to a network, modify to suit. Then as root:
cd /media
mkdir boot
mkdir system
mount /dev/mmcblk0p1 boot
mount /dev/mmcblk0p2 system
cd backup
mkdir OSMCbackup
cd …
cp -a boot backup/OSMCbackup/
cp -a system backup/OSMCbackup/
umount boot
umount system
rmdir boot (optional - could just leave it there)
rmdir system (optional)
These commands can be put in a shell script and run with cron daily or whatever.
In order to be able to re-construct OSMC on a re-formatted card, make a note of the partition size of the boot partition (currently 244MiB but I don’t think that’s a magic number, the size should be a multiple of 4MiB). For restoring, make partitions with gparted or your weapon of choice. The OSMC install image starts the FAT32 boot partition at sector 2048 (1MiB) but there are good reasons for starting it at 4MiB. It does not seem necessary to preserve the UUIDs of either partition as you would if using a boot manager. The FAT partition must be flagged as boot [edit: no, see below]. The linux partition takes up all the available space after that.
Mount the formatted card and just copy (cp -a) all the files back into the two partitions.
On a Pi, the ‘boot’ flag does not need to be set. The Pi will consult a vfat on mmcblk0p1 regardless. We don’t actually set that for OSMC on Pi
I’m not completely convinced. I’m not sure the argument re. erase block alignment is valid anymore. SD cards have virtual addresses which are mapped to physical addresses by the controller. Can you provide some stats to show evidence of this (dd speeds would be good)?
I’m not entirely convinced, either, but it does make some sense for FAT partitions to make sure the FATs themselves are contained within whole Allocation Units which do not also contain normal files. The SD Association formatter does this. It appears SD card controllers use erase-write algorithms more suited to the small writes on a FAT table either by assuming the first AU is a FAT or by adapting to the write patterns. It would be nice to think they could use similar techniques on an ext journal. Up-to-date information on this is sparse (and I assume commercially sensitive) but for the loss of only 3MiB, I’m playing safe by aligning everything to 4MiB boundaries and making sure the journals are at the beginning of the partition.
Benchmarking with dd or other tools is unlikely to give convincing results as so much depends on the previous history of the card, it seems. My concern is not for speed but for wear.
FWIW, the Raspbian images start boot at 4MiB.
After reading the following article:
http://3gfp.com/wp/2014/07/formatting-sd-cards-for-speed-and-lifetime/
I’ve come to the conclusion that optimising the start location of the FAT32 partition on OSMC doesn’t really gain us anything. Why ?
Because the FAT partition is rarely written to. The optimisation that the controller in the cards makes is to optimise the write performance of the second 4MB block on the card under the assumption that the FAT table will be stored there, because the FAT table is the most frequently updated part of a FAT partition as all metadata is stored there.
However on OSMC during normal operation the only time the FAT partition is even written (apart from the initial install) is during a kernel update, or if you are editing config.txt. All the other writes made to the disk for normal operation are on the ext4 partition.
Even if we could position the ext4 partition to start 4MB into the disk (which we can’t because the FAT partition has to come first and is 256MB in size) it still wouldn’t help, because ext4 has far more distributed storage of metadata - there are many copies of the superblock spread out over the disk, and a lot of the file metadata that would be stored in one clump in the FAT table on FAT32 are stored in file “extents” spread all over the disk.
So unfortunately as far as I can see this 4MB optimisation is only applicable to FAT32 and not at all applicable to ext4, and not useful when we must have a FAT32 partition before the ext4 partition that has the vast majority of the systems data.
Pretty much as I thought, thanks. I read Harvey’s piece (his procedure seems unnecessary, since AFAICT the SD Association’s formatter does exactly what he recommends) and a load of others while researching for setting up an SD card with just one ext4 partition. I concluded that although mis-alignment to (4MiB) allocation units is nowhere near as bad as partitioning to cylinder boundaries it was still worth doing for my case. As you say, any special treatment of the second AU can’t be made use of with the FAT in the way, but if the journals are at the beginning of the partition, at the very least you may be using fewer AUs for this activity if you don’t start it with part of an AU.
It’s a fine point, and I haven’t re-formatted my OSMC card since researching this but I would if setting up a new one.
I am just booting from the sd card to a usb memory stick. Do the sd card contents change at all after setup in this case? If they don’t , then one backup of the sd card is all I need. Then can I use your method on the usb memory stick?
I don’t know how OSMC configures itself with a USB stick but I suppose it puts just the ext4 partition on the stick instead of the card and the fat partition on the card is the same in either case. That fat partition gets updated when you do an upgrade, but I don’t think it changes day-by-day (see @DBMandrake’s advice above).
I don’t see why the method above shouldn’t work just as well. You would have to “mount /dev/sdb1 system” instead of mmcblk0p1, assuming your stick is called sdb.
HTH
