Kernel 3.14 and metadata_csum in ext4

Hello,

When formatting a filesystem in Vero 4K with:

dd if=/dev/zero of=test.raw bs=1M count=1 seek=200
mke2fs -T big -E lazy_itable_init=0,lazy_journal_init=0 -O metadata_csum,64bit test.raw

mke2fs shows the following message:

Suggestion: Use Linux kernel >= 3.18 for improved stability of the metadata and journal checksum features.

According to the mke2fs source code metadata checksumming wasn’t totally stable before 3.18.

Can someone verify if the fixes for this have been backported to the 3.14.29 kernel in Vero 4K?

Thanks

Hi

Usually users mkfs on other platforms; but I’m keen to backport anything that is missing.
Looking at 3.18, it looks like this could be the commit that ext4 utils refer to:

ext4: enable journal checksum when metadata checksum feature enabled

If metadata checksumming is turned on for the FS, we need to tell the journal to use checksumming too. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts’o <tytso@mit.edu> Cc: stable@vger.kernel.org

diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index 1eda6ab0ef9d..5c11e2153e36 100644
--- a/[fs/ext4/super.c](https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/fs/ext4/super.c?h=linux-3.18.y&amp;id=599a9b77ab289d85c2d5c8607624efbe1f552b0f)
+++ b/[fs/ext4/super.c](https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/fs/ext4/super.c?h=linux-3.18.y&amp;id=98c1a7593fa355fda7f5a5940c8bf5326ca964ba)

@@ -3526,6 +3526,10 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent)

#ifdef CONFIG_EXT4_FS_POSIX_ACL

set_opt(sb, POSIX_ACL);

#endif

+ /* don't forget to enable journal_csum when metadata_csum is enabled. */

+ if (ext4_has_metadata_csum(sb))

+ set_opt(sb, JOURNAL_CHECKSUM);

+

if ((def_mount_opts &amp; EXT4_DEFM_JMODE) == EXT4_DEFM_JMODE_DATA)

set_opt(sb, JOURNAL_DATA);

else if ((def_mount_opts &amp; EXT4_DEFM_JMODE) == EXT4_DEFM_JMODE_ORDERED)

Nothing else particularly sticks out

Sam

Hi Sam, thanks for looking into this.

Usually users mkfs on other platforms

Yes, that’s a way to not see the warning that mkfs shows, but my worries were more about the possibility that using metadata_csum in osmc could corrupt the fs.

Looking at 3.18, it looks like this could be the commit that ext4 utils refer to:

Yes, I had a look and that was the first one I saw too, and indeed it looks interesting to have it.

Since Linux 3.18 was released on 7 Dec 2014, I had a look at all commits done around that date on ext4 and jbd2 and noticed some that were also very interesting.

Here, jbd2: fix regression where we fail to initialize checksum seed when loading is a merge done by Linus on 2014-12-01 (merge for 3.18 of a regression introduced in 3.18-rc3) of a checksum related issue in the journal.

Note that this mail by Theodore Ts’o mentions “the metadata csum v3 patches that went in in 3.18-rc3” requiring an updated e2fsprogs, so I would say the warning by mkfs actually refers to many patches (in plural) and not just one fix in particular.

Reviewing the commits on ext4 and jbd2 mentioned above, I found these that look interesting and probably are worth backporting (if they’re not backported already):

jbd2: fix infinite loop when recovering corrupt journal blocks (2014-08-27)

jbd2: fix descriptor block size handling errors with journal_csum (2014-08-27)
I think this is the important patch. Note that the commit log says:

It turns out that there are some serious problems with the on-disk
format of journal checksum v2.  The foremost is that the function to
calculate descriptor tag size returns sizes that are too big.  This
causes alignment issues on some architectures and is compounded by the
fact that some parts of jbd2 use the structure size (incorrectly) to
determine the presence of a 64bit journal instead of checking the
feature flags.

Therefore, introduce journal checksum v3, which enlarges the
descriptor block tag format to allow for full 32-bit checksums of
journal blocks, fix the journal tag function to return the correct
sizes, and fix the jbd2 recovery code to use feature flags to
determine 64bitness.
[...]

I previously interpreted that mail by Theodore mentioned above as talking about v3 of some patches, but it seems it refers to v3 of the journal checksum format.

jbd2: fix journal checksum feature flag handling (2014-09-11)
jbd2: free bh when descriptor block checksum fails (2014-09-16)
ext4: Replace open coded mdata csum feature to helper function (2014-10-13) This is just some refactoring, but probably useful to ease the backporting of further commits and the commit mentions it also protects from errors [...] which may result attempt to use uninitialized csum machinery' so I’d say it’s interesting to backport.
ext4: check s_chksum_driver when looking for bg csum presence (2014-10-14)
ext4: fix oops when loading block bitmap failed (2014-10-30)
jbd2: use a better hash function for the revoke table (2014-10-30)

Hi @antlarr

Just to let you know that this hasn’t been forgotten.
We are now working on moving to Linux 4.9 which is an LTS kernel. We’ll get those ext4 improvements when we move to this kernel.

I’m not sure there’s much of a reward in us backporting these commits.

Cheers

Sam

Thanks for the update. So is the much-awaited move to Linux 4.9 really happening soon? That would be great.

Yes - it is on the immediate roadmap

Sam

1 Like

Hi, any news about this? Any schedule for when the 4.9 kernel update will be released?

End of the Summer

Sam