sargx digital garden

Filesystem

5 min read

SquashFS

https://blog.carlosgaldino.com/writing-a-file-system-from-scratch-in-rust.html

Boot Images

There are several different kinds of boot images that combine a boot loader, kernel, file system, and configuration into a single file. More popular flavors of these includes the legacy U-Boot image format, the FDT/DTB U-Boot image format and the TRX image format.

These boot images include the meta data required to extract the parts of the system, such as the offsets, size, compression, and type of entry. You can think of it as a sort of Master Boot Record or partition table that lives on an HDD.

U-Boot is open source and therefore its file formats are very well known. You can install uboot-utils package in Ubuntu/Debian and then use the mkimage tool to generate or modify existing u-boot images. You can also generate a FDT/DTB U-Boot image with any FDT/DTB tools available. Note: FDT is a flattened device tree and DTB is a device tree binary. These are the same, but essentially are a description language that is used by the kernel to describe the layout and configurations of systems. Often there is a kernel version of the device tree and a boot loader version of a device tree. The boot loader version can include the whole kernel and root filesystem.

Root/Init File System Formats

There are many root file system formats. Depending on the required functionality of the root filesystem, you’ll likely see different format selections. For example, if the root filesystem is intended to be writable, you’ll likely see it in a UBIFS, JFFS2, or YAFFS2 file system format. For read-only functionality, cpio, cramfs, and squashfs are popular options.

Read/Write File Systems

Embedded systems often store their non-volatile data on flash. Since flash wears out, there is a conceptual need for wear leveling. This means that instead of reusing the same flash erase block over and over, the file system deliberately spreads its writes over the entire flash chip to prevent favoring any one erase block and immaturely wearing it out.

  • JFFS/JFFS2 - Journaling Flash Filesystem is readable and writable. It works on NOR and NAND flash but requires a minimum of 5 free erase blocks to work. For example, if you had 128K erase block sizes, you’re JFFS2 partition could be no less than 640K. Note: JFFS2 is no longer maintained, but it is supported in the Linux Kernel. While it does work well, when you don’t have those spare erase blocks, disk space leakage may occur (i.e. when you delete a file, you can’t regain the space because JFFS2 doesn’t have the space to safely defragment the data across erase blocks safely.) JFFS2 data is primarily broken into two areas, inode metadata and file data. The former is completely loaded into memory when the file system is mounted to keep performance up. You manage jffs2 images with jffs2dump, jffs2reader, and mkfs.jffs2 from the mtd-utils package.

  • YAFFS - Yet Another Flash File System is designed to work with NAND and NOR flash devices. This file system is very similar to JFFS2 except that because it doesn’t load everything into memory at once, it has lower memory requirements than JFFS2. YAFFS arguably has a cleaner and more performant garbage collector (so says its author). You manage yaffs2 with tools from aleph1.co.uk, mkyaffs2image. It doesn’t appear that Ubuntu has official support, but you can extract yaffs2 images with unyaffs from the unyaffs package.

  • UBIFS - The spiritual successor to JFFS2 is the Unsorted Block Image File System (UBIFS). I think of UBIFS as the LVM of the flash world because it can unify and span multiple flash chips. When you create a UBIFS image, you are just creating the file system content. It then needs to be converted into a UBI image which is what gets written to actual flash.

Read-Only File Systems

Embedded systems that don’t have a need to support writable root file systems or any file system can use more space efficient or simple file system formats. Read-only file systems don’t have to worry about wear leveling because they only get erased and written during firmware updates.

  • SquashFS - SquashFS is a very commonly used read-only file system. This is because it has a bunch of flexibility with regards to how it manages compression and erase block sizes. It also supports Linux extended attributes and LZMA compression. In my experience, this is the most common read-only file system I’ve seen that isn’t a kernel CPIO archive.

  • Cramfs - Cramfs is much older than SquashFS and has a lot less flexibility. That said, its simplicity allowed it to have upstream kernel support as early as kernel 2.4.0 (it actually showed up in the 2.3 dev release cycle). Here is a commonly used comparison chart from the net:

															Squashfs    Cramfs
 
Max filesystem size:          2^64        256 MiB
Max file size:                ~ 2 TiB     16 MiB
Max files:                    unlimited   unlimited
Max directories:              unlimited   unlimited
Max entries per directory:    unlimited   unlimited
Max block size:               1 MiB       4 KiB
Metadata compression:         yes         no
Directory indexes:            yes         no
Sparse file support:          yes         no
Tail-end packing (fragments): yes         no
Exportable (NFS etc.):        yes         no
Hard link support:            yes         no
"." and ".." in readdir:      yes         no
Real inode numbers:           yes         no
32-bit uids/gids:             yes         no
File creation time:           yes         no
Xattr support:                yes         no
ACL support:                  no          no
  • CPIO - This is a general file archive format. Every 2.6 kernel has a cpio archive included in its binary. If the configuration doesn’t require the archive, its empty but still there. When the CPIO archive is used during a kernel boot, the kernel mounts an in-memory ramfs and extracts the CPIO archive there for use.

🌱 Back to Garden

1 item under this folder.