Tutorial Creating virtual file systems (Unix)

Tutorial Creating virtual file systems (Unix) - Printable Version

+- Sinisterly (https://sinister.li)
+-- Forum: Coding (https://sinister.li/Forum-Coding)
+--- Forum: Coding (https://sinister.li/Forum-Coding--71)
+--- Thread: Tutorial Creating virtual file systems (Unix) (/Thread-Tutorial-Creating-virtual-file-systems-Unix)

Creating virtual file systems (Unix) - Inori - 08-04-2017

A neat thing you can do with Unix-like systems is create a virtual file system from a regular file, mount it as you would a drive partition, and read/write to it like you would any other mounted media. This is great for many things: virtualization with qemu, simple containerized development (thing Docker and Kubernetes but infinitely simpler), exploring new file system types (useful for OS devs and SysAdmins), and much more.

To start, we need to familiarize ourselves with the "dd" command. Wikipedia defines it as a utility for "copying and converting files", which is technically accurate, but it's so much more than that. To use dd, we need to supply it with an input file (if=..., which in our case will be /dev/zero because we don't want to write any actual data to it) and an output file (of=...). Since we're not writing an ISO or anything with a fixed size to it, we can ignore block size (the "bs" parameter) and focus on "count".

The default block size (how many bytes are read at once) for dd is 512, which we'll continue to use here. This is important because "count" determines how many blocks are in the resulting output. If we want a 16 MB file system, we need to convert that size to kilobytes (multiply by 1024), then double it to match the 512 byte block size (multiply by 2, or merge both steps and multiply the initial value by 2048).

Code:
  16 * 1024 * 2
= 16 * 2048
= 32768

Now that we have all our parameters, we can create a file called "test-img" using them.

Code:
dd if=/dev/zero of=test-img count=32768

Great, now we have a nothing-filled file! Before we can actually do anything with it, we need to format it as a file system. Since I'm writing this on Linux, I've elected to use an ext4 file system (see "man 5 fs" for a list) which is the default for Linux. To convert our new file to a file system, we use the "mkfs" command and specify the type with "-t", or we can use the shorthand "mkfs.<type>" command.

Code:
mkfs -t ext4 test-img
# or this
mkfs.ext4 test-img

Cool, now we have a regular file that imitates a usable file system. There are a few more things to cover before we can start using it, though, and first up is loopback devices. Loopback devices are, like our new file, fake block devices, but they allow us to mount other fake/virtual devices as complete file systems. We need to find the first available loop device (they're bound when used, meaning we can't also use the same one), which can be done one of two ways.

We can read the contents of the /proc/mount file and pick the loop device after the last one listed, to start.

Code:
$ cat /proc/mount
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
sys /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
dev /dev devtmpfs rw,nosuid,relatime,size=4017604k,nr_inodes=1004401,mode=755 0 0
run /run tmpfs rw,nosuid,nodev,relatime,mode=755 0 0
efivarfs /sys/firmware/efi/efivars efivarfs rw,nosuid,nodev,noexec,relatime 0 0
/dev/sda5 / ext4 rw,relatime,data=ordered 0 0
...

I don't have any loop devices in use, so I would use /dev/loop0. That method works, but it's somewhat sloppy and terrible for scripting purposes. For a cleaner alternative (though it requires root privileges, but so does the next step), we can use losetup.

Code:
$ sudo losetup -f
/dev/loop0

Finally, we can mount the file system. Create a directory (any name is fine), then as root mount the file system to it using the loop device we identified earlier. This requires root because we need to use the -o option to specify the loop device.

Code:
$ mkdir fs
$ sudo mount -o loop=/dev/loop0 test-img fs

And it's done! To give access to a non-root account, recursively chown it (as root, of course), then we can cd in and do what we please!

Code:
$ sudo chown -R inori fs
$ cd fs && ls
lost+found

Some additional notes:
When unmounting the device (again, as root) we should use the "-d" flag to free the loop device we were using, and specify "-R" to recursively unmount any child devices.

Code:
$ sudo umount -dR fs

Lastly, I wrote a CLI script to automate this if anyone's interested. It's available here, and all you need to do is paste it into a file then make it executable via "chmod +x".