With the base of the system set up, the next component to add to your Raspberry Pi to get it off the ground as a file server is storage. As stated above, this tutorial follows the following setup: an array of hard drives (which do not need to be from the same manufacturer or of the same size; formatted in this example as ext4) inside an enclosure connected to the Raspberry Pi via USB 3. In this section you will merge the independent file systems so as to make your Raspberry Pi show a continuous file system (thus, for instance, four 8 TB hard drives would appear as a single 32 TB file system, making it much easier to point a media server to a single directory instead of to multiple).

User permissions

Before you can even start with the process of merging file systems, it is a good idea to create two users and one group on the system that serve a singular purpose: to own the files on the hard drives. Without going too far into file system permissions, the purpose of this is to create one user who has read and write access to the files and another user who can only read them. These two users will belong to the same group. This will allow you to grant read-write permissions to certain applications or when you specifically request them while defaulting to read-only permissions if you do not want them to be able to change anything. This, for instance, can protect against data loss if a program, such as Jellyfin, which will be installed below, attempts to delete user data, due to either user or system error. Regardless of who is at fault for a specific error, if a program does not have read-write access to a directory, it cannot modify it.

Using the groupadd command, create the group storage with the specific GID (--gid; group identification number) 200 as a system (--system) group. Having a preset GID comes in handy when granting specific programs (such as Jellyfin) the permissions of a certain group. A system group is not inherently different from a normal group but is conventionally used for technical processes, not users. For instance, no human user will belong to the group storage, as it is solely to resolve or avoid permission issues, whereas a group called family might allow multiple users on a Linux computer to share photos with one another in a single directory, with each user being able to add, modify or delete files therein.

pi@repi:~$ sudo groupadd --system --gid 200 storage

Similarly, now create two users belonging to the group storage: storagerw (rw for read-write permissions; this one will be the owner of your shared directory) and storagero (ro for read-only permissions; this one will belong to the group storage and thus be given permission to read (thus access) files but not modify or delete them). Using the useradd command, for the same reasons as above, specify the UID (--uid; user identification number) and GID (--gid; so that both accounts belong within the group storage with a GID of 200) and set both accounts as system accounts (--system). Note that user storagerw is given the UID 200, while user storagero is 201. This should be somewhat memorable: the UID that matches the GID has read-write permissions, while the UID that is slightly different (i.e. a single number off) can read but not create, modify or delete the data.

pi@repi:~$ sudo useradd --system --uid 200 --gid 200 storagerw && sudo useradd --system --uid 201 --gid 200 storagero

So that no one attempts to log in using these user accounts (potentially to gain access to the system), lock the accounts with the usermod command and the --lock option (repeated for both users).

pi@repi:~$ sudo usermod --lock storagerw && sudo usermod --lock storagero

To confirm that the new user accounts have been properly added to the group storage and to confirm their UID, use the id command followed by the username (repeated for both users).

pi@repi:~$ id storagerw && id storagero

To verify that both accounts are system accounts (and thus not intended for human use), if you list the directories in /home, you will note that only user pi is given a /home directory. For the ls (list) command, note that I prefer adding the options for horizontally (-l) listing all (--all; thus including hidden files) directories and files with human-readable (--human-readable; thus in kilobytes, megabytes, etc., not bytes) file sizes.

pi@repi:~$ ls -l --all --human-readable /home

mergerfs

Following the Perfect Media Server 2017 build guide (along with its accompanying YouTube video and general information on mergerfs; from which this section is very heavily inspired, including many commands), install mergerfs, a union file system that allows for individual block storage devices (e.g. hard drives) to appear as a single device (i.e. in a single mount directory) to the operating system and programs. As stated in Storage, this allows, for instance, for four individual drives, including those of different sizes and from different manufacturers, to be mounted in a single directory. Thus, instead of needing to add four different paths to a media server (i.e. one for each hard drive), you can add a single path which combines all the hard drives into one directory. It should be noted that you can still access each drive independently, as they each still maintain their own separate mount point.

Now, after confirming that your enclosure with hard drives installed is physically connected to the Raspberry Pi, turn on the enclosure. If you run lsblk, you will notice that your individual drives are not listed. This is exactly as it should be because the devices have not been given a mount point. To confirm that your Raspberry Pi sees the hard drives, you should instead run blkid (block device identification) with elevated privileges.

pi@repi:~$ sudo blkid

Here you are shown all block devices connected to your Raspberry Pi first identified by their location as a /dev (physical device) directory followed by information such as their label, UUID (an ID that identifies a specific block device and will probabilistically never be used for another block device), type (i.e. file system), etc. The devices that represent the attached hard drives with be listed as e.g. /dev/sda1 to /dev/sdd1 for four devices with a single partition each. I know that my enclosure lists the hard drives from top to bottom, i.e. with the topmost hard drive as /dev/sda1 and the bottommost as /dev/sdd1. If you are also so lucky as to have standardized scheme, it may help you to physically group your hard drives so that you know which data are held in a specific area (e.g. with the top three hard drives being the only ones containing movies so that you know that all movies are in /dev/sda1 to /dev/sdc1). You should now note the UUID of each hard drive, as these will be used below.

Before beginning, you must install mergerfs and fuse via apt.

pi@repi:~$ sudo apt install --yes mergerfs fuse

Now it is time to create mount points for all the hard drives. To do this, we use the mkdir (make directory) command with elevated privileges in the /mnt (mount) directory to create directories for each individual hard drive and a directory for all hard drives merged together. Using a feature of the Bash shell (the command line interpreter into which you have been typing) called brace expansion, you can create multiple directories at once so long as the unique parts are entered within curly brackets and separated with commas. For this example, I am creating 11 directories with 10 representing the content stored on each of the hard drives and 1 representing the combined contents of all others via mergerfs: /mnt/diskMovies1 to /mnt/diskMovies5, /mnt/diskSeries1 to /mnt/diskSeries5 and /mnt/storage. Note that as my hard drives only hold media (backups of my extensive DVD and Blu-Ray collection, which is legal in my jurisdiction), I am labeling the hard drive locations according to the type of media they are holding (i.e. movies and series). (Note also that my files are structured according to Jellyfin standards for movies and shows.) You should of course modify this to meet your individual needs (e.g. if you are hosting family photos and documents, /mnt/diskPhoto and /mnt/diskDoc). (The processes of labeling the directories with trailing numbers and creating more directories than I have hard drives give me flexibility were I to combine two hard drives into a single one with a larger capacity or add more drives in a larger enclosure.)

pi@repi:~$ sudo mkdir /mnt/{disk{Movies,Series}{1,2,3,4,5},storage}

From here, it is time to automatically mount the hard drives when the Raspberry Pi boots up and merge the contents into one directory, namely /mnt/storage. To do this, you need to add entries to fstab (file systems table; at /etc/fstab). The entries include the UUID noted earlier, mount location (e.g. /mnt/diskMovies1), file system type (which is assumed to be ext4), options, dump and pass. Add the following information (of course with your individual UUIDs replacing longstring) to /etc/fstab with the tee command as in Static IP. The final line is the actual mergerfs setup. The first column groups all directories that begin with /mnt/disk (using the globbing wildcard asterisk to signify anything or nothing following) and mounts them together in /mnt/storage (second column) as the file system fuse.mergerfs (third column). The fourth column lists options following the default setup in the mergerfs documentation with certain changes: first, option cache.files=off is removed, as it is not recognized via the standard upstream installation with apt; second mergerfs is now path preserving (i.e. if a directory already exists, a newly created file will go into the directory on the same hard drive instead of making a new directory on a different hard drive); and third it has been given the name mergerfs when listed using df --human-readable (disk free space in human-readable terms). Finally, dump and pass are both left as 0.

pi@repi:~$ sudo tee --append /etc/fstab << END

# Mount points for individual hard drives for use with mergerfs at /mnt/disk{Movies,Series}{1,2,3,4,5}
UUID=longstring /mnt/diskMovies1 ext4 defaults 0 0
UUID=longstring /mnt/diskMovies2 ext4 defaults 0 0
UUID=longstring /mnt/diskMovies3 ext4 defaults 0 0
UUID=longstring /mnt/diskSeries1 ext4 defaults 0 0

# mergerfs mount point at /mnt/storage
/mnt/disk* /mnt/storage fuse.mergerfs allow_other,use_ino,dropcacheonclose=true,category.create=epmfs,fsname=mergerfs 0 0
END

You can confirm that /etc/fstab has been correctly edited using the cat command.

pi@repi:~$ cat /etc/fstab

Now to mount all devices according to the entries in /etc/fstab (i.e. mount the drives we just defined at their newly defined mount points), use the mount command with the --all option with elevated privileges.

pi@repi:~$ sudo mount --all

To verify that all directories have been properly mounted, list all directories in /mnt.

pi@repi:~$ ls -l --all --human-readable /mnt

To view the new combined directory of /mnt/storage, list all directories in /mnt/storage.

pi@repi:~$ ls -l --all --human-readable /mnt/storage

Now that you can access all directories and files in the merged directory of /mnt/storage, it is time to use the chown (change owner) command with the --recursive option and elevated privileges to define user storagerw and group storage as the directory’s owner. Keeping the default permissions (drwxr-xr-x, shown when using ls -l --all --human-readable /mnt and looking at the entry for /mnt/storage), this means that user storagerw will have read-write (and execute) privileges, while anyone belonging to group storage (such as user storagero) will (only) be able to read (and execute) from the directory but neither modify, delete nor create files or directories within it. For this command, you can either specify the UID and GID (i.e. 200:200) or the user’s and group’s name (storagerw:storage). Note that depending on how many directories and files are stored within /mnt/storage, this may take a few minutes to complete (as it needs to change the permissions for every individual directory and file).

pi@repi:~$ sudo chown --recursive storagerw:storage /mnt/storage

Once you are returned to a blank command line, you will see the changes to the ownership of /mnt/storage by listing all directories in /mnt. Additionally, you will notice that all directories that merge into /mnt/storage also are now owned by user storagerw and group storage.

pi@repi:~$ ls -l --all --human-readable /mnt

Due to the change in permissions, user pi (the one which you are using) can read but not modify, delete or create files within /mnt/storage. This is not a problem, as you can always elevate your privileges with sudo to be able to have read-write permissions or grant yourself permission by specifying which user to use for a certain application (this functionality is called force user and force group in Samba parlance and PUID and PGID for Docker containers from LinuxServer.io).