Upgrade your Raspberry Pi 4 with a NVMe boot drive

You can upgrade the storage in your Raspberry Pi 4 with an NVMe drive. These drives usually plug into a PCIe lane which gives them a potential throughput of over 3000M/s for read and write access.

Yes I can hear you shouting already from Hacker News, Twitter and Reddit, so before we get into the tutorial, let’s start off with a disclaimer:

The Raspberry Pi 4 itself will not be able to achieve its full potential with the NVMe, but it will have two advantages over your SD card — reliability and sheer speed. Buying an NVMe and its associated adapter isn’t much more expensive than an older M2 drive, and you will now have that in your parts box if you need to repurpose it later (see my note on the CM4).

We will also cover UAS and TRIM support in the addendum.

If you have an external USB SSD or M2 drive you can also follow this tutorial.

Parts list:

Image for post
Image for post
Here’s all the parts ready to be assembled

Carefully insert your NVMe drive into the caddy, and connect it to your PC to see if it shows up as expected.

Image for post
Image for post
The internal PCB unit is not secured, so be sure to connect the cable whilst you have it open.

The Samsung EVO 970 Plus is faster than the WD blue drive, but also worked out about 40–50 GBP more expensive. Since our RPi’s USB3 bus will limit the speeds, we don’t need to go too crazy on our spend.

Prepare the Raspberry Pi OS

Download 32-bit RaspiOS

This tutorial “should also work”(tm) for Ubuntu 20.04, but if you run into issues try this tutorial I found on Google.

Flash the OS to an SD card, create a boot.txt file in the first partition and then plug in your RPi and boot it up.

ssh pi@raspberrypi.local

Update your OS:

sudo apt update -qy \
&& sudo apt upgrade -qy

Use the following to check your bootloader version:

sudo vcgencmd bootloader_versionSep  3 2020 13:11:43version c305221a6d7e532693cc7ff57fddfc8649def167 (release)timestamp 1599135103

Go to Advanced Options -> Boot Order -> USB Boot.

Image for post
Image for post
Enabling USB boot

You will see the follow text. This sets the order to start with the SD card, and then to try a USB device if booting fails.

Whilst there, why not go to Performance Options -> GPU Memory and change it from 64MB to 16MB. This gives you a little more RAM back, if you don’t use the RPi to run a graphical desktop.

Image for post
Image for post
Did you know that you can reclaim a little more RAM?

Now shutdown your Raspberry Pi.

On the machine you used to flash the SD card, or on another Raspberry Pi attach the NVMe over USB and the SD card via the SD card writer. I use a dedicated Intel NUC for this purpose with Linux installed.

Run lsblk, you should see your drive appear:

alex@nuc7:~$ lsblkNAME                MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTsdb                   8:16   1  29.7G  0 disk├─sdb1                8:17   1   256M  0 part└─sdb2                8:18   1  29.5G  0 partsdc                   8:32   0 465.8G  0 disk├─sdc1                8:33   0   256M  0 part└─sdc2                8:34   0  29.5G  0 part

Now mirror the SD card to the NVMe, but be really careful with the device names. You don’t want to overwrite your OS on the machine you’re using to do the mirroring operation.

sudo -I
root@nuc7:~# time dd if=/dev/sdb of=/dev/sdc bs=1M

An alternative is to plug the NVMe into the computer you used to flask the SD card, and flash it after finding its device with lsblk

Testing out our new storage

Take out the SD card and power up without the SD card and try to connect:

ssh pi@raspberrypi.local

You will want to resize the disk to fill the full capacity. Use raspi-config and go to “Expand filesystem” then reboot.

Image for post
Image for post
Getting all of our GBs

Install hdparm to test the performance of the disk:

sudo apt install -qy hdparm

Test out the speed of cached reads:

Image for post
Image for post
Nice performance boost over using an SD card!

Try writing out a 500MB file from memory:

pi@raspberrypi:~ $ dd if=/dev/zero of=test bs=1048576 count=500500+0 records in500+0 records out524288000 bytes (524 MB, 500 MiB) copied, 2.63385 s, 199 MB/spi@raspberrypi:~ $

Now when you compare this to a Samsung EVO 970 Plus in my Hex-core Intel NUC, the difference is night and day.

Image for post
Image for post

When you look at the buffered disk reads, clearly the RPi’s USB3 port is holding back the drive:

Interestingly, the SD card gave a seemingly better buffered disk read than the M2 SSD at 43.35 MB/sec.

Fear not, when the CM4 becomes widely available with nice dev boards, it has a single PCIe channel it so could attach the NVMe natively.

Here Dominic Plunkett and Eben Upton discuss the CM4 and the various I/O options.


Addendum — 29th Nov 2020

What about UASP you say? UASP stands for USB Attached SCSI and according to StarTech can enable better throughput over USB and decrease CPU usage. It just happens to be enabled on the hard-drive and enclosure combination I’ve picked out for you in the bill of materials.

If you already have a drive connected, try lsusb -t to see whether it’s enabled under the driver line.

Image for post
Image for post
Best of both worlds — TRIM and UAS

And what about TRIM support? As you can see from above, the drive and the enclosure picked both support TRIM on a Raspberry Pi 4 under Raspberry Pi OS. For an external drive, you sometimes have to enable TRIM explicitly by setting the provisioningmode. The Arch Linux wiki has an excellent article on how to do this under “Solid state drives”.

So you have the best of both worlds: UAS, TRIM and the extra potential of NVMe for random 4k reads and sustained throughput.

Wrapping up

It’s now over to you to enjoy your new more reliable and speedier storage.

Are you thinking of putting your Raspberry Pi on the Internet, or wanting a way to access it remotely? Checkout the inlets project for an unlimited open source tunnel that you can host yourself.

Perhaps you’d like to play around with some cool tech like Kubernetes or OpenFaaS?

What if buying NVMe storage is too expensive for your cluster?

You can netboot your Rasbperry Pi cluster with my step-by-step guide for RPi3 and RPi4. Free for all subscribers to the Treasure Trove.

You’ll find a bill of materials, and instructions that work with Kubernetes too.

You may also like

Build a Kubernetes cluster with a single node, or multiple hosts. Learn to manage containers and monitor your systems.

Build and deploy containers with metrics, queueing and a UI built-in. No Kubernetes or Docker is required for this and it can run on a single host.

CNCF Ambassador. OpenFaaS & Inlets founder — https://www.alexellis.io

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store