How we deployed a fleet of Raspberry Pi's across Kuwait

Several weeks ago, I took on an exciting and slightly unusual project. A client of mine had been constructing photo-booth kiosks, the old-school type where you’d walk in with a friend, get your photos taken, then collect the printed photos. Demand for the booths came from all sorts of places, from private events to public arenas.

In terms of equipment, the kiosks have several moving parts:

  • A camera to take the photos
  • A display to show the camera preview, countdown, etc.
  • A printer to print the photo strips.
  • A payment terminal to accept payment from users.
  • And finally, a PC to orchestrate all of the above.

Users approach the kiosk and swipe their card on the payment terminal to start a session. A countdown is then presented on the screen, after which a photo is captured. Once all the photos are taken, a couple of identical photo strips are printed for users to keep.

The quality of the printed shots is excellent. My client is a professional photographer and has taken great care in optimizing the camera setup, lighting, and printer selection.

The issue

The kiosk’s self-service capability is powered by an “off-the-shelf” piece of software that runs on Microsoft Windows. While it has been getting the job done so far, it was starting to get in the way as it frequently required an attendant on-site ready to catch the inevitable wrenches Windows will throw into the works. At times, Windows would nag about pending updates, interrupting normal operation. At others, it would start a system update without asking for permission - never mind what you think. Most annoying of all, however, are the times when Windows really really wants to tell you about the latest features it added to Edge and OneDrive.

Suffice it to say the software stack was not optimized for a non-interactive public-facing use case such as a self-service kiosk, and it certainly lagged behind the professional-grade quality of the photography stack. My client was aware of this, but had no choice but to begrudgingly put up with the quirks; they’ve searched far and wide and this was the most capable “off-the-shelf” option available.

Now that demand had picked up, however, my client asked me to optimize their setup. It was holding them back and they were looking to replace it with something that would accelerate their growth instead.

At this point, my client had also gained a deeper understanding of the experience they were offering, taking note of new features they want added, obstacles they want removed, and wrinkles they want ironed out. Armed with a list of a thousand cuts, we put our heads down and went to work.

The solution

  • For hardware, we chose the Raspberry Pi 4 for its balance between cost and capability.
  • nixOS is a Linux distribution that allows us to configure the entire software stack declaratively through a version-controlled configuration file. A single unified configuration means all Pi’s run the same software, from the version of the Linux kernel to the version of curl.
  • Cage is an extremely minimal window manager for Linux. It runs our app in a maximized state on boot. No logins, no menus, no intrusive ads pop-ups.
  • Svelte is a simple, yet powerful UI framework. While I was very familiar with running Svelte on the web (browser), this was the first time I’d run it as a desktop app.
  • Our app needed access to many underlying system APIs (e.g. communicate with a payment terminal over serial, control a USB-connected camera, file system manipulation, etc) that are not available through a browser. Tauri, on the other hand, grants us all of that. And allows us to use svelte!
  • Tailscale ensures we always have SSH access to each device. As long as a Pi is connected to the internet, we are able to access it. This is critical as the kiosks are constantly moved around and are not tied to a fixed location or connection. MagicDNS is the cherry on top.
  • Our (structured) logging stack includes Vector for collection, Clickhouse for storage, and Grafana for visualization.
  • Our CI/CD pipeline relied on EC2 for powerful arm64 machines to build and populate our binary cache, hosted on Cachix. Packer helps us generate our custom SD card images.

The result

The most notable improvements of the solution include:

  • Lower Equipment Cost. The costs of the new setup are only a fraction of the original. The majority of cost savings are attributable to the Raspberry Pi.
  • Reduced Complexity.
    • Before, an attendant needed to complete some steps every time the kiosk PC was powered on (such as logging in, dismissing notifications, starting the photo-booth app, etc). This gets old quickly - imagine if every vending machine had to be started by someone from the vending machine company whenever the power was cycled.
    • None of that is required now. In fact, the new setup does not even require a mouse or keyboard. You just power it on and it’s ready, every single time. This even empowers non-technical staff to set up a kiosk at a new location all on their own. To summarize, the new setup has far fewer moving parts, reducing the likelihood of failure.
  • Effortless new kiosk initialization.
    • Previously, adding a new kiosk into the rotation required some manual setup that involved downloading all the necessary software and drivers, and pairing devices like a printer or payment terminal.
    • Now, a single source of truth (our nixOS configuration) declares all the requirements. A new Pi is bootstrapped with an SD card that makes it production-ready immediately! I’m going to repeat that point because it’s such a game-changer: My client can now hand any staff member a pre-baked SD card, send them to the store to purchase a brand new Raspberry Pi 4, have them boot it up and use it in a production kiosk immediately.
  • In-sync kiosks.
    • Before, configuration across the different kiosks had to be manually kept in sync. For instance, if we wanted to alter the print settings, we would have to manually change the print settings for each kiosk by hand. Even if you only had a handful of kiosks, a trivial settings change would take up an entire morning as the kiosks are naturally located across various geographical areas. Not to mention the guaranteed configuration drift. It was not scaleable.
    • Now, not only do our Raspberry Pi’s inherit an identical configuration, but they are all listening for any updates to that configuration! We now have the ability to update the entire fleet from any PC!
  • Observability.
    • Unless an attendant was on-site, my client had previously lacked the necessary tools to know whether a particular kiosk was operating normally or if it had been suffering from any failures. And when failures did happen, it was impossible to know when, why, or how often they occurred.
    • Now, logs are collected from each Raspberry Pi, preserved, and viewable on a dashboard that can be accessed 24/7 from any phone or PC. Next, we plan to take advantage of our structured logging to implement alerts.

My favorite part of the solution has to be the remote update mechanism. A single release command bumps the version, builds, and then delivers the latest app (and OS) updates to all deployed Pi’s. Those that are powered on will immediately start the update process and will be up-to-date in seconds. The rest will simply pull the latest update the next time they are powered on. I found this to be especially satisfying to run.

Closing thoughts

In hindsight, this was a project that I had been unknowingly preparing for for a while. I only installed nixOS after being frustrated with having to tiptoe around a (comparatively) fragile popOS install. It never crossed my mind that I’d be using nixOS in a professional capacity when I first installed it, let alone that I’d use it to remotely deploy a fleet of Raspberry Pi’s. I think this is why I found this project satisfying to work on. I got to explore using some of my favorite tools in unfamiliar ways and that allowed me to learn more about them in the process.