Skip to content
book.derekmolloy.ie

1.3 The two tiers: microcontroller leaf nodes and Linux gateways

When students first encounter both microcontrollers (such as the ESP32 or STM32) and single-board Linux computers (such as the Raspberry Pi or BeagleBone), a natural question arises: Why do we need both? If a Raspberry Pi 5 has four 64-bit cores running at 2.4 GHz, gigabytes of RAM, a full Linux desktop, and a 40-pin GPIO header, why not use a Raspberry Pi for every sensor node? Conversely, if an ESP32 costs less than €3, boots in milliseconds, and includes Wi-Fi and Bluetooth on the die, why bother putting an Embedded Linux computer in the system at all?

The answer lies in the physical trade-offs of silicon architecture, memory management, power consumption, and real-time scheduling. Almost every non-trivial commercial IoT or industrial deployment resolves these trade-offs by splitting the system into a two-tier edge architecture (Figure 1.5):

  1. Tier 1, constrained microcontroller leaf nodes (Zephyr RTOS): Low-power microcontrollers placed directly at the physical transducer or actuator, executing deterministic real-time control loops, precise sensor sampling, and ultra-low-power sleep management.
  2. Tier 2, high-throughput edge gateways (Embedded Linux; “edge” in the industry sense, a fog node in the device-centric terms of Section 1.4): Application processors managing multi-protocol border routing, local data aggregation, cryptographic TLS/mTLS termination, containerised services (such as MQTT brokers and time-series databases), edge AI acceleration, and local Human-Machine Interfaces (HMIs).
Figure 1.5

The two-tier edge architecture

Click any block or link to see what it does.

Tier 1: leaf nodes (Zephyr RTOS)
Tier 2: edge gateway (Embedded Linux)
Cloud / enterprise

Select a node or a link to see its job and the chapters that cover it.

Figure 1.5. The two-tier edge architecture used throughout this book. Constrained microcontroller leaf nodes handle deterministic sensing, control, and microamp sleep budgets under Zephyr RTOS, while an Embedded Linux gateway aggregates local networks, hosts edge services, and drives a local Qt 6 user interface.

Why Embedded Linux sits in the gateway

Section titled “Why Embedded Linux sits in the gateway”

In 1991, Finnish computer science student Linus Torvalds began developing a free Unix-like operating system kernel on an Intel 80386 PC, using the GNU C Compiler (GCC) and Andrew Tanenbaum’s MINIX as his initial development host2. Released under the GNU General Public License (GPL v2) in 1992, the Linux kernel gained symmetric multiprocessing (SMP) in v2.0 (1996), scaled across enterprise servers and Android smartphones in the 2000s and 2010s, and reached the v6.x series in 2022 (with Linux 6.12 LTS in late 2024 finally merging full PREEMPT_RT real-time preemption into the mainline kernel)3.

Embedded Linux is not a stripped-down fork of Linux; it is the same Linux kernel source (often a vendor or board-support branch carrying extra patches, such as Raspberry Pi’s raspberrypi/linux) compiled for an embedded processor (usually Arm or RISC-V) and paired with a root filesystem tailored to a dedicated device. Look inside almost any commercial smart-home hub, domestic router, automotive infotainment unit, industrial gateway (such as the Kunbus Revolution Pi), or laboratory oscilloscope, and you will find Embedded Linux running underneath.

Embedded Linux dominates the gateway tier for six practical engineering reasons:

  1. A battle-tested networking and security stack: Mainline Linux includes production-grade implementations of IPv4, IPv6, 6LoWPAN, IEEE 802.15.4 (linux-wpan), SocketCAN, BlueZ (Bluetooth), NetworkManager, nftables firewalling, WireGuard VPNs, and user-space cryptographic libraries (OpenSSL, wolfSSL). Writing or validating all of these protocols from scratch on a bare-metal microcontroller is impractical.
  2. Process isolation via a hardware Memory Management Unit (MMU): An application processor (such as the Arm Cortex-A72 or Cortex-A76 on a Raspberry Pi 4 or 5) includes a hardware MMU that provides each Linux process with its own protected virtual address space. If a dashboard GUI crashes with a segmentation fault, the kernel reclaims its memory without disturbing the MQTT broker, the CAN bus logger, or the SSH daemon running alongside it.
  3. Hardware abstraction and open-source drivers: The Linux kernel ships with thousands of mainlined device drivers for USB peripherals, PCIe NVMe drives, Wi-Fi chipsets, cameras (V4L2), industrial sensors (Industrial I/O, iio), and GPIO/bus controllers, bound dynamically at boot through the Devicetree (Chapter 4).
  4. Rich local user interfaces and multi-language ecosystems: Because Embedded Linux provides a full POSIX environment (glibc, pthreads, shared libraries, and Wayland/DRM graphics), you can write high-throughput hardware services in C++ or Rust while running Qt 6 graphical interfaces (Chapter 14), Mosquitto, Node-RED, Python analysis scripts, or Docker containers on the same board.
  5. Remote administration, observability, and fleet updates: A Linux gateway can be configured, debugged, and updated remotely over SSH, inspected via systemd and journalctl (Chapter 2), and managed atomically using dual-root image updaters such as RAUC or Mender (Chapter 20).
  6. Zero per-unit licensing cost: Unlike proprietary commercial operating systems that charge per-seat or per-device royalties, Linux is open source under GPLv2 (though, as we will see in Chapter 19, shipping a product based on GPL software carries legal obligations on source-code disclosure, and the EU Cyber Resilience Act separately requires a Software Bill of Materials).

Why Linux does not belong on the leaf node

Section titled “Why Linux does not belong on the leaf node”

Given all of those advantages, why not run Linux on every node in the network?

The engineering answer comes down to what a full-featured virtual-memory operating system requires from the underlying hardware:

  • Memory footprint and package complexity: Although historical textbooks sometimes stated that a minimal Linux kernel could fit in a few megabytes of RAM, a practical modern 64-bit mainline Linux kernel with a basic systemd user space requires tens of megabytes of RAM at minimum, and typically 512 MB to 8 GB of external LPDDR4 SDRAM when running edge services. High-speed external DRAM cannot easily be embedded inside a low-cost microcontroller package; it requires complex multi-layer PCB routing, length-matched differential traces, and Ball Grid Array (BGA) packages (or System-in-Package modules such as the Octavo OSD335x). By contrast, an ESP32 microcontroller integrates its flash and 320 KB to 520 KB of SRAM inside a single low-cost QFN module that costs under €3.
  • Power budget and boot time: An Embedded Linux gateway such as a Raspberry Pi 4 or 5 draws between 2.5 W and 12 W (500 mA to 2.4 A at 5 V) depending on load, and takes 10 to 25 seconds to boot from bootloader to user space. You cannot power a 3 W computer from a coin cell, nor can you put Linux into a microamp sleep and resume it within a fraction of a millisecond to sample a sensor. A microcontroller can do both, although not at once: an ESP32 in deep sleep draws about 10 µA but takes milliseconds to restart through its bootloader, while light sleep resumes in well under a millisecond at a higher current. A microcontroller running Zephyr boots in milliseconds, draws tens of milliamps while active, and drops to microamps in deep sleep (Chapter 13).
  • Hard real-time determinism: Standard Linux (SCHED_OTHER / EEVDF scheduler) is designed to maximise average throughput and fairness across many competing processes, not to guarantee microsecond worst-case latency on a GPIO pin. When a user-space C++ program on Linux tries to bit-bang a microsecond protocol (such as a DHT22 humidity sensor) or run a 20 kHz motor current loop, its execution can be delayed by hundreds of microseconds or several milliseconds by virtual-memory page faults, TLB misses, cache flushes, kernel interrupt bottom-halves, or other processes. Even with the PREEMPT_RT real-time patch (Chapter 12), a multi-core Cortex-A processor with deep pipelines, out-of-order execution, and multi-level caches exhibits significantly higher worst-case jitter than a microcontroller core with short, predictable interrupt paths. (Even on a microcontroller, code that runs from external flash through a cache, as on the ESP32, has some jitter; time-critical ISRs on the ESP32 are placed in internal RAM with IRAM_ATTR.)
Engineering dimensionTier 1: Microcontroller Leaf Node (ESP32 + Zephyr RTOS)Tier 2: Edge Gateway (Raspberry Pi 4/5 + Embedded Linux)
Processor class32-bit MCU: Xtensa LX6/LX7 or RISC-V (160–240 MHz), or Arm Cortex-M64-bit Application Processor: quad-core Arm Cortex-A72 / Cortex-A76 (1.5–2.4 GHz)
Memory modelSingle physical address space; optional MPU; 320–520 KB SRAM, 4–16 MB FlashHardware MMU; virtual memory per process; 2–16 GB LPDDR4X, microSD / NVMe
Boot timeTens to a few hundred ms from reset to main() (under 50 ms with an optimised bootloader)10–25 s (EEPROM → kernel → systemd)
Power envelope10 µA (deep sleep) to ~100–300 mA (Wi-Fi TX peak); battery or harvesting2.5 W (idle) to 12 W+ (full load); requires mains 5 V 3–5 A USB-C supply
Interrupt latency & jitterSub-microsecond to a few microseconds; deterministicTens to hundreds of microseconds (user space); mitigated via kernel drivers or PREEMPT_RT
Primary software roleHard real-time sampling, closed-loop PID, low-power wireless mesh leafMulti-protocol border router, MQTT broker, time-series storage, Qt 6 GUI, edge AI
Approximate unit cost€2 – €8 (module / dev board)€55 – €90 (board + PSU + storage)

Shared configuration DNA across both tiers

Section titled “Shared configuration DNA across both tiers”

In traditional curricula, microcontroller firmware and Embedded Linux are taught as two unrelated worlds: one using vendor-specific C register macros and proprietary IDE wizards, and the other using Linux shell scripts and system calls.

As you saw at the end of Edge Programming (EEN1097 Chapter 12.5), one of the strongest architectural reasons to pair Zephyr RTOS on the microcontroller tier with Embedded Linux on the gateway tier is that Zephyr deliberately adopted the Linux kernel’s hardware configuration and networking abstractions4. Throughout this book, three shared mechanisms link the two tiers directly:

  1. Devicetree (.dts, .dtsi, and .overlay): Both Linux and Zephyr decouple the physical description of the board (which pins are wired to which I²C bus, SPI chip-select, or status LED) from the C/C++ application and driver source code. In Chapter 4, you will write Devicetree overlays for both the Raspberry Pi and the ESP32 using the exact same node, property, phandle, and compatible-string syntax.
  2. Kconfig (prj.conf, menuconfig, and .config): Both Linux and Zephyr use the Kconfig language to resolve compile-time dependencies and select which kernel subsystems, drivers, and protocol stacks are compiled into the binary image.
  3. POSIX threads and BSD sockets: Both tiers expose portable concurrency primitives and the standard BSD socket API (socket(), bind(), connect(), sendto(), recvfrom()), allowing networking and application protocol logic (HTTP, MQTT, CoAP) to move cleanly between the leaf node and the gateway (Chapters 15 and 16).

Preview: the three canonical operating system models

Section titled “Preview: the three canonical operating system models”

While our hands-on labs run on Zephyr RTOS (Tier 1) and Embedded Linux (Tier 2), Part III of this book (Chapters 11 and 12) places both systems inside a broader three-way architectural taxonomy used across safety-critical and industrial engineering:

  • Single-Address-Space RTOS (Zephyr): Application threads, kernel scheduler, and device drivers are linked into a single firmware binary running in physical memory (with optional hardware Memory Protection Unit regions to isolate thread stacks). Context switches and interrupt dispatch are fast because no virtual address space or page table needs to be swapped.
  • Monolithic Kernel with Virtual Memory (Embedded Linux and PREEMPT_RT): User-space applications run in isolated virtual address spaces and request services via system calls (ioctl, read, write), while all kernel subsystems, filesystems, network stacks, and device drivers execute together in privileged kernel space (EL1 on Arm). Loadable Kernel Modules (LKMs, Chapter 10) extend the kernel at runtime, and PREEMPT_RT converts kernel spinlocks into preemptible mutexes for bounded real-time latency (Chapter 12)3.
  • Microkernel RTOS (QNX Neutrino, architectural case study in Chapter 12): Only the bare minimum (thread scheduling, inter-process message passing via MsgSend/MsgReceive, and interrupt redirection) runs in privileged kernel space. Device drivers, filesystems, and network stacks run as isolated user-space processes. If a CAN bus or graphics driver faults in a microkernel, the OS restarts the driver process without bringing down the kernel, which makes fault isolation easier to argue in a safety case. QNX OS for Safety is certified to ISO 26262 ASIL D and IEC 61508 SIL 3, although certification does not require a microkernel (VxWorks Cert, SAFERTOS and PikeOS are also certified)5.

Figure 1.6 places the three models side by side. Inject a fault in an application or in a driver to see what each model loses, or follow the path an application takes to read a sensor.

Figure 1.6

Three operating system models

Choose a view, or click any box to see what it does.

Single-address-space RTOS
Zephyr (Tier 1)
User mode

Nothing: every thread runs privileged

Privileged (supervisor), one address space
Monolithic kernel, virtual memory
Embedded Linux (Tier 2)
User mode: each in its own address space
Privileged (kernel space, EL1)
Microkernel
QNX Neutrino (Chapter 12)
User mode: each in its own address space
Privileged (microkernel only)

Click a box to see its role. Drivers are drawn in gold: where they run is the main difference between the three models.

Figure 1.6. Three operating system models. Where the device drivers run decides what a driver fault takes down: the whole system in a single-address-space RTOS or a monolithic kernel, one restartable process in a microkernel.

🧩 Knowledge Check

Section titled “🧩 Knowledge Check”
Concept Match

Match the mechanism to its role in the two-tier architecture
Quiz
Select 0/1

A CAN bus driver faults. In which operating system model can the system restart the driver without bringing down the kernel?
Quiz
Select 0/2

Which of the following statements comparing the leaf node and the gateway are correct?

Footnotes

Section titled “Footnotes”

  1. N. R. Augustine, Augustine’s Laws, American Institute of Aeronautics and Astronautics (AIAA), New York, 1983. ↩

  2. L. B. Torvalds, “What would you like to see most in minix?,” comp.os.minix Usenet announcement of the Linux kernel project, 25 Aug. 1991. ↩

  3. T. Gleixner, S. Rostedt, I. Molnár, et al., Real-Time Linux (PREEMPT_RT) Project, Linux Foundation (mainlined in Linux Kernel 6.12 LTS, Nov. 2024). ↩ ↩2

  4. The Zephyr Project, Zephyr RTOS Documentation (v4.x): Architecture, Devicetree, and Kconfig Guide, Linux Foundation, https://docs.zephyrproject.org/. ↩ ↩2

  5. BlackBerry QNX, QNX Neutrino Real-Time Operating System: System Architecture Guide, QNX Software Systems. ↩

© 2026 Derek Molloy, Dublin City University. All rights reserved.