I wanted a repeatable, low-cost way to measure real-world viewer latency across different streaming stacks, so I built a little lab that uses Raspberry Pi-controlled relays to simulate viewer interactions and cloud encoders to generate test streams. The goal was simple: measure the end-to-end delay a real viewer experiences when watching a live stream served from a variety of CDNs and encoders, using hardware that’s cheap and easy to reproduce.
Why build a latency lab like this?
Most latency claims from vendors come with caveats — controlled networks, single-path delivery, or custom players. But creators care about the real experience: how long between a live moment on stage and the moment a typical viewer on Wi‑Fi sees it. I wanted measurements that reflect real conditions: cloud encoder variability, CDN hops, player buffer behavior, and last‑mile differences. And I wanted it to be inexpensive so teams without deep infra budgets could validate changes in their stack.
Core idea and components
The lab uses two simple principles:
Key components I used:
Cost outline: each Raspberry Pi ~£35–£60, relay board £8–£15, cloud instance small (~£0.01–£0.05/hr for spot/t3.micro), plus CDN egress as applicable. The whole setup can be built for under £200 if you reuse hardware.
How the test signal works
I needed a deterministic visual event the Pi "viewers" could detect. My approach:
Generating the visual: I scripted FFmpeg on a cloud VM to overlay the current UTC timestamp on the video and to insert a color flash on a specific frame interval. For example, I have a 1‑second region where a specific pixel changes color for a single frame — easy to detect by sampling pixel RGB in the player.
Viewer simulation with Raspberry Pi relays
Each Raspberry Pi runs a lightweight Chromium in kiosk mode pointed to the stream player (HLS or DASH). A small Node.js/Python script captures the video element’s canvas at a high frequency (e.g., 15–30 fps) and checks a known pixel coordinate for the flash or compares the timestamp text using OCR (Tesseract) if you prefer software detection.
When the Pi detects the flash, it logs the timestamp and toggles a relay. The relay toggle is important because it creates an external, hardware-level event you can tie back into other measuring systems — for example, a logic analyzer, a second Pi that acts as a collector, or a USB DAQ device that timestamps changes with higher precision. I also keep a local CSV log (UTC time) of detection events.
Time synchronization and accuracy
Accurate latency needs good time sync. I made sure every Raspberry Pi and the cloud encoder VM synchronize using NTP/Chrony to the same stratum‑1 or 2 servers. On the cloud side, I set up a small script to write the encoder’s frame creation timestamp into the video frame (burned-in timestamp) so the detection doesn’t rely solely on network time. That provides a secondary verification: comparing burned-in timestamp to detection timestamp gives you a measure of system clock offsets.
Typical accuracy I achieved: software detection on Pi gives ±33 ms at 30 fps sampling; with relay + external timestamp capture I could push sub-10 ms consistency for the detection event timestamp, depending on the external hardware.
Measurement workflow
Step-by-step:
What to measure and why it matters
Metrics I track:
These numbers let you answer practical questions: does switching to a low-latency HLS reduce median delay by 2 seconds? Does CDN X have worse P99 than CDN Y for European viewers? Are player buffer caps causing more delay than CDN propagation?
Common pitfalls and my troubleshooting notes
Variants and extensions
Once you have the basic lab, you can expand:
Data analysis and what I usually report
I collect CSV logs and visualize them in Grafana and simple Python notebooks. Key plots I produce:
Practical outcomes from my experiments: small encoder setting changes (keyframe interval reduction, lower GOP) and switching to chunked low-latency HLS shaved 1.2–2.5 seconds off median for European viewers in one test. However, some CDN endpoints showed worse P99 latency — indicating edge routing variability — which only showed up because I ran tests from multiple last-mile networks.
If you want, I can share the scripts I use for the FFmpeg generator, the Raspberry Pi detection agent, and a sample dashboard configuration. I’ve iterated this setup to be resilient and reproducible — it’s the kind of lab that helps you make confident, measurable choices about encoder settings, CDN selection, and player configurations without breaking the bank.
