Skip to the record

LAVA Test Rig

2026 · Ansible · Docker Compose · LAVA · Mender OTA · Grafana · Loki · Hardware-in-the-loop CI

Provisioning and runtime for the illumibot LAVA test rig: one repository owns the Ansible roles, inventory, and secrets that build it, the docker-compose LAVA stack that runs it, and the Mender OTA pipeline that upgrades 20 physical boards on every release.

20
Devices Under Test
5
Dispatcher Workers
4
DUT Racks
12
Ansible Playbooks
8
Recovery Phases

First look

Rig dashboard: LAVA job state, live logs from Loki, container health, and the camera feed
Rig dashboard: LAVA job state, live logs from Loki, container health, and the camera feed

What I built

Provisioning & Recovery

The LAVA test rig is provisioned and maintained through a combination of Ansible and Docker Compose. Twelve Ansible playbooks provision five dispatcher workers and four colour-coded DUT racks from bare metal, so the bench rebuilds from the repository rather than from memory. Live logs and DUT metrics stream into Grafana and Loki beside an always-on camera preview.

  • Built DUT-originated projector control over both IR and PJLink, reusing the production control libraries rather than reimplementing them.
  • Provisioned the rig through Ansible.
  • Shipped an always-on camera preview so the rig’s physical state is visible from the dashboard.
  • Streamed live logs and DUT metrics into Grafana and Loki.

What came of it

  • Hardware-only failures now reproduce in CI instead of in the field.
  • Traced a phantom ‘IR dongle not present’ from inside the device, a corrupted systemd journal, and a projector locked behind a password that survived repeated factory resets.
  • A failing run can be read without walking over to the bench.

The other plates

Twenty DUT bays in four colour-coded lanes
The rig’s readout, rack-mounted at the bench

Ansible · Docker Compose · LAVA · Mender OTA · Grafana · Loki · Hardware-in-the-loop CI