From Commit to the Force: How We Rebuilt Continuous Integration Feedback with Apache Pulsar, Star Wars Motifs and Haptic Fitness Trackers

The Problem: A Galaxy of Silent Builds¶

Here at ShitOps, we ship at warp speed. Every day our 4,200 engineers push more than 38,000 commits into our polyrepo-monorepo hybrid, where our continuous integration fleet — 1,400 Jenkins agents, 320 self-hosted GitLab runners and a fully ephemeral Kubernetes build farm — executes 2.1 million pipeline stages. During a recent observability deep-dive we uncovered a force-breaking issue: nobody notices our build results.

Our telemetry (OpenTelemetry into ClickHouse, naturally) showed a mean-time-to-notice of 22 minutes for failed builds. Desktop notifications were muted, Slack was in Focus Mode, and the giant NOC dashboards are invisible from the espresso machine. Failed builds cascaded into broken main branches, blocked releases and, worst of all, missed sprint goals.

We quickly ruled out the obvious suspects. Our 900-stage YAML pipeline hierarchy is best-in-class, our build cache is warmed by a dedicated GPU cluster, and our self-healing runners hit 99.99% success. Clearly the problem was not the pipeline. The problem was the feedback modality. Vision is a scarce resource, and meetings, code review and coffee consume all of it. So we asked ourselves: which channel is attached to every engineer at all times and scientifically proven to trigger emotional responses?

The answer: haptics.

Introducing CI-Force™: Haptic Feedback at Planetary Scale¶

Today we are thrilled to share the architecture of CI-Force™, our event-driven, cloud-native, AI-augmented haptic feedback platform. CI-Force™ streams every continuous integration event through Apache Pulsar, classifies it with a deep learning model, renders it as a hand-composed Star Wars vibration motif on every engineer's fitness tracker, and simultaneously celebrates it in Discord.

No failing build will ever go unnoticed again.

The Architecture¶

The Event Backbone: Apache Pulsar¶

We benchmarked Kafka, Kinesis, NATS, RabbitMQ and carrier pigeons, but Apache Pulsar was the only messaging system worthy of the Force. Every pipeline stage emits a strongly typed protobuf event onto one of 256 namespaced Pulsar topics. Tiered storage offloads the complete haptic history of the company to S3 with a seven-year retention policy, because compliance will eventually ask how often the Imperial March played in Q3. Our 47 stateless ForceRelay microservices run as Pulsar Functions and are auto-scaled by a custom Kubernetes operator that watches wrist-motion QPS.

JediOS: Custom Firmware for the Fitness Tracker¶

Off-the-shelf firmware obviously could not express the emotional depth of a failed build, so we forked FreeRTOS into JediOS, a hard real-time operating system dedicated entirely to the vibration motor of our corporate-issued fitness tracker. Our audio engineering guild composed polyphonic motifs, then transpiled them with an in-house FFT-to-envelope compiler into motor PWM envelopes:

Feeling the Imperial March crawl across your wrist during standup is, frankly, life-changing.

An IPv6-Only Haptic Mesh¶

Every fitness tracker is a first-class citizen of the internet. We assigned each wearable its own globally routable IPv6 address inside our IPv6-only 6LoWPAN mesh, because IPv4 was exhausted long before this project started. The mesh gateway terminates QUIC with mutual TLS and publishes haptic frames at 50 Hz with a p99 latency of 7.8 ms, measured by a dedicated Prometheus instance that exists solely for wrist latency.

The Holonet: Discord Integration¶

Wrists alone are not enough. Our Discord bot R2-D2C2 posts rich embeds into #build-force-feed: a Yoda GIF for green pipelines, the Imperial March GIF plus a text-to-speech announcement in the on-call voice channel for red ones, and a randomly selected Star Wars quote for flaky tests. On-call engineers can invoke /force-heal to retry a build, which round-trips through Pulsar, a Rust sidecar and our GraphQL federation layer before it reaches the Jenkins API. Total retry latency: 380 ms. Worth it.

DV->>CI: git push origin main
CI->>P: publish build.started
P->>F: consume build.started
F->>B: post A New Build Hope
CI->>P: publish build.finished
P->>F: consume build.finished
alt build successful
    F->>T: vibrate Binary Sunset motif
    F->>B: post Yoda success GIF
else build failed
    F->>T: vibrate Imperial March motif
    F->>B: post Imperial March GIF
end
T-->>DV: haptic Star Wars feedback</div>