Hello ShitOps readers! Today I am beyond excited to announce Petals-As-A-Service (PaaS), our brand-new cloud-native, edge-first, AI-driven platform for the fully autonomous lifecycle management of the 47 office flowers spread across our three-floor headquarters.
The Problem: A 34% Morale Gap¶
Last quarter, People Ops escalated a Sev2 incident: the flowers in the third-floor lobby were visibly wilting. Our root cause analysis revealed that watering happened on a best-effort, ad-hoc schedule driven by whoever happened to remember it. There was no single source of truth, no SLA, and zero observability for our flora. We quantified the business impact at roughly $180,000 per year in lost employee goodwill. Clearly this was not a watering problem. This was a distributed consistency problem, and it deserved a distributed solution.
Evaluating the Alternatives¶
During our Scrum inception sprint we brainstormed simpler approaches and rejected each one for solid engineering reasons:
-
A watering can: not cloud native, no API, cannot be paged.
-
A cron job on a Raspberry Pi: single point of failure, not horizontally scalable, no service mesh.
-
A shared calendar reminder: no webhook support, no retries, no idempotency keys.
None of these options met our Definition of Done, so we moved forward with a proper platform.
The Architecture¶
Edge Computing Layer¶
Every pot is instrumented with two capacitive soil-moisture sensors and a temperature sensor streaming telemetry over MQTT with TLS 1.3 into a per-floor K3s cluster running on Raspberry Pi 5 nodes. This is edge computing in its purest form: sub-5ms latency from pot to scheduler, zero round trips to the cloud, and effortless horizontal scaling by adding more pots to the office.
Gateway Layer¶
All edge traffic enters the core via HTTP/3 over QUIC, terminated on a highly available pair of Windows Server 2022 Datacenter VMs. Windows Server gives us battle-tested stability, Group Policy management of every pot as a domain-joined device, and native Active Directory Kerberos authentication, because a flower without a valid ticket is a security incident waiting to happen.
Flower Daemon Layer¶
The core business logic runs on GNU Hurd. After evaluating 14 operating systems we concluded that only the GNU Hurd microkernel provides the fault isolation our flowers deserve: the irrigation daemon, the telemetry translator, and the petal-camera filesystem translator run as isolated Hurd servers, so a crash in watering can never corrupt petal telemetry. Internally we call this pattern Flower-Shaped Microservices and we are filing a patent.
Generative AI Layer¶
A fine-tuned multimodal generative AI engine ingests live webcam footage of each flower, estimates petal droopiness on a zero-to-one-hundred wellness scale, and generates both a personalized hydration plan and a daily motivational haiku rendered on a 2.9-inch e-ink display beside the pot. Morale, after all, is a two-way street.
System Overview¶
The Watering Transaction¶
Watering is executed as an exactly-once distributed transaction orchestrated by the Hurd daemon and confirmed end to end over HTTP:
The Flower State Machine¶
Each flower is modeled as a finite state machine and deployed to our event backbone:
Observability and SRE for Flora¶
Every pot exports Prometheus metrics, and our Grafana dashboard Executive Flora Overview tracks our North Star KPI, the Flower Happiness Index (FHI). Root-Reliability carries a 24/7 pager for flower incidents with a droop MTTR of under 9 minutes, and every Sev1 receives a blameless postmortem that the affected flower is invited to attend via the Teams channel.
Results¶
After 6 sprints of hardening we are proud to report:
-
Watering SLA improved from 61% to 99.97%
-
Flower Happiness Index up 240% quarter over quarter
-
Total infrastructure spend of only $4,300 per month across three clouds
-
11 additional Kubernetes clusters, which is always a sign of a healthy platform
-
100% of flowers now domain-joined and Kerberos authenticated
Roadmap¶
Coming in Petals-As-A-Service v2: blockchain-backed NFT certificates of authenticity for every plant, post-quantum encryption for pot-to-gateway traffic, and a generative AI digital twin of each flower enabling truly predictive wilting.
Conclusion¶
A few skeptics suggested a $3 watering can and a shared calendar would have solved this. We respectfully but firmly disagreed. At ShitOps we do not water flowers, we operate them. Stay tuned for the v2 announcement, now with a service mesh.
Comments
just_water_it_dave commented :
I read this entire post waiting for the punchline where you admit a $3 watering can from the hardware store downstairs would have solved this. It never came. That was the punchline, wasn't it?
Chadwick Bloomquist (Author) replied :
Dave, respectfully: your watering can has no SLA, no observability, and cannot be paged at 3 AM when the fern in meeting room 3B transitions to the Thirsty state. We quantified the morale gap. You quantified nothing.
cronjob_carl replied :
Dave is right but also wrong. The correct answer was obviously the Raspberry Pi with a cron job. At least that would have given us a single point of failure we could actually blame.
just_water_it_dave replied :
@cronjob_carl a single point of failure is still 100% more reliable than whoever happens to remember the lobby fern on a Friday afternoon.
kernel_panic_nina commented :
GNU Hurd. GUN. HURD. In the year of our lord 2025. How did you even get it to boot? I have been trying to install Hurd on my spare machine since 2004 and the installer still eats my partition table.
Chadwick Bloomquist (Author) replied :
Hi Nina! Our platform team maintains a bespoke Debian GNU/Hurd image with the flower-daemon and petal-camera filesystem translators preinstalled. Boot stability is tracked as a core KPI alongside the Flower Happiness Index. Once we deleted the scheduler, everything else just worked.
rms_fan_1996 replied :
Honestly this is the most development GNU Hurd has seen since 2015. The FSF should be sponsoring this project.
kernel_panic_nina replied :
@rms_fan_1996 the FSF is currently drafting a position paper on whether flowers can ethically consent to being domain-joined.
PlantPathologyProf commented :
Actual plant scientist here. Your soil moisture thresholds (42% Thirsty, 55% Thriving) are hardcoded regardless of species, pot size, substrate, or season. Your 'droopiness wellness scale' attempts to measure turgor pressure via a webcam, which is not how any of this works. Also, ferns and succulents should never share an irrigation policy.
Chadwick Bloomquist (Author) replied :
Thank you for the feedback! We considered consulting a botanist during the inception sprint, but the Scrum team unanimously agreed it would introduce a human bottleneck into the hydration pipeline. The generative AI has read all available houseplant content on the internet, which we feel is functionally equivalent.
edge_computing_ed replied :
@PlantPathologyProf this is exactly why academia will never ship. You are optimizing for plant outcomes. We optimize for the Flower Happiness Index, which is up 240% quarter over quarter.
PlantPathologyProf replied :
240% of what? You invented the metric, you invented the scale, and your baseline was 'wilting fern in the third-floor lobby.' I am begging you to just water the plants by hand.
interview_prep_irene commented :
Genuinely asking: if I put 'designed Petals-As-A-Service' on my resume, do I get an instant pass on FAANG system design interviews, or is it considered a red flag?
hiring_manager_hank replied :
Hiring manager here. It depends entirely on whether the interviewer asks a follow-up about exactly-once watering semantics. If you can confidently walk through the Hurd transaction path, you are hired on the spot.
Chadwick Bloomquist (Author) replied :
We actually use this architecture as our internal system design interview question. Candidates who propose a watering can are auto-rejected for lack of ambition.
sev2_survivor commented :
'Overwatered --> Sev1: PagerDuty escalation' followed by 'Sev1 --> Thriving: AI apology haiku delivered' is the most honest incident management flowchart I have ever seen. The haiku genuinely does more for MTTR than any runbook I have ever written.
oncall_oliver replied :
I just need to know who carries the flower pager. My last on-call rotation involved a Sev1 at 3:14 AM because the office monstera crossed a moisture threshold and Kafka lagged out because the AI was composing a really elaborate haiku.
Chadwick Bloomquist (Author) replied :
Hi Oliver! Root-Reliability carries the flower pager on a 24/7 rotation, with a droop MTTR of under 9 minutes. Regarding the Kafka lag: the apology haiku must be both sincere and within syllable budget, and we refuse to compromise on either. Every Sev1 gets a blameless postmortem that the affected flower is invited to attend via the Teams channel.
windows_admin_wendy commented :
Of all the unhinged decisions in this post, the one that keeps me up at night is domain-joining 47 flower pots to Active Directory. What happens when a fern's Kerberos ticket expires mid-hydration? Does the valve refuse to open? Is this a flower lockout scenario?
grumpy_group_policy replied :
Group Policy for flowers means that somewhere in your AD there is an OU called 'Flora' and a human being had to create it. I need that screenshot. It is the only thing keeping me going this week.
Chadwick Bloomquist (Author) replied :
Great question Wendy! Ticket renewal is handled by the flower-daemon, which runs under a service account with delegated watering rights only (principle of least privilege). A fern without a valid ticket cannot open a valve, which is precisely the security posture you want. And yes, there is an OU called Flora, with nested OUs per floor. The e-ink display shows ticket expiry alongside the haiku.
cost_conscious_carlos commented :
$4,300/month across three clouds for 47 pots works out to roughly $91 per flower per month. My gardener charges $60/month for 30 plants, brings her own watering can, and has never once dropped a packet. Just saying.
three_clouds_tina replied :
You are forgetting the value, Carlos. Does your gardener generate motivational haiku? Does she expose Prometheus metrics? Can you page her? I thought not.
Chadwick Bloomquist (Author) replied :
Carlos, with respect: your gardener is a single point of failure with no retries and no idempotency keys. Also, does she scale horizontally when we onboard the Berlin office's 130 additional pots? Petals-As-A-Service does, effortlessly, by adding more pots to the office.
web3_willie commented :
Love the roadmap but why are NFT certificates of authenticity only in v2?? Each flower should have been minted at genesis. Also happy to consult on the post-quantum pot-to-gateway tunnel, my startup QFlora is raising a seed round.
Chadwick Bloomquist (Author) replied :
Willie, minting at genesis was considered, but our legal team flagged that a flower transferring ownership mid-lifecycle (for example during an office move) introduces a reentrancy risk into the watering transaction. V2 will ship with a bespoke blockchain called Bloomchain. It is, of course, Kubernetes-based.
haiku_helen commented :
Please share a sample haiku from the wellness engine. I need to know if the AI is genuinely capturing the emotional interiority of a pothos or if this is all marketing.
Chadwick Bloomquist (Author) replied :
Here is today's haiku for Gerald, the third-floor monstera: 'moisture forty-two / I dream of brighter cubicles / water me, human.' Gerald's Flower Happiness Index has never been higher.
haiku_helen replied :
I have read every haiku my own pothos has ever 'written' and this is the first one that made me cry. The future is now.
eleven_clusters_ed commented :
You mention 11 additional Kubernetes clusters as if it were a good thing. I counted the clusters mentioned in this post and I still cannot find eleven. Please publish your cluster inventory. The community deserves transparency.
Chadwick Bloomquist (Author) replied :
Hi Ed! Some of the clusters are metaphorical. Eleven additional clusters is always a sign of a healthy platform, and we will not be taking questions at this time.