An iOS release pipeline that checks itself first

A pipeline goes "green" when it reports success. At Memox, four tools either stayed green or pointed at the wrong culprit, while the real fault went completely unnoticed.

If you run a small product team, you already know the release tax. "Can someone make a build?" shows up in chat every day. And when a build fails, the error blames the wrong thing and the afternoon is gone.

Memox was paying that tax. Its iOS builds ran through a hosted build service and didn't trigger on merge. Its test copies of the backend lived on a shared group of servers that was being retired.

But the harder problem was silence. We found all four of these failures inside those eleven days:

• Empty secrets. A secret is a stored key the build reads. When one is missing, the build reads it as blank instead of throwing an error. So it ran for a minute or more, then blamed the key's content.
• Dead alerts. The alert sender rejected its own settings. All 22 alert rules showed healthy, and not one of them could reach anyone.
• A plan that lied. The server change plan, the summary reviewers read before approving, said "No Changes" for every single edit.
• Lost previews. From 28 August to 1 September, test copies skipped their access control step. That step still reported success on every run.

Abdullah built three things for Memox: a release pipeline, a faster build and a preview for every pull request. A pull request is a proposed change that someone reviews before it merges. Between 27 August and 6 September 2026 he merged all 40 app pull requests and all 28 cluster pull requests. Usama, our lead backend engineer, had added push notifications to the app in August.

The release path

A TestFlight-on-merge pipeline builds and uploads your iOS app every time a change merges. For Memox, the build runs on a Mac laptop that acts as the build machine. The first step checks that every required secret exists. If one is missing, the run stops and names it.

On 4 September, the first ten Memox runs failed. The eleventh, 78 minutes after the first, uploaded a build. Each failure went into a runbook, a how-to file sorted by the error you see. It lists ten symptoms: nine real failures, plus the secret check doing its job.

Sending the Memox app to Apple is a separate step. You type SUBMIT to start it, and it builds nothing. It sends the build already in TestFlight, so Apple reviews exactly what the team tested. A test fails if anyone adds a build to that step.

The cache fix

Memox builds took about 14 minutes. A cache is a stored copy of downloads, kept so the next build can skip them. Abdullah found the build pulling a 5.99 GB cache from GitHub on every run, over a home connection. The laptop already held that cache on its own disk. At 16 GB, the local copy was also above GitHub’s 10 GB limit, so every upload of it was wasted.

He removed the hosted cache on 6 September. The median successful Memox build fell from 13.7 to 6.0 minutes. Other changes landed that week too, so read that as “after”, not proof of cause.

A preview for every pull request

A preview environment is a live, temporary copy of the product for one pull request. Every Memox backend and dashboard pull request gets one. Reviewers open a link instead of setting the change up on their own machine. Each preview is deleted when its pull request closes.

Memox previews began in April on our shared cluster, a group of rented servers run as one. When that cluster was retired, Abdullah moved previews to a dedicated Memox cluster. Its first commit was 27 August, and previews moved onto it on 28 August. It runs five servers and adds up to five more when load spikes. Its README, the project’s own notes, estimates the cost at about EUR 63 a month.

Abdullah turned the extra servers on after an incident. Three previews at once pushed one server to 97% memory. The cluster’s management tool was killed 21 times in 134 minutes.

The Memox preview cluster does not run the live product. Live Memox still runs on Coolify, a self-hosted deploy tool.

The Memox pipeline now fails in seconds instead of minutes, and a successful build takes a median of 6.0 minutes. The table below uses GitHub’s own run times.

We would change two things on the Memox pipeline. First, we would move the build machine off a laptop sooner. A merge while the lid is shut waits in a queue instead of failing. Second, we would give the work a second owner earlier.

This fits you if you run a small product team with a mobile app and no platform team. You want releases on merge, previews for review, and failures that name their cause.

It is not for you if you already have a platform team with its own tooling. It is also not for you if you ship to the App Store rarely and a manual build is enough.