Three months of finished analysis were deleted in February 2026 when a client reversed the brief. The replacement work shipped three weeks later. Nobody had asked, at the start, what happens when the direction changes after the money is committed. That question costs nothing to ask in a sales call.
Vetting a development agency is not about catching liars. Most answer honestly. The difference between a good hire and an expensive one is whether the answers are specific enough to check later.
Ask a development agency five things before you sign: who writes and reviews the code, what happened on their last scope change, what you hold on the last day, what the month after final payment looks like, and when they last talked a client out of something. Judge the specificity of each answer, not the confidence.
This guide is for the operator who still owns the store after the agency leaves
You are hiring for a Shopify build, a replatform or a migration. After launch you are the one who answers when an order does not reach the warehouse, or when a wholesale customer sees the wrong price.
The hours you want back are the ones spent asking which system dropped an order. Autonomous exists to return those hours: we give Shopify and Shopify Plus operators back the time and the order errors that leak between systems after the buy button. The Shopify Plus agency work behind that is integrations, pricing logic and the data moving between them. Which makes us a firm you might vet with this guide, so the five questions below are answerable by any agency, and on two of them our own record is mixed.
This is for you if you sign for the build and stay accountable for the store afterwards. It is not for you if you are buying design work with no systems behind it. And here is where we are the wrong hire. If you need two people who know each part of your stack, press us hard on it. On the partner engagement described below, two people covered three disciplines between 2025 and 2026 and their working days never overlapped. That is deep coverage per discipline and thin coverage per person.
The people in the pitch are not always the people in the commit log
Ask: who writes the code, and who reviews it before it reaches my store?
You meet a founder and a strategist. The work may be done by someone you never met. That is not automatically wrong. An agency that names the person, the reviewer and the working hours is telling you something you can check. One that says it assigns resources by availability is telling you something too.
Write the answer down: names, the review step, and who could pick up each system next week.
| What you ask | A specific answer | A comfortable answer |
|---|---|---|
| Who writes it | 2 named people, and which part each one holds | "our development team" |
| Who reviews it | A named reviewer, and a pull request you are allowed to read | "we have a QA process" |
| When they work | Stated overlap in hours per day with your timezone | "we are flexible" |
| Who covers | How many people could pick up each system next week | "resources are assigned by availability" |
Ask how the last scope change went, not whether they handle change
Ask: tell me about the last time a client changed the brief mid build. What happened to the timeline and the invoice?
“Do you handle scope changes?” gets a yes from everybody. A scope change is any request that alters what was agreed after work has started, and every build has them. The useful question is retrospective, because it forces a real project into the answer.
You are listening for a sequence: the change is written down, priced, approved by you, and only then built. What should worry you is “do not worry, we are flexible”, which usually means the cost arrives later as a surprise.
The example at the top of this page is ours. On a white-label partner engagement, three months of measurement work was deleted in February 2026 after the client changed direction, and the replacement shipped three weeks later. That is the shape of answer you want, with a date attached.
The rule: ask about the last change, not about the process for change. A process is a paragraph anyone can write. The last change is a real project with a date, a cost and an outcome. If they cannot name one, either nothing has ever changed on one of their builds, or they would rather not say.
The clause that decides who pays belongs in the statement of work, not in the sales call. What each clause has to name is covered in what your contract scope has to name before you sign.
Owning the code is not the same as being able to run it
Ask: on the last day, what do I hold, and what still runs on your accounts?
Most will say you own everything, and most mean it. Push past it anyway: source custody is one of three things you need.
First, the code. A repository is the code plus its full change history, usually kept on GitHub. Second, the ability to run it: containers, setup docs, environment values, deploy scripts. Third, the accounts: hosting, domains, the Shopify store, analytics and email.
The honest version from our own record. On that partner engagement, all three repositories sat in the client’s own GitHub organisation, verified by git remote on 9 September 2026. One deployed service still ran on our hosting rather than theirs, from February 2026. Source custody was settled. Hosting custody was not. Ask those two questions separately, of us and of anyone else.
The platforms decide part of this. A Shopify collaborator account is the access an agency uses to work in your store without taking one of your staff seats. Know what each platform does on its own before you negotiate the rest.
| What you hand back | What the platform does | The catch |
|---|---|---|
| The code | A GitHub repository transfer carries its issues, pull requests, wiki, stars, watchers and full commit history, and old links redirect to the new location | You need permission to create a repository in the target organisation, so arrange that in week 1 |
| Store access | A Shopify collaborator account is granted with a 4 digit request code you generate, and generating a new code stops the old ones working | Collaborator accounts do not count towards your store's user limit, so an agency can stay connected long after the invoice closed |
| Ending access | Removing a collaborator account from your store is permanent and cannot be undone | Access also expires on its own after 90 days with no login, which is not the same as you revoking it |
Those facts come from GitHub’s documentation on transferring a repository and Shopify’s help page on collaborator accounts, both read on 21 September 2026. Put the handover date in your calendar on day one. Nothing will remind you.
The month after final payment is the part nobody rehearses
Ask: what happens in the 30 days after I send the final invoice?
Bugs surface when real customers arrive, not in staging. Staging is a private copy of your store where work is tested before customers can see it. The answer you want names a period, a channel and a person. “We will be here if you need us” is not a plan, so ask what “here” means on a Tuesday morning.
Ask a second question alongside it: what did “delivered” mean on their last project? Delivered code and running code are different states, often sold as one.
On the survey pipeline stream of that same partner engagement, we delivered the pipeline, its tests and a deploy script. The record holds no evidence that it was ever deployed and run on a schedule. That is a legitimate delivery when the scope says so, and an unpleasant surprise when you assumed otherwise.
Get the definition of done in writing before you sign. Is it code in a repository, code running in a staging environment, or code processing real orders in your live store? Those are three different deliverables, and the gap between them is where post-launch arguments live.
An agency that has never told a client no has not been paying attention
Ask: tell me about a time you talked a client out of something they wanted.
You are buying judgement, not typing. A team that agrees with everything is either too inexperienced to know better or too indifferent to say so.
Listen for a story with an outcome, including the ones where the pushback turned out to be wrong. An agency that only recounts its vindications is still selling.
A quieter version of the same test: ask who gets credit for the ideas. On the partner engagement, the client’s own analyst wrote the four-phase analytical logic at the centre of the measurement application, and the commit message inside the client’s repository credits her by name. The full engagement is written up at white-label partner delivery.
| The question | What this engagement's record shows | How it was checked |
|---|---|---|
| Where does the code live | All 3 repositories in the client's own GitHub organisation | git remote in each working copy, 9 September 2026 |
| Where does it run | 1 deployed service on Autonomous hosting, not the client's | Deployment commit, 16 February 2026 |
| Who wrote the core logic | The client's own analyst, credited by name in the commit message | Commit message inside the client's repository |
| What happened on the reversal | 3 months of work deleted, replacement delivered 3 weeks later | Commit history, February and March 2026 |
| Was a handover session held | Unknown. Nothing in the record answers it either way | No document found |
What breaks, and who owns it
These five questions fail in predictable ways. Each failure has an owner, and most of them are you.
Asked in a group call, they get the rehearsed answer. Ask them again of the person who would do the work. You own arranging that call.
An agency that answers all five perfectly may simply be good at answering. You own checking one answer independently. Ask for a reference whose build shipped over a year ago, not the one who launched last month.
A directory listing is a filter, not a verdict. The Shopify Partner Directory tiers partners by their history on the platform and invites you to check reviews, work samples and certifications. It also states plainly that listed partners “work independently”. The vetting is still yours.
And a warning about who wrote this. An agency publishing vetting criteria has an obvious incentive to publish criteria it passes. Two of the five answers above are ones where our own record is mixed: hosting custody that did not follow source custody, and one person of depth per discipline. Use the questions. Do not use our answers as the standard.
Questions operators ask before hiring an agency
What questions should I ask a development agency before signing?
Ask who writes and reviews the code, how the last scope change was actually handled, what you hold on the last day including hosting and store accounts, what support looks like in the 30 days after final payment, and when they last talked a client out of something. Ask for specifics you can check later, not reassurance.
How do I check that an agency really hands over the code?
Ask which GitHub organisation the repositories will live in from day one, rather than at the end. A transfer carries issues, pull requests and full commit history, and old links redirect, but you need create permission in the target organisation, so arrange it early. Then ask separately about hosting, domains and store access, because none of those move with the code.
Is a Shopify Partner Directory listing enough vetting?
No. Shopify tiers partners by their history on the platform and shows reviews, work samples and certifications, which is a useful filter. It also states that listed partners work independently. Treat a listing as a shortlist, then ask the five questions yourself.
How do I remove an agency's access when the project ends?
Decide the date before the work starts and write it into the plan. Shopify collaborator access expires on its own after 90 days without a login, which is not the same as being revoked, and removing a collaborator account is permanent with no undo. Do it deliberately, on a date you chose.
Should I still ask all of this for a small build?
Ask the ownership and access questions at minimum. They take two minutes and they are the ones that cost real money later. The scope change and post-launch questions matter more as the build gets longer.
Start by finding out what you already own
Ask these five questions of your next agency. Then find out what the last one left behind.
