Skip to content
All field notes

What Spotify’s coding agents teach us about building a better delivery system

A practical six-step guide to shared context, background coding agents, verification, review and a measured two-week team pilot.

What Spotify’s coding agents teach us about building a better delivery system
In this article

Cover image: an AI-generated fictional team review.

An AI coding tool can produce a working change while the team is still debating what the change should do. That is exciting—and it changes where delivery gets stuck. The next constraint may be an unclear brief, an overloaded reviewer, a missing test environment, or a product decision nobody owns.

Spotify’s experience offers a useful model for addressing those problems. Its June 3, 2026 engineering article reports more than 99% weekly AI-tool use among engineers, 94% reporting improved productivity, and a 76% increase in pull-request frequency. Those are Spotify’s reported adoption, perception and activity measures. They do not establish a 76% increase in customer value or a universal productivity gain.

What Spotify actually built

The foundation predates generative AI. Backstage gives developers a catalog of software components, owners and documentation. Fleetshift coordinates maintenance across that inventory. Honk, its background coding agent, handles code changes within a controlled execution environment. Soundcheck and the company’s engineering standards provide feedback about expected practices. These are connected parts of a delivery system.

Keep the historical numbers straight: Spotify’s more than 2.5 million automated maintenance PRs accumulated through Fleet Management over several years. They should not be presented as 2.5 million AI-authored PRs. The reported three-day Java migration is one company example, not an implementation timetable for every team.

There is also a product distinction. The open-source Backstage framework and Spotify’s commercial Portal offering are different things. Spotify describes Fleetshift and Honk as part of Portal. You can apply the operating principles without buying Portal or reproducing Spotify’s Kubernetes infrastructure. A small team may start with a repository inventory, clear ownership, version control and its existing continuous-integration system.

Broader research reinforces the systems perspective: DORA’s 2025 report describes AI as magnifying existing organizational strengths and weaknesses. Our practical inference is to improve the path from a ready task to an accepted result before increasing the number of agents.

1. Choose a task you can recognize as finished

Step 1: Choose one repeatable job. Task: Update one deprecated UI component; Included: Three known pages in a preview branch; Excluded: Authentication, payments, production data; Acceptance: Same behavior; keyboard and build checks pass
Step 1. A small accepted change is the first milestone. Original illustrative workflow; open to enlarge.

Start with a frequent, bounded change whose correctness you can check: replacing a deprecated component, repairing a known test, updating a documented API call, or correcting a reproducible UI defect. Use deterministic scripts when the transformation is exact. Reserve an agent for cases that require interpreting surrounding code.

Our fictional pilot updates one deprecated component across three pages. A product owner defines unchanged behavior; an engineer names the affected files; a reviewer agrees the acceptance checks. Keep unrelated cleanup out of the patch. A successful pilot ends with a reviewed, usable change, not simply a generated branch.

2. Build a small, reliable context pack

Step 2: Give the agent a map. Owner: Frontend team / named reviewer; Reference: Approved component and before / after example; Commands: Exact install, build and test commands; Stop rule: Ask when a required mapping is missing
Step 2. Make the correct path easier to find than a guess. Original illustrative workflow; open to enlarge.

Create a repository guide containing the stack, run commands, ownership, approved patterns, test fixtures and prohibited changes. Use the instruction-file format your chosen agent supports. Supply one correct example and a concrete mapping from the old behavior to the new behavior. Link to current documentation and identify who maintains it.

For multiple repositories, add a simple catalog with repository URL, service owner, runtime, test command, deployment target and last verification date. A well-maintained table is enough for a first pilot. If you expose it through an API, command-line tool or MCP server, start with only the access the task needs.

Spotify’s dataset-migration case study is instructive: explicit field-mapping tables improved results; inconsistent Scio pipelines were deferred; missing automated tests left manual checks with owners. Context and verification remained limiting factors.

3. Wrap execution in a controlled workflow

Step 3: Run in an isolated workspace. Workspace: Separate branch and disposable environment; Access: Only the target repository and approved tools; Budget: Example: 20 minutes / two repair attempts; Output: Draft pull request; never a silent production edit
Step 3. An agent session needs boundaries and a clear exit. Original illustrative workflow; open to enlarge.

A background agent needs somewhere to work, a defined tool set, a time or cost limit, and a way to stop. Give each job a separate branch and isolated environment. Limit repository and network access. Use short-lived credentials where your system supports them; keep production secrets outside the workspace.

For an initial exercise, a 20-minute budget and two repair attempts could be reasonable limits; these are illustrative settings to tune, not industry standards. Stop when a requirement is ambiguous, a forbidden file would change, or the budget is exhausted. Return a useful failure report rather than silently retrying forever.

4. Make verification part of the job

Step 4: Verify observable behavior. Automated: Build, types and relevant behavior tests; Interface: Keyboard, small screen and error states; Report: Changed files and actual test results; If blocked: Return logs and the unresolved decision
Step 4. Passing checks are evidence, not a blanket guarantee. Original illustrative workflow; open to enlarge.

Have the agent run the project’s real checks and attach the results. For a UI change, include a preview and checks of the requested behavior, keyboard operation, narrow layouts and error states. For a migration, use representative fixtures and compare the expected output. A successful compilation cannot prove the business behavior stayed correct.

Define the critical acceptance cases before generating the solution. An agent that writes both a change and a test can accidentally encode the same misunderstanding twice. A named reviewer should inspect whether the checks cover the actual requirement, including one known failure case.

5. Design the review queue before increasing volume

Step 5: Route the decision. Routine: Small change with known acceptance checks; Sensitive: Authorization, money or destructive behavior; Reviewer sees: Diff, evidence, uncertainty and rollback; Action: Approve, request changes or stop the run
Step 5. Review capacity determines how much work to start. Original illustrative workflow; open to enlarge.

Every PR should explain the requested outcome, changed files, tests actually run, remaining uncertainty, preview location and rollback approach. Label changes by consequence. Authorization, money movement, data deletion and major architecture changes need appropriate expert review. A routine documentation update can follow a simpler path.

Keep human approval for the pilot. Consider narrowly scoped automatic merging only after you have reliable checks, protected rules and evidence from repeated comparable changes. If reviewers can finish two changes a day, starting twenty agent sessions may create a larger waiting queue. Limit work in progress and reserve review time.

6. Run a two-week pilot and inspect the whole job

Step 6: Measure before expanding. Flow: Ready brief to accepted change; review wait; Effort: Preparation + review + repair minutes; Quality: Escaped defects and rollback count; Decision: Continue, narrow the task or stop
Step 6. Expand the proven workflow one step at a time. Original illustrative workflow; open to enlarge.

On days 1–2, record the baseline from comparable completed tasks and agree the scope. On days 3–4, prepare context and checks. During days 5–8, run small changes with a reviewer and log every failed or abandoned attempt. Use days 9–10 to compare total effort, elapsed time, quality and actual acceptance.

Track preparation, model/tool costs, review and repair effort separately. Compare like-for-like tasks, and record changes in scope or staffing that could explain the result. Expand only if the workflow is useful after those costs and quality checks. A short pilot provides local evidence, not proof of a lasting organization-wide effect.

A starter task brief

Outcome: Replace [old component] in [three named pages].
Owner / reviewer: [names].
Approved reference: [file and documentation version].
Must stay the same: [observable behavior].
May change: [specific directories].
Acceptance: [behavior checks and exact commands].
Stop and ask when: [missing mapping / permission / ambiguity].
Return: draft PR, test evidence, preview, uncertainty, rollback.
Budget: [time / cost / repair attempts].

The repeatable advantage is a team that can define good work, give agents relevant context, inspect the evidence and make decisions without losing ownership. Start there; add more automation when the results justify it.

Sources

Choose your next practical guide

Want this applied to your business? The current $497 Growth Fix includes three focused improvements within one customer journey: your offer, inquiry or booking step, and first reply. Review, a private working session, implementation, written handoff, and a 30-day measurement plan are included.

Ashley Kays

Ashley Kays

Founder

Founder of Waymaker. BigCo veteran (NCR, Walt Disney World, Wyndham Worldwide) turned solo operator. Building the operating layer above AI building tools.

More field notes