All postsEngineering

Why AI-generated code passes CI and still breaks in production

Generated code and generated tests share the same assumptions. Four checks that don't, with tested examples in Python, Go and Envoy.

Johnny Unar11 min read

01

green ci means something different now

A CI pipeline was built around a quiet assumption: the person who wrote the code understood the system around it, and wrote tests aimed at the parts they weren't sure about. When a model writes both the implementation and the tests in one session, that assumption is gone. The tests come from the same picture of the system as the code, so they agree with it by construction, and a green build mostly tells you the two halves are consistent with each other. If your team merges a lot of assistant or agent output into Go or Python backends, this is the gap you're exposed to.

The surveys are consistent with that, though they're all vendor-run and should be read that way. CloudBees' 2026 State of Code Abundance report, a survey of 213 enterprise technology leaders run by TrendCandy with a stated margin of error of plus or minus 8 points, found that 92% expressed confidence in the production readiness of AI-generated code while 81% reported an increase in production issues tied to it. Lightrun's 2026 State of AI-Powered Engineering report, a survey of 200 SRE and DevOps leaders at director level and above, says 43% of AI-generated code still needs manual debugging in production after passing QA and staging. That's a survey answer, not telemetry. CodeRabbit's December 2025 analysis of 470 open source pull requests (320 it classified as AI co-authored, 150 as human-only, with authorship inferred from signals rather than confirmed) found 10.83 issues per AI PR against 6.45 for human PRs, roughly 1.4 to 1.7 times more critical and major findings, logic and correctness issues 75% more common, and error-handling gaps nearly twice as common.

That last category tells you where to look. The failures cluster where the code meets something it didn't write: an upstream API that sometimes returns null, inputs nobody thought of, and behaviour that slowly diverges between two generated patches that each looked fine on its own. Adding more of the same checks doesn't reach those places. What follows are four checks that do, ordered roughly by how cheap they are to adopt.

02

ask for the assumptions before the diff

The cheapest check happens before any code exists. For a non-trivial change, ask the model to write down every assumption the implementation will depend on, and have a person read that list before reading the diff. It's far easier to spot "assumes the provider never returns a payment without settled_at" in a ten-line list than to find the same assumption buried in three hundred lines of plausible code.

A prompt along these lines works with any assistant, and the output belongs in the PR description, where a reviewer sees it first:

text
1Before writing code, list the assumptions this change depends on:
2- inputs: shapes, ranges, encodings, empty and missing cases you expect
3- upstream behaviour: fields you treat as always present, error codes you handle
4- state: what you assume about ordering, concurrency, idempotency, retries
5- money and time: rounding, currency, time zones, clock source
6For each one, say whether you verified it in the codebase (cite the file)
7or are guessing. Do not write code until I confirm the list.

The "verified or guessing" column does most of the work. Guesses about upstream behaviour are exactly where the contract tests in the next section come from, and a guess about rounding or time zones tells you which function needs a property test. Even when the list holds no surprises, reading it takes a minute and reviewing the code afterwards gets faster, because you know what the author believed.

03

contract tests at the seams

The integration boundary is where generated code fails most silently, because the model has only ever seen the happy example in the docs. A contract test pins the real shapes down in the repository: a fixture file for each response the upstream actually sends, including the ugly ones, and a test that the boundary model accepts or rejects each of them on purpose. Once those fixtures exist, every future generated change has to pass through them, whether or not the model remembers the constraint.

In Python, a strict pydantic model at the boundary does the parsing, and anything inside your code only ever sees the validated object:

python
1from datetime import datetime
2from typing import Literal
3
4from pydantic import BaseModel, ConfigDict
5
6
7class PaymentStatus(BaseModel):
8 model_config = ConfigDict(extra="ignore", strict=True)
9
10 id: str
11 status: Literal["pending", "settled", "failed", "refunded"]
12 amount_minor: int
13 currency: str
14 settled_at: datetime | None = None
15 failure_code: str | None = None

The fixtures are plain JSON files captured from the provider's sandbox or from logged production responses with personal data removed: a settled payment, a pending one where settled_at and failure_code are explicitly null, a failed one carrying a field you don't use yet, and one where the amount arrives as a string. The test says which shapes you accept and which you refuse to guess about:

python
1import json
2from pathlib import Path
3
4import pytest
5from pydantic import ValidationError
6
7from payments import PaymentStatus
8
9CONTRACTS = Path(__file__).parent / "contracts"
10ACCEPTED = ["payment_settled", "payment_pending_null_fields", "payment_failed_extra_field"]
11REJECTED = ["payment_amount_as_string"]
12
13
14@pytest.mark.parametrize("name", ACCEPTED)
15def test_accepts_real_provider_shapes(name):
16 PaymentStatus.model_validate_json((CONTRACTS / f"{name}.json").read_text())
17
18
19@pytest.mark.parametrize("name", REJECTED)
20def test_rejects_shapes_we_refuse_to_guess_about(name):
21 with pytest.raises(ValidationError):
22 PaymentStatus.model_validate_json((CONTRACTS / f"{name}.json").read_text())

strict=True is what makes the string amount fail instead of being quietly coerced to an integer, and extra="ignore" keeps a new provider field from breaking you. Those are decisions a person should make once, deliberately, and then keep in the repo. For seams between your own services owned by different teams, Pact formalises the same idea: the consumer publishes the interactions it relies on and the provider's build verifies them. Within one team, fixture files and a test like this get you most of the value without a broker.

04

properties, fuzzing and mutation testing

Example tests check the cases the author imagined, which for generated code means the cases the model imagined. Property-based tests flip that around: a person states an invariant that must hold for every valid input, and the framework generates inputs trying to break it. Here's a function that splits an amount in cents into equal parts, written the way an assistant plausibly would, along with the two example tests that came with it and passed:

python
1def split_cents(total: int, parts: int) -> list[int]:
2 share = round(total / parts)
3 return [share] * (parts - 1) + [total - share * (parts - 1)]
python
1from billing import split_cents
2
3
4def test_split_returns_all_parts():
5 assert len(split_cents(1000, 3)) == 3
6
7
8def test_split_keeps_total():
9 assert sum(split_cents(1000, 3)) == 1000

The property test states what fairness means for money: as many shares as requested, they add up to the total, none is negative, and no two differ by more than a cent.

python
1from hypothesis import given, strategies as st
2
3from billing import split_cents
4
5
6@given(total=st.integers(min_value=0, max_value=10**9), parts=st.integers(min_value=1, max_value=50))
7def test_split_is_fair_and_exact(total, parts):
8 shares = split_cents(total, parts)
9 assert len(shares) == parts
10 assert sum(shares) == total
11 assert min(shares) >= 0
12 assert max(shares) - min(shares) <= 1

Hypothesis found two counterexamples in under a second and shrank each to the smallest input that fails. split_cents(3, 5) returns [1, 1, 1, 1, -1], a negative share, and split_cents(2, 4) returns [0, 0, 0, 2], because Python's round rounds halves to even and round(0.5) is 0. Neither case is exotic, and neither would come up in a test someone wrote by hand while believing the function was right. The fix is integer arithmetic:

python
1def split_cents(total: int, parts: int) -> list[int]:
2 base, remainder = divmod(total, parts)
3 return [base + 1] * remainder + [base] * (parts - remainder)

Go has the same capability built in since Go 1.18. A fuzz target is a test function that takes *testing.F, the seed inputs run on every go test, and go test -fuzz=FuzzTruncate keeps mutating them. This one checks a function that shortens a string to a byte limit:

go
1package textutil
2
3import (
4 "strings"
5 "testing"
6 "unicode/utf8"
7)
8
9func FuzzTruncate(f *testing.F) {
10 f.Add("hello world", 5)
11 f.Add("Příliš žluťoučký kůň", 3)
12 f.Fuzz(func(t *testing.T, s string, maxBytes int) {
13 if !utf8.ValidString(s) || maxBytes < 0 {
14 t.Skip()
15 }
16 out := Truncate(s, maxBytes)
17 if len(out) > maxBytes {
18 t.Fatalf("len %d > %d", len(out), maxBytes)
19 }
20 if !strings.HasPrefix(s, out) {
21 t.Fatalf("%q is not a prefix of %q", out, s)
22 }
23 if !utf8.ValidString(out) {
24 t.Fatalf("invalid UTF-8: %q", out)
25 }
26 })
27}

Against the naive return s[:maxBytes], both seeds pass, including the Czech one, because cutting "Příliš" at 3 bytes happens to land on a character boundary. The fuzzer needed well under a second to find "řÚ" with a limit of 3, which cuts the two-byte Ú in half and returns invalid UTF-8. It wrote the input to testdata/fuzz/FuzzTruncate/, which you commit, so the case runs as a regular test from then on. The fixed version backs up to the start of a rune:

go
1package textutil
2
3import "unicode/utf8"
4
5func Truncate(s string, maxBytes int) string {
6 if len(s) <= maxBytes {
7 return s
8 }
9 cut := maxBytes
10 for cut > 0 && !utf8.RuneStart(s[cut]) {
11 cut--
12 }
13 return s[:cut]
14}

Mutation testing answers a different question: whether your tests would notice if the code were wrong. A tool such as mutmut for Python, or Stryker for JavaScript and TypeScript, makes small changes to the code (flipping <= to <, // to /, nudging constants) and reruns the tests; a mutant that survives is a behaviour change no test cares about. That makes it a direct check on tests that merely mirror the implementation. Take a late fee function:

python
1MAX_FEE_CENTS = 5_000
2
3
4def late_fee_cents(amount_cents: int, days_late: int) -> int:
5 if days_late <= 0:
6 return 0
7 return min(amount_cents * 2 // 100, MAX_FEE_CENTS)

With two example tests, one 10 days late and one 5 days early, mutmut 3.8 reported three survivors: days_late < 0, days_late <= 1, and amount_cents * 2 / 100, which would quietly return a float. Each survivor maps to a test somebody should write: what happens on day zero and day one, whether the result is still whole cents, and whether the cap holds. After adding three such tests all eleven mutants were killed. Mutation runs are slow on a large codebase, so in CI we'd restrict them to files changed in the PR.

05

shadow traffic without side effects

Tests can't show you the distribution of real requests, so the last layer runs the new version against live traffic without serving its answers. Envoy does the mirroring at the proxy: a route sends each request to the primary cluster and a copy to the candidate, and in Envoy's words it's "fire and forget", so it never waits for the shadow's response before answering the client. This config mirrors 10% of GET requests on one path and nothing else:

yaml
1static_resources:
2 listeners:
3 - name: ingress
4 address:
5 socket_address: { address: 0.0.0.0, port_value: 10000 }
6 filter_chains:
7 - filters:
8 - name: envoy.filters.network.http_connection_manager
9 typed_config:
10 "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
11 stat_prefix: ingress
12 route_config:
13 virtual_hosts:
14 - name: api
15 domains: ["*"]
16 routes:
17 - match: { prefix: "/api/invoices/", headers: [{ name: ":method", string_match: { exact: "GET" } }] }
18 route:
19 cluster: primary
20 request_mirror_policies:
21 - cluster: candidate
22 runtime_fraction:
23 default_value: { numerator: 10, denominator: HUNDRED }
24 - match: { prefix: "/" }
25 route: { cluster: primary }
26 http_filters:
27 - name: envoy.filters.http.router
28 typed_config:
29 "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
30 clusters:
31 - name: primary
32 type: STRICT_DNS
33 load_assignment:
34 cluster_name: primary
35 endpoints: [{ lb_endpoints: [{ endpoint: { address: { socket_address: { address: aidev-primary, port_value: 8080 } } } }] }]
36 - name: candidate
37 type: STRICT_DNS
38 load_assignment:
39 cluster_name: candidate
40 endpoints: [{ lb_endpoints: [{ endpoint: { address: { socket_address: { address: aidev-candidate, port_value: 8080 } } } }] }]

Running this on Envoy 1.40 in Docker with 100 GET requests, 7 reached the candidate (it's a random fraction), a POST to the same prefix went only to the primary, and the candidate saw the host header as localhost-shadow:10099, because Envoy appends -shadow unless you turn that off. Both sides received the same x-request-id, which is what lets you pair the responses afterwards.

The part that people skip is side effects, and with payment code it's the part that matters most. A mirrored request is a real request. If the candidate writes to the production database, charges a card, sends an email or publishes to a shared queue, you've just done it twice. Only mirror read-only endpoints, or give the candidate its own database copy or replica and stub every outbound integration that changes state. The simpler alternative of copying requests to a goroutine inside the same service is worse on both counts, because it shares the process, the connection pool and the downstream credentials, so the shadow can add latency to real requests or crash them along with itself.

Comparing responses only works after normalisation, otherwise every pair differs by its timestamp. Each side logs its response body with the request ID, and a small comparator drops volatile fields and sorts lists before diffing:

python
1import json
2import sys
3
4VOLATILE = {"generated_at", "request_id", "trace_id", "etag"}
5
6
7def normalize(value):
8 if isinstance(value, dict):
9 return {k: normalize(v) for k, v in sorted(value.items()) if k not in VOLATILE}
10 if isinstance(value, list):
11 return sorted((normalize(v) for v in value), key=lambda v: json.dumps(v, sort_keys=True))
12 return value
13
14
15def diff(a, b, path="$"):
16 if isinstance(a, dict) and isinstance(b, dict):
17 for key in sorted(a.keys() | b.keys()):
18 yield from diff(a.get(key), b.get(key), f"{path}.{key}")
19 elif a != b:
20 yield f"{path}: {a!r} -> {b!r}"
21
22
23primary, candidate = (normalize(json.loads(line)) for line in open(sys.argv[1]))
24for change in diff(primary, candidate):
25 print(change)

On a pair where the line items come back in a different order and the timestamp differs, it reports only the real change, $.total_minor: 1000 -> 1001, plus the version field you'd add to your ignore list. Only sort lists whose order isn't part of the contract. A one-cent drift between two versions of an invoice endpoint is exactly the kind of behavioural change no unit test catches, because each version is correct by its own tests.

06

where to start and what this won't catch

If you add one thing this week, make it the assumptions list, because it costs nothing and tells you where the other checks are needed. Then write contract fixtures for your most expensive integration, and a property test for the function that does the most money or parsing work. Shadow traffic is the biggest lift and pays off on high-traffic read paths, while for write paths you'll need record and replay against an isolated environment instead.

None of this verifies that the code does what the business wanted. Every check here depends on a person stating something true about the world, whether that's a fixture, an invariant or the list of fields you're allowed to ignore. That's the point, since it's the human input the generated tests were missing, but it means the work moves to whoever owns those statements rather than disappearing.

Thanks for reading.
Johnny Unar

Written by

Johnny Unar

Keep reading

Stop patching.Start building.

Tell us what's slowing you down. We'll bring coffee and opinions.