Generating Tests Worth Keeping (And Deleting the Ones That Aren't)

TL;DR — Ask a model for tests and you get coverage: one test per public method, largely asserting that the code does what the code does. Those tests are green, fast, and make every future refactor more expensive. Keep a test only if it fails for a reason a user would care about. AI's real strength here isn't writing tests — it's enumerating edge cases you hadn't considered.

The liability nobody counts

A test suite has a running cost. Every test is code that must be read, updated during refactors, and debugged when it goes red for a reason unrelated to a bug. We justify that cost with the defects tests prevent.

A test that asserts the implementation rather than the behaviour prevents no defects and still charges full price. Generate forty of those in an afternoon and you've quietly made your codebase harder to change — while the coverage number went up, which is why nobody notices.

The bar

One question, applied to every test:

Would this fail for a reason a user would care about?

If a plausible refactor breaks the test but the app still behaves correctly, the test is coupled to the wrong thing. If the app could break in a way users notice and no test goes red, you have a gap. That's the whole framework.

What AI produces by default

// Tests the language, not your code.
@Test fun `getName returns name`() {
    val user = User(name = "Hessam")
    assertEquals("Hessam", user.name)
}

// Asserts the implementation you're about to rewrite.
@Test fun `loadUser calls repository`() {
    viewModel.loadUser("1")
    verify(repository).fetchUser("1")
}

The second one is the more dangerous, because it looks like a real test. It passes if the ViewModel calls the repository — and it fails the moment you switch to an observable stream, even though the user-visible behaviour is identical. It is a refactor tax with a green tick.

What's worth keeping

// Boundary — the off-by-one that actually happens.
@Test fun `paging stops requesting when the last page is partial`() { /* ... */ }

// Error path — what does the user actually see?
@Test fun `network failure mid-save keeps the draft and shows retry`() = runTest {
    repository.failNextWrite()
    viewModel.save(draft)
    assertEquals(UiState.Error(RETRY), viewModel.state.value)
    assertEquals(draft, localStore.draft())    // the part users care about
}

// The bug you just fixed — proven valuable by existence.
@Test fun `rotating during checkout does not resubmit the order`() { /* ... */ }

Notice all three describe behaviour in the product's vocabulary. None mention a mock, and none would break if you rewrote the internals.

The prompt that changes the output

The default request — "write unit tests for this class" — optimises for coverage because that's what "tests" implies. Ask for something else:

Don't aim for coverage. Give me tests that would fail if a plausible refactor broke real behaviour. For each test, name the specific bug it catches and the user-visible symptom. If you can't name one, don't write the test. Prefer boundaries, error paths, and state transitions over happy paths. Assert on observable outcomes, not on mock interactions.

The sentence doing most of the work is "if you can't name one, don't write the test." It forces a justification per test, and justifications are much easier to evaluate than assertions. When the named bug is "the name field could be wrong", you delete it in two seconds.

Where AI is genuinely better than me

Not at writing tests — at enumerating cases. Before asking for any code:

Here's the function. List the inputs and state combinations where it would produce a wrong or surprising result. Don't write tests yet. Include concurrency, empty/boundary values, and Android lifecycle events.

That list is the valuable artefact. Models are excellent at systematically walking a space that humans skim, precisely because they don't share your assumptions about what "obviously can't happen". I routinely get two or three cases I hadn't considered — and I then decide which deserve a test, which deserve a code change, and which are genuinely impossible.

The review step you can't skip

Read every generated test before committing it. Not for style — for whether it can fail. The failure mode I see most often is a test that passes for the wrong reason:

// Passes whether or not the code works: nothing awaits the coroutine.
@Test fun `refresh updates state`() {
    viewModel.refresh()
    assertNotEquals(UiState.Loading, viewModel.state.value)   // races
}

A quick check: temporarily break the production code and confirm the test goes red. If it doesn't, it was never testing anything. That takes ten seconds and it's the only reliable way to tell a real test from a decorative one.

The rule

Generate freely, keep sparingly. The value of a test suite isn't how much code it touches — it's how confidently you can change code with it running. Tests coupled to implementation reduce that confidence while appearing to increase it, and AI will produce them faster than any human ever could.