Someone Described Their Garage Door Perfectly and Got 'No Exact Match'

Someone told the Virtual Technician on garagedoorscience.com exactly what their door was doing. It was a clear, accurate description, the kind a technician would hear and know where to start. The card that came back said "DIAGNOSIS · NO EXACT MATCH."

The person did their part. The tool didn't.

To see why, it helps to know how the tool works. It looks things up before it does any thinking. There's a list of known garage door problems, and each problem has the phrasings people tend to use for it. When a description comes in, the tool checks it against those patterns and nicknames. If it finds one, you get a diagnosis right away. No AI model is involved, so it costs nothing and returns in an instant.

When nothing matches, there's a second step. The tool can hand the description to Haiku, Anthropic's smallest and cheapest model, and ask it which known problem this sounds like. That step only runs if the caller asks for it with a flag. A flag is an on/off setting the caller passes in. Leave it out and it stays off.

That's what happened here. In production, the callers were AI agents, and none of them set the flag. Every agent got the matcher alone. Whenever a person's wording landed a little outside the list, they got "no exact match," even when they'd described the problem perfectly.

What each side actually costs

Look at what you pay on each path.

When the matcher hits, the fallback never runs. Leaving it on costs you nothing on those requests. It just sits there.

When the matcher misses, the fallback costs about half a second and one Haiku call. That's a fraction of a cent and a pause most people won't notice.

Leaving it off costs you the person. They described their door well, and the tool told them it didn't know. They won't think "the regex was too narrow." They'll think the tool doesn't work, and they'll be right.

So one choice costs half a second on misses only. The other costs a frustrated homeowner on every miss. That isn't close. The fix was small: agent callers now set the fallback to on by default.

Why the caller matters

I'd guess the flag started out off for a reasonable reason. If you're a developer testing the matcher, you want to see raw misses so you can grow the list. The fallback would hide those misses from you.

An agent calling the tool for a homeowner wants something else. It isn't tuning the list. It wants an answer to pass along. A default built for the person building the tool ended up applied to the person using it.

Here's the rule I'd use:

If the caller is an agent acting for someone, a miss lands on a real person, and the fallback is cheap, turn it on. If the caller is a person studying the tool, leave it off so the misses stay visible. In the garage door case every one of those checks pointed the same way, and we still had it off.

The fallback that hides

I don't trust fallbacks by default. I've seen the bad kind. Your main model goes down, a backup model quietly takes over, and everything looks fine. Nobody notices the main model is dead until something else breaks. That fallback is covering for a failure you needed to hear about.

So what's different here?

The matcher isn't broken when it misses. It's doing its job, which is to recognize the phrasings it knows. A miss is an expected event. The fallback handles a case the first step was never built to handle. It doesn't paper over a fault.

A dead primary model is a fault. Something that should work doesn't. Covering it silently means the system keeps telling you all is well while it runs on its spare tire.

I'd still log every fallback call here. Those logs show which phrasings the list is missing, so the matcher gets better over time. The rescue happens, and you can still see it happened.

If you take one thing from this: a fallback that rescues catches what the first step was never meant to catch, and you can see it doing that. A fallback that hides catches what the first step was supposed to handle, and you can't see it at all.

---

Seth Shoultes builds at garagedoorscience.com and writes here when the building produces something worth saying.