My AI research partner asserted four advantages before doing the arithmetic

2026-08-21

tl;dr

I ran 68 pre-registered experiments against my own deep-tech chip idea, a Micro-Electro-Mechanical Systems (MEMS) resonator array that classifies sound at about 1 µW, roughly a hundred times below the best digital always-on parts. Six weeks of calendar time, but only about eight working days of my actual attention.
The physics worked. The moat didn’t, and I closed the project on 2026-08-20.
The AI did the heavy lifting, and it was also confidently wrong in a specific, repeatable way. Four times it named an advantage before checking the law that governs it. All four died on measurement, and one came back with the opposite sign from the one I’d claimed.
So what I’m left with isn’t a chip. It’s a harness of eighteen methodology rules, each one bought with a real error that had produced a plausible, self-consistent, completely wrong number.
The rule that cost the most to learn is that a classification score is not a detection score. My front end scored 0.62 accuracy on curated clips, but only 0.589 Area Under the Curve (AUC), a ranking score where 0.5 is coin-flipping, once I asked it to find those same sounds hiding among many others it had never been shown. The conventional digital control collapsed the same way, to 0.686. Every one of the 1,880 non-alarm clips triggered at least once, and for an always-on part a false alarm is a power bill, so my single advantage was being spent entirely on being wrong.
Cost to find this out was eight days of work. Cost to find it out the usual way is $8–15M and four to five years, which is what I priced the path at before closing it.
The claim I’m not making is that AI reduces deep-tech failure rates. What I have is a stack of ideas killed cheaply and not one success, and a kill is only proven right when somebody else tries the same thing and fails, which hasn’t happened. The narrower claim I’ll defend is that AI moves the failure early and makes it cheap, and only if you build the jig.

Background, briefly

I have a PhD, I founded a company once, and I spent years at a deep-tech venture capital (VC) fund on the other side of the table. That last part is why I ran this the way I did. I’ve read a lot of decks where the technical risk was retired by assertion, and I didn’t want to be the one writing one.

The idea, in one paragraph

Picture a chip made of tiny mechanical resonators, tuning forks etched into silicon, each one tuned to a different pitch. Sound rings the forks it matches, so the chip sorts frequencies by physically vibrating rather than by digitising and then computing. You train it after manufacture by putting a small fixed (DC) voltage on each fork, which nudges its pitch slightly, and those voltages are the weights. Nothing in the always-on path needs a clock, a converter or a processor, which is how you get to about a microwatt where the digital equivalent needs a hundred or more.

The thesis, in one line

An AI is an ultra-sharp chainsaw. It cuts anything, instantly, in whatever direction it’s pointed, including through the thing you were standing on. The entire skill is the jig.

By jig I don’t mean prompts or model choice. I mean the pre-registered kill criteria, the negative controls, and the self-tests that force a harness to prove its own physics before it’s allowed to report a number.

That is also, I’ve come to think, what entrepreneurship always was. You don’t have an idea and then validate it. You build the apparatus that tells you where you’re wrong, and then you actually let it steer, into the pivots first, and into the kill only once the pivots have run out. Build it at the start, while the only thing it can cost you is your own time. Later it costs other people money, and, far more importantly, years of their lives.

What AI changes isn’t the difficulty. It’s that the old protection is gone, and the old protection was friction. When an experiment costs you three weeks, you think very hard before running it. When it costs forty minutes, you’ll cheerfully run sixty-eight of them straight into a wall and mistake the volume for progress.

Where the chainsaw ran off

Four times over the course of the project, an advantage got named, by the model and by me nodding along, before anyone had done the arithmetic. All four are dated in my log, because keeping the record is most of the point.

You don’t need the physics to follow them. They’re all the same shape. A number that sounded right, said out loud before anybody checked the law that decides it.

The acoustic port. “Worth 10–15 dB.” It was confident, it was plausible, and there was no calculation anywhere behind it. It also sat exactly where it was most convenient for me. When I finally measured it, it was wrong twice over. I’d not only got the size of the effect wrong, I’d been modelling it in the wrong part of the system.

The acoustic delay line. This one got as far as being proposed as a product direction before anybody wrote down that acoustic loss grows with the square of frequency. It does, and that killed it.

Per-channel dynamic range. This is the one I asserted hardest, out loud, before measuring anything. A resonator array ought to shrug off a loud interfering tone better than a digital front end does, I reasoned, because every channel carries its own full scale.

Measured, it came back with the opposite sign. My chip’s ability to ignore a loud neighbouring tone turned out to be worth about seven bits of Analog-to-Digital Converter (ADC) resolution, which means a cheap six-bit digital front end beats my twelve-bit one at the thing I claimed as an advantage.

Worse, the culprit was my most-defended design choice. “Keep Q high”, Q being how sharply each fork is tuned and how narrowly it listens, was a decision I’d justified four separate times on four independent grounds. It is precisely what loses here, because a very narrow listener hears all of a loud tone and only a sliver of the sound you actually want.

“It works where converters can’t exist.” I retracted this one without running anything, after an afternoon’s reading. Commercial ADCs already reach tens of gigasamples per second, so digital can go there. It just costs watts.

That last retraction turned out to be the whole story in miniature, because every advantage I had walked back to energy. Having no interconnect saves memory movement, having no clock saves sampling, and sorting before the converter saves the converter. That isn’t a gap in my search, it’s what physical computing is. And energy on its own doesn’t carry a venture story.

The rule that came out of all four is embarrassingly simple. Check an advantage’s governing law before you name it. It looks obvious written down. It cost me four advantages to get there.

The jig

Here’s the part I’d actually steal, if I were you. Four bugs, each of which produced a clean, stable, self-consistent and completely wrong result.

A physically impossible number that nothing flagged. One run reported 13 dB of gain from a narrow filter fed broadband noise, which simply cannot happen, because a narrow filter throws signal away and can’t manufacture it. The cause was a small modelling slip that had quietly detuned my reference rather than the thing being measured, so the comparison came out inflated. Nothing in my existing defences would have caught it, not a seed ensemble, not a control, not a determinism check, because the harness was wrong consistently rather than noisily. The rule I took from it is that a harness must assert its own physical invariants before it’s allowed to report anything, and in this case that came to three lines of self-test.

A train/test split that was redrawn on every run. Python’s hash() is salted per process, so a data split keyed on it changes silently every time you launch. That quietly mixed five to nine points of noise into every comparison I’d made, including a headline I’d already written up internally. What caught it was a reference moving. The same conventional baseline read 0.916 in one process and 0.828 in another. Physics cannot move an oracle, but data can.

A control arm that was switched off and kept its name. I’d calibrated the units going into a feedback loop and forgotten the gain that consumes them, so the arm ran 150 times under-driven and measured nothing at all, very convincingly. The tell here is a signature worth memorising, which is the same value coming back from experiments that differ in their physics. I varied three separate things across four orders of magnitude and everything landed around 0.6. When your physics changes and your number doesn’t, your code is broken. You have not discovered an invariance.

A headline that was pure luck. One cell in a sweep came back wildly unlike its neighbours, and taken at face value it said the new mechanism beat the simple baseline by 10.7 dB. That was going to be the result. In fact it came out of a harness that had fixed a phase it had no business fixing, which made two signals accidentally identical. Luck alone was worth 11.6 dB, and averaged honestly over phase the mechanism actually loses.

The common thread is what makes these dangerous. None of them look like bugs, they look like results. They’re stable across seeds, they survive a read-through, and they’re the number you were hoping for. The only defence I’ve found is structural, meaning controls, invariant assertions, and writing down what would kill the idea before you get to see the number.

Eighteen rules, each one bought with an error like these. That list is what I’d call the durable asset from all this. The chip isn’t.

What the jig actually bought

The last experiment is the one I’d have paid the whole six weeks for on its own.

Every accuracy number in the project, all sixty-seven before it, had been measured on curated clips against alternatives I chose myself. That’s standard practice, and it’s what everyone in the category publishes.

Then I asked the same features to find three alert sounds hiding among forty-seven other kinds of sound, and got an AUC of 0.589, where 0.5 is a coin flip. The conventional digital control, run identically, managed 0.686. Every one of the 1,880 non-alarm clips triggered at least once.

And for an always-on part this isn’t an annoyance, it’s the product, because every false wake-up spends power. My own power arithmetic says the budget cannot afford even five false alarms a day, and at that rate the wake-ups already cost up to 4.6× my entire 1.25 µW budget. One side of this is five a day. The other is everything I put in front of it. My single advantage, spent entirely on being wrong. No version of the chip survives that.

Which gives the rule I’d hand to anyone building in this space. A classification score is not a detection score, and only an open negative set tells you which one you’re holding. Nobody in always-on audio publishes the second number, and I now understand why.

What I think this means

Deep tech doesn’t get easier with AI. It gets more mature.

The bottleneck was never ideas. It was that each real check cost weeks, so you rationed your checks, so you raised money on a thesis you’d only partially tested and discovered the rest at Series A with a fab bill attached. That was never recklessness. It was economics.

Those economics have changed. I ran 68 falsification attempts against my own idea in about eight working days, and most of them killed something I had said out loud. That is a cheap way to be wrong, and being wrong cheaply is the whole game.

The binding constraint, interestingly, was none of the things I expected. Not compute, not ideas, not even my own time. It was token budget, meaning how much of the problem I could hold in front of the model at once. That sentence would have been meaningless two years ago, and it’s now the real limit on how fast a physics idea can be falsified.

If you write cheques for a living, here’s the part I’d think about. A founder who turns up with 68 measured negatives, four of them retractions of their own claims, should be priced differently from one who turns up with a deck. Not because they’re smarter, but because the failure has moved from “discovered after the raise” to “discovered before it.”

There’s a fair objection to all this, which is that a machine tuned to kill things will eventually kill everything, and that pessimism dressed as rigour is still pessimism. My best answer is that this project was itself killed once, wrongly, by my own early triage, and then restored two days later when the reasoning didn’t survive a second look. The kill criteria have to get audited too, or they’re just pessimism with a spreadsheet.

I’d rather be the founder who can hand you the log of everything that didn’t work than the one with the clean story. My old job was reading the clean stories.

What I’d steal from this

Pre-register the kill. Write down the number that ends the idea before you run the experiment. If you can’t name that number, you aren’t running an experiment.
Check the governing law before you name an advantage. This one cost me four.
Report every experiment against an oracle, meaning the best possible version of the conventional approach, scored on the same data. Absolute numbers mean very little alone.
Run negative controls against trained cells. A control that’s weaker only because nobody bothered optimising it measures your optimiser, not your architecture.
The same number coming out of different physics means your harness is broken. I’ve yet to see an exception.
Measure the open-set number before you quote anything to a customer. The closed-set number flatters systematically, and it’s the one your whole category publishes.

Disclosure

I ran this project with Claude, which did the planning, the simulation code, the analysis and the write-ups. The first draft of this post was written by it too, and then rewritten by me. It would be a strange post to be dishonest in.

The work is real and the numbers are real, and so are the four times the model told me something was true before it had checked. That’s rather the point.

The repo and the full rule set stay private, that part is still my edge. Everything above is real, and I’m happy to argue about any of it.