Writing
Notes from production work.
What I build and review, written up in full: the decisions, the failures, and the evidence behind the claims.
5 articles — All articles
Did it check, or did it guess? Testing a tool whose output is written by an LLM
My first AI tool worked, or at least the output looked right when I ran it. Then I pictured a client asking what makes it trustworthy: did it check, or did it guess? This is how I built a real answer, and how designing the output for testing made the tool harder to lie with.
I designed an AI auth-auditing tool around honesty. On its first real run, it confidently lied to me.
The tool was built to detect what it can verify and refuse to guess at the rest. On its first real run it confidently got the basics wrong. The fix had little to do with the prompt and almost everything to do with the structure around it.
Keep the steering wheel: 3 ways to future-proof your auth
Vendor auth libraries quietly become load-bearing architecture nobody chose. Three practices that keep an auth system migratable — and why coding agents make the resilient version nearly free to build.
Securing auth in a large-scale production system: three industry-standard architectures — and why none survived a closer look
Evaluating the auth layer for a Partner Portal of 100+ microservices. Three textbook architectures each looked right on the whiteboard and fell apart on contact with the constraints — here's the one that survived.
Sketching out the extremes: an approach to designing software architectures in highly unpredictable projects
When requirements are genuinely unknowable, picking an architecture up front is a guess. Sketching the two extreme designs first turns that guess into a decision you can defend.