sethserver.com Subscribe
A blue cartoon robot with extended arms strikes a dynamic pose on a tan platform against a coral/salmon pink background, with a stopwatch showing on the right side and decorative stars and geometric shapes scattered around. A loading bar appears at the bottom of the image.

How to Not Waste Time in a Programmer Interview

By Seth Black • Updated: September 17, 2026

Startups ยท 4 min read

Like this kind of writing? Get one email a week with notes on startups, AI, and the occasional strong opinion about Python: subscribe to the newsletter.

I stopped trusting coding quizzes the day I hired someone who aced ours and then couldn't write a single function on the job. Not one.

They passed the technical screen. They passed the personality interview. They said all the right things. Then I gave them actual work. Nothing came back. I tried pairing with them and they froze completely. Not "nervous." Not "needs a minute." Just... stuck.

After a little digging, the story got simple: they couldn't program at all. They'd somehow gamed the quiz, got themselves into a "cool startup," and figured they'd learn on the job before anyone caught on. Spoiler: they didn't.

That was the last time I used a quiz as my main filter.

The two kinds of "code quizzes" that waste everyone's time

1) Brainteasers

You know the ones. "How many golf balls fit in a 747?" These questions mostly measure whether someone has practiced that exact kind of question. Google famously said they're a waste of time. I believe them. I've also watched interviewers ask them because it lets the interviewer feel clever for five minutes.

If you want to know if someone can ship Python, asking them to estimate the mass of the sun tells you absolutely nothing.

2) LeetCode-style algorithm screens

LeetCode screens aren't completely worthless. They're just terrible at predicting who can actually do the job.

They're good at finding people who memorized 200 LeetCode problems. That's it.

They also select for "interview athletes": people who spent three months grinding algorithm problems that have zero overlap with your actual codebase. Meanwhile, solid engineers who get anxious while someone watches them type can look worse than they are.

And you can run a "fair" algorithm screen and still miss the only question that matters: can they do the actual job you're hiring for?

Most startup work is not "reverse a binary tree in 12 minutes." It's "read an ugly codebase, find the real bug, fix it without breaking five other things, and leave it cleaner than you found it."

What I do instead: a conversational interview with hard edges

I like structure. I just don't like theater.

The best setup I've found is simple. Two parts:

  1. A competency scorecard
  2. A code review

That's it.

Part 1: Competency scorecard (but keep it human)

I keep a short list of topics I actually care about: Python basics, debugging, logging, exceptions, and AWS if the job needs it.

Then I ask questions that give people room to show what they know.

Example:

"Half the repo uses print(), the other half uses logging. How would you fix that without breaking everything?"

A solid Python dev will talk about log levels, handlers, formatting, migration plans, and incremental cleanup.

Someone who's faking it will say "yeah I'd standardize it" and pray you change topics.

I'm not trying to interrogate anyone. I just want enough conversation that real experience becomes obvious.

And yes, I fill out the scorecard right then. If I wait, my brain will invent a completely different interview. It always does.

Part 2: The code review (my favorite lie detector)

Code reviews work better than almost anything else I've tried.

They're collaborative. They're bounded. They look like real work.

I keep a small rotation of short Python scripts that do normal things:

Then I ask the candidate to review it with me.

What I'm actually looking for:

This catches something specific: people who can talk but can't ship.

I worked at a SaaS startup once that sold "AI-powered" financial automation. After a few weeks I asked to see the AI. They kept dodging. Finally they flew me to an overseas office to "see the team work." The AI was people copying data between spreadsheets. That's it.

Code review has a nice property: if the product is real, it holds up. If it's smoke and mirrors, that shows up too.

The checklist: how to stop burning time

Here's what I try to hold myself to:

Hiring is messy. People lie, people freeze up, people game the system. But you can still build something that works. Stop asking trivia questions. Start looking at whether they can actually do the job.

-Sethers

Share this post
Newsletter

One email, once a week.

Notes on databases, systems, and the occasional strong opinion about Python. No spam, unsubscribe anytime.

Seth Black
Written by

Seth Black

Engineer and founder based in Texas. Writes about databases, AI, and running things in production. Embeds in small teams as lead engineer.

More from Startups

View all โ†’
Startups

What Actually Goes in a Consulting Contract

Sep 17, 2026
Startups

Productizing a Service: From Hours to Licenses

Jul 26, 2026
Startups

The MVP That Is Not a Product

Jul 11, 2026