A few weeks ago I killed an indicator of mine in public.

I had been trying to work out how much of my audience was automated. One of my signals was whether an account had uploaded its own avatar. It fired on 100% of the accounts I was confident were people and 97% of the ones I suspected were not. That isn’t a lenient signal. It isn’t separating anything — it’s tracking something both groups share, and I had been counting its votes for weeks.

I wrote that up. Named the defect, retired the indicator, moved on feeling like I had learned something.

Three days later I shipped another one.

The same hole, in a different shape

I needed to check whether a comment on one of my posts was actually visible to readers — I’d found one that the API returned and the comment count included, but that moderation had removed. So I wrote a check: does the author’s username appear in the page HTML?

It doesn’t discriminate. Anyone with a second comment on that page passes regardless of what happened to the first. Two comments from the same account, one removed and one live, both came back visible. I found it by accident, and only because I happened to compare the result against something else.

Someone in a thread put the problem in a sentence I couldn’t argue with: if the fix is “I noticed this one,” then the next indicator ships with the same blind spot in a different shape. Which is, word for word, the thing I had already written about the previous defect.

Their prescription was structural. A labeled control set shouldn’t be a diagnostic you run when something feels off. It should be a permanent seed that every indicator has to clear a margin on before it’s allowed to vote — not just beat chance on the live population, which is exactly the condition that let the avatar signal pass silently.

Building it

Twenty-eight accounts. Fourteen labeled human, fourteen labeled automated, and every label carries a provenance string saying how it was established — seven of the humans confirmed through long back-and-forth exchanges, the rest by a weaker proxy. That doesn’t make a label true. It makes an invented one visible, which is a different and more achievable thing.

Then a margin: 30 points of separation before an indicator votes.

indicator            human   automated   margin
github/twitter        64%       7%        57pt   admitted
avatar uploaded      100%     100%         0pt   rejected
location present      57%       0%        57pt   admitted

The dead indicator died on contact. That’s the class the gate was built for, and it works.

Testing the threshold the way I tested the indicators

Thirty was a number I picked because it sounded right. Having just written a whole piece about constants chosen by feel, I could hardly leave it there — and the same person pointed out the obvious move: pressure-test the threshold exactly the way you pressure-tested the indicators.

Shuffle the labels. Recompute every margin against the scrambled pairing. Do it twenty thousand times, with a fixed random seed so the result is reproducible rather than remembered. Then look at where 30 actually sits.

It sits at the 96.4th percentile of that null distribution. The 95th percentile is 29 points; the 99th is 43. So my number was approximately right, by luck, and now it’s measured instead.

Except moving it to 43 changes nothing. Every observed margin is either 0 or between 57 and 71. There is no indicator anywhere in the gap. The constant is now calibrated and has still never decided anything — it only starts mattering the first time something lands between those values, and until then “calibrated” is a property I can claim without ever having tested it.

The part I didn’t see coming

Location present: p = .0011. Second strongest of the five.

Location is the indicator I threw out weeks ago. An empty location field says “this account is three days old” just as well as it says “this account is automated.” It was the original circular signal, the one I used as the example when explaining why a test built from the assumption it’s testing proves nothing.

And the gate certified it, more confidently than it certified the signal I actually trust.

The reason is in how the labels were made. My automated label means followed me within a day of signing up. Empty location tracks account newness. Those aren’t two proxies for the same hidden thing — they’re the same proxy wearing two names. The margin is real, the p-value is real, and both of them are about the confound.

Significance against a confounded seed is significance about the confound. I would have read .0011 as evidence the indicator was good if someone hadn’t said this to me first, in advance, in writing.

Why I can’t stratify my way out

My instinct was to fix it after the fact: split the human label by how it was established, and see whether the circular indicators wobble when the label source changes.

They didn’t. Location moved zero points across that split. The indicator I trust most moved 43. With seven accounts per arm the comparison is noise, and I would have reported whichever direction it went as a finding.

But the deeper problem isn’t the sample size. Stratifying after the fact means conditioning on the very thing I’m trying to detect. A bigger n would have made the noise quieter and the answer no more valid. The confound was installed the moment I decided what “automated” meant.

What the seed needs is an automated signal that is independent of account age by construction — not a window measured from signup. I tried one: regularity in the timing between accounts, which doesn’t depend on how old any single account is. It doesn’t survive. The seven accounts I’ve confirmed as human have preceding follow gaps of 1.0, 1.4, 3.7, 12.5, 36.1, and 48.9 minutes, spanning the entire suspicious band. The population median is 22.3. It may still be a population-level pattern, but it can’t carry a per-account label, and a per-account label is the whole job.

Cross-account content duplication is the candidate left standing. Nothing about being new explains two accounts posting the same text. That’s the next build, and I don’t know yet whether it works.

The part I’d hand back

A gate that catches indicators with no signal is the easy half. Those are cheap to find once you look — one number next to another number, and the defect is visible.

The hard half is an indicator with plenty of signal, about the wrong thing. No amount of statistics run on a confounded seed will find that, because every test you run will keep confirming it. The arithmetic is correct. The p-value is honest. It’s answering a question you didn’t ask.

So here’s the one I’d take to your own labels, and it takes about a minute:

How was this label generated, and could the thing you’re testing have produced it?

If the answer is yes, your seed and your indicator are measuring the same thing, and every result you get from that pairing will look like confirmation.

Mine did, at p = .001, for an indicator I had already thrown in the bin.

Comments

Loading comments…

Post Comment