We Almost Published the Wrong Cold Email Number. Here's How We Caught It.
Updated: Oct 1
We built a version of this article last week comparing three cold email hook types by reply rate. Before it went out, a second read caught a problem: the number we were using for our own campaign's reply rate didn't match our own records. This is the article that replaced it, because the catch turned out to be a better example of the thing we were trying to say than the original piece was.
What happens when your own reports disagree with each other?
We ran into this on our own cold email program. An earlier internal progress report put our reply rate materially higher than the number we could ultimately stand behind. When we went back to the source — a full platform export we could check line by line — the audited result was 53 replies on 17,570 sends, or about 0.30 percent.
That wasn't a rounding difference. The earlier figure and the audited one told materially different stories about the same program, and only one of them came with a reply count and a send count we could set side by side. That's the one we kept.
Bottom line: when two of your own numbers disagree by that much, at least one of them is wrong, and you don't get to pick the one you like better.
Which number is right, and how do you find out?
You reconstruct the denominator, or you don't use the number.
We went back to find out where the higher number came from. It traced to an earlier progress report, and we couldn't reconstruct exactly what it was measuring against — whether it counted replies over a different date range, folded in auto-replies, or was measured against opens rather than sends. Without that denominator, the figure isn't defensible, no matter how many other documents had already repeated it by the time we checked.
The 0.30 percent figure is different. It comes from a platform export with a reply count and a send count sitting next to each other, both auditable, both from the same source, and covering the whole program rather than a number lifted from a summary. That's the one that survives the check, and it's the only reply rate this article uses going forward.
My take: a number that's been repeated in five internal documents isn't more trustworthy than a number repeated once. Repetition isn't verification. Only the export is.
Why does this matter more for this particular article than almost any other topic?
Because this article is about measuring cold email honestly, which means it has less room than almost anything else we write to get its own measurement wrong.
We've made this point before in this same cluster:
CRM numbers are self-reported and nobody audits them, and a client's own pipeline turned out to be inflated by 40 percent once someone actually checked each record by hand. The lesson there was to audit the numbers you're being handed before you build a budget or a forecast on top of them. Publishing an unverified number about our own campaign, in an article warning readers to verify theirs, would have made us the example instead of the source.
Bottom line: if you're going to tell someone else to check their denominator, check yours first.
What's a defensible way to compare your own number to a published benchmark?
Only after you've confirmed what your own number actually measures, and even then, expect the published benchmarks to disagree with each other by a wide margin.
Instantly's 2026 benchmark report, built from billions of cold email interactions across its platform, puts the average reply rate at 3.43 percent, with a top quartile above 5.5 percent and an elite tier clearing 10 percent. Belkins' 2026 study, drawn from 7.5 million emails sent to net-new contacts with a strict replies-over-total-sends denominator, puts its 2025 average at 0.45 percent, about 7 times lower than Instantly's. Both studies are real, sourced, and correctly cited. They simply measure different populations with different methodologies, which is the same lesson from the section above showing up again at the benchmark level.
Against that range, our audited 0.30 percent sits just under the Belkins net-new average and well below Instantly's platform-wide number. We're not going to round that up. It's a below-target result on a program that needed better targeting or a stronger offer, and saying so plainly is the whole point of this piece.
Bottom line: don't adopt a single external number as your target before you know what it's measuring, and don't round your own number up to make a benchmark comparison flattering.
What should you actually do before you publish or act on any reply-rate number?
Three checks, in order, before the number goes anywhere near a report, a proposal, or a public article.
First, confirm the denominator. Reply rate means replies divided by emails sent, nothing else. If a document doesn't say which, treat the number as unverified until you find out.
Second, find the export, not the summary. A summary is someone's interpretation of the export, written at some point in the past, and interpretations drift the more times they get copied into a new document. The export doesn't drift.
Third, if two of your own numbers disagree, that's not a rounding issue to average away. It's a signal that one of your reporting processes is broken, and the fix is worth more than the number you were trying to publish in the first place.
My take: the discipline that catches a wrong number before it ships is the same discipline that makes a campaign worth running. Neither one is optional.
Which hook actually performed best: Timeline, Numbers, or Problem?
Not every hook pulls the same weight. Across the same audience and the same offer, the three openers we tested landed in a clear order - and the gap is wider than most teams assume.
The Timeline hook led at 10.01 percent open rate: a specific, time-bound promise gives the reader a reason to look now. The Numbers hook followed close behind at 8.57 percent - a concrete figure signals substance before the pitch arrives. The Problem hook trailed at 4.39 percent, less than half the leader; naming the pain without a payoff attached gives the reader less reason to open.
The lesson isn't that one hook always wins. It's that the opener is a variable worth testing deliberately, not a throwaway first line. On this list, moving from the weakest hook to the strongest more than doubled the open rate - before a single word of the body copy changed.
Bottom line: test hooks on the metric they actually control, and name that metric every time you quote the result.
The pattern underneath all of it
Every number in this piece measures something specific, and none of them are interchangeable with the number next to it. A progress report isn't a platform export. An open rate isn't a reply rate. A platform-wide average isn't a strict net-new average. Treating any two of these as the same thing is where cold email "results" go wrong, and it happened to us before it happened to a reader.
Bottom line: we caught it before it published. That's the standard, not a bonus.
Related: B2B Lead Generation. How we structure outbound campaigns, including the reporting checks that catch a bad number before it reaches a client or a public page.
Want your own cold email numbers audited before you act on them? We'll check what your reply rate actually measures before you build a decision on top of it.
FAQ
What's the difference between open rate and reply rate in cold email?
Open rate measures how many delivered emails got opened. Reply rate measures how many sent emails got a response. They use different denominators and answer different questions. Neither one substitutes for the other, and a report that doesn't say which one it's showing you should be treated as unverified.
Why did two internal reports on the same program show different reply rates?
An earlier progress report stated a figure whose exact denominator could no longer be reconstructed. A later platform export — reply count and send count both visible and auditable in the same place — showed a materially lower number: 53 replies on 17,570 sends, or about 0.30 percent. When your own numbers disagree, the auditable one is the figure to use, not the one that's been repeated most often.
What's a good cold email reply rate in 2026?
Published benchmarks vary widely by methodology. Instantly's 2026 platform data puts the average at 3.43 percent with elite senders above 10 percent. Belkins' 2026 study, using a stricter replies-over-total-sends measure on net-new contacts, puts its 2025 average at 0.45 percent. Compare your own number to a benchmark that shares your denominator, not just the label "reply rate."
How do I check whether a reply-rate number I've been given is accurate?
Ask for the export, not the summary: a reply count and a send count from the same source and the same date range. If you can't get both numbers from one place, treat the percentage as unverified until you can.
What should a company do before publishing performance numbers about its own campaigns?
Reconstruct the denominator, source the number from an export rather than a prior summary, and treat any disagreement between two internal figures as a reporting problem to fix, not a discrepancy to average away.
Which cold email hook gets the highest open rate?
In our January 2026 test the Timeline hook, which opens on a date or a deadline, took a 10.01 percent open rate, against 8.57 percent for the Numbers hook and 4.39 percent for the Problem hook. Open rate measures whether the subject line earned the open. It is not a reply rate, and this test does not say which hook produced more replies.
