Field Notes/bryngtmpersonalizationsignalsmeasurementseries-personalization
The Moment, Measured
Open rates lie to you about all six stages at once. Part 3 of 3 on personalization closes the series with the measurement: four numbers that survive contact (moments seen, moments cleared, touches landed in-window, accounts that moved), why time-to-touch is the master metric, and how the record turns measurement from a project into a query.
Brad Webb, Chief Growth Officer
||10 min read|
tl;dr
Open rates lie to you about all six stages at once. They measure the channel, not the moment. This is part 3 of 3 on personalization, the measurement: four numbers that survive contact (moments seen, moments cleared, touches landed in-window, accounts that moved), why time-to-touch is the master metric, and how a per-run record turns measurement from a quarterly project into a query.
The moment from part 2 is back on the bench.
pricing → comparison → repeat (7d). Three visits, one account, seven days. In part 2 I walked through the six stages between that signal firing and a relevant touch landing: signal, identity, score, Play, bounds, record. I ended on a promise: part 3 would be the proof. What to count, what to ignore, and why open rates will lie to you about all six stages at once.
This is that note. It is shorter than you would expect, because the honest version of measurement usually is.
The open-rate lie
Here is what took me too long to admit. For about a year, the best-looking number in our funnel review was a 61 percent open rate on one nurture sequence (illustrative of the shape, not a benchmark). It went up almost every month. I kept reporting it, partly because it was the only number that reliably went up.
That sequence sourced nothing. Not one meeting I could trace. The open rate was measuring our subject lines, our sender reputation, and the aggressiveness of Apple's mail privacy proxy. It was not measuring whether we had found a real moment, resolved the right account, scored it honestly, or landed a touch while the case inside that account was still open.
Run an open rate against the six stages and watch what it can actually see. Whether the signal was real: no. Whether the identity was right: no. Whether the score was honest: no. Whether the Play was the right response: no. Whether the bounds held: no. Whether the touch landed in the window: also no. An open tells you a rendering event probably happened on a device that may belong to a person. That is the entire claim.
Metric autopsy: what can it actually see?
What each metric can see across the six stages
Metric
Sees
Verdict
Open rate
None of the six stages
A rendering event on a device. That is the entire claim.
Reply rate
Fragments of identity and Play
A human decided something. Not which moment, not whether the window was open.
MQL count
A fraction of stage one
Form fills against a definition nobody audits.
Time-to-touch
Signal, Play, record; the rest by implication
Every unbuilt stage shows up as elapsed time.
Moments cleared
Stages one through five
Every stage had to pass for this number to increment.
14-day movement
The outcome, via the record
The only outcome here; uninterpretable without fired, cleared, landed.
01 Signalblind
02 Identityblind
03 Scoreblind
04 Playblind
05 Boundsblind
06 Recordblind
Open rate sees a rendering event on a device that may belong to a person. None of the six stages. That is the entire claim.
Pick a metric (arrow keys work). Stage visibility per metric. Qualitative, not a benchmark.
Every channel metric has this problem to some degree. Reply rate at least involves a human decision, which is why it survives longer under an autopsy than opens do. But it still cannot tell you whether the reply came from the account whose moment you were answering, or three weeks after the moment expired.
Channel metrics measure the channel. If the last two parts of this series argued anything, it is that the channel was never the scarce thing. The moment is.
What the vendor counts is not what you count
Chris wrote yesterday about what your vendors started counting, and his test for a pricing meter is the right test for your own metrics too: does the number move when the value moves?
A vendor's meter answers "what am I paid for." Your funnel metrics answer "what is working." Different questions, same discipline. Most teams apply real rigor to the first (usually at renewal, under duress) and none to the second. Then they optimize the numbers that were cheapest to collect, which is how a 61 percent open rate ends up in a board deck.
Measure the stages, not the channel. Four counts, in order, each one exposing a specific part of the chain from part 2.
Moments seen. How many times did a pattern you named actually fire inside its window? pricing → comparison → repeat (7d) either fired eleven times last week or it did not. This number exposes stage one: if it is zero, you do not have a signal problem, you have a definition problem. Nobody wrote down what counts as a moment.
Moments cleared. Of those, how many made it through all six stages: identity resolved above your confidence bar, score over your threshold, an approved Play matched, bounds checked. Cleared-to-seen is the health of your machinery. This is the number I would put on the wall instead of the MQL count.
Touches landed in-window. Of the cleared moments, how many produced a touch while the moment was still a moment. Not sent. Landed, inside the window the pattern defines. This exposes the gap between having an execution layer and having a queue.
Accounts that moved. Within 14 days of the touch: a reply, a meeting, a return visit, a second person from the account showing up. Movement, not opens. This is the only one of the four that resembles an outcome, and it is deliberately last, because it is uninterpretable without the three above it.
Fired, cleared, landed, moved. Four integers. If you can produce them for one Play, you understand your motion better than most teams with a forty-tile dashboard.
The master metric
If I could keep only one number, it would be time-to-touch: the elapsed time between the signal firing and the touch landing.
Part 1 made the argument for why: the value of a response decays with the case that prompted it. Somebody inside that account is building a case or losing an argument, and the window is measured in hours. A perfect touch that lands on day four is a different, worse product than a decent touch that lands in twenty minutes.
Time-to-touch vs the moment
Touch lands
Where that is
+20 minutes
The case is still open on their screen. You joined the conversation.
+26 hours
Same week, cooling. You are re-opening the case, not joining it.
+3 days
The relay-race number. Somebody else's touch got there first, or nobody's did.
touch lands +36 hours
You are re-opening the case, not joining it. Somebody else's touch may have landed first. (illustrative, not a benchmark)
All values illustrative, not a benchmark. The window itself is whatever your pattern defines.
Time-to-touch is also the most honest summary statistic of the whole chain, because every stage you have not built shows up in it. No signal definitions: infinite. Manual identity resolution: add a day. Scoring by standup: add another. Improvised responses: add two more. Our old chain, the signal → Slack ping → spreadsheet row → standup → touch relay I confessed to in part 2, had a time-to-touch of about three days on a good week. The number was embarrassing, which is exactly why we never measured it.
The ledger
Here is the part that makes all of this cheap instead of heroic.
If every run writes a record (what was seen, why it qualified, what ran, what happened) then the four numbers are not a measurement project. They are a query. I ran pricing-follow-up-v2 against acct_4471 plus the signal, the window, the score against the threshold, the channel, and the outcome: sum those lines by week and the ledger falls out. Fired is a count of signal lines. Cleared is a count of run lines. Landed is a timestamp subtraction. Moved is a join against what the account did next.
Record grammar per the Bryn audit log; accounts invented; all values illustrative, not a benchmark.
We rebuilt our own funnel review this way, on our own funnel, and the first honest version took an afternoon, because the log already had everything. The dashboards we retired took longer to cancel than the ledger took to build. I made the general form of this argument in Proof of Claims: the record is not overhead on the work, the record is how you know the work happened.
This is the job Bryn does with the numbers. Bryn is not another dashboard to watch. It is the governed execution layer that runs Plays through your stack, and every run it executes writes the line the ledger needs: what was seen, why it qualified, what happened. The measurement is not a feature bolted onto the agent. It is the exhaust of doing the work correctly.
Run the count
The gift, this time, is a week of honesty. Pick one Play if you have one, or one manual motion if you do not. For one week, count four things: how many times your named pattern fired, how many of those cleared every stage, how many touches landed inside the window, and how many accounts did anything at all within 14 days. Ignore opens entirely. You are not allowed to look at them.
Then compare cleared to fired before you touch a single line of copy. In my experience the copy was never the problem. The chain was, and the four numbers point at the broken stage with a specificity no channel metric will ever give you.
Part 3 of 3
That closes the series. Part 1 was the why: tokens are free, moments are scarce, and {FirstName} was never personalization. Part 2 was the how: six stages between the signal and the touch, each one a place teams silently fail. This was the proof: four numbers that survive contact, one master metric, and a ledger that makes the counting free.
The moment is still not exotic. Some version of pricing → comparison → repeat (7d) fired against your funnel this week, and now you know what it has to clear and how to check whether it did.
Brad Webb is the Chief Growth Officer at Civic; he's been building the bridge between Engineering and GTM/Sales for over two decades, merging them into the science better known as Growth.
If Brad isn't running experiments or sending off Agents to verify data, he's probably building tube-based HiFi gear with his sons, hopefully remembering to drain the capacitors before soldering.