Holiday Measurement After GA4: What We Can Still Trust

Universal Analytics stopped collecting data on July 1, which makes this the first peak season the entire industry runs on GA4, and I do not think most marketing teams have absorbed what that means. The numbers on the dashboard look like the old numbers. They are not the old numbers. Every year-over-year comparison being screenshotted into a Q4 deck right now is comparing two different measurement philosophies, and somewhere a budget decision is being made on the difference.

We spent the autumn rebuilding measurement for our retained clients around this reality. Here is the field guide.

Why your YoY is lying to you

The core issue is that GA4 changed definitions, not just interface. Sessions are constructed differently and typically come in lower. Engagement replaced bounce, and the two are not inverses of each other despite every dashboard treating them that way. Conversions counted per-event rather than per-session inflate against last year's numbers. And the default attribution moved to a data-driven model, quietly reassigning credit across channels compared to the last-click views most teams lived in for a decade.

None of these are wrong. Several are improvements. But stacked together, a retailer comparing November 2023 to November 2022 is holding a ruler that changed length mid-measurement. For Northwind we built a translation layer: three months of parallel-period analysis to establish conversion factors between the old and new definitions, documented in the reporting so nobody in a Monday meeting trades on a phantom trend. Tedious, unglamorous, and it has already prevented one "email is collapsing" fire drill that was actually an attribution model doing its job.

The deeper shift: from observation to modeling

The uncomfortable adjustment is philosophical. UA-era analytics let everyone pretend we observed user behavior directly. GA4, with consent mode, behavioral modeling, and sampling in exploration reports, is honest about being partially a statistical estimate. Add the cookie deprecation Chrome has now scheduled to begin in early 2024, plus the consent tightening arriving with EU Digital Markets Act enforcement in March, and the direction is unmistakable: the era of counting individuals is ending and the era of modeling populations is here.

Our response has been to diversify what we trust, tier by tier:

• Ground truth: platform-independent numbers. Orders, revenue, margin from the commerce backend, and first-party email and SMS engagement. These anchor everything.
• Directional: GA4 and ad platform metrics, read as trends against their own recent baseline, never as absolute truths and never across the migration boundary.
• Strategic: incrementality tests and geo holdouts, which we finally got two clients to fund this year, because when attribution is modeled, experiments become the only honest arbiter of what spend actually causes.

What to do during peak, specifically

Practical guidance for the next six weeks. Decide before Black Friday which numbers trigger which decisions, and write it down; a definitions debate on November 24 is a lost day of trading. Watch revenue and blended cost per order daily, and channel-level attribution weekly at most, because the models get twitchy during traffic spikes. Annotate everything: promotions, site changes, deliverability events. GA4's data will make no sense in the January postmortem without a diary of what you did to it.

The teams that struggle this season will be the ones demanding UA-era certainty from a post-UA world. Certainty is gone, and it was always partly an illusion anyway. Judgment under honest uncertainty is the actual job now. That is less comfortable and considerably closer to the truth, which is generally the trade worth making.