Subscribe
Share
Share
Embed
Featuring "The Vanishing User: Web Analytics in an Agent-Dominated Internet" · 5 top papers
Papers about Media Measurement
Episode covers 2026-05-06 – 2026-05-13.
Themes: social media, consumer behavior, digital media, social media marketing, influencer marketing, student engagement, digital transformation, sustainability
Methods: survey, qualitative, quantitative, case-study, content analysis, cross-sectional
The premium version of this podcast covers all 40 papers selected for the episode. Subscribe to the premium podcast.
Generated by paperboy.fm.
This Week in Media Measurement tracks research on how media, platforms, and marketing are measured, from social media and web analytics to campaign evaluation, audience behavior, AI-driven content, and privacy-preserving methods.
Subscribe for the premium version of this podcast: https://paperboy.fm/podcasts/media-measurement/subscribe
Jenny: If a website says someone clicked, how sure are we that a real person actually meant to?
Davis: I'm less sure than the dashboard is, because a click can be a thumb, a bot, or now an AI assistant doing errands in the background.
Jenny: Exactly, and I don't trust any chart that treats every click like a tiny human confession when so much online activity is automated traffic in a trench coat.
Davis: So the next question isn't just how many visits happened, it's who or what acted, and whether that action changed anything a person cared about.
Jenny: That's the measurement problem this week, from vanishing users to agent-aware analytics, which just means counting the actor behind the signal...welcome to This Week In Media Measurement on paperboy.fm.
Davis: This week is bigger across the board: about sixteen hundred query hits, one hundred seventeen qualified papers, three hundred forty-nine unique authors, and thirty-one countries in the mix.
Jenny: Qualified papers rose from ninety-five to one hundred seventeen, so that's twenty-two more papers, up 23.2 percent, and the first thing I want to know is whether that growth is signal or just a wider net.
Davis: The net did widen: query hits went from one thousand three hundred seventy-five to one thousand six hundred forty-two, up two hundred sixty-seven, or 19.4 percent, while the semantic shortlist stayed fixed at two hundred, meaning the system still only kept two hundred papers for closer reading.
Jenny: So the filter got more selective, and the mix underneath is very survey-heavy: thirty-three surveys, twenty-three qualitative studies, and twenty quantitative papers, with social media at twenty-three papers, consumer behavior at ten, and digital media at eight.
Davis: That fits the through-line: measurement here is less about counting who saw media and more about asking what the signal represents, whether it's a survey response, a consumer action, or a platform trace.
Jenny: The author pool jumped from two hundred fifty-five to three hundred forty-nine, up ninety-four authors, or 36.9 percent, and countries rose from twenty-seven to thirty-one, led by Indonesia with ten papers, the U.S. with nine, and China with eight.
Davis: And the author tiers make this feel like an open week: eighty-four first-time authors, meaning their first-ever paper in the metadata, one hundred seventy-six emerging authors, and eighty-nine experienced authors, so half the field here is early-career rather than the same senior names repeating.
Jenny: Alright, let's get into the papers with a big measurement problem hiding in plain sight. Babu George and Divya Choudhary have a twenty-twenty-six position paper in Information called The Vanishing User: Web Analytics in an Agent-Dominated Internet, and the basic claim is that the old unit of web analytics, the human user, is getting shaky.
Jenny: Plain version: a click may no longer mean a person wanted something. The authors argue that crawlers, bots, AI agents, LLM-powered agents, and more autonomous agents can all create web traces, and they flag three things that make LLM agents especially messy: identity discontinuity, meaning they don't keep one stable identity; task-based instantiation, meaning they appear for one job and vanish; and agent-to-agent loops, meaning one system may be talking to another with no human in the moment.
Davis: So if a click might come from a person, a bot, or an AI agent acting for someone, what can a publisher still safely infer from it?
Jenny: Not as much as dashboards usually imply. This isn't an empirical test with a measured share of traffic; it's a conceptual roadmap that synthesizes work on bot detection, agent architecture, web measurement validity, automated-system governance, and digital trace data, then proposes five measurement primitives: task chain, actor class, interaction provenance, objective alignment, and signal authenticity.
Jenny: In normal language, they want analytics to ask what task produced the trace, what kind of actor made it, where the interaction came from, whether it matched a human goal, and whether the signal is trustworthy. The limitation is real, though: the paper names the measurement problem more than it proves how big the problem is.
Davis: That still feels like the right opening frame for this week. If AI scrambles signals, then engagement and conversion can't just be treated as little pieces of human intention anymore; they need actor labels before anyone builds a newsroom plan, an ad report, or a product decision on top of them.
Davis: That actor-label point is a nice bridge, because this next paper asks what happens after a signal gets believed by real newsrooms. S. Dvir-Gvirsman and Lidor Ivan call it Contextual gatekeeping in a platformized news ecology, meaning editorial choices made inside platform rules, dashboards, and audience feedback loops.
Davis: The plain finding is that audience engagement did predict more coverage later, but not evenly. Across two and a half years of data from thirty-nine English-language outlets in the United States, the United Kingdom, Canada, and Ireland, the strongest bump showed up on Facebook pages of digitally born outlets, weaker on Facebook pages of legacy outlets, smallest on legacy websites, with digitally born websites in the middle.
Jenny: How did they know engagement came before the extra coverage, rather than just showing that a topic was already getting hotter and everyone was chasing it?
Davis: They modeled whether engagement in a topical beat in the prior month predicted later story counts, split by platform and by outlet lineage, so the timing is built into the test. That gives this more weight than a vibes-based dashboard story, especially with thirty-nine outlets across four countries from twenty-seventeen to twenty-nineteen, but it's still a historical baseline from a social-media-dependent era, not automatically a map of subscriber-first newsrooms today.
Jenny: So the takeaway isn't, engagement controls journalism. It's sharper than that: this is the When Metrics Push Back thread, because the metric seems to push hardest where the newsroom is already organized around the platform, and weakest where the website and legacy routines still give editors more room to say no.
Jenny: That last paper treated engagement as a signal that can push a newsroom around, and this one asks what happens when the platform changes the meaning of the signal itself: Stephan Carney, Ignacio Riveros, and Stephanie M. Tully have a Journal of Consumer Research paper from twenty-twenty-six called Made With AI: Consumer Engagement with Social Media Containing AI Disclosures.
Jenny: The plain version is pretty sharp: when social posts carry an AI-generated-content disclosure, people engage less, not mainly because they think the content is worse or because they hate AI, but because the creator feels less personally present. The authors call that reduced parasocial connection, meaning the one-sided emotional bond a viewer feels with a creator, and they tie part of it to perceived effort.
Davis: So is this really about AI, or is it about people feeling like the creator did less work?
Jenny: They try to separate those with two kinds of evidence: real TikTok engagement after TikTok introduced its AI-generated content disclosure policy, and eight preregistered experiments, which means the tests were planned before the data were collected. Across those experiments, the drop didn't seem to come from quality worries, artificial-content wariness, or broad AI aversion, and disclosures that signaled more human effort softened the engagement hit, though the field evidence is centered on TikTok, so Instagram, YouTube, or niche creator communities could behave differently.
Davis: That makes the policy problem more concrete: if disclosure is required, don't just slap on a label that says made with AI and walk away. This is the AI Scrambles Signals thread again, because the same like or comment now partly measures whether the audience still believes a human did meaningful creative work.
Paperboy.fm: This is the free version of the podcast. Subscribe at paperboy.fm to access a dozen different paper review podcasts for five dollars a month.