Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events? A Systematic Evaluation of Detectors, Generators and Social Dissemination

🤗 Upvotes: 260 | cs.CV, cs.AI

Authors:
Shuo Liang, Yixing Ma, Pengfei Zhou, Zhenglin Wan, Xingyan Chen, Zihan Mei, Manting Li, Feihan Chen, Zhiwen Wang, Bin Xu, Haotian Zhang, Jiajun Song, Shiya Su, Run Liu, Zhenghang Ni, Yifa Yu, Jintao Hong, Bolong Feng, Yifei Liu, Zirui Zhang, Jingxuan Zhang, Songlin Zhao, Yifan Bai, Kang Tan, Yizhe Liu, Junhao Du, Yongtao Ge, Zhaopan Xv, Xinyuan Zhang, Mengru Ma, Chunhua Shen, Wei Wang, Yang You, Zheng Zhu, Kaipeng Zhang, Wangbo Zhao

Title:
Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events? A Systematic Evaluation of Detectors, Generators and Social Dissemination

Arxiv:
http://arxiv.org/abs/2608.14391v2

Abstract:
Recent video generators can fabricate realistic depictions of wars, disasters, public emergencies, and other real-world crises, creating substantial risks of misinformation. Existing benchmarks, however, provide limited evidence on detector and generator behavior in such settings, including how detectability varies with generation conditions, how people perceive generated videos, and whether detectors remain reliable during social dissemination. To address this gap, we introduce RA-Bench, a benchmark for AI-generated video detection that uses Real videos as Anchors. RA-Bench contains 17,886 videos, comprising 1,830 real-video anchors across 10 social-risk categories and 16,056 generated clips from four open-source and five closed-source generators. Based on RA-Bench, we organize our evaluation along three dimensions. We first assess detector generalization across seven traditional detectors, ten zero-shot multimodal models under three review settings, and two MLLMs specifically fine-tuned on AI-generated video detection. Across these methods, none of the three detector families generalizes consistently across RA-Bench instances. We then examine how detectability varies with generation quality, conditioning information, and sampling seeds. These analyses show that generation properties affect detector families differently, while source-level detection patterns remain stable across seeds. Finally, we study human authenticity judgments and detector reliability during social dissemination. We find that videos that mislead people are also difficult for current detectors, and that social dissemination makes detection harder. Together, these findings show that current methods struggle to detect realistic AI-generated videos, highlighting the need for detectors robust to evolving video generators.

What is Daily Paper Cast?

We update every weekday to discuss highest-voted papers from Huggingface Daily Paper (https://huggingface.co/papers). Both the podcast scripts and audio are generated by AI. Feedback and suggestions are welcome! Email us: [e-mail]

Creator:
Jingwen Liang, 3D ML, https://www.linkedin.com/in/jingwen-liang/
Gengyu Wang, LLM ML, http://wanggengyu.com

Listen on:
Spotify: https://open.spotify.com/show/21nrhmdaA8qoBiH8q03NXL
Apple Podcast: https://podcasts.apple.com/us/podcast/daily-paper-cast/id1777620236

Cover Image by Kawen Kuang https://kawen.art

Evan: Welcome to Daily Paper Cast, bringing you the latest research papers every day.

Ashley: Today's paper comes from Hugging Face's daily paper list of August 18, 2026, and has garnered 260 upvotes.

Evan: The title of the paper is 'Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events?

A Systematic Evaluation of Detectors, Generators, and Social Dissemination.'

Ashley: It's authored by Shuo Liang and Yixing Ma, with corresponding author Wangbo Zhao from the National University of Singapore.

Evan: Ashley, this sounds like a critical topic.

Can you start us off with some background?

Ashley: Evan.

Recently, video generation models have advanced significantly, making it possible to create very realistic videos of real-world crises.

For example, think about videos depicting wars, natural disasters, or public health emergencies.

These videos can be so lifelike that it becomes challenging for the public to distinguish them from genuine footage.

Evan: That’s concerning.

What are the main objectives of this study?

Ashley: The authors aim to address a critical gap in our understanding of how well current detectors can handle AI-generated videos in crisis settings.

They’re interested in three main aspects: how well current detectors perform, what makes AI-generated videos hard to detect, and how human perception and social dissemination impact detection reliability.

Evan: So, what are the specific contributions of this paper?

Ashley: The authors introduce a benchmark they call RA-Bench.

This benchmark uses real videos as anchors and includes 17,886 videos.

Among these, 1,830 are real-video anchors from 10 different social-risk categories, while the remaining are generated clips from both open-source and closed-source generators.

Evan: Wow, that’s quite a dataset.

What did they focus on in their evaluation?

Ashley: Their evaluation is comprehensive and organized into three main dimensions.

First, they assess how well different types of detectors generalize across various sources.

This includes traditional detectors, zero-shot multimodal models, and multimodal language models specifically fine-tuned for detecting AI-generated videos.

Evan: And the second dimension?

Ashley: In the second dimension, they study generation properties.

They explore how video detectability varies with different generation quality, conditioning information, and the consistency of detection results across different sampling seeds.

Evan: That covers quite a bit already.

What about the third dimension?

Ashley: The third dimension involves human perception and social dissemination.

They look into how human users perceive the authenticity of these videos and how the reliability of detectors changes when these videos are disseminated on social platforms.

Evan: Given the implications for misinformation, their work seems extremely important.

Ashley: Indeed, it highlights significant gaps in the current detection capabilities, especially as video generation technology continues to evolve and become more sophisticated.

Evan: So, what’s next in this paper after the Introduction?

Ashley: That brings us to the end of the Introduction section.

From here, the paper dives into related work and the specifics of their RA-Bench creation and methodologies.

Evan: Alright, Ashley, let's dive into the methods they used in this study.

They seem extensive.

Ashley: Evan.

The authors structured their investigation into several well-defined steps, beginning with the creation of their RA-Bench dataset, and proceeding through to the evaluation of various detection methods.

Evan: Let’s start with the dataset creation.

How did they go about constructing RA-Bench?

Ashley: To construct RA-Bench, the team collected 675 videos depicting real-world crises from public platforms and organized them into 10 broad social-risk categories and 44 subcategories.

These include weather and natural disasters, wars, public health emergencies, and more.

Evan: That sounds robust.

What did they do after collecting these videos?

Ashley: Once the videos were collected, they segment each video into scene-level clips and screened for near-duplicates using automated tools like PySceneDetect and a ResNet-18 model pretrained on ImageNet-1K.

Evan: PySceneDetect and ResNet-18 for quality control, huh?

Interesting.

What was the next step?

Ashley: Yes, Evan.

After automated preprocessing, they conducted a two-round manual review process where volunteers assessed the clips on various criteria like visual quality, duration, and semantic fit to the assigned subcategory.

This ensured the dataset was both diverse and high-quality.

Evan: And how did they generate the synthetic videos?

Ashley: They paired each real clip with generated counterparts using image-to-video models conditioned on the first frame of the real clips and corresponding text prompts.

They employed four open-source and five closed-source generators to create these synthetic videos, resulting in a total of 16,056 generated clips.

Evan: That's quite a large number!

What about the evaluation dimension?

Ashley: The evaluation was carried out along three main dimensions.

First, they tested the generalization of various detectors including seven traditional detectors, ten zero-shot multimodal models, and two multimodal language models (MLLMs) fine-tuned specifically for AI-generated video detection.

Evan: What did these evaluations reveal?

Ashley: They found that none of the detector families generalized consistently across all RA-Bench sources.

Traditional detectors fell significantly short, with AUCs dropping from a public-reference range of 67.6-98.6% to 43.9-57.3% on RA-Bench.

Zero-shot multimodal models remained unreliable, and fine-tuned MLLMs also faced various challenges.

Evan: And how about the properties of the generated videos?

Were there any surprises?

Ashley: Oh yes, there were several key findings.

They discovered that generation quality doesn't uniformly determine detection difficulty.

Higher generation quality scores were generally associated with weaker fake evidence.

However, factors like dynamic content strengthened fake evidence for some detectors while leaving others unaffected.

Evan: Interesting.

What about human perception and how these videos spread on social media?

Ashley: For that part, they conducted human studies to assess how deceptive these generated videos were.

They found that a significant portion of generated videos were misclassified as real by human reviewers.

Additionally, when these videos were disseminated on social media, the difficulty in detecting them increased further.

Evan: How did they simulate social media dissemination in their study?

Ashley: They introduced RA-Bench-LastMile, a controlled social dissemination simulation.

This involved additional post-processing steps like transcoding, spatial downsampling, and frame-rate conversion to replicate the effects of social media platforms.

They found that these transformations further weakened the performance of both traditional and fine-tuned detectors.

Evan: It sounds like they left no stone unturned.

Were there any findings about specific failures in human perception?

Ashley: Indeed, they constructed something they called RA-Bench-HumanProof, which consisted of videos that were consistently judged as real by human reviewers.

Detectors, including traditional ones and fine-tuned MLLMs, struggled significantly with these human-deceptive videos, showing that videos that mislead people are also difficult for detectors.

Evan: Ashley, this research covers a lot of ground.

Any notable limitations they acknowledged?

Ashley: Good question, Evan.

The authors did note some limitations, such as the need for versioned updates to the benchmark as new video generators and detectors emerge.

They also highlighted that multi-stage forgery pipelines involving selective editing, audio synthesis, and contextual captions need to be evaluated in future work.

Evan: Future research directions often highlight the evolving nature of these studies.

Did they mention any specific future directions?

Ashley: They did.

They proposed developing more robust detectors, using active watermarking during video generation, and extending detection methods to audio-visual settings.

They also suggested enhancing the realism of video generation models using detection results as feedback signals to measure generation realism.

Evan: It's fascinating how they tie detection research back into improving generation models.

This section provides insights both for current challenges and future avenues in AI-generated video detection.

Ashley: Evan.

It’s a front-line defense and continuous improvement cycle.

This ends the Method section, providing a clear path from dataset creation to comprehensive evaluation and insights.

Evan: Alright, Ashley, let's move into the experiments and results of this intriguing study.

There’s a lot to unpack here.

Ashley: Definitely, Evan.

The experiments were conducted across three main areas: detector generalization, generation properties, and human perception coupled with social dissemination.

Evan: Great.

Let’s start with the detector generalization.

What did the experiments reveal?

Ashley: In this area, they evaluated seven traditional detectors, ten zero-shot multimodal models, and two multimodal language models (MLLMs) specifically fine-tuned for AI-generated video detection.

They tested these detectors across all RA-Bench sources.

Evan: And what was the outcome?

Ashley: The results were quite revealing.

None of the detector families generalized consistently across all RA-Bench instances.

Traditional detectors experienced a fall from public-reference AUCs between 67.6% and 98.6% down to a source-level mean of 43.9% to 57.3%.

Similarly, zero-shot multimodal models exhibited high sensitivity to prompts and sources.

Evan: Sounds like a real challenge.

How did the fine-tuned MLLMs fare?

Ashley: Fine-tuned MLLMs also showed difficulties.

When timestamps were used, Skyra-SFT and Skyra-RL performed better but highlighted a protocol dependency.

When they used frame indices instead of timestamps, their accuracy fell to about 54%, illustrating a reliance on timestamp cues.

Evan: That's a significant drop.

What about the generation properties?

Ashley: In exploring how generation properties affect detectability, they analyzed 16,056 generated clips.

They used evaluators like Video–Image Subject Consistency and Motion Smoothness from VBench++.

They found that higher generation quality was associated with weaker fake evidence for detectors, but dynamic content increased fake evidence for some systems like Gemini, without affecting traditional detectors uniformly.

Evan: That’s interesting.

So, no single quality score can predict difficulty in detection accurately.

Ashley: Exactly.

For example, stronger temporal consistency and subject consistency were associated with weaker detection evidence for most detectors, but dynamic content had different impacts across detector families.

Evan: And what about their findings on human perception and social dissemination?

Ashley: This is where it gets even more critical.

They conducted a human evaluation on RA-Bench, involving 20 reviewers.

They found that reviewers could only identify 60.3% of generated videos accurately.

Real videos were recognized 71.9% of the time.

This indicates a high potential for human deception.

Evan: That’s concerning.

Did they go further into social dissemination?

Ashley: Yes, they introduced RA-Bench-LastMile, simulating social media processing on videos.

They applied transformations like transcoding, spatial downsampling, frame-rate reduction, and adding synthetic news badges.

They found that these steps further weakened detector performance, especially for fine-tuned MLLMs, reducing their fake detection recall sharply.

Evan: How much did it reduce their effectiveness?

Ashley: For example, under the Full condition, mean fake recall for fine-tuned MLLMs fell from 46.0% to just 1.4%.

This indicates a systemic shift towards classifying videos as real.

Evan: So, social media transformations make detecting these fake videos extremely hard.

Ashley: Indeed, very challenging.

Also, they observed that human deceptive videos, those that appeared real to multiple human reviewers, were particularly tough for detectors as well.

They created RA-Bench-HumanProof for this subset, and detectors performed near random, showing how misleading some generated videos can be.

Evan: This underscores the complexity and severity of the problem.

Any final takeaways from this section?

Ashley: Yes, the experiments highlighted that current detection methods are not yet reliable in various social-risk scenarios, especially as these AI-generated videos evolve and proliferate.

Their findings emphasize the urgent need for robust detection systems adaptable to new generation techniques and social media dissemination effects.

Evan: Incredible insights, Ashley.

That concludes our discussion on the Experiment section.

Evan: Next up, Ashley, the paper reviews related work extensively.

Can you guide us through the key points?

Ashley: Of course, Evan.

The related work section provides a broad context that helps us understand where this study fits within the existing landscape of AI-generated video detection.

Evan: Alright, so where do they start?

Ashley: They begin by discussing benchmarks for AI-generated video detection.

Early efforts like GVF and GenVideo started by pairing real videos with generated counterparts to evaluate detection models.

These initial benchmarks focused primarily on detection accuracy and cross-generator transfer.

Evan: Have these benchmarks evolved since then?

Ashley: Yes, they have.

Subsequent benchmarks like GenVidBench and AIGVDBench expanded in scale and included more generators and evaluation protocols.

They even moved beyond binary classification to include aspects like forensic explanations and source backtracking.

Evan: That’s quite an expansion.

How do these relate to the current paper?

Ashley: The paper introduces RA-Bench as a more comprehensive benchmark.

RA-Bench includes real-event sources, a social-risk taxonomy, real-event-conditioned generation, a human deception challenge set, and a social dissemination simulation.

It builds on the previous benchmarks but adds more dimensions to better reflect real-world challenges.

Evan: Got it.

What about the video generation models they discuss?

Ashley: The paper reviews a range of recent video generation models.

Early models extended from image generation techniques, like Stable Video Diffusion and VideoCrafter.

More advanced models, including Brooks et al.'s world simulators and Team Seedance’s Seedance 2.0, have focused on improving narrative coherence and visual consistency in generated videos.

Evan: How do these advances in video generation impact society?

Ashley: The advancements come with risks of misuse, particularly in fabricating realistic videos of crises.

These videos can deceive viewers and spread misinformation.

The paper emphasizes the need for reliable detection methods to combat this and maintain public trust in media content.

Evan: I see.

And what about detectors for AI-generated videos?

Ashley: They categorize detectors into traditional discriminative approaches and MLLM-based approaches.

Traditional detectors include video classification models, generated-image detection models, and generated-video detection models.

MLLM-based approaches have gained traction for providing detailed forensic explanations beyond simple binary classification results.

Evan: Interesting.

Can you give us examples of these traditional and MLLM-based methods?

Ashley: Sure.

Traditional methods like Feichtenhofer et al.'s SlowFast Networks or Liu et al.'s Video Swin Transformers classify videos directly.

Generated-image detectors such as those from Wang et al.

focus on image-level artifacts, while generated-video detectors like those from Ma et al.

capture temporal inconsistencies.

Evan: And the MLLM-based approaches?

Ashley: MLLM approaches, like those by Wen et al., use multimodal learning to detect forgeries and provide forensic explanations.

These models can often explain why a video is fake by identifying specific artifacts and anomalies.

Evan: It sounds like there is a significant effort in diversifying detection techniques.

How does this paper’s contribution compare to these existing methods?

Ashley: RA-Bench uniquely combines these various methods into its evaluation framework, assessing detectors on their ability to handle real-world crisis videos.

The benchmark also introduces a structured evaluation that includes human perception and the effects of social dissemination, making it more comprehensive.

Evan: Seems like RA-Bench is setting a new standard in evaluating AI-generated video detection across multiple dimensions.

Ashley: Precisely.

The integration of traditional evaluation, social media transformation, and human deceptive videos provides a holistic view that existing benchmarks have only partially addressed.

Evan: This detailed review really puts the significance of their contributions into perspective.

Ashley: Definitely.

This concludes the Related Work section, highlighting the extensive groundwork and advancements that pave the way for RA-Bench’s comprehensive approach.

Evan: Alright, Ashley, let’s wrap up this episode with a summary of the paper’s key contributions and takeaways.

Ashley: Sure thing, Evan.

The paper 'Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events?' makes several crucial contributions.

Firstly, the authors developed RA-Bench, a comprehensive benchmark designed to evaluate AI-generated video detection.

This includes real videos as anchors and a diverse set of generated clips from both open-source and closed-source generators.

Evan: Exactly.

This dataset is comprehensive, covering 10 social-risk categories and 44 subcategories.

It pairs real videos with generated counterparts to simulate a variety of crisis scenarios.

Ashley: Right.

Another significant contribution is the three-dimensional evaluation framework.

This framework assesses detector generalization, explores how generation properties affect detectability, and examines the impact of social dissemination on detection accuracy.

Evan: The experiments revealed that current detection methods struggle to generalize across different sources, with significant performance drops for traditional detectors, zero-shot multimodal models, and fine-tuned MLLMs.

Ashley: Exactly.

Another key takeaway is that higher generation quality did not uniformly translate to harder detection challenges, and dynamic content had varying impacts on different detectors.

Evan: And the human studies showed that human reviewers are often deceived by generated videos, and that social media transformations further degrade detector performance.

Ashley: Exactly.

Their findings highlight the urgent need for more robust, adaptable detection systems that can keep pace with evolving video generation technology and social dissemination effects.

Evan: It's clear the study sets a new standard for comprehensive evaluation in this field.

Ashley, thanks for breaking down this paper with me.

Ashley: My pleasure, Evan.

And to our listeners, thank you for tuning in.

We hope you found today’s discussion insightful.

Evan: We’ll be back with more cutting-edge research in our next episode.

Make sure to subscribe to Daily Paper Cast so you don't miss out on future episodes.

Ashley: Until next time, stay curious and keep exploring the world of AI research.

Goodbye!

Evan: Goodbye!