RoboCurate: Does Video-Action Verification Improve VLA Training?.

RoboCurate: Harnessing Diversity with Action-Verified Synthetic Data for Robot Learning • Seungku Kim, Suhyeok Jang et al. • KAIST and RLWRLD • arXiv v2, November 5, 2026; CoRL 2026 G2A workshop oral • 2026.

Cold Open.

Imagine a generated video of a humanoid picking up a cup. The reach looks natural, the fingers close, and the cup arrives where the instruction says it should. You approve the clip. Now inspect its action labels: they describe a different reach. The video is convincing; the supervision tells the robot to do something else.

Would you discover that mistake by watching more carefully? Not necessarily. You need to inspect the relationship between the pictures and the commands. A robot-learning dataset contains that relationship, not just a collection of attractive demonstrations.

RoboCurate makes the relationship testable: replay the predicted commands in a simulator, render the resulting movement, and compare it with the generated video. The proposal is to curate video–action pairs rather than videos alone.

That raises a sharper scaling question. When you can generate another batch overnight, should your next unit of compute buy more demonstrations—or better evidence that the demonstrations mean what their labels say?.

The Idea.

DreamGen is the direct starting point. NVIDIA’s pipeline adapts an image-to-video model to a robot, generates trajectories from an initial frame and an instruction, assigns pseudo-actions with an inverse dynamics model, and trains a visuomotor policy on the resulting examples. Its central move is to turn a video model’s repertoire into robot supervision. New task proposals come from generation; action grounding comes from a model trained on recorded robot experience.

An inverse dynamics model works backward from observations: given what the scene looks like now and later, infer the action sequence connecting them. In this pipeline, the video supplies the intended behavior and the inverse model supplies a candidate explanation in the robot’s control space. That division of labor makes synthetic video useful for imitation learning.

RoboCurate adds a round trip. Turn that candidate action sequence back into a video through simulator replay, then ask whether the two clips agree. Comparing numerical commands against pixels becomes a comparison between two visible motions.

Think of the simulator as a translator. It gives the action annotation an observable consequence. If an annotation says “move the other hand,” the replay makes that claim visible. You can now train a judge on the relationship between the clips.

A VLM first screens out visibly implausible videos. The action-consistency comparison then uses a frozen V-JEPA 2 encoder and a small attention-based classifier. V-JEPA 2 learns video representations through self-supervised prediction in feature space; its original evaluation includes motion understanding using trained probes on frozen features. That makes it a sensible backbone for extracting movement rather than surface resemblance.

A pixel-by-pixel comparison would answer the wrong question in our cup example. One clip might contain a textured kitchen and the other a plain rendered robot. The desired agreement concerns which hand moves, where it travels, and when it closes—not whether the backgrounds have the same colors.

Training the judge starts with real demonstrations. A real clip paired with replay of its recorded actions is positive. Negatives come from time-shifting that replay, swapping in another episode of the same skill, or doing both. These labels are constructed automatically. Keeping swapped episodes within a skill category pushes the task toward distinguishing trajectories rather than merely recognizing “pick” versus “pour.”.

RoboCurate also broadens the candidate pool through image editing before video generation and appearance changes to completed videos. The latter aim to preserve motion so action labels remain reusable. The same learned score supports best-of-N sampling: generate several candidate video–action pairs and retain the highest-scoring candidate.

This gives data creation two useful operations: explore a wider set of scenes, then choose among proposed demonstrations using evidence about their annotations. Generation supplies coverage. Verification supplies a preference over what enters training.

The Result.

The most informative comparison is 24.2% to 29.3% success on GR-1 Tabletop: add the learned motion verifier after the same VLM pre-filter, and downstream performance improves. This directly tests the contribution of action-aware checking.

The study pretrains GR00T N1.5 with a fresh action head on an ActionNet subset and synthetic data, applying curation late in pretraining. The low-data, 24-task GR-1 evaluation follows simulator fine-tuning with 300 demonstrations per task.

The full pipeline nearly doubles the real-only baseline. The learned probe also beats raw embedding similarity when both selectors retain the same number of samples.

That is the right kind of evidence for a curation paper. The important endpoint is what the trained policy can accomplish. A score separating attractive and unattractive videos is an intermediate instrument; policy improvement is the outcome we want. Matching retained data size also helps distinguish a better selector from simply feeding the learner fewer examples.

DexMimicGen provides a second testing context. Its original contribution was to expand demonstrations through object-relative trajectory transformations and simulated execution, including synchronized and sequential bimanual subtasks. This benchmark family emphasizes coordinated manipulation, making it a useful complement to tabletop rearrangement.

In RoboCurate’s experiments, GR-1-derived pretraining also improves performance on bimanual Panda arms after target-task fine-tuning.

The physical-robot experiments use ALLEX and OpenArm. Real demonstrations cover the familiar task; novel-object and novel-behavior examples are synthetic. Here the verifier selects among generated candidates without the scene-diversification component. This evaluates its task-specific data-selection role.

On ALLEX, synthetic training produces nonzero pouring performance where the real-data-only baseline gets none. The scientific point is that curated generated experience can change which behaviors a policy expresses, not merely how closely it imitates the familiar demonstration.

That is the evidence chain worth carrying forward: change data selection, train the policy, and measure behavior. Judge the curator by what survives this chain, rather than making its own consistency score the final arbiter.

The Biggest Weakness.

The biggest weakness is the gap between matching robot motion and validating the object interaction that makes the motion useful.

Appendix B specifies object-free simulator replays for GR-1, ALLEX, and OpenArm. The replay renders the robot moving, without reproducing its interaction with the manipulated objects.

That distinction matters most at contact. Imagine a reach with the right overall silhouette that closes the fingers just before reaching the cup. Or a motion that passes beside a handle rather than around it. A learned video representation might notice the discrepancy. The simulator-based check never establishes whether those fingers actually secure that cup, because the cup is absent from the replay.

The visual pre-filter checks apparent plausibility, rather than executing the proposed labels against the pictured objects.

Compare simulation’s role in DexMimicGen: transformed demonstrations are executed with the relevant objects, and successful trajectories are retained. Object state and task completion participate directly in accepting the demonstration.

The decisive follow-up would hold gross arm motion nearly fixed while perturbing contact-critical labels: grasp depth, finger closure, or release timing. Then test whether accepted samples still execute successfully in object-aware simulation or on the robot. That would measure the connection this pipeline needs between its consistency score and trustworthy manipulation supervision.

Pair each corrupted example with its uncorrupted source, so a filter cannot win merely by rejecting unfamiliar scenes or a different skill. It must identify the change that breaks the interaction.

Until then, the justified reading of “action-verified” is narrower than a physical guarantee. It means the annotations passed a learned motion-agreement test. That is valuable evidence, and its coverage of contact-critical errors remains the central unresolved question.

Verdict.

Robot-data teams should care most, especially those already generating videos and recovering pseudo-actions. RoboCurate makes curation an explicit part of the learning system. The practical target is the data-production pipeline: decide which proposed examples deserve repeated exposure during policy training.

That placement is important. A data-side verifier can be evaluated while holding the policy architecture fixed. It also gives the generation team a concrete feedback signal: which proposals survive because their inferred actions agree with their visible behavior? That is a more actionable development target than asking for generically “better videos.”.

There is already follow-on use from the same group. RLWRLD’s RLDX-1 technical report incorporates video-quality filtering followed by simulator-based motion-consistency filtering into its synthetic-data pipeline. Its controlled study increases the amount of filtered synthetic data while keeping the real dataset fixed, and reports improving downstream GR-1 Tabletop performance as that synthetic pool grows.

For practitioners, that demonstrates a route from an individual curation method into a broader foundation-model training recipe. The reusable unit is the verified training pair. You can improve generation, action inference, or selection separately, then ask how the resulting corpus changes control performance.

The surrounding implementation ecosystem is concrete. NVIDIA’s public GR00T-Dreams repository separates video-model adaptation, synthetic video generation, inverse-dynamics action extraction, and policy fine-tuning. Those stages expose the natural insertion point for this idea: after actions have been attached, before synthetic examples become policy supervision.

Treat quality and scale as complementary design objectives. A generator expands what the policy can encounter; a curator determines which encounters provide coherent supervision. Organize the training-data pipeline around both jobs, and you can make progress without turning every experiment into another policy-architecture search.

Takeaway: verify the relationship between observations and actions before scaling the supervision built from it.

Outro.

The next result to watch is what happens when action-aware curation becomes a routine component of larger synthetic-data pipelines.

Return to that imaginary cup. The useful question is no longer whether the generated clip deserves a place in a demo reel. It is whether the attached commands tell the same story. That change of question is the lasting idea here: a demonstration is a claim about behavior, and its labels should be examined as part of that claim.

This is Embodied AI 101.