Notes from the team
RSS-
pi0.5 LIBERO Results, Reproduced On Manifold
We ran the public pi0.5 LIBERO checkpoint for 1,999 episodes across all four suites and averaged 96.7%, against 96.85% reported by openpi. Per-suite results and rerun spread.
Read -
Robot Policy Evaluation Platforms Compared (2026)
Isaac Lab-Arena, RoboLab, LeRobot and SimplerEnv evaluate robot policies in simulation, RoboArena and AutoEval on real robots. The right one depends on your benchmark.
Read -
Synthetic Data For Defense ISR: EO/IR Training Data For Perception Teams
Synthetic data gives ISR perception teams labeled EO/IR and SAR imagery of rare targets they cannot collect or share. Train on it with some real data, then test on real.
Read -
Synthetic Data For USV Collision Avoidance And Maritime Lookout
Synthetic data gives USV lookout models the rare, dangerous cases real footage lacks, like swimmers, debris, night and rough seas. Train on it, validate on real video.
Read -
Why VLA Policies Fail: Common Failure Modes In Simulation
VLA policies fail by grabbing the wrong object, missing grasps, closing on air, stalling and not recovering, and they get worse when cameras, layouts or wording change.
Read -
Best Simulation Benchmarks For VLA Models In 2026
The best VLA simulation benchmarks in 2026 are LIBERO-Plus or LIBERO-PRO, RoboCasa365, RoboTwin 2.0, RoboLab and SimplerEnv. Plain LIBERO is saturated at 97.1%.
Read -
Does Synthetic Data Work For Object Detection? What The Evidence Says
Yes, mostly as a complement. Pretraining on synthetic images and fine-tuning on a small real set beats real-only training in most published results.
Read -
How Many Trials Do You Need To Evaluate A Robot Policy?
About 100 trials gives a 95% interval of roughly ±8 to ±10 points on one success rate. Telling two policies 10 points apart takes 199 to 388 trials each.
Read -
How To Close The Sim-To-Real Gap For Perception Models
Measure the gap on held-out real data, model your sensor, randomize appearance, match the real content distribution, then fine-tune on a small real set.
Read -
How To Run LIBERO Evaluation And Reproduce Published Results
Run each LIBERO suite from its 50 fixed initial states, 50 episodes per task, with step limits of 220 to 520, then match the training image rotation and action chunking.
Read -
Is LIBERO Saturated? What Top Scores Hide And What To Report
Largely, yes. Top VLAs report 96 to 98% averages, but LIBERO-PRO and LIBERO-Plus perturbations drop several to near 0% when objects move, tasks change or cameras shift.
Read -
Isaac Lab vs MuJoCo vs ManiSkill vs Genesis For Policy Evaluation
For policy evaluation, the benchmark usually picks the simulator. LIBERO and RoboCasa run on MuJoCo, SimplerEnv on ManiSkill, Isaac Lab-Arena tasks on Isaac Lab.
Read -
How To Evaluate Robot Policies In Simulation At Scale
Eval time goes to policy inference, sim stepping, rendering and resets. Vectorize environments, shard across GPUs, batch inference and serve the policy separately.
Read -
Can You Trust Simulated Robot Policy Evaluation?
For ranking policies, often yes. SIMPLER reports a 0.924 Pearson correlation with real results and PolaRiS 0.90, but sim success rates rarely match real ones.
Read -
SimplerEnv Alternatives For Real-To-Sim Policy Evaluation
The main SimplerEnv alternatives are PolaRiS for DROID policies, RoboLab in Isaac Lab-Arena, Gaussian-splat soft-body twins, RobotArena ∞, and real-world RoboArena.
Read -
Synthetic Data For Drone Detection: What The Research Shows
Synthetic data improves drone detectors most when mixed with real images. Published hybrids beat real-only training, while synthetic-only models trail on hard test sets.
Read -
Synthetic Data For Lunar Landing And Planetary Hazard Detection
Lunar hazard detection models train on synthetic terrain because labeled lander-scale imagery barely exists. Teams build it from LOLA DEMs, procedural rocks and low sun.
Read -
Synthetic Data Platforms For Computer Vision, Compared (2026)
The best synthetic data platform depends on your domain and sensors. We compare NVIDIA, Duality, Parallel Domain, Rendered.ai, Anyverse and others as of October 2026.
Read -
Synthetic Thermal Infrared Training Data: How To Get Labeled IR Imagery
Labeled thermal IR training data comes from three routes: hand-labeling real frames, translating RGB images to thermal, or rendering scenes with physics-based simulation.
Read -
Your robot may be learning the world flipped
We found that the most widely used LIBERO training datasets store images that are horizontally mirrored relative to the simulated scene. Here is the history, the impact, and how Manifold catches this class of bug.
Read -
Introducing Manifold
The platform for accelerating robotics research with failure analysis that explains itself. A month of experiments in a week.
Read -
Feature Launch: Multi-Camera Rendering
Place up to 5 cameras in a single world and capture every angle of the same moment in one render.
Read -
How To Evaluate A VLA Policy: Benchmarks, Rollouts, And Reproducibility
A practical guide to VLA policy evaluation, covering LIBERO and RoboCasa, how many rollouts you actually need, and a reproducibility checklist for citable results.
Read -
Top Datasets For Robot Manipulation, VLA Training & Policy Learning
A framework for evaluating manipulation datasets, plus 12 of the best open-source datasets for training VLA models and manipulation policies.
Read -
Notebooks, Now Decoupled from Live Instances
Edit notebooks and dispatch render jobs without a GPU instance, pre-warm instances for when you need them, and share packaged datasets across your organization.
Read -
A Major 3D Engine Upgrade
Our upgraded 3D engine unlocks progressive improvements in visual quality and render performance, starting with a more realistic weather system.
Read -
Top Datasets For Maritime AI, USV Perception & Obstacle Detection
A framework for evaluating maritime datasets, plus 14 of the best open-source datasets for USV perception, obstacle detection, and ship classification, scored on ontology, diversity, and label quality.
Read -
Ask, Don't Search: AI Assistant in the Bifrost Docs
Get instant answers about the Bifrost platform with the new AI assistant built into our docs.
Read -
Feature Launch: Thermal with Depth
Generate thermal imagery with pixel-aligned depth maps from a single Bifrost render.
Read -
Notebook Sessions Are Now Completely Free
Explore and experiment freely in Bifrost notebooks - your credits now go entirely to rendering.
Read -
Bootstrapping Maritime Detection with Synthetic Data in 2 days
One engineer trained a USV sailboat-and-buoy detector to 82% F1 on real test footage in two days—using just 2,500 synthetic images and two training runs on a single desktop GPU.
Read -
Feature Launch: Custom Asset Requests
Request bespoke 3D assets directly in the Bifrost platform and track every request from submission to delivered asset.
Read -
Feature Preview: Neural Rendering
Neural rendering adds photoreal detail and new environmental diversity on top of Bifrost's 3D engine, without breaking your labels.
Read -
Product Update: Panoptic Segmentation Masks
Pixel-accurate panoptic segmentation masks replace COCO polygons in Bifrost dataset downloads, now delivered as high-bit-depth TIFF files.
Read -
Feature Launch: Organization-Wide Sharing
Share notebooks, ontologies, and generations across your organization so every team builds on each other's work.
Read -
Feature Launch: Glassy Water Reflections
A new ultra-calm water material brings mirror-like ocean reflections to your maritime synthetic data.
Read -
Feature Launch: A Faster Offline Render Workflow
Send offline render jobs instantly, name collections on the way out, and track every job with clear statuses.
Read -
NTT DATA and Bifrost Partner to Test Satellite Performance Using Synthetic Data
Over the past year, NTT DATA's Innovation Center collaborated with Bifrost AI to evaluate whether synthetic data could accelerate AI model development while maintaining or improving quality and reducing costs.
Read -
Feature Launch: Dataset Gallery
Get instant access to ready-to-run datasets inspired by some of the world's top AI teams.
Read -
Building The Data Engine To Power Physical AI
AI's biggest frontier is the physical world. Why data is the bottleneck, and how Bifrost builds the generative data engines to simulate it in minutes.
Read
