Synthetic Data For Defense ISR: EO/IR Training Data For Perception Teams

The Short Answer

Synthetic data for defense ISR is rendered imagery of targets, backgrounds and sensors, labeled automatically, that perception teams use to train and test detectors where real labeled data is rare, restricted or too costly to collect. It matters most for electro-optical and infrared (EO/IR) and SAR tasks such as vessel detection, vehicle detection in overhead imagery, counter-UAS and automatic target recognition, because the targets that matter are the ones least present in public data. The evidence favors mixing it with real imagery. In RarePlanes, 50,000 synthetic satellite images with about 630,000 aircraft annotations were released next to 14,700 hand-annotated real aircraft, and the authors report that models trained on the synthetic set and fine-tuned on about 10% of the real training data reached 91 to 96% of the mAP of models trained on all of the real data.

Buyers are asking for it directly. In June 2026, DefenseScoop reported that SOCOM was seeking a self-service synthetic data platform that produces EO/IR training imagery for its unmanned systems autonomy program. As of October 2026, the pages that rank for this question are vendor product pages and whitepapers, most of them short, without public datasets, evidence or a FAQ. This post covers why ISR perception lacks labeled data, what synthetic data has to model for each task and sensor, which public datasets to use alongside it, the deployment questions that decide whether a tool can be used at all, and how the vendors compare.

Why ISR Perception Lacks Labeled Data

The targets are rare. Commercial and academic datasets are full of cars, buildings and cargo ships. A specific class of patrol craft, a mobile air defense system, or a semi-submersible appears in few public images, often at a handful of aspect angles, and seldom in the weather and lighting where a model will be used. Even xView, one of the largest public overhead datasets with over 1 million objects across 60 classes, was built around general object categories rather than any program's target list.

The real data is controlled. The imagery a program collects of the targets it cares about is often classified or controlled unclassified information. It cannot be sent to a commercial labeling vendor, moved to a public cloud, or shared with a subcontractor without approvals, so the labeled pool stays small and slow to grow.

Sensors differ. A detector trained on 0.3 m WorldView-3 imagery does not transfer to an MWIR gimbal camera on a UAV, and neither transfers to SAR. Every band (visible, SWIR, MWIR, LWIR, SAR), resolution, look angle and platform altitude needs its own training data, and many programs fly sensors that have no public dataset at all.

Adversaries vary their appearance. Camouflage, decoys, paint schemes, netting and new equipment change what a target looks like faster than a collection campaign can capture it. A model needs examples of variants that have not been photographed yet.

Small targets are hard to label. A 4.5 m car in 0.3 m satellite imagery is about 15 pixels long, and the Drone-vs-Bird challenge organizers report that most annotated drones in their training videos are smaller than 32 by 32 pixels, many under 16 by 16, in frames up to 3840 by 2160. Human labels at that size are inconsistent, and in thermal imagery they are worse. Rendered data has exact boxes and masks at any size, which is one of its practical advantages.

The SOCOM solicitation described by DefenseScoop puts the problem in similar terms. It says CV models need large volumes of labeled, operationally relevant EO/IR imagery, and manual collection is constrained by cost, access and the inability to capture rare events and contested environments.

ISR Tasks, Sensors, And What Synthetic Data Has To Model

Each ISR task pairs with a small set of sensors, and each pairing sets the things a synthetic pipeline must get right. The table lists the common ones.

Two things cut across every row. The first is the sensor model: spectral band, optics, noise, gain control, compression and any overlay the real system burns into its video. The second is metadata. Range, aspect, time of day and weather logged for every frame let a team find which conditions a model fails in, and that is where much of the value of synthetic data lies after the first model. Our post on synthetic thermal infrared training data covers IR sensor modeling in more detail, and Synthetic Data For Drone Detection covers the counter-UAS row.

Public Defense-Relevant Datasets

Synthetic data works best next to real data, for mixing and for testing. These are the public datasets most often used for ISR-style tasks, with figures from their papers and project pages.

None of these covers a program's exact sensor, targets and operating area. Their realistic role is pretraining, benchmarking and a first real test set while a program builds its own. The SpaceNet 6 paper gives a sense of how much prior data helps in a hard modality. Segmentation models pretrained on optical imagery and then trained on SAR reached an F1 of 0.21, compared with 0.135 for models trained on SAR alone.

What The Evidence Says About Synthetic Data For These Tasks

Results specific to military targets are mostly unpublished, so the public evidence comes from neighboring tasks. RarePlanes is the clearest overhead example, with real and synthetic aircraft that share 10 fine-grained attributes such as wingspan and wing shape. In counter-UAS work, the published results we collected in Synthetic Data For Drone Detection show mixed real and synthetic training beating real-only training in every hybrid comparison, while synthetic-only models fall well behind on hard test sets. Our broader review, Does Synthetic Data Work For Object Detection?, reaches the same conclusion across domains, which is to pretrain on synthetic data, fine-tune on a smaller real set, and measure on held-out real imagery.

The practical takeaway for an ISR team is to start with a real test set from the operational sensor, even a small one, before generating any synthetic data. Without it there is no way to tell whether the synthetic data is closing the gap or adding a new one. How To Close The Sim-To-Real Gap For Perception covers how to set that test up.

Deployment, Export Control, And Data Handling

For defense programs, where a tool runs and what it touches often decide the choice before image quality does. These are general considerations, not legal advice, and a program's security and export compliance staff have the final say.

On-prem and air-gapped operation. Programs working with controlled or classified data usually need the generator, the asset library and the training pipeline inside their own enclave. A cloud-only tool can still be useful for unclassified pretraining data, but any step that touches real program imagery has to run where that imagery is allowed to live. Ask vendors whether they support on-prem or air-gapped installs, how licensing works offline, and how updates and new assets reach a disconnected network.

Export control. In the United States, defense articles and technical data fall under ITAR, administered by the State Department, and dual-use items fall under the EAR, administered by the Commerce Department. Detailed 3D models of military equipment, sensor performance models and the trained models themselves can carry controls, separate from any imagery. Ask how a vendor's asset library is classified for export, and treat your own custom assets and trained weights as items that need a determination.

Data handling. Synthetic imagery does not contain collected intelligence, which is one reason teams use it, but metadata, target lists and scenario descriptions can still reveal what a program is looking for. Keep scenario definitions and any real backplates under the same handling rules as the program data they describe.

Self-service versus managed service. SOCOM's request, as reported, asked for a platform personnel can operate organically without vendor assistance and that plugs into an existing MLOps pipeline. Many defense teams want the same thing, because a managed service means sending requirements, and sometimes data, outside the program for every iteration.

Vendors And Tools For Defense Synthetic Data

Several companies sell synthetic data into defense, and they differ mainly in sensor coverage and delivery model. The descriptions below come from each company's own pages as of October 2026.

If your sensor is SAR today, Rendered.ai and L3Harris list it as available now. If you need physics-accurate thermal signatures of specific vehicles, MuSES-based pipelines are built for that. If you have the engineering staff and want everything on your own hardware, NVIDIA's tools are the strongest free option. Synthetic Data Platforms Compared covers these vendors and others in more depth.

Where Stardust Fits

Stardust is Bifrost's synthetic data platform for perception and autonomy. The U.S. Air Force, Shield AI and ST Engineering appear on its aerial page under "Trusted by airborne ISR and autonomy programs," and Saronic, Havoc, Seadronix, ClassNK and ST Engineering appear on its maritime page.

For ISR, Stardust's published use cases include maritime vessel detection from UAVs (including narcotic semi-submersibles), terrestrial detection of ground vehicles up to military vehicles, counter-UAS classification by UAV type and role, port defense patrols with virtual USVs, and maritime threat classification that distinguishes civilian from military vessels. Its maritime asset library includes aircraft carriers, destroyers, frigates, submarines, patrol vessels and minesweepers alongside commercial ships. Sensors include RGB, stereo, SWIR, MWIR and thermal, with depth and segmentation output and radar in development. On the geospatial side, it emulates WorldView-3, Legion 06 and SkySat, composites synthetic objects onto real overhead backplates, and is developing coherent SAR with NTT Data. Every frame comes labeled with boxes, pixel-level masks and scenario metadata.

Stardust's site does not publish deployment options, so ask about on-prem and air-gapped requirements directly. For SAR work today, or for an entirely self-hosted toolchain, the alternatives above are the better fit.

Sources

• Harper, SOCOM seeks 'self-service' synthetic data generation platform to boost drones' computer vision, DefenseScoop, June 9, 2026
• Lam et al., xView: Objects in Context in Overhead Imagery, 2018
• Ding et al., Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges, 2021, and the DOTA dataset page
• Sun et al., FAIR1M: A Benchmark Dataset for Fine-grained Object Recognition in High-Resolution Remote Sensing Imagery, 2021
• Gupta et al., xBD: A Dataset for Assessing Building Damage from Satellite Imagery, 2019
• Shermeyer et al., SpaceNet 6: Multi-Sensor All Weather Mapping Dataset, CVPRW 2020
• Paolo et al., xView3-SAR: Detecting Dark Fishing Activity Using Synthetic Aperture Radar Imagery, NeurIPS 2022 Datasets and Benchmarks
• Kechagias-Stamatis and Aouf, Automatic Target Recognition on Synthetic Aperture Radar Imagery: A Survey, 2020
• Shermeyer et al., RarePlanes: Synthetic Data Takes Flight, WACV 2021
• DSIAC, ATR Algorithm Development Image Database
• Coluccia et al., Drone vs. Bird Detection: Deep Learning Algorithms and Results from a Grand Challenge, Sensors, 2021