BIFROST
All posts

Synthetic Satellite Imagery For Object Detection: What Works

Synthetic satellite imagery is rendered or composited overhead data with exact labels. It works best for rare objects, as pretraining before fine-tuning on real images.

The Short Answer

Synthetic satellite imagery for object detection is overhead imagery that is either rendered from a 3D scene at a chosen ground sample distance or built by compositing rendered objects onto patches of real satellite images, with bounding boxes and masks generated from the scene rather than drawn by hand. It helps most when the target is rare in real collections. The clearest public result is RarePlanes (WACV 2021), which paired 14,700 hand-annotated real aircraft in 253 WorldView-3 scenes with 50,000 synthetic images: detectors pretrained on the synthetic set and fine-tuned on about 10% of the real training data reached 91 to 96% of the mAP of models trained on all the real data. SIMPL (2021) found that detectors trained only on synthetic aircraft composited onto real xView backgrounds performed about as well as detectors trained on 50 to 100 real instances per class.

The rest of this post covers how synthetic overhead imagery gets made, what the published experiments measured, which public datasets to test against, which sensor parameters to match, and which tools generate it as of October 2026. If your question is whether synthetic data helps object detection in general, start with Does Synthetic Data Work For Object Detection?. If your sensors are airborne EO/IR rather than satellite, Synthetic Data For Defense ISR covers that.

Why Does Satellite Object Detection Run Out Of Labeled Data?

Benchmarks make the problem look solved. xView has over 1 million objects in 60 classes across more than 1,400 km² of 0.3 m WorldView-3 imagery, and DOTA has 1,793,658 instances in 11,268 images. Both are built around common classes such as cars, buildings, ships and planes.

Real projects are usually about something else. The SIMPL authors put it plainly: applications often involve novel target classes with little or no labeled data, and finding examples means buying and inspecting imagery that may contain none. The rarer the object, the more imagery has to be purchased and searched per example, so the label budget scales with the search rather than with the number of targets found.

Synthetic imagery turns the number of target instances into a parameter rather than a search result, and every instance arrives with an exact box.

How Is Synthetic Satellite Imagery Generated?

Three approaches appear in the published work, and they differ mainly in how much of the scene is synthetic.

ApproachWhat is syntheticExampleStrengthWeakness
Full 3D renderingEverything: terrain, buildings, targets, atmosphere, sensorRarePlanes synthetic set (AI.Reverie, Unreal Engine 4)Full control of geometry, lighting, look angle and metadataLarge asset and environment build; background realism is the hard part
Compositing onto real backplatesOnly the target objectsSIMPL; Rendered.ai's RGB Satellite Imagery channelBackground is real by construction; fast to set up for a new objectTargets must be placed plausibly, with shadows and sensor effects matched at the seam
Generative refinementAppearance of rendered objects or whole imagesGAN-Reskinner (Maxar, 2021); CycleGAN domain adaptation (Rendered.ai)Narrows the visual gap between render and sensorNeeds a corpus of real imagery to learn from; can alter geometry if unconstrained

Full rendering. RarePlanes built its 50,000 synthetic images by generating airport layouts from OpenStreetMap and overhead references, importing them into Unreal Engine 4, spawning aircraft and simulating biomes and weather, then capturing at a simulated GSD of 0.3 m to match the real WorldView-3 tiles. The synthetic set covers 9,331.2 km² and carries about 630,000 aircraft annotations with the same 10 fine-grained attributes (wingspan, wing shape, propulsion, engine count, role and others) as the real labels.

Compositing. SIMPL, from Xu, Huang, Luo, Bradbury and Malof, overlays 3D target models onto randomly drawn patches of real overhead imagery. The models are imaged in a virtual world at 0.3 m per pixel, segmented from a flat background by threshold, and pasted onto 608 by 608 patches of real xView imagery that contain no instances of the target class. Colors and sizes of individual targets are sampled from distributions whose means and variances the designer sets, so prior knowledge about the object goes into a few parameters rather than hand-built models. Rendered.ai's February 2023 post describes the same pattern as a product: 3D models of aircraft, ground vehicles, ships and cranes placed within existing satellite imagery, aircraft constrained to taxiing configurations on tarmac, and metadata such as GSD, collection time and location, gamma and blur driving the placement.

Generative refinement. The Maxar paper (Clement, Schoen, Boedihardjo and Jenkins, 2021) trains a GAN-Reskinner that blends rendered 3D CAD models toward the look of real 30 cm WorldView-3 imagery, and tests neural style transfer as a post-processing step. Rendered.ai trains a CycleGAN on real xView imagery to pull its composites toward the target sensor.

The approaches stack: a compositing pipeline can render its targets with a full sensor model and finish with a generative pass for sensor noise.

What Does The Published Evidence Say?

Three papers report controlled experiments on real satellite test sets. Metrics differ by row, so read each on its own.

StudySetupSynthetic onlySynthetic plus real
RarePlanes, Shermeyer et al., WACV 2021Aircraft detection and instance segmentation, 45,000 synthetic images for training, real WorldView-3 test set, averages of 5 runs70 to 80% of the real-data average recall; mAP drops furtherFine-tuning on about 10% of the real training set reached 91 to 96% of real-only mAP
SIMPL, Xu et al., 2021Seven aircraft classes in xView, YOLOv3, 450 composited background patches per classComparable to training on 50 to 100 real instances per class; about one false detection per km²; over 90% recall on three of the seven classesAugmentation improved detectors with 25 to under 100 real instances; gains of 20% or more in AP and recall when 50 or fewer; benefit faded as real data grew
Synthetic Data and Hierarchical Object Detection in Overhead Imagery, Clement et al. (Maxar), 2021Aircraft (1,103 train, 274 test objects) and rail cars (8,438 train, 3,050 test) in 30 cm WorldView-3, AP50Adding GAN-reskinned synthetics improved AP in every zero-shot case but one, where every detector scored 0.0Mixed: GAN synthetics improved AP in 8 of 12 low-shot trials

Three conclusions hold across all three.

Synthetic data is strongest when real data is scarcest. SIMPL's gains were largest with 50 or fewer real instances and shrank as more were added, and Maxar's zero-shot results were more consistent than its low-shot ones. RarePlanes' authors describe the synthetic set as a prior that eases transfer learning.

A small real set closes most of the remaining gap. The RarePlanes 10% fine-tuning result is the number most people quote. The authors note the false positive rate after fine-tuning still sat slightly above the all-real model, and present the result as a baseline for others to beat.

Response is class dependent. The Maxar paper states that model response to synthetic data seems to be highly class dependent, and SIMPL's recall varied widely across its seven aircraft types. A pilot on one object class does not predict the next.

Vendor results point the same way. Rendered.ai reports a threefold improvement in AP for few-shot rare object detection in xView from a 2021 NGA SBIR with Orbital Insight that used over 500,000 synthetic images. Our own NTT DATA pilot generated targeted synthetic scenarios when the model under-performed on small vessels near coastlines, the same few-shot regime. Treat vendor numbers, ours included, as a reason to run your own test rather than as a transferable result.

Which Public Datasets Should You Test Against?

Synthetic data only earns its place if it moves a number on held-out real imagery, so the real test set comes first. Figures below are from each dataset's paper.

DatasetSensor and GSDContentsTerms
xView (2018)WorldView-3, 0.3 mOver 1 million objects, 60 classes, more than 1,400 km²CC BY-NC-SA 4.0
DOTA (expanded 2021 release)Google Earth, GF-2, JL-1 and aerial1,793,658 instances, 18 categories, 11,268 images, oriented boxesAcademic use only; commercial use prohibited
FAIR1M (2021)Gaofen satellites and Google Earth, 0.3 to 0.8 mMore than 1 million instances in more than 15,000 images, 5 categories and 37 sub-categoriesSee dataset page
RarePlanes (2020)WorldView-3 at 0.31 to 0.39 m depending on look angle, plus synthetic at 0.3 m14,700 real aircraft in 253 scenes across 112 locations in 22 countries; 50,000 synthetic images with about 630,000 annotationsCC BY-SA 4.0, on the AWS Open Data program

RarePlanes is the one to start with if your target is aircraft, because it is the only public set where the real and synthetic halves share a label schema. For other classes, SIMPL's released code (GitHub) is a reference implementation for compositing custom objects onto xView patches. For change detection and SAR, Synthetic Data For Defense ISR lists xBD, SpaceNet 6 and xView3-SAR.

What Does The Synthetic Imagery Have To Match?

Every paper above fixed the synthetic GSD to the real sensor before anything else, and SIMPL's sensitivity study found that errors in target size hurt AP50 substantially for several classes while errors in color had limited effect. The parameters that matter, in rough order:

  1. Ground sample distance. This sets the pixel footprint of every target. A 4.5 m car is 15 pixels at 0.3 m and 9 pixels at 0.5 m, and a detector tuned for one does not read the other. For the commercial sensors most synthetic work targets: WorldView-3 collects 31 cm panchromatic at nadir with 1.24 m multispectral and 3.7 m SWIR (Polar Geospatial Center); WorldView Legion is listed at 30 cm panchromatic and 1.20 m multispectral (UP42), or 0.29 m at nadir per eoPortal; Planet's SkySat orthorectified products have been 50 cm since June 30, 2020, after the orbit was lowered from 500 km to 450 km, with basic super-resolved products at 65 cm at nadir (Planet).
  2. Off-nadir angle. RarePlanes' real GSD ranges from 0.31 to 0.39 m across look angles, and tall objects lean and occlude differently at 30 degrees off-nadir than at nadir.
  3. Sun elevation and shadows. Shadows are often the most visible cue for small objects, and a composited target with no shadow, or a shadow at the wrong angle for the backplate, is an easy tell for a detector to learn.
  4. Product processing. Real tiles have been pan-sharpened, dynamic-range adjusted and atmospherically compensated, as the Maxar paper lists for its own data, and usually compressed on delivery. Synthetic imagery that skips those steps looks cleaner than anything the model will see in production.
  5. Placement context. Aircraft sit on tarmac, ships in shipping lanes and berths, rail cars on rails. SIMPL and Rendered.ai both constrain where targets go, because a detector will otherwise learn that the object can appear anywhere.

Record each of these per image, because when a model fails on a slice of real data the metadata is what lets you generate more of that slice. How To Close The Sim-To-Real Gap For Perception covers setting up the real test before generating anything.

Which Tools Generate Synthetic Satellite Imagery?

As of October 2026, these are the options with public documentation, listed alphabetically. For a broader comparison across domains, see Synthetic Data Platforms Compared.

ToolApproach for overhead imageryModalitiesDelivery
Bifrost StardustFull render from orbit with the optics and resolution of the target satellite, or composite synthetic objects onto real backplatesEO with WorldView-3, Legion 06 and SkySat emulations; custom sensors via a domain adaptation pipeline; coherent SAR in developmentPlatform, described below
Duality AI FalconUnreal Engine based digital twins with GIS-based environments for aerial, ground and marine work; no satellite emulation is described on the product pageCameras, lidar, radar, GPS, IMU and IR; SAR in beta since Falcon 5.3 (August 2025)FalconCloud and desktop editor; free Share tier, $20 a month EDU, commercial tiers via sales
NVIDIA Omniverse ReplicatorFramework for building your own synthetic data pipeline over OpenUSD scenes; you supply the terrain, assets and orbital camera model2D and 3D bounding boxes, depth, normals and other annotatorsSelf-hosted
Rendered.aiRGB Satellite Imagery channel that places 3D models in existing satellite imagery, with CycleGAN domain adaptation; DIRSIG integration for physics-based remote sensingRGB, SAR, IR and thermal, multispectral, hyperspectralPlatform subscriptions from $5,000 a month (Teams) and $15,000 a month (Organizations), or synthetic data as a managed service
SIMPLOpen-source compositing of 3D models onto xView patchesRGBCode, bring your own models and imagery

If your sensor is SAR today, Rendered.ai lists it as available and Duality has it in beta; Stardust's SAR is not available yet. If you have graphics engineers and want everything on your own hardware, Replicator is the most flexible route. To reproduce a paper result on aircraft before buying anything, SIMPL's code and RarePlanes' data are free.

Where Stardust Fits

Stardust is Bifrost's synthetic data platform, and its geospatial work covers both approaches above: it renders physically accurate 3D scenes from orbit with the optics and resolution of the target satellite, or composites synthetic objects onto real overhead backplates. Every ship, aircraft and vehicle is labeled with boxes and masks, and every image carries sun angle, cloud cover and acquisition geometry. Pre-built sensor emulations cover WorldView-3, Legion 06 and SkySat, and a domain adaptation pipeline calibrates custom sensors to match your imagery. Coherent SAR is in development with NTT Data and is not something to plan on today. The U.S. Air Force, NTT DATA and ST Engineering appear on the geospatial page.

The way we use it with customers matches the few-shot pattern in the papers. In the NTT DATA pilot, the team trained on a hybrid of real and synthetic satellite data, found the model under-performing on small vessels near coastlines, and generated synthetic scenarios for that case rather than for everything.

Sources

Frequently Asked Questions

Can you train a satellite object detector on synthetic imagery alone?

Partly. In RarePlanes, detectors trained only on synthetic images kept 70 to 80 percent of the real-data average recall for aircraft, and SIMPL's zero-shot detectors performed about as well as detectors trained on 50 to 100 real instances per class. Both papers still found real images necessary to close the last gap, so plan on a small real set for fine-tuning.

How much real satellite data do you need alongside synthetic data?

Less than you would think, but not zero. RarePlanes reports that pretraining on synthetic imagery and then fine-tuning on roughly 10 percent of the real training set reached 91 to 96 percent of the mAP of a model trained on all the real data. SIMPL found the benefit of synthetic augmentation was largest with 50 or fewer real instances per class and faded as more real data was added.

What is the difference between full 3D rendering and compositing onto real backplates?

Full rendering builds the terrain, buildings and objects in a 3D engine and images the whole scene from orbit, which gives control over geometry and lighting but needs a lot of asset work. Compositing places rendered 3D objects onto patches of real satellite imagery, so the background is already realistic and only the targets need design. RarePlanes used full rendering, while SIMPL and Rendered.ai composite onto real backgrounds.

Which public datasets should I use to test a detector trained on synthetic satellite imagery?

xView has over 1 million objects in 60 classes from 0.3 m WorldView-3 imagery. DOTA has 1,793,658 instances in 18 categories across 11,268 images with oriented boxes, for academic use only. FAIR1M has more than 1 million instances in 37 sub-categories. RarePlanes is the one public set that pairs 14,700 real aircraft with 50,000 synthetic images.

Which satellite sensor parameters does synthetic imagery have to match?

Ground sample distance first, then off-nadir angle, sun elevation, spectral bands and the pan-sharpening and compression the real product goes through. WorldView-3 collects 31 cm panchromatic at nadir, WorldView Legion is a 30 cm class sensor, and Planet's SkySat orthorectified products are 50 cm. A mismatch in GSD changes the pixel size of every target and is the most common reason a synthetic set fails to transfer.

Explore Stardust More Posts