Scene Retargeting: Learning Object Placement
with Analogical Transfer
*Corresponding author
Interactive simulations of embodied AI or spatial computing applications build on realistic 3D scenes that support daily activities. However, sparse, irregular layout structures impose scene-specific physical constraints, making it hard to define a generalizable framework for generating similar functional context. We formalize Scene Retargeting as stably transferring the semantically coherent spatial organization across layouts, rather than relying on textual descriptions or pairwise relationships. Our cluster-wise transfer flexibly handles mismatched object instances and adapts to distinctive floor plans. We optimize to preserve the rich semantic context of individual clusters by respecting the spatial distribution of foundation features. We can then impose physical constraints to refine wall contacts, pairwise alignment, or clear passageways and openings. Our framework outperforms state-of-the-art methods on layout generation on the 3D-FRONT dataset, and demonstrates downstream applications including real-to-sim transfer, analogical trajectory transfer, and multi-reference composition.
Method Overview. In stage 1 (Cluster Placement), functional clusters decomposed from the reference scene are placed onto the target floor plan by a learned cluster placer. In stage 2 (Object Arrangement), target object tokens cross-attend to Concerto features of the transferred reference layout within the layout decoder to assign object-level poses. Stage 3 (Analogical Refinement) matches objects across scenes, optimizes pairwise relations and wall clearances measured on the reference, and enforces physical validity to produce the final 3D layout.
Retargeting process. We visualize the full pipeline on a single example, from the reference scene, target floor plan, and target inventory to the final layout, showing how each stage progressively arranges the target room.
Qualitative comparison. Given the same reference scene and target room, baselines receive the reference through the channel they accept (a verbalized instruction or the target floor plan). Our hierarchical framework preserves fine-grained spatial relationships of the reference across diverse room geometries while strictly respecting physical constraints.
Our framework supports various downstream scenarios without fine-tuning. Because spatial relations are faithfully preserved, content authored in a reference scene carries over to the retargeted scene.
A human interaction path and a mobile robot trajectory authored in the reference scene are transferred to the retargeted room. Waypoints attached to reference objects map directly to their target counterparts, preserving navigational context across different room geometries.
A 3D camera trajectory is transferred and rendered from a first-person view in both scenes. The inset shows the camera path on the floor plan.
A real ScanNet++ scan serves as the reference. A single capture is instantiated across different synthetic floor plans and object inventories, yielding a family of simulation environments.
Since transfer operates at the functional cluster level, clusters drawn from two independent reference rooms are composed into a single target layout.
A special case where the object inventory is fixed and only the room changes, as when a household moves its furniture to a new home. Clusters are relocated and wall clearances are re-established without any retraining.
@misc{kim2026sceneretargetinglearningobject,
title={Scene Retargeting: Learning Object Placement with Analogical Transfer},
author={Minkwan Kim and Junho Kim and Seungmin Lee and Changwoon Choi and Young Min Kim},
year={2026},
eprint={2609.36801},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2609.36801},
}