ATISS: Autoregressive Transformers for Indoor Scene Synthesis

2021

Conference Paper

avg

The ability to synthesize realistic and diverse indoor furniture layouts automatically or based on partial input, unlocks many applications, from better interactive 3D tools to data synthesis for training and simulation. In this paper, we present ATISS, a novel autoregressive transformer architecture for creating diverse and plausible synthetic indoor environments, given only the room type and its floor plan. In contrast to prior work, which poses scene synthesis as sequence generation, our model generates rooms as unordered sets of objects. We argue that this formulation is more natural, as it makes ATISS generally useful beyond fully automatic room layout synthesis. For example, the same trained model can be used in interactive applications for general scene completion, partial room re-arrangement with any objects specified by the user, as well as object suggestions for any partial room. To enable this, our model leverages the permutation equivariance of the transformer when conditioning on the partial scene, and is trained to be permutation-invariant across object orderings. Our model is trained end-to-end as an autoregressive generative model using only labeled 3D bounding boxes as supervision. Evaluations on four room types in the 3D-FRONT dataset demonstrate that our model consistently generates plausible room layouts that are more realistic than existing methods. In addition, it has fewer parameters, is simpler to implement and train and runs up to 8x faster than existing methods.

@inproceedings{Paschalidou2021NEURIPS,
  title = {ATISS: Autoregressive Transformers for Indoor Scene Synthesis},
  author = {Paschalidou, Despoina and Kar, Amlan and Shugrina, Maria and Kreis, Karsten and Geiger, Andreas and Fidler, Sanja},
  booktitle = {Advances in Neural Information Processing Systems 34},
  volume = {15},
  pages = {12013--12026},
  editors = {M. Ranzato and A. Beygelzimer and Y. Dauphin and P. S. Liang and J. Wortman Vaughan},
  publisher = {Curran Associates, Inc.},
  address = {Red Hook, NY},
  month = dec,
  year = {2021},
  doi = {},
  url = {https://papers.nips.cc/paper_files/paper/2021/hash/64986d86a17424eeac96b08a6d519059-Abstract.html},
  month_numeric = {12}
}