Bibliographic Details
| Title: |
TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation |
| Authors: |
Berthelot, David, Autef, Arnaud, Lin, Jierui, Yap, Dian Ang, Zhai, Shuangfei, Hu, Siyuan, Zheng, Daniel, Talbott, Walter, Gu, Eric |
| Publication Year: |
2023 |
| Collection: |
Computer Science |
| Subject Terms: |
Computer Science - Machine Learning, Computer Science - Computer Vision and Pattern Recognition |
| More Details: |
Denoising Diffusion models have demonstrated their proficiency for generative sampling. However, generating good samples often requires many iterations. Consequently, techniques such as binary time-distillation (BTD) have been proposed to reduce the number of network calls for a fixed architecture. In this paper, we introduce TRAnsitive Closure Time-distillation (TRACT), a new method that extends BTD. For single step diffusion,TRACT improves FID by up to 2.4x on the same architecture, and achieves new single-step Denoising Diffusion Implicit Models (DDIM) state-of-the-art FID (7.4 for ImageNet64, 3.8 for CIFAR10). Finally we tease apart the method through extended ablations. The PyTorch implementation will be released soon. |
| Document Type: |
Working Paper |
| Access URL: |
http://arxiv.org/abs/2303.04248 |
| Accession Number: |
edsarx.2303.04248 |
| Database: |
arXiv |
