This is an automated archive made by the Lemmit Bot.
The original was posted on /r/stablediffusion by /u/NunyaBuzor on 2024-09-27 23:46:54+00:00.
Code:
Project Page:
Note: Everything information you see below comes from the project page, please take the results with a grain of salt on its quality.
Ctrl-X is a simple tool for generating images from text without the need for extra training or guidance. It allows users to control both the structure and appearance of an image by providing two reference images—one for layout and one for style. Ctrl-X aligns the image’s layout with the structure image and transfers the visual style from the appearance image. It works with any type of reference image, is much faster than previous methods, and can be easily integrated into any text-to-image or text-to-video model.
Ctrl-X works by first taking the clean structure and appearance data and adding noise to them using a diffusion process. It then extracts features from these noisy versions through a pretrained text-to-image diffusion model. During the process of removing the noise, Ctrl-X injects key features from the structure data and uses attention mechanisms to transfer style details from the appearance data. This allows for control over both the layout and style of the final image. The method is called “Ctrl-X” because it combines structure preservation with style transfer, like cutting and pasting.
Results of training-free and guidance-free T2I diffusion with structure and appearance control
Results of training-free and guidance-free T2I diffusion with structure and appearance control
Ctrl-X is capable of multi-subject generation with semantic correspondence between appearance and structure images across both subjects and backgrounds. In comparison, ControlNet + IP-Adapter often fails at transferring all subject and background appearances.
Ctrl-X also supports prompt-driven conditional generation, where it generates an output image complying with the given text prompt while aligning with the structure of the structure image. Ctrl-X continues to support any structure image/condition type here as well. The base model here is Stable Diffusion XL v1.0.
You must log in or register to comment.