Note that this technique and its results are unrelated to the infamous "spiral" ControlNet images a couple months back: https://arstechnica.com/information-technology/2023/09/dream...
Per the code, the technique is based off of DeepFloyd-IF, which is not as easy to run as a Stable Diffusion variant.
I missed it, what was infamous about it?
It created a backlash because a) it was too popular with AI people hyping "THIS CHANGES EVERYTHING!" and people were posting low-effort transformations to the point that it got saturated and b) non-AI people were "tricked" into thinking it was a clever trick with real art since ControlNet is not ubiquitous outside the AI-sphere, and they got mad.
I rather liked it and actually didn't get to see as many examples as I wanted to.
Is there a good repository anywhere or is it just "wade through twitter"?
not a repository as such but i linked to some good examples in my sept recap
https://www.latent.space/p/sep-2023
https://github.com/swyxio/ai-notes/blob/main/Monthly%20Notes...
It is real art.
Did you mean to say it's _related_? The original "spiral" image by Ugleh is explicitly credited in the "Related Links" section.
It’s a similar topic which is why they credit it but the mechanism is much different.
I haven't dug in yet, but it _should_ be possible to use their ideas in other diffusion networks? It may be a non-trivial change to the code provided though. Happy to be corrected of course.
I suspect the trick only works because DeepFloyd-IF operates in pixel space while other diffusion models operate in the latent space.
I always thought it was weird that this idea took off with that particular controlnet model. Many other controlnet models when combined with those same images produce excellent and striking results.
The ecosystem around Stable Diffusion in general is so massive.
Other ControlNet adapters either preserve the high-level shape not enough or preserve it too well, IMO. Canny/Depth ControlNet generations are less of an illusion.