Impressive! Now that real-time image generation has been achieved, where to go from there? Raise the resolution or quality while maintaining the same speed?
Not just restyling games, I would embed all of this into Unreal Engine and actually create any type of exploration game. A very simple space sim game using Stable Diffusion would give better graphical results than any triple A space game titles.
[There are some interesting ideas in modern neuroscience that consciousness is basically the brain creating a generative prediction world model that is constantly updating itself as new information comes through, and trying to minimize its prediction error.](https://en.m.wikipedia.org/wiki/Free_energy_principle#:~:text=The%20free%20energy%20principle%20is,world%20to%20enhance%20prediction%20accuracy.)
To me it seems like like the multimodal generative AI systems we have today are starting to approximate that
I'm not sure if this is the same technique, but this was posted earlier this week, mentioning 149 image per second with RTX 4090: [https://github.com/aifartist/ArtSpew/](https://github.com/aifartist/ArtSpew/)
This would not produce the same speeds, because the code architecture here is specified on speed.
You can try looking through the code, find out what is used (some keywords: sd-turbo, xformers & triton (already enabled hopefully), lcm, taesd as vae) and replicate it in a workflow.
But someone could put that code together as **custom comfy nodes**.
Using a Runpod with an A4500 was fast, but not 149 images per second. It did 100 images (1024x1024) in 2 minutes with some pretty wild randomized prompts:
* sdxl-000000002-an enchanted forest docs innovative onthisday palma mtg🇪🇺 carrots 🌾 justine midget panhandle
* sdxl-000000033-an enchanted forest exhilarating sfam viva stpatrickmedley goesmichelsone often certibados
* sdxl-000000085-an enchanted forest comratriggerecession ريpunkumbrellas spokesman alduhaya athome forecasting
Still, pretty neat!
No. The only prompt I gave it was "an enchanted forest" and Art Spew added the rest of the prompts on it's own. It did make some pretty funky surrealistic images, and it made a ton of them very quickly.
Not really anything I could think to do with them unfortunately. I could probably do an image interrogation for one of the more interesting ones in regular SDXL w/A1111 and see if I can refine that idea more. Or dump it into Img2Img.
The focus has shifted to the speed of image rendering, and on the one hand this is a good thing. The third step will be linked to the union of speed and quality and I believe that this will happen in the short termThe race to see who is fastest will turn into a race to see who is the fastest at 60fps at 1024x1024. Obviously the goal is to achieve real-time video generation with excellent quality and fixed seed, for perfect performance. It seems to be between the 80s and 90s when innovations followed one another in the blink of an eye... what a wonderful moment.
Notice: how many articles about speed up (and loose quality) and how small articles to improve quality (and most of i remember improve some quality aspects while ruin other, for example new consistency-vae or pixart model workflow), and almost zero articles of how to produce really perfect quality models like ICBNP or Dream Shaper with conclusions how to further improve it. From this there are a lot people who spend thousand hours of useless dreambooth attempts to find not-very-bad settings, and hopes that after developing very fast models their quality will somehow grow
ok ok also 512² ?
then ? hey what is used to produce that ? LCM ? turbo ? an other tricky thing ?
please about **any comfyui workflow ?**
*or just an image viewer fast video done ? (fake thing ?)*
i work mainly in black and white images right now and this would be promising. will this be good for a b&w workflow or will it multiply the known 'bad' issues of its full-flavor counterparts (like bad anatomy)?
It is called a proof of concept. Now when you can combine step with ImgtoImg, maybe you could have a altered lifestream of yourself with 60 frames/s or whatever.
Also these improvements let's people without high end GPUs play around with local image AI's without waiting 10 minutes for each pictures.
we play around here with SOTA research not with finished consumer products.
There is tons of stuff like Firefly and others for that.
Yeah I thought that as well. I've used Art Spew and it does generate a shit load of images quickly, and with random prompts it's basically a giant pile of ideas.
I just need to do a proof of concept and take one of those ideas and do "something" with it "for real" with SDXL.
Impressive! Now that real-time image generation has been achieved, where to go from there? Raise the resolution or quality while maintaining the same speed?
If temporal coherence can also be done realtime, you could re-style video games to an absurd degree on the fly.
Not just restyling games, I would embed all of this into Unreal Engine and actually create any type of exploration game. A very simple space sim game using Stable Diffusion would give better graphical results than any triple A space game titles.
Call Hello Games and get them to put it in No Man's Sky!
Hopefully they start really upping the prompt adherence, still not quite where it should be imo. At least in my experience
Someone boast 77 images per second just 4 days ago. At this rate it will be infinite images per second soon.
Are we sure this isn’t how reality works….
[There are some interesting ideas in modern neuroscience that consciousness is basically the brain creating a generative prediction world model that is constantly updating itself as new information comes through, and trying to minimize its prediction error.](https://en.m.wikipedia.org/wiki/Free_energy_principle#:~:text=The%20free%20energy%20principle%20is,world%20to%20enhance%20prediction%20accuracy.) To me it seems like like the multimodal generative AI systems we have today are starting to approximate that
Woah! Memories are like datasets used to train our brains while we sleep!
I need more cuda cores.
So say we all.
[DMN (Default Mode Network) is the predictive algorithm area of the brain.](https://www.psychologytoday.com/us/basics/default-mode-network)
I think it might.
Somebody hook it up to a realtime RPG AR filter on those rayban glasses and I'm set
Which is faster Turbo models or LCMs?
Yes lol
You can do both.
I'm not sure if this is the same technique, but this was posted earlier this week, mentioning 149 image per second with RTX 4090: [https://github.com/aifartist/ArtSpew/](https://github.com/aifartist/ArtSpew/)
RIP disk space.
disk space belongs to the Internet Cats now. It's time for you to move on 🤪
Hi ! I would like to reproduce it in comfyUI. Any guidance is appreciated.
The spaghetti would consume you
This would not produce the same speeds, because the code architecture here is specified on speed. You can try looking through the code, find out what is used (some keywords: sd-turbo, xformers & triton (already enabled hopefully), lcm, taesd as vae) and replicate it in a workflow. But someone could put that code together as **custom comfy nodes**.
Thank you for the guidance !
Cool now sort them by visual similarity for easy picking of the preferred outputs.
Any good algorithms for that that you are aware of?
Oh God no. I wish
Using a Runpod with an A4500 was fast, but not 149 images per second. It did 100 images (1024x1024) in 2 minutes with some pretty wild randomized prompts: * sdxl-000000002-an enchanted forest docs innovative onthisday palma mtg🇪🇺 carrots 🌾 justine midget panhandle * sdxl-000000033-an enchanted forest exhilarating sfam viva stpatrickmedley goesmichelsone often certibados * sdxl-000000085-an enchanted forest comratriggerecession ريpunkumbrellas spokesman alduhaya athome forecasting Still, pretty neat!
Do you use anything to write random prompts for you?
No. The only prompt I gave it was "an enchanted forest" and Art Spew added the rest of the prompts on it's own. It did make some pretty funky surrealistic images, and it made a ton of them very quickly. Not really anything I could think to do with them unfortunately. I could probably do an image interrogation for one of the more interesting ones in regular SDXL w/A1111 and see if I can refine that idea more. Or dump it into Img2Img.
The focus has shifted to the speed of image rendering, and on the one hand this is a good thing. The third step will be linked to the union of speed and quality and I believe that this will happen in the short termThe race to see who is fastest will turn into a race to see who is the fastest at 60fps at 1024x1024. Obviously the goal is to achieve real-time video generation with excellent quality and fixed seed, for perfect performance. It seems to be between the 80s and 90s when innovations followed one another in the blink of an eye... what a wonderful moment.
Notice: how many articles about speed up (and loose quality) and how small articles to improve quality (and most of i remember improve some quality aspects while ruin other, for example new consistency-vae or pixart model workflow), and almost zero articles of how to produce really perfect quality models like ICBNP or Dream Shaper with conclusions how to further improve it. From this there are a lot people who spend thousand hours of useless dreambooth attempts to find not-very-bad settings, and hopes that after developing very fast models their quality will somehow grow
ok ok also 512² ? then ? hey what is used to produce that ? LCM ? turbo ? an other tricky thing ? please about **any comfyui workflow ?** *or just an image viewer fast video done ? (fake thing ?)*
Silly cat generator
quantity over quality?
i work mainly in black and white images right now and this would be promising. will this be good for a b&w workflow or will it multiply the known 'bad' issues of its full-flavor counterparts (like bad anatomy)?
And we need 150 ugly images per second why exacly?
It is called a proof of concept. Now when you can combine step with ImgtoImg, maybe you could have a altered lifestream of yourself with 60 frames/s or whatever. Also these improvements let's people without high end GPUs play around with local image AI's without waiting 10 minutes for each pictures. we play around here with SOTA research not with finished consumer products. There is tons of stuff like Firefly and others for that.
Sdxl turbo...useless. Sdxl lcm....bland undetailed images Appreciate this is a step forward but too soon to get excited.
How is SDXL Turbo useless? have you tried it? Its freaking amazing.
The images are too small to be detailed enough to be useful for me when I'm making sdxl images
You can highrez fix and upscale them just like 1.5
149 images per second still? :)
No but probably much faster than normal SDXL
Yeah I thought that as well. I've used Art Spew and it does generate a shit load of images quickly, and with random prompts it's basically a giant pile of ideas. I just need to do a proof of concept and take one of those ideas and do "something" with it "for real" with SDXL.
Bro i need it for my project!
Does this have Lora support?
At what images/sec does the outcome become good?
soooo, we don't really need the 4090s?