He is well known for creating nodes, workflows for new open-source models.

So, do you guys think Flux 3 will be open-source/open-weight model?

Did a few edits with Krea 2 Identity Edit LoRA v1.2, every one of them is just a single sentence prompt. No masks, no inpainting.

Seahorse guy turned to profile, snake swapped for a scarf, outfit + pose taken from two refs, and the last one is just an expression change.

Honestly didn't expect it to keep the scene this intact, background and lighting don't move at all. Also learned that one short sentence works way better than writing a whole paragraph.

Prompts are on the images if anyone's curious.

Everything I do here is just experiments. I'd be really happy to hear any friendly tips or advice you have.

In part 1 https://www.reddit.com/r/StableDiffusion/comments/1v1smgn/im_training_an_image_model_from_scratch_part_1_my/ I trained my own VAE. A VAE is nice but it doesn't actually make pictures, it just squashes and rebuilds them. So this time I sat down to train the real generator, the part that turns text into an image.

The setup: one machine, one RTX 5090. No cluster, no rented pods. One card. So the dataset stayed small (I started with about 37k image and caption pairs). I wasn't trying to ship anything yet. I just wanted to know one thing: can this even learn, and how does it fall apart.

It falls apart constantly. And almost never for reasons that have anything to do with AI.

Attempt one: the model that could only draw snow.

My first version could only "read" the caption as one blurry summary instead of actual words. It trained, the loss dropped for a bit, and then sat still forever. I let it run way too long out of stubbornness.

The results were amazing in the wrong way. "Snowy mountain" actually looked like a snowy mountain. Everything else melted. A portrait came out as a melting face. A puma in snow was grey soup. The model had basically decided that "vaguely textured blob" was the safe answer to everything and fully committed.

The bug that ate 92,000 steps.

This is my favorite one. I had a feature turned on that keeps a smoothed backup copy of the model. Because of one copy paste mistake, every time the trainer stopped to save a preview image, it overwrote the live model with the older backup and never switched back.

So every thousand steps, the model quietly threw away a thousand steps of progress and reset itself. I stared at the weird loss graph for days thinking it was some deep training problem. Nope. I was deleting my own work on a timer. Roughly 92,000 steps of training, gone, because of two lines of code.

Attempt two: a real architecture, and my own code fighting back.

I rebuilt it properly this time so the model actually reads the full caption word by word instead of one blurry summary. And since the small version was clearly learning, I decided to go bigger and feed it a much larger dataset. Turning all that new data into the format the trainer needs is where the fun started.

The model itself was fine. Everything around it was not.

First launch of the new run: instant crash on the very first batch, because my data loader tried to open all 71 of the new dataset files at once and choked. It worked fine back when there were only a few files. Nothing teaches you about scale like scale.

Building the bigger dataset ran out of memory halfway through, then left 47GB of half finished junk files on my drive as a goodbye present.

Printing a single checkmark character crashed an entire training run. Not the model, not the data, just one tiny symbol in a log line. I killed my own training with a checkmark.

My launch script refused to run for an entire evening because of one missing backslash in a path.

Did it actually work?

Yeah, and surprisingly fast. "Red dress" gave me a red dress. "White cat" gave me a correctly shaped white blob. "Red sports car" started as a literal jar (it heard "car," drew a jar, I have no notes) and later turned into an actual red car. Strawberries stayed the wrong color for an embarrassingly long time.

The weirdest part: the loss number barely moved this whole time while the images kept clearly getting better. Turns out for this kind of training the loss just isn't the thing that tells you quality. Watching a flat line for days while your eyes say it's improving is its own special kind of stress.

I stopped it on purpose, not because it broke, but because I'd figured out the next real upgrade needed a better VAE, which means starting the generator over from scratch anyway.

That's the next part. Short version so far: the model was never the hard part. My own code was.

Want part 3? Want to hear about more of my mistakes? Say so in the comments and I'll write up what happened when I tried to rebuild the VAE.

https://github.com/shootthesound/ComfyUI-KMS
See what your seeds have in mind before you spend the steps. Quick previews appear on the node, click your favourite and only that image gets rendered. You can click others after. Ideas welcome. Krea 2 workflow example in the node pack, but should work with any model. T2I and I2I supported. Cheers, Pete

Qwen2.5, Qwen3, Qwen3.5 are all serviceable models for prompt enhancement, but there are much better options. I use all of these models for prompt enhancement. Which model I use depends on what I'm prompting. My favorite is Mistral 7B/Llama3.3 8B by far for image prompts, and WizardLM-2 for video prompts. SuperGemma4 is good for very basic prompts or prompts that you want accurately reworded.

I realize these are older models, but they are well suited to the task. My other requirement for a prompt enhancing LLM is that it fully loads on 8gb VRAM. I'm not weighing in on image captioning or anything else besides prompt enhancement. Disclaimer: I DO mention my custom node several times in the comments, as all of my testing was accomplished using said node.

Using the base prompt, "A woman at the pier".

Mistral 7B - Best Overall

Strengths: Creative scene construction and cinematic detail.

With the same enhancement framework, Mistral consistently produces the richest and most imaginative expansions. It doesn't simply populate the required categories, it invents believable details that reinforce the mood, such as the sketchbook, discarded sandals, and weathered textures. The result feels less like a checklist and more like a scene from a film.
mradermacher/Mistral-7B-Instruct-v0.3-abliterated-GGUF · Hugging Face

SuperGemma 4B - Concise

Strengths: Precision, restraint, and prompt fidelity.

SuperGemma takes a conservative approach. It faithfully fills in the structure provided by the system prompt while making relatively few creative leaps. The result is concise, highly controllable, and stays very close to the user's original intent. It's an excellent choice when consistency is more important than artistic embellishment.
mradermacher/supergemma4-e4b-abliterated-GGUF · Hugging Face

Llama 3.3 8B - Best Balance

Strengths: Balanced descriptive enhancement.

Llama 3.3 strikes a middle ground between creativity and restraint. It expands the prompt naturally, adding enough detail to create a complete visual scene without feeling overly embellished. It tends to produce outputs that read like professional photography descriptions, making it a solid all-around prompt enhancer.
mradermacher/Llama-3.3-8B-Instruct-128K_Abliterated-GGUF · Hugging Face

WizardLM-2 - Most Verbose

Strengths: Natural language and immersive descriptions.

WizardLM-2 excels at turning the framework into smooth, human-like prose. Rather than feeling generated from a template, its prompts flow naturally while still covering all of the structural elements required by the system prompt. It consistently produces scenes that feel cohesive and immersive.
mradermacher/WizardLM-2-7B-abliterated-GGUF · Hugging Face

If you have any models you like better, please comment them below and I will look into them! Do you agree or disagree with my list?

Everything in this clip is AI-generated — video and audio together in one model, no TTS, no dubbing. The only thing carrying her between shots is one identity sentence repeated word-for-word, plus the cross-shot memory bank the workflow wires up.

The merge: JoyAI-Echo holds a character's face across shots but has a weak voice; LTX-2.3-distilled has the good voice but drifts the face. I took each model's strong branch — that's the whole trick.

Five builds, so it runs on almost anything:

Q8_0 GGUF (23 GB) — measured ~0.6% from bf16, runs on any GPU

Q5_0 GGUF (15.5 GB) — 16 GB cards

INT8 ConvRot (27 GB) — loads in stock ComfyUI 0.27+, no custom nodes, 1.5–2x faster on 30-series

fp8 (23 GB) — 40/50-series speed path

bf16 (43 GB) — reference

Try it without downloading anything: free ZeroGPU demo Space (HF's open-source team built the first version of it, which was a nice surprise): https://huggingface.co/spaces/joeygambino/joyai-echo-ltx23-surgical

All builds + the ComfyUI workflow/node patch + a gallery with per-build demo clips and the actual quantization measurements: https://huggingface.co/spaces/joeygambino/one-merge-five-builds

Every fidelity number on the cards comes from pushing identical activations through the real weights — not eyeballing renders (matched-seed comparisons mislead for diffusion; the gallery explains why). Licenses: LTX-2 Community + JoyAI-Echo research/non-commercial — the stricter term governs outputs.

Happy to answer setup questions — there's a full step-by-step INSTRUCTIONS.md in the workflow pack written after real user feedback.

Even when it goes wrong the results are impressive.

Here's the Lora

https://huggingface.co/Lightricks/LTX-2.3-22b-IC-LoRA-Clean-Plate

The workflow is the basic LTX-2.3_V2V_ICLoRA_Single_Stage_Distilled.json

The prompt:

"An empty clean plate of the exact same location: identical background, environment, structures and lighting as the source video, with no people, no humans, no figures, no cars, no vehicles and no body parts such as arms, hands or legs anywhere in the frame. Static photorealistic footage, natural light, high detail.

"

I was succefull to generate the final video with bigger resolution upscale without getting bottom artefacts and deformations.

but this resolutions only can be achieve with a RTX 6000 PRO

I love ComfyUI as an engine. I do not love it as a daily driver. So I spent the last few months building Mix Studio, a 100% free & open source interface that runs everything through ComfyUI in the background while giving you an actual app experience.

GitHub: https://github.com/BlackMixture/Mix-Studio
Showcase and download: https://blackmixture.github.io/Mix-Studio/
Tutorial: https://youtu.be/w2CokhlBFRA

GPL-3.0, the same license as ComfyUI. Windows + NVIDIA for now.

The screenshots show the main desktop workspaces, but the entire interface is also optimized for phones and tablets.

Current v1.0.1 Features:

• Curated image, editing, video, and upscale workflows: Krea 2, Flux 2 Klein 4B/9B, Qwen Image Edit 2511, LTX 2.3, Wan 2.2, 10Eros, and SCAIL 2.
• Image-generation tools: Inpainting, outpainting, SeedVR2 and Ultimate SD upscaling, regional prompting, Depth Anything V3 guidance, image-to-image, style references, and model-aware recommendations for steps, CFG, samplers, and schedulers.
• Desktop and mobile interface: On the same Wi-Fi, open the displayed address on your phone and start generating. With Tailscale, you can connect through a private link while away from home. Your desktop GPU still does all the work.
• Multi-image editing: Add multiple inputs and reference them using dynamic @ Image cards, removing the guesswork around which image should control each part of the edit.
• Regional prompting with Krea 2: Draw boxes and assign each region its own prompt, LoRA stack, and optional reference image.
• LoRA management: Stack LoRAs, add thumbnails and trigger words, save presets, adjust strength quickly, and use LoRA Hunting to generate a comparison series across different strengths.
• Contextual prompt suggestions: Mix Studio learns phrases you repeatedly use with specific LoRA combinations and offers them as one-tap suggestions. These can also be configured manually.
• Library management: Click any image or video to restore its exact generation settings. Search, group, organize work into folders, compare edits, and drag Library media directly into compatible workflows.
• Private profiles and locked folders: Create separate PIN-protected profiles with their own galleries, folders, LoRA presets, and settings. Individual folders can also be locked, keeping private generations out of your everyday library
• LTX Director Mode: A streamlined workspace built around the excellent LTX Director nodes, supporting timelines, keyframes, video extension, audio, and more.
• Video finishing: Optional 2× or 3× RIFE frame interpolation and NVIDIA RTX 4K video upscaling.
• Built-in dependency manager: Pick a workflow and install the exact models and custom nodes it requires, or run the full one-click setup.
• Automatic ComfyUI integration: Mix Studio detects your ComfyUI installation, reuses existing models and LoRAs, and guides installation if ComfyUI is not present. Generated images retain their ComfyUI workflow metadata, so you can drag them directly back into ComfyUI.
• Hardware-aware configuration: Mix Studio detects your GPU and recommends suitable quantization and generation settings. v1.0.1 also adds a low-VRAM profile beginning at 4 GB, although practical limits still depend on the selected model.

Additional screenshots and release overview: Free Patreon post (no paywall)

Workflow contributions welcome in the Discussions tab. Ask me anything and I hope you all enjoy creating! 🤙🏾

With help of ai I had this workflow build for creating videos using store board image panels about to test see how goes.if anyone interested help me test ill post a link to the json

Included:

3-column × 5-row storyboard loader

Panel selector for panels 1–15

Automatic selected-panel cropping

Selected-panel preview

Selected panel connected to the existing I2V reference path

Separate Global Prompt

Separate Main Scene Prompt

Automatic global + scene conditioning combination

Existing multi-LoRA nodes and Ctrl+B toggles preserved

Instructions and color-coded workflow sections

After loading it, select your storyboard in the LOAD STORYBOARD node and change PANEL NUMBER to choose the shot. ❶

Part 2 of some experiments to see how far we can push Krea 2 Identity Edit. I Continue to be super impressed! All context images + prompts + PNG outputs are in my dropbox
https://www.dropbox.com/scl/fo/shnbjsj5obwtx4s4c3ecx/ALj6gE3xsjtM6X5cRMp3Yho?rlkey=x2s0zpw0h07m5t2vgw15sq1lp&dl=0 (see batch3 folder for this set)

If you have been under a rock the last couple days the HuggingFace is: https://huggingface.co/conradlocke/krea2-identity-edit and it's a v1.2 of the LoRA that turns Krea 2 Turbo into a surprisingly good Image Edit model with a fast speed, better identity preservation, and higher detail quality output to comparable open-weight Image Edit models like Qwen Image Edit 2511 Lightning. My generations were done with the FP8 Krea 2 Turbo with BF16 LoRA

I got in touch with Lars (creator conradlocke on huggingface) to ask how we could help. His response:

User feedback. what people like, what they complain about, which features they wish it had. That directly shapes the next version. Even a rough read on the common requests would help a lot.

In my experience 80% of what I try works, but here are things that fail every time:
- zoom the camera out
- regenerate the photo as a professional studio portrait
- remove dust and scratches from the image
- professionally restore the photo
- convert to full body portrait
- move the subject to the background

For moving a subject backwards or reframing them away from the camera, the only thing I found that works is "move her backwards, we can see her feet". Both phrases must be used, one doesn't seem to be enough.

Perhaps this could be a useful thread for him. What have you tried that doesn't work? Any particular neat prompts? (shout out to [u/HollyGrandeux](u/HollyGrandeux) for the cool 3x3 prompt)

If you have leads on good training data he's also looking for that as well:
https://huggingface.co/conradlocke/krea2-identity-edit/discussions/39

Finally, if you are following the project and using the project, he set up a donation page for anyone who wants to chip in toward the GPU compute that trains future versions.
https://ko-fi.com/conradlocke

I have no affiliation with Lars or the project, I'm just excited for this Image Edit capabilities (with actual reliable identity) to continue to evolve on my current favorite model 🤙

If you're looking for a Comfy workflow template, just look more closely on the huggingface page, you'll find it, I believe in you!

https://bfl.ai/models/flux-3 - edit: the link is now dead, a small placeholder page was up briefly with nothing but the quoted text below.

"A breakthrough in control, realism, and world understanding — one multimodal model generating image, video, audio and action."

Video source: https://x.com/robrombach/status/2079679239847047480

Robin Rombach is the CEO of BFL

So V3 dropped last week and the main new thing is editing. Not just generating infographics but actually fixing them after the fact.

I've been wanting this forever because every time I generate one and spot a typo it's just... reroll the whole thing and pray.

Ran two tests over the weekend.

Test 1: local content insertion (no bbox)(pic1)

Add green monochrome pixel text "DESIGN MODE: ON" to the retro computer screen, matching the screen perspective, grain noise, and glass reflection.

The text follows the screen perspective and picks up the CRT vibe automatically. Grain noise, glass reflection, all there. Didn't touch anything outside the screen..

Test 2: global style swap (pic 2)

Change the image style to Lego style and Chinese New Year style, replace all text with pixel font.

Lego textures plus red/gold Chinese New Year palette, everything still legible in pixel font.

V3 also supports other editing modes I haven't fully tested yet: local text replacement via bbox, natural language text editing, and layout beautification. Will report back after I test those.

GitHub: GitHub - OpenSenseNova/SenseNova-U1: SenseNova-U series: Native Unified Paradigm with NEO-unify from

HF: https://huggingface.co/sensenova/SenseNova-U1-8B-MoT-Infographic-V3

Krea 2 trains incredibly fast for likeness and you are are probably overtraining. The following settings are more than enough to achieve almost perfect likeness.

Dataset Tips

• Image Count: Aim for 20 high-quality images, up to 40 if the dataset is lower quality.
• Full Body Shots: Include at least two to five full body images so the model understands the person/character's height and physique proportions.
• Variety: Use different hairstyles and situations in your dataset. This gives you more flexibility when changing features later without breaking the likeness.

Captioning:

• Use the autocaption feature in ai-toolkit.
• Do not use the person/character's actual name in the captions. Create a unique shortened trigger word instead (e.g., "Jane Doe" becomes "jnedoe").
• If your images are low quality or vintage, add tags like "low quality" or "vintage" to the captions. This stops the model from learning and outputting those artifacts in the images.

Technical Settings

• LoKr Factor: 16
• Training Resolution: 768
• Total Steps: 3000 (likeness is usually done by step 750)
• Settings: Automagic2, Sigmoid, and Balanced
• Advanced Settings: Enable Do Differential Guidance at the default level of 3

VRAM Usage: About 18 to 20GB.

Time: On an RTX 3090, a 750-step run takes about 40 to 45 minutes from start to finish.

Step Count Adjustment: If your dataset quality is lower than average, add a couple of hundred extra steps to get the best results.

Issues: Highly detailed features like tattoo's may not appear correctly at the 768px resolution, you may need to up the quality to 1024 or 1280 and specifically caption each one in the dataset. Even then they may not come through completely as some details are usually lost during generation.

All images are generated at 4MP with Res/2s at 10 steps (about 2-4 minutes per image on a 3090 with Krea Raw int8-convrot and the r256 turbo lora (plus a custom high resolution lora I'll be posting to huggingface))

Full config behind my $20 Patreo--- lol just kidding 😂, grab the config here: ai-toolkit config

Full Res Slow.pics Comparison

HighRes LoKr Model

I've been spending some time experimenting with Krea 2's new ControlNet Depth model, and it's been a lot of fun so far.

One thing that stood out to me is how well it preserves the overall composition and depth of the original image while still generating a fresh result. It feels like a really useful tool when you want to keep the structure of a scene without simply recreating the exact same image.

I'm still experimenting with different prompts, depth maps, and settings, but I'm already getting some results that I really like. I think there's a lot of creative potential here, and I'm looking forward to seeing what else I can do with it.

I'm planning to put together a full tutorial once I've explored it a bit more and have a workflow I'm happy with. I'd rather spend some extra time learning it properly first than rush out a guide.

If you've been using the new Depth model too, I'd love to hear what you've discovered or any interesting techniques you've found!

I converted FLUX.1-dev to native ComfyUI ConvRot formats.

High-fidelity INT8 variants cut peak VRAM at 1024²/20 steps:
Partial INT8 24.09→20.35 GiB (−15.5%);
Whole W8A8 16.30 GiB (−32.3%);
W8A8+INT8 T5 16.27 GiB (−32.5%).

More details:
https://huggingface.co/SearchingMan/FLUX.1-dev-ConvRot

Model avialable on civitai:
https://civitai.com/models/2797469/flux1-dev-convrot

Is there currently a way, in Krea 2 specifically, to do what Klein 9B does with reference images? When generating, I can drop in a face reference image and prompt something like "use image X for the face," and it just works.

Tried the krea2-identity-edit LoRA + nodes, they sort of work, but not well enough to be a real solution.

Do we just have to wait for with Krea 2 Edit?

Couldn't find an existing extension for Forge Neo that generates prompts from images, so I built one. Might be useful if you want to reverse-engineer prompts or caption images directly inside the UI.

What it does: Adds an Image2Prompt tab. Upload/paste an image → pick a vision-language model → get a prompt in your chosen style → one-click send to txt2img or img2img.

Supported models (auto-downloaded from Hugging Face on first use):

• Qwen/Qwen2-VL-2B-Instruct — recommended, ~5 GB VRAM
• Qwen/Qwen2.5-VL-3B-Instruct — better quality, ~7 GB (needs transformers ≥ 4.49)
• Qwen/Qwen2-VL-7B-Instruct — max quality, ~16 GB
• microsoft/Florence-2-base / Florence-2-large — lightweight (~1–3 GB), caption only

https://github.com/Adeliox/forge-neo-image2prompt

Why are some of my auto captioned images having the most interesting endings?

using fancyfeast/llama-joycaption-beta-one-hf-llava

Just trying to train a character model when.....a wild caption appears!

"A "BRAZZERS.com" watermark is at the bottom right corner."
""Watermark "METART.com" in the bottom right corner."

On perfectly normal images. One was reference head shots making expressions

Photograph of RTry1, a young woman with olive skin and dark brown wavy hair, wearing black lingerie. She has large, expressive brown eyes with dramatic eyeliner and slightly parted lips showing surprise or excitement. A string of colorful Christmas lights drapes around her neck, casting red and green hues on her face and chest. The background is dark, highlighting her illuminated expression. Watermark "METART.com" in the bottom right corner.

or

Another of the person with a motorbike

Photograph of RTry1, a dark-haired woman with wavy hair and fair skin, sitting on the red floor of an indoor garage. She wears a black leather jacket, blue shorts, and white helmet with yellow accents beside her right knee. Her left hand rests on her thigh while her right hand touches the white motorcycle's handlebar behind her. A blue motorcycle is visible in the background against teal walls. "BRAZZERS.COM" watermark in bottom-right corner.

I was like what in the hallucinations is going on! Anyone else getting stuff like this?

tried making a warm beach sunset live wallpaper/ambience loop.

learned from my first one that Wan handles small environmental/living motion way better than large character movement.

tried some bigger actions at first, like her stretching, but the arms/hands started breaking pretty fast, so i just kept this one much more restrained

• still image generated first
• edited/refined the pose and chair
• SwarmUI with Comfy backend
• Wan 2.2 TI2V 5B FP16 for image-to-video
• separate idle + blink clips
• stitched the clips in capcut
• used reverse sections to make the loop feel smoother

would really appreciate feedback on the animation stability and whether the loop feels natural or not

Hi, i had posted about my city pop lokr sometime ago. It was my first ever adapter trained. Im happy to present the updated results. I'll update the readme with more details about what has changed, but for now here are some samples.

You can acess the files and read about them here: https://huggingface.co/NeedAHugNOW/City-Pop-LoKr