Base Models

The Krita AI Diffusion plugin works with various different model architectures, also known as base models. These models are refined, extended and supported by various other models (LoRA, ControlNet, IP-Adapter) which must match the base architecture. Extensions between different base models are not compatible!

You can install multiple base models with their extensions side by side. The plugin supports:

• Stable Diffusion 1.5 (SD1.5)
• Stable Diffusion XL (SDXL)
• Illustrious / NoobAI
• Flux 1
• Flux 2
• Z-Image

The following base models have limited support:

• Chroma
• Stable Diffusion 3.5 (SD3.5)
• Qwen Image

Stable Diffusion 1.5

Released in October 2022 this is the older of the two base models. It was trained on 512x512 images originally, although custom checkpoints also work well for resolutions up to 768. It is still suitable for much higher resolutions by generating a low resolution image first and then upscaling it - the Plugin will do this automatically.

Advantages of SD 1.5:

• Requires less GPU memory (VRAM)
• Requires less disk space
• More community content available (checkpoints, LoRA)
• Wider range of control modes available

Stable Diffusion XL

Released in July 2023 this is a newer base model designed for higher fidelity. It was trained on images with a total of 1024x1024 pixels (various aspect ratios).

Advantages of SD XL:

• Better quality at high resolutions
• Improved understanding of text prompts

Illustrious / NoobAI

A line of anime-focused models. They are based on the SDXL architecture but have been trained on a different dataset. This makes them largely incompatible with extensions and LoRA from “regular” SDXL models. Instead they are supported by their own set of extensions.

The base NoobAI XL model is very versatile within its domain, but not straight-forward to use. It is recommended to follow its tag-based prompt structure. There are also derivative models which are easier to use but typically more restricted to a particular style.

Tip

Enabling tag auto-completion in Interface settings helps with writing keywords that are recognized by the model.

Caution

For epsilon-prediction models the base model has to be set to “Illustrious” manually in Style settings ▸ Advanced checkpoint configuration. That’s because they cannot be distinguished from SDXL models.

The v-prediction models are also supported. They are detected automatically.

Flux 1

Flux is a modern diffusion model architecture with superior text understanding and image quality compared to Stable Diffusion. It was released by Black Forest Labs in August 2024. It comes in various configurations, with the most capable models not freely available. The open-weights models are less flexible and favour photographic content.

• Good at understanding natural language
• Excellent image composition and consistency
• Relatively large and slow (requires powerful GPU)

The plugin also supports the edit model Flux Kontext which allows making specific changes to images via text instructions.

Tip

Models marked [Nunchaku] are quantized versions which are smaller and much faster. They retain most of the quality, and are available for NVIDIA GPUs only.

Flux 2

Flux 2 is the successor to Flux with improved quality of image details. The instruction-based edit capabilities are integrated into the main model, and recognize more complex instructions.

The plugin supports the “Klein” variants of Flux 2, which are small optimized versions suitable for consumer-grade GPUs. They are faster and more lightweight than Flux 1 despite their superior capabilities. Sometimes they struggle with image continuity, eg. wrong anatomy or inconsistent objects.

Flux 2 Klein also comes as a larger “9B” variant which requires more VRAM but produces higher quality results. It is supported by the plugin as well, but must be downloaded manually.

Z-Image

Z-Image is a medium-sized diffusion model, somewhere between Flux 1 and Flux 2 4B in terms of memory requirements and speed. The “Turbo” variant has made a large impact by delivering extremely convincing realistic images with reasonable performance. It has good text understanding in Chinese and English.

There is also a “Base” variant of Z-Image which is not as heavily tuned towards photographic content. It can produce a much wider variety of styles, but images are less consistent and generation time is much slower. You can download it manually if you want to experiment.

Installation

When using the Plugin’s automatic installer you can select which base model (“workload”) you want to install. This will install a minimal set of models which the plugin needs to perform certain common tasks. Base models which don’t have a workload listed don’t need one.

Optional packages and extensions listed below are available for selection depending on whether the base workload is installed. Some packages like upscalers are unrelated to diffusion and work for both, others have separate packages available for each version. Not all extensions exist for all base models.

Styles

Styles indicate which base model version they use with an icon. This is derived from the selected checkpoint. Any LoRA you add must match the base model.

Note that the Plugin automatically filters styles for workloads you don’t have installed. In the Style configuration menu you will see a notification if there is an issue.