Since its introduction, Stable Diffusion has revolutionized the way we create images: anyone can use AI to create images from a textual description of what they want to represent. But while this works pretty well, you'll have found that knowing how to write that description is essential to getting satisfactory results. That's why we're going to take a look in this article at a simple but effective technique for better controlling image generation with Stable Diffusion: Prompt Weighting.

---

ℹ️ This article is a translation from the French article Prompt Weight : le poids des mots originally published on my Stable Diffusion Blog.

---

Prompt Weighting is a tool that allows you to give more or less importance to certain parts of the text you submit to Stable Diffusion. In other words, it's a way of guiding the AI's attention to the key elements you want to appear in the generated image.

It may sound trivial, but in reality, this feature opens the door to very powerful creative control - especially as it doesn't require you to install any other complex templates or plugins: it's just a matter of using a particular syntax in the prompt.

Let's explore in detail what Prompt Weighting is, how it works and, above all, how it can help you take your creativity to the next level.

Prompt Weighting

What is Prompt Weighting?

In practical terms, Prompt Weighting uses the principle of weighting to change the relative importance of concepts or words in your prompt by changing their Weight.

Increasing word weights

Let's imagine a simple (and simplistic) prompt like "Woman, Beach, Pizza". In this prompt, each of the words has the same importance as the others - they all have a Weight of 1. Put another way, each one weighs 1/3 of the prompt's total.

To increase the importance of a word, and influence the result accordingly, we need to increase its weight, so that it weighs more than the others in the prompt total.

Let's compare 4 image series, generated with the same seed but varying word weights:

As you can see, the images remain quite similar, but by changing the prompt weight to give more weight to a word, its influence on the image is more marked.

Look, for example, at how, when importance is given to "Beach", the hills and cliffs in the background disappear to give more room to the beach and the sea. Or how the pizza is a little larger when this word is given more weight.

Every word counts

The relative importance of each word also depends on the length of the prompt. In our simplified example, we only had 3 words - so each one is already very important in the prompt (1/3 of the total).

When there are more words in the prompt, the relative importance of each word decreases. And increasing the weight of any one of them can have a greater effect than in this first example.

Let's take a similar but longer and more detailed prompt for a new test: Photography of a Woman with dark hair and blue eyes eating a slice of pepperoni pizza on a white beach: at sunset, ocean, cinematic shot, natural light

The most noticeable difference is obtained when the weight of "beach" is increased, as this leads to wider shots leaving more room for the beach in the image. This is particularly noticeable in the first two images of the series.

As you can see, Prompt Weighting allows you to refine the description of your image to give more importance to certain words or expressions and thus modify the result.

How do I use it?

Most Stable Diffusion interfaces allow you to vary the weight of words directly in the prompt - the relative importance of each word being calculated before image generation.

Each interface has its own way of implementing this feature - but the way of using it is quite similar from one to another.

Prompt Weight in Automatic1111

Automatic1111 was one of the first interfaces to implement the use of Prompt Weight using a syntax based on () and [] - which inspired the syntax for most modern interfaces.

The simple way to change the weight of a word in Automatic is therefore to enclose the word in () to increase its weight - and enclose it in [] to decrease its weight. For example, "Photography of a woman with (blue) eyes" for a prompt that reinforces the importance of the color blue (blue). And "Photography of a woman with [blue] eyes" to decrease it.

For greater precision, use a slightly different syntax: the word or expression is enclosed in brackets, followed by ":" and the weight modifier you wish to add. For example, "Photography of a man wearing jeans and a (pink:1.5) shirt" will increase the weight of the color pink (pink). To decrease the importance of a word, use a weight less than 1.

In practice, (keyword) is equivalent to (keyword:1.1) and [keyword] is equivalent to (keyword:0.9).

Prompt Weight in Fooocus

Fooocus uses the same syntax as Automatic1111 and applies it in the same way. The aim is to make it easy to copy/paste image prompts created with Automatic1111 to obtain similar results with Fooocus.

The Fooocus interface also implements a shortcut for using this function: by selecting a word or expression, it is possible to change its weight by pressing Ctrl+⬆️ to increase it and Ctrl+⬇️ to decrease it.

Prompt Weight in ComfyUI

The syntax used in ComfyUI to change the weight of a word is also very similar to that of Automatic1111.

ComfyUI uses the () accompanied by the weight, such as (keyword:1.1) to adjust the weight of a word or expression in the prompt.

You can also select a word or expression and press Ctrl+⬆️ and Ctr+⬇️ to increase or decrease its weight. The extent to which you increase or decrease the weight of these shortcuts can be adjusted in the parameters.

Difference between Automatic1111 and ComfyUI

Even though the concept is the same and the syntax is practically the same, Automatic1111 and ComfyUI's prompt weighting is different.

Indeed, the way weights are applied is not the same in the two interfaces. ComfyUI processes and applies weights as specified, whereas Automatic1111 will normalize the weights so that they sum to 1.

In practice, this means that the same weight in ComfyUI will generally have a stronger effect than in Automatic1111 - so it's best to use lower values in ComfyUI than in Automatic1111.

Indeed, with the same prompt weights, A1111 generally has a very weak effect compared to ComfyUI. You therefore need to apply much higher weights in A1111 to achieve a similar effect. And higher weights often produce poorer quality, more "cartoony" results in ComfyUI.

Because of this difference, it's also very difficult to replicate the same results between the two interfaces, even with identical parameters, as the weighting of words in the prompt is not ultimately the same between the two interfaces.

Fooocus has chosen to apply the same normalization calculation as Automatic1111 to enable reuse of prompts from one to the other.

Prompt Weighting is therefore a powerful technique for fine-tuning and precisely controlling the generation of images by Stable Diffusion. By adjusting the weight of words and phrases in your prompts, you can subtly or radically influence the final result, opening up new creative possibilities.

By mastering this technique, you can refine your creations and achieve results closer to your artistic vision. Whether you're an occasional user or a seasoned artist, this technique gives you greater control over the AI creative process.

Feel free to experiment with different weights and combinations to discover the full potential of this feature. Prompt Weighting is an invaluable tool in your creative toolbox, enabling you to push the boundaries of what's possible with Stable Diffusion.