Overview — dgenerate 3.7.1 documentation (2024)

dgenerate is a command line tool and library for generating images and animation sequencesusing Stable Diffusion and related techniques / models. Now Featuring a Console UI andREPL shell mode for the dgenerate configuration / scripting language.

You can use dgenerate to generate multiple images or animated outputs using multiple combinations ofdiffusion input parameters in batch, so that the differences in generated output can be compared / curated easily.

Simple txt2img generation without image inputs is supported, as well as img2img and inpainting, and ControlNets.

Animated output can be produced by processing every frame of a Video, GIF, WebP, or APNG through various implementationsof diffusion in img2img or inpainting mode, as well as with ControlNets and control guidance images, in any combination thereof.MP4 (h264) video can be written without memory constraints related to frame count. GIF, WebP, and PNG/APNG can bewritten WITH memory constraints, IE: all frames exist in memory at once before being written.

Video input of any runtime can be processed without memory constraints related to the video size.Many video formats are supported through the use of PyAV (ffmpeg).

Animated image input such as GIF, APNG (extension must be .apng), and WebP, can also be processed WITHmemory constraints, IE: all frames exist in memory at once after an animated image is read.

PNG, JPEG, JPEG-2000, TGA (Targa), BMP, and PSD (Photoshop) are supported for static image inputs.

In addition to diffusion, dgenerate also supports the processing of any supported image, video, oranimated image using any of its built in image processors, which include various edge detectors,depth detectors, segment generation, normal map generation, pose detection, non-diffusion based AI upscaling,and more.

This software requires an Nvidia GPU supporting CUDA 12.1+, CPU rendering is possible forsome operations but extraordinarily slow.

For library documentation, and a better README reading experience whichincludes proper syntax highlighting for examples, and side panel navigation,please visit readthedocs.

  • How to install
    • Windows Install

    • Linux or WSL Install

  • Usage Examples
    • Basic Usage

    • Negative Prompt

    • Multiple Prompts

    • Image Seed

    • Inpainting

    • Per Image Seed Resizing

    • Animated Output

    • Animation Slicing

    • Inpainting Animations

    • Deterministic Output

    • Specifying a specific GPU for CUDA

    • Specifying a Scheduler (sampler)

    • Specifying a VAE

    • VAE Tiling and Slicing

    • Specifying a UNet

    • Specifying an SDXL Refiner

    • Specifying a Stable Cascade Decoder

    • Specifying LoRAs

    • Specifying Textual Inversions

    • Specifying Control Nets

    • Utilizing CivitAI links and Other Hosted Models

    • Specifying Generation Batch Size

    • Image Processors

    • Upscaling with Diffusion Upscaler Models

    • Sub Commands (image-process)

    • Upscaling with chaiNNer Compatible Upscaler Models

    • Writing and Running Configs
      • Basic config syntax

      • Built in template variables and functions

      • Directives, and applying templating

      • Setting template variables, in depth

      • Globbing and path manipulation

      • The \print and \echo directive

      • The \image_process directive

      • The \exec directive

      • The \download directive

      • The download() template function

      • The \exit directive

      • Running configs from the command line

      • Config argument injection

    • Console UI

    • Writing Plugins

    • File Cache Control

Help Output

usage: dgenerate [-h] [-v] [--version] [--file | --shell | --no-stdin | --console] [--plugin-modules PATH [PATH ...]] [--sub-command SUB_COMMAND] [--sub-command-help [SUB_COMMAND ...]] [-ofm] [--templates-help [VARIABLE_NAME ...]] [--directives-help [DIRECTIVE_NAME ...]] [--functions-help [FUNCTION_NAME ...]] [-mt MODEL_TYPE] [-rev BRANCH] [-var VARIANT] [-sbf SUBFOLDER] [-atk TOKEN] [-bs INTEGER] [-bgs SIZE] [-un UNET_URI] [-un2 UNET_URI] [-vae VAE_URI] [-vt] [-vs] [-lra LORA_URI [LORA_URI ...]] [-ti URI [URI ...]] [-cn CONTROL_NET_URI [CONTROL_NET_URI ...]] [-sch SCHEDULER_URI] [-mqo | -mco] [--s-cascade-decoder MODEL_URI] [-dqo] [-dco] [--s-cascade-decoder-prompts PROMPT [PROMPT ...]] [--s-cascade-decoder-inference-steps INTEGER [INTEGER ...]] [--s-cascade-decoder-guidance-scales INTEGER [INTEGER ...]] [--s-cascade-decoder-scheduler SCHEDULER_URI] [--sdxl-refiner MODEL_URI] [-rqo] [-rco] [--sdxl-refiner-scheduler SCHEDULER_URI] [--sdxl-refiner-edit] [--sdxl-second-prompts PROMPT [PROMPT ...]] [--sdxl-aesthetic-scores FLOAT [FLOAT ...]] [--sdxl-crops-coords-top-left COORD [COORD ...]] [--sdxl-original-size SIZE [SIZE ...]] [--sdxl-target-size SIZE [SIZE ...]] [--sdxl-negative-aesthetic-scores FLOAT [FLOAT ...]] [--sdxl-negative-original-sizes SIZE [SIZE ...]] [--sdxl-negative-target-sizes SIZE [SIZE ...]] [--sdxl-negative-crops-coords-top-left COORD [COORD ...]] [--sdxl-refiner-prompts PROMPT [PROMPT ...]] [--sdxl-refiner-clip-skips INTEGER [INTEGER ...]] [--sdxl-refiner-second-prompts PROMPT [PROMPT ...]] [--sdxl-refiner-aesthetic-scores FLOAT [FLOAT ...]] [--sdxl-refiner-crops-coords-top-left COORD [COORD ...]] [--sdxl-refiner-original-sizes SIZE [SIZE ...]] [--sdxl-refiner-target-sizes SIZE [SIZE ...]] [--sdxl-refiner-negative-aesthetic-scores FLOAT [FLOAT ...]] [--sdxl-refiner-negative-original-sizes SIZE [SIZE ...]] [--sdxl-refiner-negative-target-sizes SIZE [SIZE ...]] [--sdxl-refiner-negative-crops-coords-top-left COORD [COORD ...]] [-hnf FLOAT [FLOAT ...]] [-ri INT [INT ...]] [-rg FLOAT [FLOAT ...]] [-rgr FLOAT [FLOAT ...]] [-sc] [-d DEVICE] [-t DTYPE] [-s SIZE] [-na] [-o PATH] [-op PREFIX] [-ox] [-oc] [-om] [-p PROMPT [PROMPT ...]] [-cs INTEGER [INTEGER ...]] [-se SEED [SEED ...]] [-sei] [-gse COUNT] [-af FORMAT] [-if FORMAT] [-nf] [-fs FRAME_NUMBER] [-fe FRAME_NUMBER] [-is SEED [SEED ...]] [-sip PROCESSOR_URI [PROCESSOR_URI ...]] [-mip PROCESSOR_URI [PROCESSOR_URI ...]] [-cip PROCESSOR_URI [PROCESSOR_URI ...]] [--image-processor-help [PROCESSOR_NAME ...]] [-pp PROCESSOR_URI [PROCESSOR_URI ...]] [-iss FLOAT [FLOAT ...] | -uns INTEGER [INTEGER ...]] [-gs FLOAT [FLOAT ...]] [-igs FLOAT [FLOAT ...]] [-gr FLOAT [FLOAT ...]] [-ifs INTEGER [INTEGER ...]] [-mc EXPR [EXPR ...]] [-pmc EXPR [EXPR ...]] [-umc EXPR [EXPR ...]] [-vmc EXPR [EXPR ...]] [-cmc EXPR [EXPR ...]] model_pathBatch image generation and manipulation tool supporting Stable Diffusion and related techniques /algorithms, with support for video and animated image processing.positional arguments: model_path huggingface model repository slug, huggingface blob link to a model file, path to folder on disk, or path to a .pt, .pth, .bin, .ckpt, or .safetensors file.options: -h, --help show this help message and exit -v, --verbose Output information useful for debugging, such as pipeline call and model load parameters. --version Show dgenerate's version and exit --file Convenience argument for reading a configuration script from a file instead of using a pipe. This is a meta argument which can not be used within a configuration script and is only valid from the command line or during a popen invocation of dgenerate. --shell When reading configuration from STDIN (a pipe), read forever, even when configuration errors occur. This allows dgenerate to run in the background and be communicated with by another process sending it commands. Launching dgenerate with this option and not piping it input will attach it to the terminal like a shell. Entering configuration into this shell will require two newlines to submit a command due to parsing lookahead. IE: two presses of the enter key. This is a meta argument which can not be used within a configuration script and is only valid from the command line or during a popen invocation of dgenerate. --no-stdin Can be used to indicate to dgenerate that it will not receive any piped in input. This is useful for running dgenerate via popen from python or another application using normal arguments, where it would otherwise try to read from STDIN and block forever because it is not attached to a terminal. This is a meta argument which can not be used within a configuration script and is only valid from the command line or during a popen invocation of dgenerate. --console Launch a terminal-like tkinter GUI that communicates with an instance of dgenerate running in the background. This allows you to interactively write dgenerate config scripts as if dgenerate were a shell / REPL. This is a meta argument which can not be used within a configuration script and is only valid from the command line or during a popen invocation of dgenerate. --plugin-modules PATH [PATH ...] Specify one or more plugin module folder paths (folder containing __init__.py) or python .py file paths, or python module names to load as plugins. Plugin modules can currently implement image processors and config directives. --sub-command SUB_COMMAND Specify the name a sub-command to invoke. dgenerate exposes some extra image processing functionality through the use of sub-commands. Sub commands essentially replace the entire set of accepted arguments with those of a sub-command which implements additional functionality. See --sub-command-help for a list of sub-commands and help. --sub-command-help [SUB_COMMAND ...] List available sub-commands, providing sub-command names will produce their documentation. Calling a subcommand with "--sub-command name --help" will produce argument help output for that subcommand. -ofm, --offline-mode Whether dgenerate should try to download huggingface models that do not exist in the disk cache, or only use what is available in the cache. Referencing a model on huggingface that has not been cached because it was not previously downloaded will result in a failure when using this option. --templates-help [VARIABLE_NAME ...] Print a list of template variables available in the interpreter environment used for dgenerate config scripts, particularly the variables set after a dgenerate invocation occurs. When used as a command line option, their values are not presented, just their names and types. Specifying names will print type information for those variable names. --directives-help [DIRECTIVE_NAME ...] Print a list of directives available in the interpreter environment used for dgenerate config scripts. Providing names will print documentation for the specified directive names. When used with --plugin-modules, directives implemented by the specified plugins will also be listed. --functions-help [FUNCTION_NAME ...] Print a list of template functions available in the interpreter environment used for dgenerate config scripts. Providing names will print documentation for the specified function names. When used with --plugin- modules, functions implemented by the specified plugins will also be listed. -mt MODEL_TYPE, --model-type MODEL_TYPE Use when loading different model types. Currently supported: torch, torch- pix2pix, torch-sdxl, torch-sdxl-pix2pix, torch-upscaler-x2, torch- upscaler-x4, torch-if, torch-ifs, torch-ifs-img2img, or torch-s-cascade. (default: torch) -rev BRANCH, --revision BRANCH The model revision to use when loading from a huggingface repository, (The git branch / tag, default is "main") -var VARIANT, --variant VARIANT If specified when loading from a huggingface repository or folder, load weights from "variant" filename, e.g. "pytorch_model.<variant>.safetensors". Defaults to automatic selection. This option is ignored if using flax. -sbf SUBFOLDER, --subfolder SUBFOLDER Main model subfolder. If specified when loading from a huggingface repository or folder, load weights from the specified subfolder. -atk TOKEN, --auth-token TOKEN Huggingface auth token. Required to download restricted repositories that have access permissions granted to your huggingface account. -bs INTEGER, --batch-size INTEGER The number of image variations to produce per set of individual diffusion parameters in one rendering step simultaneously on a single GPU. When using flax, batch size is controlled by the environmental variable CUDA_VISIBLE_DEVICES which is a comma separated list of GPU device numbers (as listed by nvidia-smi). Usage of this argument with --model-type flax* will cause an error, diffusion with flax will generate an image on every GPU that is visible to CUDA and this is currently unchangeable. When generating animations with a --batch-size greater than one, a separate animation (with the filename suffix "animation_N") will be written to for each image in the batch. If --batch-grid-size is specified when producing an animation then the image grid is used for the output frames. During animation rendering each image in the batch will still be written to the output directory along side the produced animation as either suffixed files or image grids depending on the options you choose. (Torch Default:1) -bgs SIZE, --batch-grid-size SIZE Produce a single image containing a grid of images with the number of COLUMNSxROWS given to this argument when --batch-size is greater than 1, or when using flax with multiple GPUs visible (via the environmental variable CUDA_VISIBLE_DEVICES). If not specified with a --batch-size greater than 1, images will be written individually with an image number suffix (image_N) in the filename signifying which image in the batch they are. -un UNET_URI, --unet UNET_URI Specify a UNet using a URI. Examples: "huggingface/unet", "huggingface/unet;revision=main", "unet_folder_on_disk". Blob links / single file loads are not supported for UNets. The "revision" argument specifies the model revision to use for the UNet when loading from huggingface repository or blob link, (The git branch / tag, default is "main"). The "variant" argument specifies the UNet model variant, it is only supported for torch type models it is not supported for flax. If "variant" is specified when loading from a huggingface repository or folder, weights will be loaded from "variant" filename, e.g. "pytorch_model.<variant>.safetensors. "variant" defaults to the value of --variant if it is not specified in the URI. The "subfolder" argument specifies the UNet model subfolder, if specified when loading from a huggingface repository or folder, weights from the specified subfolder. The "dtype" argument specifies the UNet model precision, it defaults to the value of -t/--dtype and should be one of: auto, bfloat16, float16, or float32. If you wish to load weights directly from a path on disk, you must point this argument at the folder they exist in, which should also contain the config.json file for the UNet. For example, a downloaded repository folder from huggingface. -un2 UNET_URI, --unet2 UNET_URI Specify a second UNet, this is only valid when using SDXL or Stable Cascade model types. This UNet will be used for the SDXL refiner, or Stable Cascade decoder model. -vae VAE_URI, --vae VAE_URI Specify a VAE using a URI. When using torch models the URI syntax is: "AutoEncoderClass;model=(huggingface repository slug/blob link or file/folder path)". Examples: "AutoencoderKL;model=vae.pt", "AsymmetricAutoencoderKL;model=huggingface/vae", "AutoencoderTiny;model=huggingface/vae", "ConsistencyDecoderVAE;model=huggingface/vae". When using a Flax model, there is currently only one available encoder class: "FlaxAutoencoderKL;model=huggingface/vae". The AutoencoderKL encoder class accepts huggingface repository slugs/blob links, .pt, .pth, .bin, .ckpt, and .safetensors files. Other encoders can only accept huggingface repository slugs/blob links, or a path to a folder on disk with the model configuration and model file(s). If an AutoencoderKL VAE model file exists at a URL which serves the file as a raw download, you may provide an http/https link to it and it will be downloaded to dgenerates web cache. Aside from the "model" argument, there are four other optional arguments that can be specified, these include "revision", "variant", "subfolder", "dtype". They can be specified as so in any order, they are not positional: "AutoencoderKL;model=huggingface/vae;revision=main;variant=fp1 6;subfolder=sub_folder;dtype=float16". The "revision" argument specifies the model revision to use for the VAE when loading from huggingface repository or blob link, (The git branch / tag, default is "main"). The "variant" argument specifies the VAE model variant, it is only supported for torch type models it is not supported for flax. If "variant" is specified when loading from a huggingface repository or folder, weights will be loaded from "variant" filename, e.g. "pytorch_model.<variant>.safetensors. "variant" in the case of --vae does not default to the value of --variant to prevent failures during common use cases. The "subfolder" argument specifies the VAE model subfolder, if specified when loading from a huggingface repository or folder, weights from the specified subfolder. The "dtype" argument specifies the VAE model precision, it defaults to the value of -t/--dtype and should be one of: auto, bfloat16, float16, or float32. If you wish to load a weights file directly from disk, the simplest way is: --vae "AutoencoderKL;my_vae.safetensors", or with a dtype "AutoencoderKL;my_vae.safetensors;dtype=float16". All loading arguments except "dtype" are unused in this case and may produce an error message if used. If you wish to load a specific weight file from a huggingface repository, use the blob link loading syntax: --vae "AutoencoderKL;https://huggingface.co/UserName/repository- name/blob/main/vae_model.safetensors", the "revision" argument may be used with this syntax. -vt, --vae-tiling Enable VAE tiling (torch Stable Diffusion only). Assists in the generation of large images with lower memory overhead. The VAE will split the input tensor into tiles to compute decoding and encoding in several steps. This is useful for saving a large amount of memory and to allow processing larger images. Note that if you are using --control-nets you may still run into memory issues generating large images, or with --batch-size greater than 1. -vs, --vae-slicing Enable VAE slicing (torch Stable Diffusion models only). Assists in the generation of large images with lower memory overhead. The VAE will split the input tensor in slices to compute decoding in several steps. This is useful to save some memory, especially when --batch-size is greater than 1. Note that if you are using --control-nets you may still run into memory issues generating large images. -lra LORA_URI [LORA_URI ...], --loras LORA_URI [LORA_URI ...] Specify one or more LoRA models using URIs (flax not supported). These should be a huggingface repository slug, path to model file on disk (for example, a .pt, .pth, .bin, .ckpt, or .safetensors file), or model folder containing model files. If a LoRA model file exists at a URL which serves the file as a raw download, you may provide an http/https link to it and it will be downloaded to dgenerates web cache. huggingface blob links are not supported, see "subfolder" and "weight-name" below instead. Optional arguments can be provided after a LoRA model specification, these include: "scale", "revision", "subfolder", and "weight-name". They can be specified as so in any order, they are not positional: "huggingface/lora;scale=1.0;revision=main;subfolder=repo_subfolder;weight- name=lora.safetensors". The "scale" argument indicates the scale factor of the LoRA. The "revision" argument specifies the model revision to use for the LoRA when loading from huggingface repository, (The git branch / tag, default is "main"). The "subfolder" argument specifies the LoRA model subfolder, if specified when loading from a huggingface repository or folder, weights from the specified subfolder. The "weight-name" argument indicates the name of the weights file to be loaded when loading from a huggingface repository or folder on disk. If you wish to load a weights file directly from disk, the simplest way is: --loras "my_lora.safetensors", or with a scale "my_lora.safetensors;scale=1.0", all other loading arguments are unused in this case and may produce an error message if used. -ti URI [URI ...], --textual-inversions URI [URI ...] Specify one or more Textual Inversion models using URIs (flax and SDXL not supported). These should be a huggingface repository slug, path to model file on disk (for example, a .pt, .pth, .bin, .ckpt, or .safetensors file), or model folder containing model files. If a Textual Inversion model file exists at a URL which serves the file as a raw download, you may provide an http/https link to it and it will be downloaded to dgenerates web cache. huggingface blob links are not supported, see "subfolder" and "weight-name" below instead. Optional arguments can be provided after the Textual Inversion model specification, these include: "token", "revision", "subfolder", and "weight-name". They can be specified as so in any order, they are not positional: "huggingface/ti_model;revision=main;subfolder=repo_subfolder;weight- name=ti_model.safetensors". The "token" argument can be used to override the prompt token used for the textual inversion prompt embedding. For normal Stable Diffusion the default token value is provided by the model itself, but for Stable Diffusion XL the default token value is equal to the model file name with no extension and all spaces replaced by underscores. The "revision" argument specifies the model revision to use for the Textual Inversion model when loading from huggingface repository, (The git branch / tag, default is "main"). The "subfolder" argument specifies the Textual Inversion model subfolder, if specified when loading from a huggingface repository or folder, weights from the specified subfolder. The "weight-name" argument indicates the name of the weights file to be loaded when loading from a huggingface repository or folder on disk. If you wish to load a weights file directly from disk, the simplest way is: --textual-inversions "my_ti_model.safetensors", all other loading arguments are unused in this case and may produce an error message if used. -cn CONTROL_NET_URI [CONTROL_NET_URI ...], --control-nets CONTROL_NET_URI [CONTROL_NET_URI ...] Specify one or more ControlNet models using URIs. This should be a huggingface repository slug / blob link, path to model file on disk (for example, a .pt, .pth, .bin, .ckpt, or .safetensors file), or model folder containing model files. If a ControlNet model file exists at a URL which serves the file as a raw download, you may provide an http/https link to it and it will be downloaded to dgenerates web cache. Optional arguments can be provided after the ControlNet model specification, for torch these include: "scale", "start", "end", "revision", "variant", "subfolder", and "dtype". For flax: "scale", "revision", "subfolder", "dtype", "from_torch" (bool) They can be specified as so in any order, they are not positional: "huggingface/controlnet;scale=1.0;start=0.0;end=1.0;revision=main;variant= fp16;subfolder=repo_subfolder;dtype=float16". The "scale" argument specifies the scaling factor applied to the ControlNet model, the default value is 1.0. The "start" (only for --model-type "torch*") argument specifies at what fraction of the total inference steps to begin applying the ControlNet, defaults to 0.0, IE: the very beginning. The "end" (only for --model-type "torch*") argument specifies at what fraction of the total inference steps to stop applying the ControlNet, defaults to 1.0, IE: the very end. The "revision" argument specifies the model revision to use for the ControlNet model when loading from huggingface repository, (The git branch / tag, default is "main"). The "variant" (only for --model-type "torch*") argument specifies the ControlNet model variant, if "variant" is specified when loading from a huggingface repository or folder, weights will be loaded from "variant" filename, e.g. "pytorch_model.<variant>.safetensors. "variant" defaults to automatic selection and is ignored if using flax. "variant" in the case of --control-nets does not default to the value of --variant to prevent failures during common use cases. The "subfolder" argument specifies the ControlNet model subfolder, if specified when loading from a huggingface repository or folder, weights from the specified subfolder. The "dtype" argument specifies the ControlNet model precision, it defaults to the value of -t/--dtype and should be one of: auto, bfloat16, float16, or float32. The "from_torch" (only for --model-type flax) this argument specifies that the ControlNet is to be loaded and converted from a huggingface repository or file that is designed for pytorch. (Defaults to false) If you wish to load a weights file directly from disk, the simplest way is: --control-nets "my_controlnet.safetensors" or --control-nets "my_controlnet.safetensors;scale=1.0;dtype=float16", all other loading arguments aside from "scale" and "dtype" are unused in this case and may produce an error message if used ("from_torch" is available when using flax). If you wish to load a specific weight file from a huggingface repository, use the blob link loading syntax: --control-nets "https://huggingface.co/UserName/repository- name/blob/main/controlnet.safetensors", the "revision" argument may be used with this syntax. -sch SCHEDULER_URI, --scheduler SCHEDULER_URI Specify a scheduler (sampler) by URI. Passing "help" to this argument will print the compatible schedulers for a model without generating any images. Passing "helpargs" will yield a help message with a list of overridable arguments for each scheduler and their typical defaults. Arguments listed by "helpargs" can be overridden using the URI syntax typical to other dgenerate URI arguments. Torch schedulers: (DDIMScheduler, DDPMScheduler, PNDMScheduler, LMSDiscreteScheduler, EulerDiscreteScheduler, HeunDiscreteScheduler, EulerAncestralDiscreteScheduler, DPMSolverMultistepScheduler, DPMSolverSinglestepScheduler, KDPM2DiscreteScheduler, KDPM2AncestralDiscreteScheduler, DEISMultistepScheduler, UniPCMultistepScheduler, DPMSolverSDEScheduler, EDMEulerScheduler). -mqo, --model-sequential-offload Force sequential model offloading for the main pipeline, this may drastically reduce memory consumption and allow large models to run when they would otherwise not fit in your GPUs VRAM. Inference will be much slower. Mutually exclusive with --model-cpu-offload -mco, --model-cpu-offload Force model cpu offloading for the main pipeline, this may reduce memory consumption and allow large models to run when they would otherwise not fit in your GPUs VRAM. Inference will be slower. Mutually exclusive with --model-sequential-offload --s-cascade-decoder MODEL_URI Specify a Stable Cascade (torch-s-cascade) decoder model path using a URI. This should be a huggingface repository slug / blob link, path to model file on disk (for example, a .pt, .pth, .bin, .ckpt, or .safetensors file), or model folder containing model files. Optional arguments can be provided after the decoder model specification, these include: "revision", "variant", "subfolder", and "dtype". They can be specified as so in any order, they are not positional: "huggingface/decoder_model;revision=main;v ariant=fp16;subfolder=repo_subfolder;dtype=float16". The "revision" argument specifies the model revision to use for the Textual Inversion model when loading from huggingface repository, (The git branch / tag, default is "main"). The "variant" argument specifies the decoder model variant and defaults to the value of --variant. When "variant" is specified when loading from a huggingface repository or folder, weights will be loaded from "variant" filename, e.g. "pytorch_model.<variant>.safetensors. The "subfolder" argument specifies the decoder model subfolder, if specified when loading from a huggingface repository or folder, weights from the specified subfolder. The "dtype" argument specifies the Stable Cascade decoder model precision, it defaults to the value of -t/--dtype and should be one of: auto, bfloat16, float16, or float32. If you wish to load a weights file directly from disk, the simplest way is: --sdxl-refiner "my_decoder.safetensors" or --sdxl-refiner "my_decoder.safetensors;dtype=float16", all other loading arguments aside from "dtype" are unused in this case and may produce an error message if used. If you wish to load a specific weight file from a huggingface repository, use the blob link loading syntax: --s-cascade-decoder "https://huggingface.co/UserName/repository- name/blob/main/decoder.safetensors", the "revision" argument may be used with this syntax. -dqo, --s-cascade-decoder-sequential-offload Force sequential model offloading for the Stable Cascade decoder pipeline, this may drastically reduce memory consumption and allow large models to run when they would otherwise not fit in your GPUs VRAM. Inference will be much slower. Mutually exclusive with --s-cascade-decoder-cpu-offload -dco, --s-cascade-decoder-cpu-offload Force model cpu offloading for the Stable Cascade decoder pipeline, this may reduce memory consumption and allow large models to run when they would otherwise not fit in your GPUs VRAM. Inference will be slower. Mutually exclusive with --s-cascade-decoder-sequential-offload --s-cascade-decoder-prompts PROMPT [PROMPT ...] One or more prompts to try with the Stable Cascade decoder model, by default the decoder model gets the primary prompt, this argument overrides that with a prompt of your choosing. The negative prompt component can be specified with the same syntax as --prompts --s-cascade-decoder-inference-steps INTEGER [INTEGER ...] One or more inference steps values to try with the Stable Cascade decoder. (default: [10]) --s-cascade-decoder-guidance-scales INTEGER [INTEGER ...] One or more guidance scale values to try with the Stable Cascade decoder. (default: [0]) --s-cascade-decoder-scheduler SCHEDULER_URI Specify a scheduler (sampler) by URI for the Stable Cascade decoder pass. Operates the exact same way as --scheduler including the "help" option. Passing 'helpargs' will yield a help message with a list of overridable arguments for each scheduler and their typical defaults. Defaults to the value of --scheduler. --sdxl-refiner MODEL_URI Specify a Stable Diffusion XL (torch-sdxl) refiner model path using a URI. This should be a huggingface repository slug / blob link, path to model file on disk (for example, a .pt, .pth, .bin, .ckpt, or .safetensors file), or model folder containing model files. Optional arguments can be provided after the SDXL refiner model specification, these include: "revision", "variant", "subfolder", and "dtype". They can be specified as so in any order, they are not positional: "huggingface/refiner_model_xl;re vision=main;variant=fp16;subfolder=repo_subfolder;dtype=float16". The "revision" argument specifies the model revision to use for the Textual Inversion model when loading from huggingface repository, (The git branch / tag, default is "main"). The "variant" argument specifies the SDXL refiner model variant and defaults to the value of --variant. When "variant" is specified when loading from a huggingface repository or folder, weights will be loaded from "variant" filename, e.g. "pytorch_model.<variant>.safetensors. The "subfolder" argument specifies the SDXL refiner model subfolder, if specified when loading from a huggingface repository or folder, weights from the specified subfolder. The "dtype" argument specifies the SDXL refiner model precision, it defaults to the value of -t/--dtype and should be one of: auto, bfloat16, float16, or float32. If you wish to load a weights file directly from disk, the simplest way is: --sdxl-refiner "my_sdxl_refiner.safetensors" or --sdxl-refiner "my_sdxl_refiner.safetensors;dtype=float16", all other loading arguments aside from "dtype" are unused in this case and may produce an error message if used. If you wish to load a specific weight file from a huggingface repository, use the blob link loading syntax: --sdxl-refiner "https://huggingface.co/UserName/repository- name/blob/main/refiner_model.safetensors", the "revision" argument may be used with this syntax. -rqo, --sdxl-refiner-sequential-offload Force sequential model offloading for the SDXL refiner pipeline, this may drastically reduce memory consumption and allow large models to run when they would otherwise not fit in your GPUs VRAM. Inference will be much slower. Mutually exclusive with --refiner-cpu-offload -rco, --sdxl-refiner-cpu-offload Force model cpu offloading for the SDXL refiner pipeline, this may reduce memory consumption and allow large models to run when they would otherwise not fit in your GPUs VRAM. Inference will be slower. Mutually exclusive with --refiner-sequential-offload --sdxl-refiner-scheduler SCHEDULER_URI Specify a scheduler (sampler) by URI for the SDXL refiner pass. Operates the exact same way as --scheduler including the "help" option. Passing 'helpargs' will yield a help message with a list of overridable arguments for each scheduler and their typical defaults. Defaults to the value of --scheduler. --sdxl-refiner-edit Force the SDXL refiner to operate in edit mode instead of cooperative denoising mode as it would normally do for inpainting and ControlNet usage. The main model will preform the full amount of inference steps requested by --inference-steps. The output of the main model will be passed to the refiner model and processed with an image seed strength in img2img mode determined by (1.0 - high-noise-fraction) --sdxl-second-prompts PROMPT [PROMPT ...] One or more secondary prompts to try using SDXL's secondary text encoder. By default the model is passed the primary prompt for this value, this option allows you to choose a different prompt. The negative prompt component can be specified with the same syntax as --prompts --sdxl-aesthetic-scores FLOAT [FLOAT ...] One or more Stable Diffusion XL (torch-sdxl) "aesthetic-score" micro- conditioning parameters. Used to simulate an aesthetic score of the generated image by influencing the positive text condition. Part of SDXL's micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952]. --sdxl-crops-coords-top-left COORD [COORD ...] One or more Stable Diffusion XL (torch-sdxl) "negative-crops-coords-top- left" micro-conditioning parameters in the format "0,0". --sdxl-crops- coords-top-left can be used to generate an image that appears to be "cropped" from the position --sdxl-crops-coords-top-left downwards. Favorable, well-centered images are usually achieved by setting --sdxl- crops-coords-top-left to "0,0". Part of SDXL's micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952]. --sdxl-original-size SIZE [SIZE ...], --sdxl-original-sizes SIZE [SIZE ...] One or more Stable Diffusion XL (torch-sdxl) "original-size" micro- conditioning parameters in the format (WIDTH)x(HEIGHT). If not the same as --sdxl-target-size the image will appear to be down or up-sampled. --sdxl- original-size defaults to --output-size or the size of any input images if not specified. Part of SDXL's micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952] --sdxl-target-size SIZE [SIZE ...], --sdxl-target-sizes SIZE [SIZE ...] One or more Stable Diffusion XL (torch-sdxl) "target-size" micro- conditioning parameters in the format (WIDTH)x(HEIGHT). For most cases, --sdxl-target-size should be set to the desired height and width of the generated image. If not specified it will default to --output-size or the size of any input images. Part of SDXL's micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952] --sdxl-negative-aesthetic-scores FLOAT [FLOAT ...] One or more Stable Diffusion XL (torch-sdxl) "negative-aesthetic-score" micro-conditioning parameters. Part of SDXL's micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952]. Can be used to simulate an aesthetic score of the generated image by influencing the negative text condition. --sdxl-negative-original-sizes SIZE [SIZE ...] One or more Stable Diffusion XL (torch-sdxl) "negative-original-sizes" micro-conditioning parameters. Negatively condition the generation process based on a specific image resolution. Part of SDXL's micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952]. For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208 --sdxl-negative-target-sizes SIZE [SIZE ...] One or more Stable Diffusion XL (torch-sdxl) "negative-original-sizes" micro-conditioning parameters. To negatively condition the generation process based on a target image resolution. It should be as same as the " --sdxl-target-size" for most cases. Part of SDXL's micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952]. For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208. --sdxl-negative-crops-coords-top-left COORD [COORD ...] One or more Stable Diffusion XL (torch-sdxl) "negative-crops-coords-top- left" micro-conditioning parameters in the format "0,0". Negatively condition the generation process based on a specific crop coordinates. Part of SDXL's micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952]. For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208. --sdxl-refiner-prompts PROMPT [PROMPT ...] One or more prompts to try with the SDXL refiner model, by default the refiner model gets the primary prompt, this argument overrides that with a prompt of your choosing. The negative prompt component can be specified with the same syntax as --prompts --sdxl-refiner-clip-skips INTEGER [INTEGER ...] One or more clip skip override values to try for the SDXL refiner, which normally uses the clip skip value for the main model when it is defined by --clip-skips. --sdxl-refiner-second-prompts PROMPT [PROMPT ...] One or more prompts to try with the SDXL refiner models secondary text encoder, by default the refiner model gets the primary prompt passed to its second text encoder, this argument overrides that with a prompt of your choosing. The negative prompt component can be specified with the same syntax as --prompts --sdxl-refiner-aesthetic-scores FLOAT [FLOAT ...] See: --sdxl-aesthetic-scores, applied to SDXL refiner pass. --sdxl-refiner-crops-coords-top-left COORD [COORD ...] See: --sdxl-crops-coords-top-left, applied to SDXL refiner pass. --sdxl-refiner-original-sizes SIZE [SIZE ...] See: --sdxl-refiner-original-sizes, applied to SDXL refiner pass. --sdxl-refiner-target-sizes SIZE [SIZE ...] See: --sdxl-refiner-target-sizes, applied to SDXL refiner pass. --sdxl-refiner-negative-aesthetic-scores FLOAT [FLOAT ...] See: --sdxl-negative-aesthetic-scores, applied to SDXL refiner pass. --sdxl-refiner-negative-original-sizes SIZE [SIZE ...] See: --sdxl-negative-original-sizes, applied to SDXL refiner pass. --sdxl-refiner-negative-target-sizes SIZE [SIZE ...] See: --sdxl-negative-target-sizes, applied to SDXL refiner pass. --sdxl-refiner-negative-crops-coords-top-left COORD [COORD ...] See: --sdxl-negative-crops-coords-top-left, applied to SDXL refiner pass. -hnf FLOAT [FLOAT ...], --sdxl-high-noise-fractions FLOAT [FLOAT ...] One or more high-noise-fraction values for Stable Diffusion XL (torch- sdxl), this fraction of inference steps will be processed by the base model, while the rest will be processed by the refiner model. Multiple values to this argument will result in additional generation steps for each value. In certain situations when the mixture of denoisers algorithm is not supported, such as when using --control-nets and inpainting with SDXL, the inverse proportion of this value IE: (1.0 - high-noise-fraction) becomes the --image-seed-strengths input to the SDXL refiner. (default: [0.8]) -ri INT [INT ...], --sdxl-refiner-inference-steps INT [INT ...] One or more inference steps values for the SDXL refiner when in use. Override the number of inference steps used by the SDXL refiner, which defaults to the value taken from --inference-steps. -rg FLOAT [FLOAT ...], --sdxl-refiner-guidance-scales FLOAT [FLOAT ...] One or more guidance scale values for the SDXL refiner when in use. Override the guidance scale value used by the SDXL refiner, which defaults to the value taken from --guidance-scales. -rgr FLOAT [FLOAT ...], --sdxl-refiner-guidance-rescales FLOAT [FLOAT ...] One or more guidance rescale values for the SDXL refiner when in use. Override the guidance rescale value used by the SDXL refiner, which defaults to the value taken from --guidance-rescales. -sc, --safety-checker Enable safety checker loading, this is off by default. When turned on images with NSFW content detected may result in solid black output. Some pretrained models have no safety checker model present, in that case this option has no effect. -d DEVICE, --device DEVICE cuda / cpu. (default: cuda). Use: cuda:0, cuda:1, cuda:2, etc. to specify a specific GPU. This argument is ignored when using flax, for flax use the environmental variable CUDA_VISIBLE_DEVICES to specify which GPUs are visible to cuda, flax will use every visible GPU. -t DTYPE, --dtype DTYPE Model precision: auto, bfloat16, float16, or float32. (default: auto) -s SIZE, --output-size SIZE Image output size, for txt2img generation, this is the exact output size. The dimensions specified for this value must be aligned by 8 or you will receive an error message. If an --image-seeds URI is used its Seed, Mask, and/or Control component image sources will be resized to this dimension with aspect ratio maintained before being used for generation by default. Unless --no-aspect is specified, width will be fixed and a new height (aligned by 8) will be calculated for the input images. In most cases resizing the image inputs will result in an image output of an equal size to the inputs, except in the case of upscalers and Deep Floyd --model-type values (torch-if*). If only one integer value is provided, that is the value for both dimensions. X/Y dimension values should be separated by "x". This value defaults to 512x512 for Stable Diffusion when no --image- seeds are specified (IE txt2img mode), 1024x1024 for Stable Diffusion XL (SDXL) model types, and 64x64 for --model-type torch-if (Deep Floyd stage 1). Deep Floyd stage 1 images passed to superscaler models (--model-type torch-ifs*) that are specified with the 'floyd' keyword argument in an --image-seeds definition are never resized or processed in any way. -na, --no-aspect This option disables aspect correct resizing of images provided to --image-seeds globally. Seed, Mask, and Control guidance images will be resized to the closest dimension specified by --output-size that is aligned by 8 pixels with no consideration of the source aspect ratio. This can be overriden at the --image-seeds level with the image seed keyword argument 'aspect=true/false'. -o PATH, --output-path PATH Output path for generated images and files. This directory will be created if it does not exist. (default: ./output) -op PREFIX, --output-prefix PREFIX Name prefix for generated images and files. This prefix will be added to the beginning of every generated file, followed by an underscore. -ox, --output-overwrite Enable overwrites of files in the output directory that already exists. The default behavior is not to do this, and instead append a filename suffix: "_duplicate_(number)" when it is detected that the generated file name already exists. -oc, --output-configs Write a configuration text file for every output image or animation. The text file can be used reproduce that particular output image or animation by piping it to dgenerate STDIN or by using the --file option, for example "dgenerate < config.dgen" or "dgenerate --file config.dgen". These files will be written to --output-path and are affected by --output-prefix and --output-overwrite as well. The files will be named after their corresponding image or animation file. Configuration files produced for animation frame images will utilize --frame-start and --frame-end to specify the frame number. -om, --output-metadata Write the information produced by --output-configs to the PNG metadata of each image. Metadata will not be written to animated files (yet). The data is written to a PNG metadata property named DgenerateConfig and can be read using ImageMagick like so: "magick identify -format "%[Property:DgenerateConfig] generated_file.png". -p PROMPT [PROMPT ...], --prompts PROMPT [PROMPT ...] One or more prompts to try, an image group is generated for each prompt, prompt data is split by ; (semi-colon). The first value is the positive text influence, things you want to see. The Second value is negative influence IE. things you don't want to see. Example: --prompts "shrek flying a tesla over detroit; clouds, rain, missiles". (default: [(empty string)]) -cs INTEGER [INTEGER ...], --clip-skips INTEGER [INTEGER ...] One or more clip skip values to try. Clip skip is the number of layers to be skipped from CLIP while computing the prompt embeddings, it must be a value greater than or equal to zero. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings. This is only supported for --model-type values "torch" and "torch-sdxl", including with --control-nets. -se SEED [SEED ...], --seeds SEED [SEED ...] One or more seeds to try, define fixed seeds to achieve deterministic output. This argument may not be used when --gse/--gen-seeds is used. (default: [randint(0, 99999999999999)]) -sei, --seeds-to-images When this option is enabled, each provided --seeds value or value generated by --gen-seeds is used for the corresponding image input given by --image-seeds. If the amount of --seeds given is not identical to that of the amount of --image-seeds given, the seed is determined as: seed = seeds[image_seed_index % len(seeds)], IE: it wraps around. -gse COUNT, --gen-seeds COUNT Auto generate N random seeds to try. This argument may not be used when -se/--seeds is used. -af FORMAT, --animation-format FORMAT Output format when generating an animation from an input video / gif / webp etc. Value must be one of: mp4, png, apng, gif, or webp. You may also specify "frames" to indicate that only frames should be output and no coalesced animation file should be rendered. (default: mp4) -if FORMAT, --image-format FORMAT Output format when writing static images. Any selection other than "png" is not compatible with --output-metadata. Value must be one of: png, apng, blp, bmp, dib, bufr, pcx, dds, ps, eps, gif, grib, h5, hdf, jp2, j2k, jpc, jpf, jpx, j2c, icns, ico, im, jfif, jpe, jpg, jpeg, tif, tiff, mpo, msp, palm, pdf, pbm, pgm, ppm, pnm, pfm, bw, rgb, rgba, sgi, tga, icb, vda, vst, webp, wmf, emf, or xbm. (default: png) -nf, --no-frames Do not write frame images individually when rendering an animation, only write the animation file. This option is incompatible with --animation- format frames. -fs FRAME_NUMBER, --frame-start FRAME_NUMBER Starting frame slice point for animated files (zero-indexed), the specified frame will be included. (default: 0) -fe FRAME_NUMBER, --frame-end FRAME_NUMBER Ending frame slice point for animated files (zero-indexed), the specified frame will be included. -is SEED [SEED ...], --image-seeds SEED [SEED ...] One or more image seed URIs to process, these may consist of URLs or file paths. Videos / GIFs / WEBP files will result in frames being rendered as well as an animated output file being generated if more than one frame is available in the input file. Inpainting for static images can be achieved by specifying a black and white mask image in each image seed string using a semicolon as the separating character, like so: "my-seed-image.png;my- image-mask.png", white areas of the mask indicate where generated content is to be placed in your seed image. Output dimensions specific to the image seed can be specified by placing the dimension at the end of the string following a semicolon like so: "my-seed-image.png;512x512" or "my- seed-image.png;my-image-mask.png;512x512". When using --control-nets, a singular image specification is interpreted as the control guidance image, and you can specify multiple control image sources by separating them with commas in the case where multiple ControlNets are specified, IE: (--image- seeds "control-image1.png, control-image2.png") OR (--image-seeds "seed.png;control=control-image1.png, control-image2.png"). Using --control-nets with img2img or inpainting can be accomplished with the syntax: "my-seed-image.png;mask=my-image-mask.png;control=my-control- image.png;resize=512x512". The "mask" and "resize" arguments are optional when using --control-nets. Videos, GIFs, and WEBP are also supported as inputs when using --control-nets, even for the "control" argument. --image-seeds is capable of reading from multiple animated files at once or any combination of animated files and images, the animated file with the least amount of frames dictates how many frames are generated and static images are duplicated over the total amount of frames. The keyword argument "aspect" can be used to determine resizing behavior when the global argument --output-size or the local keyword argument "resize" is specified, it is a boolean argument indicating whether aspect ratio of the input image should be respected or ignored. The keyword argument "floyd" can be used to specify images from a previous deep floyd stage when using --model-type torch-ifs*. When keyword arguments are present, all applicable images such as "mask", "control", etc. must also be defined with keyword arguments instead of with the short syntax. -sip PROCESSOR_URI [PROCESSOR_URI ...], --seed-image-processors PROCESSOR_URI [PROCESSOR_URI ...] Specify one or more image processor actions to preform on the primary image specified by --image-seeds. For example: --seed-image-processors "flip" "mirror" "grayscale". To obtain more information about what image processors are available and how to use them, see: --image-processor-help. -mip PROCESSOR_URI [PROCESSOR_URI ...], --mask-image-processors PROCESSOR_URI [PROCESSOR_URI ...] Specify one or more image processor actions to preform on the inpaint mask image specified by --image-seeds. For example: --mask-image-processors "invert". To obtain more information about what image processors are available and how to use them, see: --image-processor-help. -cip PROCESSOR_URI [PROCESSOR_URI ...], --control-image-processors PROCESSOR_URI [PROCESSOR_URI ...] Specify one or more image processor actions to preform on the control image specified by --image-seeds, this option is meant to be used with --control-nets. Example: --control-image-processors "canny;lower=50;upper=100". The delimiter "+" can be used to specify a different processor group for each image when using multiple control images with --control-nets. For example if you have --image-seeds "img1.png, img2.png" or --image-seeds "...;control=img1.png, img2.png" specified and multiple ControlNet models specified with --control-nets, you can specify processors for those control images with the syntax: (--control-image-processors "processes-img1" + "processes-img2"), this syntax also supports chaining of processors, for example: (--control- image-processors "first-process-img1" "second-process-img1" + "process- img2"). The amount of specified processors must not exceed the amount of specified control images, or you will receive a syntax error message. Images which do not have a processor defined for them will not be processed, and the plus character can be used to indicate an image is not to be processed and instead skipped over when that image is a leading element, for example (--control-image-processors + "process-second") would indicate that the first control guidance image is not to be processed, only the second. To obtain more information about what image processors are available and how to use them, see: --image-processor-help. --image-processor-help [PROCESSOR_NAME ...] Use this option alone (or with --plugin-modules) and no model specification in order to list available image processor module names. Specifying one or more module names after this option will cause usage documentation for the specified modules to be printed. -pp PROCESSOR_URI [PROCESSOR_URI ...], --post-processors PROCESSOR_URI [PROCESSOR_URI ...] Specify one or more image processor actions to preform on generated output before it is saved. For example: --post-processors "upcaler;model=4x_ESRGAN.pth". To obtain more information about what processors are available and how to use them, see: --image-processor-help. -iss FLOAT [FLOAT ...], --image-seed-strengths FLOAT [FLOAT ...] One or more image strength values to try when using --image-seeds for img2img or inpaint mode. Closer to 0 means high usage of the seed image (less noise convolution), 1 effectively means no usage (high noise convolution). Low values will produce something closer or more relevant to the input image, high values will give the AI more creative freedom. (default: [0.8]) -uns INTEGER [INTEGER ...], --upscaler-noise-levels INTEGER [INTEGER ...] One or more upscaler noise level values to try when using the super resolution upscaler --model-type torch-upscaler-x4 or torch-ifs. Specifying this option for --model-type torch-upscaler-x2 will produce an error message. The higher this value the more noise is added to the image before upscaling (similar to --image-seed-strengths). (default: [20 for x4, 250 for torch-ifs/torch-ifs-img2img, 0 for torch-ifs inpainting mode]) -gs FLOAT [FLOAT ...], --guidance-scales FLOAT [FLOAT ...] One or more guidance scale values to try. Guidance scale effects how much your text prompt is considered. Low values draw more data from images unrelated to text prompt. (default: [5]) -igs FLOAT [FLOAT ...], --image-guidance-scales FLOAT [FLOAT ...] One or more image guidance scale values to try. This can push the generated image towards the initial image when using --model-type *-pix2pix models, it is unsupported for other model types. Use in conjunction with --image-seeds, inpainting (masks) and --control-nets are not supported. Image guidance scale is enabled by setting image-guidance- scale > 1. Higher image guidance scale encourages generated images that are closely linked to the source image, usually at the expense of lower image quality. Requires a value of at least 1. (default: [1.5]) -gr FLOAT [FLOAT ...], --guidance-rescales FLOAT [FLOAT ...] One or more guidance rescale factors to try. Proposed by [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://arxiv.org/pdf/2305.08891.pdf) "guidance_scale" is defined as "φ" in equation 16. of [Common Diffusion Noise Schedules and Sample Steps are Flawed] (https://arxiv.org/pdf/2305.08891.pdf). Guidance rescale factor should fix overexposure when using zero terminal SNR. This is supported for basic text to image generation when using --model-type "torch" but not inpainting, img2img, or --control-nets. When using --model-type "torch-sdxl" it is supported for basic generation, inpainting, and img2img, unless --control-nets is specified in which case only inpainting is supported. It is supported for --model-type "torch- sdxl-pix2pix" but not --model-type "torch-pix2pix". (default: [0.0]) -ifs INTEGER [INTEGER ...], --inference-steps INTEGER [INTEGER ...] One or more inference steps values to try. The amount of inference (de- noising) steps effects image clarity to a degree, higher values bring the image closer to what the AI is targeting for the content of the image. Values between 30-40 produce good results, higher values may improve image quality and or change image content. (default: [30]) -mc EXPR [EXPR ...], --cache-memory-constraints EXPR [EXPR ...] Cache constraint expressions describing when to clear all model caches automatically (DiffusionPipeline, VAE, and ControlNet) considering current memory usage. If any of these constraint expressions are met all models cached in memory will be cleared. Example, and default value: "used_percent > 70" For Syntax See: [https://dgenerate.readthedocs.io/en/v 3.7.1/dgenerate_submodules.html#dgenerate.pipelinewrapper.CACHE_MEMORY_CON STRAINTS] -pmc EXPR [EXPR ...], --pipeline-cache-memory-constraints EXPR [EXPR ...] Cache constraint expressions describing when to automatically clear the in memory DiffusionPipeline cache considering current memory usage, and estimated memory usage of new models that are about to enter memory. If any of these constraint expressions are met all DiffusionPipeline objects cached in memory will be cleared. Example, and default value: "pipeline_size > (available * 0.75)" For Syntax See: [https://dgenerate.re adthedocs.io/en/v3.7.1/dgenerate_submodules.html#dgenerate.pipelinewrapper .PIPELINE_CACHE_MEMORY_CONSTRAINTS] -umc EXPR [EXPR ...], --unet-cache-memory-constraints EXPR [EXPR ...] Cache constraint expressions describing when to automatically clear the in memory UNet cache considering current memory usage, and estimated memory usage of new UNet models that are about to enter memory. If any of these constraint expressions are met all UNet models cached in memory will be cleared. Example, and default value: "unet_size > (available * 0.75)" For Syntax See: [https://dgenerate.readthedocs.io/en/v3.7.1/dgenerate_submodul es.html#dgenerate.pipelinewrapper.UNET_CACHE_MEMORY_CONSTRAINTS] -vmc EXPR [EXPR ...], --vae-cache-memory-constraints EXPR [EXPR ...] Cache constraint expressions describing when to automatically clear the in memory VAE cache considering current memory usage, and estimated memory usage of new VAE models that are about to enter memory. If any of these constraint expressions are met all VAE models cached in memory will be cleared. Example, and default value: "vae_size > (available * 0.75)" For Syntax See: [https://dgenerate.readthedocs.io/en/v3.7.1/dgenerate_submodul es.html#dgenerate.pipelinewrapper.VAE_CACHE_MEMORY_CONSTRAINTS] -cmc EXPR [EXPR ...], --control-net-cache-memory-constraints EXPR [EXPR ...] Cache constraint expressions describing when to automatically clear the in memory ControlNet cache considering current memory usage, and estimated memory usage of new ControlNet models that are about to enter memory. If any of these constraint expressions are met all ControlNet models cached in memory will be cleared. Example, and default value: "control_net_size > (available * 0.75)" For Syntax See: [https://dgenerate.readthedocs.io/en/v 3.7.1/dgenerate_submodules.html#dgenerate.pipelinewrapper.CONTROL_NET_CACH E_MEMORY_CONSTRAINTS]

You can install using the Windows installer provided with each release on theReleases Page, or you can manuallyinstall with pipx, (or pip if you want) as described below.

Manual Install

Install Visual Studios (Community or other), make sure “Desktop development with C++” is selected, unselect anything you do not need.

https://visualstudio.microsoft.com/downloads/

Install rust compiler using rustup-init.exe (x64), use the default install options.

https://www.rust-lang.org/tools/install

Install Python:

https://www.python.org/ftp/python/3.12.3/python-3.12.3-amd64.exe

Make sure you select the option “Add to PATH” in the python installer,otherwise invoke python directly using it’s full path while installing the tool.

Install GIT for Windows:

https://gitforwindows.org/

Install dgenerate

Using Windows CMD

Install pipx:

pip install pipxpipx ensurepath# Log out and log back in so PATH takes effect

Install dgenerate:

pipx install dgenerate ^--pip-args "--extra-index-url https://download.pytorch.org/whl/cu121/"# If you want a specific versionpipx install dgenerate==3.7.1 ^--pip-args "--extra-index-url https://download.pytorch.org/whl/cu121/"# You can install without pipx into your own environment like sopip install dgenerate==3.7.1 --extra-index-url https://download.pytorch.org/whl/cu121/

It is recommended to install dgenerate with pipx if you are just intendingto use it as a command line program, if you want to develop you can install it froma cloned repository like this:

# in the top of the repo make# an environment and activate itpython -m venv venvvenv\Scripts\activate# Install with pip into the environmentpip install --editable .[dev] --extra-index-url https://download.pytorch.org/whl/cu121/

Run dgenerate to generate images:

# Images are output to the "output" folder# in the current working directory by defaultdgenerate --helpdgenerate stabilityai/stable-diffusion-2-1 ^--prompts "an astronaut riding a horse" ^--output-path output ^--inference-steps 40 ^--guidance-scales 10

First update your system and install build-essential

sudo apt update && sudo apt upgradesudo apt install build-essential

Install CUDA Toolkit 12.*: https://developer.nvidia.com/cuda-downloads

I recommend using the runfile option:

# CUDA Toolkit 12.2.1 For Ubuntu / Debian / WSLwget https://developer.download.nvidia.com/compute/cuda/12.2.1/local_installers/cuda_12.2.1_535.86.10_linux.runsudo sh cuda_12.2.1_535.86.10_linux.run

Do not attempt to install a driver from the prompts if using WSL.

# On linux, if you intend to use flax, you may or may not need to create a symlink for libnvrtc# flax will look for libnvrtc.so, and may not be able to find it.ln -s /usr/local/cuda/lib64/libnvrtc.so.12 /usr/local/cuda/lib64/libnvrtc.so

Add libraries to linker path:

# Add to ~/.bashrc# For Linux add the followingexport LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH# For WSL add the followingexport LD_LIBRARY_PATH=/usr/lib/wsl/lib:/usr/local/cuda/lib64:$LD_LIBRARY_PATH# Add this in both cases as wellexport PATH=/usr/local/cuda/bin:$PATH

When done editing ~/.bashrc do:

source ~/.bashrc

Install Python 3.10+ (Debian / Ubuntu) and pipx

sudo apt install python3.10 python3-pip pipx python3.10-venv python3-wheelpipx ensurepathsource ~/.bashrc

Install dgenerate

pipx install dgenerate \--pip-args "--extra-index-url https://download.pytorch.org/whl/cu121/"# With flax/jax supportpipx install dgenerate[flax] \--pip-args "--extra-index-url https://download.pytorch.org/whl/cu121/ \-f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html"# If you want a specific versionpipx install dgenerate==3.7.1 \--pip-args "--extra-index-url https://download.pytorch.org/whl/cu121/"# Specific version with flax/jax supportpipx install dgenerate[flax]==3.7.1 \--pip-args "--extra-index-url https://download.pytorch.org/whl/cu121/ \-f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html"# You can install without pipx into your own environment like sopip3 install dgenerate==3.7.1 --extra-index-url https://download.pytorch.org/whl/cu121/# Or with flaxpip3 install dgenerate[flax]==3.7.1 --extra-index-url https://download.pytorch.org/whl/cu121/ \-f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html

It is recommended to install dgenerate with pipx if you are just intendingto use it as a command line program, if you want to develop you can install it froma cloned repository like this:

# in the top of the repo make# an environment and activate itpython3 -m venv venvsource venv/bin/activate# Install with pip into the environmentpip3 install --editable .[dev] --extra-index-url https://download.pytorch.org/whl/cu121/# With flax if you wantpip3 install --editable .[dev,flax] --extra-index-url https://download.pytorch.org/whl/cu121/ \-f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html

Run dgenerate to generate images:

# Images are output to the "output" folder# in the current working directory by defaultdgenerate --helpdgenerate stabilityai/stable-diffusion-2-1 \--prompts "an astronaut riding a horse" \--output-path output \--inference-steps 40 \--guidance-scales 10

The example below attempts to generate an astronaut riding a horse using 5 differentrandom seeds, 3 different inference steps values, and 3 different guidance scale values.

It utilizes the stabilityai/stable-diffusion-2-1 model repo on Hugging Face.

45 uniquely named images will be generated (5 x 3 x 3)

Also Adjust output size to 512x512 and output generated images to the astronaut folder in the current working directory.

When --output-path is not specified, the default output location is the output folderin the current working directory, if the path that is specified does not exist then it will be created.

dgenerate stabilityai/stable-diffusion-2-1 \--prompts "an astronaut riding a horse" \--gen-seeds 5 \--output-path astronaut \--inference-steps 30 40 50 \--guidance-scales 5 7 10 \--output-size 512x512

Loading models from huggingface blob links is also supported:

dgenerate https://huggingface.co/stabilityai/stable-diffusion-2-1/blob/main/v2-1_768-ema-pruned.safetensors \--prompts "an astronaut riding a horse" \--gen-seeds 5 \--output-path astronaut \--inference-steps 30 40 50 \--guidance-scales 5 7 10 \--output-size 512x512

SDXL is supported and can be used to generate highly realistic images.

Prompt only generation, img2img, and inpainting is supported for SDXL.

Refiner models can be specified, fp16 model variant and a datatype of float16 isrecommended to prevent out of memory conditions on the average GPU :)

dgenerate stabilityai/stable-diffusion-xl-base-1.0 --model-type torch-sdxl \--sdxl-high-noise-fractions 0.6 0.7 0.8 \--gen-seeds 5 \--inference-steps 50 \--guidance-scales 12 \--sdxl-refiner stabilityai/stable-diffusion-xl-refiner-1.0 \--prompts "real photo of an astronaut riding a horse on the moon" \--variant fp16 --dtype float16 \--output-size 1024

In order to specify a negative prompt, each prompt argument is splitinto two parts separated by ;

The prompt text occurring after ; is the negative influence prompt.

To attempt to avoid rendering of a saddle on the horse being ridden, youcould for example add the negative prompt saddle or wearing a saddleor horse wearing a saddle etc.

dgenerate stabilityai/stable-diffusion-2-1 \--prompts "an astronaut riding a horse; horse wearing a saddle" \--gen-seeds 5 \--output-path astronaut \--inference-steps 50 \--guidance-scales 10 \--output-size 512x512

Multiple prompts can be specified one after another in quotes in orderto generate images using multiple prompt variations.

The following command generates 10 uniquely named images using twoprompts and five random seeds (2x5)

5 of them will be from the first prompt and 5 of them from the second prompt.

All using 50 inference steps, and 10 for guidance scale value.

dgenerate stabilityai/stable-diffusion-2-1 \--prompts "an astronaut riding a horse" "an astronaut riding a donkey" \--gen-seeds 5 \--output-path astronaut \--inference-steps 50 \--guidance-scales 10 \--output-size 512x512

The --image-seeds argument can be used to specify one or more image input resource groupsfor use in rendering, and allows for the specification of img2img source images, inpaint masks,control net guidance images, deep floyd stage images, image group resizing, and frame slicing valuesfor animations. It possesses it’s own URI syntax for defining different image inputs used for image generation,the example described below is the simplest case for one image input (img2img).

This example uses a photo of Buzz Aldrin on the moon to generate a photo of an astronaut standing on marsusing img2img, this uses an image seed downloaded from wikipedia.

Disk file paths may also be used for image seeds and generally that is the standard use case,multiple image seed definitions may be provided and images will be generated from each imageseed individually.

# Generate this image using 5 different seeds, 3 different inference-step values, 3 different# guidance-scale values as above.# In addition this image will be generated using 3 different image seed strengths.# Adjust output size to 512x512 and output generated images to 'astronaut' folder, the image seed# will be resized to that dimension with aspect ratio respected by default, the width is fixed and# the height will be calculated, this behavior can be changed globally with the --no-aspect option# if desired or locally by specifying "img2img-seed.png;aspect=false" as your image seed# If you do not adjust the output size of the generated image, the size of the input image seed will be used.# 135 uniquely named images will be generated (5x3x3x3)dgenerate stabilityai/stable-diffusion-2-1 \--prompts "an astronaut walking on mars" \--image-seeds https://upload.wikimedia.org/wikipedia/commons/9/98/Aldrin_Apollo_11_original.jpg \--image-seed-strengths 0.2 0.5 0.8 \--gen-seeds 5 \--output-path astronaut \--inference-steps 30 40 50 \--guidance-scales 5 7 10 \--output-size 512x512

--image-seeds serves as the entire mechanism for determining if img2img or inpainting is going to occur viait’s URI syntax described further in the section Inpainting.

In addition to this it can be used to provide control guidance images in the case of txt2img, img2img, or inpaintingvia the use of a URI syntax involving keyword arguments.

The syntax --image-seeds "my-image-seed.png;control=my-control-image.png" can be used with --control-nets to specifyimg2img mode with a ControlNet for example, see: Specifying Control Nets for more information.

Inpainting on an image can be preformed by providing a mask image with your image seed. This mask should be a black and white imageof identical size to your image seed. White areas of the mask image will be used to tell the AI what areas of the seed image should be filledin with generated content.

For using inpainting on animated image seeds, jump to: Inpainting Animations

Some possible definitions for inpainting are:

  • --image-seeds "my-image-seed.png;my-mask-image.png"

  • --image-seeds "my-image-seed.png;mask=my-mask-image.png"

The format is your image seed and mask image separated by ;, optionally mask can be named argument.The alternate syntax is for disambiguation when preforming img2img or inpainting operations while Specifying Control Netsor other operations where keyword arguments might be necessary for disambiguation such as per image seed Animation Slicing,and the specification of the image from a previous Deep Floyd stage using the floyd argument.

Mask images can be downloaded from URL’s just like any other resource mentioned in an --image-seeds definition,however for this example files on disk are used for brevity.

You can download them here:

The command below generates a cat sitting on a bench with the images from the links above, the mask image masks outareas over the dog in the original image, causing the dog to be replaced with an AI generated cat.

dgenerate stabilityai/stable-diffusion-2-inpainting \--image-seeds "my-image-seed.png;my-mask-image.png" \--prompts "Face of a yellow cat, high resolution, sitting on a park bench" \--image-seed-strengths 0.8 \--guidance-scales 10 \--inference-steps 100

If you want to specify multiple image seeds that will have different output sizes irrespectiveof their input size or a globally defined output size defined with --output-size,You can specify their output size individually at the end of each provided image seed.

This will work when using a mask image for inpainting as well, including when using animated inputs.

This also works when Specifying Control Nets and guidance images for control nets.

Here are some possible definitions:

  • --image-seeds "my-image-seed.png;512x512" (img2img)

  • --image-seeds "my-image-seed.png;my-mask-image.png;512x512" (inpainting)

  • --image-seeds "my-image-seed.png;resize=512x512" (img2img)

  • --image-seeds "my-image-seed.png;mask=my-mask-image.png;resize=512x512" (inpainting)

The alternate syntax with named arguments is for disambiguation when Specifying Control Nets, orpreforming per image seed Animation Slicing, or specifying the previous Deep Floyd stage outputwith the floyd keyword argument.

When one dimension is specified, that dimension is the width, and the height.

The height of an image is calculated to be aspect correct by default for all resizingmethods unless --no-aspect has been given as an argument on the command line or theaspect keyword argument is used in the --image-seeds definition.

The the aspect correct resize behavior can be controlled on a per image seed definition basisusing the aspect keyword argument. Any value given to this argument overrides the presenceor absense of the --no-aspect command line argument.

the aspect keyword argument can only be used when all other components of the image seeddefinition are defined using keyword arguments. aspect=false disables aspect correct resizing,and aspect=true enables it.

Some possible definitions:

  • --image-seeds "my-image-seed.png;resize=512x512;aspect=false" (img2img)

  • --image-seeds "my-image-seed.png;mask=my-mask-image.png;resize=512x512;aspect=false" (inpainting)

The following example preforms img2img generation, followed by inpainting generation using 2 image seed definitions.The involved images are resized using the basic syntax with no keyword arguments present in the image seeds.

dgenerate stabilityai/stable-diffusion-2-1 \--image-seeds "my-image-seed.png;1024" "my-image-seed.png;my-mask-image.png;512x512" \--prompts "Face of a yellow cat, high resolution, sitting on a park bench" \--image-seed-strengths 0.8 \--guidance-scales 10 \--inference-steps 100

dgenerate supports many video formats through the use of PyAV (ffmpeg), as well as GIF & WebP.

See --help for information about all formats supported for the --animation-format option.

When an animated image seed is given, animated output will be produced in the format of your choosing.

In addition, every frame will be written to the output folder as a uniquely named image.

By specifying --animation-format frames you can tell dgenerate that you just needthe frame images and not to produce any coalesced animation file for you. You may alsospecify --no-frames to indicate that you only want an animation file to be producedand no intermediate frames, though using this option with --animation-format framesis considered an error.

If the animation is not 1:1 aspect ratio, the width will be fixed to the width of therequested output size, and the height calculated to match the aspect ratio of the animation.Unless --no-aspect or the --image-seeds keyword argument aspect=false are specified,in which case the video will be resized to the requested dimension exactly.

If you do not set an output size, the size of the input animation will be used.

# Use a GIF of a man riding a horse to create an animation of an astronaut riding a horse.dgenerate stabilityai/stable-diffusion-2-1 \--prompts "an astronaut riding a horse" \--image-seeds https://upload.wikimedia.org/wikipedia/commons/7/7b/Muybridge_race_horse_~_big_transp.gif \--image-seed-strengths 0.5 \--output-path astronaut \--inference-steps 50 \--guidance-scales 10 \--output-size 512x512 \--animation-format mp4

The above syntax is the same syntax used for generating an animation with a controlimage when --control-nets is used.

Animations can also be generated using an alternate syntax for --image-seedsthat allows the specification of a control image source when it is desired to use--control-nets with img2img or inpainting.

For more information about this see: Specifying Control Nets

As well as the information about --image-seeds from dgenerates --helpoutput.

Animated inputs can be sliced by a frame range either globally using--frame-start and --frame-end or locally using the named argumentsyntax for --image-seeds, for example:

  • --image-seeds "animated.gif;frame-start=3;frame-end=10".

When using animation slicing at the --image-seed level, all image input definitionsother than the main image must be specified using keyword arguments.

For example here are some possible definitions:

  • --image-seeds "seed.gif;frame-start=3;frame-end=10"

  • --image-seeds "seed.gif;mask=mask.gif;frame-start=3;frame-end=10

  • --image-seeds "seed.gif;control=control-guidance.gif;frame-start=3;frame-end=10

  • --image-seeds "seed.gif;mask=mask.gif;control=control-guidance.gif;frame-start=3;frame-end=10

  • --image-seeds "seed.gif;floyd=stage1.gif;frame-start=3;frame-end=10"

  • --image-seeds "seed.gif;mask=mask.gif;floyd=stage1.gif;frame-start=3;frame-end=10"

Specifying a frame slice locally in an image seed overrides the global frameslice setting defined by --frame-start or --frame-end, and is specific onlyto that image seed, other image seed definitions will not be affected.

Perhaps you only want to run diffusion on the first frame of an animated input inorder to save time in finding good parameters for generating every frame. You couldslice to only the first frame using --frame-start 0 --frame-end 0, which will be muchfaster than rendering the entire video/gif outright.

The slice range zero indexed and also inclusive, inclusive means that the starting and ending framesspecified by --frame-start and --frame-end will be included in the slice. Both slice pointsdo not have to be specified at the same time. You can exclude the tail end of a video withjust --frame-end alone, or seek to a certain start frame in the video with --frame-start aloneand render from there onward, this applies for keyword arguments in the --image-seeds definition as well.

If your slice only results in the processing of a single frame, an animated file format willnot be generated, only a single image output will be generated for that image seed during thegeneration step.

# Generate using only the first framedgenerate stabilityai/stable-diffusion-2-1 \--prompts "an astronaut riding a horse" \--image-seeds https://upload.wikimedia.org/wikipedia/commons/7/7b/Muybridge_race_horse_~_big_transp.gif \--image-seed-strengths 0.5 \--output-path astronaut \--inference-steps 50 \--guidance-scales 10 \--output-size 512x512 \--animation-format mp4 \--frame-start 0 \--frame-end 0

Image seeds can be supplied an animated or static image mask to define the areas for inpainting while generating an animated output.

Any possible combination of image/video parameters can be used. The animation with least amount of frames in the entirespecification determines the frame count, and any static images present are duplicated across the entire animation.The first animation present in an image seed specification always determines the output FPS of the animation.

When an animated seed is used with an animated mask, the mask for every corresponding frame in the input is taken from the animated mask,the runtime of the animated output will be equal to the shorter of the two animated inputs. IE: If the seed animation and the mask animationhave different length, the animated output is clipped to the length of the shorter of the two.

When a static image is used as a mask, that image is used as an inpaint mask for every frame of the animated seed.

When an animated mask is used with a static image seed, the animated output length is that of the animated mask. A video iscreated by duplicating the image seed for every frame of the animated mask, the animated output being generated by maskingthem together.

# A video with a static inpaint mask over the entire videodgenerate stabilityai/stable-diffusion-2-inpainting \--prompts "an astronaut riding a horse" \--image-seeds "my-animation.mp4;my-static-mask.png" \--output-path inpaint \--animation-format mp4# Zip two videos together, masking the left video with corresponding frames# from the right video. The two animated inputs do not have to be the same file format# you can mask videos with gif/webp and vice versadgenerate stabilityai/stable-diffusion-2-inpainting \--prompts "an astronaut riding a horse" \--image-seeds "my-animation.mp4;my-animation-mask.mp4" \--output-path inpaint \--animation-format mp4dgenerate stabilityai/stable-diffusion-2-inpainting \--prompts "an astronaut riding a horse" \--image-seeds "my-animation.mp4;my-animation-mask.gif" \--output-path inpaint \--animation-format mp4dgenerate stabilityai/stable-diffusion-2-inpainting \--prompts "an astronaut riding a horse" \--image-seeds "my-animation.gif;my-animation-mask.gif" \--output-path inpaint \--animation-format mp4dgenerate stabilityai/stable-diffusion-2-inpainting \--prompts "an astronaut riding a horse" \--image-seeds "my-animation.gif;my-animation-mask.webp" \--output-path inpaint \--animation-format mp4dgenerate stabilityai/stable-diffusion-2-inpainting \--prompts "an astronaut riding a horse" \--image-seeds "my-animation.webp;my-animation-mask.gif" \--output-path inpaint \--animation-format mp4dgenerate stabilityai/stable-diffusion-2-inpainting \--prompts "an astronaut riding a horse" \--image-seeds "my-animation.gif;my-animation-mask.mp4" \--output-path inpaint \--animation-format mp4# etc...# Use a static image seed and mask it with every frame from an# Animated mask filedgenerate stabilityai/stable-diffusion-2-inpainting \--prompts "an astronaut riding a horse" \--image-seeds "my-static-image-seed.png;my-animation-mask.mp4" \--output-path inpaint \--animation-format mp4dgenerate stabilityai/stable-diffusion-2-inpainting \--prompts "an astronaut riding a horse" \--image-seeds "my-static-image-seed.png;my-animation-mask.gif" \--output-path inpaint \--animation-format mp4dgenerate stabilityai/stable-diffusion-2-inpainting \--prompts "an astronaut riding a horse" \--image-seeds "my-static-image-seed.png;my-animation-mask.webp" \--output-path inpaint \--animation-format mp4# etc...

If you generate an image you like using a random seed, you can later reuse that seed in another generation.

Updates to the backing model may affect determinism in the generation.

Output images have a name format that starts with the seed, IE: s_(seed here)_ ...png

Reusing a seed has the effect of perfectly reproducing the image in the case that allother parameters are left alone, including the model version.

You can output a configuration file for each image / animation produced that will reproduce itexactly using the option --output-configs, that same information can be written to themetadata of generated PNG files using the option --output-metadata and can be read backwith ImageMagick for example as so:

magick identify -format "%[Property:DgenerateConfig]" generated_file.png

Generated configuration can be read back into dgenerate via a pipe or file redirection.

magick identify -format "%[Property:DgenerateConfig]" generated_file.png | dgeneratedgenerate < generated-config.dgen

Specifying a seed directly and changing the prompt slightly, or parameters such as image seed strengthif using a seed image, guidance scale, or inference steps, will allow for generating variations closeto the original image which may possess all of the original qualities about the image that you liked as well asadditional qualities. You can further manipulate the AI into producing results that you want with this method.

Changing output resolution will drastically affect image content when reusing a seed to the point where trying toreuse a seed with a different output size is pointless.

The following command demonstrates manually specifying two different seeds to try: 1234567890, and 9876543210

dgenerate stabilityai/stable-diffusion-2-1 \--prompts "an astronaut riding a horse" \--seeds 1234567890 9876543210 \--output-path astronaut \--inference-steps 50 \--guidance-scales 10 \--output-size 512x512

The desired GPU to use for CUDA acceleration can be selected using --device cuda:N where N isthe device number of the GPU as reported by nvidia-smi.

# Console 1, run on GPU 0dgenerate stabilityai/stable-diffusion-2-1 \--prompts "an astronaut riding a horse" \--output-path astronaut_1 \--inference-steps 50 \--guidance-scales 10 \--output-size 512x512 \--device cuda:0# Console 2, run on GPU 1 in paralleldgenerate stabilityai/stable-diffusion-2-1 \--prompts "an astronaut riding a cow" \--output-path astronaut_2 \--inference-steps 50 \--guidance-scales 10 \--output-size 512x512 \--device cuda:1

A scheduler (otherwise known as a sampler) for the main model can be selected via the use of --scheduler.

And in the case of SDXL the refiner’s scheduler can be selected independently with --sdxl-refiner-scheduler.

For Stable Cascade the decoder scheduler can be specified via the argument -s-cascade-decoder-schedulerhowever only one scheduler type is supported for Stable Cascade (DDPMWuerstchenScheduler).

Both of these default to the value of --scheduler, which in turn defaults to automatic selection.

Available schedulers for a specific combination of dgenerate arguments can bequeried using --scheduler help, --sdxl-refiner-scheduler help, or --s-cascade-decoder-scheduler helpthough they cannot be queried simultaneously.

In order to use the query feature it is ideal that you provide all the other argumentsthat you plan on using while making the query, as different combinations of argumentswill result in different underlying pipeline implementations being created, each of whichmay have different compatible scheduler names listed. The model needs to be loaded in order togather this information.

For example there is only one compatible scheduler for this upscaler configuration:

dgenerate stabilityai/sd-x2-latent-upscaler --variant fp16 --dtype float16 \--model-type torch-upscaler-x2 \--prompts "none" \--image-seeds my-image.png \--output-size 256 \--scheduler help# Outputs:## Compatible schedulers for "stabilityai/sd-x2-latent-upscaler" are:## "EulerDiscreteScheduler"

Typically however, there will be many compatible schedulers:

dgenerate stabilityai/stable-diffusion-2 \--inference-steps 40 \--guidance-scales 8 \--output-size 1024 \--gen-seeds 2 \--prompts "none" \--scheduler help# Outputs:## Compatible schedulers for "stabilityai/stable-diffusion-2" are:## "DDIMScheduler"# "DDPMScheduler"# "DEISMultistepScheduler"# "DPMSolverMultistepScheduler"# "DPMSolverSDEScheduler"# "DPMSolverSinglestepScheduler"# "EDMEulerScheduler"# "EulerAncestralDiscreteScheduler"# "EulerDiscreteScheduler"# "HeunDiscreteScheduler"# "KDPM2AncestralDiscreteScheduler"# "KDPM2DiscreteScheduler"# "LCMScheduler"# "LMSDiscreteScheduler"# "PNDMScheduler"# "UniPCMultistepScheduler"

Passing helpargs to a --scheduler related option will reveal configuration arguments thatcan be overridden via a URI syntax, for every possible scheduler.

dgenerate stabilityai/stable-diffusion-2 \--inference-steps 40 \--guidance-scales 8 \--output-size 1024 \--gen-seeds 2 \--prompts "none" \--scheduler helpargs# Outputs (shortened for brevity...):## Compatible schedulers for "stabilityai/stable-diffusion-2" are:# ...## PNDMScheduler:# num-train-timesteps=1000# beta-start=0.0001# beta-end=0.02# beta-schedule=linear# trained-betas=None# skip-prk-steps=False# set-alpha-to-one=False# prediction-type=epsilon# timestep-spacing=leading# steps-offset=0## ...

As an example, you may override the mentioned arguments for any scheduler in this manner:

# Change prediction type of the scheduler to "v_prediction".# for some models this may be necessary, not for this model# this is just a syntax exampledgenerate stabilityai/stable-diffusion-2 \--inference-steps 40 \--guidance-scales 8 \--output-size 1024 \--gen-seeds 2 \--prompts "none" \--scheduler PNDMScheduler;prediction-type=v_prediction

To specify a VAE directly use --vae.

VAEs are supported for these model types:

  • --model-type torch

  • --model-type flax

  • --model-type torch-pix2pix

  • --model-type torch-upscaler-x2

  • --model-type torch-upscaler-x4

  • --model-type torch-sdxl

  • --model-type torch-sdxl-pix2pix

The URI syntax for --vae is AutoEncoderClass;model=(huggingface repository slug/blob link or file/folder path)

Named arguments when loading a VAE are separated by the ; character and are not positional,meaning they can be defined in any order.

Loading arguments available when specifying a VAE for torch --model-type valuesare: model, revision, variant, subfolder, and dtype

Loading arguments available when specifying VAE for flax --model-type valuesare: model, revision, subfolder, dtype

The only named arguments compatible with loading a .safetensors or other model filedirectly off disk are model and dtype

The other named arguments are available when loading from a huggingface repository or folderthat may or may not be a local git repository on disk.

Available encoder classes for torch models are:

  • AutoencoderKL

  • AsymmetricAutoencoderKL (Does not support --vae-slicing or --vae-tiling)

  • AutoencoderTiny

  • ConsistencyDecoderVAE

Available encoder classes for flax models are:

  • FlaxAutoencoderKL (Does not support --vae-slicing or --vae-tiling)

The AutoencoderKL encoder class accepts huggingface repository slugs/blob links,.pt, .pth, .bin, .ckpt, and .safetensors files. Other encoders can only accept huggingfacerepository slugs/blob links, or a path to a folder on disk with the modelconfiguration and model file(s).

dgenerate stabilityai/stable-diffusion-2-1 \--vae "AutoencoderKL;model=stabilityai/sd-vae-ft-mse" \--prompts "an astronaut riding a horse" \--output-path astronaut \--inference-steps 50 \--guidance-scales 10 \--output-size 512x512

If you want to select the repository revision, such as main etc, use the named argument revision,subfolder is required in this example as well because the VAE model file exists in a subfolderof the specified huggingface repository.

dgenerate stabilityai/stable-diffusion-2-1 \--revision fp16 \--dtype float16 \--vae "AutoencoderKL;model=stabilityai/stable-diffusion-2-1;revision=fp16;subfolder=vae" \--prompts "an astronaut riding a horse" \--output-path astronaut \--inference-steps 50 \--guidance-scales 10 \--output-size 512x512

If you wish to specify a weights variant IE: load pytorch_model.<variant>.safetensors, from a huggingfacerepository that has variants of the same model, use the named argument variant. This usage is onlyvalid when loading VAEs if --model-type is either torch or torch-sdxl. Attemptingto use it with FlaxAutoencoderKL with produce an error message. When not specified in the URI,this value does NOT default to the value --variant to prevent errors during common use cases.If you wish to select a variant you must specify it in the URI.

dgenerate stabilityai/stable-diffusion-2-1 \--variant fp16 \--vae "AutoencoderKL;model=stabilityai/stable-diffusion-2-1;subfolder=vae;variant=fp16" \--prompts "an astronaut riding a horse" \--output-path astronaut \--inference-steps 50 \--guidance-scales 10 \--output-size 512x512

If your weights file exists in a subfolder of the repository, use the named argument subfolder

dgenerate stabilityai/stable-diffusion-2-1 \--vae "AutoencoderKL;model=stabilityai/stable-diffusion-2-1;subfolder=vae" \--prompts "an astronaut riding a horse" \--output-path astronaut \--inference-steps 50 \--guidance-scales 10 \--output-size 512x512

If you want to specify the model precision, use the named argument dtype,accepted values are the same as --dtype, IE: ‘float32’, ‘float16’, ‘auto’

dgenerate stabilityai/stable-diffusion-2-1 \--revision fp16 \--dtype float16 \--vae "AutoencoderKL;model=stabilityai/stable-diffusion-2-1;revision=fp16;subfolder=vae;dtype=float16" \--prompts "an astronaut riding a horse" \--output-path astronaut \--inference-steps 50 \--guidance-scales 10 \--output-size 512x512

If you are loading a .safetensors or other file from a path on disk, only the model, and dtypearguments are available.

# These are only syntax examplesdgenerate huggingface/diffusion_model \--vae "AutoencoderKL;model=my_vae.safetensors" \--prompts "Syntax example"dgenerate huggingface/diffusion_model \--vae "AutoencoderKL;model=my_vae.safetensors;dtype=float16" \--prompts "Syntax example"

You can use --vae-tiling and --vae-slicing to enable to generation of huge imageswithout running your GPU out of memory. Note that if you are using --control-nets you maystill be memory limited by the size of the image being processed by the ControlNet, and stillmay run in to memory issues with large image inputs.

When --vae-tiling is used, the VAE will split the input tensor into tiles tocompute decoding and encoding in several steps. This is useful for saving a large amount ofmemory and to allow processing larger images.

When --vae-slicing is used, the VAE will split the input tensor in slices tocompute decoding in several steps. This is useful to save some memory, especiallywhen --batch-size is greater than 1.

# Here is an SDXL example of high resolution image generation utilizing VAE tiling/slicingdgenerate stabilityai/stable-diffusion-xl-base-1.0 --model-type torch-sdxl \--variant fp16 --dtype float16 \--vae "AutoencoderKL;model=madebyollin/sdxl-vae-fp16-fix" \--vae-tiling \--vae-slicing \--sdxl-refiner stabilityai/stable-diffusion-xl-refiner-1.0 \--sdxl-high-noise-fractions 0.8 \--inference-steps 30 \--guidance-scales 8 \--output-size 2048 \--sdxl-target-size 2048 \--prompts "Photo of a horse standing near the open door of a red barn, high resolution; artwork"

An alternate UNet model can be specified via a URI with the --unet option, in asimilar fashion to --vae and other model arguments that accept URIs.

Specifying a --unet value is supported for all model types which support --vae

This is useful in particular for using the latent consistency scheduler.

The first component of the --unet URI is the model path itself.

You can provide a path to a huggingface repo, or a folder on disk (downloaded huggingface repository).

dgenerate stabilityai/stable-diffusion-xl-base-1.0 --model-type torch-sdxl \--variant fp16 --dtype float16 \--unet latent-consistency/lcm-sdxl \--scheduler LCMScheduler \--inference-steps 4 \--guidance-scales 8 \--gen-seeds 2 \--output-size 1024 \--prompts "a close-up picture of an old man standing in the rain"

Loading arguments available when specifying a UNet for torch --model-type valuesare: revision, variant, subfolder, and dtype

In the case of --unet the variant loading argument defaults to the valueof --variant if you do not specify it in the URI.

Loading arguments available when specifying UNet for flax --model-type valuesare: revision, subfolder, dtype. variant is not used for flax.

The --unet2 option can be used to specify a UNet for theSDXL Refiner or Stable Cascade Decoder,and uses the same syntax as --unet.

When the main model is an SDXL model and --model-type torch-sdxl is specified,you may specify a refiner model with --sdxl-refiner.

You can provide a path to a huggingface repo/blob link, folder on disk, or a model fileon disk such as a .pt, .pth, .bin, .ckpt, or .safetensors file.

This argument is parsed in much the same way as the argument --vae, except themodel is the first value specified.

Loading arguments available when specifying a refiner are: revision, variant, subfolder, and dtype

The only named argument compatible with loading a .safetensors or other file directly off disk is dtype

The other named arguments are available when loading from a huggingface repo/blob link,or folder that may or may not be a local git repository on disk.

# Basic usage of SDXL with a refinerdgenerate stabilityai/stable-diffusion-xl-base-1.0 --model-type torch-sdxl \--variant fp16 --dtype float16 \--sdxl-refiner stabilityai/stable-diffusion-xl-refiner-1.0 \--sdxl-high-noise-fractions 0.8 \--inference-steps 40 \--guidance-scales 8 \--output-size 1024 \--prompts "Photo of a horse standing near the open door of a red barn, high resolution; artwork"

If you want to select the repository revision, such as main etc, use the named argument revision

dgenerate stabilityai/stable-diffusion-xl-base-1.0 --model-type torch-sdxl \--variant fp16 --dtype float16 \--sdxl-refiner "stabilityai/stable-diffusion-xl-refiner-1.0;revision=main" \--sdxl-high-noise-fractions 0.8 \--inference-steps 40 \--guidance-scales 8 \--output-size 1024 \--prompts "Photo of a horse standing near the open door of a red barn, high resolution; artwork"

If you wish to specify a weights variant IE: load pytorch_model.<variant>.safetensors, from a huggingfacerepository that has variants of the same model, use the named argument variant. By default thisvalue is the same as --variant unless you override it.

dgenerate stabilityai/stable-diffusion-xl-base-1.0 --model-type torch-sdxl \--variant fp16 --dtype float16 \--sdxl-refiner "stabilityai/stable-diffusion-xl-refiner-1.0;variant=fp16" \--sdxl-high-noise-fractions 0.8 \--inference-steps 40 \--guidance-scales 8 \--output-size 1024 \--prompts "Photo of a horse standing near the open door of a red barn, high resolution; artwork"

If your weights file exists in a subfolder of the repository, use the named argument subfolder

# This is a non working example as I do not know of a repo with an SDXL refiner# in a subfolder :) this is only a syntax exampledgenerate huggingface/sdxl_model --model-type torch-sdxl \--variant fp16 --dtype float16 \--sdxl-refiner "huggingface/sdxl_refiner;subfolder=repo_subfolder"

If you want to select the model precision, use the named argument dtype. Bydefault this value is the same as --dtype unless you override it. Acceptedvalues are the same as --dtype, IE: ‘float32’, ‘float16’, ‘auto’

dgenerate stabilityai/stable-diffusion-xl-base-1.0 --model-type torch-sdxl \--variant fp16 --dtype float16 \--sdxl-refiner "stabilityai/stable-diffusion-xl-refiner-1.0;dtype=float16" \--sdxl-high-noise-fractions 0.8 \--inference-steps 40 \--guidance-scales 8 \--output-size 1024 \--prompts "Photo of a horse standing near the open door of a red barn, high resolution; artwork"

If you are loading a .safetensors or other file from a path on disk, simply do:

# This is only a syntax exampledgenerate huggingface/sdxl_model --model-type torch-sdxl \--sdxl-refiner my_refinermodel.safetensors

When preforming inpainting or when using ControlNets, therefiner will automatically operate in edit mode instead of cooperative denoising mode.Edit mode can be forced in other situations with the option --sdxl-refiner-edit.

Edit mode means that the refiner model is accepting the fully (or mostly) denoised outputof the main model generated at the full number of inference steps specified, and actingon it with an image strength (image seed strength) determined by (1.0 - high-noise-fraction).

The output latent from the main model is renoised with a certain amount of noise determinedby the strength, a lower number means less noise and less modification of the latent outputby the main model.

This is similar to what happens when using dgenerate in img2img with a standalone model,technically it is just img2img, however refiner models are better at enhancing detailsfrom the main model in this use case.

When the main model is a Stable Cascade prior model and --model-type torch-s-cascade is specified,you may specify a decoder model with --s-cascade-decoder.

The syntax (and URI arguments) for specifying the decoder model is identical to specifying an SDXL refinermodel as mentioned above.

dgenerate stabilityai/stable-cascade-prior \--model-type torch-s-cascade \--variant bf16 \--dtype bfloat16 \--model-cpu-offload \--s-cascade-decoder-cpu-offload \--s-cascade-decoder "stabilityai/stable-cascade;dtype=float16" \--inference-steps 20 \--guidance-scales 4 \--s-cascade-decoder-inference-steps 10 \--s-cascade-decoder-guidance-scales 0 \--gen-seeds 2 \--prompts "an image of a shiba inu, donning a spacesuit and helmet"

It is possible to specify one or more LoRA models using --loras

LoRAs are supported for these model types:

  • --model-type torch

  • --model-type torch-pix2pix

  • --model-type torch-upscaler-x4

  • --model-type torch-sdxl

  • --model-type torch-sdxl-pix2pix

When multiple specifications are given, all mentioned models will be fused intothe main model at a given scale.

The plural form of the argument is identical to the non-plural version, which onlyexists for backward compatibility.

You can provide a huggingface repository slug, .pt, .pth, .bin, .ckpt, or .safetensors files.Blob links are not accepted, for that use subfolder and weight-name described below.

The LoRA scale can be specified after the model path by placing a ; (semicolon) andthen using the named argument scale

When a scale is not specified, 1.0 is assumed.

Named arguments when loading a LoRA are separated by the ; character and arenot positional, meaning they can be defined in any order.

Loading arguments available when specifying a LoRA are: scale, revision, subfolder, and weight-name

The only named argument compatible with loading a .safetensors or other file directly off disk is scale

The other named arguments are available when loading from a huggingface repository or folderthat may or may not be a local git repository on disk.

This example shows loading a LoRA using a huggingface repository slug and specifying scale for it.

# Don't expect great results with this example,# Try models and LoRA's downloaded from CivitAIdgenerate runwayml/stable-diffusion-v1-5 \--loras "pcuenq/pokemon-lora;scale=0.5" \--prompts "Gengar standing in a field at night under a full moon, highquality, masterpiece, digital art" \--inference-steps 40 \--guidance-scales 10 \--gen-seeds 5 \--output-size 800

Specifying the file in a repository directly can be done with the named argument weight-name

Shown below is an SDXL compatible LoRA being used with the SDXL base model and a refiner.

dgenerate stabilityai/stable-diffusion-xl-base-1.0 --model-type torch-sdxl \--inference-steps 30 \--sdxl-refiner stabilityai/stable-diffusion-xl-refiner-1.0 \--prompts "sketch of a horse by Leonardo da Vinci" \--variant fp16 --dtype float16 \--loras "goofyai/SDXL-Lora-Collection;scale=1.0;weight-name=leonardo_illustration.safetensors" \--output-size 1024

If you want to select the repository revision, such as main etc, use the named argument revision

dgenerate runwayml/stable-diffusion-v1-5 \--loras "pcuenq/pokemon-lora;scale=0.5;revision=main" \--prompts "Gengar standing in a field at night under a full moon, highquality, masterpiece, digital art" \--inference-steps 40 \--guidance-scales 10 \--gen-seeds 5 \--output-size 800

If your weights file exists in a subfolder of the repository, use the named argument subfolder

# This is a non working example as I do not know of a repo with a LoRA weight in a subfolder :)# This is only a syntax exampledgenerate huggingface/model \--prompts "Syntax example" \--loras "huggingface/lora_repo;scale=1.0;subfolder=repo_subfolder;weight-name=lora_weights.safetensors"

If you are loading a .safetensors or other file from a path on disk, only the scale argument is available.

# This is only a syntax exampledgenerate runwayml/stable-diffusion-v1-5 \--prompts "Syntax example" \--loras "my_lora.safetensors;scale=1.0"

One or more Textual Inversion models (otherwise known as embeddings) may be specified with --textual-inversions

Textual inversions are supported for these model types:

  • --model-type torch

  • --model-type torch-pix2pix

  • --model-type torch-upscaler-x4

  • --model-type torch-sdxl

  • --model-type torch-sdxl-pix2pix

You can provide a huggingface repository slug, .pt, .pth, .bin, .ckpt, or .safetensors files.Blob links are not accepted, for that use subfolder and weight-name described below.

Arguments pertaining to the loading of each textual inversion model may be specified in the sameway as when using --loras minus the scale argument.

Available arguments are: token, revision, subfolder, and weight-name

Named arguments are available when loading from a huggingface repository or folderthat may or may not be a local git repository on disk, when loading directly from a .safetensors fileor other file from a path on disk they should not be used.

The token argument may be used to override the prompt token value, which is the text tokenin the prompt that triggers the inversion, textual inversions for stable diffusion usuallyinclude this token value in the model itself, for instance in the example below the tokenfor Isometric_Dreams-1000.pt is Isometric_Dreams.

The token value used for SDXL (Stable Diffusion XL) models is a bit different, a defaultvalue is not provided in the model file. If you do not provide a token value, dgeneratewill assign the tokens default value to the filename of the model with any spaces converted tounderscores, and with the file extension removed.

# Load a textual inversion from a huggingface repository specifying it's name in the repository# as an argumentdgenerate Duskfallcrew/isometric-dreams-sd-1-5 \--textual-inversions "Duskfallcrew/IsometricDreams_TextualInversions;weight-name=Isometric_Dreams-1000.pt" \--scheduler KDPM2DiscreteScheduler \--inference-steps 30 \--guidance-scales 7 \--prompts "a bright photo of the Isometric_Dreams, a tv and a stereo in it and a book shelf, a table, a couch,a room with a bed"

You can change the token value to affect the prompt token used to trigger the embedding

# Load a textual inversion from a huggingface repository specifying it's name in the repository# as an argumentdgenerate Duskfallcrew/isometric-dreams-sd-1-5 \--textual-inversions "Duskfallcrew/IsometricDreams_TextualInversions;weight-name=Isometric_Dreams-1000.pt;token=<MY_TOKEN>" \--scheduler KDPM2DiscreteScheduler \--inference-steps 30 \--guidance-scales 7 \--prompts "a bright photo of the <MY_TOKEN>, a tv and a stereo in it and a book shelf, a table, a couch,a room with a bed"

If you want to select the repository revision, such as main etc, use the named argument revision

# This is a non working example as I do not know of a repo that utilizes revisions with# textual inversion weights :) this is only a syntax exampledgenerate huggingface/model \--prompts "Syntax example" \--textual-inversions "huggingface/ti_repo;revision=main"

If your weights file exists in a subfolder of the repository, use the named argument subfolder

# This is a non working example as I do not know of a repo with a textual# inversion weight in a subfolder :) this is only a syntax exampledgenerate huggingface/model \--prompts "Syntax example" \--textual-inversions "huggingface/ti_repo;subfolder=repo_subfolder;weight-name=ti_model.safetensors"

If you are loading a .safetensors or other file from a path on disk, simply do:

# This is only a syntax exampledgenerate runwayml/stable-diffusion-v1-5 \--prompts "Syntax example" \--textual-inversions "my_ti_model.safetensors"

One or more ControlNet models may be specified with --control-nets, and multiple controlnet guidance images can be specified via --image-seeds in the case that you specifymultiple control net models.

ControlNet models are supported for these model types:

  • --model-type torch

  • --model-type flax

  • --model-type torch-sdxl

You can provide a huggingface repository slug / blob link, .pt, .pth, .bin, .ckpt, or .safetensors files.

Control images for the Control Nets can be provided using --image-seeds

When using --control-nets specifying control images via --image-seeds can be accomplished in these ways:

  • --image-seeds "control-image.png" (txt2img)

  • --image-seeds "img2img-seed.png;control=control-image.png" (img2img)

  • --image-seeds "img2img-seed.png;mask=mask.png;control=control-image.png" (inpainting)

Multiple control image sources can be specified in these ways when using multiple control nets:

  • --image-seeds "control-1.png, control-2.png" (txt2img)

  • --image-seeds "img2img-seed.png;control=control-1.png, control-2.png" (img2img)

  • --image-seeds "img2img-seed.png;mask=mask.png;control=control-1.png, control-2.png" (inpainting)

It is considered a syntax error if you specify a non-equal amount of control guidanceimages and --control-nets URIs and you will receive an error message if you do so.

resize=WIDTHxHEIGHT can be used to select a per --image-seeds resize dimension for all imagesources involved in that particular specification, as well as aspect=true/false and the frameslicing arguments frame-start and frame-end.

ControlNet guidance images may actually be animations such as MP4s, GIFs etc. Frames can betaken from multiple videos simultaneously. Any possible combination of image/video parameters can be used.The animation with least amount of frames in the entire specification determines the frame count, andany static images present are duplicated across the entire animation. The first animation presentin an image seed specification always determines the output FPS of the animation.

Arguments pertaining to the loading of each ControlNet model specified with --control-nets may bedeclared in the same way as when using --vae with the addition of a scale argument and from_torchargument when using flax --model-type values.

Available arguments when using torch --model-type values are: scale, start, end, revision, variant, subfolder, dtype

Available arguments when using flax --model-type values are: scale, revision, subfolder, dtype, from_torch

Most named arguments apply to loading from a huggingface repository or folderthat may or may not be a local git repository on disk, when loading directly from a .safetensors fileor other file from a path on disk the available arguments are scale, start, end, and from_torch.from_torch can be used with flax for loading pytorch models from .pt or other files designed for torch from a repo or file/folder on disk.

The scale argument indicates the affect scale of the control net model.

For torch, the start argument indicates at what fraction of the total inference stepsat which the control net model starts to apply guidance. If you have multiplecontrol net models specified, they can apply guidance over different segmentsof the inference steps using this option, it defaults to 0.0, meaning start at thefirst inference step.

for torch, the end argument indicates at what fraction of the total inference stepsat which the control net model stops applying guidance. It defaults to 1.0, meaningstop at the last inference step.

These examples use: vermeer_canny_edged.png

# Torch example, use "vermeer_canny_edged.png" as a control guidance imagedgenerate runwayml/stable-diffusion-v1-5 \--inference-steps 40 \--guidance-scales 8 \--prompts "Painting, Girl with a pearl earing by Leonardo Da Vinci, masterpiece; low quality, low resolution, blank eyeballs" \--control-nets "lllyasviel/sd-controlnet-canny;scale=0.5" \--image-seeds "vermeer_canny_edged.png"# If you have an img2img image seed, use this syntaxdgenerate runwayml/stable-diffusion-v1-5 \--inference-steps 40 \--guidance-scales 8 \--prompts "Painting, Girl with a pearl earing by Leonardo Da Vinci, masterpiece; low quality, low resolution, blank eyeballs" \--control-nets "lllyasviel/sd-controlnet-canny;scale=0.5" \--image-seeds "my-image-seed.png;control=vermeer_canny_edged.png"# If you have an img2img image seed and an inpainting mask, use this syntaxdgenerate runwayml/stable-diffusion-v1-5 \--inference-steps 40 \--guidance-scales 8 \--prompts "Painting, Girl with a pearl earing by Leonardo Da Vinci, masterpiece; low quality, low resolution, blank eyeballs" \--control-nets "lllyasviel/sd-controlnet-canny;scale=0.5" \--image-seeds "my-image-seed.png;mask=my-inpaint-mask.png;control=vermeer_canny_edged.png"# Flax exampledgenerate runwayml/stable-diffusion-v1-5 --model-type flax \--revision bf16 \--dtype float16 \--inference-steps 40 \--guidance-scales 8 \--prompts "Painting, Girl with a pearl earing by Leonardo Da Vinci, masterpiece; low quality, low resolution, blank eyeballs" \--control-nets "lllyasviel/sd-controlnet-canny;scale=0.5;from_torch=true" \--image-seeds "vermeer_canny_edged.png"# SDXL exampledgenerate stabilityai/stable-diffusion-xl-base-1.0 --model-type torch-sdxl \--variant fp16 --dtype float16 \--vae "AutoencoderKL;model=madebyollin/sdxl-vae-fp16-fix" \--sdxl-refiner stabilityai/stable-diffusion-xl-refiner-1.0 \--inference-steps 30 \--guidance-scales 8 \--prompts "Taylor Swift, high quality, masterpiece, high resolution; low quality, bad quality, sketches" \--control-nets "diffusers/controlnet-canny-sdxl-1.0;scale=0.5" \--image-seeds "vermeer_canny_edged.png" \--output-size 1024

If you want to select the repository revision, such as main etc, use the named argument revision

# This is a non working example as I do not know of a repo that utilizes revisions with# ControlNet weights :) this is only a syntax exampledgenerate huggingface/model \--prompts "Syntax example" \--control-nets "huggingface/cn_repo;revision=main"

If your weights file exists in a subfolder of the repository, use the named argument subfolder

# This is a non working example as I do not know of a repo with a textual# inversion weight in a subfolder :) this is only a syntax exampledgenerate huggingface/model \--prompts "Syntax example" \--control-nets "huggingface/cn_repo;subfolder=repo_subfolder"

If you are loading a .safetensors or other file from a path on disk, simply do:

# This is only a syntax exampledgenerate runwayml/stable-diffusion-v1-5 \--prompts "Syntax example" \--control-nets "my_cn_model.safetensors"

Any model accepted by dgenerate that can be specified as a single fileinside of a URI or otherwise can be specified by a URL link to a modelfile itself. dgenerate will attempt to download the file from the link,store it in the web cache, and then use it.

You may also use the \download config directive to assist in predownloading other resources from the internet. The directive has the abilityto specify arbitrary storage locations. See: The \download directive

You can also use the download() template function for similarpurposes. See: The download() template function

In the case of CivitAI you can use this to bake models into your scriptthat will be automatically downloaded for you, you just need a CivitAIaccount and API token to download models.

Your API token can be created on this page: https://civitai.com/user/account

Near the bottom of the page in the section: API Keys

To get a direct link to a CivitAI model, in your browser(right click -> Copy link address...) on the download link for thespecific file on the model page. This will yield a link that pointsdirectly at the model file, which is what dgenerate needs.

If you plan to download many large models to the web cache inthis manner you may wish to adjust the global cache expiry timeso that they exist in the cache longer than the default of 12 hours.

You can see how to change the cache expiry time in this section File Cache Control

#!/usr/bin/env bash# Download the main model from civitai using an api token# https://civitai.com/models/122822?modelVersionId=133832TOKEN=your_api_token_hereMODEL="https://civitai.com/api/download/models/133832?type=Model&format=SafeTensor&size=full&fp=fp16&token=$TOKEN"dgenerate $MODEL \--model-type torch-sdxl \--variant fp16 --dtype float16 \--sdxl-refiner stabilityai/stable-diffusion-xl-refiner-1.0 \--sdxl-high-noise-fractions 0.8 \--guidance-scales 8 \--inference-steps 40 \--prompts "a fluffy cat playing in the grass"

This method can be used for VAEs, LoRAs, ControlNets, and Textual Inversionsas well whenever single file loads are supported by the argument.

Multiple image variations from the same seed can be produce on a GPU simultaneouslyusing the --batch-size option of dgenerate. This can be used in combination with--batch-grid-size to output image grids if desired.

When not writing to image grids the files in the batch will be written to diskwith the suffix _image_N where N is index of the image in the batch of imagesthat were generated.

When producing an animation, you can either write N animation output fileswith the filename suffixes _animation_N where N is the index of the imagein the batch which makes up the frames. Or you can use `--batch-grid-size towrite frames to a single animated output where the frames are all image gridsproduced from the images in the batch.

With larger --batch-size values, the use of --vae-slicing can make the differencebetween an out of memory condition and success, so it is recommended that youtry this option if you experience an out of memory condition due to the use of--batch-size.

Images provided through --image-seeds can be processed before being used for image generationthrough the use of the arguments --seed-image-processors, --mask-image-processors, and--control-image-processors. In addition, dgenerates output can be post processed with theused of the --post-processors argument, which is useful for using the upscaler processor.An important note about --post-processors is that post processing occurs before any image gridrendering is preformed when --batch-grid-size is specified with a --batch-size greater than one,meaning that the output images are processed with your processor before being put into a grid.

Each of these options can receive one or more specifications for image processing actions,multiple processing actions will be chained together one after another.

Using the option --image-processor-help with no arguments will yield a list of available image processor names.

dgenerate --image-processor-help# Output:## Available image processors:## "sam"# "pidi"# "normal-bae"# "upscaler"# "grayscale"# "invert"# "posterize"# "mirror"# "flip"# "mlsd"# "leres"# "hed"# "solarize"# "midas"# "canny"# "lineart"# "openpose"# "lineart-anime"

Specifying one or more specific processors for example: --image-processor-help canny openpose will yielddocumentation pertaining to those processor modules. This includes accepted arguments and their types for theprocessor module and a description of what the module does.

Custom image processor modules can also be loaded through the --plugin-modules option as discussedin the Writing Plugins section.

All processors posses the arguments: output-file, output-overwrite, device, and model-offload

The output-file argument can be used to write the processed image to a specific file, if multipleprocessing steps occur such as when rendering an animation or multiple generation steps, a numbered suffixwill be appended to this filename. Note that an output file will only be produced in the case that theprocessor actually modifies an input image in some way. This can be useful for debugging an image thatis being fed into diffusion or a ControlNet.

The output-overwrite is a boolean argument can be used to tell the processor that you do not want numberedsuffixes to be generated for output-file and to simply overwrite it.

The device argument can be used to override what device any hardware accelerated image processingoccurs on if any. It defaults to the value of --device and has the same syntax for specifying deviceordinals, for instance if you have multiple GPUs you may specify device=cuda:1 to run image processingon your second GPU, etc. Not all image processors respect this argument as some image processing is onlyever CPU based.

The model-offload is a boolean argument that can be used to force any torch modules / tensorsassociated with an image processor to immediately evacuate the GPU or other non CPU processing deviceas soon as the processor finishes processing an image. Usually, any modules / tensors will bebrought on to the desired device right before processing an image, and left on the device untilthe image processor object leaves scope and is garbage collected. This can be useful for achievingcertain GPU or processing device memory constraints, however it is slower when processing multipleimages in a row, as the modules / tensors must be brought on to the desired device repeatedlyfor each image. In the context of dgenerate invocations where processors can be used as preprocessorsor postprocessors, the image processor object is garbage collected when the invocation completes,this is also true for the \image_process directive. Using this argument with a preprocessspecification, such as --control-image-processors may yield a noticeable memory overheadreduction when using a single GPU, as any models from the image processor will be moved to theCPU immediately when it is done, clearing up VRAM space before the diffusion models enter GPU VRAM.

For an example, images can be processed with the canny edge detection algorithm or OpenPose (rigging generation)before being used for generation with a model + a ControlNet.

This image of a horseis used in the example below with a ControlNet that is trained to generate images from canny edge detected input.

# --control-image-processors is only used for control images# in this case the single image seed is considered a control image# because --control-nets is being useddgenerate stabilityai/stable-diffusion-xl-base-1.0 --model-type torch-sdxl \--variant fp16 --dtype float16 \--vae "AutoencoderKL;model=madebyollin/sdxl-vae-fp16-fix" \--sdxl-refiner stabilityai/stable-diffusion-xl-refiner-1.0 \--inference-steps 30 \--guidance-scales 8 \--prompts "Majestic unicorn, high quality, masterpiece, high resolution; low quality, bad quality, sketches" \--control-nets "diffusers/controlnet-canny-sdxl-1.0;scale=0.5" \--image-seeds "horse.jpeg" \--control-image-processors "canny;lower=50;upper=100" \--gen-seeds 2 \--output-size 1024 \--output-path unicorn

The --control-image-processors has a special additional syntax that the other processor specificationoptions do not, which is used to describe which processor group is affecting which control guidance imagesource in an --image-seeds specification.

For instance if you have multiple control guidance images, and multiple control nets which are goingto use those images, or frames etc. and you want to process each guidance image with a separateprocessor OR processor chain. You can specify how each image is processed by delimiting theprocessor specification groups with + (the plus symbol)

Like this:

  • --control-nets "huggingface/controlnet1" "huggingface/controlnet2"

  • --image-seeds "image1.png, image2.png"

  • --control-image-processors "affect-image1" + "affect-image2"

Specifying a non-equal amount of control guidance images and --control-nets URIs isconsidered a syntax error and you will receive an error message if you do so.

You can use processor chaining as well:

  • --control-nets "huggingface/controlnet1" "huggingface/controlnet2"

  • --image-seeds "image1.png, image2.png"

  • --control-image-processors "affect-image1" "affect-image1-again" + "affect-image2"

In the case that you would only like the second image affected:

  • --control-nets "huggingface/controlnet1" "huggingface/controlnet2"

  • --image-seeds "image1.png, image2.png"

  • --control-image-processors + "affect-image2"

The plus symbol effectively creates a NULL processor as the first entry in the example above.

When multiple guidance images are present, it is a syntax error to specify more processor chainsthan control guidance images. Specifying less processor chains simply means that the trailingguidance images will not be processed, you can avoid processing leading guidance imageswith the mechanism described above.

This can be used with an arbitrary amount of control image sources and control nets, takefor example the specification:

  • --control-nets "huggingface/controlnet1" "huggingface/controlnet2" "huggingface/controlnet3"

  • --image-seeds "image1.png, image2.png, image3.png"

  • --control-image-processors + + "affect-image3"

The two + (plus symbol) arguments indicate that the first two images mentioned in the control imagespecification in --image-seeds are not to be processed by any processor.

Stable diffusion image upscaling models can be used via the model types torch-upscaler-x2 and torch-upscaler-x4.

The image used in the example below is this low resolution cat

# The image produced with this model will be# two times the --output-size dimension IE: 512x512 in this case# The image is being resized to 256x256, and then upscaled by 2xdgenerate stabilityai/sd-x2-latent-upscaler --variant fp16 --dtype float16 \--model-type torch-upscaler-x2 \--prompts "a picture of a white cat" \--image-seeds low_res_cat.png \--output-size 256# The image produced with this model will be# four times the --output-size dimension IE: 1024x1024 in this case# The image is being resized to 256x256, and then upscaled by 4xdgenerate stabilityai/stable-diffusion-x4-upscaler --variant fp16 --dtype float16 \ --model-type torch-upscaler-x4 \--prompts "a picture of a white cat" \--image-seeds low_res_cat.png \--output-size 256 \--upscaler-noise-levels 20

dgenerate implements additional functionality through the option --sub-command.

For a list of available sub-commands use --sub-command-help, which by defaultwill list available sub-command names.

For additional information on a specific sub-command use --sub-command-help NAME multiplesub-command names can be specified here if desired however currently there is only one available.

All sub-commands respect the --plugin-modules and --verbose argumentseven if their help output does not specify them, these arguments are handledby dgenerate and not the sub-command.

currently the only implemented sub-command is image-process, which you canread the help output of using dgenerate --sub-command image-process --help

The image-process sub-command can be used to run image processors implementedby dgenerate on any file of your choosing including animated images and videos.

It has a similar but slightly different design/usage to the main dgeneratecommand itself.

It can be used to run canny edge detection, openpose, etc. on any image orvideo/animated file that you want.

The help output of image-process is as follows:

usage: \image_process [-h] [-p PROCESSORS [PROCESSORS ...]] [--plugin-modules PATH [PATH ...]] [-o OUTPUT [OUTPUT ...]] [-ff FRAME_FORMAT] [-ox] [-r RESIZE] [-na] [-al ALIGN] [-d DEVICE] [-fs FRAME_NUMBER] [-fe FRAME_NUMBER] [-nf | -naf] input [input ...]This command allows you to use dgenerate image processors directly on files of your choosing.positional arguments: input Input file paths, may be a static images or animated files supported by dgenerate. URLs will be downloaded.options: -h, --help show this help message and exit -p PROCESSORS [PROCESSORS ...], --processors PROCESSORS [PROCESSORS ...] One or more image processor URIs, specifying multiple will chain them together. See: dgenerate --image-processor-help --plugin-modules PATH [PATH ...] Specify one or more plugin module folder paths (folder containing __init__.py) or python .py file paths to load as plugins. Plugin modules can implement image processors. -o OUTPUT [OUTPUT ...], --output OUTPUT [OUTPUT ...] Output files, parent directories mentioned in output paths will be created for you if they do not exist. If you do not specify output files, the output file will be placed next to the input file with the added suffix '_processed_N' unless --output-overwrite is specified, in that case it will be overwritten. If you specify multiple input files and output files, you must specify an output file for every input file, or a directory (indicated with a trailing directory seperator character, for example "my_dir/" or "my_dir\" if the directory does not exist yet). Failure to specify an output file with a URL as an input is considered an error. Supported file extensions for image output are equal to those listed under --frame-format. -ff FRAME_FORMAT, --frame-format FRAME_FORMAT Image format for animation frames. Must be one of: png, apng, blp, bmp, dib, bufr, pcx, dds, ps, eps, gif, grib, h5, hdf, jp2, j2k, jpc, jpf, jpx, j2c, icns, ico, im, jfif, jpe, jpg, jpeg, tif, tiff, mpo, msp, palm, pdf, pbm, pgm, ppm, pnm, pfm, bw, rgb, rgba, sgi, tga, icb, vda, vst, webp, wmf, emf, or xbm. -ox, --output-overwrite Indicate that it is okay to overwrite files, instead of appending a duplicate suffix. -r RESIZE, --resize RESIZE Preform naive image resizing (LANCZOS). -na, --no-aspect Make --resize ignore aspect ratio. -al ALIGN, --align ALIGN Align images / videos to this value in pixels, default is 8. Specifying 1 will disable resolution alignment. -d DEVICE, --device DEVICE Processing device, for example "cuda", "cuda:1". -fs FRAME_NUMBER, --frame-start FRAME_NUMBER Starting frame slice point for animated files (zero-indexed), the specified frame will be included. (default: 0) -fe FRAME_NUMBER, --frame-end FRAME_NUMBER Ending frame slice point for animated files (zero-indexed), the specified frame will be included. -nf, --no-frames Do not write frames, only an animation file. Cannot be used with --no- animation-file. -naf, --no-animation-file Do not write an animation file, only frames. Cannot be used with --no- frames.

Overview of specifying image-process inputs and outputs

# Overview of specifying outputs, image-process can do simple operations# like resizing images and forcing image alignment with --align, without the# need to specify any other processing operations with --processors. Running# image-process on an image with no other arguments simply aligns it to 8 pixels,# given the defaults for its command line arguments# More file formats than .png are supported for static image output, all# extensions mentioned in the image-process --help documentation for --frame-format# are supported, the supported formats are identical to that mentioned in the --image-format# option help section of dgenerates --help output# my_file.png -> my_file_processed_1.pngdgenerate --sub-command image-process my_file.png --resize 512x512# my_file.png -> my_file.png (overwrite)dgenerate --sub-command image-process my_file.png --resize 512x512 --output-overwrite# my_file.png -> my_file.png (overwrite)dgenerate --sub-command image-process my_file.png -o my_file.png --resize 512x512 --output-overwrite# my_file.png -> my_dir/my_file_processed_1.pngdgenerate --sub-command image-process my_file.png -o my_dir/ --resize 512x512 --no-aspect# my_file_1.png -> my_dir/my_file_1_processed_1.png# my_file_2.png -> my_dir/my_file_2_processed_2.pngdgenerate --sub-command image-process my_file_1.png my_file_2.png -o my_dir/ --resize 512x512# my_file_1.png -> my_dir_1/my_file_1_processed_1.png# my_file_2.png -> my_dir_2/my_file_2_processed_2.pngdgenerate --sub-command image-process my_file_1.png my_file_2.png \-o my_dir_1/ my_dir_2/ --resize 512x512# my_file_1.png -> my_dir_1/renamed.png# my_file_2.png -> my_dir_2/my_file_2_processed_2.pngdgenerate --sub-command image-process my_file_1.png my_file_2.png \-o my_dir_1/renamed.png my_dir_2/ --resize 512x512

A few usage examples with processors:

# image-process can support any input format that dgenerate itself supports# including videos and animated files. It also supports all output formats# supported by dgenerate for writing videos/animated files, and images.# create a video rigged with OpenPose, frames will be rendered to the directory "output" as well.dgenerate --sub-command image-process my-video.mp4 \-o output/rigged-video.mp4 --processors "openpose;include-hand=true;include-face=true"# Canny edge detected video, also using processor chaining to mirror the frames# before they are edge detecteddgenerate --sub-command image-process my-video.mp4 \-o output/canny-video.mp4 --processors mirror "canny;blur=true;threshold-algo=otsu"

chaiNNer compatible upscaler models from https://openmodeldb.info/and elsewhere can be utilized for tiled upscaling using dgenerates upscaler image processor and the--post-processors option. The upscaler image processor can also be used for processinginput images via the other options mentioned in Image Processors such as --seed-image-processors

The upscaler image processor can make use of URLs or files on disk.

In this example we reference a link to the SwinIR x4 upscaler from the creators github release.

This uses the upscaler to upscale the output image by x4 producing an image that is 4096x4096

The upscaler image processor respects the --device option of dgenerate, and is CUDA accelerated by default.

dgenerate stabilityai/stable-diffusion-xl-base-1.0 --model-type torch-sdxl \--variant fp16 --dtype float16 \--sdxl-refiner stabilityai/stable-diffusion-xl-refiner-1.0 \--sdxl-high-noise-fractions 0.8 \--inference-steps 40 \--guidance-scales 8 \--output-size 1024 \--prompts "Photo of a horse standing near the open door of a red barn, high resolution; artwork" \--post-processors "upscaler;model=https://github.com/JingyunLiang/SwinIR/releases/download/v0.0/001_classicalSR_DIV2K_s48w8_SwinIR-M_x4.pth"

In addition to this the \image_process config directive, or --sub-command image-process can be used to upscaleany file that you want including animated images and videos. It is worth noting that the sub-command and directivewill work with any named image processor implemented by dgenerate.

# print the help output of the sub command "image-process"# the image-process subcommand can process multiple files and do# and several other things, it is worth reading :)dgenerate --sub-command image-process --help# any directory mentioned in the output spec is created automaticallydgenerate --sub-command image-process my-file.png \--output output/my-file-upscaled.png \--processors "upscaler;model=https://github.com/JingyunLiang/SwinIR/releases/download/v0.0/001_classicalSR_DIV2K_s48w8_SwinIR-M_x4.pth"

Config scripts can be read from stdin using a shell pipe or file redirection, or byusing the --file argument to specify a file to interpret.

Config scripts are processed with model caching and other optimizations, in orderto increase speed when many dgenerate invocations with different arguments are desired.

Loading the necessary libraries and bringing models into memory is quite slow, so using dgeneratethis way allows for multiple invocations using different arguments, without needing to load themachine learning libraries and models multiple times in a row.

When a model is loaded dgenerate caches it in memory with it’s creation parameters, which includesamong other things the pipeline mode (basic, img2img, inpaint), user specified UNets, VAEs, LoRAs,Textual Inversions, and ControlNets.

If another invocation of the model occurs with creation parameters that are identical, it will beloaded out of an in memory cache, which greatly increases the speed of the invocation.

Diffusion Pipelines, user specified UNets, VAEs, and ControlNet models are cached individually.

UNets, VAEs, and ControlNet model objects can be reused by diffusion pipelines in certainsituations when specified explicitly and this is taken advantage of by using an inmemory cache of these objects.

In effect, the creation of a diffusion pipeline is memoized, as well as the creation ofany pipeline subcomponents when you have specified them explicitly with a URI.

A number of things effect cache hit or miss upon a dgenerate invocation, extensive informationregarding runtime caching behavior of a pipelines and other models can be observed using -v/--verbose

When loading multiple different models be aware that they will all be retained in memory forthe duration of program execution, unless all models are flushed using the \clear_model_cache directive orindividually using one of: \clear_pipeline_cache, \clear_unet_cache, \clear_vae_cache, or \clear_control_net_cache.

dgenerate uses heuristics to clear the in memory cache automatically when needed, including a size estimationof models before they enter system memory, however by default it will use system memory very aggressivelyand it is not entirely impossible to run your system out of memory if you are not careful.

Basic config syntax

The basic idea of the dgenerate config syntax is that it is a pseudo Unix shell mixed with Jinja2 templating.

The config language provides many niceties for batch processing large amounts of imagesand image output in a Unix shell like environment with Jinja2 control constructs.

Shell builtins, known as directives, are prefixed with \, for example: \print

Environmental variables will be expanded in config scripts using both Unix and Windows CMD syntax

# these all expand from your system environment# if the variable is not set, they expand to nothing\print $VARIABLE\print ${VARIABLE}\print %VARIABLE%

Empty lines and comments starting with # will be ignored, comments that occur at the end of lines will also be ignored.

You can create a multiline continuation using \ to indicate that a line continues similar to bash.

Unlike bash, if the next line starts with - it is considered part of a continuation as welleven if \ had not been used previously. This allows you to list out many Posix style shelloptions starting with - without having to end every line with \.

Comments can be interspersed with invocation or directive argumentson their own line with the use of \ on the last line beforecomments and whitespace begin. This can be used to add documentationabove individual arguments instead of at the tail end of them.

The following is a config file example that covers the most basic syntax concepts.

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1# If a hash-bang version is provided in the format above# a warning will be produced if the version you are running# is not compatible (SemVer), this can be used anywhere in the# config file, a line number will be mentioned in the warning when the# version check fails# Comments in the file will be ignored# Each dgenerate invocation in the config begins with the path to a model,# IE. the first argument when using dgenerate from the command line, the# rest of the options that follow are the options to dgenerate that you# would use on the command line# Guarantee unique file names are generated under the output directory by specifying unique seedsstabilityai/stable-diffusion-2-1 --prompts "an astronaut riding a horse" --seeds 41509644783027 --output-path output --inference-steps 30 --guidance-scales 10stabilityai/stable-diffusion-2-1 --prompts "a cowboy riding a horse" --seeds 78553317097366 --output-path output --inference-steps 30 --guidance-scales 10stabilityai/stable-diffusion-2-1 --prompts "a martian riding a horse" --seeds 22797399276707 --output-path output --inference-steps 30 --guidance-scales 10# Guarantee that no file name collisions happen by specifying different output paths for each invocationstabilityai/stable-diffusion-2-1 --prompts "an astronaut riding a horse" --output-path unique_output_1 --inference-steps 30 --guidance-scales 10stabilityai/stable-diffusion-2-1 --prompts "a cowboy riding a horse" --output-path unique_output_2 --inference-steps 30 --guidance-scales 10# Multiline continuations are possible implicitly for argument# switches IE lines starting with '-'stabilityai/stable-diffusion-2-1 --prompts "a martian riding a horse"--output-path unique_output_3 # there can be comments at the end of lines--inference-steps 30 \ # this comment is also ignored# There can be comments or newlines within the continuation# but you must provide \ on the previous line to indicate that# it is going to happen--guidance-scales 10# The continuation ends (on the next line) when the last line does# not end in \ or start with -# the ability to use tail comments means that escaping of the # is sometimes# necessary when you want it to appear literally, see: examples/config_syntax/tail-comments-config.dgen# for examples.# Configuration directives provide extra functionality in a config, a directive# invocation always starts with a backslash# A clear model cache directive can be used inbetween invocations if cached models that# are no longer needed in your generation pipeline start causing out of memory issues\clear_model_cache# Additionally these other directives exist to clear user loaded models# out of dgenerates in memory cache individually# Clear specifically diffusion pipelines\clear_pipeline_cache# Clear specifically user specified UNet models\clear_unet_cache# Clear specifically user specified VAE models\clear_vae_cache# Clear specifically ControlNet models\clear_control_net_cache# This model was used before but will have to be fully instantiated from scratch again# after a cache flush which may take some timestabilityai/stable-diffusion-2-1 --prompts "a martian riding a horse"--output-path unique_output_4

Built in template variables and functions

There is valuable information about the previous invocation of dgenerate thatis set in the environment and available to use via Jinja2 templating or inthe \setp directive, some of these include:

  • {{ last_images }} (An iterable of un-quoted filenames which were generated)

  • {{ last_animations }} (An iterable of un-quoted filenames which were generated)

There are template variables for prompts, containing the previous prompt values:

  • {{ last_prompts }} (List of prompt objects with the un-quoted attributes ‘positive’ and ‘negative’)

  • {{ last_sdxl_second_prompts }}

  • {{ last_sdxl_refiner_prompts }}

  • {{ last_sdxl_refiner_second_prompts }}

Some available custom jinja2 functions/filters are:

  • {{ first(list_of_items) }} (First element in a list)

  • {{ last(list_of_items) }} (Last element in a list)

  • {{ unquote('"unescape-me"') }} (shell unquote / split, works on strings and lists)

  • {{ quote('escape-me') }} (shell quote, works on strings and lists)

  • {{ format_prompt(prompt_object) }} (Format and quote one or more prompt objects with their delimiter, works on single prompts and lists)

  • {{ gen_seeds(n) }} (Return a list of random integer seeds in the form of strings)

  • {{ cwd() }} (Return the current working directory as a string)

  • {{ download(url) }} (Download from a url to the web cache and return the file path)

The above functions which possess arguments can be used as either a function or filter IE: {{ "quote_me" | quote }}

The option --functions-help and the directive \functions_help can be used to printdocumentation for template functions. When the option or directive is used alone all builtin functions will be printed with their signature, specifying function names as argumentswill print documentation for those specific functions.

To receive information about Jinja2 template variables that are set after a dgenerate invocation.You can use the \templates_help directive which is similar to the --templates-help optionexcept it will print out all of the template variables assigned values instead of just theirnames and types. This is useful for figuring out the values of template variables set aftera dgenerate invocation in a config file for debugging purposes. You can specify one ormore template variable names as arguments to \templates_help to receive help for onlythe mentioned variable names.

Template variables set with the \set, \setp, and \sete directive willalso be mentioned in this output.

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1# Invocation will proceed as normalstabilityai/stable-diffusion-2-1 --prompts "a man walking on the moon without a space suit"# Print all set template variables\templates_help

The \templates_help output from the above example is:

Config template variables are: Name: "last_model_path" Type: typing.Optional[str] Value: stabilityai/stable-diffusion-2-1 Name: "last_subfolder" Type: typing.Optional[str] Value: None Name: "last_sdxl_refiner_uri" Type: typing.Optional[str] Value: None Name: "last_sdxl_refiner_edit" Type: typing.Optional[bool] Value: None Name: "last_batch_size" Type: typing.Optional[int] Value: 1 Name: "last_batch_grid_size" Type: typing.Optional[tuple[int, int]] Value: None Name: "last_prompts" Type: collections.abc.Sequence[dgenerate.prompt.Prompt] Value: ['a man walking on the moon without a space suit'] Name: "last_sdxl_second_prompts" Type: typing.Optional[collections.abc.Sequence[dgenerate.prompt.Prompt]] Value: [] Name: "last_sdxl_refiner_prompts" Type: typing.Optional[collections.abc.Sequence[dgenerate.prompt.Prompt]] Value: [] Name: "last_sdxl_refiner_second_prompts" Type: typing.Optional[collections.abc.Sequence[dgenerate.prompt.Prompt]] Value: [] Name: "last_seeds" Type: collections.abc.Sequence[int] Value: [98030306583037] Name: "last_seeds_to_images" Type: <class 'bool'> Value: False Name: "last_guidance_scales" Type: collections.abc.Sequence[float] Value: [5] Name: "last_inference_steps" Type: collections.abc.Sequence[int] Value: [30] Name: "last_clip_skips" Type: typing.Optional[collections.abc.Sequence[int]] Value: [] Name: "last_sdxl_refiner_clip_skips" Type: typing.Optional[collections.abc.Sequence[int]] Value: [] Name: "last_image_seeds" Type: typing.Optional[collections.abc.Sequence[str]] Value: [] Name: "last_parsed_image_seeds" Type: typing.Optional[collections.abc.Sequence[dgenerate.mediainput.ImageSeedParseResult]] Value: [] Name: "last_image_seed_strengths" Type: typing.Optional[collections.abc.Sequence[float]] Value: [] Name: "last_upscaler_noise_levels" Type: typing.Optional[collections.abc.Sequence[int]] Value: [] Name: "last_guidance_rescales" Type: typing.Optional[collections.abc.Sequence[float]] Value: [] Name: "last_image_guidance_scales" Type: typing.Optional[collections.abc.Sequence[float]] Value: [] Name: "last_s_cascade_decoder_uri" Type: typing.Optional[str] Value: None Name: "last_s_cascade_decoder_prompts" Type: typing.Optional[collections.abc.Sequence[dgenerate.prompt.Prompt]] Value: [] Name: "last_s_cascade_decoder_inference_steps" Type: typing.Optional[collections.abc.Sequence[int]] Value: [] Name: "last_s_cascade_decoder_guidance_scales" Type: typing.Optional[collections.abc.Sequence[float]] Value: [] Name: "last_sdxl_high_noise_fractions" Type: typing.Optional[collections.abc.Sequence[float]] Value: [] Name: "last_sdxl_refiner_inference_steps" Type: typing.Optional[collections.abc.Sequence[int]] Value: [] Name: "last_sdxl_refiner_guidance_scales" Type: typing.Optional[collections.abc.Sequence[float]] Value: [] Name: "last_sdxl_refiner_guidance_rescales" Type: typing.Optional[collections.abc.Sequence[float]] Value: [] Name: "last_sdxl_aesthetic_scores" Type: typing.Optional[collections.abc.Sequence[float]] Value: [] Name: "last_sdxl_original_sizes" Type: typing.Optional[collections.abc.Sequence[tuple[int, int]]] Value: [] Name: "last_sdxl_target_sizes" Type: typing.Optional[collections.abc.Sequence[tuple[int, int]]] Value: [] Name: "last_sdxl_crops_coords_top_left" Type: typing.Optional[collections.abc.Sequence[tuple[int, int]]] Value: [] Name: "last_sdxl_negative_aesthetic_scores" Type: typing.Optional[collections.abc.Sequence[float]] Value: [] Name: "last_sdxl_negative_original_sizes" Type: typing.Optional[collections.abc.Sequence[tuple[int, int]]] Value: [] Name: "last_sdxl_negative_target_sizes" Type: typing.Optional[collections.abc.Sequence[tuple[int, int]]] Value: [] Name: "last_sdxl_negative_crops_coords_top_left" Type: typing.Optional[collections.abc.Sequence[tuple[int, int]]] Value: [] Name: "last_sdxl_refiner_aesthetic_scores" Type: typing.Optional[collections.abc.Sequence[float]] Value: [] Name: "last_sdxl_refiner_original_sizes" Type: typing.Optional[collections.abc.Sequence[tuple[int, int]]] Value: [] Name: "last_sdxl_refiner_target_sizes" Type: typing.Optional[collections.abc.Sequence[tuple[int, int]]] Value: [] Name: "last_sdxl_refiner_crops_coords_top_left" Type: typing.Optional[collections.abc.Sequence[tuple[int, int]]] Value: [] Name: "last_sdxl_refiner_negative_aesthetic_scores" Type: typing.Optional[collections.abc.Sequence[float]] Value: [] Name: "last_sdxl_refiner_negative_original_sizes" Type: typing.Optional[collections.abc.Sequence[tuple[int, int]]] Value: [] Name: "last_sdxl_refiner_negative_target_sizes" Type: typing.Optional[collections.abc.Sequence[tuple[int, int]]] Value: [] Name: "last_sdxl_refiner_negative_crops_coords_top_left" Type: typing.Optional[collections.abc.Sequence[tuple[int, int]]] Value: [] Name: "last_unet_uri" Type: typing.Optional[str] Value: None Name: "last_second_unet_uri" Type: typing.Optional[str] Value: None Name: "last_vae_uri" Type: typing.Optional[str] Value: None Name: "last_vae_tiling" Type: <class 'bool'> Value: False Name: "last_vae_slicing" Type: <class 'bool'> Value: False Name: "last_lora_uris" Type: typing.Optional[collections.abc.Sequence[str]] Value: [] Name: "last_textual_inversion_uris" Type: typing.Optional[collections.abc.Sequence[str]] Value: [] Name: "last_control_net_uris" Type: typing.Optional[collections.abc.Sequence[str]] Value: [] Name: "last_scheduler" Type: typing.Optional[str] Value: None Name: "last_sdxl_refiner_scheduler" Type: typing.Optional[str] Value: None Name: "last_s_cascade_decoder_scheduler" Type: typing.Optional[str] Value: None Name: "last_safety_checker" Type: <class 'bool'> Value: False Name: "last_model_type" Type: <enum 'ModelType'> Value: ModelType.TORCH Name: "last_device" Type: <class 'str'> Value: cuda Name: "last_dtype" Type: <enum 'DataType'> Value: DataType.AUTO Name: "last_revision" Type: <class 'str'> Value: main Name: "last_variant" Type: typing.Optional[str] Value: None Name: "last_output_size" Type: typing.Optional[tuple[int, int]] Value: (512, 512) Name: "last_no_aspect" Type: <class 'bool'> Value: False Name: "last_output_path" Type: <class 'str'> Value: output Name: "last_output_prefix" Type: typing.Optional[str] Value: None Name: "last_output_overwrite" Type: <class 'bool'> Value: False Name: "last_output_configs" Type: <class 'bool'> Value: False Name: "last_output_metadata" Type: <class 'bool'> Value: False Name: "last_animation_format" Type: <class 'str'> Value: mp4 Name: "last_image_format" Type: <class 'str'> Value: png Name: "last_no_frames" Type: <class 'bool'> Value: False Name: "last_frame_start" Type: <class 'int'> Value: 0 Name: "last_frame_end" Type: typing.Optional[int] Value: None Name: "last_auth_token" Type: typing.Optional[str] Value: None Name: "last_seed_image_processors" Type: typing.Optional[collections.abc.Sequence[str]] Value: [] Name: "last_mask_image_processors" Type: typing.Optional[collections.abc.Sequence[str]] Value: [] Name: "last_control_image_processors" Type: typing.Optional[collections.abc.Sequence[str]] Value: [] Name: "last_post_processors" Type: typing.Optional[collections.abc.Sequence[str]] Value: [] Name: "last_offline_mode" Type: <class 'bool'> Value: False Name: "last_model_cpu_offload" Type: <class 'bool'> Value: False Name: "last_model_sequential_offload" Type: <class 'bool'> Value: False Name: "last_sdxl_refiner_cpu_offload" Type: typing.Optional[bool] Value: None Name: "last_sdxl_refiner_sequential_offload" Type: typing.Optional[bool] Value: None Name: "last_s_cascade_decoder_cpu_offload" Type: typing.Optional[bool] Value: None Name: "last_s_cascade_decoder_sequential_offload" Type: typing.Optional[bool] Value: None Name: "last_plugin_module_paths" Type: collections.abc.Sequence[str] Value: [] Name: "last_verbose" Type: <class 'bool'> Value: False Name: "last_cache_memory_constraints" Type: typing.Optional[collections.abc.Sequence[str]] Value: [] Name: "last_pipeline_cache_memory_constraints" Type: typing.Optional[collections.abc.Sequence[str]] Value: [] Name: "last_unet_cache_memory_constraints" Type: typing.Optional[collections.abc.Sequence[str]] Value: [] Name: "last_vae_cache_memory_constraints" Type: typing.Optional[collections.abc.Sequence[str]] Value: [] Name: "last_control_net_cache_memory_constraints" Type: typing.Optional[collections.abc.Sequence[str]] Value: [] Name: "last_images" Type: collections.abc.Iterable[str] Value: <dgenerate.renderloop.RenderLoop.written_images.<locals>.Iterable object at ...> Name: "last_animations" Type: collections.abc.Iterable[str] Value: <dgenerate.renderloop.RenderLoop.written_animations.<locals>.Iterable object at ...> Name: "injected_args" Type: collections.abc.Sequence[str] Value: [] Name: "injected_device" Type: typing.Optional[str] Value: None Name: "injected_verbose" Type: typing.Optional[bool] Value: False Name: "injected_plugin_modules" Type: typing.Optional[collections.abc.Sequence[str]] Value: [] Name: "saved_modules" Type: dict[str, dict[str, typing.Any]] Value: {} Name: "glob" Type: <class 'module'> Value: <module 'glob'> Name: "path" Type: <class 'module'> Value: <module 'ntpath' (frozen)>

The following is output from \functions_help showing every implemented template function signature.

Available config template functions: abs(args, kwargs) all(args, kwargs) any(args, kwargs) ascii(args, kwargs) bin(args, kwargs) bool(args, kwargs) bytearray(args, kwargs) bytes(args, kwargs) callable(args, kwargs) chr(args, kwargs) complex(args, kwargs) cwd() -> str dict(args, kwargs) divmod(args, kwargs) download(url: str, output: str | None = None, overwrite: bool = False, text: bool = False) -> str enumerate(args, kwargs) filter(args, kwargs) first(iterable: collections.abc.Iterable[typing.Any]) -> typing.Any float(args, kwargs) format(args, kwargs) format_prompt(prompts: dgenerate.prompt.Prompt | collections.abc.Iterable[dgenerate.prompt.Prompt]) -> str format_size(size: collections.abc.Iterable[int]) -> str frozenset(args, kwargs) gen_seeds(n: int) -> list[str] getattr(args, kwargs) hasattr(args, kwargs) hash(args, kwargs) hex(args, kwargs) int(args, kwargs) iter(args, kwargs) last(iterable: list | collections.abc.Iterable[typing.Any]) -> typing.Any len(args, kwargs) list(args, kwargs) map(args, kwargs) max(args, kwargs) min(args, kwargs) next(args, kwargs) object(args, kwargs) oct(args, kwargs) ord(args, kwargs) pow(args, kwargs) quote(strings: str | collections.abc.Iterable[typing.Any]) -> str range(args, kwargs) repr(args, kwargs) reversed(args, kwargs) round(args, kwargs) set(args, kwargs) slice(args, kwargs) sorted(args, kwargs) str(args, kwargs) sum(args, kwargs) tuple(args, kwargs) type(args, kwargs) unquote(strings: str | collections.abc.Iterable[typing.Any], expand: bool = False) -> list zip(args, kwargs)

Directives, and applying templating

You can see all available config directives with the commanddgenerate --directives-help, providing this option with a name, or multiplenames such as: dgenerate --directives-help save_modules use_modules will printthe documentation for the specified directives. The backslash may be omitted.This option is also available as the config directive \directives_help.

Example output:

Available config directives: "\help" "\templates_help" "\directives_help" "\functions_help" "\image_processor_help" "\clear_model_cache" "\clear_pipeline_cache" "\clear_unet_cache" "\clear_vae_cache" "\clear_control_net_cache" "\save_modules" "\use_modules" "\clear_modules" "\gen_seeds" "\download" "\pwd" "\ls" "\cd" "\pushd" "\popd" "\exec" "\mv" "\cp" "\mkdir" "\rmdir" "\rm" "\exit" "\image_process" "\import_plugins" "\set" "\sete" "\setp" "\unset" "\print" "\echo"

Here are examples of other available directives such as \set, \setp, and\print as well as some basic Jinja2 templating usage. This example also coversthe usage and purpose of \save_modules for saving and reusing pipeline modulessuch as VAEs etc. outside of relying on the caching system.

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1# You can define your own template variables with the \set directive# the \set directive does not do any shell args parsing on its value# operand, meaning the quotes will be in the string that is assigned# to the variable my_prompt\set my_prompt "an astronaut riding a horse; bad quality"# If your variable is long you can use continuation, note that# continuation replaces newlines and surrounding whitespace# with a single space\set my_prompt "my very very very very very very very \ very very very very very very very very \ long long long long long prompt"# You can print to the console with templating using the \print directive# for debugging purposes\print {{ my_prompt }}# The \setp directive can be used to define python literal template variables\setp my_list [1, 2, 3, 4]\print {{ my_list | join(' ') }}# Literals defined by \setp can reference other template variables by name.# the following creates a nested list\setp my_list [1, 2, my_list, 4]\print {{ my_list }}# \setp can evaluate template functions\setp directory_content glob.glob('*')\setp current_directory cwd()# the \gen_seeds directive can be used to store a list of# random seed integers into a template variable.# (they are strings for convenience)\gen_seeds my_seeds 10\print {{ my_seeds | join(' ') }}# An invocation sets various template variables related to its# execution once it is finished runningstabilityai/stable-diffusion-2-1 --prompts {{ my_prompt }} --gen-seeds 5# Print a quoted filename of the last image produced by the last invocation# This could potentially be passed to --image-seeds of the next invocation# If you wanted to run another pass over the last image that was produced\print {{ quote(last(last_images)) }}# you can also get the first image easily with the function "first"\print {{ quote(first(last_images)) }}# if you want to append a mask image file name\print "{{ last(last_images) }};my-mask.png"# Print a list of properly quoted filenames produced by the last# invocation separated by spaces if there is multiple, this could# also be passed to --image-seeds# in the case that you have generated an animated output with frame# output enabled, this will contain paths to the frames\print {{ quote(last_images) }}# For loops are possible\print {% for image in last_images %}{{ quote(image) }} {% endfor %}# For loops are possible with normal continuation# when not using a heredoc template continuation (mentioned below),# such as when the loop occurs in the body of a directive or a# dgenerate invocation, however this sort of continuation usage will# replace newlines and whitespace with a single space.# IE this template will be: "{% for image in last_images %} {{ quote(image) }} {% endfor %}"\print {% for image in last_images %} \ {{ quote(image) }} \ {% endfor %}# Access to the last prompt is available in a parsed representation# via "last_prompt", which can be formatted properly for reuse# by using the function "format_prompt"stabilityai/stable-diffusion-2-1 --prompts {{ format_prompt(last(last_prompts)) }}# You can get only the positive or negative part if you want via the "positive"# and "negative" properties on a prompt object, these attributes are not# quoted so you need to quote them one way or another, preferably using the# dgenerate template function "quote" which will shell quote any special# characters that the argument parser is not going to understandstabilityai/stable-diffusion-2-1 --prompts {{ quote(last(last_prompts).positive) }}# "last_prompts" returns all the prompts used in the last invocation as a list# the "format_prompt" function can also work on a liststabilityai/stable-diffusion-2-1 --prompts "prompt 1" "prompt 2" "prompt 3"stabilityai/stable-diffusion-2-1 --prompts {{ format_prompt(last_prompts) }}# Execute additional config with full templating.# The sequence !END is interpreted as the end of a# template continuation, a template continuation is# started when a line begins with the character {# and is effectively a heredoc, in that all whitespace# within is preserved including newlines{% for image in last_images %} stabilityai/stable-diffusion-2-1 --image-seeds {{ quote(image) }} --prompts {{ my_prompt }}{% endfor %} !END# Multiple lines can be used with a template continuation# the inside of the template will be expanded to raw config# and then be ran, so make sure to use line continuations within# where they are necessary as you would do in the top level of# a config file. The whole of the template continuation is# processed by Jinja, from { to !END, so only one !END is# ever necessary after the external template# when dealing with nested templates{% for image in last_images %} stabilityai/stable-diffusion-2-1 --image-seeds {{ quote(image) }} --prompts {{ my_prompt }}{% endfor %} !END# The above are both basically equivalent to thisstabilityai/stable-diffusion-2-1 --image-seeds {{ quote(last_images) }} --prompts {{ my_prompt }}# You can save modules from the main pipeline used in the last invocation# for later reuse using the \save_modules directive, the first argument# is a variable name and the rest of the arguments are diffusers pipeline# module names to save to the variable name, this is an advanced usage# and requires some understanding of the diffusers library to utilize correctlystabilityai/stable-diffusion-2-1--variant fp16--dtype float16--prompts "an astronaut walking on the moon"--safety-checker--output-size 512\save_modules stage_1_modules feature_extractor safety_checker# that saves the feature_extractor module object in the pipeline above,# you can specify multiple module names to save if desired# Possible Module Names:# unet# vae# text_encoder# text_encoder_2# tokenizer# tokenizer_2# safety_checker# feature_extractor# controlnet# scheduler# To use the saved modules in the next invocation use \use_modules\use_modules stage_1_modules# now the next invocation will use those modules instead of loading them from internal# in memory cache, disk, or huggingfacestabilityai/stable-diffusion-x4-upscaler--variant fp16--dtype float16--model-type torch-upscaler-x4--prompts {{ format_prompt(last_prompts) }}--image-seeds {{ quote(last_images) }}--vae-tiling# you should clear out the saved modules if you no longer need them# and your config file is going to continue, or if the dgenerate# process is going to be kept alive for some reason such as in# some library usage scenarios, or perhaps if you are using it# like a server that reads from stdin :)\clear_modules stage_1_modules

Setting template variables, in depth

The directives \set, \sete, and \setp can be used to set the valueof template variables within a configuration. The directive \unset can beused to undefine template variables.

All three of the assignment directives have unique behavior.

The \set directive sets a value with templating and environmental variable expansion applied to it,and nothing else aside from the value being striped of leading and trailing whitespace. The value that isset to the template variables is essentially the text that you supply as the value, as is. Or the text thatthe templates or environment variables in the value expand to, unmodified or parsed in any way.

This is for assigning literal text values to a template variable.

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1\set my_variable "I am an incomplete string and this is completely fine because I am a raw value# prints exactly what is above, including the quote at the beginning\print {{ my_variable }}# add a quote to the end of the string using templates\set my_variable {{ my_variable }}"# prints a fully quoted string\print {{ my_variable }}

The \sete directive can be used to assign the result of shell parsing and expansion to atemplate variable, the value provided will be shell parsed into tokens as if it were a line ofdgenerate config. This is useful because you can use the config languages built in shell globbingfeature to assign template variables.

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1# lets pretend the directory "my_files" is full of files\sete my_variable --argument my_files/*# prints the python array ['--argument', 'my_files/file1', 'my_files/file2', ...]\print {{ my_variable }}# Templates and environmental variable references# are also parsed in the \sete directive, just as they are with \set\set directory my_files\sete my_variable --argument {{ directory }}/*

The \setp directive can be used to assign the result of evaluating a limited subset of pythonexpressions to a template variable. This can be used to set a template variable to the resultof a mathematical expression, python literal value such as a list, dictionary, set, etc…python comprehension, or python ternary statement. In addition, all template functionsimplemented by dgenerate are available for use in the evaluated expressions.

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1\setp my_variable 10*10# prints 100\print {{ my_variable }}# you can reference variables defined in the environment\setp my_variable [my_variable, my_variable*2]# prints [100, 200]\print {{ my_variable }}# all forms of python comprehensions are supported# such as list, dict, and set comprehensions\setp my_variable [i for i in range(0,5)]# prints [0, 1, 2, 3, 4]\print {{ my_variable }}# declare a literal string value\setp my_variable "my string value"# prints the string without quotes included, the string was parsed\print {{ my_variable }}# templates and environmental variable references# are also expanded in \setp values\setp my_variable [my_variable, "{{ my_variable }}"]# prints ["my string value", "my string value"]\print {{ my_variable }}# my_variable is a literal list so it can be# looped over with a jinja template continuation{% for value in my_variable %} \print {{ value }}{% endfor %} !END

Globbing and path manipulation

The entirety of pythons builtin glob and os.path module are also accessible during templating, youcan glob directories using functions from the glob module, you can also glob directory’s using shellglobbing.

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1# globbing can be preformed via shell expansion or using# the glob module inside jinja templates# note that shell globbing and home directory expansion# does not occur inside quoted strings# \echo can be use to show the results of globbing that# occurs during shell expansion. \print does not preform shell# expansion nor does \set or \setp, all other directives do, as well# as dgenerate invocations# shell globs which produce 0 files are considered an error\echo ../media/*.png\echo ~# \sete can be used to set a template variable to the result# of one or more shell globs\sete myfiles ../media/*.png# with Jinja2:# The most basic usage is full expansion of every file\set myfiles {{ quote(glob.glob('../media/*.png')) }}\print {{ myfiles }}# If you have a LOT of files, you may want to# process them using an iterator like so{% for file in glob.iglob('../media/*.png') %} \print {{ quote(file) }}{% endfor %} !END# usage of os.path via path\print {{ path.abspath('.') }}# Simple inline usagestabilityai/stable-diffusion-2-1--variant fp16--dtype float16--prompts "In the style of picaso"--image-seeds {{ quote(glob.glob('../media/*.png')) }}--output-path {{ quote(path.join(path.abspath('.'), 'output')) }}# equivalentstabilityai/stable-diffusion-2-1--variant fp16--dtype float16--prompts "In the style of picaso"--image-seeds ../media/*.png--output-path ./output

The \print and \echo directive

The \print and \echo directive can both be used to output text to the console.

The difference between the two directives is that \print only ever printsthe raw value with templating and environmental variable expansion applied,similar to the behavior of \set

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1# the text after \print(space) will be printed verbatim\print I am a raw value, I have no ability to * glob# Print the PATH environmental variable\set header Path Elements:\print {{ header }} $PATH\print {{ header }} ${PATH}\print {{ header }} %PATH%

The \echo directive preforms shell expansion into tokens before printing, like \sete,This can be useful for debugging / displaying the results of a shell expansion.

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1# lets pretend "directory" is full of files# this prints: directory/file1 directory/file2 ...\echo directory/*# Templates and environmental variables are expanded# this prints: Files: directory/file1 directory/file2 ...\set header Files:\echo {{ header }} directory/*

The \image_process directive

The dgenerate sub-command image-process has a config directive implementation.

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1# print the help message of --sub-command image-process, this does# not cause the config to exit\image_process --help\set myfiles {{ quote(glob.glob('my_images/*.png')) }}# this will create the directory "upscaled"# the files will be named "upscaled/FILENAME_processed_1.png" "upscaled/FILENAME_processed_2.png" ...\image_process {{ myfiles }} \--output upscaled/--processors upscaler;model=https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesr-general-x4v3.pth# the last_images template variable will be set, last_animations is also usable if# animations were written. In the case that you have generated an animated output with frame# output enabled, this will contain paths to the frames\print {{ quote(last_images) }}

The \exec directive

The \exec directive can be used to run native system commands and supports bashpipe and file redirection syntax in a platform independent manner. All fileredirection operators supported by bash are supported. This can be usefulfor running other image processing utilities as subprocesses from within aconfig script.

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1# run dgenerate as a subprocess, read a config# and send stdout and stderr to a file\exec dgenerate < my_config.dgen &> log.txt# chaining processes together with pipes is supported# this example emulates 'cat' on Windows using cmd\exec cmd /c "type my_config.dgen" | dgenerate &> log.txt# on a Unix platform you could simply use cat\exec cat my_config.dgen | dgenerate &> log.txt

The \download directive

Arbitrary files can be downloaded via the \download directive.

This directive can be used to download a file and assign itsdownloaded path to a template variable.

Files can either be inserted into dgenerates web cache ordownloaded to a specific directory or absolute path.

This directive is designed with using cached files in mind,so it will reuse existing files by default when downloadingto an explicit path.

See the directives help output for more details: \download --help

If you plan to download many large models to the web cache inthis manner you may wish to adjust the global cache expiry timeso that they exist in the cache longer than the default of 12 hours.

You can see how to do this in the section File Cache Control

This directive is primarily intended to download models and or otherbinary file formats such as images and will raise an error if it encountersa text mimetype. This behavior can be overridden with the -t/--text argument.

Be weary that if you have a long-running loop in your config usinga top level jinja template, which refers to your template variable,cache expiry may invalidate the file stored in your variable.

You can rectify this by putting the download directive inside ofyour processing loop so that the file is simply re-downloaded ifit expires in the cache.

Or you may be better off using the downloadtemplate function which provides this functionalityas a template function. See: The download() template function

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1# download a model into the web cache,# assign its path to the variable "path"\download path https://modelhost.com/somemodel.safetensors# download to the models folder in the current directory# the models folder will be created if it does not exist# if somemodel.safetensors already exists it will be reused# instead of being downloaded again\download path https://modelhost.com/somemodel.safetensors -o models/somemodel.safetensors# download into the folder without specifying a name# the name will be derived from the URL or content disposition# header from the http request, if you are not careful you may# end up with a file named in a way you were not expecting.# only use this if you know how the service you are downloading# from behaves in this regard\download path https://modelhost.com/somemodel.safetensors -o models/# download a model into the web cache an overwrite any cached model using -x\download path https://modelhost.com/somemodel.safetensors -x# Download to an explicit path without any cached file reuse# using the -x/--overwrite argument. In effect, always freshly# download the file\download path https://modelhost.com/somemodel.safetensors -o models/somemodel.safetensors -x\download path https://modelhost.com/somemodel.safetensors -o models/ -x

The download() template function

The template function download is analogous to the \download directive

And can be used to download a file with the same behaviour and return itspath as a string, this may be easier to use inside of certain jinja flowcontrol constructs.

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1\set my_variable {{ download('https://modelhost.com/model.safetensors') }}\set my_variable {{ download('https://modelhost.com/model.safetensors', output='model.safetensors') }}\set my_variable {{ download('https://modelhost.com/model.safetensors', output='directory/') }}# you can also use any template function with \setp (python expression evaluation)\setp my_variable download('https://modelhost.com/model.safetensors')

The signature for this template function is: download(url: str, output: str | None = None, overwrite: bool = False, text: bool = False) -> str

The \exit directive

You can exit a config early if need be using the \exit directive

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1# exit the process with return code 0, which indicates success\print "success"\exit

An explicit return code can be provided as well

#! /usr/bin/env dgenerate --file#! dgenerate 3.7.1# exit the process with return code 1, which indicates an error\print "some error occurred"\exit 1

Running configs from the command line

To utilize configuration files use the --file option,or pipe them into the command, or use file redirection:

Use the --file option

dgenerate --file my-config.dgen

Piping or redirection in Bash:

# Pipecat my-config.dgen | dgenerate# Redirectiondgenerate < my-config.dgen

Redirection in Windows CMD:

dgenerate < my-config.dgen

Piping Windows Powershell:

Get-Content my-config.dgen | dgenerate

Config argument injection

You can inject arguments into every dgenerate invocation of a batch processingconfiguration by simply specifying them. The arguments will added to the endof the argument specification of every call.

# Pipecat my-animations-config.dgen | dgenerate --frame-start 0 --frame-end 10# Redirectiondgenerate --frame-start 0 --frame-end 10 < my-animations-config.dgen

On Windows CMD:

dgenerate --frame-start 0 --frame-end 10 < my-animations-config.dgen

On Windows Powershell:

Get-Content my-animations-config.dgen | dgenerate --frame-start 0 --frame-end 10

If you need arguments injected from the command line within the config forsome other purpose such as for using with the \image_process directivewhich does not automatically recieve injected arguments, use theinjected_args and related injected_* template variables.

# all injected args\print {{ quote(injected_args) }}# just the injected device\print {{ '--device '+injected_device if injected_device else '' }}# was -v/--verbose injected?\print {{ '-v' if injected_verbose else '' }}# plugin module paths injected with --plugin-modules\print {{ quote(injected_plugin_modules) if injected_plugin_modules else '' }}
Overview — dgenerate 3.7.1 documentation (2)

You can launch a cross platform Tkinter GUI for interacting with alive dgenerate process using dgenerate --console or via the optionallyinstalled desktop shortcut on Windows.

This provides a basic REPL for the dgenerate config language utilizinga dgenerate --shell subprocess to act as the live interpreter, italso features full context aware syntax highlighting for the dgenerateconfig language.

It can be used to work with dgenerate without encountering the startupoverhead of loading large python modules for every command line invocation.

The GUI console supports command history via the up and down arrow keys as anormal terminal would, optional multiline input for sending multiline commands / configurationto the shell. And various editing niceties such as GUI file / directory path insertion,the ability to insert templated command recipes for quickly getting started and getting results,and a selection menu for inserting karras schedulers by name.

Also supported is the ability to view the latest image as it is produced by dgenerate or\image_process via an image pane or standalone window.

The console UI always starts in single line entry mode (terminal mode), multiline input modeis activated via the insert key and indicated by the presence of line numbers, you must deactivate this modeto submit commands via the enter key, however you can use the run button from the run menu (or Ctrl+Space)to run code in this mode. You cannot page through command history in this mode, and code will remain in theconsole input pane upon running it making the UI function more like a code editor than a terminal.

The console can be opened with a file loaded in multiline input modeby using the command: dgenerate --console filename.dgen

Ctrl+Q can be used in input pane for killing and then restarting the background interpreter process.

Ctrl+F (find) and Ctrl+R (find/replace) is supported for both the input and output panes.

All common text editing features that you would expect to find in a basic text editor are present,as well as python regex support for find / replace, with group substitution supporting the syntax\n or \{n} where n is the match group number.

Scroll back history in the output window is currently limited to 10000 lines however the consoleapp itself echos all stdout and stderr of the interpreter, so you can save all output to a logfile via file redirection if desired when launching the console from the terminal.

This can be configured by setting the environmental variable DGENERATE_CONSOLE_MAX_SCROLLBACK=10000

Command history is currently limited to 500 commands, multiline commands are alsosaved to command history. The command history file is stored at -/.dgenerate_console_history,on Windows this equates to %USERPROFILE%\.dgenerate_console_history

This can be configured by setting the environmental variable DGENERATE_CONSOLE_MAX_HISTORY=500

Any UI settings that persist on startup are stored in -/.dgenerate_console_settings oron Windows %USERPROFILE%\.dgenerate_console_settings

dgenerate has the capability of loading in additional functionality through the use ofthe --plugin-modules option and \import_plugins config directive.

You simply specify one or more module directories on disk, paths to python files, or referencesto modules installed in the python environment using the argument or import directive.

dgenerate supports implementing image processors and config directives through plugins.

A code example as well as a usage example for image processor plugins can be foundin the “writing_plugins/image_processor”folder of the examples folder.

The source code for the built in canny processor,the openpose processor, and the simplepillow image operations processors can alsobe of reference as they are written as internal image processor plugins.

An example for writing config directives can be found in the“writing_plugins/config_directive” folderof the examples folder. Config template functions can also be implemented by plugins,see: “writing_plugins/template_function”

Currently the only internal directive that is implemented as a plugin is the \image_process directive,who’s source file can be located here,the source file for this directive is terse as most of \image_process is implemented as reusable code as mentioned below.

The behavior of \image_process which is also used for --sub-command image-process isis implemented here.

dgenerate will cache downloaded non hugging face models, downloaded --image-seeds files,files downloaded by the \download directive, download template function, and downloadedfiles used by image processors in the directory ~/.cache/dgenerate/web

On Windows this equates to: %USERPROFILE%\.cache\dgenerate\web

You can control where these files are cached with the environmental variable DGENERATE_WEB_CACHE.

Files are cleared from the web cache automatically after an expiry time upon running dgenerate orwhen downloading additional files, the default value is after 12 hours.

This can be controlled with the environmental variable DGENERATE_WEB_CACHE_EXPIRY_DELTA.

The value of DGENERATE_WEB_CACHE_EXPIRY_DELTA is that of the named arguments of pythonsdatetime.timedelta classseperated by semicolons.

For example: DGENERATE_WEB_CACHE_EXPIRY_DELTA="days=5;hours=6"

Specifying "forever" or an empty string will disable cache expiration for every downloaded file.

Files downloaded from huggingface by the diffusers/huggingface_hub library will be cached under~/.cache/huggingface/, on Windows this equates to %USERPROFILE%\.cache\huggingface\.

This is controlled by the environmental variable HF_HOME

In order to specify that all large model files be stored in another location,for example on another disk, simply set HF_HOME to a new path in your environment.

You can read more about environmental variables that affect huggingface libraries on thishuggingface documentation page.

Overview — dgenerate 3.7.1 documentation (2024)
Top Articles
Latest Posts
Article information

Author: Tyson Zemlak

Last Updated:

Views: 6320

Rating: 4.2 / 5 (43 voted)

Reviews: 90% of readers found this page helpful

Author information

Name: Tyson Zemlak

Birthday: 1992-03-17

Address: Apt. 662 96191 Quigley Dam, Kubview, MA 42013

Phone: +441678032891

Job: Community-Services Orchestrator

Hobby: Coffee roasting, Calligraphy, Metalworking, Fashion, Vehicle restoration, Shopping, Photography

Introduction: My name is Tyson Zemlak, I am a excited, light, sparkling, super, open, fair, magnificent person who loves writing and wants to share my knowledge and understanding with you.