Merge branch 'master' into master

    id: what-should

    attributes:

      label: What should have happened?

      description: tell what you think the normal behavior should be

      description: Tell what you think the normal behavior should be

    validations:

      required: true

  - type: input

    id: commit

    attributes:

      label: Commit where the problem happens

      description: Which commit are you running ? (Do not write *Latest version/repo/commit*, as this means nothing and will have changed by the time we read your issue. Rather, copy the **Commit hash** shown in the cmd/terminal when you launch the UI)

      description: Which commit are you running ? (Do not write *Latest version/repo/commit*, as this means nothing and will have changed by the time we read your issue. Rather, copy the **Commit** link at the bottom of the UI, or from the cmd/terminal if you can't launch it.)

    validations:

      required: true

  - type: dropdown

    id: platforms

    attributes:

      label: What platforms do you use to access UI ?

      label: What platforms do you use to access the UI ?

      multiple: true

      options:

        - Windows

    id: cmdargs

    attributes:

      label: Command Line Arguments

      description: Are you using any launching parameters/command line arguments (modified webui-user.py) ? If yes, please write them below

      description: Are you using any launching parameters/command line arguments (modified webui-user .bat/.sh) ? If yes, please write them below. Write "No" otherwise.

      render: Shell

    validations:

      required: true

  - type: textarea

    id: extensions

    attributes:

      label: List of extensions

      description: Are you using any extensions other than built-ins? If yes, provide a list, you can copy it at "Extensions" tab. Write "No" otherwise.

    validations:

      required: true

  - type: textarea

    id: logs

    attributes:

      label: Console logs

      description: Please provide **full** cmd/terminal logs from the moment you started UI to the end of it, after your bug happened. If it's very long, provide a link to pastebin or similar service.

      render: Shell

    validations:

      required: true

  - type: textarea

    id: misc

    attributes:

      label: Additional information, context and logs

      description: Please provide us with any relevant additional info, context or log output.

      label: Additional information

      description: Please provide us with any relevant additional info or context.

    - a man in a (tuxedo:1.21) - alternative syntax

    - select text and press ctrl+up or ctrl+down to automatically adjust attention to selected text (code contributed by anonymous user)

- Loopback, run img2img processing multiple times

- X/Y plot, a way to draw a 2 dimensional plot of images with different parameters

- X/Y/Z plot, a way to draw a 3 dimensional plot of images with different parameters

- Textual Inversion

    - have as many embeddings as you want and use any names you like for them

    - use multiple embeddings with different numbers of vectors per token

- Idea for Composable Diffusion - https://github.com/energy-based-model/Compositional-Visual-Generation-with-Composable-Diffusion-Models-PyTorch

- xformers - https://github.com/facebookresearch/xformers

- DeepDanbooru - interrogator for anime diffusers https://github.com/KichangKim/DeepDanbooru

- Sampling in float32 precision from a float16 UNet - marunine for the idea, Birch-san for the example Diffusers implementation (https://github.com/Birch-san/diffusers-play/tree/92feee6)

- Instruct pix2pix - Tim Brooks (star), Aleksander Holynski (star), Alexei A. Efros (no star) - https://github.com/timothybrooks/instruct-pix2pix

- Security advice - RyotaK

- Initial Gradio script - posted on 4chan by an Anonymous user. Thank you Anonymous user.

- (You)

# File modified by authors of InstructPix2Pix from original (https://github.com/CompVis/stable-diffusion).

# See more details in LICENSE.

model:

  base_learning_rate: 1.0e-04

  target: modules.models.diffusion.ddpm_edit.LatentDiffusion

  params:

    linear_start: 0.00085

    linear_end: 0.0120

    num_timesteps_cond: 1

    log_every_t: 200

    timesteps: 1000

    first_stage_key: edited

    cond_stage_key: edit

    # image_size: 64

    # image_size: 32

    image_size: 16

    channels: 4

    cond_stage_trainable: false   # Note: different from the one we trained before

    conditioning_key: hybrid

    monitor: val/loss_simple_ema

    scale_factor: 0.18215

    use_ema: true

    load_ema: true

    scheduler_config: # 10000 warmup steps

      target: ldm.lr_scheduler.LambdaLinearScheduler

      params:

        warm_up_steps: [ 0 ]

        cycle_lengths: [ 10000000000000 ] # incredibly large number to prevent corner cases

        f_start: [ 1.e-6 ]

        f_max: [ 1. ]

        f_min: [ 1. ]

    unet_config:

      target: ldm.modules.diffusionmodules.openaimodel.UNetModel

      params:

        image_size: 32 # unused

        in_channels: 8

        out_channels: 4

        model_channels: 320

        attention_resolutions: [ 4, 2, 1 ]

        num_res_blocks: 2

        channel_mult: [ 1, 2, 4, 4 ]

        num_heads: 8

        use_spatial_transformer: True

        transformer_depth: 1

        context_dim: 768

        use_checkpoint: True

        legacy: False

    first_stage_config:

      target: ldm.models.autoencoder.AutoencoderKL

      params:

        embed_dim: 4

        monitor: val/rec_loss

        ddconfig:

          double_z: true

          z_channels: 4

          resolution: 256

          in_channels: 3

          out_ch: 3

          ch: 128

          ch_mult:

          - 1

          - 2

          - 4

          - 4

          num_res_blocks: 2

          attn_resolutions: []

          dropout: 0.0

        lossconfig:

          target: torch.nn.Identity

    cond_stage_config:

      target: ldm.modules.encoders.modules.FrozenCLIPEmbedder

data:

  target: main.DataModuleFromConfig

  params:

    batch_size: 128

    num_workers: 1

    wrap: false

    validation:

      target: edit_dataset.EditDataset

      params:

        path: data/clip-filtered-dataset

        cache_dir:  data/

        cache_name: data_10k

        split: val

        min_text_sim: 0.2

        min_image_sim: 0.75

        min_direction_sim: 0.2

        max_samples_per_prompt: 1

        min_resize_res: 512

        max_resize_res: 512

        crop_res: 512

        output_as_edit: False

        real_input: True

model:

  base_learning_rate: 1.0e-4

  target: ldm.models.diffusion.ddpm.LatentDiffusion

  base_learning_rate: 7.5e-05

  target: ldm.models.diffusion.ddpm.LatentInpaintDiffusion

  params:

    parameterization: "v"

    linear_start: 0.00085

    linear_end: 0.0120

    num_timesteps_cond: 1

    cond_stage_key: "txt"

    image_size: 64

    channels: 4

    cond_stage_trainable: false

    conditioning_key: crossattn

    cond_stage_trainable: false   # Note: different from the one we trained before

    conditioning_key: hybrid   # important

    monitor: val/loss_simple_ema

    scale_factor: 0.18215

    use_ema: False # we set this to false because this is an inference only config

    finetune_keys: null

    scheduler_config: # 10000 warmup steps

      target: ldm.lr_scheduler.LambdaLinearScheduler

      params:

        warm_up_steps: [ 2500 ] # NOTE for resuming. use 10000 if starting from scratch

        cycle_lengths: [ 10000000000000 ] # incredibly large number to prevent corner cases

        f_start: [ 1.e-6 ]

        f_max: [ 1. ]

        f_min: [ 1. ]

    unet_config:

      target: ldm.modules.diffusionmodules.openaimodel.UNetModel

      params:

        use_checkpoint: True

        use_fp16: True

        image_size: 32 # unused

        in_channels: 4

        in_channels: 9  # 4 data + 4 downscaled image + 1 mask

        out_channels: 4

        model_channels: 320

        attention_resolutions: [ 4, 2, 1 ]

        num_res_blocks: 2

        channel_mult: [ 1, 2, 4, 4 ]

        num_head_channels: 64 # need to fix for flash-attn

        num_heads: 8

        use_spatial_transformer: True

        use_linear_in_transformer: True

        transformer_depth: 1

        context_dim: 1024

        context_dim: 768

        use_checkpoint: True

        legacy: False

    first_stage_config:

        embed_dim: 4

        monitor: val/rec_loss

        ddconfig:

          #attn_type: "vanilla-xformers"

          double_z: true

          z_channels: 4

          resolution: 256

          target: torch.nn.Identity

    cond_stage_config:

      target: ldm.modules.encoders.modules.FrozenOpenCLIPEmbedder

      params:

        freeze: True

        layer: "penultimate"

 No newline at end of file

      target: ldm.modules.encoders.modules.FrozenCLIPEmbedder

from modules import extra_networks

from modules import extra_networks, shared

import lora

class ExtraNetworkLora(extra_networks.ExtraNetwork):

        super().__init__('lora')

    def activate(self, p, params_list):

        additional = shared.opts.sd_lora

        if additional != "" and additional in lora.available_loras and len([x for x in params_list if x.items[0] == additional]) == 0:

            p.all_prompts = [x + f"<lora:{additional}:{shared.opts.extra_networks_default_multiplier}>" for x in p.all_prompts]

            params_list.append(extra_networks.ExtraNetworkParams(items=[additional, shared.opts.extra_networks_default_multiplier]))

        names = []

        multipliers = []

        for params in params_list: