stable diffusion webui 搭建和初步使用

官方repo: GitHub - AUTOMATIC1111/stable-diffusion-webui: Stable Diffusion web UI

关于stable-diffusion的介绍：Stable Diffusion｜图解稳定扩散原理 - 知乎

一、环境搭建和启动

准备在容器里面搞一下

以 ubuntu22.04 为基础镜像，新建/code, /dataset目录，添加非root用户auser并添加到sudo（webui.sh要以非root用户执行）

mkdir /sd
chmod -R 777 /sd
useradd auser
passwd auser
usermod -aG sudo auser
mkdir -p /home/auser
chmod -R 777 /home
# 安装sudo
apt-get update
apt-get install sudo

commit 上述container为一个新镜像

用以上述新镜像起container，把webui.sh， weibui-user.sh下载到/sd挂在的地方并给与+x权限，进入docker后：

sudo apt update
sudo apt install libgoogle-perftools4 libtcmalloc-minimal4
sudo apt install wget git python3 python3-venv libgl1 libglib2.0-0
sudo apt install bc

# 添加国内pip源（比如中科大源）
mkdir -p ~/.pip
echo "[global]" > ~/.pip/pip.conf
echo "index-url = https://pypi.mirrors.ustc.edu.cn/simple/" >> ~/.pip/pip.conf

# automatic intall
./webui.sh

# 安装xformer，安装之前需要看下 torch 的版本，2.1.2对应：
pip install xformers==0.0.23.post1

# Can‘t load tokenizer for ‘openai/clip-vit-large-patch14‘ 参考:
# https://cloud.baidu.com/qianfandev/topic/268119

启动：

# docker内启动
cd /sd/stable-diffusion-webui
source ./venv/bin/activate
python webui.py --xformers

二、初步体验

这次用了sd_xl_base_1.0.safetensors，使用下来觉得小场景不错，大场景一般，用LoRA效果很好，但是AI还是不能画Ai自己（prompt 2）

1024x1024的分辨率显存占用大概11.5GB。

很多模型可去hugging face上搜，text-to-image 标签，国内也有 hf 的镜像 ByteDance/SDXL-Lightning at main

1. prompt: In a sunny morning, at a busy road cross, peoples are waiting for the green light. a car is breaking to avoid impacting a cyclist.

2. prompt: A diagram of a ViT commonly used in computer vision.

3. prompt: A busy 21-century candy factory. Trucks are waiting at the gate. It's going to raining, so lots of workers are try their best to cover the raw materials. The boss of the factory is on a meeting with some important guests behind the office window. There're also some students visitors.

4. 是否使用LoRA对比, 使用LoRA参考：Yangdf/mini-mecha-sdxl-lora · HF Mirror

prompt: A dynamic scene of two mecha battle in the air, one is flying and shooting blue energy beam to another which standing on building roof, unreal engine 5 with epic cinematography, in an industrial snow covered planet with green grass

with lora prompt: mini-mecha-sdxl-lora_emb,lora:mini-mecha-sdxl-lora:1>, A dynamic scene of two mecha battle in the air, one is flying and shooting blue energy beam to another which standing on building roof, unreal engine 5 with epic cinematography, in an industrial snow covered planet with green grass