NoobAI Notes

NoobAI Notes

Introduction

NoobAI is based on the SDXL architecture specifically tailored for anime and furry image generation. Currently I’m using the v-prediction 0.65S version of the model. This guide will focus mostly on that particular version. It was trained on Danbooru and e621.

Download

CivitAI

Training Tips

I’m currently on the sd3 branch so there is nothing special to passing:

--v_parameterization
--zero_terminal_snr

Fitting in 24Gb VRAM

I used the following settings to fit perfectly inside a 4090.

Click to expand content.
args=(
    --debiased_estimation_loss
    --max_token_length=225
    --keep_tokens=1
    --keep_tokens_separator="|||"
    --pretrained_model_name_or_path=/home/kade/ComfyUI/models/checkpoints/noobaiXLVpredv06.safetensors
    --v_parameterization
    --zero_terminal_snr
    --log_with=tensorboard
    --seed=1728871242
    --dataset_repeats=1
    --resolution="1024,1024"
    --enable_bucket
    --bucket_reso_steps=64
    --min_bucket_reso=256
    --max_bucket_reso=2048
    --flip_aug
    --shuffle_caption
    --cache_latents
    --cache_latents_to_disk
    --max_data_loader_n_workers=8
    --persistent_data_loader_workers
    --network_dim=100000
    --network_alpha=64
    --network_module="lycoris.kohya"
    --network_args
            "preset=full"
            "conv_dim=100000"
            "decompose_both=False"
            "conv_alpha=64"
            "rank_dropout=0"
            "module_dropout=0"
            "use_tucker=True"
            "use_scalar=False"
            "rank_dropout_scale=False"
            "algo=lokr"
            "bypass_mode=False"
            "factor=16"
            "dora_wd=True"
            "train_norm=False"
    --network_dropout=0
    --optimizer_type=ClybW
    --train_batch_size=14
    --max_grad_norm=1
    --gradient_checkpointing
    --lr_warmup_steps=0
    --learning_rate=0.0003
    --unet_lr=0.0003
    --text_encoder_lr=0.00015
    --lr_scheduler="cosine"
    --lr_scheduler_args="num_cycles=0.375"
    --multires_noise_iterations=12
    --multires_noise_discount=0.4
    --no_half_vae
    --sdpa
    --mixed_precision="bf16"
    --save_model_as="safetensors"
    --save_precision="fp16"
    --save_every_n_steps=100
    --sample_every_n_steps=100
    --sample_sampler="euler_a"
    --sample_at_first
    --caption_extension=".txt"
)

You can reduce the batch size and/or resolution to 768x or 512x to fit any GPU. If you still struggle, use an 8-bit optimizer. Here is the optimizer I prefer currently, there is a guide on how to integrate custom optimizers to sd-scripts here.

Prompting Tips

Date Tags

There are two types of date tags: year tags and period tags.

Year Tags

Use year xxxx format. Example: 2021.

Period Tags

For period tags, please refer to the following table:

Year RangePeriod Tag
2005-2010old
2011-2014early
2014-2017mid
2018-2020recent
2021-2024newest

Related Posts

Yiff Toolkit

A comprehensive set of tools and guides for creating furry artwork using AI. With a detailed guide on LoRA training, this toolkit provides everything you need to create your own synthetic art.

LoRA Training Guide

The LoRA Training Guide explains Low-Rank Adaptation (LoRA), a technique for fine-tuning large language and diffusion models efficiently by introducing small, trainable low-rank matrices instead of modifying all model parameters. This approach keeps the original model weights frozen and injects two additional matrices into each layer to learn necessary adjustments. LoRA is lightweight, making it feasible to train multiple adaptations without hefty storage requirements. The guide also compares LoRA with LyCORIS, an advanced extension that offers more control and flexibility, and introduces LoKr, which uses Kronecker products for matrix decomposition, enhancing memory efficiency and control over the adaptation process.

🐍 FurryTagger

Loads `eva02-vit-large-448-8046`, applies it to a set of images in a specified directory, and write the model’s output tags to a text file for each image.