var(AI)tions on Lorville

Testing the waters with the new Community Hub interface. I recently posted these on reddit but maybe they will have a better place here.

So.. Before anyone else decides to accuse me of using prompts to get my results. Spoiler alert. There are none! Here's my current process: https://i.imgur.com/W1Tzr9q.jpg

In more detail:

1. I took very specific screenshots with very specific details from Lorville, keeping in mind the current training limitations that I got to know by lots of testing in the last 1-2 weeks. It's just like photography with extra steps or photogrammetry with a lot less steps. Somewhere in between but it's more about finding the right lines, shapes, values, atmosphere, colours and CONTEXT for the algorithm to pick up.

2. I filter the results down to a minimal dataset of 20-50 images that perfectly fill all the requirements and contain the elements the model needs to learn in order to fulfil my vision. There are basically infinite things the model could capture, and I want consistency. The data needs to be as focused as possible for its intended scope.

3. Generalizing the vision. Now I take the sample data and figure out a way to create what I call a skeleton word. A basic token or class that the encoder can associate with my sample data. This will be used to generate at least a 1000 of "random" images. For example, if I want to train a specific person, I need images lots of random persons. This serves as regularization data and pulls the sample data into the right weight class. The inherit nature of these classes could be inherited by the limited training data making it more flexible to work with. For Lorville I mostly used brutalist interiors and industrial machines in the desert.

4. I rent a server with a strong GPU with lots and lots and lots of VRAM. Store up all the thousands of images and other gathered data on github for easy access on the server. I pull the Open-Source tools (Dreambooth) and configure them. Do some debuging (if necessarry) in Python and adding my own usual snippets for easier workflow. Start the training and wait for an hour. Download the finished model.

5. I'm running the model locally (NMKD stable diffusion) with almost no prompts at all because the model is strong as hell (that's what I want) and prompts are like a fart in a tornado at this point. For getting controlled results I use input images bashed together in photoshop. Due to the nature of the reg images only a few shapes needed to get precisely controlled results. I also like to merge models and apply different filters from textual inversion methods to get larger or more limited scope on the original samples.

That's it. "No" prompts used in the process. (only the basic calls for the trained data)

Thats all. If you as a fellow citizen or tech enthusiast want to try something similar feel free to reach out and I can point you to the right sources to get you up and running. The tech is progressing rapidly and in my humble opinion as a Technical Artist for the last 5 years or so: With this method it's production ready. No joke. It's all very controlled and consistent. Some of these images contain more than enough to push some background stuff into graybox or further.

The current prerequisites:

Minimal Python knowledge (mainly package management) Windows/Linux CLI basics Batch processing concepts in general Lots of free time (alternatively: sleepdeprivation.exe)

Thanks for being here, Dave_Spareblade out!