Fine-tuned Image Model
SDXL / Replicate
September 2023
Representing the fusion of machine learning and creativity — I teamed up with Anne Milan to fine-tune the SDXL image model with a dataset of 21 digital illustrations from her "A Toast a Day" series. You can now enter short prompts and generate images of toast in Anne's distinctive style.
Background
Fine-tuning is the process of taking a pre-trained model (sometimes referred to as a foundation model) and training it with your own data to create a new model better suited to a specific task.
Replicate makes it easy to deploy machine learning models in the cloud using your own code.
SDXL is the latest iteration of Stable Diffusion, an image generation model developed by Stability AI.
This project was born out of curiosity, and Anne provided the perfect original dataset to work with. While we were positively surprised by the results, it left us pondering the implications for art.
dataset
setting up the training
the model identifying the content of each image
first output generated for the prompt "bread with pesto rosso"
generated results
yash-banka/moon
A few days later, we fine-tuned another model using 27 digital illustrations from Anne's "Month of the Moon" series. It was featured in Replicate's email newsletter.
