Overpainting: Trimap-Controlled Localized Image Editing via Finetuned FLUX.1 Kontext

Researchers have introduced "overpainting," a diffusion-based image editing method that gives users precise spatial control over where edits occur while keeping the model aware of the original image content. The system uses a trimap annotation scheme — white pixels must be edited, black pixels must not, and gray pixels may be changed — enabling both tight and loose editing constraints depending on the task. The model is implemented by finetuning FLUX.1 Kontext with a "Teamwork LoRA" combined with joint attention and attention dropout to balance information flow between the noise, source image, and trimap mask.

To train the model, the researchers built an automated data generation pipeline that samples freely licensed photographs, uses Qwen2.5-VL to suggest edit prompts, runs FLUX.1 Kontext to produce edited versions, filters out poor results, and extracts human-like trimaps from usable pairs using an adapted BiRefNet model. The method is demonstrated across a wide range of localized editing tasks, including fine-grained changes like adding whiskers or wrapping objects in twine — cases where conventional binary masks are cumbersome or ill-defined.