Content Based Image Synthesis


The idea for content based image synthesis is a natural extension of ideas in Content Based Image Retrieval (CBIR) and graphics. Using a database of annotated imagery, our application supports an image editor in recombining these annotated image pieces in novel ways. Annotations follow some tradition CBIR methods (e.g. hue, saturation, lightness), as well as much higher level semantic categories (e.g. sky, mountain, forest) so that the editor can quickly query for content that is relevant to his editing task. 


N. Diakopoulos, I. Essa, and R. Jain. Content Based Image Synthesis. Conference on Image and Video Retrieval (CIVR) 2004, Dublin, Ireland, July 2004. pp 299-307. [PDF]

Example Results

In each of the following blocks, the source texture is in the upper left, the input image is in the lower left, and the final output is shown right. 

A city skyline is synthesized above a park in Oslo.

A field of flowers is replaced with a field of rocks.

The cloudy sky over Florence is replaced with a sunset.

A distant island is inserted.

Van Gogh’s “Starry Night” is used as a source to replace the night sky over Atlanta.

A rocky mountain is replaced with a snowy one.

A cloudy sky is inserted behind a statue of Stalin.