‘Quantum Mirror’ Sony camera app reimagines photos with DALL-E 2

Photographer Nicholas Sherlock created an app for Sony cameras that submits any photos taken directly to OpenAI’s DALL-E 2, which he says allows users to “peek into parallel universes” and see reimagined versions of those scenes .

The app, called Quantum Mirror, and published on Github, is designed for Android-based Sony cameras, including older models like the Alpha 7 and Alpha 7R II, but not the Alpha 7 III or newer, and will shoot The photos are submitted to the DALL-E2 artificial intelligence system, which then uses the photos to generate their variants.

“Because the app allows you to immediately see the results of the AI ​​in the field, it drives an interesting cycle of discovery and experimentation, and it also provides a great shoot-and-react format for video presentations,” Sherlock told PetaPixel.

quantum mirror

“I connected my camera to my phone’s hotspot via WiFi so it could submit my images in the field and display the results for playback after about 20 seconds of AI processing.”

DALL-E 2 is an artificial intelligence (AI) system developed by OpenAI capable of creating photo-realistic images based only on brief descriptions. The app was first shown in April and entered beta on July 20.

“When I take pictures, my pictures are sent to the DALL-E 2 AI, which dreams up new versions of them, revealing a world that might have once existed,” he explained. “Join me for a photoshoot in an alternate reality version of Dunedin, New Zealand.”

The original photos below were taken using Sherlock’s Sony Alpha 7R in combination with a Canon FD 35mm TS SSC lens or a Laowa 85mm macro lens.

quantum mirror
The original
quantum mirror
Dar-E 2
quantum mirror
Dar-E 2
quantum mirror
The original
quantum mirror
Dar-E 2
quantum mirror
Dar-E 2
quantum mirror
The original
quantum mirror
Dar-E 2
quantum mirror
Dar-E 2

DALL-E 2 seems to be the easiest to recreate simple landscapes, because when the AI ​​tries to recreate images that contain text or logos, they get weird, distorted, nightmarish versions.

As Sherlock puts it, “Spelling is not AI’s strong suit.”

quantum mirror
The original
quantum mirror
Dar-E 2
quantum mirror
Dar-E 2
quantum mirror
The original
quantum mirror
Dar-E 2
quantum mirror
Dar-E 2
quantum mirror
The original
quantum mirror
Dar-E 2
quantum mirror
Dar-E 2

While the results are interesting and the process for creating them is unique, Sherlock said OpenAI believes his app violates its beta program’s terms of service.

“Currently, using this app automatically bans your DALL-E 2 account because OpenAI considers it a ‘web scraper’ that violates its terms and conditions,” he said.

Sherlock intends to adjust the application after OpenAI releases its public API and reformats the version to comply with these new terms.

The Quantum Mirror Github page is the best place to read more about the app for anyone interested in seeing how it works. In addition, Sherlock’s Quantum Mirror website has more photo comparisons between the photos he took with the Sony camera and some of the interactions the DALL-E 2 has reimagined from them.


Image Source: Photo by Nicholas Sherlock, AI image generated using DALL-E 2

Leave a Comment

Your email address will not be published.