OpenAI’s new AI system DALL-E 2 can create mesmerizing images from text
Back in January of 2021, OpenAI introduced DALL-E, a neural network the company said can "take any text and make an image out of it," according to OpenAI's chief scientist and co-founder, Ilya Sutskever. This included concepts it may never have chanced upon during training.
Now, the firm is back with an even more impressive version of the previous program called DALL-E. According to OpenAI's website, "DALL-E 2 is a new AI system that can create realistic images and art from a description in natural language."
Astronauts on horses and more
The website gives many examples of the works of the AI system and they are nothing short of impressive. For example, the words "an astronaut" "riding a horse" and "in a photorealistic style" produced the image we feature above.
What's astonishing here is that the AI system understands where to put the astronaut on the horse and what a photorealistic image is. OpenAI further states that "DALL·E 2 can make realistic edits to existing images from a natural language caption. It can add and remove elements while taking shadows, reflections, and textures into account."
Examples show how the program can indeed add features or make them disappear entirely within a picture while still leaving it in good condition. Finally, OpenAI reveals that "DALL·E 2 can take an image and create different variations of it inspired by the original." This is illustrated by a painting that is made slightly different with each version while remaining true to the painting's overall features.
How does DALL-E achieve all this?
OpenAI explains that "DALL·E 2 has learned the relationship between images and the text used to describe them. It uses a process called “diffusion,” which starts with a pattern of random dots and gradually alters that pattern towards an image when it recognizes specific aspects of that image." Even better, OpenAI's content policy does not allow users to generate violent, adult, or political content, among other categories making the program safe for everyone.
We had the chance to speak to Dr. Stiavelli, the head of NASA’s James Webb Space Telescope project