DALL-E Mini: Everything you need to know about the AI art generator ruling the internet

• DALL-E Mini, a powerful text-to-image generator, has taken the internet by storm
• DALL-E Mini is a portmanteau of Salvador Dali and WALL-E
• Users have been using the tool to produce bizarre, surreal images that solely existed in their imagination

If you have been using Instagram and Twitter in the past two months, you probably wouldn't have missed memes such as Donald Trump as the Nevermind baby. Or the presidential inauguration of Jack Black, minions fighting in the Vietnam War, or Jesus Christ laughing at a meme on his phone.

These are all examples of AI-generated art, and it is giving visual artists and graphic designers a run for their money. Many didn't think it was possible until the AI image-generation tool, called DALL-E, was released by Open AI in January 2021. (DALL-E is a portmanteau of Salvador Dali and WALL-E.) However, Open-Ai did not make DALL-E widely available due to concerns that it would be misused.

Enter Boris Dayma, a machine learning consultant in Houston, Texas, who was fascinated by DALL-E. He created the first version of DALL-E Mini at a hackathon organized by Hugging Face, a company known for its open source artificial intelligence projects, and Google later in 2021. The project was later rebranded as Craiyon, to avoid confusion with the original DALL-E project.

How does DALL-E Mini conjure images?

All one has to do is type in a prompt on the website describing an image — it could even be something that exists only in your mind — and click on 'run'. Within a couple of seconds, the algorithm will display nine images in the form of a 3x3 grid, matching the algorithm's interpretation of the prompt with different variations (If you get an error message about traffic to the tool, you'll have to try again).

Users quickly got their creative juices flowing, inputting the most ridiculous prompts, resulting in hilarious, surreal pictures that triggered a meme fest across the Web.

A note about the tool's website says that the algorithm was trained on "unfiltered data from the internet." DALL-E Mini has been trained by analyzing about 30 million labeled images to extract connections between words and pixels. The training data was compiled from a number of public image collections, including one released by OpenAI. The program then builds up a database that connects word prompts to what an image can look like. Because it lacks an understanding of how objects look and behave in the real world, the software can make mistakes and struggle to produce a perfect image; most faces are distorted.

However, improvements by Dayma and a few viral memes did their magic, and the tool has exploded in popularity.

DALL-E Mini versus DALL-E

According to reports, Dayma plans to launch a premium version of his image generator.

"I would get interesting feedback and suggestions from the AI community," Dayma told NPR. "And it became better, and better, and better" at generating images until it reached what Dayma referred to as "a viral threshold".

This April, OpenAI revealed a more powerful AI text-to-image making tool DALL-E 2. According to the platform, this successor to DALL-E can produce images resembling photographs and illustrations that look extremely real and as if a professional artist made them.

Partly because of its concerns about misuse, OpenAI has provided access to DALL-E and DALL-E 2 only to select users, including artists and computer scientists, who must abide by stringent rules. As of July 12, the company has invited 100,516 people to try DALL-E. The company follows this approach to "learn about the technology’s capabilities and limitations.”

The ethical implications of AI media generators

Several developers have admitted to the potential for harm from using such generators when in the wrong hands. Deepfakes, or rather convincing applications of machine-learning models to render false images of celebrities and politicians, have become a concern for AI researchers, lawmakers, and nonprofits that work on online abuse and harassment.

Prof Toby Walsh, AI researcher and author of a book on the morality of AI, told The Guardian that the kind of technology that powers DALL-E makes it easier to create fake images. "We are seeing deep fakes being used all the time, and the technology is going to allow still images, but ultimately also video images, to be synthesized [more easily] by bad actors," he says.

Users have rendered questionable prompts, resulting in images that probably shouldn't exist. The company OpenAI said that it filters the system's training data and restricts keywords that could provide explicit content. OpenAI had implemented a safety policy for all images, filtering images that could be sexual, violent, or inappropriate. It even has a second filter that is used to try and remove images that could be propagandistic. The DALL-E 2 team is also checking every image to weed out anything inappropriate.

However, DALL-E Mini works differently in terms of filters and restrictions - it has none. Though the team has given a disclaimer about offensive and biased images, no search terms have been banned yet. As a result, some prompts wouldn't pass the original model's safety protocols, resulting in the outlandish images one must have seen online.

The rise of AI artists?

Dr. Oliver Bown, a researcher in computational creativity at the University of New South Wales, told The Guardian that the public also needs to be educated to be more discerning in addition to regulatory framework and company policies around the use of the technology about what they see online.

Walsh also drew attention to the potential for text-to-image AI to replace jobs in graphic design.

"You can imagine that more of us are going to be able to do graphic design because we could say ‘paint me a picture with the specification when we want, and we’ll get that picture. Whereas previously, there was a graphic designer who produced that picture. Graphic design isn’t going to go away, it will lead to even more graphic design because all of us can access these tools, but graphic designers might have less work themselves," he said.

Brown added that the AI image generator would also allow for "prompt creativity", meaning the thought that goes into the image request will require more creativity. "This new challenge is on for creative people to think about what they want to put into a system like this," he said.

DALL-E Mini is not the last such system; other companies are also building their image-generating tools. This May, Google announced Imagen, a research system capable of generating images of a level similar to DALL-E 2. It had also announced another called Parti. Both the systems are publicly unavailable.