While we were still contesting whether or not we could tell the difference between a real or 'fake' photo, along comes a text-to-video program named Sora (Japanese for sky) that can create scarily photo-realistic one minute clips.
OpenAI's website states that 'Sora is able to generate complex scenes with multiple characters, specific types of motion, and accurate details of the subject and background. The model understands not only what the user has asked for in the prompt, but also how those things exist in the physical world.'
Sora's abilities will be sending shockwaves to the film and animation industries where it will be undoubtedly used to create highly detailed animation or green screen backgrounds. And even though the program is in its infancy, and not yet open to the public, you can guarentee that the 'footage' created will only become more realistic over the next few years.
The company behind the ChatGPT chatbot and the still-image generator DALL-E also notes that Sora can “accurately interpret props and generate compelling characters that express vibrant emotions.” Image: OpenAI
In addition, Sora can also generate a video based on a still image, as well as fill in missing frames on an existing video or extend it. This would enable the 'improving' of old, turn of last century 16fps B&W film footage – to look as if it was shot as 25fps colour footage.
Incredible as it is, OpenAI says the tool does have weaknesses, and “may struggle with accurately simulating the physics of a complex scene,” and may not understand specific instances of cause and effect.
Despite this, the results so far are pretty impressive.
The team behind the technology, including the researchers Tim Brooks and Bill Peebles, said the company was not yet releasing Sora to the public yet because it was still working to understand the system’s dangers.
Instead, OpenAI is sharing the technology with a small group of academics and other researchers who will “red team” it, a term for looking for ways it can be misused.
Despite this, with still images now capably produced by most AI tools, video appears to be the next frontier, with companies like Runway Pika and Google's Lumiere already demonstrating impressive text-to-video models in an increasingly crowded space.
You can watch some of the examples of the text prompted technology on the OpenAI website.
