Text-to-video AI inches closer as startup Runway announces new model


Text-to-image AI is mainstream now, but just waiting in the wings is text-to-video. The pitch for this technology is that you’ll be able to type a description and generate a corresponding video in any style you like. Current capabilities lag behind this dream, but for those tracking the tech’s progress, an announcement today by AI startup Runway of a new AI video generation model is noteworthy nonetheless.
Runway offers a web-based video editor that specializes in AI tools like background removal and pose detection. The company helped develop open-source text-to-image model Stable Diffusion and announced its first AI video editing model, Gen-1, in February.
Gen-1 focused on transforming existing video footage, letting users input a rough 3D animation or shaky smartphone clip and apply an AI-generated overlay. In the clip below, for example, footage of cardboard packaging is paired with an image of an industrial factory to produce a clip that could be used for storyboarding or pitching a more polished feature.
Gen-2, by comparison, seems more focused on generating videos from scratch, though there are lots of caveats to note. First, the demo clips shared by Runway are short, unstable, and certainly not photorealistic, and second, access is limited. Bloomberg News reports that users will have to sign up to join a waitlist for Gen-2 via Runway’s Discord, and a spokesperson for the company, Kelsey Rondenet, told The Verge that Runway will be “providing broad access in the coming weeks.”
In other words, all we have to judge Gen-2 right now is a demo reel and a handful of clips (most of which were already being advertised as part of Gen-1).
Still, the results are fascinating, and the prospect of text-to-video AI is certainly intoxicating — promising both new creative opportunities and new threats for misinformation, etc. It’s also worth comparing Runway’s work with text-to-video research shared by behemoths like Meta and Google. The work by these companies is more advanced (their AI-generated clips are longer and more cohesive) but not in a way that necessarily reflects these firms’ massive resources. (Runway, by comparison, is only a 45-person team.)
In other words: startups continue to do exciting work in generative AI, including the still-unexplored territory of text-to-video. Watch for more soon, AI-generated or not.
Text-to-image AI is mainstream now, but just waiting in the wings is text-to-video. The pitch for this technology is that you’ll be able to type a description and generate a corresponding video in any style you like. Current capabilities lag behind this dream, but for those tracking the tech’s progress,…
Recent Posts
- Fortnite’s new season has heists, pickles, and Cowboy Bebop
- The best microSD cards in 2025
- I tried this new online AI agent, and I can’t believe how good Convergence AI’s Proxy 1.0 is at completing multiple online tasks simultaneously
- I cannot describe how strange Elon Musk’s CPAC appearance was
- Over a million clinical records exposed in data breach
Archives
- February 2025
- January 2025
- December 2024
- November 2024
- October 2024
- September 2024
- August 2024
- July 2024
- June 2024
- May 2024
- April 2024
- March 2024
- February 2024
- January 2024
- December 2023
- November 2023
- October 2023
- September 2023
- August 2023
- July 2023
- June 2023
- May 2023
- April 2023
- March 2023
- February 2023
- January 2023
- December 2022
- November 2022
- October 2022
- September 2022
- August 2022
- July 2022
- June 2022
- May 2022
- April 2022
- March 2022
- February 2022
- January 2022
- December 2021
- November 2021
- October 2021
- September 2021
- August 2021
- July 2021
- June 2021
- May 2021
- April 2021
- March 2021
- February 2021
- January 2021
- December 2020
- November 2020
- October 2020
- September 2020
- August 2020
- July 2020
- June 2020
- May 2020
- April 2020
- March 2020
- February 2020
- January 2020
- December 2019
- November 2019
- September 2018
- October 2017
- December 2011
- August 2010