Last year I worked on an intensive research into AI & Video, with a deep dive into Web3 use cases.

Since then there has been a rapid evolution of the technology as predicted by many. Suddenly companies like Runway ML (founded in 2018) were everywhere. And seemingly overnight (but in reality years of development), AI video evolved from simple animations to a true storytelling tool. Below is an example of how AI video capabilities have improved over the last six months.

One of the biggest developments has been the release and rapid growth of ChatGPT. ChatGPT has already been injected into almost every sector, from creating beer recipes for Beck's to automating administrative tasks. And the recently opened ChatGPT store will continue to unlock what's possible as developers and entrepreneurs build on top of this LLM with its eyes on AGI.

Before we highlight some trends, tools, and predictions, let's recap last year's report.

This is the TLDR of the Q1 2023 as told by ChatGPT-4:

In 2023, The Gallery DAO's report on 'AI & Web3 Video: Tools Trends & Resources for the Creator Economy' provided an in-depth analysis of the evolving landscape of video creation in the context of AI and Web3 technologies. It highlighted the integration of traditional video tools with innovative Web3 products, offering insights into a range of creator tools across various segments. The report also emphasized the significance of ethical practices in AI, discussing monetization and artist credit in this new digital era.

This is the TLDR of the Q1 2023 as told by ChatGPT-4

• Major advancements in video quality usher in a wave of new consumer /B2B2C companies though text-to-video is still early. Not just generation, but techniques like deblurring and upscaling.

• Real-time video and motion capture also improves as seen with Domo AI.

• Decentralization and accessibility to build brings new startups to the space as seen by the growing list on There's an AI for That.

Early 2024 News

Livepeer started the year announcing their foray into AI Video Compute. The decision to focus resources on AI video is a sign that this technology will have a wide-reaching impact on society.

As the first motion pictures and animated characters enter the public domain, we will likely see more boundary pushing when it comes to IP usage. However general public sentiment remains negative, as seen by backlash to artwork surrounding Nicki Minaj's latest release (how many fingers are on the human hand again)?

In terms of content types, short form video and live streaming still lead the pack in traditional social and media. It will be interesting to see how AI video is consumed and if this also ushers in more long form, "cinematic" content.

Made using VPM, a visual assistant tool from ORIGIN STØRIES (model SDXL)

New Tools and Technical Advancements

Early last year Deforum lead the pack in terms of image-to-video tools and companies like Kaiber integrated these latent space creations into their offerring. There were many research papers, but not many easy-to-use tools (other than Replicate). Now the options and quality continue to expand. Here are the top tools at the moment for AI video creation.

Runway

Stable Video Diffusion

Pika

Assistive

The range of tools emerging will allow for more diverse storytelling and so it seems for now creatives will help to steer the "how" when it comes to use cases for this new type of media. This includes the announcement of a nature-focused large language model, released by Refik Anadol.

Many projects are still in research or stealth phase, including the recently announced text-to-video model from ByteDance (the parent company of TikTok).

AI Hits Hollywood

Visual direction for "Pray" by ORIGIN STØRIES

The Hollywood strike uncovered some of the brutal truths about how fast artificial intelligence will be implemented and how it could impact jobs. The importance of unions to protect workers helped significant protections for writer's from the Writer's Guild. Actors part of SAG-AFTRA accepted a deal with lots of grey area. It will be interesting how this plays out and what AI looks like at the next negotiations. It's clear the landscape will look much different, especially following recently deals between SAG-AFTRA and Replica voice studios and the major text-to-voice advancements that have been made by Eleven Labs.

AI Video Creation Onchain

Artists, especially those who release on chain have been experimenting with AI for much more than album covers. And while not all were video, there were more than 2k submissions in the AI Art Category alone for We Love The Art.

ORIGIN STØRIES is building custom generation tools and co-created experiences between music artists and audiences. This experience provides a new way to discover new talent and also be a part of the show. Next dates are 17 February in Oslo and 22nd February during NFT Paris. I submitted a piece from the creative tech studio for the We Love The Art Contest on Optimism.

Ethics and AI

While there might be a lot of public discourse and artists anger over the rapid rise of artificial intelligence and the training usedor datasets, many cultural institutions and businesses are diving deep into AI in both a research and product development phase. One example is Cultural AI, a collaborative initiative in the Netherlands , focused on the "study, design and development of socio-technological AI systems that are implicitly or explicitly aware of the subtle and subjective complexity of human culture."

What's Next?

The timeline and specifics of how exactly AI will be layered into society is still to be seen, but it's clear that it will happen. I'm sure the advancements when we check in for 2025 are going to be even bigger than the ones highlighted today.

What do you think about AI? How are you using it in creative work? Do you think there's space for businesses that aren't multinational tech companies to build scalable and sustainable products? Feel free to share your thoughts in the comments.

Resources for Creatives:
AIFF- Film Festival
Runway Creative Partners Program
Assistive Creator Program

AI News & Updates on X:
@CodeByPoonam
@rowancheung
@scobleizer
@nickfloats