It all started by accident!
Midjourney
I was using Midjourney A.I. to generate some images of futuristic concept motorbikes for another video, and had a few browser windows open with some concept motorbikes that I was using as image prompts for Midjourney.
Later that day, I was seeking an image of some “people talking on cell phones ignoring each other” for a totally different video, and was using that as the prompt combined with some other image prompts.
Well you guessed it, I accidentally used the concept motorbike image with the prompt “people talking on cell phones ignoring each other” and the 4 images above were generated. Not exactly what I was after, but they looked cool so I put them aside for another day.
When I finally got back to these I just kept generating the same prompt and getting different results each time. I also tried using a concept car as the prompt, then used the already generated images as prompts with the same text prompt over and over.
What I ended up with
Getting a few more images
With that set of images ready I tried a few different prompts:
“Short attention spans, biomorphic ignorance”
“Humanoid portrait, Biomorphic style”
“Divided attention spans, biomorphic style”
Generating Some A.I. Animation
Before piecing it all together I used a few different A.I. tools to generate some video clips, but first here’s a simple free A.I. tool that you can use to cut out parts of an image or remove the background:
PhotoRoom
https://www.photoroom.com/background-remover
For this image I removed the background and replaced it with black. Photo room has a load of other free tools and some paid plans for more quick tricks.
Leiapix
https://convert.leiapix.com/draw
Leiapix is a simple to use free tool that can bring an image to life by making shord 3d clips that can be exported as different file formats such as MP4.
Kaiber A.I.
I used Kaiber A.I. for certain parts of this video by basically uploading images, setting a few simple parameters, and a text prompt then letting it do its thing. You can try it for free but then you will need to pay. The lowest plan is $5 per month.
The talking Heads
For the talking heads there are various options available, some free and others paid, and the results vary.
For a few of the talking heads in this video I used this free Lip-sync demo tool I found.
The Tool: https://bhaasha.iiit.ac.in/lipsync/
The Explainer Video on Youtube: YoutTube video explaining how to use the tool
The results are not great and you first need a video prompt. If you want to use your own Midjourney or other A.I. generated image you need to convert it to a video clip. I used Leiapix for this. The explainer video guides you through this.
As I said the results are not great and the characters I generated don’t blink. The output is also low res, which is why I didn’t use any of them full screen in my video. You could of course use an A.I. video upscaler such as Hitpaw bur that’s not free so you would be better off using a paid talking head generator such as: https://www.d-id.com/ which will give you much better quality.
For the purpose of this A.I. mash up video the free generators served their purpose. Another free one I used for this video is https://huggingface.co/spaces/vinthony/SadTalker
I got better results with SadTaker but it can take ages to generate and often fails due too many people using it.
Lip-sync
Sad Talker
The Voice Generation
For the voices I used Eleven Labs which for me is still the best there is. There are tons of them out there but Eleven Labs is so realistic and you get quite a lot for free.
Video Generation
The two video generators I used both can be used for free but with limitations and some paid options with one of them:
Runway ML
First of all Runway ML have a frame interpolation that allows you to add images to a sequence and have it morph them together like this:
Actually runway ML have loads of cool A.I. tools for image and video generation. Their Gen 2 video generator creates 5 second clips from image and text prompts. Its still early days for video generation but this technology is growing so fast. Here’s what I generated. with the following text prompt
“Fast moving VR in a world of robotics”
Pika Labs
For now Pika Labs is completely free. You will need a discord account to use it. The generations are only 3 seconds long at this stage so I strung a few of them together for various parts of my video. Here’s an example of a three second clip
Piecing it all together
While these A.I. tools are great for creating art and animation very quickly there is still plenty of work required to piece it all together. For this video I used Open Shot Video Editor which is free to download and use for your projectsOpen Shot has limitations but its simple to use and served me well for this video.
You will of course have to use your own imagination and come up with a plan and a script (Although Chat GTP could help you with that too) I just pieced this all together on the time line using masks, transitions, zooms, fades, key frames etc.
Watch on Youtube: https://youtu.be/Kmv9FbBDe0c
Useful Resources
So I’ve touched on how I accidentally got started with this video and the tools I used, but here are a few YouTube channels I find very useful for tips and keeping updated with the latest A.I.
Matt Wolfe (Stay up to date with A.I.)
https://www.youtube.com/@mreflow
Matt constantly uploads videos of the latest A.I. tech available and has a website https://www.futuretools.io/ where you can search and filter all of the cool tools available.
Theoretically Media (Tutorials on Creative Tools )
Lots of videos and tutorials on the latest creative tools for film making, music, writing, and art. Very direct and straight to the point without the waffle.
Obscurious (Getting creative with A.I.)
https://www.youtube.com/@obscuriousmind
Good tutorials on the latest A.I. creative tools. Another one of my favorites. Gets to the point.
Tokenized AI by Christian Heidorn (Midjourney Tutorials)
https://www.youtube.com/@TokenizedAI
In depth tutorials for Midjouney. This guy is the prompt master and will get you started with Midjourney and have you creating awesome images.