Background
I started filming with my fathers Super 8 camera back in the 80s. The film was super expensive. The quality poor and it was a mess when you wanted to watch. It was always a slight chance that you would burn the filmstrip. Do I need to say that backups were impossible. Later on C VHS Camcorders came. The quality was even worse but at least the tapes were inexpensive.
I started to film more frequently in 2000 during a trip to Tanzania. I had my grandfathers Sony Digital 8 camera. Later also a Canon XM 2 Mini DV. I remember I thought the quality was amazing. It was digital. The camera is sold since many years. However I was able to buy an old mini dv camera just in order to capture some more old tapes.
The files created from a DV camera are avi files. The container is avi.
- 720*576 PAL (50 Hz)
- 4:3 format
- Bitrate is 25 Mbps
- Interlaced
- Lossy compression
- DCT to compress video on a frame-by-frame basis
I have saved most of my original avi files. I also have an edited film made in Ulead I want to preserve. And also converted them to mp4. I have some other posts about this. The main issue when converting or updating dv files is that you want to use 16:9. 4:3 looks old and bad. Also de interlacing the footage is crucial. This can be done in many ways. In general the quality of the dv or in some cases VHS is so poor anyway so you do not notice the difference. I have used Yadif parameter.
Goal
I record video with my DJI Mini 4 pro as well as my Sony RX10IV. The results is amazing. Crisp 4k video. My old videos looks terrible when compared. I could have been getting better results with an low budget phone today. My Canon XM 2 costed me 2500$ at the time. So what I would like to do is.
- Upscale to 4K (or Full HD)
- Convert to other codec
- Make it 16:9 (In some cases)
- Remove noise
- Enhance video in general
- Remake my Kilimanjaro film from 2000.
If someone asked me some years ago if you could improve quality of a bad blurred unsharp video I would say no. It is impossible. You can do some general improvements but if the source is bad you will get bad output. Now with the AI revolution it is possible. So lets start exploring what can be done.
Challenges
Upscale an old dv video might sound easy. But lets consider how much upscale that is actually needed. DV is 0.415 Megapixels. 4K is 8.3 Megapixels.
DV is 720*576 and has the aspect ratio 5:4. In old TVs is is displayed as 4:3 and the image is adjusted. Full HD is 1920*1080 pixels and the ratio is 16:9. So we will have issues both with the large upscale as well as there will be problems with ratio. You will have two options here. Put letterbox on the sides creating black sides. Or crop and remove some parts of the original video. What you choose may vary.
In the image above you can see how small the DV video is. Upscaling this to 4k just do not make sense.
In this image you see that if you upscale the dv video to full size there will be letterboxes on the sides. Letterboxing is a technique used in video and film to display widescreen content on standard-width screens while preserving the original aspect ratio. This is achieved by adding black bars (mattes) at the top and bottom of the screen
And if we make it fill the full HD screen we will lose footage (the purple in the image). In most cases you want to reedit this. If you just have raw videos I would not use this method.
Open source alternatives to Topaz
Before we start exploring Topaz Video AI you may have asked yourself if there is no free alternatives. I love open source and some of the best software is open-source. Topaz may seem expensive so why not use something else? I will do a brief summary. I really wanted to find a perfect free tool.
Video2X
This would most likely be my first choice if I had to choose. However… It does not deinterlace correct. If the image is deinterlaced the result is quite poor compared to Topaz. And it takes forever. Also the models are focused on anime. I did use Real-ESGAN with real esgran plus (non anime). 4 x upscale in Topaz took 44 s. In Video2X 12 minutes. I did run some small clips to compare. They were done with an h264 mp4 file and not the dv avi. Note that I use Topaz Video AI. And stills. I suppose you could get impressive results with Topaz Photo but in this case it does not work.
As you can see Vide2X makes the bird look very strange. It also adds some random artifacts that makes using this impossible. If you look at the Planet4 logo you get the typical melted icecream AI syndrome. So the final verdict is that Topaz is far better and faster.
RealScaler
RealScaler installation is quite hard if you do not buy it via itch.io. It is focused on Anime film.
Waifu2x-Extension-GUI
Waifu2x-Extension-GUI is focused on Anime to. The gui is perfect if you want it like it was in 1997. To be honest. Is the a tool you would like to use?
Running Topaz Video AI
Just to be clear from the start. There is no one click button to get a perfect video. You will have to try different settings, models and inputs. It will also take time. My last render took 26 hours. You will need time and/or a powerful computer.
Input
To start you will need a source file. To make this as good as possible save your old file first. Do not overwrite it. Also use the original file if possible. If you have the dv avi file for example try this first. Let´s say you filmed this years ago. You converted the file in 2003 to divx, then some years later you re saved it as mpeg2 to use for DVDs. Each conversion that is done will change the original file. And add extra artifacts and compression that makes the original file loose details. In my case I have DV Avi files as well as some other files converted to h264 with ffmpeg. I will show you some results later.
Conversion and Topaz
How you work with your video may be different from how I do. There might be other AI models that might produce even better results. I found that this way worked fine for me.
- De interlace and upscale. Increase FPS.
- Upscale to FHD. With proteus model. Crop from 4:3 to 16:9
- Edit in Davinci. Export as h265
First step is t de-interlace and upscale. I do not really understand why but my file is 720*576 (DV PAL) but Topaz thinks the original file is 768*576? I choose to set the output manually to 1440 x1152. Also choose codec and audio. Clone or convert. Otherwise the audiotrac is lost. Choose to double the framerate. Use squarepixel. (se FAQ) and the model Dione DV.
If you set the custom resolution to 1440 and tick “keep aspect ratio” in Topaz it will be 1440*1080 and you get the choice of “resize method” and prioritize width (or height). I noticed i used this in some clips. Not sure why as 720*2=1440 and 576*2 =1152. Anyway I do not think the result differs that mucj if it is 1080 (cropped) or 1152. I will leave this as something to learn later on.
This will create a deineterlaced ai fixed videofile twice as big as the old one. A good start. Save the project. It will take a long time. I did it overnight.
In step two we take the newly created file and open it in Topaz Video AI. Now we have some choices. As I want to use this to recreate my old Kilimanjaro film from 2000 i want it 16:9. Read more above under challenges. First choose Full HD (1920*1080). Set the crop setting to “crop to fill”. Maybe you want one file letterboxed and one cropped? Choose Proteus model. Export.
Results, Models and upscaling
Before showing some results I would like to mention that it is video. Stills does not really shows the full picture. Also the bushed in the back looks a little bit strange. It will look better when played. When I shoot this I most likely used the digital zoom on my Sony camcorder. I did not really understand back then that it was a bad idea. Note that the last giraffe is cropped and in some cases this will not look good.
Now I would like to show some videos of giraffes. The quality is poor from the start. The first film is the orginal dv footage. The second one is the Dione DV upscale 50 fps and the third is the cropped Proteus file. The fourh film is the original file but it was deinterlaced with ffmpeg. It would maybe be the closest how the orginal footage would look on an old crt tv. The result may not be perfect. But it looks better when the original video is better.
You may wonder what happens if we skip Topaz and just take the original DV file, convert it to mp4 and deinterlace it with ffmpeg. And the just upscale it to full HD. I will show three images.
If just upscaling it will be very blurry. I have seen some of my clips looking better with just Dione and others better with both Dione and Proteus. I just not had the time to analyze each clip. But if you have time you might get better results.
In some parts i decided to use Artemis and Theia as some parts of the forsest just looked very strange. It looked good but it took a very long time to process. Se example image below. (artemis to the right)
FAQ?
Square vs non-square pixels.
If pixels are not square, they are typically referred to as rectangular pixels or non-square pixels. These pixels have an aspect ratio that is not 1:1, meaning the width and height of each pixel are different. This can lead to distortions if not correctly interpreted by the display or software.
- NTSC: 720×480 with a pixel aspect ratio of 0.91 (narrower than tall)
- PAL: 720×576 with a pixel aspect ratio of 1.09 (wider than tall)
- Anamorphic Video
- Anamorphic 16:9: Uses a non-square pixel aspect ratio to fit widescreen content into a standard aspect ratio container. This is common in DVDs and some broadcast formats.
Using non-square pixels can be advantageous for certain broadcast standards and older video formats, but it requires proper handling during editing and playback to avoid issues with aspect ratio and image quality.
Why do we use full HD and not 8K?
In this case the input quality is so low so I just wont work. If we add more AI models and add details faces will start to look strange. Artifacts may appear randomly. In some parts it may look perfect and in other parts not.
Why do we run two steps?
Good question. You do not have to. Maybe you like the Dione DV model better? You could run a third step with Nyx. But it took so long time and created other issues so I decided this was good enough.
Why do we use Proteus as AI model?
Because it is a general model that will not tamper to much with the original. Feel free to test. Also it is quite fast. Make sure to read all details at the Topaz AI Video site. It will explain each model in detail.
I have tried with the Artemis model as well as this is a dedicated denoise and sharpen model. I did see some improvements but it was not amazing. I have see other people use this with great success but I thin my videos were to poor.
Why does it not get super clear as in the examples I have seen?
Well, In some cases it does. Below is a video in HD I filmed with a 2006 camcorder. Better quality but not 4k. I did try to upscale this to 4k and the result is amazing. The main difference is that the original video was quite ok as well. But not 4k.
The image/film below is a random team from Gothia Cup. I did upscale this with Rhea model and the results is amazing. Hard to see in this image but the results are just as good as you see when advertised.
Summary
When I did my film 25 years back I thought it would be impossible to improve it. Now it is done. Topaz makes it possible. The quality of the original was not the best so do not expect any miracles. If you have HD footage you could most likely create very good 4k upscaled videos. in my case not.
I have also used Davinci Resolve. As the transitions and text in the film was not perfect I just added the old film in Davinvi. It imported the Topaz generated full HD material in the editor and recut the movie. It took some time but now it feels like the film is brand new. I did the original film on a celeron computer with 196 Mb RAM. So times have changed.
So here it is. The final result and the goal of this project. Kilimanjaro 2000 digitally remastered.
Thanks for Topas Labs for letting me test their software. i would definitely recommend anyone working with old footage to give it a try. Next step is upscaling Super 8 movies from the 80s.
The final result can be found here
Useful Links & commands
Thread about DV and Topaz
Another Reddit Thread
Topaz AI Models Docs
Video Compare. A tool to play two vides at the same time. Usefull for comparing.
Commands
Some general ffmpeg commands I have used. To convert avi to mp4 if not using Topaz. And to get some parts out of a video for testing. And make sure you get low render times.
ffmpeg -ss 01:02:09 -to 01:02:15 -i .\0-step1.mp4 -vsync vfr -vf yadif -c:v libx264 -pix_fmt yuv420p -c:a mp3 giraffe.mp4
ffmpeg -ss 00:00:05 -to 00:00:10 -i .\kilimanjaro2000-final-good.mp4 -c copy reference2.mp4
