offering my help - enhancing cut scenes like Wing Commander

In here you can discuss what is going on in the project, offer help, praise or critique!
Post Reply

offering my help - enhancing cut scenes like Wing Commander

PopoPirat
Cadet 4th Class
Posts: 12
Joined: Sun Dec 27, 2020 12:09 pm

Post by PopoPirat » Mon Dec 28, 2020 2:17 pm

Hi guys,

I just managed to install the update to XWA and I am amazed by how good it looks! While watching the cut scenes though I thought, how about upscaling them like Owen did for Wing Commander : https://www.wcnews.com/chatzone/threads ... ack.30415/
I know it's CGI but it can too be improved.

Only problem is the video file extension is .snm and I can't open those. Does anyone know how to extract/convert those so I could manipulate them with Topaz Enhance AI?

User avatar
Ace Antilles
Admiral (Moderator)
Posts: 7825
Joined: Sat Jan 22, 2000 12:01 am
Contact:

Post by Ace Antilles » Mon Dec 28, 2020 3:40 pm

PopoPirat wrote:
Mon Dec 28, 2020 2:17 pm
Hi guys,

I just managed to install the update to XWA and I am amazed by how good it looks! While watching the cut scenes though I thought, how about upscaling them like Owen did for Wing Commander : https://www.wcnews.com/chatzone/threads ... ack.30415/
I know it's CGI but it can too be improved.

Only problem is the video file extension is .snm and I can't open those. Does anyone know how to extract/convert those so I could manipulate them with Topaz Enhance AI?
It's already been by forum user Marcop000.
They are hosted on my own Website for people to use.
https://www.xwingoutpost.com/

XWO small.png

Now if you could improve the ones in TIE Fighter, XvT or X-Wing. That would be good.
To extract SNM you need Xwa Snm Converter from the Downloads page.
https://raw.github.com/JeremyAnsel/XwaT ... loader.zip
You do not have the required permissions to view the files attached to this post.
Chief XWAU Team annoying nitpicker.
Ace Antilles - The X-Wing Outpost
Image

PopoPirat
Cadet 4th Class
Posts: 12
Joined: Sun Dec 27, 2020 12:09 pm

Post by PopoPirat » Tue Dec 29, 2020 10:09 am

Hi, that's fantastic I will check it out!

I own all of the games you listed (gog.com versions) and I will check out the tool too and post asap.

Porkins_BB
Cadet 2nd Class
Posts: 64
Joined: Wed Nov 25, 2020 1:42 pm

Post by Porkins_BB » Tue Dec 29, 2020 4:31 pm

Ace, is there a reason why the HD cutscenes were not included in the latest XWAU 2020 Update? Just curious. I was also wondering if the new HD format will help those who are having choppy/stuttering or out of sync cutscenes.

PopoPirat
Cadet 4th Class
Posts: 12
Joined: Sun Dec 27, 2020 12:09 pm

Post by PopoPirat » Tue Dec 29, 2020 6:56 pm

Ok guys, I've spent the whole day with testing the following:

Framerate interpolation using FlowFrames:
- Increasing framerate from 15 to 30 / result: big improvement! It makes all the videos instantly much nicer!
- Increasing framerate from 30 to 60 / result: only a marginal improvement over 30 fps. I'd say it doesn't justify the increased diskspace and processing time. The source material is simply too bad as to make a bigger difference.
I get minor artifacts in some videos (when objects get blurry in the source) but I would prefer those and have a crisper image over the blurry + pixelated low-res stuff.


Resolution & enhancement using Topaz Enhance AI:
- I quickly found that cropping is not an option. F.ex. Datapad would not be displayed correctly.
- Upscaling to 16:9 1080p or 4k results in black bars left and right baked into the video. So I would stick to the original aspect ratio. That would leave me with two reasonable options:

- Upscaling 300% to 1920x1440 without cropping / result: great "FHD" without the black bars baked into the video.

- Upscaling 400% to 2560x1920 without cropping / result: even more pixels so technically better and probably the way to go

Filesize comparison:
original TGINTRO.SNM 6.5MB
1920x1440 MP4 (CRF 20) 19.5MB
1920x1440 MP4 (CRF 24) 12.3MB
2560x1920 MP4 (CRF 20) 29.9MB
2560x1920 MP4 (CRF 24) 18.4MB
MarcoP 12.6MB

I'd be ok with CRF24 MP4 so we get:
3x increase in resolution + 2x the framerate for double the filesize of original
or 4x increase in resolution + 2x the framerate for 2.8x the filesize of original

Marcoop footage is sharper (maybe even too much at times) but keeps 15 FPS. I think the 30FPS is the "gamechanger" here which is missing in Marcoop footage. I will try to increase my sharpness a bit tomorrow and post some examples.



Regarding the other games: they use a different video format so I would need tools again. Any idea?
Last edited by PopoPirat on Wed Dec 30, 2020 10:48 am, edited 1 time in total.

PopoPirat
Cadet 4th Class
Posts: 12
Joined: Sun Dec 27, 2020 12:09 pm

Post by PopoPirat » Wed Dec 30, 2020 8:50 am

Quick update

I am using a different interpolation setting now and I get an even sharper image.

But I ran into several problems:

1. the SNM Converter tool puts out a really heavy compressed and therefore unusable video many times. FIrst I thought the SNM are just such a bad source, but when it converted the crawler text, O M G! It had artifacts like crazy, complete garbage. So I stopped using it and used Flowframes internal converter with a super clean endresult!

2. the timecodes of the videos are messed up and the converted files are being cut off many times. As soon as you do anything to the freshly converted file like interpolation or upscaling, the newliy saved file is shorter. The same can be observed in MarcoPs footage. For example LOGOFINAL should have 10s, final result has 7s of length. Using "butterflow-ui" for interpolation instead of "Flowframes" lead to better results in some cases. So each file has to be inspected several times.

3. As soon as I insert more than one MP4 into the MOVIES folder, only the first gets played and then the game jumps right into the pilots room. I can not get all of the new files to play. Is this a bug in TgSmush DLL?

So a lot of work had to be scrapped and redone. My conclusion is I can get a cleaner, smoother video than MarcoP but it still has flaws. So I don't know if I shall continue or not. I'm kinda torn.
You do not have the required permissions to view the files attached to this post.

User avatar
Ace Antilles
Admiral (Moderator)
Posts: 7825
Joined: Sat Jan 22, 2000 12:01 am
Contact:

Post by Ace Antilles » Thu Dec 31, 2020 12:02 am

3 - Could be a number of reasons for this. Maybe codec. Try AVI or WMV.
I personally can't see much of a difference in either picture.
It's going to be near impossible to make it perfect due to the age of the videos I think.

Going to move this topic into the XWA Discussion forum section.
Chief XWAU Team annoying nitpicker.
Ace Antilles - The X-Wing Outpost
Image

PopoPirat
Cadet 4th Class
Posts: 12
Joined: Sun Dec 27, 2020 12:09 pm

Post by PopoPirat » Fri Jan 01, 2021 8:08 pm

Ace Antilles wrote:
Thu Dec 31, 2020 12:02 am
3 - Could be a number of reasons for this. Maybe codec. Try AVI or WMV.
I personally can't see much of a difference in either picture.
It's going to be near impossible to make it perfect due to the age of the videos I think.

Going to move this topic into the XWA Discussion forum section.
You can see many artifacts depending on your screenbrightness. The MP4 I made are free of these kind of artifacts.
You do not have the required permissions to view the files attached to this post.

User avatar
Ace Antilles
Admiral (Moderator)
Posts: 7825
Joined: Sat Jan 22, 2000 12:01 am
Contact:

Post by Ace Antilles » Fri Jan 01, 2021 11:02 pm

I see now. If you can make a cleaner, smoother version I think it's worth having a go.
But if you can't get them to play then that needs to be looked at.
Chief XWAU Team annoying nitpicker.
Ace Antilles - The X-Wing Outpost
Image

User avatar
darklord
Galactic Empire
Posts: 836
Joined: Sun Jan 16, 2005 12:01 am
Contact:

Post by darklord » Sat Jan 02, 2021 5:50 am

Age of the movies shouldn't be a barrier to giving an attempt. There are upscales of Dos era games, ODVS's upscale of Wing Commander III for instance. I wish you luck in your attempt. And if nothing else, I'm sure it will be a great learning experience. It's amazing what things I would have said were impossible 5 years ago are being done now.

PopoPirat
Cadet 4th Class
Posts: 12
Joined: Sun Dec 27, 2020 12:09 pm

Post by PopoPirat » Sat Jan 09, 2021 4:14 pm

Still experimenting a little bit. Here are just a couple of stills from a 4k AI enhance upscale test for anyone to enjoy.
You do not have the required permissions to view the files attached to this post.

PopoPirat
Cadet 4th Class
Posts: 12
Joined: Sun Dec 27, 2020 12:09 pm

Post by PopoPirat » Sat Jan 09, 2021 4:15 pm

more
You do not have the required permissions to view the files attached to this post.

User avatar
Ace Antilles
Admiral (Moderator)
Posts: 7825
Joined: Sat Jan 22, 2000 12:01 am
Contact:

Post by Ace Antilles » Sat Jan 09, 2021 4:29 pm

Probably a good idea if you and @marcop000 join forces.
Looks like you are both working on the same thing and would save someone a lot of extra effort
Chief XWAU Team annoying nitpicker.
Ace Antilles - The X-Wing Outpost
Image

marcop000
Cadet 1st Class
Posts: 213
Joined: Fri Dec 27, 2019 7:08 pm

Post by marcop000 » Sat Jan 09, 2021 5:13 pm

Hi PopoPirat,

I'am the author of upscaled cutscene I just read this post I can send more info about:

To to these I have used:

topaz video enhance ai
https://topazlabs.com/downloads/

and butterflow-ui to do a frame interpolation.

This software use ai neural algorithm, that sometime can do a very good work.

Unfortunatenly this tecnique is not perfect and can introduce different artefact as you can see.

I don't known if exist exist another software that can bo better without side effects, but I can confirm you that every time they upload a new version I see some improvement.

What you see is not done by the last version of video enhance ai, becouse I can't repeat the work each time.

So is not a surprise for me if your work is better then my.

One note about butterflow-ui:

Now there is DAIN_APP
https://grisk.itch.io/dain-app

That in all my test is much better then DAIN_APP . So if you have time I suggest to use it for frame interpolation, and you can be sure to do a work better then my.

About your requests:

The SNM Converter tool was done by jeremy. You can ask to him.

"So a lot of work had to be scrapped and redone. My conclusion is I can get a cleaner, smoother video than MarcoP but it still has flaws. So I don't know if I shall continue or not. I'm kinda torn."

If you can upload some of your upscaled video we can compare with my and if are better we can ask to ACE to use it on his web site.

To be honest I have planned in the next weeks to rework all cutscene with the last upated version of topaz video enhance ai / DAIN_APP, but If you can do the work in my place you get me a favor.

Thanks !

Marco

PopoPirat
Cadet 4th Class
Posts: 12
Joined: Sun Dec 27, 2020 12:09 pm

Post by PopoPirat » Tue Jan 12, 2021 3:39 pm

marcop000 wrote:
Sat Jan 09, 2021 5:13 pm
Hi PopoPirat,

I'am the author of upscaled cutscene I just read this post I can send more info about:

To to these I have used:

topaz video enhance ai
https://topazlabs.com/downloads/

and butterflow-ui to do a frame interpolation.

This software use ai neural algorithm, that sometime can do a very good work.

Unfortunatenly this tecnique is not perfect and can introduce different artefact as you can see.

I don't known if exist exist another software that can bo better without side effects, but I can confirm you that every time they upload a new version I see some improvement.

What you see is not done by the last version of video enhance ai, becouse I can't repeat the work each time.

So is not a surprise for me if your work is better then my.

One note about butterflow-ui:

Now there is DAIN_APP
https://grisk.itch.io/dain-app

That in all my test is much better then DAIN_APP . So if you have time I suggest to use it for frame interpolation, and you can be sure to do a work better then my.

About your requests:

The SNM Converter tool was done by jeremy. You can ask to him.

"So a lot of work had to be scrapped and redone. My conclusion is I can get a cleaner, smoother video than MarcoP but it still has flaws. So I don't know if I shall continue or not. I'm kinda torn."

If you can upload some of your upscaled video we can compare with my and if are better we can ask to ACE to use it on his web site.

To be honest I have planned in the next weeks to rework all cutscene with the last upated version of topaz video enhance ai / DAIN_APP, but If you can do the work in my place you get me a favor.

Thanks !

Marco
Thank you so much for chiming in Marco! I also use Topaz Enhance AI for upscaling and found that for some video Gaia-CG v5 and for others Artemis LQ v9 works better.

You are right about artifacts. It's still not perfect. I am still experimenting how to get the best result. Most of the time the interpolation of frame rate brings the bad artifacts.

For framerate interpolation I tried Butterflow UI 1.1.0.3797 and also Flowframes 1.18.2. I can't use DAIN because I don't have a NVIDIA card, I have a RX 580 AMD. My computer is mainly a hackintosh with MacOS for Audio production. This here I just did as an experiment under windows.

Let's compare what we have and see which looks better and then go with that.

Here are a couple of tests on youtube. Higher resolutions than HD are still processing as of now.

Here is my WQXGA@15 upscale remaster:

https://www.youtube.com/watch?v=z_o-cvdDVEk


Here is my 4k@15 (cropped) upscale remaster:

https://www.youtube.com/watch?v=4gQ-sobozpQ


Here is my 4k@30 upscale remaster:

https://www.youtube.com/watch?v=vd914CXuqAE

marcop000
Cadet 1st Class
Posts: 213
Joined: Fri Dec 27, 2019 7:08 pm

Post by marcop000 » Tue Jan 12, 2021 5:05 pm

Hi PopoPirat,

I agree with you that with "Artemis LQ v9" is the better choice.

Your test "WQXGA@15" seem similar to my apart that is 15 FPS. My is 30 FPS but was done by a outdated version of Topaz Enhance AI.

In your test the text while scrolling is correct, but my introduce artifact becouse my version is outdated.

To avoid to strecth the video I use 4X upscaling without force to FullHD or 4K or other and for my experience with this we get the best results and play in widescreen without compromise.

To be honest I don't remember in my cutscene the initial artifact present in "4k@15 (cropped) upscale remaster:" and "4k@30 upscale remaster:". Why there are ? I don't see it in "WQXGA@15".

Apart this, youtube is not good way to see the result, becouse we need to see directly the file avi.

However If you agree we can do this test:

I can upload this cuscene at 60FPS (with DAIN) at original size, and you can use Topaz Enhance AI to upscale.

And then we can compare the result, I have no duobt that the result will be better then actual

Please let me known if you agree.

Thanks !

PopoPirat
Cadet 4th Class
Posts: 12
Joined: Sun Dec 27, 2020 12:09 pm

Post by PopoPirat » Tue Jan 12, 2021 9:06 pm

marcop000 wrote:
Tue Jan 12, 2021 5:05 pm
Hi PopoPirat,

I agree with you that with "Artemis LQ v9" is the better choice.

Your test "WQXGA@15" seem similar to my apart that is 15 FPS. My is 30 FPS but was done by a outdated version of Topaz Enhance AI.

In your test the text while scrolling is correct, but my introduce artifact becouse my version is outdated.

To avoid to strecth the video I use 4X upscaling without force to FullHD or 4K or other and for my experience with this we get the best results and play in widescreen without compromise.

To be honest I don't remember in my cutscene the initial artifact present in "4k@15 (cropped) upscale remaster:" and "4k@30 upscale remaster:". Why there are ? I don't see it in "WQXGA@15".

Apart this, youtube is not good way to see the result, becouse we need to see directly the file avi.

However If you agree we can do this test:

I can upload this cuscene at 60FPS (with DAIN) at original size, and you can use Topaz Enhance AI to upscale.

And then we can compare the result, I have no duobt that the result will be better then actual

Please let me known if you agree.

Thanks !
Ok let's try! But I would think MP4 is a better format than avi or not?

User avatar
Ace Antilles
Admiral (Moderator)
Posts: 7825
Joined: Sat Jan 22, 2000 12:01 am
Contact:

Post by Ace Antilles » Tue Jan 12, 2021 9:09 pm

You want the best quality for the lowest file size really.
Personally I don't really see it's worth the effort trying to make them 4k when HD is an improvement as it is.
I wouldn't want to add several gigabytes of files to my game when HDD space is always needed more these days.
Plus I don't have a 4k monitor yet :D
Chief XWAU Team annoying nitpicker.
Ace Antilles - The X-Wing Outpost
Image

marcop000
Cadet 1st Class
Posts: 213
Joined: Fri Dec 27, 2019 7:08 pm

Post by marcop000 » Wed Jan 13, 2021 8:03 am

Hi Ace,

I agree with you, and I can confirm that upscale a video more then 4x of original size is not a good think for different reasons:

1) Over this limit you get "zero" quality improvement.
2) You need more space in your drive.
3) The load time will increase.

In my cutscene I have choose 4x (2560x1200) becouse with this size the aspect ratio is keeped.

In my personal experience I don't raccomand to upscale to 2x (1280x800) becouse is too low or more then 4x or force to FullHD to avoid the risk to lose the aspect ratio.

This is the my point of view.
Last edited by marcop000 on Wed Jan 13, 2021 6:21 pm, edited 1 time in total.

PopoPirat
Cadet 4th Class
Posts: 12
Joined: Sun Dec 27, 2020 12:09 pm

Post by PopoPirat » Wed Jan 13, 2021 5:43 pm

I find 2560x1600 (400% increase without crop) with 30fps would be the sweet spot. We can have better quality videos with the same filesize thanks to MP4 superior compression over AVI. At least that's what my test result is.

The 4K is just a test and proof of concept. I might keep it on Youtube for the fans to enjoy.

Marco and I will split work and post results soon.

Post Reply