[OPTing] texture limits

A Forum dedicated to the Suggestion, Creation and Editing of XWA Dynamic Link Library Files

Moderator: JeremyaFr

[OPTing] texture limits

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Sat Jan 11, 2014 5:32 pm

Hello,
I think I have found a way to break the texture size limit (not fully tested).

I tested a few models and they seem to work:
  • an opt with 9 texture sizes from 16x16 to 4096x4096
  • an opt with 200 256x256 textures
  • an opt with 200 512x512 textures
  • an opt with 200 1024x1024 textures
  • an opt with 50 meshes (to test the mesh count limit)
To test these opt models, you will need to modify xwingalliance.exe:
At offset 19564A replace 81790C000100007717 with 909090909090909090
At offset 1956C2 replace 817A10000100007717 with 909090909090909090

This will disable the texture size check (If the texture size is greater than 256, the game set it to 256).

I've only tested textures with a single mipmap level, so the max mipmap levels count is probably still 6.
You do not have the required permissions to view the files attached to this post.
Last edited by JeremyaFr on Sun Jan 12, 2014 5:40 pm, edited 1 time in total.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Sun Jan 12, 2014 5:34 pm

After the texture size limit (256x256), I have just tested the textures count limit (200).

I set the max textures count from 200 to 1024:
At offset 0CC432 replace 2003 with 0010
At offset 0CC43A replace 2803 with 0810
At offset 0CC445 replace 2C03 with 0C10
At offset 0CC44C replace 2003 with 0010
At offset 0CC466 replace C800 with 0004
At offset 0CC497 replace 3003 with 1010
At offset 0CC4C9 replace 3003 with 1010
At offset 0CC4E6 replace 3003 with 1010
At offset 0CC4EE replace 2003 with 0010

I tested an opt with 1000 32x32 textures.

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Mon Jan 13, 2014 8:09 am

Awesome. You have our attention JeremyaFr. On a side note, assuming these all come to fruition after testing, is there a way to change the 8-bit (256 color resolution limit) of textures to 24-bit high color ?
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

Rasalas
Cadet 2nd Class
Posts: 50
Joined: Fri Jan 17, 2014 8:09 pm

Post by Rasalas » Fri Jan 17, 2014 8:48 pm

Hello everybody!

JeremyaFr, I tested your .exe-modifications and the opts you provided. It worked like a charm! This is really great stuff. :gott:
To bad this hasn't been discovered 10 years ago. I hope your discovery will lead to some more activity of this community.

Wow, after 12 years of forum lurking I finally had to register to say something. :shock:

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Sat Jan 18, 2014 12:28 pm

Bman wrote:Is there a way to change the 8-bit (256 color resolution limit) of textures to 24-bit high color ?
It might that I have found a way to do this. I will test that.

User avatar
Q
Ensign
Posts: 456
Joined: Sat Dec 28, 2002 12:01 am
Contact:

Post by Q » Sat Jan 18, 2014 9:23 pm

JeremyaFr wrote:Hello,
I think I have found a way to break the texture size limit (not fully tested).

I tested a few models and they seem to work:
  • an opt with 9 texture sizes from 16x16 to 4096x4096
  • an opt with 200 256x256 textures
  • an opt with 200 512x512 textures
  • an opt with 200 1024x1024 textures
  • an opt with 50 meshes (to test the mesh count limit)
To test these opt models, you will need to modify xwingalliance.exe:
At offset 19564A replace 81790C000100007717 with 909090909090909090
At offset 1956C2 replace 817A10000100007717 with 909090909090909090

This will disable the texture size check (If the texture size is greater than 256, the game set it to 256).

I've only tested textures with a single mipmap level, so the max mipmap levels count is probably still 6.
Wow, you are a genius! I can't believe no one has thought of this before, this is awesome!
"I like work; it fascinates me. I can sit and look at it for hours."

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Sun Jan 19, 2014 3:48 pm

Great News!
I have tried to replace the 8 bpp indexed texture format with the 32 bpp ARGB texture format and it seems to work.

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Wed Jan 22, 2014 6:51 am

Woo-hoo! 32-bit color bitmap textures! What about 24-bit color bitmap textures?
Is that with software mode and accelerated 3D HW (and mip-mapping) mode for any given .opt file ?

Jumping ahead, could this allow importing higher quality .bmp files to replace any .dat files in \Resdata folder ? :-)
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Wed Jan 22, 2014 2:56 pm

24-bit RGB texture can be handled with 32-bit ARGB texture by setting A to 255.
If we modify the game to use 24-bit textures, the textures without an alpha map will be in R8G8B8, but the textures with an alpha map will remain in A4R4G4B4.
If we modify the game to use 32-bit textures, the textures with or without an alpha map will be in A8R8G8B8 (with A = 255 for textures without an alpha map).

The modifications works on hardware mode (the game may crash when software mode is used).
They require that all the textures of all the .opt files are in 32-bit ARGB format with a full mipmap chain. Otherwise the game may crash.

These modifications only alter the loading of opt files. They do not impact the loading of other resources (dat, bmp, ...).

Code: Select all

*textures bpp 8 to 32
At offset 1964C5 replace 8A028801 with 8B028901
At offset 1964D1 replace 01 with 04
At offset 1964E0 replace 01 with 04
At offset 1955BE replace 8B4804 with 909090
At offset 1955C1 replace 898DE8FEFFFF83BDE8FEFFFF00751A with C785E8FEFFFF00000000909090EB1A
At offset 195D30 replace 755E with EB5E
At offset 040ED9 replace 8D040B with 8D0499
At offset 040F16 replace 08 with 20
At offset 040F1A replace 74098D140B8B4C24208911 with C1A43CFC05000002909090
At offset 041064 replace 399424080B0000 with BE03000000EB56
At offset 198A2F replace 8B484C83E120 with 8B4864C1E918
At offset 198AED replace 83E108 with 83E128

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Thu Jan 23, 2014 9:16 pm

For those who want to test that, here are a tool to apply patches and a tool to convert the textures.

EDIT: updated to support Windows XP.
Requirements:
Windows 32/64 bits with the .NET framework 4.0.
(Windows XP, Windows Vista, Windows 7, Windows 8, or superior)

EDIT: link removed
These tools are now part of XwaOptEditor.
Last edited by JeremyaFr on Thu Mar 13, 2014 7:03 pm, edited 3 times in total.

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Fri Jan 24, 2014 3:01 am

Thanks Jeremy, this is great.
Last edited by Bman on Wed Feb 05, 2014 7:49 am, edited 1 time in total.
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
Ramshu
Cadet 2nd Class
Posts: 74
Joined: Sat Sep 21, 2013 10:42 am
Contact:

Post by Ramshu » Sun Jan 26, 2014 11:02 am

Since we are breaking limits ... I would like to propose something, as we all know, in multiplayer a "free view" inside cockpit is disabled and would be very well remove that detail.

"Nasios pa vola"

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Wed Feb 05, 2014 8:04 am

Jeremy, when using your test tools, I'm also getting errors. First, applying the XwaExePatcher to game engine for offset conversions I get this error:
XwaExePatcher.PNG
You do not have the required permissions to view the files attached to this post.
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Wed Feb 05, 2014 8:10 am

When I tested the ModelConverter patch to change AssaultTransport.opt textures from 8-bit to 32-bit, I get the following when viewing the model with the OptGL tool. Maybe that's the cause, but assuming it can display 32-bit textures. Having said that, I have not tested this .opt in-game (yet):
You do not have the required permissions to view the files attached to this post.
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Wed Feb 05, 2014 3:19 pm

Concerning XwaOptPatcher, can you verify that the program has not been blocked and that it have access to the temporary folder?

Concerning ModelConverter, it's not a bug, it's a feature. The program converts texture data from 8 bits to 32 bits. When 32 bits colors are interpreted as 8 bits index, textures are displayed like above.

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Thu Feb 06, 2014 8:45 am

Regarding ModelConverter, understood. Great job. Thanks.

Regarding XwaOptPatcher, nothing is blocking it to my knowledge. It seems it wants to access the csc.exe resource file in the Windows .net Framework subfolder. I've moved your unzip folder and tools to different directories. Tried turning Read-only folder attributes off, but still getting same dos-box error message. I've been running it in your default \Tools directory as shown above in the first image. Did I unzip this correctly ?
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Thu Feb 06, 2014 5:12 pm

Yes you did.
I've modified the way to parse the xml file.
Hope this will work. If not, I will rewrite the program to use a simple txt file.

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Fri Feb 07, 2014 3:45 am

Sorry, no go. Your latest XwaOptPatcher started to run it's course, but my security software immediately removed it and flagged it as Sonar.Hueristic.112 Trojan. No worries. In the meantime I can use my Be.HexEditor manually, but my Offsets are slightly different, but show the same Hex Code strings. i.e. Offset: 1964C5 vs. 1964C0 ... Thanks.
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Fri Feb 07, 2014 1:38 pm

It's a false positive.
Here is virustotal check:
XwaOptTestTools.zip
XwaExePatcher.exe
ModelConverter.exe
JeremyAnsel.Xwa.Opt4.dll

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Sat Feb 08, 2014 2:38 am

Thanks, assumed it was false. I'll have to reinstall from the .zip file and fix my security settings for it. A lot faster process.
Setting texture size limit for each bitmap file to the maximum 1024x1024, and the maximum bitmap files per .opt 1,024 is good universal standard going forward for community. Good idea. A lot of flexibility there. And portions of bitmap files can be used if the whole size is not needed by an author for his craft. Wondering if there is a trade off though for slowing down game engine.

Idea: how about consolidating these two tools as part of your main Opting tool ? Separate modules... maybe a pull down menu with a different screen? Also the ability for users to change texture files from an Optional 8-bit to 32-bit format or vice-versa would be great.
But people can use their own paint programs to do that too. Might put a flag in there so if it detects someone is using both 8-bit and 32-bit textures, it will popup a message dialog box and warn the user of incompatibility. I suspect authors could release future opts using both default 8-bit and separate 32-bit high detailed colored models.
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Sun Feb 09, 2014 11:57 pm

For the performance, the limit will be the available memory (system and video). You can have a few large textures or a lot of small textures.
For the opting tool, I note your ideas.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Mon Feb 10, 2014 12:32 am

I tested an opt with 50 meshes of 3200 triangles.
flightscreen0.png
framerate, SceneNumVerts, SceneNumTris, SceneNumTexChanges, SceneNumStateChanges
XWA Framerate cap
meshes_50_40_40.7z
This opt doesn't require any modification of the exe.
You do not have the required permissions to view the files attached to this post.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Thu Mar 13, 2014 7:01 pm

The exe modifications for breaking the texture limits works. I haven't notice any problem with them.
So I have included the two testing tools in the main XwaOptEditor.

Note that to fully enjoy 32-bit textures, you will have to use Reimar's 32-bit mode hack.

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Fri Mar 14, 2014 5:51 am

Thanks Jeremy. Great work.
Ok, so to re-cap all the steps in the following order . . .

1.) Install the appropriate video/graphics card drivers (directX9.0C compatible with this game) if needed.
2.) Install default XWA game from scratch.
3.) Install the LA/TG XWA 2.02 patch.
4.) Install XWAUP Pack/models or DS's UCP (optional step)
5.) Install a Mod Conversion Project: i.e. TFTC, TBPP, XWC, etc. (optional step)
6.) Run ModelConverter.exe from the XwaOptEditor Program to change all existing .opt files in ...XWA/Flightmodels folder to 32-bit high colored textures one-time--if needed. A little time consuming but worth it. :-)

7.) Run XwaExePatcher.exe one-time, from the XwaOptEditor Program (or manually edit offsets), to allow "the use" of 32-bit high-colored .opt models that were just changed.

8.) Apply Reimar's 32bitmode.bat and/or ChangeRes.bat files (or manually edit offsets) to match the desired resolution of your LCD monitor/HDTV. (optional step?)

9.) Run the game (or mod i.e.--TFTC)

So are you suggesting Step 8 be more than an optional step?
Also someone please correct me if the steps are out of order or missing. Thanks
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Fri Mar 14, 2014 8:05 pm

Applying Reima's 32bitmode.bat is not required to use 32-bit textures. But, since the game engine is 16 bits, if you don't apply it, then Direct3D will display textures in 16 bits.

Regarding textures greater than 256x256, their name must begin with an underscore '_'.

To convert textures from command line, you can run this PowerShell script:

Code: Select all

# Convert textures from 8 bits to 32 bits
# replace JeremyAnsel.Xwa.Opt.dll with JeremyAnsel.Xwa.Opt4.dll to use the .NET Framework 4.0 instead of 4.5
Add-Type -Path JeremyAnsel.Xwa.Opt.dll

foreach( $file in Get-ChildItem "FLIGHTMODELS\*.OPT" )
{
    Write-Host $file

    $opt = [JeremyAnsel.Xwa.Opt.OptFile]::FromFile( $file )

    Write-Host $opt.TexturesBitsPerPixel "bits"

    if( $opt.TexturesBitsPerPixel -ne 32 )
    {
        Write-Host "convert 8 -> 32"
        $opt.ConvertTextures8To32()
        $opt.Save($opt.FileName)
    }
}
To apply the patch from command line, you can run this PowerShell script:

Code: Select all

# Apply patcher.xml to XWINGALLIANCE.EXE
Add-Type -Path JeremyAnsel.Xwa.ExePatcher.dll

$patcher = [JeremyAnsel.Xwa.ExePatcher.Patcher]::Read($PSScriptRoot + "\patcher.xml")

# $true to apply the new values, $false to restore the original values
$patcher.Apply( $PSScriptRoot + "\XWINGALLIANCE.EXE", $true )
After step 9.), I suggest:
10.) Check the game configuration: use of hardware mode and other settings.

Post Reply