Page 1 of 5

[OPTing] texture limits

Posted: Sat Jan 11, 2014 5:32 pm
by JeremyaFr
Hello,
I think I have found a way to break the texture size limit (not fully tested).

I tested a few models and they seem to work:
  • an opt with 9 texture sizes from 16x16 to 4096x4096
  • an opt with 200 256x256 textures
  • an opt with 200 512x512 textures
  • an opt with 200 1024x1024 textures
  • an opt with 50 meshes (to test the mesh count limit)
To test these opt models, you will need to modify xwingalliance.exe:
At offset 19564A replace 81790C000100007717 with 909090909090909090
At offset 1956C2 replace 817A10000100007717 with 909090909090909090

This will disable the texture size check (If the texture size is greater than 256, the game set it to 256).

I've only tested textures with a single mipmap level, so the max mipmap levels count is probably still 6.

Re: [OPTing] texture limits

Posted: Sun Jan 12, 2014 5:34 pm
by JeremyaFr
After the texture size limit (256x256), I have just tested the textures count limit (200).

I set the max textures count from 200 to 1024:
At offset 0CC432 replace 2003 with 0010
At offset 0CC43A replace 2803 with 0810
At offset 0CC445 replace 2C03 with 0C10
At offset 0CC44C replace 2003 with 0010
At offset 0CC466 replace C800 with 0004
At offset 0CC497 replace 3003 with 1010
At offset 0CC4C9 replace 3003 with 1010
At offset 0CC4E6 replace 3003 with 1010
At offset 0CC4EE replace 2003 with 0010

I tested an opt with 1000 32x32 textures.

Re: [OPTing] texture limits

Posted: Mon Jan 13, 2014 8:09 am
by Bman
Awesome. You have our attention JeremyaFr. On a side note, assuming these all come to fruition after testing, is there a way to change the 8-bit (256 color resolution limit) of textures to 24-bit high color ?

Re: [OPTing] texture limits

Posted: Fri Jan 17, 2014 8:48 pm
by Rasalas
Hello everybody!

JeremyaFr, I tested your .exe-modifications and the opts you provided. It worked like a charm! This is really great stuff. :gott:
To bad this hasn't been discovered 10 years ago. I hope your discovery will lead to some more activity of this community.

Wow, after 12 years of forum lurking I finally had to register to say something. :shock:

Re: [OPTing] texture limits

Posted: Sat Jan 18, 2014 12:28 pm
by JeremyaFr
Bman wrote:Is there a way to change the 8-bit (256 color resolution limit) of textures to 24-bit high color ?
It might that I have found a way to do this. I will test that.

Re: [OPTing] texture limits

Posted: Sat Jan 18, 2014 9:23 pm
by Q
JeremyaFr wrote:Hello,
I think I have found a way to break the texture size limit (not fully tested).

I tested a few models and they seem to work:
  • an opt with 9 texture sizes from 16x16 to 4096x4096
  • an opt with 200 256x256 textures
  • an opt with 200 512x512 textures
  • an opt with 200 1024x1024 textures
  • an opt with 50 meshes (to test the mesh count limit)
To test these opt models, you will need to modify xwingalliance.exe:
At offset 19564A replace 81790C000100007717 with 909090909090909090
At offset 1956C2 replace 817A10000100007717 with 909090909090909090

This will disable the texture size check (If the texture size is greater than 256, the game set it to 256).

I've only tested textures with a single mipmap level, so the max mipmap levels count is probably still 6.
Wow, you are a genius! I can't believe no one has thought of this before, this is awesome!

Re: [OPTing] texture limits

Posted: Sun Jan 19, 2014 3:48 pm
by JeremyaFr
Great News!
I have tried to replace the 8 bpp indexed texture format with the 32 bpp ARGB texture format and it seems to work.

Re: [OPTing] texture limits

Posted: Wed Jan 22, 2014 6:51 am
by Bman
Woo-hoo! 32-bit color bitmap textures! What about 24-bit color bitmap textures?
Is that with software mode and accelerated 3D HW (and mip-mapping) mode for any given .opt file ?

Jumping ahead, could this allow importing higher quality .bmp files to replace any .dat files in \Resdata folder ? :-)

Re: [OPTing] texture limits

Posted: Wed Jan 22, 2014 2:56 pm
by JeremyaFr
24-bit RGB texture can be handled with 32-bit ARGB texture by setting A to 255.
If we modify the game to use 24-bit textures, the textures without an alpha map will be in R8G8B8, but the textures with an alpha map will remain in A4R4G4B4.
If we modify the game to use 32-bit textures, the textures with or without an alpha map will be in A8R8G8B8 (with A = 255 for textures without an alpha map).

The modifications works on hardware mode (the game may crash when software mode is used).
They require that all the textures of all the .opt files are in 32-bit ARGB format with a full mipmap chain. Otherwise the game may crash.

These modifications only alter the loading of opt files. They do not impact the loading of other resources (dat, bmp, ...).

Code: Select all

*textures bpp 8 to 32
At offset 1964C5 replace 8A028801 with 8B028901
At offset 1964D1 replace 01 with 04
At offset 1964E0 replace 01 with 04
At offset 1955BE replace 8B4804 with 909090
At offset 1955C1 replace 898DE8FEFFFF83BDE8FEFFFF00751A with C785E8FEFFFF00000000909090EB1A
At offset 195D30 replace 755E with EB5E
At offset 040ED9 replace 8D040B with 8D0499
At offset 040F16 replace 08 with 20
At offset 040F1A replace 74098D140B8B4C24208911 with C1A43CFC05000002909090
At offset 041064 replace 399424080B0000 with BE03000000EB56
At offset 198A2F replace 8B484C83E120 with 8B4864C1E918
At offset 198AED replace 83E108 with 83E128

Re: [OPTing] texture limits

Posted: Thu Jan 23, 2014 9:16 pm
by JeremyaFr
For those who want to test that, here are a tool to apply patches and a tool to convert the textures.

EDIT: updated to support Windows XP.
Requirements:
Windows 32/64 bits with the .NET framework 4.0.
(Windows XP, Windows Vista, Windows 7, Windows 8, or superior)

EDIT: link removed
These tools are now part of XwaOptEditor.

Re: [OPTing] texture limits

Posted: Fri Jan 24, 2014 3:01 am
by Bman
Thanks Jeremy, this is great.

Re: [OPTing] texture limits

Posted: Sun Jan 26, 2014 11:02 am
by Ramshu
Since we are breaking limits ... I would like to propose something, as we all know, in multiplayer a "free view" inside cockpit is disabled and would be very well remove that detail.

Re: [OPTing] texture limits

Posted: Wed Feb 05, 2014 8:04 am
by Bman
Jeremy, when using your test tools, I'm also getting errors. First, applying the XwaExePatcher to game engine for offset conversions I get this error:
XwaExePatcher.PNG

Re: [OPTing] texture limits

Posted: Wed Feb 05, 2014 8:10 am
by Bman
When I tested the ModelConverter patch to change AssaultTransport.opt textures from 8-bit to 32-bit, I get the following when viewing the model with the OptGL tool. Maybe that's the cause, but assuming it can display 32-bit textures. Having said that, I have not tested this .opt in-game (yet):

Re: [OPTing] texture limits

Posted: Wed Feb 05, 2014 3:19 pm
by JeremyaFr
Concerning XwaOptPatcher, can you verify that the program has not been blocked and that it have access to the temporary folder?

Concerning ModelConverter, it's not a bug, it's a feature. The program converts texture data from 8 bits to 32 bits. When 32 bits colors are interpreted as 8 bits index, textures are displayed like above.

Re: [OPTing] texture limits

Posted: Thu Feb 06, 2014 8:45 am
by Bman
Regarding ModelConverter, understood. Great job. Thanks.

Regarding XwaOptPatcher, nothing is blocking it to my knowledge. It seems it wants to access the csc.exe resource file in the Windows .net Framework subfolder. I've moved your unzip folder and tools to different directories. Tried turning Read-only folder attributes off, but still getting same dos-box error message. I've been running it in your default \Tools directory as shown above in the first image. Did I unzip this correctly ?

Re: [OPTing] texture limits

Posted: Thu Feb 06, 2014 5:12 pm
by JeremyaFr
Yes you did.
I've modified the way to parse the xml file.
Hope this will work. If not, I will rewrite the program to use a simple txt file.

Re: [OPTing] texture limits

Posted: Fri Feb 07, 2014 3:45 am
by Bman
Sorry, no go. Your latest XwaOptPatcher started to run it's course, but my security software immediately removed it and flagged it as Sonar.Hueristic.112 Trojan. No worries. In the meantime I can use my Be.HexEditor manually, but my Offsets are slightly different, but show the same Hex Code strings. i.e. Offset: 1964C5 vs. 1964C0 ... Thanks.

Re: [OPTing] texture limits

Posted: Fri Feb 07, 2014 1:38 pm
by JeremyaFr
It's a false positive.
Here is virustotal check:
XwaOptTestTools.zip
XwaExePatcher.exe
ModelConverter.exe
JeremyAnsel.Xwa.Opt4.dll

Re: [OPTing] texture limits

Posted: Sat Feb 08, 2014 2:38 am
by Bman
Thanks, assumed it was false. I'll have to reinstall from the .zip file and fix my security settings for it. A lot faster process.
Setting texture size limit for each bitmap file to the maximum 1024x1024, and the maximum bitmap files per .opt 1,024 is good universal standard going forward for community. Good idea. A lot of flexibility there. And portions of bitmap files can be used if the whole size is not needed by an author for his craft. Wondering if there is a trade off though for slowing down game engine.

Idea: how about consolidating these two tools as part of your main Opting tool ? Separate modules... maybe a pull down menu with a different screen? Also the ability for users to change texture files from an Optional 8-bit to 32-bit format or vice-versa would be great.
But people can use their own paint programs to do that too. Might put a flag in there so if it detects someone is using both 8-bit and 32-bit textures, it will popup a message dialog box and warn the user of incompatibility. I suspect authors could release future opts using both default 8-bit and separate 32-bit high detailed colored models.

Re: [OPTing] texture limits

Posted: Sun Feb 09, 2014 11:57 pm
by JeremyaFr
For the performance, the limit will be the available memory (system and video). You can have a few large textures or a lot of small textures.
For the opting tool, I note your ideas.

Re: [OPTing] texture limits

Posted: Mon Feb 10, 2014 12:32 am
by JeremyaFr
I tested an opt with 50 meshes of 3200 triangles.
flightscreen0.png
framerate, SceneNumVerts, SceneNumTris, SceneNumTexChanges, SceneNumStateChanges
XWA Framerate cap
meshes_50_40_40.7z
This opt doesn't require any modification of the exe.

Re: [OPTing] texture limits

Posted: Thu Mar 13, 2014 7:01 pm
by JeremyaFr
The exe modifications for breaking the texture limits works. I haven't notice any problem with them.
So I have included the two testing tools in the main XwaOptEditor.

Note that to fully enjoy 32-bit textures, you will have to use Reimar's 32-bit mode hack.

Re: [OPTing] texture limits

Posted: Fri Mar 14, 2014 5:51 am
by Bman
Thanks Jeremy. Great work.
Ok, so to re-cap all the steps in the following order . . .

1.) Install the appropriate video/graphics card drivers (directX9.0C compatible with this game) if needed.
2.) Install default XWA game from scratch.
3.) Install the LA/TG XWA 2.02 patch.
4.) Install XWAUP Pack/models or DS's UCP (optional step)
5.) Install a Mod Conversion Project: i.e. TFTC, TBPP, XWC, etc. (optional step)
6.) Run ModelConverter.exe from the XwaOptEditor Program to change all existing .opt files in ...XWA/Flightmodels folder to 32-bit high colored textures one-time--if needed. A little time consuming but worth it. :-)

7.) Run XwaExePatcher.exe one-time, from the XwaOptEditor Program (or manually edit offsets), to allow "the use" of 32-bit high-colored .opt models that were just changed.

8.) Apply Reimar's 32bitmode.bat and/or ChangeRes.bat files (or manually edit offsets) to match the desired resolution of your LCD monitor/HDTV. (optional step?)

9.) Run the game (or mod i.e.--TFTC)

So are you suggesting Step 8 be more than an optional step?
Also someone please correct me if the steps are out of order or missing. Thanks

Re: [OPTing] texture limits

Posted: Fri Mar 14, 2014 8:05 pm
by JeremyaFr
Applying Reima's 32bitmode.bat is not required to use 32-bit textures. But, since the game engine is 16 bits, if you don't apply it, then Direct3D will display textures in 16 bits.

Regarding textures greater than 256x256, their name must begin with an underscore '_'.

To convert textures from command line, you can run this PowerShell script:

Code: Select all

# Convert textures from 8 bits to 32 bits
# replace JeremyAnsel.Xwa.Opt.dll with JeremyAnsel.Xwa.Opt4.dll to use the .NET Framework 4.0 instead of 4.5
Add-Type -Path JeremyAnsel.Xwa.Opt.dll

foreach( $file in Get-ChildItem "FLIGHTMODELS\*.OPT" )
{
    Write-Host $file

    $opt = [JeremyAnsel.Xwa.Opt.OptFile]::FromFile( $file )

    Write-Host $opt.TexturesBitsPerPixel "bits"

    if( $opt.TexturesBitsPerPixel -ne 32 )
    {
        Write-Host "convert 8 -> 32"
        $opt.ConvertTextures8To32()
        $opt.Save($opt.FileName)
    }
}
To apply the patch from command line, you can run this PowerShell script:

Code: Select all

# Apply patcher.xml to XWINGALLIANCE.EXE
Add-Type -Path JeremyAnsel.Xwa.ExePatcher.dll

$patcher = [JeremyAnsel.Xwa.ExePatcher.Patcher]::Read($PSScriptRoot + "\patcher.xml")

# $true to apply the new values, $false to restore the original values
$patcher.Apply( $PSScriptRoot + "\XWINGALLIANCE.EXE", $true )
After step 9.), I suggest:
10.) Check the game configuration: use of hardware mode and other settings.