[OPTing] texture limits

A Forum dedicated to the Suggestion, Creation and Editing of XWA Dynamic Link Library Files

Moderator: JeremyaFr

Re: [OPTing] texture limits

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Sun Mar 16, 2014 3:11 am

Can those lines code/scripts be copied and saved to a text file say for example using Notepad.exe ? If so then we can change the file extension from *.txt to *.bat and double-click to run as a batch file.

The first script(batch) file would need to be in \Flightmodels folder with your ....Opt(4).dll file to convert each existing .opt files to 32-bit color all at once rather than piece meal.
The second script(batch) file would need to be in main XWA directly with your ....ExePatcher.dll file.

Do I have all of this understood correctly ? Thanks.
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Sun Mar 16, 2014 11:20 pm

PowerShell scripts can be written in a text file with the '.ps1' extension. '.ps1' files can be executed like a batch file by double-clicking on them.

The OPTs can be converted and the patch apply from XwaOptEditor or from command-line or manually.
From command-line, the scripts can be executed from the shell or from a '.ps1' text file.
From the interface of XwaOptEditor, you can convert a single OPT or all the OPTs of a folder.
The patch can be manually applied with an hex editor.
XwaOptEditor_ConvertTextures.png
The files needed to run the script are:
  • ....Opt(4).dll to convert textures
  • ....ExePatcher.dll and patcher.xml to apply the patch
You can place these files whereever you want by writting the correct path in the script.
You do not have the required permissions to view the files attached to this post.

User avatar
Atx
Lieutenant Commander
Posts: 1217
Joined: Mon Feb 11, 2002 12:01 am

Post by Atx » Mon Mar 17, 2014 1:52 pm

OK, I take what I said back - I had no idea you found the actual code in the EXE itself and NOP'd it!
Good stuff and yeah this would revolutionize the game itself BIG TIME!
One thing, I believe the game does some sort of caching of textures in real time as the missions progress which caused a bit of a stutter/lag in-game. I wonder if machine specs are fast enough without this becoming noticable? I'm guessing so since I was running a 500MHz machine back in the day compared to the monsters now.

Again, good stuff.
The Navbuoy is closed...long live the XWAU!!!

Sadsailor
Cadet 4th Class
Posts: 21
Joined: Tue Nov 08, 2011 5:19 am

Post by Sadsailor » Fri Mar 21, 2014 5:16 pm

This is wonderful news, JeremyaFr. Having you around is almost as good as the source code. Now if only we could do the same thing for OPT dimension limits.

User avatar
DTM
Fleet Admiral (Administrator)
Posts: 2119
Joined: Tue Apr 22, 2003 11:01 pm
Contact:

Post by DTM » Sun Mar 30, 2014 8:27 am

Good news! What about the DAT size limits? To upgrade the backdrop image size could be very important to refresh this old game....

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Sun Mar 30, 2014 4:40 pm

DAT files can contain 8-bit, 16-bit, or 32-bit colors images, with or without alpha.
When the game engine load dat images, it convert them to 16 bits and limit their size to 256x256.
Those limits may be removed in a manner similar to the opt textures limits.
I will work on that.

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Mon Mar 31, 2014 3:21 am

.
Last edited by Bman on Sun Mar 29, 2015 4:30 am, edited 1 time in total.
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
DTM
Fleet Admiral (Administrator)
Posts: 2119
Joined: Tue Apr 22, 2003 11:01 pm
Contact:

Post by DTM » Wed Apr 02, 2014 5:26 pm

To use large backdrops could be the key to complete Frakel's (aka Dodonna) work! He created beutiful skies using multiple 256x256 backdrops. See topic

The limit was the number of backdrops used...but with greater backdrops it could be possible to have beutiful skies using few inages and even a new strafield...

Unfortunately my programming skills are zero, and I also find it hard to understand your explanations in English...sorry

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Thu Apr 03, 2014 5:57 am

.
Last edited by Bman on Sun Mar 29, 2015 4:29 am, edited 1 time in total.
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Thu Apr 03, 2014 6:58 pm

*** dat files ***
In dat files, images are 8-bits indexed colors (each color is 24-bit R8G8B8 ), with or without an 8-bit or 1-bit alpha channel.

The main image types in dat files are:
  • type 7
    8-bit indexed colors and 1-bit alpha, rle compressed
  • type 23
    8-bit indexed colors and 8-bit alpha, rle compressed
  • type 24
    8-bit indexed colors and 8-bit alpha
  • type 25
    similar to type 24
There is no dat file that contain an image type 25. So I will use this type to store 32-bit (A8R8G8B8) images.

Concourse images use type 7, 23.
In-flight images use type 7, 24, 25.

If a concourse image is not type 7 or 23, then the image is not drawn.
If an in-flight image is not type 7, 24 or 24, then the image is not loaded into a texture and the game crashes.

When an in-flight image is loaded, it is converted to one of the following formats:
  • 32 bits A8R8G8B8 (type 1)
  • 16 bits A4R4G4B4 (type 4)
  • 16 bits A1R5G5B5
  • 16 bits R5G6B5
  • 16 bits R5G5B5
When 8-bit alpha is converted to 1-bit, values below 0x4B (75) are considered as transparent, values above or equal are considered as opaque.
Since the game engine is 16-bit, the code to convert images to 32-bit is never used.

*** 32-bit images ***
The 32-bit image patch will concern in-flight images, not concourse images.
The size limit (256x256) concerns only in-flight images, and it is probably already removed with the opt texture size limit patch.

When creating 32-bit image, the image type will be 25. The format will be A8R8G8B8.
The patch that allow usage of 32-bit textures in opt will be a requirement.
An other patch will be needed.
I will activate the function that convert image type 24 (or 7) to 32 bits. So there will be no need to convert existing images to type 25 (32-bit).

*** applying opt textures patched ***
There are 3 ways to apply the patches:
  • from the GUI interface: XwaOptEditor
  • from the command line or programming environnement
  • manually

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Sat Apr 05, 2014 1:43 am

In summary, you're saying the concourse and historical combat mission image screens have to remain as 256 color bitmap source files when importing and replacing items inside the .dat files like "FrontEnd.dat" etc. Well, at least in-flight image issue seems promising. Thanks
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

Tony Knightcrawler
Commander
Posts: 2263
Joined: Sat Nov 29, 2003 12:01 am
Contact:

Post by Tony Knightcrawler » Sat Apr 26, 2014 8:25 pm

Great work on all this, it really brings up some cool possibilities! :) I'm also a big fan of the mesh face-count upgrade, and the mesh-count upgrade. It'd be cool if there was a single, simple patch that did all this at once: Added the higher-color, higher-resolution image types, increased the max mesh count, increased the max poly-per-mesh count, changed all the resolutions to modern and upcoming resolutions, set it to 32-bit, maybe even added that head tracking/mouse support while flying (from a few years ago), set the max framerate to 60 or 90 FPS, and whatever else can be hacked/changed (S-Foil slots?).

Long ago there was a stigma against editing certain parts of the game, but I doubt anyone at Disney cares about what we do to this game.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Sun May 18, 2014 10:52 pm

JeremyaFr wrote: The size limit (256x256) concerns only in-flight images, and it is probably already removed with the opt texture size limit patch.
I have tested dat images larger than 256x256 :
planet.jpg
This is a single image. Its size is 1024x1024 (4 planets x 4 planets).
You do not have the required permissions to view the files attached to this post.

Gank
XWAU Member
Posts: 577
Joined: Sun Mar 31, 2002 11:01 pm
Contact:

Post by Gank » Mon May 19, 2014 2:56 am

Great work, i agree with Tony Knightcrawler, it would be great if other parts of the engine could be changed, s-foil slots, more flight group colours, more ship slots, this would be a far more useful way for you to spend your time then building in texturing capabilities which can be done with the large number of free uv mapping tools available out there.

Some of us are working on new models to take advantage of the new limits, but be some time before anythings complete, has anyone tried to convert any of the high poly models out there? The corvette and star destroyer from swgalaxies would make great test subjects i think, i dont have xwa installed but i do have the models if anyone wants to give it a shot

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Tue May 20, 2014 6:29 pm

To facilitate the application of the 'texture size limit' patch, I have created a GUI tool.
XwaDatEditor.jpg
XwaDatEditor.zip

Apply the texture size patch

When the user click on "Apply Exe Patcher", a dialog box asks for XWINGALLIANCE.EXE path and then the patch is applied to it.

The patch allows in-flight images larger than 256x256. You can apply it via a gui, or with an hexadecimal editor.

Code: Select all

<?xml version="1.0" encoding="utf-8"?>
<ArrayOfPatch>
  <Patch Name="textures size limit">
    <Item Offset="19564A" From="81790C000100007717" To="909090909090909090" />
    <Item Offset="1956C2" From="817A10000100007717" To="909090909090909090" />
  </Patch>
</ArrayOfPatch>
Create DAT files

To create DAT files, you can use X-Wing Alliance Dat Customizer (XDC), or any other dat editor.
You do not have the required permissions to view the files attached to this post.

User avatar
Atx
Lieutenant Commander
Posts: 1217
Joined: Mon Feb 11, 2002 12:01 am

Post by Atx » Fri May 23, 2014 12:09 am

Holy moses Gank is that you :shock:
Getting all the old veterans out of the woodwork. LOL.

Your texture mods are much appreciated man - good work. It takes what we have and at least brings it up to a much more current/respectable standard.

I hate tiling textures with a passion so this is a big deal...no doubt. Especially when doing large opts and landscapes this is a game-changer.
The Navbuoy is closed...long live the XWAU!!!

User avatar
DTM
Fleet Admiral (Administrator)
Posts: 2119
Joined: Tue Apr 22, 2003 11:01 pm
Contact:

Post by DTM » Sat May 24, 2014 2:29 pm

I have tested it with a 1024x1024 planet image, using the original 530 pixel large size image used to create the XWAUP planet.dat file.

Original XWAUP 256x256:
http://www.mediafire.com/view/wf13ed9lp ... an_256.jpg

New image 1024x1024:
http://www.mediafire.com/view/9ev7kbzc7 ... n_1024.jpg


It's a very good upgrade!!! :D :D :D

The only problem: the backdrop is 4x larger.
If a should make a 1024 pixel version of the XWAUP planet dat file, I should also fix all campaign mission files :hilfe: !!!

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Sat May 24, 2014 4:34 pm

Yes, if you change the backrop size, you will also have to change the backdrop scale.
Doing this manually can be tiring. So it will be better to automatize the process.

Another solution would be to change the scale of all backdrops.
This can be made with the following exe modification:

To replace the default scaling factor of 256.0f:

Code: Select all

At offset 0x1A83AC, replace 00008043 with XXXXXXXX.
XXXXXXXX is a float value. You can try with different float values to see the difference.
Here is a few possible values:
  • 512.0f (2) --> 00000044
  • 128.0f (1/2) --> 00000043
  • 64.0f (1/4) --> 00008042
  • 32.0f (1/8) --> 00000042
Last edited by JeremyaFr on Tue Dec 02, 2014 5:07 pm, edited 1 time in total.

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Mon May 26, 2014 5:54 am

.
Last edited by Bman on Tue Dec 02, 2014 8:27 am, edited 1 time in total.
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Mon Jul 07, 2014 10:02 pm

An update of Xwa Dat Editor:
added:
  • setting transparency with a color key

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Tue Dec 02, 2014 8:37 am

Hi Jeremy, can you double check to see if your "Convert opt models from 8-bit textures to 32-bit" pallettes does not affect other opting features ?
When i converted all of my opt models to 32-bit pallette, the models seem a little dark in-flight .... specifically "illumination" seems turned off or striped away. I noticed this especially while inside hangar models. Hangar ceiling is dark. It's possible I don't have enough light sources in a mission I tested like more stars or nebula FG's and their brightness. But the shadows effect seems to work more realistically than before. Thanks.
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Tue Dec 02, 2014 5:21 pm

Hi,
I have checked the XwaOptEditor zip and it seems that it uses an out-of-date version of the Opt.dll.
"illumination" is not yet active in 32-bit mode.
This feature is available in the dll for 8-bit mode. It will be available from the editor for 8-bit opt in the next version and for both 8-bit and 32-bit modes in the next one.

UPDATE for:
  • Xwa Dat Editor
  • Xwa Cbm Editor
  • Xwa Opt Editor
If you use any of these tools, please re-download them to ensure that you use an up-to-date version.

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Sat Dec 06, 2014 6:49 pm

Thanks. Question about the XWADatEditor: when I import my 32-bit colored images into a .dat file, i'm saving them as Type 24 which seems to preserve the original color better than Type 7. Problem is in-game... my background and edges of my images are a darker black than the game engine space environment. When I click on your black (color strip) to set transparency, it doesn't seem to do anything. Maybe black is different shade ?

I'm patched to 32-bit OFFSETs via your XWAOptEditor patcher file. I'm using GIMP to export my images as 32-bit X8R8B8G8 files. My backgrounds are deleted before I save. Should I set the background as the transparent alpha channel in GIMP program, or should this be done via your XWAdatEditor ? Both ? Thanks
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
JeremyaFr
XWAU Member
Posts: 3918
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Sat Dec 06, 2014 8:41 pm

To make a color transparent, you have to select the color and then click on the 'Make transparent' button.
dat_star.jpg
When importing images, it is better to use the alpha channel for transparency.
You do not have the required permissions to view the files attached to this post.

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Mon Dec 08, 2014 3:37 am

.
Last edited by Bman on Sun Mar 29, 2015 4:47 am, edited 1 time in total.
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

Post Reply