Craft limit, Starship limit, Starfighter limit, Object limit

[AngeI]

Impressive, most impressive!

[Darksaber]

How the hell did he do that I wonder

[Vince T]

Gives me ideas for a Battle over Coruscant remake ....

[Trevor]

wow.... I want that
Although, perhaps not quite so many since the conglomerate squadrons looked a bit clunky - all 20 craft or whatever moved together and in steps rather than smooth.

Trev

[Bman]

192 FG's with more than 6 wingmen per FG ????

[Driftwood]

Well... that many objects in a mission really opens up questions on how was that done, and shows the game can run smooth regardless of how many model and texture details are rendered... anybody else notice how there was an object tracker overlay that worked real time?

The question is how can this be achieved, because it would really open up performance if it could be figured out...

[Darksaber]

Yes but you probably couldn't do this with XWAUP models, it a vanilla game he's using

[Driftwood]

Yes but you probably couldn't do this with XWAUP models, it a vanilla game he's using

True maybe not, we dont know, but with 2000 original models in the game with varying levels of detail it should still be a slideshow and it doesnt appear to be. It makes me wonder if there's multiple changes involved as that's still a hell of a lot of vertices and textures loaded in to the game all at once.

The most detailed model I had ingame had 170k vertices and after 5 minutes the game just shut down randomly, there were only two ships in the skirmish, missileboat and the big one. Stationary.

Even if we assume that all 2000 ships contained 500 vertices that's still 1000000 vertices ingame not counting missiles, flares, ejected pilots, ect. The game should crash instantly. Especially with all the movement and combat.

Sad thing is we may well never know how this was achieved or if it could open up performance for us either way.

[Bman]

Anyone who has YT account could just easily ask him.

[AngeI]

To clarify, is the 192 count the number of ships or objects per region or FGs (or both)? If its FG's then given you can only have 6 per FG, the max number of ships or objects should be 1152? Unless you can modify the number of ships per FG beyond 6?

[AngeI]

Well according to the video authour:

"I repartitioned the object table with different ranges for everything. More craft, more lasers, more explosions. I'm not sure how stable it actually is, but it seemed to work fine during my tests."

[AngeI]

Double post

[Trevor]

that's cool that the author replied. Maybe Jeremy can shed some light on how that would work and if its possible to use a similar technique?
Also, given that his Video was not a slideshow, he must be optimizing model throughput, maybe any suggestions from him could help current bottlenecks?

Just throwing ideas out there incase someone isn't already working on it

Trev

[AngeI]

Well frankly we're already hitting the upper limits of FPS with all the new effects and models with just the 192 limit as it is. Increasing the limit might be situationally useful for adding say more asteroids in a mission or a technically impressive fleet display.

[Random Starfighter]

Hello, I made the video with the craft counts. After receiving a few more comments I decided to register here to answer any questions directly. I'll try to post an explanation of the patch later, and how to patch it. I still have the patch, but my notes are sparse. It's been several months so I have to relearn how it was done.

It started off an experimental mod for XvT since that's the game engine I'm familiar with. But since XWA's engine is derived from XvT I decided to try applying it here too. After some research I was able to get it to work. The concept is the same as XvT but XWA's approach is slightly more complicated because of regions. Luckily the actual patch turned out to be pretty simple. I can't vouch for its stability, but if I remember from my tests it seemed to work fine after the bugs were fixed. And there were bugs, side effects of exceeding the craft count that required further modifications in other places.

Although, perhaps not quite so many since the conglomerate squadrons looked a bit clunky - all 20 craft or whatever moved together and in steps rather than smooth.

When pushing the limits to absurdity, the bottleneck was the 192 FGs, so the only way to add more craft was to make the waves bigger. The "school of fish" behavior is a side effect of how simplistic the AI behavior is.

Well... that many objects in a mission really opens up questions on how was that done, and shows the game can run smooth regardless of how many model and texture details are rendered... anybody else notice how there was an object tracker overlay that worked real time?

The question is how can this be achieved, because it would really open up performance if it could be figured out...

There were no modifications or optimizations, just a plain vanilla install with the limit patch applied. The overlay is an external program reading the game memory in real time and counting how many objects of each class are present in the mission.

I tried using XWAU (well DSUCP before the thing happened) and that was a slideshow. As I understand it, the performance bottleneck is the CPU since pretty much everything goes through it. All the gameplay logic, all the AI behavior, determining what's visible on screen, sorting them, preparing the meshes, culling, clipping, and sorting geometry, and sending the results to the GPU. Every frame, done in software on the CPU, all in real-time, on a single thread. The new models especially seem to be more demanding than the craft counts. I'm basing that knowledge off XvT, I'm not sure what the differences are in XWA. I do know that XWA has much better framerate performance than XvT for similar craft counts, so something in the rendering pipeline is more optimized in XWA.

[Berruga]

Thank you for registering and posting in the forum. I think this is a topic that we all really would like to know much more about, and although personally I don't have the knowledge to understand engine workarounds I'm sure other team members like JeremyaFr or blue_max would love to know more about it.

Pushing the limits of X-wing Alliance is something that we all love here, so any new ways to do it is always welcome!

[Random Starfighter]

The basic explanation is everything in the game (craft, projectiles, explosions, and space objects like asteroids, etc) at its lowest level is an object. There is a large array that stores all these objects. The array is partitioned so that different classes of objects occupy certain range of slots. Craft have one section, projectiles have another, etc. XWA complicates this by adding regions, but it handles everything under the hood as long as you patch the correct counts.

The limit patch isn't much different than the 192 craft and 256 projectiles that JeremyaFr has already posted, the main difference is the bugs that need to be fixed in order to prevent crashes.

Here's the part of the patch that replaces the instructions that set the craft count, while managing some of the registers so that the remaining unpatched code works correctly. When I was running tests I wanted to be able to choose a specific amount, in this case 0xC00 (3072 craft) which is way overkill. The framerate is unplayable even on vanilla. For something closer to the video, use something like BB00080000 (0x800, 2048 craft).

Code: Select all

At offset 14C25, replace 8D1C76C705 with BB000C0000
At offset 14C2A, replace 944B7D006000 with 891D944B7D00
At offset 14C30, replace 0000C1 with 0FAFDE
At offset 14C33, replace E305 with 9090

Now for the projectile count, in this case a specific amount of 0x1000 (4096) which is way more than than my testing scenario needs, even with 3072 craft which maxes out somewhere around 3500 active projectiles. The scenario in the video can go lower, maybe around 3000.

Code: Select all

At offset 14C68, replace 750BBA1000 with B800100000
At offset 14C6D, replace 0000 with EB0A

Now for increasing the explosions from 32 to 512.

Code: Select all

At offset 14C85, replace C1E005 with C1E009

Now come the bugfixes. The friendly and enemy craft lists populate an array that has a maximum size of 192 craft. But since the craft count is vastly increased, it will overflow the array. This patch sets a hard limit of 192 slots. The downside is the craft list won't show anything above index 192. Small price to pay for not crashing.

Code: Select all

At offset BFC61, replace A1B8A37C00 with B8C0000000
At offset C0D63, replace A1B8A37C00 with B8C0000000

Another bug comes from initially clearing an array related to the officer sounds. This is based on the old craft limit of 192, and will now overflow. Again needs a hard limit. NOTE: There's a bug here that the patch doesn't account for, since the officer sounds are used elsewhere to generate craft reports (shields down, hull damaged, etc). It can introduce crashes for officer reports above a certain craft index. I'm not sure how large that array actually is or what the "safe" craft index range is.

Code: Select all

At offset 39517, replace 8B0DB8A37C00 with B9C000000090

Another bug is the message log. XWA has a legit bug where too many messages in the message log will overflow and cause undefined behavior, eventually resulting in a crash. The simplest way to avoid this is to increase the heap size of the message log array. This results in enough space for over 10,000 messages, which should be plenty even if you use multiple waves (like I did in the video) unless you go insane with even more.

Code: Select all

At offset 30216, replace 68007D0000 with 6880A21900

The final bug concerns an array that is used for AI handling. In XvT it's for turret targeting. Without investigating more I can only assume it's the same thing in XWA. I don't know how to resize this, so my solution was to move it somewhere else. In this case I moved it near the end of the FlightGroup array. However, this trade-off does subtract from the total number of FGs you can have in your mission (I don't recall exactly how many, I think in this example it's 180 max FGs instead of 192).

Code: Select all

At offset A95AA replace 8A847980F19A00 with 8A8479E8E28A00
At offset A9AD5 replace C6847080F19A0000 with C68470E8E28A0000
At offset A9B5F replace C6847180F19A0001 with C68471E8E28A0001

So with all of that said, there can be some alterations and optimizations to the numbers depending on how high or low you really want to go. Lower numbers may improve performance slightly, as most of the game logic loops over the entire section of the object list in order to perform frame updates.

Remember that the officer sound array isn't patched here. This exercise was more about science rather than gameplay.

Also remember these craft counts are for vanilla. Expect heavy performance penalties if you run XWAU with these high counts.

Edit: forgot to mention a couple things. XvT uses a certain craft index as a special condition for certain orders. I'm not sure if that's the case for XWA, but it could introduce some subtle AI bugs. The sensor display has its own limit, which is not changed here. None of this has been tested in multiplayer, nor has it been tested extensively in singleplayer. It's possible there are more bugs or crashes that I haven't encountered yet.

[Bman]

Random, that is awesome. Thank you for posting this! This gives a lot of material to work with that Jeremy and others might be able to fine tune and enhance. I would think even having a smaller, scalable setting like 200 to 700 FG's for fleet battles say i.e. battle of Endor or Courscant for example would be realistic vs. 2048. Definitely LOD meshes would be needed for .opt models. Having said all that, have a question regarding the sound files. Right now XWA is limited on how it plays the flybyCraft.wav sounds. For example, say you're flying your X-wing and you have 3 tie-fighters that pass by your craft all within one/two seconds or less separated by distance and direction. The game will only play one FlyByTi.wav file in it's entirety (sequentially) before it ques for next flyby craft a few seconds later. How would you get the other two tiefighter crafts to play their FlyByTi.wav files concurrently and independently from the first tiefighter craft? Put another way, how would you invoke a separate instance of each object's flyby.wav file based on their own separate trigger distance/timing from the player's craft, so that you would realistically hear all three tie fighters and other craft flying past you independently ? Does that make sense? Maybe you've already accomplished that. I'll have to watch your video again. Thanks.

[JeremyaFr]

@Random Starfighter

Thank you very much for the explanations

[Avarice1987]

Short question about the ship limit

Some players said that the engine is exploited in a way that the game with mods doesn't run smoothly anymore. Is this still the case or was this fixable for the mods?

[Random Starfighter]

Regarding the fly-by sounds, yeah the games are specifically programmed not to play the same fly-by sound again if there's a channel already playing it.

You can fix that with this patch. Instead of exiting the function if there's a matching sound, it will instead proceed to queue it up.

Code: Select all

At offset 3B4E9, replace 7407 with EB07

I've been running some tests with 700 craft. It seems to be perfectly playable in vanilla. In XWAU the framerate plummets if there's too many ships, especially capital ships, on screen. It might take some testing to find an acceptable balance of performance between fleet size and frame rate.

For 700 craft (0x02BC), the first line of the craft limit can be changed, but you'll still need the rest:

Code: Select all

At offset 14C25, replace 8D1C76C705 with BBBC020000     <--
At offset 14C2A, replace 944B7D006000 with 891D944B7D00
At offset 14C30, replace 0000C1 with 0FAFDE
At offset 14C33, replace E305 with 9090

There could still be a ton of combat action. 2048 projectiles (0x0800) is enough for maximum carnage:

Code: Select all

At offset 14C68, replace 750BBA1000 with B800080000     <--
At offset 14C6D, replace 0000 with EB0A

As well as 256 explosions:

Code: Select all

At offset 14C85, replace C1E005 with C1E008

It seems that 700 craft is low enough that the officer bugfix may not be required. The turret bugfix may not be required either, which means that nothing needs to be shuffled, and the mission FlightGroup capacity remains intact at 192.

You'll still need the message log patch though.

Some players said that the engine is exploited in a way that the game with mods doesn't run smoothly anymore. Is this still the case or was this fixable for the mods?

Not fixed since the XWAU models are the main bottleneck in performance. It will take some testing to find a good balance.

[Bman]

Tested the concurrent flyby sound hook. It works. Thank you!!!

[AngeI]

Random Starfighter, I sense you will be of great service for the cause of XWAU and TFTC

A question, does the projectile limit refer to physical objects like warheads or lasers and ions too?

[JeremyaFr]

Hello,
Here is a screenshot with 1074 tie fighters:

crafts_limit_179_6.jpg

[Random Starfighter]

To clarify, is the 192 count the number of ships or objects per region or FGs (or both)? If its FG's then given you can only have 6 per FG, the max number of ships or objects should be 1152? Unless you can modify the number of ships per FG beyond 6?

I missed this question from an earlier post. 192 is the total number of FGs that can appear in a mission. They're not limited to 6 craft per FG. The older games had a limit of 6 craft in the player's wave because of limitations with wingmen sounds using lots of fixed-size arrays. As for XWA, I'm not familiar enough with the game to know what the player wave limits are. But for AI FGs there's no such limitation.

As far as gameplay is concerned, the formations would be broken because they also have a max of 6 craft, and extra craft would overflow into positions used by other formations. Also the craft numbering would get wonky above 9, since it was just simple addition to produce an ASCII character.

Normally (without patching the limits) 96 was the hard limit for the total craft that could spawn in the mission, and 192 being another hard limit if the craft counts were increased, as I explained in the bugfixes section of the patch. Space objects (mines and asteroids) historically shared a separate limit of 64. In XWA it appears they increased the limit of space objects to 128 per region.

Random Starfighter, I sense you will be of great service for the cause of XWAU and TFTC

A question, does the projectile limit refer to physical objects like warheads or lasers and ions too?

Haha well my specialty is XvT, not XWA. Lucky in these cases that the engines are so similar. Like the fly-by sound thing, I only knew where to look because I knew it was in XvT, only had to find it and compare.

Projectiles are basically any weapon with a velocity, yes. Lasers, ions, warheads, flares. Chaff is technically an explosion, I think. Players have their own projectile limit, which allows them to keep firing even if the AI limit is full. In multiplayer games each player is supposed to have their own limit, so they don't interfere with one another.

In the earlier games it was very easy for the AI projectile limit to be maxed out in large combat. Gameplay balance of certain missions can be altered by increasing the limit, when ships in certain situations would take more damage than they otherwise would just because more lasers/warheads are capable of spawning. This might be something to consider when designing large battles involving capital ships.

Hello,
Here is a screenshot with 1074 tie fighters: