Approximate Global Illumination Shaders

Want to edit the game, build your own craft and missions? Here you'll find help, tools, guides and people to discuss with.

Approximate Global Illumination Shaders

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Tue Oct 22, 2019 10:57 pm

I'm not sure if it's actually accurate to call it "Global Illumination": in space there's very little where light can bounce around and produce this effect; but I'm coding a few shaders that are used to implement GI in videogames and there's diffuse inter-reflections in the spots where they should be so... I'm going to claim that anyway :P

Building on my last Ambient Occlusion shader, I'm now implementing a mixture of three things:

1. SSDO -- Screen-Space Directional Occlusion. This is an extension on SSAO that uses the same basic infrastructure. The difference is that SSAO does not check lights to compute occlusion; but SSDO does. In other words. SSAO looks like this, no matter where the light is coming from:
ssdo1.jpg
With the SSDO shader I'm working on, the occlusion is now sensitive to the light source, so it looks like this:
ssdo2.jpg
or like this:
ssdo4.jpg
Depending on where the light is coming from. SSDO includes one light bounce (which I'm not showing here, will post more pics later) -- in other words, diffuse inter-reflection. The effect is a little subtle; but it can be noticed. Note also that the shader is generating "contact shadows" (small shadows that happen when two surfaces are in contact to each other). Although, technically, the contact shadows are happening because I'm also adding...

2. Bent Normals: this is the average unoccluded direction when computing SSAO. Using this normal to compute lighting tends to produce an effect that is closer to GI and also creates the shadows I've described; and finally, I'm also adding the following (since I'm here):

3. Normal Mapping. However, this isn't the regular normal mapping we find in other games. I'm approximating the normal map from the derivative of the luminosity value of the texture being rendered. But it does provide the illusion of relief and it responds to changes in illumination, so... probably as good as it's going to get without actually generating normal maps for each model.

As usual, there are a number of technical details that I have to fix: I don't have access to the smooth normals from the game (Jeremy has tried to explain to me how to do it; but I'm not yet smart enough to understand him), so some of the shadows look a bit faceted; there's some noise in certain areas; and we have the same limitations we had for SSAO: the effect disappears around the edges of the screen, etc. However, I think it still looks good enough in spite of these details. I'll post more pictures later...
You do not have the required permissions to view the files attached to this post.

User avatar
Trevor
Lieutenant JG
Posts: 541
Joined: Thu Dec 04, 2014 7:11 pm

Post by Trevor » Tue Oct 22, 2019 11:32 pm

Oh WOW, those pics look good.

While you are right that AO doesn't entirely make sence in space, you will get DIR around planets and from objects themselves.
I count AO as a cheap/simple DIR that doesn't take into account sky colour or reflections from other surfaces.

What you have is a replacement to diffuse lighting with raytraced shadows AND DIR

If you were somehow able to keep backdrops in memory and use them for DIR too this would definatly make it (almost)true GI.
I don't have access to the smooth normals from the game (Jeremy has tried to explain to me how to do it; but I'm not yet smart enough to understand him)
I would like to know more about what you mean, I have seen reshade attempt to make normals and from what I could tell it was a mess and would rather not do that without real normal maps supplied for ships.

Oh, light direction, are you still backtracing lights or do you have access to the real light positions yet?

Trev

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Wed Oct 23, 2019 12:01 am

ssdo3.jpg
Yes Trevor, we would get additional illumination from the environment in space. Normally this would be done with a sky/environment texture; but I think the game randomly generates the background, so that complicates things a bit. It's not impossible, though, just... a bit of a challenge. Of course we could provide additional environment maps that I can load in ddraw, too...

I'm reconstructing the normals from the depth buffer and they're actually quite good. Problem is the resulting normals are faceted instead of smooth and you can easily see where each triangle begins and ends. It's basically like flat shading. Jeremy told me where to look for the smooth normals that XWA is using; but I'm going to need to do more research before I can read them.

I don't have access to in-game light positions. I'm supplying my own light source and then I'm trying to make it match with the in-game orientation. I'm running into gimbal-lock problems though... so that's going to need more research.
You do not have the required permissions to view the files attached to this post.

User avatar
keiranhalcyon7
Lieutenant JG
Posts: 599
Joined: Tue Jan 02, 2018 6:41 am

Post by keiranhalcyon7 » Wed Oct 23, 2019 5:12 am

Nice. So to check my understanding, SSDO with zero bounces would be an approximation of self-shadowing?

Light sources are part of the mission file flightgroup list, which I think means they're accessible from the mission objects array. And yes, they can be directly above and/or below.

User avatar
JeremyaFr
XWAU Member
Posts: 3921
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Wed Oct 23, 2019 5:15 am

Hello,
To get the lights:

The structure that contains the lights data looks like that:

Code: Select all

// S0x07D4FA0
struct XwaGlobalLight
{
/* 0x0000 */ int PositionX ;
/* 0x0004 */ int PositionY ;
/* 0x0008 */ int PositionZ ;
/* 0x000C */ float DirectionX ;
/* 0x0010 */ float DirectionY ;
/* 0x0014 */ float DirectionZ ;
/* 0x0018 */ float Intensity ;
/* 0x001C */ float XwaGlobalLight_m1C ;
/* 0x0020 */ float ColorR ;
/* 0x0024 */ float ColorB ;
/* 0x0028 */ float ColorG ;
/* 0x002C */ float BlendStartIntensity ;
/* 0x0030 */ float BlendStartColor1C ;
/* 0x0034 */ float BlendStartColorR ;
/* 0x0038 */ float BlendStartColorB ;
/* 0x003C */ float BlendStartColorG ;
/* 0x0040 */ float BlendEndIntensity ;
/* 0x0044 */ float BlendEndColor1C ;
/* 0x0048 */ float BlendEndColorR ;
/* 0x004C */ float BlendEndColorB ;
/* 0x0050 */ float BlendEndColorG ;
};
The variables are:

Code: Select all

// V0x00782848
dword s_XwaGlobalLightsCount;

// V0x007D4FA0
Array<XwaGlobalLight,8> s_XwaGlobalLights;

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Wed Oct 23, 2019 6:17 am

keiranhalcyon7 wrote:
Wed Oct 23, 2019 5:12 am
Nice. So to check my understanding, SSDO with zero bounces would be an approximation of self-shadowing?
Not exactly. Single-bounce SSDO looks like SSAO; but the shadows can be colored and they respond to lights, which means that some areas that SSAO would render as shadowed actually are illuminated with SSDO. Also, SSDO adds the contribution from all light sources, so when you use an environment map it can produce some very interesting results that look a bit like GI. The contact shadows happen because I'm using Bent Normals (and, to be honest, I'm not 100% sure I'm using them correctly; but it looks fine to me).

Thanks for the pointers, keiranhalcyon7 and Jeremy. However, I tried reading those offsets (0x00782848 and 0x007D4FA0) and it looks like the coordinates/direction of the lights are in world coordinates. So I still need to transform them into view coordinates. Would you guys happen to know where the world-view transform matrix is stored?

User avatar
JeremyaFr
XWAU Member
Posts: 3921
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Wed Oct 23, 2019 4:51 pm

Hello,
The world and view matrix are stored in the SceneCompDatas structure.

You can get the direction of the lights with this code:

Code: Select all

void DumpGlobalLights()
{
	std::ostringstream str;

	int s_XwaGlobalLightsCount = *(int*)0x00782848;
	XwaGlobalLight* s_XwaGlobalLights = (XwaGlobalLight*)0x007D4FA0;

	int s_XwaCurrentSceneCompData = *(int*)0x009B6D02;
	int s_XwaSceneCompDatasOffset = *(int*)0x009B6CF8;

	XwaTransform* ViewTransform = (XwaTransform*)(s_XwaSceneCompDatasOffset + s_XwaCurrentSceneCompData * 284 + 0x0008);
	XwaTransform* WorldTransform = (XwaTransform*)(s_XwaSceneCompDatasOffset + s_XwaCurrentSceneCompData * 284 + 0x0038);

	for (int i = 0; i < s_XwaGlobalLightsCount; i++)
	{
		str << std::endl;
		str << "\t\t" << s_XwaGlobalLights[i].PositionX << ";" << s_XwaGlobalLights[i].PositionY << ";" << s_XwaGlobalLights[i].PositionZ;
		str << "\t\t" << s_XwaGlobalLights[i].DirectionX << ";" << s_XwaGlobalLights[i].DirectionY << ";" << s_XwaGlobalLights[i].DirectionZ;

		XwaVector3 viewDir = { s_XwaGlobalLights[i].DirectionX, s_XwaGlobalLights[i].DirectionY, s_XwaGlobalLights[i].DirectionZ };
		XwaVector3Transform(&viewDir, &ViewTransform->Rotation);
		str << "\t\t" << viewDir.x << ";" << viewDir.y << ";" << viewDir.z;
	}

	LogText(str.str());
}
The XwaVector3Transform function looks like that;

Code: Select all

// L00439B30
void XwaVector3Transform(XwaVector3* A4, const XwaMatrix3x3* A8)
{
	float A4_00 = A8->_31 * A4->z + A8->_21 * A4->y + A8->_11 * A4->x;
	float A4_04 = A8->_32 * A4->z + A8->_22 * A4->y + A8->_12 * A4->x;
	float A4_08 = A8->_33 * A4->z + A8->_23 * A4->y + A8->_13 * A4->x;

	A4->x = A4_00;
	A4->y = A4_04;
	A4->z = A4_08;
}
The vector and matrix structures are defined like that:

Code: Select all

// S0x0000001
struct XwaVector3
{
	/* 0x0000 */ float x;
	/* 0x0004 */ float y;
	/* 0x0008 */ float z;
};

// S0x0000002
struct XwaMatrix3x3
{
	/* 0x0000 */ float _11;
	/* 0x0004 */ float _12;
	/* 0x0008 */ float _13;
	/* 0x000C */ float _21;
	/* 0x0010 */ float _22;
	/* 0x0014 */ float _23;
	/* 0x0018 */ float _31;
	/* 0x001C */ float _32;
	/* 0x0020 */ float _33;
};

// S0x0000012
struct XwaTransform
{
	/* 0x0000 */ XwaVector3 Position;
	/* 0x000C */ XwaMatrix3x3 Rotation;
};

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Wed Oct 23, 2019 5:04 pm

Thanks Jeremy! I'll try that today!

This is what SSDO + Bent Normals looks like (note the shadows being projected from the pillars):
ssdo7-nobounce.jpg
and this is what it looks like with 1 diffuse bounce (the effect has been exaggerated a bit so that it's easier to notice):
ssdo7-bounce.jpg
and this is with normal mapping on (easier to see the differences in-game by toggling the effect):
ssdo7-bounce-nm.jpg
You do not have the required permissions to view the files attached to this post.

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Wed Oct 23, 2019 5:07 pm

The reflected light from the diffuse bounce is also sensitive to the position of the incoming light, this is the same scene; but with the light coming from a different direction:

Did I mention that I took all these screenshots with XWA's lighting disabled? All the shading is computed by the SSDO effect in ddraw (however, XWA's diffuse shading can be combined with this effect)
ssdo7-bounce-nm-2.jpg
You do not have the required permissions to view the files attached to this post.

User avatar
Trevor
Lieutenant JG
Posts: 541
Joined: Thu Dec 04, 2014 7:11 pm

Post by Trevor » Wed Oct 23, 2019 5:58 pm

AWSOME!!!!

Ok, your fake normals actually look quite good, is it analogus to bump mapping (using luminosity as "height" or rather, difference in height)

If you are able to add (at a later date) specular highlights/SSReflections we could start removing a lot of the static highlights on textures (like the tanks above)

Now, one last question regarding shadows, can you calculate sharp shadows?

So, what I mean is;
In atmosphere,
- Cloudy - AO (or more strictly DIR from the atmosphere and surroundings) is prevalent
- Sunny - Sharp shadows are prevalent on sunlit /single occluded surfaces.
in Orbit around a planet
- Planetside - AO/DIR is prevalent
- Spaceside - Sharp shadows (starkly lit) are prevalent
in deep space - sharp shadows (starkly lit) are prevalent

Taking the platform example above, its lighting closely resembles atmospheric/planetside.
if the light is a star, sharp shadows would be projected from the arms and tanks onto the hull (where the shadows would then be secondarily re-lit by DIR)

Trev

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Wed Oct 23, 2019 6:19 pm

JeremyaFr wrote:
Wed Oct 23, 2019 4:51 pm
Hello,
The world and view matrix are stored in the SceneCompDatas structure.
Unfortunately, I'm still seeing artifacts with the light direction using this code. It's as if the light was jumping 180 degrees in certain places. I only used XWA's light direction ignoring the position (that should be enough for a quick test); I'll keep trying though...

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Wed Oct 23, 2019 6:26 pm

Thanks Trevor. Yes, this normal mapping is essentially what you described.

From what I've read in various sources, I think it's possible to compute sharp shadows using SSDO. However, I *think* they would stop abruptly when the SSDO mask stops, so they have to either fade or become soft as they leave the vicinity of the center point. I don't think this is a problem in regular games, because they use light maps to complement the effect; but that probably won't work very well in this game due to clipped geometry.

Specular highlights are possible; but they will look faceted right now -- until I can figure out how to read the smooth normals from XWA. But maybe in the future...

Turgidson
Cadet 2nd Class
Posts: 52
Joined: Sun Aug 25, 2019 10:02 pm

Post by Turgidson » Wed Oct 23, 2019 8:52 pm

Wow - impressive. Maybe I wasn't kidding, when I wrote we'll soon have state-of-the-art graphics in this game. :)

User avatar
Trevor
Lieutenant JG
Posts: 541
Joined: Thu Dec 04, 2014 7:11 pm

Post by Trevor » Thu Oct 24, 2019 5:30 pm

yeah, so for sharp shadows we fall back to the requirement to disable frustum culling so the whole scene is visible to ddraw.

I was thinking today, ddraw seems to come "at the end" of all graphics processing, can it not also replace other graphics calls that come before culling etc?
I mean, for example, having the data (ok, your having difficulty reading now, but I hope Jeremy can elaborate and fix) to position lights, load textures, load models can you not just draw the scene in ddraw and skip XWA's old drawing?

I mean, its not like starting from scratch, xwa will still position objects and generate display lists, but ddraw can shade + texture (bumpmapping) + shadows + custom frustum (based on VR position or 0,0 if VR disabled) and final view matrix transform with subsequent specular and SSDO

Trev

User avatar
JeremyaFr
XWAU Member
Posts: 3921
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Thu Oct 24, 2019 7:58 pm

Replacing the entire rendering engine may be possible but if it is possible, it will require a lot of work.

When a new scene is started, the visible objects list is first cleared.
Then an object is added to the list based on the result of these functions:
- L004F1CA0
- L004F1E90
- L0049F550

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Fri Oct 25, 2019 9:12 pm

A few more samples, this is regular SSDO with Bent Normals:
ssdo8-no-nm.jpg
This is what it looks like with Normal Mapping:
ssdo8-nm.jpg
And this is what it looks like with everything put together (XWA's shading is disabled in this picture and there's no diffuse inter-reflections either):
ssdo8-nm-col.jpg
Normal Mapping can be disabled or it can be fine-tuned to increase/decrease the effect.
You do not have the required permissions to view the files attached to this post.

User avatar
Vince T
Fleet Admiral (Administrator)
Posts: 14045
Joined: Fri Apr 27, 2001 11:01 pm
Contact:

Post by Vince T » Fri Oct 25, 2019 10:28 pm

OMG is this actually happening? Quasi bump mapping!?!
Your ship, Captain. I need a drink. - Vince Trageton
Vince T's X-Wing HQ - where the bad guys get their gear

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Fri Oct 25, 2019 10:43 pm

Yes, it's happening :)

I think that if we created actual normal maps we could have real normal mapping instead of this quasi bump mapping. It's one of the experiments I want to try later; but it might work...

User avatar
JeremyaFr
XWAU Member
Posts: 3921
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Sat Oct 26, 2019 8:24 am

Good work :applaus:

User avatar
keiranhalcyon7
Lieutenant JG
Posts: 599
Joined: Tue Jan 02, 2018 6:41 am

Post by keiranhalcyon7 » Sat Oct 26, 2019 5:18 pm

Hmm... Looks too lumpy to me. Maybe if there was a threshold, below which no bending happens?

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Sat Oct 26, 2019 6:34 pm

The "bumpiness" of effect can be adjusted -- I probably exaggerated it a bit in the images above.

User avatar
Vince T
Fleet Admiral (Administrator)
Posts: 14045
Joined: Fri Apr 27, 2001 11:01 pm
Contact:

Post by Vince T » Sat Oct 26, 2019 7:38 pm

Is there any way to have it not being affected by the text on the dynamic displays? It's odd to have those bumped, too...
Your ship, Captain. I need a drink. - Vince Trageton
Vince T's X-Wing HQ - where the bad guys get their gear

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Sun Oct 27, 2019 5:30 am

General_Trageton wrote:
Sat Oct 26, 2019 7:38 pm
Is there any way to have it not being affected by the text on the dynamic displays? It's odd to have those bumped, too...
Yes, that's possible. The regular HUD doesn't get any bump-mapping, though.

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Thu Oct 31, 2019 8:01 am

I'm still trying to fix a few artifacts with SSDO (I'm getting lots of ugly halos); but I was able to add normal mapping to the regular SSAO:

Regular SSAO:
ssao14.jpg
SSAO + Normal Mapping:
ssao14-nm.jpg
You do not have the required permissions to view the files attached to this post.

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Thu Oct 31, 2019 8:04 am

A few more teasers. This is regular SSDO:
ssdo8.jpg
and SSDO + Normal Mapping:
ssdo8-nm.jpg
You do not have the required permissions to view the files attached to this post.

Post Reply