Approximate Global Illumination Shaders

[blue_max]

I'm not sure if it's actually accurate to call it "Global Illumination": in space there's very little where light can bounce around and produce this effect; but I'm coding a few shaders that are used to implement GI in videogames and there's diffuse inter-reflections in the spots where they should be so... I'm going to claim that anyway

Building on my last Ambient Occlusion shader, I'm now implementing a mixture of three things:

1. SSDO -- Screen-Space Directional Occlusion. This is an extension on SSAO that uses the same basic infrastructure. The difference is that SSAO does not check lights to compute occlusion; but SSDO does. In other words. SSAO looks like this, no matter where the light is coming from:

ssdo1.jpg

With the SSDO shader I'm working on, the occlusion is now sensitive to the light source, so it looks like this:

ssdo2.jpg

or like this:

ssdo4.jpg

Depending on where the light is coming from. SSDO includes one light bounce (which I'm not showing here, will post more pics later) -- in other words, diffuse inter-reflection. The effect is a little subtle; but it can be noticed. Note also that the shader is generating "contact shadows" (small shadows that happen when two surfaces are in contact to each other). Although, technically, the contact shadows are happening because I'm also adding...

2. Bent Normals: this is the average unoccluded direction when computing SSAO. Using this normal to compute lighting tends to produce an effect that is closer to GI and also creates the shadows I've described; and finally, I'm also adding the following (since I'm here):

3. Normal Mapping. However, this isn't the regular normal mapping we find in other games. I'm approximating the normal map from the derivative of the luminosity value of the texture being rendered. But it does provide the illusion of relief and it responds to changes in illumination, so... probably as good as it's going to get without actually generating normal maps for each model.

As usual, there are a number of technical details that I have to fix: I don't have access to the smooth normals from the game (Jeremy has tried to explain to me how to do it; but I'm not yet smart enough to understand him), so some of the shadows look a bit faceted; there's some noise in certain areas; and we have the same limitations we had for SSAO: the effect disappears around the edges of the screen, etc. However, I think it still looks good enough in spite of these details. I'll post more pictures later...

[Trevor]

Oh WOW, those pics look good.

While you are right that AO doesn't entirely make sence in space, you will get DIR around planets and from objects themselves.
I count AO as a cheap/simple DIR that doesn't take into account sky colour or reflections from other surfaces.

What you have is a replacement to diffuse lighting with raytraced shadows AND DIR

If you were somehow able to keep backdrops in memory and use them for DIR too this would definatly make it (almost)true GI.

I don't have access to the smooth normals from the game (Jeremy has tried to explain to me how to do it; but I'm not yet smart enough to understand him)

I would like to know more about what you mean, I have seen reshade attempt to make normals and from what I could tell it was a mess and would rather not do that without real normal maps supplied for ships.

Oh, light direction, are you still backtracing lights or do you have access to the real light positions yet?

Trev

[blue_max]

ssdo3.jpg

Yes Trevor, we would get additional illumination from the environment in space. Normally this would be done with a sky/environment texture; but I think the game randomly generates the background, so that complicates things a bit. It's not impossible, though, just... a bit of a challenge. Of course we could provide additional environment maps that I can load in ddraw, too...

I'm reconstructing the normals from the depth buffer and they're actually quite good. Problem is the resulting normals are faceted instead of smooth and you can easily see where each triangle begins and ends. It's basically like flat shading. Jeremy told me where to look for the smooth normals that XWA is using; but I'm going to need to do more research before I can read them.

I don't have access to in-game light positions. I'm supplying my own light source and then I'm trying to make it match with the in-game orientation. I'm running into gimbal-lock problems though... so that's going to need more research.

[keiranhalcyon7]

Nice. So to check my understanding, SSDO with zero bounces would be an approximation of self-shadowing?

Light sources are part of the mission file flightgroup list, which I think means they're accessible from the mission objects array. And yes, they can be directly above and/or below.

[JeremyaFr]

Hello,
To get the lights:

The structure that contains the lights data looks like that:

Code: Select all

// S0x07D4FA0
struct XwaGlobalLight
{
/* 0x0000 */ int PositionX ;
/* 0x0004 */ int PositionY ;
/* 0x0008 */ int PositionZ ;
/* 0x000C */ float DirectionX ;
/* 0x0010 */ float DirectionY ;
/* 0x0014 */ float DirectionZ ;
/* 0x0018 */ float Intensity ;
/* 0x001C */ float XwaGlobalLight_m1C ;
/* 0x0020 */ float ColorR ;
/* 0x0024 */ float ColorB ;
/* 0x0028 */ float ColorG ;
/* 0x002C */ float BlendStartIntensity ;
/* 0x0030 */ float BlendStartColor1C ;
/* 0x0034 */ float BlendStartColorR ;
/* 0x0038 */ float BlendStartColorB ;
/* 0x003C */ float BlendStartColorG ;
/* 0x0040 */ float BlendEndIntensity ;
/* 0x0044 */ float BlendEndColor1C ;
/* 0x0048 */ float BlendEndColorR ;
/* 0x004C */ float BlendEndColorB ;
/* 0x0050 */ float BlendEndColorG ;
};

The variables are:

Code: Select all

// V0x00782848
dword s_XwaGlobalLightsCount;

// V0x007D4FA0
Array<XwaGlobalLight,8> s_XwaGlobalLights;

[blue_max]

Nice. So to check my understanding, SSDO with zero bounces would be an approximation of self-shadowing?

Not exactly. Single-bounce SSDO looks like SSAO; but the shadows can be colored and they respond to lights, which means that some areas that SSAO would render as shadowed actually are illuminated with SSDO. Also, SSDO adds the contribution from all light sources, so when you use an environment map it can produce some very interesting results that look a bit like GI. The contact shadows happen because I'm using Bent Normals (and, to be honest, I'm not 100% sure I'm using them correctly; but it looks fine to me).

Thanks for the pointers, keiranhalcyon7 and Jeremy. However, I tried reading those offsets (0x00782848 and 0x007D4FA0) and it looks like the coordinates/direction of the lights are in world coordinates. So I still need to transform them into view coordinates. Would you guys happen to know where the world-view transform matrix is stored?

[JeremyaFr]

Hello,
The world and view matrix are stored in the SceneCompDatas structure.

You can get the direction of the lights with this code:

Code: Select all

void DumpGlobalLights()
{
	std::ostringstream str;

	int s_XwaGlobalLightsCount = *(int*)0x00782848;
	XwaGlobalLight* s_XwaGlobalLights = (XwaGlobalLight*)0x007D4FA0;

	int s_XwaCurrentSceneCompData = *(int*)0x009B6D02;
	int s_XwaSceneCompDatasOffset = *(int*)0x009B6CF8;

	XwaTransform* ViewTransform = (XwaTransform*)(s_XwaSceneCompDatasOffset + s_XwaCurrentSceneCompData * 284 + 0x0008);
	XwaTransform* WorldTransform = (XwaTransform*)(s_XwaSceneCompDatasOffset + s_XwaCurrentSceneCompData * 284 + 0x0038);

	for (int i = 0; i < s_XwaGlobalLightsCount; i++)
	{
		str << std::endl;
		str << "\t\t" << s_XwaGlobalLights[i].PositionX << ";" << s_XwaGlobalLights[i].PositionY << ";" << s_XwaGlobalLights[i].PositionZ;
		str << "\t\t" << s_XwaGlobalLights[i].DirectionX << ";" << s_XwaGlobalLights[i].DirectionY << ";" << s_XwaGlobalLights[i].DirectionZ;

		XwaVector3 viewDir = { s_XwaGlobalLights[i].DirectionX, s_XwaGlobalLights[i].DirectionY, s_XwaGlobalLights[i].DirectionZ };
		XwaVector3Transform(&viewDir, &ViewTransform->Rotation);
		str << "\t\t" << viewDir.x << ";" << viewDir.y << ";" << viewDir.z;
	}

	LogText(str.str());
}

The XwaVector3Transform function looks like that;

Code: Select all

// L00439B30
void XwaVector3Transform(XwaVector3* A4, const XwaMatrix3x3* A8)
{
	float A4_00 = A8->_31 * A4->z + A8->_21 * A4->y + A8->_11 * A4->x;
	float A4_04 = A8->_32 * A4->z + A8->_22 * A4->y + A8->_12 * A4->x;
	float A4_08 = A8->_33 * A4->z + A8->_23 * A4->y + A8->_13 * A4->x;

	A4->x = A4_00;
	A4->y = A4_04;
	A4->z = A4_08;
}

The vector and matrix structures are defined like that:

Code: Select all

// S0x0000001
struct XwaVector3
{
	/* 0x0000 */ float x;
	/* 0x0004 */ float y;
	/* 0x0008 */ float z;
};

// S0x0000002
struct XwaMatrix3x3
{
	/* 0x0000 */ float _11;
	/* 0x0004 */ float _12;
	/* 0x0008 */ float _13;
	/* 0x000C */ float _21;
	/* 0x0010 */ float _22;
	/* 0x0014 */ float _23;
	/* 0x0018 */ float _31;
	/* 0x001C */ float _32;
	/* 0x0020 */ float _33;
};

// S0x0000012
struct XwaTransform
{
	/* 0x0000 */ XwaVector3 Position;
	/* 0x000C */ XwaMatrix3x3 Rotation;
};

[blue_max]

Thanks Jeremy! I'll try that today!

This is what SSDO + Bent Normals looks like (note the shadows being projected from the pillars):

ssdo7-nobounce.jpg

and this is what it looks like with 1 diffuse bounce (the effect has been exaggerated a bit so that it's easier to notice):

ssdo7-bounce.jpg

and this is with normal mapping on (easier to see the differences in-game by toggling the effect):

ssdo7-bounce-nm.jpg

[blue_max]

The reflected light from the diffuse bounce is also sensitive to the position of the incoming light, this is the same scene; but with the light coming from a different direction:

Did I mention that I took all these screenshots with XWA's lighting disabled? All the shading is computed by the SSDO effect in ddraw (however, XWA's diffuse shading can be combined with this effect)

ssdo7-bounce-nm-2.jpg

[Trevor]

AWSOME!!!!

Ok, your fake normals actually look quite good, is it analogus to bump mapping (using luminosity as "height" or rather, difference in height)

If you are able to add (at a later date) specular highlights/SSReflections we could start removing a lot of the static highlights on textures (like the tanks above)

Now, one last question regarding shadows, can you calculate sharp shadows?

So, what I mean is;
In atmosphere,
- Cloudy - AO (or more strictly DIR from the atmosphere and surroundings) is prevalent
- Sunny - Sharp shadows are prevalent on sunlit /single occluded surfaces.
in Orbit around a planet
- Planetside - AO/DIR is prevalent
- Spaceside - Sharp shadows (starkly lit) are prevalent
in deep space - sharp shadows (starkly lit) are prevalent

Taking the platform example above, its lighting closely resembles atmospheric/planetside.
if the light is a star, sharp shadows would be projected from the arms and tanks onto the hull (where the shadows would then be secondarily re-lit by DIR)

Trev

[blue_max]

Hello,
The world and view matrix are stored in the SceneCompDatas structure.

Unfortunately, I'm still seeing artifacts with the light direction using this code. It's as if the light was jumping 180 degrees in certain places. I only used XWA's light direction ignoring the position (that should be enough for a quick test); I'll keep trying though...

[blue_max]

Thanks Trevor. Yes, this normal mapping is essentially what you described.

From what I've read in various sources, I think it's possible to compute sharp shadows using SSDO. However, I *think* they would stop abruptly when the SSDO mask stops, so they have to either fade or become soft as they leave the vicinity of the center point. I don't think this is a problem in regular games, because they use light maps to complement the effect; but that probably won't work very well in this game due to clipped geometry.

Specular highlights are possible; but they will look faceted right now -- until I can figure out how to read the smooth normals from XWA. But maybe in the future...

[Turgidson]

Wow - impressive. Maybe I wasn't kidding, when I wrote we'll soon have state-of-the-art graphics in this game.

[Trevor]

yeah, so for sharp shadows we fall back to the requirement to disable frustum culling so the whole scene is visible to ddraw.

I was thinking today, ddraw seems to come "at the end" of all graphics processing, can it not also replace other graphics calls that come before culling etc?
I mean, for example, having the data (ok, your having difficulty reading now, but I hope Jeremy can elaborate and fix) to position lights, load textures, load models can you not just draw the scene in ddraw and skip XWA's old drawing?

I mean, its not like starting from scratch, xwa will still position objects and generate display lists, but ddraw can shade + texture (bumpmapping) + shadows + custom frustum (based on VR position or 0,0 if VR disabled) and final view matrix transform with subsequent specular and SSDO

Trev

[JeremyaFr]

Replacing the entire rendering engine may be possible but if it is possible, it will require a lot of work.

When a new scene is started, the visible objects list is first cleared.
Then an object is added to the list based on the result of these functions:
- L004F1CA0
- L004F1E90
- L0049F550

[blue_max]

A few more samples, this is regular SSDO with Bent Normals:

ssdo8-no-nm.jpg

This is what it looks like with Normal Mapping:

ssdo8-nm.jpg

And this is what it looks like with everything put together (XWA's shading is disabled in this picture and there's no diffuse inter-reflections either):

ssdo8-nm-col.jpg

Normal Mapping can be disabled or it can be fine-tuned to increase/decrease the effect.

[Vince T]

OMG is this actually happening? Quasi bump mapping!?!

[blue_max]

Yes, it's happening

I think that if we created actual normal maps we could have real normal mapping instead of this quasi bump mapping. It's one of the experiments I want to try later; but it might work...

[JeremyaFr]

Good work

[keiranhalcyon7]

Hmm... Looks too lumpy to me. Maybe if there was a threshold, below which no bending happens?

[blue_max]

The "bumpiness" of effect can be adjusted -- I probably exaggerated it a bit in the images above.

[Vince T]

Is there any way to have it not being affected by the text on the dynamic displays? It's odd to have those bumped, too...

[blue_max]

Is there any way to have it not being affected by the text on the dynamic displays? It's odd to have those bumped, too...

Yes, that's possible. The regular HUD doesn't get any bump-mapping, though.

[blue_max]

I'm still trying to fix a few artifacts with SSDO (I'm getting lots of ugly halos); but I was able to add normal mapping to the regular SSAO:

Regular SSAO:

ssao14.jpg

SSAO + Normal Mapping:

ssao14-nm.jpg

[blue_max]

A few more teasers. This is regular SSDO:

ssdo8.jpg

and SSDO + Normal Mapping:

ssdo8-nm.jpg