After I posted my blog, I carried on cleaning up code and improving the shadow rendering for another 14 hour plus marathon, and although there are inevitable tweaks still to come I am happy with the final result.
We now have 'Percentage Closer Filtering (3x3)' for a nice soft shadow and cascade blending to remove the banding artifacts as the shadows steps into the lower quality levels in the distance (kind of like LOD for shadows).
Here is a video (minus the new art) as I was very excited to show you the latest version (and also to show off the technique below):
Notice how the shadow goes into the brickwork? By using the depth map information from the Parallax Mapping fragment shader, I was able to hijack these most previous variables and use them for my shadows.
Shadows in the Depth Map
One of the many articles I read on this subject had the idea of using the normal map (depth map) as part of the shadowing operation, so that the shadows could fall into the grooves of the normally flat surface. Although I did not read up on the correct math, my own idea seemed sensible enough, which was to shift the world position of the texel into the surface of the polygon using the tangent aligned normal. The shadow mapper would then assume that pixel was further away (deeper) and thus shade it as such, and it did. Imagine my delight when the shadow now contoured around the details in the texture depth map as well as the large blocks.
The only noticeable issues I have now is that at extreme angles the shadow calculation stops short of the width of the cascade but it's rare to see this artifact. A more serious one though is the edge bleed you can see at the base of the objects where they meet the floor. As I am using Clockwise Culling to defeat self shadowing on the side facing the light source, I only get a polygon thick wall to cast the actual shadow which bleeds non-shadow values thanks to the new PCF effect. There are numerous solutions but each one carries a side effect that is worse than the fix, so I am going to let that one brew a while. The current fix is to shrink the geometry slightly when I render the depth views, which helps a little but it's now what I am after. I am sure the solution will present itself before too long.
My mission now is to clean the code up some more (as there is a lot of commented out experimental code that never worked), and then see if I can apply this process to the main engine (just to see what happens). My new implementation does almost all of the shadowing in the C++ DLL side or the shader, so it should be as simple as placing a light and calling a few commands in the main engine to get my shadows in there.