Some More Nice Performances
The star if the show this evening was my friends the QUADs. These little pair of pesky polygons helped me fill the gap caused when the LOW LOD models disappeared from the scene to save render drain. They look much like the LOW LODs but are entirely flat. They are also so far away you don't really notice they are flat, or much care, when your attention is on the foreground. Here is my horrid little prototype, with visuals only a mother could love.
As an antidote to two developer shots in two days, Rolfy comes to the rescue (once more) with THREE new skies for Reloaded. Amazingly, he gifted them to us completely free to be used for the Reloaded product, and the moon one is my new favorite! One day I hope to add the technology to make our moon (and sun) round ;)
I could go into the amazing levels of brain numbing detail, but it's 3AM and my pizza is in the oven, so this will be brief. The new QUAD buffer work completes the polygon fill for the screen and all that remains (all he says!) is to orient the quads to always face the camera and to render the contents of the quad with the correct little texture to fool the user into thinking it is still the LOW LOD model you are looking at.
I was going to fake it with seven angled shots of each texture, but the more I think about the cleverness of the Dark Imposter (not Dark Occlusion - which is also pretty clever), the more I am keen to at least TRY to see if I can get each quad to have an accurate representation of the original model. So what if it takes 100MB of texture memory, hits the performance for a few FPS, the finished render will be gorgeous, with almost polygon perfect accuracy across a vista stretching over a mile wide. I must keep reminding myself to constantly aim HIGH!
Signing Off
A few more minutes before I burn my pizza. Wednesday I will finish the last two QUAD tasks then see if I can integrate the lot into the engine to see where we are. I am not too precious on this point and won't leave the prototype until it does everything I need it to do, and I have a meeting on Thursday to show my wares. I was hoping to show the final software with a huge speed increase, but given the complexity of what I've had to go through, I am now very happy if it's a complete and competent prototype that gets demonstrated. I can smell, burning, so time to sign off :)
Performance In The House
My plan for Monday was QUAD batches, so of course by the end of Monday I've just put the finishing touches to the new GPU Occlusion and Culling system for Reloaded. Now prepare yourself for a prototype screenshot, so those of a weak disposition, please avert your eyes now.
As you might see, the numbers in the top left tell an interesting story about where we are in engine terms. The first number to note is the O value, which tells us how many buildings I added to the 50,000 x 50,000 world. Each building has three LOD levels of 5000 polygons, 4000 polygons and 3000 polygons (roughly). The next important number is 310MB, which shows we only used 109MB to render all the buildings within visual distance, and have been created in real-time as part of the dynamic buffering system. As we move through the world, this memory usage will not increase much more than this if the spread of buildings stays consistent as buffers are efficiently reused.
My favorite statistic is CALLS:4 which tells us that only FOUR draw calls where made to create the scene, and the POLYGONS of 24796 shows the population within these four drawn buffers. A quick calculation and you will figure out that we have static geometry batching going on as well!
The 1750 fps is a little deceptive given we don't have the rest of the engine in play, but when you consider that before the above techniques the same scene was rendering over 40 draw calls, nearly half a million polygons and a frame rate of around 400-500 fps.
GPU Occlusion
This worked better than I thought and really takes advantage of the speed increase you can get when you draw to a scene (in front to back order) using a non-color write mode to get the query to say if pixels where drawn.
Amazingly after lots of careful thinking and slow coding, the engine worked first time out the gate. It was a shock to see the call rate drop to single digits and everything still rendered as though EVERYTHING was being rendered. Neat.
Signing Off
As tempting as it is to move this over to the main engine and test against a variety of entities, sizes, textures and so on, I really want to finish the render scene with QUADs. Right now the LOW LOD just disappears after a certain distance and is not replaced with anything, so they just ping out of existence. I had a mind to prototype using the Dark Occlusion module which provides this functionality out of the box, but on deeper reflection I would need it very closely integrated into the instance stamp occlusion system I have now and there would be some compliance issues in the middle of all that.
I will have it as my fall-back, but for Tuesday I am hoping I can create a large static pool of quads and write a shader to have them face the camera with the right texture. Generating the texture will be another matter entirely, and it will be a dice throw as to whether I go for my previous idea which was seven angles around the Y axis and one from above (for the shadow cameras), or opt for the more sophisticated technique used by Dark Occlusion which renders each object perfectly to a communal quad texture, and only refreshes the render when absolutely necessary. The result is a MUCH more believable transition in the far distance, and is perhaps the hallmark of a quality future proof engine. My concern is that if you place down 100,000 trees, that's rendering 100K of entity views to a sequence of community textures, and that is only going to consume video memory and real-time performance.
My dilemma is whether the extra time to test the more advanced technique is worth it, or whether it's more important to get everything in the main engine as soon as possible, and continue testing and improving from that base.
The Return Of Instance Stamps
As part of my batching code, I returned to a batching system I did a while ago for the old segments called Instance Stamps. The idea that instances are converted to reference data within a large grid, and the geometry is only created and rendered on a need to know basis.
I had to heavily adapt it to the dimensions and unique style of the Reloaded level size and preferred ranges, and took all day, but the end result was a prototype that positively eats polygons for breakfast. My last demo had 7800 combat buildings added all across the world, with HIGH and LOW LOD buildings being rendered in batches only where I was standing at the time. The draw calls where less than 50 and my frame rate was well over 500 fps. yes it included the actual entity shader, but did not include all the other gump required to run a games engine. The real teller was the impact of the dynamic creation as you moved through the world which kept a smooth motion even at 500 fps. One fear of course is that there is so much new geometry being dynamically created that it slows down and jitters the smooth running of the engine. So far, so good.
Next Steps
The two next steps are QUAD buffers for the far distance and OCCLUSION volumes for the stuff immediately around me, drawn front to back starting with the HIGH lod, then LOW lod, then QUADS, and all through the filter of a camera frustum. All these together should allow me to render only a handful of polygons for the view you are current watching, and disregard much of the rest of the level. It should also present very little performance hit as the stuff is created and managed, especially as the front to back sorting is done on a permanent visible object list so only needs to tax the processor when a change is required (which happens in small bursts as you move around and the order changes).
Signing Off
I am happy with the weeks work. Even though my two hour read-up on spacial databases has not contributed directly to the speed up work being undertaken it has given me a sense of clarity on how to approach a situation to gain speed in simple ways. It also means when the time comes to do quick searches based on spacial items in the world, I have this knowledge to fall back on. I even discovered a cool query you can do with spacial databases which is to project a line through 2D collection of boxes and it will instantly report which of those boxes where involved in the intersect test. Much better than a for next loop through every object in range of the bullet ray(s) and can quickly detect when it hits something solid and only check for dynamic entities within that reduced range! It's the little things that will give us that super smooth performance, and even if I cannot afford the distraction of coding these ideas, making a small note in the source code will ensure I can come back to these little pearls.
I will be attending an Intel Game Fest this Saturday morning for 1 hour to help out would-be Android developers understand the wonderful world of AGK. If you know me, you know I can talk for an hour on practically any subject, but it is usually a rant or a long verbal waffle. Being center stage for an hour and not repeat myself too many times will be a challenge. Should be a nice end of week distraction to kick-start my weekend. It's about 1:30AM now and my Hangout is at 5AM, so I am going to prepare some AGK material for half an hour and then spend the rest of the time adding a QUAD buffer idea I just had to my instance stamp engine prototype. Sorry there is no video or shot, but when I grabbed it, the quality was to 'programmer art' I dare not inflict it on you. I think the next substantial shot will be a new view of my 'run to the river' demo with a substantially better FPS score :)
A Rare Day
It is not often I read up on contemporary literature of the day, as my old school habits lead me astray towards the 'do it yourself' approach. Today however I was impressed with the promise of a C++ library called BOOST to spend some time on pure reading.
A pretty well rounded library, and a fiendishly fast set of routines to speed up spacial searching and database creation. After 2 hours of reading, I had learned the API and all the examples, and was ready to do some prototyping. I then decided to sketch out what I needed to build a much faster scene management system for Reloaded.
After two pages of A4, it was apparent that for my initial coding I did not need a spacial query system as the current system I was using in the Instance Stamp system was FASTER than the BOOST library, and used only marginally more memory. I thus decided plan A was to re-activate the Instance Stamp solution with the changes required to reflect the way static entities exist in the current engine. So I don't chew off too much, I have put occlusion and quad engine to one side so I can focus on the 'up close and personal' polygons that would surround the player immediately in the scene. From there, I can create bounding volumes of geometry to reflect the instance stamp buffer collections and do the occlusion, and finally fill the missing 'distance' objects with quads. My sketch told me the order of the work as well as highlighting the specific details of what was missing to get from task A to task B.
Instance Stamp
As a recap, instance stamps is the idea that instances of objects are placed in a reference grid, and when the player is close enough to these references, a dynamic copy of that geometry is created in a common buffer for rendering. As it moves out of view, the dynamic buffer is destroyed and as new references come into view, they are created using the discarded buffers. Essentially the level can be extremely large without consuming a large amount of static geometry memory. The performance hit is the dynamic creation, which if kept low enough becomes completely unnoticeable (and in prototypes have proven I can achieve hundreds of FPS no problem).
It's about 9:15PM now and my plan of a regular day went out the window today as I only rose after dinner time, meaning my day was already on a skew. I also have a 5AM appointment on Saturday which forces me to either stay up amazingly late on Friday, or go to bed insanely early to get up on Saturday before the larks and seagulls. I have no doubt I will manage it either way, but it makes for an interesting schedule between now and then. It is for this reason, and while my brain is keen, to get some of the instance stamp engine coded now so I can see some early results. All the commands are still in place from last time, so it's just a case of familiarizing myself with the old prototype, and migrating those ideas over to how the static entities are situated right now. Hopefully before Friday I can report a speed up :)
Signing Off
Looking back on my blogs, I must apologies for the lack of screenshots and lovely eye candy. This is the essential truth of the programmer, and what most weeks actually looks like down in the coding dungeon. Lots and lots of ideas, much code and very little visuals to show for it. You only get visuals when some PR guy taps you on the shoulder and asks for something 'real' to show :) Alas, this is the way of the world. Until my next 'real' blog, I shall continue to crank out more of this alien gobbledygook and look forward to the day when I can report some serious performance boosts.
In the meantime, here is the small scene I am using as a template to first test my instance stamp batcher and ultimately my occluder. As you can see, from an FPS point of view, and with occlusion working, very few of the buildings should ever actually be rendered and result in low polygons and draw calls, and high FPS as a result. Here's hoping!!
Performance Is King
As a mere subject of the king, my work today was rather dull with plenty of nothing to screen shot for your delectation! I had a big six hour thought last night due to insomnia and came up with a radical plan to transform performance metrics in Reloaded. It involved a bottom-up approach to the solution which is 'what don't I need to draw right now'.
Given this simple premise, I started by getting rid of all the objects until I was left with the floor and sky. The terrain floor was happily gobbling up over 30K in polygons for a completely flat floor. I immediate remembered the QUAD REDUCE function of the Blitz Terrain system so decided Wednesday would be about using that feature. I already knew it could not simply be turned on and I had to be flagged before passing in a height-map. Generating and testing that height map took most of the day.
The Performance Plan
Essentially I am going to build a spacial R-tree database of all entities, create 3D box volumes for the enclosed groups of entities, then generate pools of quads that share a communal texture. The quads and communal texture is generated just before the game starts and will take very little memory. I then re-use my Instance Stamp system batch together all the HIGH and MID static entities relative to the local position of the player, which means we only create what we need to see within out close range. I then use GPU occlusion detection with the r-tree volumes to work out what I don't need to render, and then add dynamic entities only where volumes are visible for that cycle. The quad will have multiple textures depending on which angle you are looking at it, and one for when looking at it from above. A shader will control the quad rotation and texture selection to avoid writing into the vertex buffer, and the same batches of static buffers and quads can be used by the shadow rendering process. The upshot is that we will only render the large pools of static common geometry that is visible, and only dynamic geometry that is associated with a visible r-tree volume, giving us the minimum amount to draw for the maximum performance.
The quad system is pretty similar to how Dark Imposter does things, but won't be required to regenerate the object view as the player moves. By pre-rendering eight fixed directions around the entity for our quad texturing, we will sacrifice that millimeter perfect quad visual for more speed, which is the objective at the moment. I must say though I really like the way Dark Imposter can produce a quad texture in real-time and regenerate it as you move around the object, and still manage to keep hundreds of FPS in the bag! If only I knew how they did it :)
Signing Off
All sounds impressive right :) Well now I need to code it, and I am giving myself a few weeks to do this, but the end result should be entities as far as the eye can see, and every draw call that can be saved, saved. I can also use the r-tree spacial database to localize entity searches for things like physics interaction, sound playing, gun and missile ray-casting and to some degree entity logic. Traditionally in FPSC Classic, we would have to step through EVERY entity in the level to perform our tasks, even if we only needed access to just one or a few close items. Having a spacial database means our searches can be MUCH more targeted.
I am just finishing off some terrain quad reduction work, and then Thursday I will delve into the spacial functions of the BOOST library which promises some nice algorithms to make my database implementation go smoothly. Once I have my spacial hierarchy in place, creating batches of common buffers from them and hiding them via GPU occlusion should be a little easier!
Aha!
The sound of victory, perhaps, the sound of a small mystery uncovered, definitely. You will be disenchanted to learn I was not working on performance today. I was working on that annoying horrible vertex corruption we have been seeing in the gun and characters. You will NEVER guess what I found inside my engine source code:
// sort into time ascending order at init stage
// very time consuming - so skip until see adverse affects!
//if ( pAnim ) SortAnimationDataByTime ( pAnim );
Amazing or what! I must have been insanely tired or blind to comment this function out, not provide a date, to even think that I could get away with it and finally to lose all trace of every doing it and carrying on with my day.
I have not actually uncommented and compiled yet but I would bet a pint of Guinness that this is the bug. I vaguely remember removing this line to speed up character loading, madness!!
Other News
As yesterday's blog was a little small, I want to report on what happened on Monday in a little more detail. My PC decided to corrupt one file (just one) at the I/O level, which means a PHYSICAL hole in my data. All the usual suspects such as CHKSDK, Defrag, rename, remove and rebel where all in vain. I left the PC to do a full defrag overnight in the hopes it would move the offending file to some new sector which did not have a hole. No joy. First thing today I renamed my SVN repository, moved all the files over and left that one file in the old location locked and alone. It's such a dangerous file that even if I cursor over it Windows File Explorer will crash!
As this is hardly a professional status quo, I have ordered a new replacement 128GB SSD drive and will have the miserable job of a full Windows re-install sometime this week. The good news is that the new drive is much faster than the old one and starting Windows a fresh always provides a nice fast OS (at least to begin with) to enjoy for a few months. It arrives Wednesday, so that will probably be the day.
My Mission
I decided my mission for today was to either fix the vertex corruption or make the engine faster. Having tried reducing the shaders and noticing almost no performance improvement on my 650 Ti Boost (likely due to being CPU bound rather than GPU), and also failing to get the NSIGHT tool to do anything practical (after re-installing drivers, updating the tool, changing my WDD settings, fiddling with configurations and ultimately getting absolutely nothing from my attempt to profile my frames) I finally got to the line of code you saw above, which means in theory I have achieved something today!
Signing Off
A meeting was held today in other parts of TGC land, and it was agreed by all that performance and key bug fixes are to remain our top priority, and this attitude will continue until we have a solid and fast architecture. I am the first to leap up and down when new features are busting to be coded, but I am also a realist, and I learn from my past mistakes, and unless we get the core of the engine correct there is little point in adding pretty tinsel over it.
So today, we 'maybe' fix the animating corruption issue, tommorow I fix my HDD corruption issue and Thursday I dive headlong into shadow batching, which I can confidently predict will give me a 10fps speed up, which when you consider going from 26fps to 35fps, that's a substantial performance boost!
Well I Never
I write to you from my mobile as my main machine is in emergency defrag. It looks like it is going to be an HDD failure. Darn. Lots done today but can only report Tuesday if my PC comes back to me. Plenty backuos so code is safe.
Signing Off
At least I discovered I can blog from my phone. Very useful. Catch you all Tuesday. Fingers crossed.
