Wednesday Midnight Oil

Phew What A Night

After the daily grind of emails, minor distractions, conference calls and assembling a new FPS Creator Model Pack (looks pretty good), I had maybe an hour to do some Reloaded work.  I fell asleep watching Time Team after my tea and when I got back to the computer I got the feeling I need to do more.

FPSC Model Pack 73

I spent a quick hour trying out a few levels with this new model pack we are preparing for sale, and I must say it's a huge effort. The shear number of entities and the scale of the content is awesome, and I cannot wait to announce it once we have completed full testing on the product. Watch out for the release soon, and I am personally looking forward to what Reloaded can do to the pack when I drop it in a few months from now :)  After my quick model pack test hour, I jumped into the land of Reloaded...

Coming Up To 6AM

Technically Thursday now, but I burned through the twilight hours as is my occasional custom to get some juicy code done, and it's gone well. I now have a system that creates as many buffers as is needed to get the render done, and a few surprises greeted me along the way.

All my plans to create local buffers within the reference grid worked fine, which will now collect polygons locally in regions, which will be good later for occlusion work (visibility) and for performance and also for memory usage as I can abandon buffers that leave the scope of the camera without messing about too much with the memory holding data.  It's still all white, but it is a sea of white for as far as I can scroll.

I did a quick scroll from left to right on a 200 segment wide level and it just kept going and going, and when I imaged the size of room you would need to design to fill it, I soon realised that 200 is a fine maximum width for V1 of Reloaded, and I challenge anyone to make a building complex that needs such a wide run of segments :)

My next surprise was when I pushed it to the maximum in terms of memory. My initial prototype used one LARGE buffer to hold the polygons and I forgot to reduce the size of the buffer when I started doing the multiple buffer system, so when I asked it to create localised buffers every other tile, it gobbled huge amounts of memory per buffer.  Thanks to some careful coding, which ensures I am releasing everything I create, and putting plenty of error trapping in so when something does not get created, it gracefully ignores it and/or cleans it up, what I found was that as soon as it hit 1867MB the program simply stopped creating any more buffers or reserving any more memory, just like that. I could carry on scrolling around, it was just that the machine decided to not try to render any more rather than crash. A great testimony to some sensible coding practises.

I also added a system which improved initial memory usage too. Instead of reserving a grid of 200x200x20, each with a data structure to store the reference information of the instance, I realised this was a waste. Instead I am now storing a single pointer at each reference grid location and only creating the instance reference data when that grid location is used. It takes me from 37MB down to 3MB when initially creating the map, and it only gets back up to 37MB if you fill the ENTIRE level from top to bottom.  Further, I made the pointer a linked list pointer so I could chain more than once instance reference per tile, which when you think about it was pretty vital.

Signing Off

My next two jobs are to reduce the buffer sizes to something reasonable. Right now nine segments are typically stored in a single localised buffer (maybe 500 polygons), but I am reserving enough space for 666,000 polygons which is a little overkill :) Ultimate the buffer will resize once it goes above a determined average, but keeping buffer recreations to a minimum will improve performance.  The second thing to do is to add my 'Paradroid System' so instead of selecting the entire universe to add to all buffers, it will add and remove reference instances to the buffers as I move the camera location around. This will be the moment of truth as it will reveal how much memory the buffers need and whether the vanishing point for geometry is sufficient for real-time rendering. One solution to polygon overkill will be the use of LOD (level of detail) to use lower polygon content in the distance, but to support older model packs from FPSC Classic I want the new engine to handle content that has no LOD levels in place as there is no sense making a new engine that cannot do everything the old engine could do.  

Also, in case you thought this blog was itself a time consuming and distracting task from the pursuit of Reloaded code, this whole ramble took about 17 minutes and is probably therapeutic too.  Going to have a long sleep now and start Thursday sometime in the afternoon ;)