Sunday Christmas Holiday #3

Still Not At Work

Another unofficial blog to close off 2013. Probably going to have a few days away from the PC next week before the slog of 2014 begins in earnest.  To close off nicely, I have added a few more things.

More Non-Performance Bits

The crouch now works within the physics system properly so you can crawl under things, given the enemies a little more strength and connected weapon damage statistics to the shots so different weapons deal a specific amount of damage, solved the culling issues for static batching when things like tree veg is used where culling is not welcome, sky now updates when you enter test game depending on the previously selected choice and one or two small internal tweaks.

The most fun feature added was that I sorted the weapon firing physics, so now the weapons apply forces to the dynamic objects, can detect when solid objects block bullets, no longer shoots all characters in line of the weapon shot, added decal blood for character hurt points and dust flumes for all solid impacts that are non-organic.

Signing Off

Once I added blood decals for characters and gave them more strength, it was VERY apparent that I need the characters to react properly when I shoot them. Right now a decal spraying from them is not a convincing effect any more, and we need some kind of per-limb or general hurt animation to show they have been wounded.  As I have not done ragdoll yet, when I implement this I might find a solution there to create some good reaction effects. The cheat of course is a few extra character animations for basic body parts like head shot, body, arms, legs, e.t.c.  

Still, with the additions from today it's really starting to liven up with crates being shotgun blasted and blood flying everywhere as the player gets ganged up on.  Found myself doing a 'lot' of testing, that is, running around shooting dynamic kettles over and over again :)

Saturday Christmas Holiday #2

Still On Holiday

Spending much time playing Bioshock Infinity, eating food and chilling out. Started the day at 12:15AM (midnight) and ended the day with this blog around 4:30PM. Added lovely non-performance treats to Reloaded:

Sky Scrolling Support - Artists can add scrolling sky

Water splashes and ripples - Shooting water produces a splash

Flashlight now assigned to F key - Switch on a cone of light at will


HUD Directional blood splat damage - Directional indicator and blood splats hit the screen when the player is hurt

More work around the decal engine and bullet ray cast so bullets create a decal flume of dust when the bullet hits the terrain. Next will be bullets that can detect solid surfaces such as buildings and dynamic stuff like crates. Also makes sense to add a little blood splat on the characters too when they are hit, and allow them to be hit a few times before they drop.

With the extra decal work, scrolling sky, blood splats and better up-close combat it's starting to pull together nicely now.  Plenty more to come of course, but my minds-eye can see the mighty oak it will eventually become.

Signing Off

First official day back at TGC is 2nd February (January-Ed) 2014 so watch that space for a more bulky blog post. I had only intended to do a full year of daily blogging for 2013, and it's had it's share of controversy during it's time.  Have not decided whether to continue a daily one for 2014, or make it more of a twice weekly post with more pertinent content (and less waffle).  We'll see :)

Monday Christmas Holiday #1

What Are You Doing Here?

As you may know, I have officially clocked off TGC hours until January 2nd, so I repeat, why are you reading today's blog? Are you expecting me to stop my little holiday and carry on working for you, endlessly, without rest, forever?

My Christmas Listening Recommendation

I found Jose earlier Christmas Album on YouTube, and liked it so much I bought his new one (which he and his family created 22 years later).

http://www.cdbaby.com/cd/josemarichan14

My hope is that some of the royalties get back to him as I do like to reward creativity when I can afford to fit it into my universe.  Some of his songs (old and new), are so Christmas-sing-a-long-able, and now I have him on a loop in my office it feels like a Christmas office ;)

What I Done So Far

I have given myself permission to work on some non-Performance stuff as it's my holiday away from the day job.  I have added the decal engine back in, added muzzle flash on character weapons, entities now respect the SCALE field when in test game, physics now machine independent so player jump now works normally, characters can no longer be penetrated (oh er), characters now fight melee better and player now gets knocked back when struck with some cool camera tilt action too.

Signing Off

I am quite excited to do more decal related work, so things like water splashes, bullet impacts, blood splats, this sort of thing. Not for any reason of priority, just because I have one email in my inbox that's had water splash decal textures in there for months and I want to get rid of it :)  Hope your Christmas is going well so far!

Friday Ho Ho Ho

The Last Day - Officially

According to the spreadsheet I laughably call my Holiday Calender, this is the last TGC work day.  Being CEO and Founder, you might think I would not bother with such a regimented system for recovery and rest, but it's all part of the magical machine. 

I say officially because I don't imagine, in my wildest dreams, that I am actually going to clock off this evening and then return to work on the 2nd January 2014.  I am pretty sure my time between here and there will be occupied between playing some FPS games and adding to our FPS creator :)

In testing the latest beta, due later today, I found myself in 'play mode' and came up with this shot:

I started to imagine all sorts of cool things such as shop keepers I could trade items with, civilians walking about minding their own business, ladders so I could get to the roof tops, closed and locked doors and windows I could bash or pick open, the sounds of the night and the daily buzz of a small village, some creepy zombies hiding in the shadows, dynamic lighting effects for camp fires, lanterns, shop interiors and glowing eyed vampires, dust and tumble weeds kicking about in small wind swirls, maybe some light rain causing the cobbles to slowly reflect the buildings above, cloth physics hangings draped from open doorways and windows.  All this was swimming in my head as I wondered around my completely empty village, and it was exciting to know that most of the hard work is either behind me or in my immediate sights. I knew 2014 would be the year when all my imaginary features would slowly transform the reality of the engine, and found myself tempted to start all of them at once.

Of course the reality is that I am but one person, and I chose a very small team to help me get there.  For this I apologize, and beg your patience as I build this edifice one brick at a time.  I want this castle to stand the test of time. Other coders thought I was daft to build a castle here, but I built it anyway. It sank into the swamp. So I built another one,  that sank into the swamp. I built a third one...that burned down, fell over then sank into the swamp. But the forth one stayed up, and that's what you're going to get from me, the strongest castle in these isles.

BETA TIME

I am presently putting final touches and testing to the 1.004 BETA version, set for release later tonight, which I think you will approve as a good version to play with over Christmas.  As I hinted, I will be continuing the development over the festive season, focusing a little more on some critical yet fun stuff like more sounds, basic logic that's missing and any critical horrors that are found with 1.004 in December.

I have not decided whether this minor Christmas patch will be for GOLD pledgers only, as it falls outside the scheduled releases, so if you're not a GOLD pledger I strongly advise you to pledge now while the discount is in effect so you can get your hands on this 'potential' fun update ;)

Signing Off

Still plenty hours in my day so will carry on testing and tweaking. It's 6:44PM here in the UK and I aim to start uploading the final installer for 1.004 around 10PM so wherever you are in the world, do the time-zone math and get ready for a new update!

Thursday All Nighter

True Imposters Integrated

I had a deadline of having an internal testing version for Thursday morning, and due to the nature of the remaining work I had to work through. I started what I thought was a simple clean-up and integrate at 3PM on Wednesday and have just now finished at 7AM on Thursday (with about two hours break). As this is technically a lot of my Thursday energy used up, I thought I would make my blog now, just in case I sleep through the rest of the day.

It was grueling, but managed to solve the occlusion popping issue and tame the quad renderer in time to make an installer and upload.  The quad stuff was a real challenge and included a LOT of stuff I had not really considered such as regenerating based on various camera factors, requiring multiple quad buffer data structures for each camera and the balance between per-cycle workload and overall performance in real terms.  Fortunately, much vexing and trial and error solved the issues and I was able to come up with something passable.

Quad Visuals

I am still not entirely happy with the transition effect between the LOW LOD model geometry and the QUAD imposter, one because there seems to be some lighting differences between the two renders and also due to the clipping thresholds which will need to be model specific rather than hard coded into the shader.  I am pretty sure the lighting is due to incorrect use of shadow and dynamic lighting, both of which I can either simulate in the quad render or alternatively, to fade the LOW LOD render slowly into the lighting attributes which the quad uses. The former is more desirable, but may have a cost given their distance from the player camera.

Remaining Things

The goal of a 1.004 beta for Friday night is now very reasonable to achieve given the work just done. I have a few tweaks I want to get in, such as saving the slider settings between sessions, and perhaps some way to reset them. The question of whether they should be saved with the level files, or saved globally is something I should like to think on. I suspect both, but I will need to come up with a system that is intuitive and not clumsy.

There are some other tweaks that are less important, but would transform the overall look and sound of the beta to give it a nice feel, but we will see how much actual time I have for these trinkets.

Signing Off

Hopefully I can get the larger bitmap fonts for the ammo and health numeric before the update as I think it would improve the in-game stat panels quite a lot.  Eventually we will have a lot more stat panel choices, but for now I am aiming for something generic and functional, also clean and polished so it would not look out of place in any modern game.  There are some other graphics in the wings too, but I will leave those as a surprise for when you get the next beta update!

Wednesday Walk

Yes, And No

It is true that I went for another walk today, but this time I came straight back and did not go to sleep :)  I gave myself one hour before dinner to finish off the prototype with communal textures which will allow me to populate a single texture with many quad texture renders. It's now two hours and I've just finished the quad texture regeneration system and not even started on the communal multi-render stuff. Just goes to show how even a simple developer task can be broken down into small unpredictable extra work that had no previous time estimate assigned.

Imagine, a task that I did not even think about, and did not have a time estimate for, taking twice as long as the task I started to do. Now multiply that few hours with those in week, and then an entire project and you will get a sense of how an estimate and reality can differ. I set myself an hour as I was famished, and as I write this I am doubly famished. Will stop now and eat, then return to do the task I set myself at 5PM :)

Signing Off

In the two hours of quad regeneration work however I tracked down some bugs in the engine which caused HIGH and MEDIUM LOD objects to render over each other which was rather redundant, and introduced the system which only renders the quad texture when it absolutely needed to be done. I also found that I needed to update the quad vertices each cycle in order to keep the quad polygons rendering to the screen smoothly.  Whether this will effect overall performance when we start locking hundreds of buffers, we will see, but I wanted to achieve super-smooth AND speed. When we have some real world levels to throw at it, we can then decide to keep speed and have the quads stepping along, or keep smooth, and sacrifice some speed.  One of the many thousands upon thousands of tiny decisions that do into making a games engine.

Also got an email with a prototype showing some rather nifty player hurt, directional damage and blood splatting effects. I am now resisting the urge to play with that integration (five minutes work which is actually five hours work) and remain focused on two objectives. Finish the prototype for engine integration and solve the object popping caused by occlusion. I can then prepare an installer for some internal testing before a planned release for the weekend. I think you will like this update, we have lots of tweaks in there!

Tuesday Got IMPOSTERS!!

And Phew!

At long last, today I saw my first peek at the working technology of true imposters within my static batching and occlusion system.

Impressive I know!  So what are you looking at?  Well the blue square is the background color of the camera and nothing to get excited about. The red square is our quad imposter which has been calculated automatically based on the position of the camera and the target object. The building in the center is the target object and it is being rendered via the imposter pixel for pixel exactly where the real model would have been rendered. Instead of hundreds of polygons, there are now just two.

Shadows About To Get Fast

The reason I show you a view from above is that this is the perspective the shadow light sees objects from. By using imposters when rendering the shadow objects, we will vastly accelerate the entire shadow drawing cycle too. Not forgetting the reflection views and light ray cameras!

Signing Off

I still have to do some finishing off and house-keeping, and of course add multiple target objects per communal texture, and then get it integrated into the main engine, but the crazy math part has been done, which is a big relief for me.  The immediate goal for Wednesday is to stop the popping effect currently vexing the Reloaded community, but the side effect of solving that should be some additional performance improvements.  That, together with the shader panel, might be enough performance to allow us to shift the percentage over to some non PPP work, such as player damage, enemy strengths and some improving tweaks.

Monday Halved

Four Hours With The Secret Coder

A good early start to Monday preceded a marathon four hour conversation as I prepped the new coder for his first foray into the Reloaded engine. Covering everything from brand vision and project scope through to coding style and the new DBP compiler.  He is now armed with most of the tools he needs to produce something cool for Reloaded.

The Walk

I then went for a walk before dinner, which turned into a long walk and I only got back at close to 5PM, picking my car up en-route from the garage. After some refreshment I suddenly felt fatigued so had myself an early night.

Signing Off


For some reason, my previous entry for Monday was half-deleted and unpublished, so I am publishing it again now. Hopefully it will stay undeleted long enough for you to read Monday's capers.

Sunday Weekend @ Work

Busy Busy

As the weekend was populated by rain and wind, and wrapping presents, I decided to give a few hours over to some coding to make up for any lost hours during the week gone.  Turns out Saturday I clocked in over 12 hours and made some nice improvements. 

We have been internally building a list of priority A bugs that need fixing, but performance came in at an even higher priority so these duties where postponed. Performance is still in the works, but with my recent success with shader reduction I decided to knock a few of the more undesirable issues off the list and improve the engine.

What Improvements?

The most striking is that you now have a weapons and health HUD panel when you place down a Start Marker, fully functional with final graphics. I am not too happy with the size of them, and the font used, but otherwise it's in the ball-park. To take screen shots, simply remove the start marker from the scene (or don't start with a weapon and set health to zero).

As you have read, the start marker weapon is now respected so you can start with a weapon and ammo, the weird bar at the bottom of the test game screen has been removed (a real pig that one), foliage no longer causes collision issues or AI obstacles to be created, allowing the character to run through low grasses and shrubs. Lots of little tweaks you probably won't pick up on but had sat in the list for a good while and was starting to alloy me.

The Secret Coder

I have logged on Sunday for one purpose, which is to prepare some files for a new coder to the Reloaded camp. His mandate is simple enough, and for the moment I will keep his task secret, but he brings with him as much experience at coding as I have (which means he's an old git like me).

At the start of the year, I re-designed the new Reloaded engine to allow modules to be worked on independent of the main code. This has been used to fashion small prototypes that could quickly implement and test sub-modules of the main engine and allow super rapid development. The system now lends itself to allowing another coder to share things like the data structures and common helper subroutines, but implement a wholly separate piece of code which can drop into the main engine without resource or coding conflicts.  It is this system that will be tested over the next few weeks.

The bottom line is that you now have 'potentially' two Lee's working on Reloaded, and I will be happy to report the results of this experiment.

Classic Makes Good

A bit of retro news for you. The original product FPS Creator Classic was created many (many) years ago and has been responsible for some great games. Two such examples have made it into the top 100 IndieDB charts and is now spoiling for a top award. The games need votes, and every vote is one for the FPSC brand, so I invite you to go and help the author click his way up the ladder.

http://www.indiedb.com/events/2013-indie-of-the-year-awards/top100#vote16039

http://www.indiedb.com/events/2013-indie-of-the-year-awards/top100#vote25758

My hope is that in years to come, Reloaded will be the instrument that will allow many users to create IndieDB chart topping games, showing the same level of customization as 'Into the Dark' and 'Into the Ice' reveals.

Signing Off

With shaders (for the moment) done, on Monday I will be returning to the subject of QUADs (True Imposters) and finishing what I started. I could readily leave this out for the next update if it was not for the fact that objects are popping in and out of existence. A handy quad will resolve this visual artifact and at the same time move us in the right direction. A substantial byproduct of this work will be the acceleration of the shadow, light-ray and reflection renderers which means high-end users are going to get even higher. I could do the work today, but I think it makes sense to have at least half a day off, maybe playing some relevant FPS game to stay tuned to the overall goals of this tool.

Steam Is Green

One of our ambitions is to get Reloaded onto Steam, which means getting a Green Light, helped along with public votes. We are off to a flier, but need to keep the momentum going. If you know anyone with Steam, invite them to check out the page and see if they would also like to see Reloaded reach millions of new FPS players (and budding creators):

http://steamcommunity.com/sharedfiles/filedetails/?id=203481310

Also, if you have ideas how we can improve our steam page, description, shots, videos, e.t.c do get in touch!

Friday Fun

The Storm After The Calm

A good rest does one a world of good, and I started at noon today and just finishing off now at 45 minutes past midnight, so a good long stint. Made a huge boo-boo half way through when I realized I was operating from two different source code bases and in danger of loosing some work from some unspecified number of days. Fortunately WinDiff came to my rescue and I once again had the best of both worlds compiling as normal.

As you can see, a new Shader Panel has emerged which can set the terrain to MEDIUM, which retains the texturing and fog effects, but removes normals and shadows from the mix. I plan to continue working on this shader level, plus add a few more so that you have a good selection of visual options that represent the best compromise between visuals and performance.

So What's Been Done

The shader panel, which has been my sole goal today, is now refined and working well. It can step right down to the most basic of Pixel Shader 2.0 shaders, allowing us to get a real feel for the performance 'starting point' for low-end devices.  High end effects are unaffected for those with meaty systems, but I am hopeful that my shader work at the lower end will start to show some real gains.  From these benchmarks, I can decide which visual effects to promote into the shader until we basically run out of GPU cycles.

Too Ultra

I am already getting super frame rates on my Ultrabook now, so I decided to dig out a Windows Surface Tablet (Intel ATOM processor using a GMA chip). As a point of comparison, running the 1.003 software fully loaded AND at low settings I got 1-2fps, and was essentially un-usable.  I am not sure yet whether this should be a platform to target, but I do know there will be a lot of devices out there at this processor level and you can imagine some users will want to play their FPS games on the Windows Tablet.  I am not going to obsess over this far end of the spectrum, but I will be very interested to see what basic shaders and a massive reduction in polygons would yield here. If the engine was able to create and play games at this level, it would effectively span any type of system you might have and create a very accessible solution.  We will see...

Signing Off

My last task for the week will be to build an installer from this version which has the shader panel and then try on a few systems I have here.  It will also be a stand-by version should the QUAD work which is next week turn out to be a nightmare (again).  I also plan to do a little over the weekend to make up for my transient life earlier this week.

I must also thank Rodrigo for his suggestions for shader improvements, who went to the trouble of going through all the shaders and documenting small savings which no doubt will add up to a few extra FPS units for some of you guys out there.  I am sure the community will want to thank you as well!

We Want Your Vote

One last thing before I go. I noticed in our new Voting System that Performance is scoring less than the Character Creator feature :)  

http://fpscrvote.thegamecreators.com/

Fortunately I started Performance work so that box is being actively developed now, but it was interesting to note that you are using your votes to push a new feature into the engine ahead of more performance. It will be interesting to see how the chart moves after BETA 1.004 :)

We Want Your GREEN Vote Too

Today we also launched our bid to get a Green Light on Steam, which would open the doors to a lot more eyeballs and interest in what we are building. I encourage you to check it out, sign in and click YES to Reloaded's invasion of Steam :)

http://steamcommunity.com/sharedfiles/filedetails/?id=203481310

I am not sure how many YES votes we need, but once I get some statistics from somewhere, I will keep you updated on how we are fairing there!!

Thursday Slumber

...he slept muchly that day...tune in Friday night for more exciting tales...

Wednesday Has A Shader Panel

Mobile Coding

A strange thing using the Ultrabook as a daily coding machine, not being able to reach for my mouse and relying on arrow keys so small you need laser guided fingers to hit them :)  Still, it's quite liberating, and in a way forces me to think more consciously of users who use these types of devices as their main machine. Which I think is a good thing to be thinking about!

The Progress

We now have a shader panel in the software which can control terrain, entity and vegetation shader levels. That is, select from a range of techniques from HIGHEST which is everything plus the kitchen sink down to LOWEST which is the most basic rendering path you can imagine.  I have read the comments about whether time should be given to the lowest of the low systems, but I figure this low end work ultimately benefits the high end result (as it will get faster as small corrections and refinements at the basement level iterate upwards to the lofty heights we enjoy in mid-range land).

Signing Off

Almost completed my exile from the office, and should be writing the next blog from my PC beast.  I am pretty confident BETA 1.004 will deliver sizable performance increases for those who have yet to see their FPS value budge. I DO think it's important that lower end users can run at a decent speed, and get to enjoy game creation and play back, even if their visuals are not as drop dead gorgeous as the high end ones. Let's face it, users invest in spending LOTS on high end equipment so they CAN see better bigger faster visuals.  Once the majority of the Reloaded community is off the ground and flying with the software (and not stuck in the mud at 10 fps), we can then address the concerns over visual touches and tricks to recover those aspects you feel are important at the low end.  You can probably sense my approach to working, in that when I say top priority is given to performance, I pretty much exclude all else to that end.  I was reminded of this fact by Rick on Tuesday, in that I am even excluding features that used to work (and the fixes that would remedy those changes).  Rest assured as soon as shader panels and quads are in and working fully, the remaining time will be spent triple checking everything that used to work still works, and we have a successful 'positive' update ;)

Tuesday Meet Up

Out And About

As you may know today was a meeting day so between driving and talking, not much left in the tank for coding tonight. I did show off that speed increases are possible on integrated systems with room to spare, which means we can retain some level of visual fidelity in the modified shaders.

It is not realistic to get ALL the effects, running ALL shaders at white hot speeds on low-end hardware and expect high frame rates to compete with the mid and high end systems.  There MUST be compromise. The trick is in compromising those shader hungry effects that are not immediately apparent, thus allowing performance to increase without significantly affecting the end shot.

My jibe at removing EVERYTHING from the shader was really just to highlight my starting point, from the point of view of performance. The work now is to introduce shadows, texture variations, normal mapping and other elements in an efficient way, and as there is no one solution here I will probably opt for several shader levels to choose from. It will then be up to the individual creator what settings they prefer for making games, and what settings they recommend and preset for their users.

The End User

We will be having option screens so the end user of your game can likewise choose which shaders are employed for best results, and during the meeting we talked briefly about being able to 'post' recommended settings filed by graphics card name so new users of Reloaded and it's standalone games to quickly find and apply the best settings as discovered by fellow members of the community.

Signing Off

I plan to work on bringing the basics back into the terrain shader Wednesday as I am pretty sure can restore unfiltered shadows and terrain selection without hurting performance, and I have a question whether you can really appreciate the normal mapping as you're running through a level (and more importantly, whether it's worth the few FPS it costs to render them on low-end systems). Similar measures can then be applied the entity, weapon and other-game render surfaces. Combine these with the savings on scene and terrain polygons, we might start to see some serious improvements for many Reloaded users.  

Whether the sacrifice of visuals is apparent or welcome is the subject of another discussion, but it may be possible to supplement any short falls with some pre-baking. Many modern games that run on low-end positively depend on various forms of pre-baking to get the performance you expect, whereas Reloaded is attempting a complete real-time engine with zero-pre-baking.  In laymen terms, pre-baking is sitting and waiting for your level to 'build', and something I really want to avoid for as long as possible, and hopefully find a nice solution that satisfied the majority.

P.S. If...

Anyone wants to have a look at my terrain shader (in effectbank/reloaded/) and suggest how it can be made 'more efficient' without loosing 'any' of the visual effects, please do get in touch ;)

Monday Ultra Look

An Ultrabook Day

Normally I plant myself into a seat facing a mid-range PC. Today I am in front of a Ultra-mobile PC experience typing from my lap. My mission was to see what I could gain from an integrated graphics chip, and the best way was to use one to do the testing and tweaking.

Shader Slicer

I began and ended my work chopping up the shaders until they started to return performance. The bottom line is that I went from 18 fps to 40 fps by reducing the complexity of the shader, and when I switched off some of the features I got upwards of 70 fps with a terrain and some characters running about.

It is fair to say my shader slicing removed some things you might have wanted to keep such as multiple textures, normals, shadows and other small effects but the experiment was to see 'WHO' was spending the frame rate.

It was clear that on an integrated (and I suspect low end card), that the shader is real bottleneck here and it's a case of figuring out which things I can keep and which things I cannot have to get the performance and visuals that pleases. I dare say some pre-baked processing and hacking would be required to get the visuals on lower end systems, but it's a great start to see the performance hit 90 fps in some cases on what is essentially a fan-less graphics chip!

Signing Off

I still have some reductions to chase through quads, reduced terrain polygons and some more occlusion tricks but it is clear to me now that it's the shaders themselves which slow down the non-mid range graphics solutions. From this info I can add a few extra settings to the slider system to allow these ideas to be tried on the variety of low end systems out there.

Sunday Shaderthon

Revelations Chapter One

Spent a little time today on a different tact from the usual chase of fewer polygons and draw calls, and made my home on the Ultrabook I often use when I am about and out. My latest effort did quite well at 15 fps fully loaded and over 40 fps on silent running, and together with some anecdotal evidence from one of our beta testers, it occurs that perhaps there is gain to be had by chopping down the shaders after all (as right now they are consuming over 85% of my Ultrabook processes). I tried this before but tested on my monster mid range card and so no benefit. I started to suspect I WOULD have seen a benefit had I tried it on my mobile computer!

The Work

Therefore, my work for Sunday evening will be to trim the shaders directly on the Ultrabook and see what gains are to be had. If they report well, then it is very likely this solution will work for many users in the same boat as a few respected testers who have reported almost no performance improvement since I started my antics. Hopefully this is the key that unlocks that door to happiness.

Signing Off

I am on the road over the next few days so my blogs may come from strange places. My mission will be to continue working on performance, through better use of draw calls, polygons and now shader code. I know it can seem like nothing new is being done, but I can assure you, and I think I have a few supporters who will agree from long experience, that the absence of 'new things' in this case is a very good thing.  Keeping Lee on performance until we crack the case shows that we've learned something from products of the past and brave enough to stick to our gun.

Friday Progresso

Some Good Math

A quick blog today to say I have now gone through the new maths for True Imposters and quad texturing, and it's all pretty plain sailing now. I am still in the prototype dealing with front facing and top down facing quads, all of which need to be in the engine before I move to the main software, but it's going well. I plan to do some work over the weekend so I have my quad system in place for Monday week. Have 'another' meeting on Tuesday and want to show off the result of this little side track into the world of strange imposters.

Signing Off

Might be a Saturday/Sunday blog to make up for this short one, so check back over the weekend in case there is a surprise for you :)

Thursday Email Mountain

The Email Tractor Beam

It's funny how some days you can get a lot of coding done, and other days you can get none done. Ever since I clocked on around 1PM (slowly getting back to day shift) I have been answering emails, writing 'Ask Lee' answers, writing up reports, reading articles, Google pinging and sketching out little paper ideas, I've not had any actual code time :(  

I have got some good news in that the recent 60 FPS performance work has now been tested on a few other internal team systems and the improvement is definitely measurable.  This bodes very well for the next BETA update and so I can promise you have something to look forward to!

Signing Off

The good news is that I've been thinking of the imposter system so much the actual work should not take too long (in theory).  Going to have something to eat and then see if I can get a few hours of coding in, and hopefully get a big result from my quads!

Wednesday 60 FPS Day

Performanciniaring

Another signature day in the saddle of the Good Horse Reloaded. As you know (owing to the unforgivable lack of a blog post yesterday) I was in Manchester on Tuesday having a meeting and in the process having a good time. Back in Wales now and the code fest continues, and seemingly as a result of my short break (and a rather nice Irish Coffee I had on Monday from a rather nice bar-dude) I have ACHIEVED one of my private goals of getting the 'Run To The River' level up to 60 fps!

And how did I manage that I hear you scream!  Well...first I fixed a bug which caused ALL the grass to render in the shadow camera renders, which is not required at all, and then second thing was to detect (using some occlusion commands I wrote a while ago) whether the water quad was rendering any pixels that where visible on the screen. If that value was zero, I skipped the reflection camera render.  These two measures saw me jump from 40 fps up to 60 fps and the end of one of my little goals.

A New Hope

I need a new 'mini' goal now of course. Not the usual stuff that we all know needs doing, just something I can giggle at when I've achieved it.  To keep it simple, my new goal is to get over 100 fps on the same sample level as I think it is reasonable to expect a small compound with a few buildings and characters in should not tax a professional games engine too much, and as the goal is not 'mandatory' it should make the achieving of it all the sweeter!

Signing Off

Another goal I'm aiming for is to get back on day shift. It's fair to say I'm not feeling any worse than I did last week, but my body does feel out of sorts. Going back on the old 9 to 5 is my first step to correcting this case of affairs, and as the time is 6:35 I'm going to clock off and get an early night. Thursday will see the final coding of the True Imposter system, but this time with some proper maths!!

Monday Jet Setting

Signing In

A very short day today, consisting of emails, this blog, packing some bits and then a nice drive to Manchester. I have a meeting all day Tuesday which might provide some extra ammo in the world of Reloaded. More news on this if the mission is successful.

Signing Out

Continuation of performance work will resume Wednesday, as it has been almost unanimously agreed that PPP should remain top of the list for the time being. The mileage of the results you may have got with BETA 1.003 will vary wildly and I am not too surprised at the reports of slowdown in some cases and substantial increase in others.

I suspect the slow down comes from lower-end systems which are having to do more CPU work to calculate occlusion, and on the GPU side building vertex buffers dynamically might upset some graphics cards which do not expect this approach. Many games would build the entire world as an optimized static soup of polygons, and NEVER build them in real-time. It could be that once we have identified and singled out a group of machines that are reacting negatively to dynamic building of static batches, we can use an alternative technique that best fits those cards.  If anyone can demonstrate a substantial slow-down of a side by side test of 1.002 and 1.003, please send me your CPU and graphics card details so I can start to get an idea which kinds of systems react in this way.

Saturday Report

Advice Taken

I took my bodies advice and slept for over 12 hours, then logged on long enough to check for any urgent emails. The plan was to find nothing and log off immediately, and so to enjoy my weekend off.  Alas.

First thing I saw was that one of the internal versions had escaped, and was the star in a blog video on the FPSC-R forums ;) I knew it was an internal version on account of the sand cubes where corrupted, which has been fixed in the official final version of BETA 1.003. I also got an email from Rick green lighting this version for release, so I stayed logged on to supervise the release of the beta for today. Upload takes a while, so watch your Products page and your inbox for news of the new update.

Occluding In Brief

As you explore the new update, you will notice two things. One is that your frame rate goes up (or should) and the second is that some objects are disappearing (and reappearing) on you.

The visibility issue is due to the way the occlusion system has been created, and relies on the fact the larger invisible bound-box that surrounds each object will be visible BEFORE the actual object to which the box relates is visible to the player. Unfortunately there are situations where the player can turn a corner and surprise the system, meaning for a second the object is not there and then it appears. This happens for only one render cycles, but it is enough to spot the artifact!  I am currently looking for options to minimize this effect but all the Google links I found so far elude to the fact that this is acceptable and common for occlusion systems.  If anyone can find a link which identifies this issue, AND SOLVES IT, then please let me know through the comments section.

Another issue you MAY spot is that really distant objects now completely disappear. In the final version, objects at this distance are replaced with quads (two polygon billboards) to fake the object's appearance for increased rendering performance. Alas my R&D into the technique of generating the textures for these quads over-ran the planned release schedule for this update and so the quads had to be hidden for their own sake.

A Secret Tip

If you want to see what the quads look like, you can comment back in the two CLIP commands in the entity and quad shaders.  I only recommend this if you don't mind the technical challenge and the less than satisfactory visual effect.

A Special Thanks

All is not lost however as on the eve of the official 1.003 release, I received essential advice from my new best friend Matty, the author of DarkImposters who had already solved the issue of projecting objects onto quads and very kindly shared his secret sauce with me.

I have glanced through his equations and I 'just about' understand the technique used, and am now very eager to code this into the engine and see what happens.

The Question Of Performance

We still have a LONG way to go on the subject of performance, and I still have some hellish optimizations I want to attack the engine with. I did take a little time out to solve some legacy model issues and adding the store to start the whole legacy support ball rolling, but I am curious what the community (you guys and gals) would have me do at this point.

I could continue to work on PPP (Performance Performance Performance) at the exclusion of all else, or I could blend in a little non-Performance work such as more legacy support, Weapon HUDs (of which the art is now available to me), auto-fence builder, adding the RPG, e.t.c. I am sure I will do one or two of these things for the next update in December, but I am curious to what percentage the wider world would set for this division of labors.  Imagine it was another slider between 0% and 100% to control Lee's non-performance activity :)

Signing Off

I know it's against policy to blog at the weekend, but I have 450MB of data to upload and decided to give you an update while I waited for the files to leave Wales. I have a meeting Monday and Tuesday next week so I can't imagine I would have much Reloaded news to blog, but if anything does occur, I will be sure to include it!

Friday Finally

Captain Sensible

In order to put my degenerating over-worked body on a better footing, I started today as normal as I could manage. Got up at 7AM, breakfast, work, dinner, work, finish up and test, upload.  Knowing full well the quad situation could easily soak up all my remaining daily hours, I decided to shift focus to getting a solid update ready for the Alpha Cascade. It's uploading for an internal test now, and then I will be ready to uploaded to the main FTP this weekend if Rick agrees that you can all enjoy the performance improvements so far. He's also prepared to delay the update a week in order to add more polish to the release before you get your hands on it, so watch this space for that!

What's In Store

As I did not find the time to code the auto-fence maker, I quickly added in the Game Creator Store feature with a few modifications for Reloaded.

In the shot above, I have taken a rather cool refinery looking construct from the store and added it to a simple level.  I took the liberty of marking the item as Reloaded compatible as I have since fixed the collision and orientation issues that originally prevented this model from working properly. It is also the first and only Reloaded approved model in the store right now, but rest assured more will quickly follow now the legacy support is going into the engine as each new issue is identified and resolved.

Some other issues such as static object collision has been improved so there is no more falling through ladders when you try to make a bridge out of them, you can now climb the staircase object and you can no longer sneak through invisible gaps in the fencing.  I am still dubious about the dynamic collision shapes, but when I have more objects to test against I can put those through the mill.

Signing Off

I have temporarily disabled quad rendering, and enabled extra low LOD static buffers. The shadow, reflection, light ray and the main camera all use these dynamic static batch buffers now so you should see some speed improvement. I once reported my 'run to the river' demo went from 24 to 29 then up to 50. Well it went back down to 42 once I re-activated some needed rendering code, but it's still heading in the right direction.

I have since learned some new techniques and have had a few ideas of my own to solve the final quad problem, so hopefully we can re-introduce quads next week to gain some more performance.  I also have a mind to add some code to the terrain system to replace a field of polygons with two polygons AND apply occlusion techniques to the terrain segments too, to see if that helps performance.

Although occlusion is in and working, I am not 100% happy with it yet. You will read in many occlusion articles that the momentary 'popping in' of objects once the engine has determined they should be visible is acceptable and barely noticeable. Having tested it for many days, I DO notice it and I DON'T find it acceptable. The solution will probably hog some performance as I suspect I will have to do a pre-draw with the occluded shapes, stall the GPU then draw the 'now visible' once-occluded objects. When you get hold of the update, check out the artifacts on this and do send me any links you find from anyone in the world who has solved this 'popping in' effect without huge GPU stalls.

Wednesday Or Thursday

Blurry Work

First up, thanks for all the calls to action and recommendations of rest, a very sensible idea and anyone listening would be a fool to ignore such sage advice.  
Unfortunately software seldom writes itself and deadlines have a habit of getting closer and larger, and it would not do to just leave the update half-finished which is why I must carry on and finish it.  Perhaps I can find a few days to rest after the update is out the door!

Perspective Pains

The more I work on this QUAD texture rendering, the more I am convinced my brain was not wired for 3D thinking. I've literally spent 10 hours on the same thirty lines of code, trying every combination of crazy ass ideas I can think of to get a section of render to re-render centrally at the correct angle and distance. You would think it was easy :)

The good news is that I am making progress, all be it slowly. I figured out some neat things like how you can scale up the perspective matrix to create a sort of zoom effect. I am currently working on a way to tweak the view matrix generation to tilt towards the target object that needs rendering as simply shifting the view camera was not enough.  I also need to handle the multitude of little attributes for things like resolution changes, aspect changes, camera position and angle changes, object dimension changes and then quad resolution and densities which can be adjusted in real-time.  It's clear to me that this is not a 'few days work' as I had originally thought and will take some considerable time and thought to become fully rounded and complete.

Signing Off

A nice cup of tea me thinks, and then see if I can get 'something' concluded from all this work, and build an internal version for Ricky baby.  The next update is imminent so internal testing is the plan, I only wished I had more done to test ;)

Tuesday At Quadington

Short Day

It's very rare I allow physical discomfort to stop me coding, but for whatever reason Tuesday has been a real mare for me. Might be overwork, or just the weather, but my energy levels are barely registering.  I did however have the professional sense to make today count so I booted up and carried on regardless for a few hours.

Quad Progress

More progress made here, with each quad now having the correct UV coordinates assigned, the viewport system rendering a single object to each gap in the community texture and I have started the code to correctly align the view camera with the target object.

I am currently struggling with ensuring the view camera alignment remains unaffected, and to work out the optimum distance to place the camera from the object to encompass the whole object and match it perfectly with the quad size too.  Doing this correctly means the quad system will work from any camera, and any angle, and any quad size.

I remember my early days at school and I don't think we did advanced 3D maths at GCSE level, but I wish I did.  For what might take a modern programmer five minutes to work through the relationships between all my vectors and matrices, will take my brain a considerable amount of time, usually through trial, error and stubbornness.

Other Bits

Also managed to catch up with my emails, do this blog and start one or two plates spinning so no-one is waiting on my feedback.

Signing Off

Hopefully Wednesday I will feel better. For now, I am going to put the kettle on, watch an episode of Time Team and then get an early night.  I did not sleep a wink (that I remember) in the last 36 hours so hopefully I can drift off. The quad system is the last visual hold-up to the next update, as after this it will be clean-up and testing which requires less 3D math and more donkey.

Monday Quadz

Houston, We Have A Quad Texture!

After much tinkering inside the bowels of the engine, we now have quads using a render target texture created entirely on the video card and rendered with the object that created it.  Here is the glorious mess for your enjoyment:

As you can see, we are rendering the whole texture for each quad, and rendering all the objects to the target surface as they happen.

The next step, now we can see something along the right lines, is to sub-divide the texture into grid squares, to use a view-port to only render to the correct square, to only render the object once per generation request, to modify the quad UV to only show the texture that applies to it's own original object, and so on.  I had thought I would be further along, but it's always a good place to be when you render something and you can actually 'see it'.

Signing Off

It seems my body is once again on night shift, so I write this at the end of my day which is about 5AM.  Hopefully I can get up before the sun goes down as I so light to see some daylight before I start my day.

Sunday Pottering

Legacy Bits and Bobs

After spending MOST of yesterday playing FEAR 2 and the start of today watching an episode of The Planets, I decided to boot up the ol' computer and do some non-performance coding to give my brain a break.

What you are goggling at are three objects from the Game Creator Store, which is being integrated for the next update.  To this end, I spent half of Sunday making sure some legacy models and entities would load in first time without modification.  I am certainly not going to make Reloaded backwards compatible with everything in the store, and will be adding a filter by default to show only Reloaded content, but this filter will be adjustable so you can find older legacy static entities and see how they fair (including any that you have bought previously via FPSC Classic).

Occlusion Works

Also made some continuing fixes with the new renderer to stop it flickering due to multiple queries and cameras confusing it.  I am saving the true imposter work until Monday as that's a full work day of headache that one!

Signing Off

My plan for this coming week is to have a solid version by Wednesday, on which plenty of testing can be performed THU and FRI. The old regime of releasing new updates that break old projects should really be put to rest here at TGC, and we do that by spending two days NOT adding features and JUST testing (that is, if I don't get strong armed into adding anything during those two days).  The objective is stability, consistency, THEN performance, THEN the store integration. You can then be at least assured that the update is not break stuff you have working right now.

Friday Foundations

Lee Stops Digging

After many days of performance tuning and investigating, I have finally got to bedrock on everything that spends GPU calls.

As you can see, when looking up at the sky there are only SEVEN draw calls now being made. Four of those are sky box surfaces (one for each 'side' of the sky being rendered), one for the post process quad to render the main camera to the screen, one for the dummy terrain object which is needed to regulate the terrain shader system and finally one for the world sized water plane.

I also discovered the cause of the 'super high drain' from the post processing which was caused by the lightray camera rendering everything, even if you switched it off in a previous session, now fixed.

After hacking my last few singular calls out, and noticed no performance gain beyond the high 300's mark, I decided to leave them be.  I am now curious what FPS scores users will get from a blank scene looking at the sky with everything switched off. I agree it's not the makings of the next killer game, but it will be very interesting to see how low-end systems handle this little experiment, and if they DO perform badly, then we can start to look elsewhere for the culprit such as the CPU, swap files or even the panel that displays the score in the first place :)

Further Ideas

There are one or two further ideas beyond this minimalist position, such as using the occlusion to detect if the water plane pixel(s) was rendered, and if not, hide until the occluder says it is visible once more.  The reason I am not pursuing at this time is that with the new front to back draw order, all of the underground water plane is Z-rejected instantly so does not hit performance except for the extremely minor call hit.

Moving On

Now I know every draw call personally, and can account for everything the GPU is doing, I can start to build things back up.  One of the first will be the True Imposter System which will create community textures for each Instance Stamp buffer and then GPU render objects into them to provide my quad textures.  It's a priority as right now I am using a place holder 'building' texture for the quads and allowing that to be in the next update would make all your screenshots from that version very odd indeed.

I also have it penciled in to add some more legacy support too, but more on that when I discover I have the time to do what I have planned. It should be a welcome addition if it all works out :)

Signing Off

Well I did say it was going to be all Performance, Performance, Performance and I think you will agree I have been rather single minded on this point. There is more performance to be had elsewhere such as faster physics for polygon style objects, faster AI by diving into the source code and seeing how I can save speed and of course going through each shader and seeing if I can write less-hungry ones that produce similar results as the top end ones. Hopefully before too long we'll get to the point where it 'just works' for most of you and we can start on the really exciting stuff like alternative camera views when editing, completion of the weapon systems and refinement of the character and AI behaviors.

P.S.

Also, just backing up for the 'evening' and realized I was showing a debug release metric shot. Compiled in release mode and got a slightly better FPS for the sky-test ;) 

I also quickly loaded my 'run to the river' level, and originally I was getting 24fps fully loaded, and after later performance work I got 29fps on Wednesday. Just tried with this 'evenings' version and with everything switched on I am now getting 50fps!  This was one of my pet goals, to get this small level playable, and it certainly is now.  I still want to hit 60 fps, and once I look more into AI, Physics and Shader optimizations I am sure I will not only get this but exceed it.

A good week for me. Until Monday, have a good weekend! More to come next week when we start our live-quad adventures, and if we're lucky, our half a million trees test!

Thursday Tinker

Meeting Day

With a six hour talking appointment starting at 8AM, there was not much time left in the day for coding, but I managed to look into a few small areas.  The issue of the FISH-EYE issue has now been fully explored and the problem was that the Reloaded engine was passing in a horizontal FOV, but using it in the perspective matrix calculation as a vertical FOV value, and the aspect was fixed at 1.777.  The new version sets the aspect ratio based on the resolution you are running under, and the FOV is now providing a vertical angle. By default now, the buildings and high walls don't skew out of all proportion when you look at the sky, and for those users who want the old style back, just ramp up the slider until you get what you want.

Now when you look up, the fish-eye effect is virtually gone and you get the kind of perspective you might expect from more traditional games that set a reasonable 85 degrees for horizontal FOV rather than the 170 degrees that I was using before.

Where Those Draw Calls Go

I also wanted to find out why an empty terrain scene took 73 (SEVENTY THREE) draw calls to perform, and so before Thursday was out I decided to make a shopping list of what GPU calls I was making. Here they are:

12 draw calls for the two overlapping skies (day and night, one side per call)
48 draw calls for the BlitzTerrain module segment renderings (one viewpoint)
3 terrain object calls (to populate the texture painter, editor water plane, etc)
1 water plane draw call which lies under the flat terrain
9 draw calls for the nine-stage post process for Bloom, Etc
1 vegmap draw call to paint the texture paint to the paint camera

As you can see, we have a few questions to ask of our engine here. We don't need 12 calls for the sky, and when not transitioning, one sky mesh would do. So that's 11 calls that can be saved. The editor water plane is not needed in game, so should be removed, as well as the texture painter when in-game! A simpler post process shader could manage Bloom in fewer that nine passes I am sure. All told, I could probably shave off 18 draw calls from the above 'empty' scene. I also want to investigate the 48 draw calls to produce the terrain, just to make sure they are not being rendered twice, and at least to confirm I need 48 segment meshes. Maybe I can find some settings to fiddle with in this regard too.

Signing Off

Quite tired after today's various antics, so I will be starting my research into True Imposter technology on Friday, which I hope the fill in the last of the visual gaps in my performance boosting trinity and get those distant distant QUAD polygons textured properly. I had thought of fake QUAD textures, but it would take me as long, and probably longer to create a system to produce the fixed baked textures to accompany each entity you bring in.  I have a plan, and it should not take too long, so it's worth spending a day on.

It was also aired that perhaps progress is not as fast as the community may wish, and perhaps too much time is being given over to pure performance work. I think however that in the long term this 'boring' performance work will repeat rewards much later on when we take high frame rates and clever scene handling for granted.

Wednesday Delight

A Sweet & Sour Day

No I am not having a Chinese meal, I am summarizing my adventure from testing the all new 'integrated' batch buffer and occlusion system. I swapped in the required code, made sure it did what it was supposed to do at the time it was supposed to do it in, and LO, I went from 24fps to 29fps. Sigh.

You can almost imagine the OOOs and AAAs in the meeting on Thursday when I display my 5fps increase for all the office to see!  That was the sour.

The Sweet

I decided to pick my confidence off the floor and start from the beginning, creating a flat landscape and then populating it with a few hundred high polygon trees.  My current Beta 1.002 update rendered the amazing vista at 42fps (with everything switched off) and an astounding 10 fps with everything switch on. I re-instated my new code which optimizes the terrain and uses the triple whammy of GPU occlusion, geometry batching and QUAD system and my new statistics on the exact same level was 100fps with everything on and 155fps with everything switched off.  Silence.  Basically, a 1000% increase in performance for that level :)

Lee's New Mission

I still have to sort out some textures for the quad polygons and maybe fiddle with the transition distances (ultimate for a new slider), but I had a new mission from my mixed results.  I was going to account for every draw call and every rogue performance metric in the system, to find out WHY my run-to-the-river level is not showing the same increase in performance.  As you can tell, I am not quite finished with the 'make the engine seriously fast' work, and hopefully you can bear with me as we go from small victory to small victory in the pursuit of a decent games engine.

Signing Off

I am conscious of the fact I have a 7AM wake-up call tomorrow so I cannot indulge in a large blog post :)  All in all, pretty happy with my performance boost on some levels, and I know pretty much what I am doing over the next few days to get some more!!