Blurry Work
First up, thanks for all the calls to action and recommendations of rest, a very sensible idea and anyone listening would be a fool to ignore such sage advice.
Unfortunately software seldom writes itself and deadlines have a habit of getting closer and larger, and it would not do to just leave the update half-finished which is why I must carry on and finish it. Perhaps I can find a few days to rest after the update is out the door!
Perspective Pains
The more I work on this QUAD texture rendering, the more I am convinced my brain was not wired for 3D thinking. I've literally spent 10 hours on the same thirty lines of code, trying every combination of crazy ass ideas I can think of to get a section of render to re-render centrally at the correct angle and distance. You would think it was easy :)
The good news is that I am making progress, all be it slowly. I figured out some neat things like how you can scale up the perspective matrix to create a sort of zoom effect. I am currently working on a way to tweak the view matrix generation to tilt towards the target object that needs rendering as simply shifting the view camera was not enough. I also need to handle the multitude of little attributes for things like resolution changes, aspect changes, camera position and angle changes, object dimension changes and then quad resolution and densities which can be adjusted in real-time. It's clear to me that this is not a 'few days work' as I had originally thought and will take some considerable time and thought to become fully rounded and complete.
Signing Off
A nice cup of tea me thinks, and then see if I can get 'something' concluded from all this work, and build an internal version for Ricky baby. The next update is imminent so internal testing is the plan, I only wished I had more done to test ;)
Wednesday, 27 November 2013
Tuesday, 26 November 2013
Tuesday At Quadington
Short Day
It's very rare I allow physical discomfort to stop me coding, but for whatever reason Tuesday has been a real mare for me. Might be overwork, or just the weather, but my energy levels are barely registering. I did however have the professional sense to make today count so I booted up and carried on regardless for a few hours.
Quad Progress
More progress made here, with each quad now having the correct UV coordinates assigned, the viewport system rendering a single object to each gap in the community texture and I have started the code to correctly align the view camera with the target object.
I am currently struggling with ensuring the view camera alignment remains unaffected, and to work out the optimum distance to place the camera from the object to encompass the whole object and match it perfectly with the quad size too. Doing this correctly means the quad system will work from any camera, and any angle, and any quad size.
I remember my early days at school and I don't think we did advanced 3D maths at GCSE level, but I wish I did. For what might take a modern programmer five minutes to work through the relationships between all my vectors and matrices, will take my brain a considerable amount of time, usually through trial, error and stubbornness.
Other Bits
Also managed to catch up with my emails, do this blog and start one or two plates spinning so no-one is waiting on my feedback.
Signing Off
Hopefully Wednesday I will feel better. For now, I am going to put the kettle on, watch an episode of Time Team and then get an early night. I did not sleep a wink (that I remember) in the last 36 hours so hopefully I can drift off. The quad system is the last visual hold-up to the next update, as after this it will be clean-up and testing which requires less 3D math and more donkey.
It's very rare I allow physical discomfort to stop me coding, but for whatever reason Tuesday has been a real mare for me. Might be overwork, or just the weather, but my energy levels are barely registering. I did however have the professional sense to make today count so I booted up and carried on regardless for a few hours.
Quad Progress
More progress made here, with each quad now having the correct UV coordinates assigned, the viewport system rendering a single object to each gap in the community texture and I have started the code to correctly align the view camera with the target object.
I am currently struggling with ensuring the view camera alignment remains unaffected, and to work out the optimum distance to place the camera from the object to encompass the whole object and match it perfectly with the quad size too. Doing this correctly means the quad system will work from any camera, and any angle, and any quad size.
I remember my early days at school and I don't think we did advanced 3D maths at GCSE level, but I wish I did. For what might take a modern programmer five minutes to work through the relationships between all my vectors and matrices, will take my brain a considerable amount of time, usually through trial, error and stubbornness.
Other Bits
Also managed to catch up with my emails, do this blog and start one or two plates spinning so no-one is waiting on my feedback.
Signing Off
Hopefully Wednesday I will feel better. For now, I am going to put the kettle on, watch an episode of Time Team and then get an early night. I did not sleep a wink (that I remember) in the last 36 hours so hopefully I can drift off. The quad system is the last visual hold-up to the next update, as after this it will be clean-up and testing which requires less 3D math and more donkey.
Monday, 25 November 2013
Monday Quadz
Houston, We Have A Quad Texture!
After much tinkering inside the bowels of the engine, we now have quads using a render target texture created entirely on the video card and rendered with the object that created it. Here is the glorious mess for your enjoyment:
As you can see, we are rendering the whole texture for each quad, and rendering all the objects to the target surface as they happen.
The next step, now we can see something along the right lines, is to sub-divide the texture into grid squares, to use a view-port to only render to the correct square, to only render the object once per generation request, to modify the quad UV to only show the texture that applies to it's own original object, and so on. I had thought I would be further along, but it's always a good place to be when you render something and you can actually 'see it'.
Signing Off
It seems my body is once again on night shift, so I write this at the end of my day which is about 5AM. Hopefully I can get up before the sun goes down as I so light to see some daylight before I start my day.
After much tinkering inside the bowels of the engine, we now have quads using a render target texture created entirely on the video card and rendered with the object that created it. Here is the glorious mess for your enjoyment:
As you can see, we are rendering the whole texture for each quad, and rendering all the objects to the target surface as they happen.
The next step, now we can see something along the right lines, is to sub-divide the texture into grid squares, to use a view-port to only render to the correct square, to only render the object once per generation request, to modify the quad UV to only show the texture that applies to it's own original object, and so on. I had thought I would be further along, but it's always a good place to be when you render something and you can actually 'see it'.
Signing Off
It seems my body is once again on night shift, so I write this at the end of my day which is about 5AM. Hopefully I can get up before the sun goes down as I so light to see some daylight before I start my day.
Sunday, 24 November 2013
Sunday Pottering
Legacy Bits and Bobs
After spending MOST of yesterday playing FEAR 2 and the start of today watching an episode of The Planets, I decided to boot up the ol' computer and do some non-performance coding to give my brain a break.
What you are goggling at are three objects from the Game Creator Store, which is being integrated for the next update. To this end, I spent half of Sunday making sure some legacy models and entities would load in first time without modification. I am certainly not going to make Reloaded backwards compatible with everything in the store, and will be adding a filter by default to show only Reloaded content, but this filter will be adjustable so you can find older legacy static entities and see how they fair (including any that you have bought previously via FPSC Classic).
Occlusion Works
Also made some continuing fixes with the new renderer to stop it flickering due to multiple queries and cameras confusing it. I am saving the true imposter work until Monday as that's a full work day of headache that one!
Signing Off
My plan for this coming week is to have a solid version by Wednesday, on which plenty of testing can be performed THU and FRI. The old regime of releasing new updates that break old projects should really be put to rest here at TGC, and we do that by spending two days NOT adding features and JUST testing (that is, if I don't get strong armed into adding anything during those two days). The objective is stability, consistency, THEN performance, THEN the store integration. You can then be at least assured that the update is not break stuff you have working right now.
After spending MOST of yesterday playing FEAR 2 and the start of today watching an episode of The Planets, I decided to boot up the ol' computer and do some non-performance coding to give my brain a break.
What you are goggling at are three objects from the Game Creator Store, which is being integrated for the next update. To this end, I spent half of Sunday making sure some legacy models and entities would load in first time without modification. I am certainly not going to make Reloaded backwards compatible with everything in the store, and will be adding a filter by default to show only Reloaded content, but this filter will be adjustable so you can find older legacy static entities and see how they fair (including any that you have bought previously via FPSC Classic).
Occlusion Works
Also made some continuing fixes with the new renderer to stop it flickering due to multiple queries and cameras confusing it. I am saving the true imposter work until Monday as that's a full work day of headache that one!
Signing Off
My plan for this coming week is to have a solid version by Wednesday, on which plenty of testing can be performed THU and FRI. The old regime of releasing new updates that break old projects should really be put to rest here at TGC, and we do that by spending two days NOT adding features and JUST testing (that is, if I don't get strong armed into adding anything during those two days). The objective is stability, consistency, THEN performance, THEN the store integration. You can then be at least assured that the update is not break stuff you have working right now.
Friday, 22 November 2013
Friday Foundations
Lee Stops Digging
After many days of performance tuning and investigating, I have finally got to bedrock on everything that spends GPU calls.
As you can see, when looking up at the sky there are only SEVEN draw calls now being made. Four of those are sky box surfaces (one for each 'side' of the sky being rendered), one for the post process quad to render the main camera to the screen, one for the dummy terrain object which is needed to regulate the terrain shader system and finally one for the world sized water plane.
I also discovered the cause of the 'super high drain' from the post processing which was caused by the lightray camera rendering everything, even if you switched it off in a previous session, now fixed.
After hacking my last few singular calls out, and noticed no performance gain beyond the high 300's mark, I decided to leave them be. I am now curious what FPS scores users will get from a blank scene looking at the sky with everything switched off. I agree it's not the makings of the next killer game, but it will be very interesting to see how low-end systems handle this little experiment, and if they DO perform badly, then we can start to look elsewhere for the culprit such as the CPU, swap files or even the panel that displays the score in the first place :)
Further Ideas
There are one or two further ideas beyond this minimalist position, such as using the occlusion to detect if the water plane pixel(s) was rendered, and if not, hide until the occluder says it is visible once more. The reason I am not pursuing at this time is that with the new front to back draw order, all of the underground water plane is Z-rejected instantly so does not hit performance except for the extremely minor call hit.
Moving On
Now I know every draw call personally, and can account for everything the GPU is doing, I can start to build things back up. One of the first will be the True Imposter System which will create community textures for each Instance Stamp buffer and then GPU render objects into them to provide my quad textures. It's a priority as right now I am using a place holder 'building' texture for the quads and allowing that to be in the next update would make all your screenshots from that version very odd indeed.
I also have it penciled in to add some more legacy support too, but more on that when I discover I have the time to do what I have planned. It should be a welcome addition if it all works out :)
Signing Off
Well I did say it was going to be all Performance, Performance, Performance and I think you will agree I have been rather single minded on this point. There is more performance to be had elsewhere such as faster physics for polygon style objects, faster AI by diving into the source code and seeing how I can save speed and of course going through each shader and seeing if I can write less-hungry ones that produce similar results as the top end ones. Hopefully before too long we'll get to the point where it 'just works' for most of you and we can start on the really exciting stuff like alternative camera views when editing, completion of the weapon systems and refinement of the character and AI behaviors.
P.S.
Also, just backing up for the 'evening' and realized I was showing a debug release metric shot. Compiled in release mode and got a slightly better FPS for the sky-test ;)
I also quickly loaded my 'run to the river' level, and originally I was getting 24fps fully loaded, and after later performance work I got 29fps on Wednesday. Just tried with this 'evenings' version and with everything switched on I am now getting 50fps! This was one of my pet goals, to get this small level playable, and it certainly is now. I still want to hit 60 fps, and once I look more into AI, Physics and Shader optimizations I am sure I will not only get this but exceed it.
A good week for me. Until Monday, have a good weekend! More to come next week when we start our live-quad adventures, and if we're lucky, our half a million trees test!
After many days of performance tuning and investigating, I have finally got to bedrock on everything that spends GPU calls.
As you can see, when looking up at the sky there are only SEVEN draw calls now being made. Four of those are sky box surfaces (one for each 'side' of the sky being rendered), one for the post process quad to render the main camera to the screen, one for the dummy terrain object which is needed to regulate the terrain shader system and finally one for the world sized water plane.
I also discovered the cause of the 'super high drain' from the post processing which was caused by the lightray camera rendering everything, even if you switched it off in a previous session, now fixed.
After hacking my last few singular calls out, and noticed no performance gain beyond the high 300's mark, I decided to leave them be. I am now curious what FPS scores users will get from a blank scene looking at the sky with everything switched off. I agree it's not the makings of the next killer game, but it will be very interesting to see how low-end systems handle this little experiment, and if they DO perform badly, then we can start to look elsewhere for the culprit such as the CPU, swap files or even the panel that displays the score in the first place :)
Further Ideas
There are one or two further ideas beyond this minimalist position, such as using the occlusion to detect if the water plane pixel(s) was rendered, and if not, hide until the occluder says it is visible once more. The reason I am not pursuing at this time is that with the new front to back draw order, all of the underground water plane is Z-rejected instantly so does not hit performance except for the extremely minor call hit.
Moving On
Now I know every draw call personally, and can account for everything the GPU is doing, I can start to build things back up. One of the first will be the True Imposter System which will create community textures for each Instance Stamp buffer and then GPU render objects into them to provide my quad textures. It's a priority as right now I am using a place holder 'building' texture for the quads and allowing that to be in the next update would make all your screenshots from that version very odd indeed.
I also have it penciled in to add some more legacy support too, but more on that when I discover I have the time to do what I have planned. It should be a welcome addition if it all works out :)
Signing Off
Well I did say it was going to be all Performance, Performance, Performance and I think you will agree I have been rather single minded on this point. There is more performance to be had elsewhere such as faster physics for polygon style objects, faster AI by diving into the source code and seeing how I can save speed and of course going through each shader and seeing if I can write less-hungry ones that produce similar results as the top end ones. Hopefully before too long we'll get to the point where it 'just works' for most of you and we can start on the really exciting stuff like alternative camera views when editing, completion of the weapon systems and refinement of the character and AI behaviors.
P.S.
Also, just backing up for the 'evening' and realized I was showing a debug release metric shot. Compiled in release mode and got a slightly better FPS for the sky-test ;)
I also quickly loaded my 'run to the river' level, and originally I was getting 24fps fully loaded, and after later performance work I got 29fps on Wednesday. Just tried with this 'evenings' version and with everything switched on I am now getting 50fps! This was one of my pet goals, to get this small level playable, and it certainly is now. I still want to hit 60 fps, and once I look more into AI, Physics and Shader optimizations I am sure I will not only get this but exceed it.
A good week for me. Until Monday, have a good weekend! More to come next week when we start our live-quad adventures, and if we're lucky, our half a million trees test!
Thursday, 21 November 2013
Thursday Tinker
Meeting Day
With a six hour talking appointment starting at 8AM, there was not much time left in the day for coding, but I managed to look into a few small areas. The issue of the FISH-EYE issue has now been fully explored and the problem was that the Reloaded engine was passing in a horizontal FOV, but using it in the perspective matrix calculation as a vertical FOV value, and the aspect was fixed at 1.777. The new version sets the aspect ratio based on the resolution you are running under, and the FOV is now providing a vertical angle. By default now, the buildings and high walls don't skew out of all proportion when you look at the sky, and for those users who want the old style back, just ramp up the slider until you get what you want.
Now when you look up, the fish-eye effect is virtually gone and you get the kind of perspective you might expect from more traditional games that set a reasonable 85 degrees for horizontal FOV rather than the 170 degrees that I was using before.
Where Those Draw Calls Go
I also wanted to find out why an empty terrain scene took 73 (SEVENTY THREE) draw calls to perform, and so before Thursday was out I decided to make a shopping list of what GPU calls I was making. Here they are:
12 draw calls for the two overlapping skies (day and night, one side per call)
48 draw calls for the BlitzTerrain module segment renderings (one viewpoint)
3 terrain object calls (to populate the texture painter, editor water plane, etc)
1 water plane draw call which lies under the flat terrain
9 draw calls for the nine-stage post process for Bloom, Etc
1 vegmap draw call to paint the texture paint to the paint camera
As you can see, we have a few questions to ask of our engine here. We don't need 12 calls for the sky, and when not transitioning, one sky mesh would do. So that's 11 calls that can be saved. The editor water plane is not needed in game, so should be removed, as well as the texture painter when in-game! A simpler post process shader could manage Bloom in fewer that nine passes I am sure. All told, I could probably shave off 18 draw calls from the above 'empty' scene. I also want to investigate the 48 draw calls to produce the terrain, just to make sure they are not being rendered twice, and at least to confirm I need 48 segment meshes. Maybe I can find some settings to fiddle with in this regard too.
Signing Off
Quite tired after today's various antics, so I will be starting my research into True Imposter technology on Friday, which I hope the fill in the last of the visual gaps in my performance boosting trinity and get those distant distant QUAD polygons textured properly. I had thought of fake QUAD textures, but it would take me as long, and probably longer to create a system to produce the fixed baked textures to accompany each entity you bring in. I have a plan, and it should not take too long, so it's worth spending a day on.
It was also aired that perhaps progress is not as fast as the community may wish, and perhaps too much time is being given over to pure performance work. I think however that in the long term this 'boring' performance work will repeat rewards much later on when we take high frame rates and clever scene handling for granted.
With a six hour talking appointment starting at 8AM, there was not much time left in the day for coding, but I managed to look into a few small areas. The issue of the FISH-EYE issue has now been fully explored and the problem was that the Reloaded engine was passing in a horizontal FOV, but using it in the perspective matrix calculation as a vertical FOV value, and the aspect was fixed at 1.777. The new version sets the aspect ratio based on the resolution you are running under, and the FOV is now providing a vertical angle. By default now, the buildings and high walls don't skew out of all proportion when you look at the sky, and for those users who want the old style back, just ramp up the slider until you get what you want.
Now when you look up, the fish-eye effect is virtually gone and you get the kind of perspective you might expect from more traditional games that set a reasonable 85 degrees for horizontal FOV rather than the 170 degrees that I was using before.
Where Those Draw Calls Go
I also wanted to find out why an empty terrain scene took 73 (SEVENTY THREE) draw calls to perform, and so before Thursday was out I decided to make a shopping list of what GPU calls I was making. Here they are:
12 draw calls for the two overlapping skies (day and night, one side per call)
48 draw calls for the BlitzTerrain module segment renderings (one viewpoint)
3 terrain object calls (to populate the texture painter, editor water plane, etc)
1 water plane draw call which lies under the flat terrain
9 draw calls for the nine-stage post process for Bloom, Etc
1 vegmap draw call to paint the texture paint to the paint camera
As you can see, we have a few questions to ask of our engine here. We don't need 12 calls for the sky, and when not transitioning, one sky mesh would do. So that's 11 calls that can be saved. The editor water plane is not needed in game, so should be removed, as well as the texture painter when in-game! A simpler post process shader could manage Bloom in fewer that nine passes I am sure. All told, I could probably shave off 18 draw calls from the above 'empty' scene. I also want to investigate the 48 draw calls to produce the terrain, just to make sure they are not being rendered twice, and at least to confirm I need 48 segment meshes. Maybe I can find some settings to fiddle with in this regard too.
Signing Off
Quite tired after today's various antics, so I will be starting my research into True Imposter technology on Friday, which I hope the fill in the last of the visual gaps in my performance boosting trinity and get those distant distant QUAD polygons textured properly. I had thought of fake QUAD textures, but it would take me as long, and probably longer to create a system to produce the fixed baked textures to accompany each entity you bring in. I have a plan, and it should not take too long, so it's worth spending a day on.
It was also aired that perhaps progress is not as fast as the community may wish, and perhaps too much time is being given over to pure performance work. I think however that in the long term this 'boring' performance work will repeat rewards much later on when we take high frame rates and clever scene handling for granted.
Wednesday, 20 November 2013
Wednesday Delight
A Sweet & Sour Day
No I am not having a Chinese meal, I am summarizing my adventure from testing the all new 'integrated' batch buffer and occlusion system. I swapped in the required code, made sure it did what it was supposed to do at the time it was supposed to do it in, and LO, I went from 24fps to 29fps. Sigh.
You can almost imagine the OOOs and AAAs in the meeting on Thursday when I display my 5fps increase for all the office to see! That was the sour.
The Sweet
I decided to pick my confidence off the floor and start from the beginning, creating a flat landscape and then populating it with a few hundred high polygon trees. My current Beta 1.002 update rendered the amazing vista at 42fps (with everything switched off) and an astounding 10 fps with everything switch on. I re-instated my new code which optimizes the terrain and uses the triple whammy of GPU occlusion, geometry batching and QUAD system and my new statistics on the exact same level was 100fps with everything on and 155fps with everything switched off. Silence. Basically, a 1000% increase in performance for that level :)
Lee's New Mission
I still have to sort out some textures for the quad polygons and maybe fiddle with the transition distances (ultimate for a new slider), but I had a new mission from my mixed results. I was going to account for every draw call and every rogue performance metric in the system, to find out WHY my run-to-the-river level is not showing the same increase in performance. As you can tell, I am not quite finished with the 'make the engine seriously fast' work, and hopefully you can bear with me as we go from small victory to small victory in the pursuit of a decent games engine.
Signing Off
I am conscious of the fact I have a 7AM wake-up call tomorrow so I cannot indulge in a large blog post :) All in all, pretty happy with my performance boost on some levels, and I know pretty much what I am doing over the next few days to get some more!!
No I am not having a Chinese meal, I am summarizing my adventure from testing the all new 'integrated' batch buffer and occlusion system. I swapped in the required code, made sure it did what it was supposed to do at the time it was supposed to do it in, and LO, I went from 24fps to 29fps. Sigh.
You can almost imagine the OOOs and AAAs in the meeting on Thursday when I display my 5fps increase for all the office to see! That was the sour.
The Sweet
I decided to pick my confidence off the floor and start from the beginning, creating a flat landscape and then populating it with a few hundred high polygon trees. My current Beta 1.002 update rendered the amazing vista at 42fps (with everything switched off) and an astounding 10 fps with everything switch on. I re-instated my new code which optimizes the terrain and uses the triple whammy of GPU occlusion, geometry batching and QUAD system and my new statistics on the exact same level was 100fps with everything on and 155fps with everything switched off. Silence. Basically, a 1000% increase in performance for that level :)
Lee's New Mission
I still have to sort out some textures for the quad polygons and maybe fiddle with the transition distances (ultimate for a new slider), but I had a new mission from my mixed results. I was going to account for every draw call and every rogue performance metric in the system, to find out WHY my run-to-the-river level is not showing the same increase in performance. As you can tell, I am not quite finished with the 'make the engine seriously fast' work, and hopefully you can bear with me as we go from small victory to small victory in the pursuit of a decent games engine.
Signing Off
I am conscious of the fact I have a 7AM wake-up call tomorrow so I cannot indulge in a large blog post :) All in all, pretty happy with my performance boost on some levels, and I know pretty much what I am doing over the next few days to get some more!!
Subscribe to:
Posts (Atom)