Prototype Fun
The prior evening was something of a success with no real roadblocks. I now have some new commands INSTANCE STAMP, INIT INSTANCE STAMP and FREE INSTANCE STAMP which usurps an innocent object and causes it to activate a new feature of a DBP object to render a series of meshes based on reference data passed in from an instance object through the INSTANCE STAMP command.
Right now it still uses the internal DrawMesh command which essentially draws the parents mesh in a different location based on the reference data, but my task this evening will be to delete this in favor of a brand new piece of code which writes the DirectX buffer creation and staging code up from scratch to only service the features I want.
Features I Want
The first feature is to draw some polygons on the screen at the location representative of the instance object reference data passed in. Not too bothered about shaders, textures, render states, performance or memory usage right now, I just want to 'replace' the existing draw system with a new one which renders a polygon in the right location.
Once we have that, it will be possible to then break this up to use multiple buffers (one per shader/texture) and accumulate the polygons within an area to be rendered. From this we can then do a realistic memory and performance test and figure out whether it's worth doing all the rest, or just revert to some instance object style approach.
In Other News
I appreciate the graphics and videos have been thin on the grounds since I have been banned from releasing crappy videos, so let me enlighten you with a new video produced from the recently built quality-control building located in Rickville. While I have been coding maddening memory stuff, Mark has been creating cool character stuff and here is something he prepared earlier:
http://www.youtube.com/watch?v=QUJXW8F6j0c&feature=youtu.be
In Other Other News
As a little trumpet blowing moment, I wanted to add that the AGK apps I wrote at the start of the year and quite brutally distracted me from Reloaded are doing very well in the charts for both iOS and Android. There are about six versions floating about, but here are the Hazard ones:
https://itunes.apple.com/gb/app/hazard-perception-uk-driving/id609649044?mt=8
https://play.google.com/store/apps/details?id=uk.co.focusmm.dts_hazard
I am really happy with how they came out, and it was all thanks to AGK (http://www.appgamekit.com) which allowed me to create the apps very quickly from scratch and then get it onto Windows, iOS and Android in a few clicks. I have now handed the project over so that iMac and Blackberry versions can be produced which will mean my range of Driving Test apps cover all the main distribution points with the possible exception of Windows 8. The aim is to get them into the top ten for both iOS and Android Educational charts, and with the quality of the apps compared to ALL the competition put together, they should manage that very easily ;)
Signing Off
Back to Reloaded, hopefully the new render system will not take 2 months, but I have a long way to go to even get back to the visual level we already had with classic, but I think it will be worth the wait if the theory lines up roughly with the practice. In my latest proto, I pasted 800,000 object references and my memory usage did not even get past 300MB. True it was all the same mesh, but you get the idea ;)
Tuesday, 30 April 2013
Monday, 29 April 2013
Monday Proto Day
What Happened After Friday
After the revelation last week about the memory situation with FPSC (old and new), I got to thinking about how the solution might be coded. To that end I started a small prototype which directly compared creating segment objects for a 200x200x20 using different techniques.
Using a really intense segment object, the CLONE OBJECT method got to around 2500 objects before running out of memory (11x11x20). The current INSTANCE OBJECT did much better and I stopped it at 500MB which is a nice round number which would yield 200,000 objects and a rather nice 300x300x20. This of course excludes any light mapping or CSG operations, but gives you an idea of the power of instances if we took that approach.
My theoretical third option called 'Workspace Object' uses the idea that you don't need a whole 'instance object container' to reference the placement of a parent object, but just a few simple pieces of data such as 'parent object index, position, rotation, scale, limb visibility' and then extras such as 'lightmap UVs, CSG operations' and so on. Placing this data on a map grid, the workspace object would generate the mesh data directly in the DirectX buffers when then hone into view. My work over the next few days is to finish the prototype along these lines and see what kind of memory activity is produced.
Predicted Results
My guess is that this new system will occupy virtually no additional memory beyond the initial reference data of 200x200x20x8x4=25MB which stores the 'where', the memory for the parent resources such as original segments, entities and other objects, the textures for all these parents, additional UV data from the light-mapping process, any CSG meshes (to be decided) and other incidentals. We also need memory for the vertex/index buffers of course which will represent a size representative of how large we make the visible rendering area. For the technique to work we also need LOD models for distant objects so the polygon count can be controlled when looking at scenes from a distance as well. The good news is that apart from the rendering buffer sizes which will ebb and flow with the content close to the camera, there will be no additional memory cost for painting segments and objects in the scene, which is the holy grail we are after. It's a lot of development, but the basic functionality should be up and running quite quickly.
Community Response
I asked the community for their thoughts on this subject, and the community response was fast and decisive. Within days we had a thread running and users voting their opinions, and the result was as far as I could make out, 100% YES. I even threw in a scary 2 month delay on the release, and that did not seem to scare them from their decision.
I still have to clear it with the internal team as there are marketing and commercial realities to consider, but I think this is one of those features we have to do properly as it's the bedrock of the whole product. I have a meeting on the 8th May to make a concrete decision on this, but in the meantime I am creating prototypes and technology to work out the total feasibility of implementing this new system.
Signing Off
It has been one of those late to rise days, so I will be returning this evening after a spot of food to finish my first 'workspace object' prototype when I hope to tame a DirectX buffer into generating my segment mesh directly from its parent, and then see how many I can cram in before it starts to wobble. Another bonus of this technique is that you get 'automatic batching' of polygons, pre-sorted into texture and shader lists. This means you don't get a slow down as you start to pour in more polygons as graphics cards will take the same time to draw whether it's 15 polygons or 150 polygons. I also need to insert a mechanism for object culling and occlusion as well, but I will be able to look closer at this once I have the buffers up and running.
After the revelation last week about the memory situation with FPSC (old and new), I got to thinking about how the solution might be coded. To that end I started a small prototype which directly compared creating segment objects for a 200x200x20 using different techniques.
Using a really intense segment object, the CLONE OBJECT method got to around 2500 objects before running out of memory (11x11x20). The current INSTANCE OBJECT did much better and I stopped it at 500MB which is a nice round number which would yield 200,000 objects and a rather nice 300x300x20. This of course excludes any light mapping or CSG operations, but gives you an idea of the power of instances if we took that approach.
My theoretical third option called 'Workspace Object' uses the idea that you don't need a whole 'instance object container' to reference the placement of a parent object, but just a few simple pieces of data such as 'parent object index, position, rotation, scale, limb visibility' and then extras such as 'lightmap UVs, CSG operations' and so on. Placing this data on a map grid, the workspace object would generate the mesh data directly in the DirectX buffers when then hone into view. My work over the next few days is to finish the prototype along these lines and see what kind of memory activity is produced.
Predicted Results
My guess is that this new system will occupy virtually no additional memory beyond the initial reference data of 200x200x20x8x4=25MB which stores the 'where', the memory for the parent resources such as original segments, entities and other objects, the textures for all these parents, additional UV data from the light-mapping process, any CSG meshes (to be decided) and other incidentals. We also need memory for the vertex/index buffers of course which will represent a size representative of how large we make the visible rendering area. For the technique to work we also need LOD models for distant objects so the polygon count can be controlled when looking at scenes from a distance as well. The good news is that apart from the rendering buffer sizes which will ebb and flow with the content close to the camera, there will be no additional memory cost for painting segments and objects in the scene, which is the holy grail we are after. It's a lot of development, but the basic functionality should be up and running quite quickly.
Community Response
I asked the community for their thoughts on this subject, and the community response was fast and decisive. Within days we had a thread running and users voting their opinions, and the result was as far as I could make out, 100% YES. I even threw in a scary 2 month delay on the release, and that did not seem to scare them from their decision.
I still have to clear it with the internal team as there are marketing and commercial realities to consider, but I think this is one of those features we have to do properly as it's the bedrock of the whole product. I have a meeting on the 8th May to make a concrete decision on this, but in the meantime I am creating prototypes and technology to work out the total feasibility of implementing this new system.
Signing Off
It has been one of those late to rise days, so I will be returning this evening after a spot of food to finish my first 'workspace object' prototype when I hope to tame a DirectX buffer into generating my segment mesh directly from its parent, and then see how many I can cram in before it starts to wobble. Another bonus of this technique is that you get 'automatic batching' of polygons, pre-sorted into texture and shader lists. This means you don't get a slow down as you start to pour in more polygons as graphics cards will take the same time to draw whether it's 15 polygons or 150 polygons. I also need to insert a mechanism for object culling and occlusion as well, but I will be able to look closer at this once I have the buffers up and running.
Friday, 26 April 2013
Friday Thoughts
Large Levels
I have done some DBP Object tests and the way memory is created and deleted, and it is clear to me that whatever scheme I choose the slow fragmentation of memory in the heap is a critical limitation of the system I am currently using to build segment levels.
Even if I use instancing and shared parent geometry, the size of the memory required for the Vertex and Index buffers, plus the reduction in memory over time means there will be a problem sooner or later when it comes to the capabilities of creating 'very large levels'.
I am pretty sure I can get larger levels with the system as it is, but a small voice is telling me that squeezing a little more from the current system might not be the best plan. I don't have a solution, but I have been sketching out some ideas of possible implementations that allows 'extremely large levels' to be constructed.
Even as I contemplate the possibilities, I am acutely aware that such a detour will affect the project timetable. Even though the existing system will allow you to create 200x200x20 terrain levels with lots of buildings and objects, at some point it will 'run out'. And when it does that, it will most likely crash out. I have run four or five different ways to monitor and track the memory usage of the current system (last night), but in each case I had to reject the solution due to either in-accuracy or memory monitored out of context.
I also gained a great insight into all the different elements within the engine that reserve memory and free it, and when they do this. It was clear that operating within this finite world of just 1.8GB to store everything was quite restricting.
The New Approach
If I decide to take a detour, the new system will be based on a system that has a single aim of serving data to the DirectX buffers so you can see what you are looking at. Everything else will be about generating that data only when needed, and avoid the creation of new pockets of memory during the editing and game play processes. The ideal mechanism would occupy memory only when the camera needed to render a view, and use LOD (level of detail) to reduce polygon quantities down for wide angle or high views.
I don't have the whole picture in my mind right now, but the upshot of such a system would mean you would no longer have the traditional problem of 'stopping editing when the memory ran out'. Instead, the memory is used as a workspace in which to bring things in when you are using them, and remove them when you no longer require their services. This means the limitation is no longer the memory size, but the local storage system which I think we can all agree is plentiful.
I am curious what the voting might be on this possible new approach. Keep with the current plan and be ready for October 2013 or implement this more far reaching technology and maybe add two months onto the project.
Signing Off
As you can see, big thoughts on Friday and some coding to back-up my concern that memory management in it's current form is a little too wasteful. There are also a few solutions that sit between these two approaches, and I need to finish thinking about the many assets required to populate a typical level before I know how big the change might be. I do know that when users buy and start using Reloaded, they should be able to paint segments for as long as they want and as far as they want. It's not everything, but I think it is one of the important ones to get right and I don't mind sacrificing a deadline if it means turning a 'good' result into a 'great' result.
I have done some DBP Object tests and the way memory is created and deleted, and it is clear to me that whatever scheme I choose the slow fragmentation of memory in the heap is a critical limitation of the system I am currently using to build segment levels.
Even if I use instancing and shared parent geometry, the size of the memory required for the Vertex and Index buffers, plus the reduction in memory over time means there will be a problem sooner or later when it comes to the capabilities of creating 'very large levels'.
I am pretty sure I can get larger levels with the system as it is, but a small voice is telling me that squeezing a little more from the current system might not be the best plan. I don't have a solution, but I have been sketching out some ideas of possible implementations that allows 'extremely large levels' to be constructed.
Even as I contemplate the possibilities, I am acutely aware that such a detour will affect the project timetable. Even though the existing system will allow you to create 200x200x20 terrain levels with lots of buildings and objects, at some point it will 'run out'. And when it does that, it will most likely crash out. I have run four or five different ways to monitor and track the memory usage of the current system (last night), but in each case I had to reject the solution due to either in-accuracy or memory monitored out of context.
I also gained a great insight into all the different elements within the engine that reserve memory and free it, and when they do this. It was clear that operating within this finite world of just 1.8GB to store everything was quite restricting.
The New Approach
If I decide to take a detour, the new system will be based on a system that has a single aim of serving data to the DirectX buffers so you can see what you are looking at. Everything else will be about generating that data only when needed, and avoid the creation of new pockets of memory during the editing and game play processes. The ideal mechanism would occupy memory only when the camera needed to render a view, and use LOD (level of detail) to reduce polygon quantities down for wide angle or high views.
I don't have the whole picture in my mind right now, but the upshot of such a system would mean you would no longer have the traditional problem of 'stopping editing when the memory ran out'. Instead, the memory is used as a workspace in which to bring things in when you are using them, and remove them when you no longer require their services. This means the limitation is no longer the memory size, but the local storage system which I think we can all agree is plentiful.
I am curious what the voting might be on this possible new approach. Keep with the current plan and be ready for October 2013 or implement this more far reaching technology and maybe add two months onto the project.
Signing Off
As you can see, big thoughts on Friday and some coding to back-up my concern that memory management in it's current form is a little too wasteful. There are also a few solutions that sit between these two approaches, and I need to finish thinking about the many assets required to populate a typical level before I know how big the change might be. I do know that when users buy and start using Reloaded, they should be able to paint segments for as long as they want and as far as they want. It's not everything, but I think it is one of the important ones to get right and I don't mind sacrificing a deadline if it means turning a 'good' result into a 'great' result.
Thursday, 25 April 2013
Thursday Tiptoes
Disclaimer
Please don't take this blog TOO seriously guys and gals! Think of it as a simple development diary of an ordinary human documenting his journey. Stringing me up on the nearest yardarm might seem like a good idea right now, but if I am not allowing to make mistakes, I won't learn my best lessons! I learn more from a total catastrophe than an easy win. With a little luck, many of these lessons will find their way into the final product.
Progress
The first order of business was to measure memory usage. Before you can reduce the footprint and solve the huge consumption issue, we first need a good system for measuring process that does not involve toggling to task manager and keeping up with the heap size :)
I have implemented a new command called OBJECT STATISTICS INTEGER which will interrogate a new set of values I am generating internally per object, one of which is the actual memory created when the NEW command is used. It restores it when DELETE is used as well so we get an accurate measure over the scope of a specific command. I tried to using the various techniques to interrogate the heap directly but it's a mysterious beast that does not lend itself to instant interrogations. The best system is to intercept every instance of memory creation and count it, which is why I have overwridden NEW and DELETE for all Basic3D DLL operations.
It's not perfect as it excludes memory creation from things like creating VBO and texture objects, but now I am dealing with the problem internally I can add code quite easily now and track this. I also made sure that my additions where using the pCustomData structure within an sObject structure so old plugins will still work with the new DBP DLL should I release it at some point. From the pCustomData pointer, I can pretty much create as much additional property storage as I like without ruining the legacy data structure.
Signing Off
Once I have a reliable way to measure and account for memory usage, the next step will be to retrace my steps and visit those locations where obvious memory usage occurred, and ask the key question 'is this necessary?'. Also, apologies to readers who don't want all the technical jargon and horror stories, alas this is the real and gritty world of development :)
I would also like to tip my hat to the honest and gritty comments provided so far regarding the memory issue. It is EVEN MORE clear to me that this is a 'deal breaker' within the FPSC community, and you can consider my efforts redoubled to solve it for ya! Physics and shaders will have to wait until we have huge levels and an inexhaustible capacity to contain your content.
Please don't take this blog TOO seriously guys and gals! Think of it as a simple development diary of an ordinary human documenting his journey. Stringing me up on the nearest yardarm might seem like a good idea right now, but if I am not allowing to make mistakes, I won't learn my best lessons! I learn more from a total catastrophe than an easy win. With a little luck, many of these lessons will find their way into the final product.
Progress
The first order of business was to measure memory usage. Before you can reduce the footprint and solve the huge consumption issue, we first need a good system for measuring process that does not involve toggling to task manager and keeping up with the heap size :)
I have implemented a new command called OBJECT STATISTICS INTEGER which will interrogate a new set of values I am generating internally per object, one of which is the actual memory created when the NEW command is used. It restores it when DELETE is used as well so we get an accurate measure over the scope of a specific command. I tried to using the various techniques to interrogate the heap directly but it's a mysterious beast that does not lend itself to instant interrogations. The best system is to intercept every instance of memory creation and count it, which is why I have overwridden NEW and DELETE for all Basic3D DLL operations.
It's not perfect as it excludes memory creation from things like creating VBO and texture objects, but now I am dealing with the problem internally I can add code quite easily now and track this. I also made sure that my additions where using the pCustomData structure within an sObject structure so old plugins will still work with the new DBP DLL should I release it at some point. From the pCustomData pointer, I can pretty much create as much additional property storage as I like without ruining the legacy data structure.
Signing Off
Once I have a reliable way to measure and account for memory usage, the next step will be to retrace my steps and visit those locations where obvious memory usage occurred, and ask the key question 'is this necessary?'. Also, apologies to readers who don't want all the technical jargon and horror stories, alas this is the real and gritty world of development :)
I would also like to tip my hat to the honest and gritty comments provided so far regarding the memory issue. It is EVEN MORE clear to me that this is a 'deal breaker' within the FPSC community, and you can consider my efforts redoubled to solve it for ya! Physics and shaders will have to wait until we have huge levels and an inexhaustible capacity to contain your content.
Wednesday, 24 April 2013
Wednesday Memories
Crashes and Memory
Both words are at the forefront of my thoughts this day. I drop in lots of segments, it gobbles up all the memory, then crashes. Wonderful. My task today has been to first track exactly how much memory is being used on segments (and by extension entities) and then relay that to an on-screen resource bar so I can see the resources grow the larger the level gets.
First Person Game Creator
I did an educational version of FPSC called FPGC which used some extra features such as an entity resource bar so I already have code for this, I have added segments to it now so it's comprehensive.
Notice the green bar at the bottom of the screen. It is nice and out of the way, and will grow as you add things to your level. It will turn red when you are over 900MB of memory used. I plan to cap all levels at 1GB of content, to then allow 800MB for the real time light mapping process and other operations of the application. This should keep me within the virtual address cap and prevent catastrophic crashes. The above bar tracks both segment and entity usage so it's pretty good measure of what your level consists of.
Next Step
Now this is coded, my next step is to add some code to stop allowing the user to add more entities and segments if the maximum resource is reached. I think this is better than a random crash and allows the user to backtrack and remove some items to finish their level. I would have liked to provide an infinite amount of space for your creations, but for this product we'll have to live with placing content across multiple levels. There is a technique of loading in content as you get near it, but that's a whole new engine and another product! Maybe next year :)
Signing Off
The day started late so I will be coming back after dinner to do a little more on this and try to get it to the point where the software cannot be crashed, no matter what I add and what I do. No sense moving on before this is cracked 100% which is the sign of a professional coder (that is, not get excited about a new feature go off to code it, but to stay with what you've done and make sure it's bullet proof). I'm not saying that's me, but I try!
Both words are at the forefront of my thoughts this day. I drop in lots of segments, it gobbles up all the memory, then crashes. Wonderful. My task today has been to first track exactly how much memory is being used on segments (and by extension entities) and then relay that to an on-screen resource bar so I can see the resources grow the larger the level gets.
First Person Game Creator
I did an educational version of FPSC called FPGC which used some extra features such as an entity resource bar so I already have code for this, I have added segments to it now so it's comprehensive.
Notice the green bar at the bottom of the screen. It is nice and out of the way, and will grow as you add things to your level. It will turn red when you are over 900MB of memory used. I plan to cap all levels at 1GB of content, to then allow 800MB for the real time light mapping process and other operations of the application. This should keep me within the virtual address cap and prevent catastrophic crashes. The above bar tracks both segment and entity usage so it's pretty good measure of what your level consists of.
Next Step
Now this is coded, my next step is to add some code to stop allowing the user to add more entities and segments if the maximum resource is reached. I think this is better than a random crash and allows the user to backtrack and remove some items to finish their level. I would have liked to provide an infinite amount of space for your creations, but for this product we'll have to live with placing content across multiple levels. There is a technique of loading in content as you get near it, but that's a whole new engine and another product! Maybe next year :)
Signing Off
The day started late so I will be coming back after dinner to do a little more on this and try to get it to the point where the software cannot be crashed, no matter what I add and what I do. No sense moving on before this is cracked 100% which is the sign of a professional coder (that is, not get excited about a new feature go off to code it, but to stay with what you've done and make sure it's bullet proof). I'm not saying that's me, but I try!
Tuesday, 23 April 2013
Tuesday In The Shade
Shader Success
It took many hours of torture but I managed to wrangle the shader and real time light mapper so they cooperated. They now exist happily together and although my initial shader is not up to Reloaded standards, it's enough to demonstrate that it works.
The shader you can see creating a specular reflection off the wet floor and the light map is basically hiding around the scene. You can tell good light mapping when you cannot see it ;) Again I apologise for FPSC classic graphics used in this screen shot, please forgive me.
The Crash
I also discovered the cause of the crash, and it was nothing more or less than the total lack of virtual address space. Yes friendly, the 1.8GB memory limit strikes again! Arg!
Okay, so this time rather than put an error message in there saying "make a smaller level" I have decided to tackle it head on and see what I can do. I have started the process by investigating exactly where the memory is being spent and you might be quite surprised.
The first 100MB is just the basic map editor, graphics and reserved arrays. That's fine and only represents 5% of the total spend, not bad for a foundation. It's all downhill from there though. Each small segment takes as little as 4Kb but the largest 'WW2 Cellar Segment Full' takes up 2.4Mb. Amazing huh! I did some initial digging and a function called CloneDBO is the cause for most of this, and then the 'Sync' of course which creates any new VBO objects for DirectX. Every byte is used and nothing is really wasted so it is not so much about fixing a bug but figuring a better way to do this. I had the idea of cloning all the repeated segment and entity data, and only holding the unique second UV layer for the texture lightmapping but that would be a bit of engineering to sort out. It WOULD solve the issue though so it is still on my 'possible' radar. The issue remains unresolved as of right now though.
Good Work Though
I did not spend the day making the PC crash and counting how big the memory usage was. Another crash was caused by the light mapper taking too many segments to light. If you give it a few hundred segments, and each segment with many polygons can take up to 20MB to store the light calculations, it soon eats through the 1.8GB barrier.
I solved this by slicing up the lighting task into 100 object chunks. Not scientific and I plan to measure and store the size of each segment to make this a more accurate lighting limit. For now it solves the crash due to too much lighting going on, and at least in this scenario you won't crash at the light mapping stage. This solves the classic issue with the original FPSC engine. Phew!
I just have to work out how to crunch the size of individual cloned segments and entities now. There is always a way, I just have to wait for my brain to tell me what it is.
Signing Off
All in all a good day, some nice looking renders now with the shader (from certain angles). I wanted to delve into a more sophisticated shader, but there will be time for that. For now, I want to reduce the memory usage to reasonable levels and then jump into player controls and physics. Cosmetics are critically important, but they can wait until the real work is done ;)
It took many hours of torture but I managed to wrangle the shader and real time light mapper so they cooperated. They now exist happily together and although my initial shader is not up to Reloaded standards, it's enough to demonstrate that it works.
The shader you can see creating a specular reflection off the wet floor and the light map is basically hiding around the scene. You can tell good light mapping when you cannot see it ;) Again I apologise for FPSC classic graphics used in this screen shot, please forgive me.
The Crash
I also discovered the cause of the crash, and it was nothing more or less than the total lack of virtual address space. Yes friendly, the 1.8GB memory limit strikes again! Arg!
Okay, so this time rather than put an error message in there saying "make a smaller level" I have decided to tackle it head on and see what I can do. I have started the process by investigating exactly where the memory is being spent and you might be quite surprised.
The first 100MB is just the basic map editor, graphics and reserved arrays. That's fine and only represents 5% of the total spend, not bad for a foundation. It's all downhill from there though. Each small segment takes as little as 4Kb but the largest 'WW2 Cellar Segment Full' takes up 2.4Mb. Amazing huh! I did some initial digging and a function called CloneDBO is the cause for most of this, and then the 'Sync' of course which creates any new VBO objects for DirectX. Every byte is used and nothing is really wasted so it is not so much about fixing a bug but figuring a better way to do this. I had the idea of cloning all the repeated segment and entity data, and only holding the unique second UV layer for the texture lightmapping but that would be a bit of engineering to sort out. It WOULD solve the issue though so it is still on my 'possible' radar. The issue remains unresolved as of right now though.
Good Work Though
I did not spend the day making the PC crash and counting how big the memory usage was. Another crash was caused by the light mapper taking too many segments to light. If you give it a few hundred segments, and each segment with many polygons can take up to 20MB to store the light calculations, it soon eats through the 1.8GB barrier.
I solved this by slicing up the lighting task into 100 object chunks. Not scientific and I plan to measure and store the size of each segment to make this a more accurate lighting limit. For now it solves the crash due to too much lighting going on, and at least in this scenario you won't crash at the light mapping stage. This solves the classic issue with the original FPSC engine. Phew!
I just have to work out how to crunch the size of individual cloned segments and entities now. There is always a way, I just have to wait for my brain to tell me what it is.
Signing Off
All in all a good day, some nice looking renders now with the shader (from certain angles). I wanted to delve into a more sophisticated shader, but there will be time for that. For now, I want to reduce the memory usage to reasonable levels and then jump into player controls and physics. Cosmetics are critically important, but they can wait until the real work is done ;)
Monday, 22 April 2013
Monday Shooting High
A Day Of Lasers, Shaders and Light Maps
My day continues from where it left off on Sunday, trying to get a shader and a light map to co-exist in my real-time light mapping process while editing levels. The afternoon was spent checking and deleting emails and apart from a five minute AGK task, it was all FPSC related :)
On The Subject Of Lasers
I started the ball rolling a few days ago to add lasers to Reloaded, but was convinced by Rick that lasers don't really sit well in the middle of a modern day generic graphics set so today we dropped them, along with laser booby traps and a beefy new laser gun for the player.
Instead we are going with something similar called tracer rounds, which show the person firing the gun where their bullet went. They will provide great additional visuals during a fire fight and allow us to add extra things like sounds when the bullet just misses your head, and showing where the enemies are shooting their guns. The feature will be enabled, like in real life, with the use of 'tracer rounds' which are alternative forms of ammo you can use with your weapons.
The best news is that the system can be easily extended in the future to bring lasers back, but we will probably do that when we do a big futuristic model pack for you.
Shader Plus Light Map Equals Hell
The burning task today is to get these two elements to play nicely. I have got quite a way already, now at 4:30PM, but I am getting random crashes with no easy way to find the cause. I know it has something to do with writes into naught memory so the search continues.
The good news is though that when not crashing, I can now see my lovely rendered shader floor and the real-time light mapping textures blending nicely together and the map editor has jumped up in visual quality. A long way to go though as the shader I chose only really works on floors, and I need a universal shader that ticks all the boxes for whatever object it has to render. I have plenty source material so it should be a fun process (that is, when I can stop it crashing).
Signing Off
I am also taking the opportunity to move the DBP DLL projects over to Visual Studio 2012 as well, as and when it becomes necessary to debug and work on them. My last machine had all three Visual Studio editions on and it was a bit of a mess, so moving to a single development kit should make life simpler. I dare say VS 2012 also has some nice improvements in terms of compiler speed, optimisation of byte code and better production of Windows 8 friendly applications.
My day continues from where it left off on Sunday, trying to get a shader and a light map to co-exist in my real-time light mapping process while editing levels. The afternoon was spent checking and deleting emails and apart from a five minute AGK task, it was all FPSC related :)
On The Subject Of Lasers
I started the ball rolling a few days ago to add lasers to Reloaded, but was convinced by Rick that lasers don't really sit well in the middle of a modern day generic graphics set so today we dropped them, along with laser booby traps and a beefy new laser gun for the player.
Instead we are going with something similar called tracer rounds, which show the person firing the gun where their bullet went. They will provide great additional visuals during a fire fight and allow us to add extra things like sounds when the bullet just misses your head, and showing where the enemies are shooting their guns. The feature will be enabled, like in real life, with the use of 'tracer rounds' which are alternative forms of ammo you can use with your weapons.
The best news is that the system can be easily extended in the future to bring lasers back, but we will probably do that when we do a big futuristic model pack for you.
Shader Plus Light Map Equals Hell
The burning task today is to get these two elements to play nicely. I have got quite a way already, now at 4:30PM, but I am getting random crashes with no easy way to find the cause. I know it has something to do with writes into naught memory so the search continues.
The good news is though that when not crashing, I can now see my lovely rendered shader floor and the real-time light mapping textures blending nicely together and the map editor has jumped up in visual quality. A long way to go though as the shader I chose only really works on floors, and I need a universal shader that ticks all the boxes for whatever object it has to render. I have plenty source material so it should be a fun process (that is, when I can stop it crashing).
Signing Off
I am also taking the opportunity to move the DBP DLL projects over to Visual Studio 2012 as well, as and when it becomes necessary to debug and work on them. My last machine had all three Visual Studio editions on and it was a bit of a mess, so moving to a single development kit should make life simpler. I dare say VS 2012 also has some nice improvements in terms of compiler speed, optimisation of byte code and better production of Windows 8 friendly applications.
Sunday, 21 April 2013
Sunday Reloaded Marathon
In Point Of Fact
Actually, I have just started this blog and the day, and have no idea if today will turn into a Reloaded marathon, but this is my intent. I had planned to start the player control system but having used the editor to do some fun play testing, I realised there are some loose ends that need finishing off. The blueprint graphic from the old system shows through from time to time which is not helpful, the was I slice the visuals so you can see the layer you are editing chops off cursors and the level does not light completely when you load a new map.
This is enough to make the whole thing feel unfinished, and I think it makes sense to tie these off now while I am closest to the time I messed it up. Even though control systems and physics is exciting and cool to work on, the sensible course is to make the current editor as close to final as is practicable.
You might be wandering why I am not outside enjoying the sun shine? Well, the weather man said heat wave, and right on schedule, the exact opposite blessed my Sunday with rain and clouds. So no repeat BBQ and no outside wheel barrow (or barrel) work. I consider myself quite smug now that I snook off and enjoyed the sun while it was available, and will probably make it a policy to do so this summer so I don't miss any 'good days'.
2PM : Playtesting
As is my ritual, I spend 10 minutes just playing with the current state of the software to get a feel for what is blatantly bad. It is obvious the editing process has been disrupted a little and I need to 'put it right' so it feels as solid and visually crisp as ever. I have made a small list for my reference, and intend to work through it as the day unfolds.
3PM : Loading In
The old system would load the map and then 'maybe' use the last loaded light to illuminate the scene. Not ideal as the scene could be full of details that need lighting. I have added a system to create a cascade of light map updates for each light in the scene just after the level loads in, so the user only has to wait and watch as the lighting is restored. I am not going the other route of saving the light maps with each FPM file as I like the fact they are very small and interchangeable.
It's a minor thing, but when you load your level you want to see it in the proper light (pardon the pun and the classic graphics).
3:35PM : Object Cascading
Similar to above, when you draw lots of segments very fast, the light mapper can skips objects during the light mapping and so they appear unlit in the scene, which looks bad. I am adding a system which will store all objects in a list, and then use that list to feed the light mapper. This will create a trail which the real time light mapper can follow to ensure ALL objects are light mapped as you are editing.
3:45PM : Very Nice
The object cascade system works very nice, it actually ran backwards with the newest object being light mapped before the older ones, but I like this as it maintains focus on what the user is doing right now and not older stuff that might be far away from the scene of editing. I think I will keep this order. I also noticed the cursor is flickering all over the place so I will see if I can tame that and also sort out the visibility of the cube cursor (such as those used when editing segments) so they once again appear whole and unclipped so I can see the top of them. Right now my layer slicing system is cutting their tops off!
4:10PM : Performance Boost
The first implementation of the object cascade light mapper seemed to be light mapping objects around the newly added object which was not ideal as complex areas would trigger a lengthy light map step multiplied by all the objects in the area, lightmapping objects that had already been lit. By adding some extra code which removed objects from the active cascade list when they where indirectly lit by a previous lighting step, all redundant light map requests are eliminated and a faster lighting experience ensues. Hurray!
4:28PM : Not Entirely Happy
There are cases where segments which have already been light mapped in previous passes are being included in the cascade list, but there is no 'memory' to tell me these objects should not be included so take up precious time. It's not critical, but as an observer, you might be left wondering why the light mapper is going ten to the dozen without any light map changes taking place. Maybe I will come back to this if it becomes intolerable. On top of that, my latest monster test just crashed without giving me a call stack trace which is not very helpful. Will start running in debug mode in case it happens again.
4:40PM : Polish Early
I noticed when there was a quick light map step such as one segment tile the light map status bar flickered as it went from zero to one hundred in a few milliseconds. Not pretty. I have added some smoothing code which increases the alpha of the status graphic the longer it's being used, so a quick step should not show and a longer process will show the bar fading in and then fading out. Not easy either as it's very much a case of looking at the activity of the light map percentage value and working out when to bring up the alpha an when to hide the status bar altogether. Still, should be worth it when done correctly.
5:10PM : A Spot Of Lunch
I've just finished the above polished light map status bar and also removed the camera range slicer which messed up so much. After lunch I am going to see if I can go back to the idea of hiding layers of no interest during the edit process. Ideally I only need to traverse the 'massive' object group when changing layers so theoretically should be a good solution. Time and experimentation will tell...but first, food.
5:40PM : Soupee-Twist
Just finished my spot of lunch and finished watching half an hour of great comedy on Netflix. Congratulations if you can figure out what it was. Just ran Reloaded to see where I was, and the clip slicing as gone which means I need to replace it with something else. That will be my mission now..
6:00PM : Mission Aborted
The object cascade list system is not ideal. A simple segment room with two storeys and a single light seems to have a lot of objects to repeatedly light which is nuts! I think I will return to it and 'dump' the whole object list into the light mapping pass so it gets it over with in one go. It might mean less frequent light map updates, but it will eliminate the current system issue which seems to want to process objects fully 60 seconds after the last light map change was detected. Grr.
6:15PM : Label Text Moved To Status Bar
When I adjusted the size of the back-buffer for perfect 1:1 pixel quality in the test game mode, the map editors smaller window scrunched up the text. The two back buffer sizes cannot be reconciled and naturally we prefer the higher quality test game render, so I have moved ALL text from the map editor over to the status bar in the IDE. It means the editor is less cluttered and the text is also 1:1 perfect too so another win-win.
7:25PM : Three In One
In moving to the next bit which was hiding non-current layers, I realised the alpha mapping code was not working. Rather than mess about with fixed function questions, I realised that the editor will be using shaders directly in the final product so decided to skip to adding a basic universal shader to the map editor so the alpha factor functionality would once again work. It also means we're one step closer to the final editor visuals which will feature the full shader rendering, not a washed out representation as in the old engine.
8:35PM : Shaders Are Buggers
I always have a fun old time with shaders, that is, they are a royal pain in the butt sometimes. Most of the time they have errors, and just disappear, but you don't know whether they are invisible because of the shader, or invisible because they are working with alpha at zero. Grr. And don't get me started on the task of integrating the shader with the fixed functionality of the light mapped object after the real time light mapper has finished with it. So much to do that's so obvious, but so far away...
10:28M : Making Slow But Steady Progress
I have the layers separated by layer now based on the layer you are editing using the new alpha mapper on the freshly made universal shader. The new shader is based on one of Mark Blosser's creations and works out the box so I can adapt as I go. My first adoption was to allow the alpha to be overridden so the editor could fade out individual objects.
I re-introduced the light mapper and of course, boom, all alpha stuff ignored. Now I need to make sure the real time light mapper can handle shaded objects as I don't fancy making the code re-apply the shader after light mapping as it seems redundant when the back-end could do it quicker and more elegantly. That is my next port of call. I think I made the right call sorting these varied issues out and not jumping into cool physics and control systems, it's the boring stuff that really needs pinning down to ensure it's a great user experience at the edit end of the product.
Signing Off
Well it's a few minutes to midnight and I've not eaten since my spot of lunch, and right now when I drop in a large segment my lovely light mapper crashes. I probably should not have starting blending the shader and light mapper into one thing as it's a big thought to do and it was the back end of the day. Just going to stop it crashing by backtracking a little, then calling it a night. Can't abide leaving a project with a crash at the very front of it. Once I've tackled that, a spot of 'midnight dinner' and then off to beddy bye boes.
Actually, I have just started this blog and the day, and have no idea if today will turn into a Reloaded marathon, but this is my intent. I had planned to start the player control system but having used the editor to do some fun play testing, I realised there are some loose ends that need finishing off. The blueprint graphic from the old system shows through from time to time which is not helpful, the was I slice the visuals so you can see the layer you are editing chops off cursors and the level does not light completely when you load a new map.
This is enough to make the whole thing feel unfinished, and I think it makes sense to tie these off now while I am closest to the time I messed it up. Even though control systems and physics is exciting and cool to work on, the sensible course is to make the current editor as close to final as is practicable.
You might be wandering why I am not outside enjoying the sun shine? Well, the weather man said heat wave, and right on schedule, the exact opposite blessed my Sunday with rain and clouds. So no repeat BBQ and no outside wheel barrow (or barrel) work. I consider myself quite smug now that I snook off and enjoyed the sun while it was available, and will probably make it a policy to do so this summer so I don't miss any 'good days'.
2PM : Playtesting
As is my ritual, I spend 10 minutes just playing with the current state of the software to get a feel for what is blatantly bad. It is obvious the editing process has been disrupted a little and I need to 'put it right' so it feels as solid and visually crisp as ever. I have made a small list for my reference, and intend to work through it as the day unfolds.
3PM : Loading In
The old system would load the map and then 'maybe' use the last loaded light to illuminate the scene. Not ideal as the scene could be full of details that need lighting. I have added a system to create a cascade of light map updates for each light in the scene just after the level loads in, so the user only has to wait and watch as the lighting is restored. I am not going the other route of saving the light maps with each FPM file as I like the fact they are very small and interchangeable.
BEFORE
AFTER
It's a minor thing, but when you load your level you want to see it in the proper light (pardon the pun and the classic graphics).
3:35PM : Object Cascading
Similar to above, when you draw lots of segments very fast, the light mapper can skips objects during the light mapping and so they appear unlit in the scene, which looks bad. I am adding a system which will store all objects in a list, and then use that list to feed the light mapper. This will create a trail which the real time light mapper can follow to ensure ALL objects are light mapped as you are editing.
3:45PM : Very Nice
The object cascade system works very nice, it actually ran backwards with the newest object being light mapped before the older ones, but I like this as it maintains focus on what the user is doing right now and not older stuff that might be far away from the scene of editing. I think I will keep this order. I also noticed the cursor is flickering all over the place so I will see if I can tame that and also sort out the visibility of the cube cursor (such as those used when editing segments) so they once again appear whole and unclipped so I can see the top of them. Right now my layer slicing system is cutting their tops off!
4:10PM : Performance Boost
The first implementation of the object cascade light mapper seemed to be light mapping objects around the newly added object which was not ideal as complex areas would trigger a lengthy light map step multiplied by all the objects in the area, lightmapping objects that had already been lit. By adding some extra code which removed objects from the active cascade list when they where indirectly lit by a previous lighting step, all redundant light map requests are eliminated and a faster lighting experience ensues. Hurray!
4:28PM : Not Entirely Happy
There are cases where segments which have already been light mapped in previous passes are being included in the cascade list, but there is no 'memory' to tell me these objects should not be included so take up precious time. It's not critical, but as an observer, you might be left wondering why the light mapper is going ten to the dozen without any light map changes taking place. Maybe I will come back to this if it becomes intolerable. On top of that, my latest monster test just crashed without giving me a call stack trace which is not very helpful. Will start running in debug mode in case it happens again.
4:40PM : Polish Early
I noticed when there was a quick light map step such as one segment tile the light map status bar flickered as it went from zero to one hundred in a few milliseconds. Not pretty. I have added some smoothing code which increases the alpha of the status graphic the longer it's being used, so a quick step should not show and a longer process will show the bar fading in and then fading out. Not easy either as it's very much a case of looking at the activity of the light map percentage value and working out when to bring up the alpha an when to hide the status bar altogether. Still, should be worth it when done correctly.
5:10PM : A Spot Of Lunch
I've just finished the above polished light map status bar and also removed the camera range slicer which messed up so much. After lunch I am going to see if I can go back to the idea of hiding layers of no interest during the edit process. Ideally I only need to traverse the 'massive' object group when changing layers so theoretically should be a good solution. Time and experimentation will tell...but first, food.
5:40PM : Soupee-Twist
Just finished my spot of lunch and finished watching half an hour of great comedy on Netflix. Congratulations if you can figure out what it was. Just ran Reloaded to see where I was, and the clip slicing as gone which means I need to replace it with something else. That will be my mission now..
6:00PM : Mission Aborted
The object cascade list system is not ideal. A simple segment room with two storeys and a single light seems to have a lot of objects to repeatedly light which is nuts! I think I will return to it and 'dump' the whole object list into the light mapping pass so it gets it over with in one go. It might mean less frequent light map updates, but it will eliminate the current system issue which seems to want to process objects fully 60 seconds after the last light map change was detected. Grr.
6:15PM : Label Text Moved To Status Bar
When I adjusted the size of the back-buffer for perfect 1:1 pixel quality in the test game mode, the map editors smaller window scrunched up the text. The two back buffer sizes cannot be reconciled and naturally we prefer the higher quality test game render, so I have moved ALL text from the map editor over to the status bar in the IDE. It means the editor is less cluttered and the text is also 1:1 perfect too so another win-win.
7:25PM : Three In One
In moving to the next bit which was hiding non-current layers, I realised the alpha mapping code was not working. Rather than mess about with fixed function questions, I realised that the editor will be using shaders directly in the final product so decided to skip to adding a basic universal shader to the map editor so the alpha factor functionality would once again work. It also means we're one step closer to the final editor visuals which will feature the full shader rendering, not a washed out representation as in the old engine.
8:35PM : Shaders Are Buggers
I always have a fun old time with shaders, that is, they are a royal pain in the butt sometimes. Most of the time they have errors, and just disappear, but you don't know whether they are invisible because of the shader, or invisible because they are working with alpha at zero. Grr. And don't get me started on the task of integrating the shader with the fixed functionality of the light mapped object after the real time light mapper has finished with it. So much to do that's so obvious, but so far away...
10:28M : Making Slow But Steady Progress
I have the layers separated by layer now based on the layer you are editing using the new alpha mapper on the freshly made universal shader. The new shader is based on one of Mark Blosser's creations and works out the box so I can adapt as I go. My first adoption was to allow the alpha to be overridden so the editor could fade out individual objects.
I re-introduced the light mapper and of course, boom, all alpha stuff ignored. Now I need to make sure the real time light mapper can handle shaded objects as I don't fancy making the code re-apply the shader after light mapping as it seems redundant when the back-end could do it quicker and more elegantly. That is my next port of call. I think I made the right call sorting these varied issues out and not jumping into cool physics and control systems, it's the boring stuff that really needs pinning down to ensure it's a great user experience at the edit end of the product.
Signing Off
Well it's a few minutes to midnight and I've not eaten since my spot of lunch, and right now when I drop in a large segment my lovely light mapper crashes. I probably should not have starting blending the shader and light mapper into one thing as it's a big thought to do and it was the back end of the day. Just going to stop it crashing by backtracking a little, then calling it a night. Can't abide leaving a project with a crash at the very front of it. Once I've tackled that, a spot of 'midnight dinner' and then off to beddy bye boes.
Saturday, 20 April 2013
Saturday At Play
More Sunshine
Apparently we are getting sun tomorrow as well as today, which is a rare thing for us Welsh-bound souls at the moment so I'm hitting the 'outdoors' every day so far. Right now I am still recovering from a BBQ, and will probably enjoy more sunshine Sunday too!
As promised, I did find time to fire up the PC to do some extra work and once my email was cleared out a had a few gems to play with. Rather that blog about my own coding, I wanted to highlight two DBP coders who will be influencing and perhaps contributing to the Reloaded effort.
Darkest Occlusion
I have just put the finishers on a new Dark Occlusion installer update which takes the current DBP coded system over to a DLL, which means more speed and more security. I gave the demo a quick run, and boy is it fast. I was going to use the Intel Software Occluder but it's DirectX 11 and will take some untangling. The new Dark Occluder is a serious contender though, and I will be looking at it closer in the days to come:
I plan to run some tests to see how it handles object limbs, and/or large small objects which is what FPSC will throw at it when occlusion is required, but I have a feeling it will fly along very nicely in terms of performance. I gave it 10,000 objects to occlude and it sped along at over 300fps ;) Nice! If you want to learn more, check out the current version here:
http://www.thegamecreators.com/?m=view_product&id=2311
Shameless plug I know, but when your DBP projects start to get into the 'thousands of objects' territory, it's time to bring in the big guns!
Darkest Lighting
I also checked out a link from a blog comment and was impressed with what I discovered at the other end. Evolved is a name many DBP coders will recognise and respect, and he continues to dazzle with his latest Advanced Lighting demo. Check out his site and the excellent technologies on offer here: http://www.evolved-software.com/advancedlighting/advancedlighting
Right now my plan is to write my own unified shader system to combine many of the effects you would expect from Reloaded, by Evolved has produced a great module which seems to do it all. If you have to check out just one of his demos, try the 'Materials' once where you can roam about an interior brick scene. Check out the depth of field, parallax mapping, dynamic shadows and metallic surface shader. All top draw!
Signing Off
I am very careful when it comes to other coders copyright and approvals, so I won't be able to just copy their code and stick it into Reloaded. Once I have a play, and if it's a good fit, I will tentatively approach the authors to see if I can 'compliment' Reloaded with their excellent technologies. I am sold on the visuals of the Advanced Lighting and the speed of the Occluder, but like most things in coder land, many planets have to line up before things start to work.
Apparently we are getting sun tomorrow as well as today, which is a rare thing for us Welsh-bound souls at the moment so I'm hitting the 'outdoors' every day so far. Right now I am still recovering from a BBQ, and will probably enjoy more sunshine Sunday too!
As promised, I did find time to fire up the PC to do some extra work and once my email was cleared out a had a few gems to play with. Rather that blog about my own coding, I wanted to highlight two DBP coders who will be influencing and perhaps contributing to the Reloaded effort.
Darkest Occlusion
I have just put the finishers on a new Dark Occlusion installer update which takes the current DBP coded system over to a DLL, which means more speed and more security. I gave the demo a quick run, and boy is it fast. I was going to use the Intel Software Occluder but it's DirectX 11 and will take some untangling. The new Dark Occluder is a serious contender though, and I will be looking at it closer in the days to come:
I plan to run some tests to see how it handles object limbs, and/or large small objects which is what FPSC will throw at it when occlusion is required, but I have a feeling it will fly along very nicely in terms of performance. I gave it 10,000 objects to occlude and it sped along at over 300fps ;) Nice! If you want to learn more, check out the current version here:
http://www.thegamecreators.com/?m=view_product&id=2311
Shameless plug I know, but when your DBP projects start to get into the 'thousands of objects' territory, it's time to bring in the big guns!
Darkest Lighting
I also checked out a link from a blog comment and was impressed with what I discovered at the other end. Evolved is a name many DBP coders will recognise and respect, and he continues to dazzle with his latest Advanced Lighting demo. Check out his site and the excellent technologies on offer here: http://www.evolved-software.com/advancedlighting/advancedlighting
Right now my plan is to write my own unified shader system to combine many of the effects you would expect from Reloaded, by Evolved has produced a great module which seems to do it all. If you have to check out just one of his demos, try the 'Materials' once where you can roam about an interior brick scene. Check out the depth of field, parallax mapping, dynamic shadows and metallic surface shader. All top draw!
Signing Off
I am very careful when it comes to other coders copyright and approvals, so I won't be able to just copy their code and stick it into Reloaded. Once I have a play, and if it's a good fit, I will tentatively approach the authors to see if I can 'compliment' Reloaded with their excellent technologies. I am sold on the visuals of the Advanced Lighting and the speed of the Occluder, but like most things in coder land, many planets have to line up before things start to work.
Friday, 19 April 2013
Friday sun
A day outdoors
The day started normally with work then the sun started shining so I left the keyboard and picked up the wheel barrel. After a long day digging I have decided to have an early night and work over the weekend. Tune in for a rare weekend blog ;)
The day started normally with work then the sun started shining so I left the keyboard and picked up the wheel barrel. After a long day digging I have decided to have an early night and work over the weekend. Tune in for a rare weekend blog ;)
Thursday, 18 April 2013
Thursday Non-Loaded
A Day In Bits
Today was designated 'bits day' where I move other projects along that had been building up, and it has been a successful one. Got two apps free of bugs and a few preliminary versions ready for release. Currently retrieving a pre-Perceptual version of DBP from my backups to build a new FPSC V120 without the MFC dependencies :)
Reloaded News
I can report some Reloaded news from last night which saw the rest of the light mapping tweaks put in, and it hums along nicely now. I changed the text font to improve visuals but it still lets it down so I have decided to have no text in the map editor part, and allow the full screen text game to dictate the back-buffer size. This basically means higher quality test game renders.
I also took a few hours to play Borderlands 2 as well, which was a very good idea of mine. I am about to start the player control system and Borderlands 2 is a nice game to teach you about a control system. It's simple, but works. The speed of movement, jump height and timing, strafing, stop/start inertias and mouse look speeds are all nicely calibrated. My hope is to get something similar in the next week or so (the controls that is) ;)
It also teaches some great character AI as well, with some enemies just charging in, others hiding and lobbing metal objects at you and others sniping from positions almost as good as the player can find. I think it's a great game to bounce back and forth from and should improve the Reloaded offering over time.
Signing Off
It is about 6PM now and I have one more hour of 'bit duty', then going to reward myself with a huge steak with stinky cheese on top. Maybe be back this evening to play borderlands 2, or get started on the control system, or finalise any beta uploads that might be required. As it was an early start (relatively) for me, I am looking for an early night so we will see how that goes. All in all, a very productive day (and an almost empty inbox).
Today was designated 'bits day' where I move other projects along that had been building up, and it has been a successful one. Got two apps free of bugs and a few preliminary versions ready for release. Currently retrieving a pre-Perceptual version of DBP from my backups to build a new FPSC V120 without the MFC dependencies :)
Reloaded News
I can report some Reloaded news from last night which saw the rest of the light mapping tweaks put in, and it hums along nicely now. I changed the text font to improve visuals but it still lets it down so I have decided to have no text in the map editor part, and allow the full screen text game to dictate the back-buffer size. This basically means higher quality test game renders.
I also took a few hours to play Borderlands 2 as well, which was a very good idea of mine. I am about to start the player control system and Borderlands 2 is a nice game to teach you about a control system. It's simple, but works. The speed of movement, jump height and timing, strafing, stop/start inertias and mouse look speeds are all nicely calibrated. My hope is to get something similar in the next week or so (the controls that is) ;)
It also teaches some great character AI as well, with some enemies just charging in, others hiding and lobbing metal objects at you and others sniping from positions almost as good as the player can find. I think it's a great game to bounce back and forth from and should improve the Reloaded offering over time.
Signing Off
It is about 6PM now and I have one more hour of 'bit duty', then going to reward myself with a huge steak with stinky cheese on top. Maybe be back this evening to play borderlands 2, or get started on the control system, or finalise any beta uploads that might be required. As it was an early start (relatively) for me, I am looking for an early night so we will see how that goes. All in all, a very productive day (and an almost empty inbox).
Wednesday, 17 April 2013
Wednesday And Light Design
Illuminating Day
A good day today with all those little niggles being squashed one by one. The system is so nice now I almost made a video for you guys and gals. I passed it to quality control (Rick) and with a few more polish tweaks it will be ready for public consumption.
The above picture shows some of the alley graphics you saw in the original Kickstarter. What you might find quite cool is that this is a shot from the FPSC map editor. The shadows being cast are real time generated, and will move when you move the yellow posts (or if you move the light too). The combined effect of this feature is quite amazing as you start to appreciate the subtle and powerful art of lighting design!
A Run Down
Right now I can add and remove entities and segments as fast as I like, and the light-mapping keeps pace very nicely. I switched from area lights to point lights and naturally it got even faster, so much so that they could be mistaken for dynamic lights in a low poly area :) I dare say the light mapping routines can be optimised even more, and with control over quality settings while editing the whole thing could potentially zip along at a higher rate. That said, I am happy with what I have now which means I am nearly ready to move on.
I was going to add a system which chained all your changes into a list of required updates, and the light mapper could work through the list, but as I edit the scene now it seems not to be required. Not wanting to make work for myself I will skip this feature and come back to it if needed. This omission reveals itself when you might place five entities very quickly in a wide area of the map, and the middle entity might get missed as the 'latest' entity placement overwrites it. This is normally hidden as entities are usually placed close together when editing, and the next entity is already cued up ready as the present one is being light mapped, so it usually gets hidden well.
Another optimisation I had was to refine the way the light mapper detects exactly which objects need lighting as right now it is a rather crude distance calculation. A better way would be to check for objects clipping the light frustum volume from the point light to the object being placed (thereby objects within the shadow cast by the newly placed object need updating). Again, this is an optimisation I can call on if I need more speed in this feature later on.
Going to go through the usual steps of checking for stability, memory leaks and of course restoring editing to the same functional level as the classic engine. Right now all this happens on one layer. There is no longer alpha mapping and slicing of the edited level so I need to reintroduce a new system to cope with that. For some reason alpha mapping is not working with the new cloned objects so that needs looking at. Once I can step up and down the layers, and edit as before, I will be happy to move on.
Signing Off
I will be moving onto player controls after real time light mapping, which will be a lot of fun. Right now the mouse has a limited mouse look capability as I use keyboard and mouse sharing technology, which means when I move to the left of the screen I switch to another PC. Not good if you are mouse looking in the test game. Once I solve that, I can bring in the old control system from the classic engine and make any improvements required. One comment from a previous blog used the word 'fluid' and I think that sums up the goal perfectly. You really need to feel the mouse is an extension of your hand, and that sliding around the level feels responsive and slick. We have the frame rate, we just need the subtle art of inertia tweaking and speed control. I will probably need some mouse sensitivity settings too, as everyone's mouse is different!
A good day today with all those little niggles being squashed one by one. The system is so nice now I almost made a video for you guys and gals. I passed it to quality control (Rick) and with a few more polish tweaks it will be ready for public consumption.
The above picture shows some of the alley graphics you saw in the original Kickstarter. What you might find quite cool is that this is a shot from the FPSC map editor. The shadows being cast are real time generated, and will move when you move the yellow posts (or if you move the light too). The combined effect of this feature is quite amazing as you start to appreciate the subtle and powerful art of lighting design!
A Run Down
Right now I can add and remove entities and segments as fast as I like, and the light-mapping keeps pace very nicely. I switched from area lights to point lights and naturally it got even faster, so much so that they could be mistaken for dynamic lights in a low poly area :) I dare say the light mapping routines can be optimised even more, and with control over quality settings while editing the whole thing could potentially zip along at a higher rate. That said, I am happy with what I have now which means I am nearly ready to move on.
I was going to add a system which chained all your changes into a list of required updates, and the light mapper could work through the list, but as I edit the scene now it seems not to be required. Not wanting to make work for myself I will skip this feature and come back to it if needed. This omission reveals itself when you might place five entities very quickly in a wide area of the map, and the middle entity might get missed as the 'latest' entity placement overwrites it. This is normally hidden as entities are usually placed close together when editing, and the next entity is already cued up ready as the present one is being light mapped, so it usually gets hidden well.
Another optimisation I had was to refine the way the light mapper detects exactly which objects need lighting as right now it is a rather crude distance calculation. A better way would be to check for objects clipping the light frustum volume from the point light to the object being placed (thereby objects within the shadow cast by the newly placed object need updating). Again, this is an optimisation I can call on if I need more speed in this feature later on.
Going to go through the usual steps of checking for stability, memory leaks and of course restoring editing to the same functional level as the classic engine. Right now all this happens on one layer. There is no longer alpha mapping and slicing of the edited level so I need to reintroduce a new system to cope with that. For some reason alpha mapping is not working with the new cloned objects so that needs looking at. Once I can step up and down the layers, and edit as before, I will be happy to move on.
Signing Off
I will be moving onto player controls after real time light mapping, which will be a lot of fun. Right now the mouse has a limited mouse look capability as I use keyboard and mouse sharing technology, which means when I move to the left of the screen I switch to another PC. Not good if you are mouse looking in the test game. Once I solve that, I can bring in the old control system from the classic engine and make any improvements required. One comment from a previous blog used the word 'fluid' and I think that sums up the goal perfectly. You really need to feel the mouse is an extension of your hand, and that sliding around the level feels responsive and slick. We have the frame rate, we just need the subtle art of inertia tweaking and speed control. I will probably need some mouse sensitivity settings too, as everyone's mouse is different!
Tuesday, 16 April 2013
Tuesday Cave Dwelling
The Mound Rises
Since the fateful decision was made to go into official obsession mode over Reloaded, it's been a blast. That said, it is also apparent that small jobs that would otherwise have been done as a matter of course are now piling up nicely in a mound of todo's. Some of them actually quite substantial product releases too :)
Rather than revert to the old ways, I have decided to block book Thursday to do all those tasks and get the inbox clear and all Lee related matters resolved. Some developers can happily bounce between a few different projects, but I find the going quite slow when I do that. There is a certain amount of re-acquainting with the subject matter before you can get into a rhythm and then of course you change records again. At least confining all the potential distractions to one day caps each end and contains the matter.
Reloaded Progress So Far
I now have segments, overlays, entities and lights being added and removed using the new object system and the real time light mapping adjusting the map in the background without slowing down proceedings. A great result for what was simply a theory a few months ago.
I also solved a few crash issues caused by the light mapper lighting objects in the background, which where then deleted by me as it was happening, meaning it would be using data that had been deleted elsewhere. This is a common problem when writing multi-core software and something I should be on my guard for. Now I simply hide the objects until the process is complete and the threads finished, and then I can delete them safely.
Right now I am battling with deleting lights and having that event trigger more lighting in the background. Once I have the full gamut of adding and deletion when editing sorted, and hiding the markers in the main test mode, I should be ready to move on a little.
Small But Vital Tweak
One little change I made while playing with entity placement was for the cursor to remember the actual position of the entity before you attach it to your mouse cursor. This way, when you click to drag an entity it no longer jumps position to centre on the mouse pointer and instead stays put. This means micro changes can be made using the top down view without the frustration of constantly having to re-position every entity you lift off the map. It's a small change, but really improves the editing process.
Signing Off
It's 4PM now so a few more hours of lovely toil and then we'll be further along. The next thing I want to do is a good player control system so I can move, strafe, jump, mouse look and manouver like a good 'un. Taking inspiration from a few of the FPS games I have installed will help in this, and before I add bullet physics I want to have a basic player control system in place so I can see how the physics affects it once implemented. The physics will be an exciting chunk of work, but I want to completely finish the light mapping stuff so I don't have to come back in a months time to fix things. Better do it now while it is fresh in my mind.
Since the fateful decision was made to go into official obsession mode over Reloaded, it's been a blast. That said, it is also apparent that small jobs that would otherwise have been done as a matter of course are now piling up nicely in a mound of todo's. Some of them actually quite substantial product releases too :)
Rather than revert to the old ways, I have decided to block book Thursday to do all those tasks and get the inbox clear and all Lee related matters resolved. Some developers can happily bounce between a few different projects, but I find the going quite slow when I do that. There is a certain amount of re-acquainting with the subject matter before you can get into a rhythm and then of course you change records again. At least confining all the potential distractions to one day caps each end and contains the matter.
Reloaded Progress So Far
I now have segments, overlays, entities and lights being added and removed using the new object system and the real time light mapping adjusting the map in the background without slowing down proceedings. A great result for what was simply a theory a few months ago.
I also solved a few crash issues caused by the light mapper lighting objects in the background, which where then deleted by me as it was happening, meaning it would be using data that had been deleted elsewhere. This is a common problem when writing multi-core software and something I should be on my guard for. Now I simply hide the objects until the process is complete and the threads finished, and then I can delete them safely.
Right now I am battling with deleting lights and having that event trigger more lighting in the background. Once I have the full gamut of adding and deletion when editing sorted, and hiding the markers in the main test mode, I should be ready to move on a little.
Small But Vital Tweak
One little change I made while playing with entity placement was for the cursor to remember the actual position of the entity before you attach it to your mouse cursor. This way, when you click to drag an entity it no longer jumps position to centre on the mouse pointer and instead stays put. This means micro changes can be made using the top down view without the frustration of constantly having to re-position every entity you lift off the map. It's a small change, but really improves the editing process.
Signing Off
It's 4PM now so a few more hours of lovely toil and then we'll be further along. The next thing I want to do is a good player control system so I can move, strafe, jump, mouse look and manouver like a good 'un. Taking inspiration from a few of the FPS games I have installed will help in this, and before I add bullet physics I want to have a basic player control system in place so I can see how the physics affects it once implemented. The physics will be an exciting chunk of work, but I want to completely finish the light mapping stuff so I don't have to come back in a months time to fix things. Better do it now while it is fresh in my mind.
Monday, 15 April 2013
Monday Amazing Mark Day
Late Last Night
Whilst dozing in front of a few episodes of Wheeler Dealer I got an email, and boy what an email. Mark has finished his first draft of the Reloaded character model and it's stunning.
If you don't believe me, check out the close-up of the face and especially the eyes. Now imagine when this guy is talking, and you'll get some idea of the level of quality Reloaded will inflict on you!
I am super excited to see this, and can't wait to start a new prototype that will have him put through his paces. Behaviour and AI will be a big feature of the new FPSC Reloaded product and the character will be the vehicle by which we will demonstrate this feature. Happy Days!
On The Coding Front
As I reported over the weekend I increased the speed at which I can compile the FPSC engine which means it is almost four times quicker for me to change something then see the results running. A breath of fresh air so far, and it really feels like the development has become more fluid. No more surfing the net every compile which I wait for it!
I'm working on the entity creation and placement now, and also the entity floater object which is the entity attached to your cursor just before you place them down. With these in place, and added to the segment stuff which is done, I will have rewritten the whole real object shroud system of the old FPSC forever and from here on in, all objects are created and retained.
A nice bonus to this new system is that all textures and shaders are applied to full objects, not instances, which means when you customise an entity, you will see that customisation in the editor right away. In the old FPSC engine you had to wait until you played the game before you saw any changes you made to the entity texture or shader.
Signing Off
It's about 2PM now, so time for some beans and toast, then onto finishing the entity stuff, and a big stress test on what I have so far before moving much further. It is vital I ensure there are no leaks, no loss in functionality and that what I have done will stand up well for the rest of the project. It's the best time to do stuff like this as the code is still fresh in my mind. Once we wave good bye to these tasks, the next fun task is Collision and tackling the dreaded 'fall through floor or get stuck' issues of the old universe.
Whilst dozing in front of a few episodes of Wheeler Dealer I got an email, and boy what an email. Mark has finished his first draft of the Reloaded character model and it's stunning.
If you don't believe me, check out the close-up of the face and especially the eyes. Now imagine when this guy is talking, and you'll get some idea of the level of quality Reloaded will inflict on you!
I am super excited to see this, and can't wait to start a new prototype that will have him put through his paces. Behaviour and AI will be a big feature of the new FPSC Reloaded product and the character will be the vehicle by which we will demonstrate this feature. Happy Days!
On The Coding Front
As I reported over the weekend I increased the speed at which I can compile the FPSC engine which means it is almost four times quicker for me to change something then see the results running. A breath of fresh air so far, and it really feels like the development has become more fluid. No more surfing the net every compile which I wait for it!
I'm working on the entity creation and placement now, and also the entity floater object which is the entity attached to your cursor just before you place them down. With these in place, and added to the segment stuff which is done, I will have rewritten the whole real object shroud system of the old FPSC forever and from here on in, all objects are created and retained.
A nice bonus to this new system is that all textures and shaders are applied to full objects, not instances, which means when you customise an entity, you will see that customisation in the editor right away. In the old FPSC engine you had to wait until you played the game before you saw any changes you made to the entity texture or shader.
Signing Off
It's about 2PM now, so time for some beans and toast, then onto finishing the entity stuff, and a big stress test on what I have so far before moving much further. It is vital I ensure there are no leaks, no loss in functionality and that what I have done will stand up well for the rest of the project. It's the best time to do stuff like this as the code is still fresh in my mind. Once we wave good bye to these tasks, the next fun task is Collision and tackling the dreaded 'fall through floor or get stuck' issues of the old universe.
Saturday, 13 April 2013
Weekend Blog
Surprise Blog
I know I don't normally blog the weekend (otherwise the ritual would paint my addled brain into a corner from whence escape would be impossible), but the momentum of doing the Perceptual Computing Challenge has given me a sense that weekends are okay for extra coding.
The day light was spent shovelling shale from one side of my garden to the other, and the evening was spent watching the Magicians with David Mitchell and Robert Webb (very funny guys). A bit of inspiring end credits music got me thinking that maybe I could throw a few hours at Reloaded, and so I did.
My Few Thrown Hours
It occurred to me at the end of a full week of Reloaded development that one of the biggest obstacles to quick progress was the speed of the DBP compiler (currently 3 minutes on my monster Reloaded PC). I figured if I could have a faster compiler, I could do more compiles in the day and the thing I was tweaking or fixing or adding would be fresher in my mind and next week will produce even more Reloaded goodness.
Secret Super Coder
After ten plus years running a software company, you cannot help but perfect the art of delegation, and sometimes it pays dividends. One such delegation brokered the assistance of a top coder, who kindly volunteered his time to help me spruce up the internal version of DBP, starting with the compiler, in anticipation of larger DBP projects such as Reloaded.
The fantastic news is that an early version of the new compiler has taken my compile time from 3 minutes down to 45 seconds, which is an immense speed boost when you are typically compiling about 100 times a day minimum. My few hours this weekend in integrating this new version with the Reloaded dev set-up will save me literally days of time over the course of the remainder of this project! Those acquired days can be fed back into the engine to make it even better and make my coding life sane once again.
Before every DBP coder starts jumping up and down, the compiler is far from finished and the 'super secret top coder' will want paying for his efforts too, so it will not be a free update. My priority remains Reloaded all the way through to October, and all forces will be steadily employed in this direction, but along the way if the opportunity exists to commercialise these internal DBP enhancements, you will be the first to know!
A Note On Concurrency
The term 'concurrency' is often used to describe how many cores your app is using at the same time (i.e. running concurrently side by side). For fun I draw a few rooms and dropped a light into the map editor of the current Reloaded product. I then opened the performance profiler which indicates what my four cores where doing. Guess what. Processor utilisation was 100% from the moment the light was placed through to when the light mapping appeared. That means all four cores where red hot to get the light mapping done as quickly as possible on the monster machine it finds itself on. Intel would be very proud of an app like Reloaded, and for the first time FPSC will be tapping every corner of your processing power even as you edit and tweak your levels.
I also plan to get the same (or near) when you play the game too, thanks to the multi-core DarkAI system which will be asked to calculate some fiendish math to figure out the best way to strike at the player. Getting full concurrency when you edit and when you play will mean we have a piece of software that pushes your machine to the maximum.
Signing Off
It's now 2:30AM and I will be turning in very soon. I'm happy with this coding stint, not least because it has just improved my life by 2 minutes 15 seconds per compile and once I have stripped out the old 'build process' and 'blue print shroud system', the code base will be even smaller which means even faster compiles. That is, until I add the new bullet physics system, terrain system and combat intelligence systems the likes of which you'll have nightmares about. A big player in the internal Reloaded team suggested this product needs a killer feature to make it stand out from the crowd. Rick, our resident pessimist[realist], concluded there are no killer ideas left, and everything has been done, and small teams like us don't have the manpower for that level of innovation. I disagree. I think it is precisely because we are small that we can take a risk on a killer idea that might very well transform the FPS genre forever. I don't know what the idea is of course, and I dare say it may never surface, but if you have an awesome idea you've never seen in an FPS, post your comments here, I read them all!
I know I don't normally blog the weekend (otherwise the ritual would paint my addled brain into a corner from whence escape would be impossible), but the momentum of doing the Perceptual Computing Challenge has given me a sense that weekends are okay for extra coding.
The day light was spent shovelling shale from one side of my garden to the other, and the evening was spent watching the Magicians with David Mitchell and Robert Webb (very funny guys). A bit of inspiring end credits music got me thinking that maybe I could throw a few hours at Reloaded, and so I did.
My Few Thrown Hours
It occurred to me at the end of a full week of Reloaded development that one of the biggest obstacles to quick progress was the speed of the DBP compiler (currently 3 minutes on my monster Reloaded PC). I figured if I could have a faster compiler, I could do more compiles in the day and the thing I was tweaking or fixing or adding would be fresher in my mind and next week will produce even more Reloaded goodness.
Secret Super Coder
After ten plus years running a software company, you cannot help but perfect the art of delegation, and sometimes it pays dividends. One such delegation brokered the assistance of a top coder, who kindly volunteered his time to help me spruce up the internal version of DBP, starting with the compiler, in anticipation of larger DBP projects such as Reloaded.
The fantastic news is that an early version of the new compiler has taken my compile time from 3 minutes down to 45 seconds, which is an immense speed boost when you are typically compiling about 100 times a day minimum. My few hours this weekend in integrating this new version with the Reloaded dev set-up will save me literally days of time over the course of the remainder of this project! Those acquired days can be fed back into the engine to make it even better and make my coding life sane once again.
Before every DBP coder starts jumping up and down, the compiler is far from finished and the 'super secret top coder' will want paying for his efforts too, so it will not be a free update. My priority remains Reloaded all the way through to October, and all forces will be steadily employed in this direction, but along the way if the opportunity exists to commercialise these internal DBP enhancements, you will be the first to know!
A Note On Concurrency
The term 'concurrency' is often used to describe how many cores your app is using at the same time (i.e. running concurrently side by side). For fun I draw a few rooms and dropped a light into the map editor of the current Reloaded product. I then opened the performance profiler which indicates what my four cores where doing. Guess what. Processor utilisation was 100% from the moment the light was placed through to when the light mapping appeared. That means all four cores where red hot to get the light mapping done as quickly as possible on the monster machine it finds itself on. Intel would be very proud of an app like Reloaded, and for the first time FPSC will be tapping every corner of your processing power even as you edit and tweak your levels.
I also plan to get the same (or near) when you play the game too, thanks to the multi-core DarkAI system which will be asked to calculate some fiendish math to figure out the best way to strike at the player. Getting full concurrency when you edit and when you play will mean we have a piece of software that pushes your machine to the maximum.
Signing Off
It's now 2:30AM and I will be turning in very soon. I'm happy with this coding stint, not least because it has just improved my life by 2 minutes 15 seconds per compile and once I have stripped out the old 'build process' and 'blue print shroud system', the code base will be even smaller which means even faster compiles. That is, until I add the new bullet physics system, terrain system and combat intelligence systems the likes of which you'll have nightmares about. A big player in the internal Reloaded team suggested this product needs a killer feature to make it stand out from the crowd. Rick, our resident pessimist[realist], concluded there are no killer ideas left, and everything has been done, and small teams like us don't have the manpower for that level of innovation. I disagree. I think it is precisely because we are small that we can take a risk on a killer idea that might very well transform the FPS genre forever. I don't know what the idea is of course, and I dare say it may never surface, but if you have an awesome idea you've never seen in an FPS, post your comments here, I read them all!
Friday, 12 April 2013
Friday Lights
Getting Real
It is great to be away from a prototype and into the main engine. It has been some time since I really got my hands dirty inside the FPSC source code and it is all coming back to me, and really how complex the whole process of building the map actually is. Lots of considerations that seam completely natural at the front-end need plenty of code to make it a smooth editing process. Of course there are some bits of code that are quite shocking, and are scheduled to be removed and rewritten at some point.
Current Progress
I managed to get segment painting working with adding multiple lights this morning, which was great to see, and even fixed a texture issue which means you can light map the same object over and over.
I was going to make a screen shot for you, but then decided the current code was obsolete as it assumed the objects needed recreating each time something was added. The old system did this using instance objects, but the new system does not require this traversal. Simply adding and removing the object at the point of the event is sufficient, but it meant removing a chunk of legacy code, and as I write this I dun broke it.
No worries though, it has to get worse before it gets better. The new system will mean that as I increase the map size to 200x200x20, the engine does not have to traverse every single map reference to check if a grid tile has been changed (which would have been the case had I kept the old system). The new one will create the object on demand. Right now it is perhaps creating it, but bot showing it in the right place. Going well though!
Signing Off
My hope for the end of Friday is to be able to draw a simple Alley scene comprising segments and static entities, then drop in a few lights and watch as they auto light map. Then to be able to remove both segments and entities and when I add a new light the scene amends itself. I am not aiming to get light mapping to trigger on segment and entity removal just yet, as I want to make sure the "add/remove light" trigger works 100%. Currently having a ball, and it's nice to see FPSC take a few tentative steps into the future :)
It is great to be away from a prototype and into the main engine. It has been some time since I really got my hands dirty inside the FPSC source code and it is all coming back to me, and really how complex the whole process of building the map actually is. Lots of considerations that seam completely natural at the front-end need plenty of code to make it a smooth editing process. Of course there are some bits of code that are quite shocking, and are scheduled to be removed and rewritten at some point.
Current Progress
I managed to get segment painting working with adding multiple lights this morning, which was great to see, and even fixed a texture issue which means you can light map the same object over and over.
I was going to make a screen shot for you, but then decided the current code was obsolete as it assumed the objects needed recreating each time something was added. The old system did this using instance objects, but the new system does not require this traversal. Simply adding and removing the object at the point of the event is sufficient, but it meant removing a chunk of legacy code, and as I write this I dun broke it.
No worries though, it has to get worse before it gets better. The new system will mean that as I increase the map size to 200x200x20, the engine does not have to traverse every single map reference to check if a grid tile has been changed (which would have been the case had I kept the old system). The new one will create the object on demand. Right now it is perhaps creating it, but bot showing it in the right place. Going well though!
Signing Off
My hope for the end of Friday is to be able to draw a simple Alley scene comprising segments and static entities, then drop in a few lights and watch as they auto light map. Then to be able to remove both segments and entities and when I add a new light the scene amends itself. I am not aiming to get light mapping to trigger on segment and entity removal just yet, as I want to make sure the "add/remove light" trigger works 100%. Currently having a ball, and it's nice to see FPSC take a few tentative steps into the future :)
Thursday, 11 April 2013
Thursday 90% Loaded
AM to PM
Another long Reloaded day, with almost all of it within the FPSC source code :) I was distracted for no more than 20 minutes on a quick AGK fix, but right back into the light mapping work! Success.
Current Light Mapper
I have moved from the prototype to the main engine now, and redesigned the way in which the segments and entities are painted in the editor. Not outwardly, but internally the objects are no longer instances but clones so they can be light mapped, and with this I was able to paint down a few floors, drop in a light and presto, an almost instant light mapped ring of light.
After a little more work I could add several lights in, and again the light map would automatically adjust and the scene would be pre-lit within seconds.
As expected, now came the army of small unpredictable gremlins to spoil the party. When you click New to create a new scene, the old objects had to be deleted, and of course light mapping is still occurring so of course the whole thing crashed when I tried to reset the level. I also cannot delete any segments right now as the new system only handles additions to the scene, and of course when I start deleting these objects I will have the same issue as above with the light mapper not quite ready to let the object go.
This might not be a problem if the light map process is a few seconds, but a complex scene can take 10-20 seconds to light, and does the user want to wait that long before they can load a new level or start again, no.
Signing Off
My task for the remaining few hours and perhaps this evening is to bring these crashes to heal by adding a new light mapper command which can extract an object immediately from the scope and responsibility of the process, so one or all objects can be removed and the light map process terminated gracefully. It's also not straight forward as when using threads (which the real time light mapper certainly does), a crash is often reported in a different location with no call stack to fall back on, so it's pretty much a case of trial and error tweaks until the true cause of these drop-outs becomes apparent. Hopefully tomorrow I can bring news of victory over these gremlins.
Another long Reloaded day, with almost all of it within the FPSC source code :) I was distracted for no more than 20 minutes on a quick AGK fix, but right back into the light mapping work! Success.
Current Light Mapper
I have moved from the prototype to the main engine now, and redesigned the way in which the segments and entities are painted in the editor. Not outwardly, but internally the objects are no longer instances but clones so they can be light mapped, and with this I was able to paint down a few floors, drop in a light and presto, an almost instant light mapped ring of light.
After a little more work I could add several lights in, and again the light map would automatically adjust and the scene would be pre-lit within seconds.
As expected, now came the army of small unpredictable gremlins to spoil the party. When you click New to create a new scene, the old objects had to be deleted, and of course light mapping is still occurring so of course the whole thing crashed when I tried to reset the level. I also cannot delete any segments right now as the new system only handles additions to the scene, and of course when I start deleting these objects I will have the same issue as above with the light mapper not quite ready to let the object go.
This might not be a problem if the light map process is a few seconds, but a complex scene can take 10-20 seconds to light, and does the user want to wait that long before they can load a new level or start again, no.
Signing Off
My task for the remaining few hours and perhaps this evening is to bring these crashes to heal by adding a new light mapper command which can extract an object immediately from the scope and responsibility of the process, so one or all objects can be removed and the light map process terminated gracefully. It's also not straight forward as when using threads (which the real time light mapper certainly does), a crash is often reported in a different location with no call stack to fall back on, so it's pretty much a case of trial and error tweaks until the true cause of these drop-outs becomes apparent. Hopefully tomorrow I can bring news of victory over these gremlins.
Wednesday, 10 April 2013
Wednesday Locked And Loaded
And Their Off
Taking my own advice, and that of others, as soon as I logged onto my PC this morning I deliberately avoided checking emails and instead booted the Reloaded PC and started from where I left off.
Where I Left Off
The last thing I did before writing Tuesday's blog was identify where the prototype was crashing, the result of textures being released even though they where still being used by other objects. The trick with lightmaps is that one texture can be shared by hundreds of objects, and not all the objects will be updated with the real time light map each frame. This means you have new light map textures and old not yet affected textures side by side. You cannot just delete a light map texture because ONE object started using a new texture.
The solution which I implemented last night was to create a reference count list of all textures assigned to objects, and then decrease that count when the texture was no longer used by an object. Only when the count reached zero did I now that NO-ONE was using the ageing light map texture and I could remove it. This worked well, and further work ensured there where no leaks and the process could be left running with only minimal fragmentation in memory.
This Mornings Work
The next task was to allow entity creations, movements and deletions to affect the infinilights structure which controls what lights are recorded in the game scene. This structure was not used previously in the map editor part so had to be moved across and made to play nice.
I also added some code which prevented the light mapper from saving out the light map files as we only really need the texture to exist in memory at this stage. The files are only needed during a final standalone export, so removing this saved precious performance.
The final step was to extract the code from the prototype that was to be used in the engine and create a separate code file for it (FPSC-M-RTLightmap.dba) and move the new data structures required by this new feature to the types code file. With this complete, and the prototype still working nicely, it was time to switch over the monster code file, the FPSC engine itself!
The Monster FPSC Engine
After integrating the new code file and inserting four subroutine calls into the map editor part of the code, I compiled, ran and got my first (of many) crashes. Back into the RTLM code file, made a tweak, ran again and no more crash. Added my first light entity, crash. This is where you find me now at 2:30PM. I do have a few hours I need to put into the email and some last minute Android AGK bits, but from now on I think this formulae will work out just fine.
FPSC Compile Speed
Let's face it, three minutes per compile is atrocious, and it will severely hinder the speed at which I integrate the various parts of the engine and of course that all important final testing and tweaking.
I have made steps to improve this in the DBP compiler but cannot reveal more at this stage, but I am confident that I will be able to reduce the compile time significantly in the weeks to come. My priority though is to finish the light mapper with the present compiler so I can reach a milestone that should have been finished over a month ago. Grr.
Lovely New Art
I am sure Rick will not mind me sharing videos that are representations of final artwork, so I have uploaded the Proximity Mine that Mark has kindly created for the project:
It's a proximity mine which you can magnetically place on any surface, and will detonate when anyone gets too close. Combined with the new explosion and in the future augmented with a physics blast that will make you think twice about venturing closer, we're going to have a blast with this new toy!
Signing Off
The extreme compile times of FPSC allow me much spare time to write my blogs so I will probably be writing them in the Morning/Mid-Day as I code Reloaded, and it also means the stuff is being reported while it is fresh in my mind. It's been really great waking up and spending my best most sharpened hours on Reloaded, reminds me of the olden days!
Taking my own advice, and that of others, as soon as I logged onto my PC this morning I deliberately avoided checking emails and instead booted the Reloaded PC and started from where I left off.
Where I Left Off
The last thing I did before writing Tuesday's blog was identify where the prototype was crashing, the result of textures being released even though they where still being used by other objects. The trick with lightmaps is that one texture can be shared by hundreds of objects, and not all the objects will be updated with the real time light map each frame. This means you have new light map textures and old not yet affected textures side by side. You cannot just delete a light map texture because ONE object started using a new texture.
The solution which I implemented last night was to create a reference count list of all textures assigned to objects, and then decrease that count when the texture was no longer used by an object. Only when the count reached zero did I now that NO-ONE was using the ageing light map texture and I could remove it. This worked well, and further work ensured there where no leaks and the process could be left running with only minimal fragmentation in memory.
This Mornings Work
The next task was to allow entity creations, movements and deletions to affect the infinilights structure which controls what lights are recorded in the game scene. This structure was not used previously in the map editor part so had to be moved across and made to play nice.
I also added some code which prevented the light mapper from saving out the light map files as we only really need the texture to exist in memory at this stage. The files are only needed during a final standalone export, so removing this saved precious performance.
The final step was to extract the code from the prototype that was to be used in the engine and create a separate code file for it (FPSC-M-RTLightmap.dba) and move the new data structures required by this new feature to the types code file. With this complete, and the prototype still working nicely, it was time to switch over the monster code file, the FPSC engine itself!
The Monster FPSC Engine
After integrating the new code file and inserting four subroutine calls into the map editor part of the code, I compiled, ran and got my first (of many) crashes. Back into the RTLM code file, made a tweak, ran again and no more crash. Added my first light entity, crash. This is where you find me now at 2:30PM. I do have a few hours I need to put into the email and some last minute Android AGK bits, but from now on I think this formulae will work out just fine.
FPSC Compile Speed
Let's face it, three minutes per compile is atrocious, and it will severely hinder the speed at which I integrate the various parts of the engine and of course that all important final testing and tweaking.
I have made steps to improve this in the DBP compiler but cannot reveal more at this stage, but I am confident that I will be able to reduce the compile time significantly in the weeks to come. My priority though is to finish the light mapper with the present compiler so I can reach a milestone that should have been finished over a month ago. Grr.
Lovely New Art
I am sure Rick will not mind me sharing videos that are representations of final artwork, so I have uploaded the Proximity Mine that Mark has kindly created for the project:
It's a proximity mine which you can magnetically place on any surface, and will detonate when anyone gets too close. Combined with the new explosion and in the future augmented with a physics blast that will make you think twice about venturing closer, we're going to have a blast with this new toy!
Signing Off
The extreme compile times of FPSC allow me much spare time to write my blogs so I will probably be writing them in the Morning/Mid-Day as I code Reloaded, and it also means the stuff is being reported while it is fresh in my mind. It's been really great waking up and spending my best most sharpened hours on Reloaded, reminds me of the olden days!
Tuesday, 9 April 2013
Tuesday Bits
Death By A Thousand Cuts
After a brief phone call with Rick today, I realised that 24 hours, a plate of beans and a round of toast was enough to completely side track my brain into replaying the old mind set of answering emails, dealing with little bits of jobs here and there, and not actually spending a great deal of time on Reloaded.
Yes, all the tasks where important to some degree, but they seemed to automatically get placed on a high priority for the simple fact they cluttered my inbox. Back in the day, we did not have in-boxes or emails for that matter, and a lot of ruthless development got done.
Some Ruthless Development
As of today, I am going to play a game I like to call 'Ruthless Reloaded Development' and see how it pans out. Many worthy causes will be sacrificed this week, but it's an experiment I am curious to see the outcome from. First quick job is to finish the memory management of displaced texture memory from the real time light mapper and get it leak free. From there, I can drop it into the main engine and see what it looks like for performance and visuals.
In other camps, Mark is fired up to produce a proximity mine for our latest munition prototype, and also to begin creation of the new character template for Reloaded which I intend to be very awesome indeed. I might be showing videos for a while, but I think I can show you the cool artwork as it comes in.
We also have our 2D artist producing mock-ups for the new IDE and map editor visuals as well, so hopefully I can show you the various themes we will be choosing from. The idea is to give FPSC a totally new modern look and of course resolve those little interface niggles we have come to know and love.
A Day Of Two Halves
As I woke up at noon today, I have promised myself a few more hours this evening to make up for my lie-in. If I have not cracked the real time light mapping memory leak and added it into the main engine I will be most vexed. I will be able to report my progress on Wednesday's blog and let you know how it went.
Signing Off
For the rest of this week my email inbox will be playing second fiddle to booting up the Reloaded PC and getting stuck in, so I apologise in advance if anyone is waiting around for a reply. Consider me cave-bound!
After a brief phone call with Rick today, I realised that 24 hours, a plate of beans and a round of toast was enough to completely side track my brain into replaying the old mind set of answering emails, dealing with little bits of jobs here and there, and not actually spending a great deal of time on Reloaded.
Yes, all the tasks where important to some degree, but they seemed to automatically get placed on a high priority for the simple fact they cluttered my inbox. Back in the day, we did not have in-boxes or emails for that matter, and a lot of ruthless development got done.
Some Ruthless Development
As of today, I am going to play a game I like to call 'Ruthless Reloaded Development' and see how it pans out. Many worthy causes will be sacrificed this week, but it's an experiment I am curious to see the outcome from. First quick job is to finish the memory management of displaced texture memory from the real time light mapper and get it leak free. From there, I can drop it into the main engine and see what it looks like for performance and visuals.
In other camps, Mark is fired up to produce a proximity mine for our latest munition prototype, and also to begin creation of the new character template for Reloaded which I intend to be very awesome indeed. I might be showing videos for a while, but I think I can show you the cool artwork as it comes in.
We also have our 2D artist producing mock-ups for the new IDE and map editor visuals as well, so hopefully I can show you the various themes we will be choosing from. The idea is to give FPSC a totally new modern look and of course resolve those little interface niggles we have come to know and love.
A Day Of Two Halves
As I woke up at noon today, I have promised myself a few more hours this evening to make up for my lie-in. If I have not cracked the real time light mapping memory leak and added it into the main engine I will be most vexed. I will be able to report my progress on Wednesday's blog and let you know how it went.
Signing Off
For the rest of this week my email inbox will be playing second fiddle to booting up the Reloaded PC and getting stuck in, so I apologise in advance if anyone is waiting around for a reply. Consider me cave-bound!
Monday, 8 April 2013
Monday Magnetica
A Happy Place
After the intensity of juggling several projects and a super intense competition (and no weekends or sleep), it's great to finally be able to sit back and have only one project in front of me, and what better project than FPSC Reloaded!
I had a strategy meeting today to discuss overall plans and directions, and you will be pleased to hear that we are putting more eggs into the Reloaded basket and making sure it really delivers when the time comes.
Magnetic Mines
A new version of the magnetic proximity mines came in today which was great to see. You are still throwing them like grenades, but the mechanism is now there to throw, place, drop, hoist them where ever you please. Mark will be working on a nice graphic so we can see them pulse dangerously and should make a great addition to your arsenal.
Due to recent comments from various social feeds, I have now been banned from showing early video footage of Reloaded prototypes. The decision has been made to only show videos that contain final or near final artwork which reflects what you will get in the finished product.
You will still get rants from me, and pictures, but video production and release has been handed to Rick so he can control the quality here. I often commit the sin of thinking everyone looking at the videos can see past the obviously ancient art and imagine what it could be like, but of course the world does not work that way.
Signing Off
Meeting days always sap my energy so this will be a shorter blog (and day). I also worked to 4AM Sunday to finish my final UCCII blog post and video. I have done all I can bar some final external testing and tweaks to the installer and I hope the app behaves itself in front of the judges.
Here is a taster of what the final blog had in store for it's readers. Often is the case the coolest stuff happens in the last five minutes, and the PerceptuCam app is no exception:
The above idea is the first stab at creating an instant visual cue from the gesture data before allowing the user to summon commands with the wave of a wand. It turns out that even though wands are fictitious story elements, they also make surprisingly good ways to produce accurate gestures! It has also not escaped my thinking that once Reloaded V1 is out the door, adding gesture control to the create and play elements of FPSC is not such a bad idea. Until next time, have a good evening!
After the intensity of juggling several projects and a super intense competition (and no weekends or sleep), it's great to finally be able to sit back and have only one project in front of me, and what better project than FPSC Reloaded!
I had a strategy meeting today to discuss overall plans and directions, and you will be pleased to hear that we are putting more eggs into the Reloaded basket and making sure it really delivers when the time comes.
Magnetic Mines
A new version of the magnetic proximity mines came in today which was great to see. You are still throwing them like grenades, but the mechanism is now there to throw, place, drop, hoist them where ever you please. Mark will be working on a nice graphic so we can see them pulse dangerously and should make a great addition to your arsenal.
Due to recent comments from various social feeds, I have now been banned from showing early video footage of Reloaded prototypes. The decision has been made to only show videos that contain final or near final artwork which reflects what you will get in the finished product.
You will still get rants from me, and pictures, but video production and release has been handed to Rick so he can control the quality here. I often commit the sin of thinking everyone looking at the videos can see past the obviously ancient art and imagine what it could be like, but of course the world does not work that way.
Signing Off
Meeting days always sap my energy so this will be a shorter blog (and day). I also worked to 4AM Sunday to finish my final UCCII blog post and video. I have done all I can bar some final external testing and tweaks to the installer and I hope the app behaves itself in front of the judges.
Here is a taster of what the final blog had in store for it's readers. Often is the case the coolest stuff happens in the last five minutes, and the PerceptuCam app is no exception:
The above idea is the first stab at creating an instant visual cue from the gesture data before allowing the user to summon commands with the wave of a wand. It turns out that even though wands are fictitious story elements, they also make surprisingly good ways to produce accurate gestures! It has also not escaped my thinking that once Reloaded V1 is out the door, adding gesture control to the create and play elements of FPSC is not such a bad idea. Until next time, have a good evening!
Subscribe to:
Posts (Atom)