All ideas expressed in this blog are temporal and any resemblance to manifested reality are purely co-incidental. The forgoing is a casual musing from a fellow coder, not an iron clad promise of features and products - The Law
There I Was Yes indeed, there I was, minding my own business when Mark submits an email of intense significance to the Reloaded project. He only went and re-wrote my shaders, and replaced my textures for the Terrain Demo. Music to my ears it was, and upon running the video he prepared and the new prototype in dropbox, I was not disappointed. In the space of a few hours, Mark transformed my prototype into a drab unlit world into a scorching explosion of sharp contrast and infinite detail. He rescued a texture stage from my terrain shader and made the result look even better than it did before. Now that's talent ladies and gentlemen! And Now A Visual Oh, that's right, we're on a visual ban aren't we? :) Ah well, patience is a virtue, or so they say. That is, if you believe 'them', whoever they are. Anyhoo, I can confirm it looks darn good and I have a video and a screenshot for the newsletter so all is well. Sorry for the visual black-out. I will try to convince Rick at the meeting on Thursday that SOME visuals would be okay for the blog, just not horrible ones that squarely belong in the programmers cassette box. The Trouble With Physics In fact, there is no trouble, whatsoever. The bullet physics wrapping is going very well, and resembles ODE in many ways in the way everything is set up, created, released, e.t.c. I am currently toying with getting static terrain into a simple DBP prototype. I already have static and dynamic cubes and spheres, and thanks to a reminder from the blog comments section, I even have a dynamic cylinder! We never managed to get good cylinder physics in FPSC Classic, so their introduction in Reloaded will be a great plus. Always wanted to roll a barrel down some stairs, then strafe bullets at it half way down and have it explode amongst my enemies, who just that moment started up the stairs. It's the little things that make you smile!
No doubt, largely thanks to precious little examples on the subject, I will be working on static terrain for a while, but once I've overcome that I am onto Character Controllers and that's where it gets interesting for me. More on this later in the week, as I hope to cram some work in Thursday evening when the meeting is over. Signing Off It makes sense to make this one an early night which makes a shorter day than usual, but I can make up for it Friday I think. Hopefully Friday's blog will reveal something you can use, and push the boundaries of what we've been used to so far with FPSC. A few more notes before I resume terrain physics fun, our AGK V2 Kickstarter is starting i's last 10 days run-down to get to that mythic 1000 pledger's. It's a good wholesome number to get to, and we're tantalizingly close. I recommend everyone pledge another £3 (or get their friend to do it) so we can push it over the edge and watch the wonderful splash when it hits the sea (sorry, my mind is still in terrain mode).
I also wanted to provide a link to a great write-up of the AGK V2 effort, in the form of a review of our Kickstarter from Gadget Girl. Great to see such enthusiasm, and a real kick to see the photos of the old products and the stack of TGC inspired books! Here is the link for your delectation: http://www.gadgetgirlreviews.com/2013/07/app-game-kit-v2-calling-all-budding.html Oh, and before I forget, the current AGK V2 total (with TGC Paypal pledges) comes to a rocketing £27,918 and an amazing 770 backers. Good luck to that project, and wish me luck with terrain physics. It should be straight-forward, but I don't have sufficient documentation for half of the required parameters, but that's life for a developer, right? ;)
Box, Meet Sphere I have a sphere which falls to a static box and slowly rolls away. I know it was not the most ground breaking start to my physics system masterpiece but it's a start :) Most of the day in fact was spent re factoring the terrain prototype so it would integrate much more easily with the engine, which in real terms means an almost complete rewrite. The data structures and subroutines had to be separated out, cleaned up and commented, the resources such as objects, images, shaders and cameras had to be accounted for and externally controllable (as it would integrate with an engine which already uses many DBP resource indices) and finally I had to move the files around and remove old ones, and place new ones in clever places for future proofing. Test the whole thing to make sure nothing got broke in the conversion too! Finally, I moved the media files and the newly created source files to the main engine, in such a way that I can still load and compile the terrain prototype as a standalone, but the types and main module can now be used in the main engine compile as well. I stopped short of integrating the IDE (that is, seeing the terrain in the map editor window and linking in the controls) as I had an appointment with physics I did not want to delay. Integration Delay It might be a mistake postponing integration in favor of starting the physics module but there are many more integration modules to come and it might make sense to do them all at once. The task is also quite involved and I should really treat it as a project in itself and give it the proper amount of time, not squeeze it in between modules. I have done my re-factoring correctly, which is the main thing, and I can still compile and run the terrain module proto as a standalone which means the new art which Mark is burning overtime on can be tested and dropped in when they are ready. Signing Off As you may have read from yesterday's blog comments, it appears Rick discovered a secret version of the terrain module on dropbox I was using to work with Mark on the final terrain texture sets. Seems he liked it. You might think that means you get a screen shot but I rather like the idea of a big reveal through the newsletter and the Friday blog. Something to start your weekend with a little bang. Who knows, maybe it will get better in the next few days and you do want some WOW factor out of this project right? I just hope I've not built the suspense too much, worst thing you can do that! Wednesday's plan is a bit scatter gun, with some art integration's, more physics module work, new FPSC animated sprite pack to create an installer around, get rid of some office clutter so I can see the chairs again, create some GITHUB repositories and see if they all compile for me, prepare presentation notes for Thursday AM and who knows what else. All in all, pretty happy with today's run of work.
More Terrain Time Today I get a few more days terrain time thanks to a postponed meeting, which was really supposed to be day one of my Physics work. I just could not resist continuing to add the eye candy and get the demo 'just so' before moving too far off my present mark. Monday I finished the water shader, water effects code and added a better sky box texture to give a high definition feel to the whole scene. I then must have spent two hours just moping about making terrain, cutting out little rivers, playing with my height and paint editing tools to see what kind of landscapes I could come up with. During my 'fun' my mind raced about more editing tools I could implement, more refinements to the shaders, optimizations that fit the way I did some things and a general need for more plants, rocks and bits of scenery detail. I think the screen shot I have not is pretty darn good, but I am going to stick to Rick's guns and refrain from putting any 'suspicious' art out there :) Only a few more days to wait, and I think we will be using a 'terrain shot' for the next TGC newsletter so there will be no doubt you will see all this soon enough. Details Details It might seem adding water and new sky was a relatively trivial affair, but the very subtle code I added attempted to best some of the tricks done in high rolling games. Little things like a subtle fog effect, not to cull distant objects but to provide a misty sense of distance (and I mean VERY subtle). Another trick. courtesy of Evolved, is to use a water mask texture to describe the edges of a lake or body of water. This ensures the water does not look strange when bumping up against solid geometry and is a great substitute for a depth read. I also had a few ideas where I could hijack the other channels of the water mask to describe depth, and use a nifty pixel shader calculation to shade the refraction map darker as the water gets deeper, like you would see in the real world. Alas these are all what I call 'nice to have' B list items and should not get in the way of my goals on this occasion. No Blooming I was tempted to make the 'maybe awesome' demo use post processing effects such as bloom, depth of field and haze effects (for the desert heat) but again for the above reason refrained. These can be added later, and are not what we all agree are 'essential features' of the engine. Hopefully my terrain demo now stands on it's own without the smudging effects of post processing. Next Things Now I feel the demo stands alone for a presentation, I can move onto more serious matters such as re-coding and cleaning the prototype to use a data structure and resource numbering set that can drop right into the main engine and map editor IDE. A terrain editor is no use as a prototype! Hopefully this task won't take too long, and the subsequent introduction to the IDE will go smoothly if I do this preliminary step correctly. Once the above is done, I will be free to resurrect the Bullet Physics prototype I very quickly knocked up before my holiday. It attempts to operate a 'character controller' walking over a 'height map', which is actually quite ideal as I could drop it into the present terrain module. From there, which is no small trick to get all that set up and running properly, is to then add a few dynamic and static objects to test performance and stability of the floor. We don't want a repeat of FPSC Classic with enemy characters suddenly disappearing through the floor and into infinity! I am hopeful the new Bullet physics system will provide many wonderful toys to Reloaded in time, and if I can get the basics working great, the rest will follow smoothly. Signing Off My schedule puts this week down as pure Physics coding, so the sooner I can get off terrain and onto physics, the sooner I can get them both integrated and start the inevitable tweaks to ensure they all play nicely. I am quite looking forward to creating a sphere and rolling it down a terrain hill and watch it hit the water and start bobbing about, then cast some wind around and see it subtly shift position as it islands it's way around the pond.
Terrain Save, Load and Random Generate Another good (but hot) night in the coders dungeon. At night I have to switch off the blessed air-con as the open window lets all the moths in. After 10PM the office gets hotter and hotter and hotter this time of year! It's now 3AM and I'm practically in a sauna, sweating like a coder without a compiler and hoping this blog entry is nice and succinct so I can go to bed. Some good news in that the Monday meeting has been pushed to Thursday which gives me much more time to add some polish and even start the Physics prototype next week! The downside of course is that without a demo for management to approve, there will be no pictures or videos now until next Friday. If you want this policy to change, you know Rick's email ;) Terrain Saver and Loader It was key to be able to save all edits performed and reload the terrain in the exact same state to resume editing and of course to load in when the level requires it. A few hitches though. One bug one is that when I construct the height data manually in the terrain module, I drop from 200 fps down to 30 fps for some unfathomable reason. I have yet to dive into the source code of the module and might just be something silly, but it has me a little vexed. I also found that converting the terrain height data to a height map image, and then feeding that back through the loader created import distortion of some kind which would slowly (but subtly) change the terrain heights each time you saved and loaded. Not a good idea as your terrain would have washed away after the 100th save :) I am instead directly storing the heights in a Dark Basic Pro memblock file so I can keep a very accurate eye on this most valuable data. I also save the veg shadow map too, this time as a simple image and map it back to the system when I restart the prototype. It works just great. Holy Random Terrains Batman! A tickbox on my schedule was the successful generation of a believable 'random' terrain as part of the Reloaded terrain editing suite. I was certain many users would not want to define every hill, dip, lake, bump, slope and mountain over a 50,000 x 50,000 area. They would want a single button to make one up, and then allow some editing in the part of interest. To that end I borrowed inspiration from our friendly Dark Basic Pro forum, and an entry into our 20 line challenge which is a bit of fun which tasks any programmer to write something useful in under 20 lines of code. It's great to see the amazing things coders can do in such a small space and in this case, someone produced a terrain generator in 20 lines! The name of this DBP star is Lewis999 who adapted a Diamond-Square algorithm to populate a matrix with realistic height data. Using the same technique, I generated pure height data using the same idea and applied it to the terrain constructor. I then painted textures based on distance from the water level, and used smoothing to create a nice transition. I also painted a band of grass at the point I felt grass would ideally grow (not too high in the mountains, not underwater or on the shore). You can still paint all this manually, but the terrain generator starts off as an automated process. Finally I played with the grass sizes to create a more dynamic and rich veg experience, and concluded that a few more grass and vegetation models and textures would really lift my terrain visual. Something to add to Mark's list when he's not looking! Is Something Missing? Running around my newly generated terrain for a while, as good as it was, there was something missing. Naturally there is no game, so no enemies, no buildings, no game objects, no nout of course. Still, you can have a convincing terrain without all the above, right? As I am making a deliberate effort not to add trees (as that is for another pack and another day), I wondered what else I could add. Perhaps a few cactus plants models when/if they are created, sure, but I need something more substantial! Is it a bird, is it a plane, no it's a Rock! Sure enough, what my bleak terrain level needed was rocks, and lots of them! Not the little pebbles that can be baked into the floor texture but large ugly rocks left after the last glacier strolled through the land. From my extensive terrain editing (playing) I knew how to create a rock by making a sharp raised land, then squashing the top bit, then adding a rock texture and surrounding it with grass. Given that knowledge, I proceeded to add a system which would strategically add a rock, and not just a single rock but what I term a rock clump. These clumps comprise of about 15 rocks in close proximity and they really do look natural when sprawled over the floor. The system then moves somewhere else in the land and makes another clump, and so on until there are enough. The terrain generator is pretty good for a V1 now, but it does take a while (30 seconds for 150 rocks) and the final level would probably need a few thousand of them. The good news is that once the rocks are in the land, you can save and load the whole terrain within seconds :)
Signing Off So that's the week, and apart from IDE integration, all my terrain objectives have been met on schedule which I am pleased about. Still needs spit and polish of course, but we're well on the way now. It also occurs to me that our AI system will need to know about height map terrain so enemies can work out a path through cliffs, valleys, slopes, work out line of sight, know what height advantage means, e.t.c. I have already started a list :)
I plan to use the weekend to quickly add shadow generation to the prototype, experimenting with things like burning the normals as shadows in the texture to improve LOD transitions, calculating when a large hill or rock casts a shadow onto the terrain and the big one, rendering the entire 'outdoor' scene of a level such that I can burn the resulting shadows into the veg shadow map too. This of course means we cannot have a moving sun for day and night cycles, but adding cascade shadow mapping to the terrain proved too costly and unless there is a performance friendly solution out there I have decided to aim for quality visuals rather than a day / night cycle in V1. I should have plenty of time to work on my shaders, and it won't be long before a solution pops into my head which allows us to have our cake and eat it. As is my custom, no blog entries this weekend so I can be free to do nothing, or burn the candle at both ends. I was going to sneak out a screenshot of the latest progress, but I resisted (Rick will be proud). See ya Monday!
Veg Improvements Thanks to an urgent rush on a veg model, some tinkering and tweaking, the terrain module how has a better vegetation system which looks better, does not overhang cliffs, fits better on the ground and is faster for performance too. Terrain Edit Cursor Highlighter Thanks to the fact I am using a shader for my terrain, and I no longer need to save four texture stages for cascade shadow mapping, I have added three more diffuse texture options and a cursor highlighter texture as the last texture stage. Using some pokery, I am able to have the cursor render at the mouse coordinate during editing, and also reflect the radius of the brush which is now under shortcut control. It looks pretty snazzy too, as though a glowing light of creation spawns terrain from it's terrible orb of power :) More Shader Fun To combine the different textures coming into the terrain, I am using a control map (the veg shader map I mentioned in previous blogs) and rather that gobble up my useful channels, I can specify which of the five diffuse textures from a single Blue channel of the control map. Here some code which does it: float texselectorV = tex2D(NormalSampler,IN.TexCoord/500.0f).b; float texselectorcol1 = max(0,0.25f-abs(texselectorV-0.00))*4.0f; float texselectorcol2 = max(0,0.25f-abs(texselectorV-0.25))*4.0f; float texselectorcol3 = max(0,0.25f-abs(texselectorV-0.50))*4.0f; float texselectorcol4 = max(0,0.25f-abs(texselectorV-0.75))*4.0f; float texselectorcol5 = max(0,0.25f-abs(texselectorV-1.00))*4.0f; float3 textpart1 = tex2D(DiffuseSampler1,IN.TexCoord).xyz*texselectorcol1; float3 textpart2 = tex2D(DiffuseSampler2,IN.TexCoord).xyz*texselectorcol2; float3 textpart3 = tex2D(DiffuseSampler3,IN.TexCoord).xyz*texselectorcol3; float3 textpart4 = tex2D(DiffuseSampler4,IN.TexCoord).xyz*texselectorcol4; float3 textpart5 = tex2D(DiffuseSampler5,IN.TexCoord).xyz*texselectorcol5;
Neat I know, but when you have limited resources you need to get creative with what you have. Old school 8 bit experience that one! I've also been impressed with the raw power of the GPU when running shaders. Almost all my bottlenecks have come from poor coding or some intense CPU activity, which is very encouraging for all the wonderful ideas one can perform through a shader! Signing Off The plan is to carry on and get the prototype so I can paint to the 'veg shader control map' by writing the texture directly from my program (performance alert) which can then reflect in the textures used by the terrain in real-time. This will lead onto being able to paint the veg color and at the same time the actual veg data which the vegetation system will use to populate the terrain. Right now the veg is being used for the entire terrain, which works fine and fast, but I dare say not everyone will want veg everywhere! After that, I will add some quick 'randomize terrain' functionality and perhaps some 'save what I have done' code to make sure this edit can be reloaded and continued. The final step will then be to add it to the main FPSC IDE which is what I hope will occupy my Friday and have the weekend as an overflow buffer before the meeting on Monday. I still have to play S.T.A.L.K.E.R (deferred renderer) which is still in it's shrink wrap :)
Terrain Takes A True Shape An amazingly good day today, so much that it's one of those days you forgot what you had 24 hours prior to the very moment you think back. I started a new prototype to collect together the terrain painter and the terrain shader, so that I could have a relatively blank canvas to try my new technique ideas. Thanks to the Blitzwerks Terrain upgrade, vertex normals now provide some great data to my newly concocted terrain shader which applied nicely without a fuss to the terrain geometry. I also got some super terrain textures from Mark and my shader experiments started right away. The trick is to get it looking good from the distance and from up close, which meant choosing the right textures, applying the right amount of scale, selecting a good detail map, balancing the effect of various lighting calculations and a good deal of trial and error. Finally I got something that looked pretty cool. Being able to paint land and see it rise out of the water was a real treat, and I even applied a nice trick which automatically map rocky texture to the upward slopes of the terrain and sand on the flat bits. Coupled with the normal shading, the whole thing was quite addictive during testing and I found myself drawing all manner of terrain islands :) Bringing Forth Mr Vegetable I was going to add some really nice reflective/refractive water which I realised that my schedule did not call for water, but DID call for vegetation effects. I already have a water shader waiting in FPSC engine so the smart move was to avoid the easy eye fest and opt to solve the veg issue. Thankfully I already had some preliminary desert vegetation and a rather nice shader from Mark, so was able to get started adding veg into the prototype quickly. For those who don't know, veg is my way of describing the grasses that rise out of the terrain to soften the hard surface of the floor and provide some nice decoration in what might be a rather barren landscape. Mark's shader also provided some soft movement and colour offsetting as the wind blows through the grass, very nice! All well and good rendering some veg, but I needed enough to cover an area 50,000 x 50,000 units. What I needed was a routine to create rolling vegetation based on the players position. After a few hours, this was created by allocating 500x500 grid squares a single batch of veg polygons, calculated in real-time to be placed on the terrain. By spacing out when these objects are created, I can keep performance even. By fading them out at the far distance, I can introduce them slowly into the scene to make some great veg LOD. I then had the neat idea of creating a new texture 4096x4096 (the biggest I dare support for Reloaded) which stores what I call a 'veg shadow map' which uses the separate R,G,B channels for information to both the terrain and the veg shader. Right now I have RED control the paint which is placed on the terrain where any veg is to be located. This is cool in that when the veg fades out, this colour remains to fool the user that distant patches of grass colour is grass, when in fact it's just painted to the terrain. The GREEN is being used as a shadow mask, which allows me to darken any area of the terrain as a separate process. My hope is that I can coax the cascade shadow map to draw into this channel and provide a 'sort of baked' full universe shadow to apply to the terrain. I can also use the exact same texture and channel to shadow the grass in the exact same way and position, creating unity between the two shaders. In developing the veg, I realised that the cascade shadow mapping would be too performance intensive to apply to everything, so the 'painted veg shadow map' will be my very nice looking fall-back. I also realised that terrain normals maps get lost in everything I am rendering to the floor right now, and also the fact I needed the texture slot for something else. Now I have removed the need for cascade shadow maps on the terrain, I have freed up four texture stages which means I could bring back normals, but I think I might opt for something better like more texture paint choices and use the extra channels of the veg shadow map to control which texture gets selected for terrain painting. Before these experiments can begin, I still need to add some 'editing' controls so I can paint the veg and shadows as part of the terrain modification system. The Question Of Art Looks like my Tuesday blog got quite a few comments there ;) Don't be too hard on Rick, he's doing the job he's been assigned for this project, and if my horrid screenshots are going to disrupt the marketing job he has to do later then a cease and desist order is probably the wisest course. It will be hard as I do like to gabble on as you know, but perhaps it will force me to insert better art in my prototypes just in case a screenshot happens to escape :) Some good news is that I have a meeting on Monday morning, showing off the progress of the last two weeks terrain work. If the feedback from that meeting are positive, I will post some screenshots and maybe a video of the demo. It's internal name is called 'awesome demo', but that might be a tall order as I don't have the final graphics, no post process shaders, no real scene content and only a few days left to code it all. Signing Off A happy day all told. Even a 3PM appointment with the Dentist today did not dampen my spirits, and apart from the infernal heat, flies, moths and other creatures which insist on tupping me all evening, it's a day that deserves a tick. At some point I want to get the whole shabang into the IDE, but I think before this I need to complete the whole range of 'painting controls', find out how to burn the cascade shadow into the veg shadow map and orient and align the prototype and code so that it can drop smoothly into the main FPSC Reloaded engine without too many compiles. More good news is that my weekend looks free of social engagements which means I can dedicate those days to more terrain work, which means I might find time to add some of that 'oh so tasty' eye candy you can only get from a lovely water shader ;)
Some Shader Success You will never find a more challenging maze that is the shader coders daily life. Every little dust of wind will affect the output of your shader, and when you are in experimental shader writing mode the problems magnify beyond belief. I spent most of the day in a pitch battle against thin air for the most part, unable to make contact with anything and not even knowing whether I was winning or not. Two hours before writing this text I decided to go right back to first principals and get something small working and work my way forwards. The technique worked and I can now sigh a little with my terrain now accepting a shader shadow and rendering alongside my other shaders. Phew! Mark Art After my green and white splodge screenshot I have now been banned from posting any more 'developer art'. The idea is that as we get closer to the release, this blog becomes part of the product marketing and we don't want any 'dislikes' or 'misunderstandings' over the final quality of the project. To that end, I will only post prepared artwork from our esteemed artist Mr Blosser and Mr Peter. Here is one such sample, a render of some rock which I will attempt to coax into my new terrain shader very soon.
Normal Success Perhaps I celebrate prematurely but I just got an email from the Terrain Module author with some new source code which solves the normal issues I was experiencing. Fingers crossed that everything goes smoothly this evening and we can get normals updating as the height changes. After my early shader work, I have concluded the normals are vital to the success of our terrain so I am very nervous about checking out the new code! Signing Off I would have shown you a sneak peek at the 'decent demo' with a terrain background plus shadowing, but it's not top draw so you will never see that version of it :) Once we get something stunning, and our QA guy approves it, just maybe, you can have a sneak peek at Reloaded terrain. More work due this evening, but looking at the clock I seem to have done an 8 hour stint without a break and currently light-headed. Some food, then more lovely coding!!
At The Beginning I have decided to do another of my rolling blogs, which time stamps my day to give you fledgling and ageing developers alike an idea of the pace of life in front of a coders keyboard. My day started about 12:30, around noon, and greeted me with a small mountain of emails and paper correspondence which was quickly opened and discarded. Same too in most part to the emails which now split into Social and Promotional categories thanks to Google. Much easier to mass delete both sections and increase the working week I found. With the handful of 'relevant' emails remaining I fired off suitably brief replies all round and began my day. 1:20PM : Character Tweaks An email containing feedback on my quick character proto required a few tweaks to speed up the character animations and modify the animation set selection so that diagonal forward and backward walking uses traditional forward and backward animations rather than sidestep animations. The result was a marked improvement and sent back to source for further study. 2:00PM : Emails' Gone I finally obliterated the final email from my inbox, and it's now terrain time. The last thing I did was a quick and dirty terrain painter with solid colours. It really needs a texture there, and it also needs to fade into other textures on the terrain also. This will be my next task, and I feel a shader coming on! 5:43PM : Texture Splatter After some scrolling through the source code of Blitzwerks Terrain, it became apparent that although it does support shaders internally, I could not find the commands to apply a shader externally, nor did I feel confident to modify the source sufficiently to allow full shader support in the time available to me. I had to think out of the box 'again' and created a strange yet working system which creates a camera and points it at a large textured plane containing the current version of the terrain 'paint map'. I then splat some quads on the plane as I draw, and then the final camera image is sent to the terrain system as the terrain texture. I then combine the quad blobs into the main terrain texture source image by creating a bitmap and transferring the resulting camera image to a texture which in turn textures the original large textured plane. In effect, this cyclic process allows me to draw quickly to the terrain using texture splatters, then combine in real-time the results of those drawings to the original terrain texture image. This technique also allows me to float a highlighter over the terrain for more accurate terrain editing, and I can save out the final terrain texture at any time (ideally when the level is saved). Here is the current terrain map after all my experiments up to this point:
The downside is that the step which combines the texture splats to the new final terrain texture image takes about 1 second, so the editing sequence will need to account for this, perhaps as a 'commit' button. The advantage of a commit button is that any editing during that sequence can be reversed or entirely discarded without penalty which might prove a good feature. It's also apparent to me that I need shadows here. Raising terrain (currently combined with the painter) offers little clue as height until you scroll across said terrain. Alas without the ability to override the terrain renderer with a shader, I cannot add the shadow mapping extras needed to pull from a cascade shadow map source. I am left once again to think out of the box. 6:00PM : Small Shadows I tried a few height maps to get a better canvas to continue my painting experiments and realised (rather belatedly) that the terrain employs the use of normals in the geometry which are aligned when the height map is first created, which in plain speak means I DO have a method of creating basic shadows (of a kind) on the terrain. Enough of a kind to indicate to the level designer exactly where the lumps and bumps are located as they edit.
Alas there does not seem to be a command to generate the new normals once the height data has been changed, nor low level commands to edit the data myself. I have thus decided to 'enhance' the raise terrain command and modify the normal data as the position vertices are changed. This will add some performance hit to the overall procedure, but it will be well worth it I suspect. 8:25PM : Stubborn Normals I have tried four ways now to update the normals associated with the edited vertices. The terrain module is divided between two DLL blocks which means access to ALL functions and members of the feature are not readily available. It is also clear that normal generation was only meant to be done during the build step so extricating the required code from the QuadMap structure is proving to be less than straight forward. Couple this with the fact I've coded non-stop and am now very hungry makes for a frustrating time all round. My latest attempt involves re-using the normal calculation performed when the terrain is generated, but this brings the FPS down to 11 though 'seems' to generate correct normals of the original height data, just not my edited data. I suspect I am acting on the wrong height/vertex data here and some more investigation could see correct normals from height edits a reality. The extremely slow update is a concern, but once I have it running I can look at ways to localise the update to speed things up. I think the smart move now is to eat, and return later with a fresh head on my shoulders in the hopes of cracking this before the day is done. 12:22PM : Darn It Many hours later, I can conclude the current method in the module for generating normals is so intertwined with the origination of the terrain from a height map texture, to add the normal commands would equate to writing substantial new chunks of code. Such work requires a very good understanding of the entire system, not just a small window into it. The bitter conclusion for this evening is that I cannot affect the normals based on height changes. The height modification is only being made at the very end, the vertex data stage, and does not carry back in any way to the origination data. Therefore using the existing normal generation system is out of the question unless I can affect the core height data, which of course leads me deeper into the rabbit hole!
Signing Off And there you go, sometimes you get good days and others, bad days. Some days are both, and at least I managed to add texture painting and highlighting the terrain cursor on this day. I only hope Tuesday brings good fortune and some new ideas. I am now going to take me and my massive headache to bed in the hopes of slumber born inspiration!
Perceptual Computing Gets An Extra Perspective Just a small note to say that an interview I gave for Intel has transformed into an in-depth case study on the trials and triumphs of Perceptual Computing development, and for those with more reading time available, you can discover this freshly released article here: http://software.intel.com/en-us/articles/perceptucam-case-study
Painting and Textures I got up early today to get back into the terrain stuff, which today was experiments into terrain painting. Here is a quick and dirty video of where we are right now:
Be aware this video is scratch graphics only and does not contain any final artwork or editing controls. It is a technology demo to quickly test the editing capabilities of the terrain system and gain a performance metric. I added a new parameter to Blitzwerks Terrain to cap the height of newly created land mass based on the layer/zoom height of the camera. This allows floors and outcrops to be created easily. As an experiment I also added some basic grass colouring to the terrain affected to see what I might be needing when it comes to applying real textures. It turns out that even a basic colour can look good when multiplied by a detail map. That said, one of my objectives is to produce a high quality up close texture result on the terrain and I don't think blob colours are going to cut it for Reloaded. The good news is that both height and texture can be edited in real-time and it's a great starting point for the shader I will need to write for the next phase of this component. Signing Off As you might guess from my early rise and my early blog, it's an early knock for me today as jump in the car for a nice long drive. Join me again on Monday's blog when we test out the terrains ability to accept a shader and some real textures for up-close editing.
After The Rain, Sunshine Another good day of development and a decision of sorts. My day started by reading a recent paper on a technique called CDLOD terrain (Continuous Distance Level Of Detail Terrain) which basically uses the GPU with Shader Model 3.0 to blend the vertex geometry so that successively lower polygon meshes can be substituted seamlessly in the scene. As this technique uses texture lookups in the vertex shader, it is quite hungry on cards prior to SM3.0 and perfect for the newer generation of cards which is why I liked it so much. The technique produces almost perfect and undetectable LOD transitions, is very fast with almost no CPU hit and quite memory efficient with the possibility of streaming the terrain data in and upgrading it to DX11 tessellation in the future. All sounds good don't it? The downside is that to learn the technique, modify the same code to fit in as a DBP module, work out the kinks to make it stable and usable for the Reloaded project then run it through a battery of usage scenarios would take much more than the time I have left for terrain work. Step Forward Our Hero Once I had accepted the fact that my plan A was too ambitious and risky, I once again looked at the solution that was sitting right in front of me screaming to be heard. After only a few minutes of tinkering, I started to realise just how powerful and complete the Blitzwerks Terrain module for Dark Basic Pro actually was.
Not only can you create extremely large terrain in memory, the automated LOD system works seamlessly, the whole system runs very fast and as a massive bonus I discovered half way through the day that it also contains a huge array of real-time terrain modifier commands. The same commands I was going to write myself until I found them in the documentation, which again was top draw and a hidden gem of this module. The author very kindly loaned me the entire source code to the entire project, and with it I was able to quickly step through and get comfortable with the various components. This now gives me confidence that should I need to add or fix anything, it's there waiting for me. It all compiled first time and turned out to be a very well written set of projects despite a warning that it was a bit 'bloated'. If this code is bloated, I dare not imagine what it would look like streamlined! My universe was back on solid ground and the power of terrain was bristling off my trembling fingers! And Then There Was A Big Hole With no time to lose I started re-shaping one of the BT2 examples along the lines Reloaded would require. A terrain that stretches 50,000 units north and east. A top down camera perspective and of course, a set of basic terrain editing features such as adding and subtracting terrain heights using a brush size based on zoom distance from the terrain to the camera viewer. I also added a cheap and quick water table plane so I could define the land masses from that part of the terrain underwater.
At the moment all the editing controls are super crude but the objective at this stage is to get all my ducks in a row, which means get each functionality in place. We have height modification, but we also need tighter terrain height controls so I can create flat (yet blended) areas, specific shape stamps, layer level control and who knows what else. I also need to make massive strides into the texture side of things. I want a highlighter actually part of the terrain texture, texture splatting, some sort of dynamic texture renderer based on the player position so I can do LOD for the texture to compliment the geometry LOD system. Things like the water shader and other such eye candy might also be worth doing just to please those certain mentioning no names people who get giddy at the sight of gorgeous pixels. I have some terrain textures being brewed up a week from now so hopefully the whole thing should jump in quality as we move forwards. There is also things like geometry vegetation, terrain shadows, maybe some depth normal editing and more ideas to try out.
I also added a first person perspective to the terrain prototype so I could run around my finished creation and see what it looks like from the players point of view. It's here you realise that I will need some sophisticated terrain height editing brushes to get the kind of results most users would expect from their landscapes. Huge smooth spherical gouges ain't one of them :) Signing Off The next step will be to create some kind of dynamic texture system so that I can paint multiple levels of detail onto the terrain for things like grass, rock and underwater surfaces, but also normal/depth painting so I can slice very subtle details into the very texture (rather than trying to use the geometry). I can then use this functionality to automatically texture the terrain based on height so flattish areas get some grass and steep slopes get some rock. Hopefully the effect will be that painting a high grass layer above a lower grass layer will look like a raised escarpment. See the above video of ice and rock to see the effect of a series of escarpment layers. I am sure users will be taking control of this painting eventually, but for the user who just wants to paint the terrain quickly and start playing their game, an automated terrain painting system is vital.
Rather than feed you another incorrect AGK V2 pledge total, I will simply leave you the link so you can check it out yourself. Maybe Rick will step in and provide the 'real' pledge level: http://www.kickstarter.com/projects/tgc/app-game-kit-v2
And Then Came The Rain As much as Tuesday was fun and productive, Wednesday was muddy and slow in the world of terrain coding. I had a few missions today, mainly to improve the shape of the lower LODs so they don't change shape too much on the horizon line, add multi texture support and improve the distant LOD level as it was too low polygon. I started the day well and completed the first task, which increased my polygon count from 8800 to 13500 for the terrain geometry and dropped my frame rate from 850 to more like 450 fps. At this point I decided to apply the brakes as there is no point continuing with texturing if I'm already loosing a significant amount of performance at the first hurdle. The third task was entirely academic as the solution 'was' to add more polygons but this clearly was not the direction I needed to go in. Calling In The Blitz As a point of comparison, I decided to bring in one of the newer DBP modules from the massive library of extra-features of the Dark Basic language has at it's disposal. This one was called Blitzwerks Terrain and with it, I created a terrain 500x500 with two levels of LOD. When run, the geometry shapes where very well retained across all LOD levels, it already has basic texturing support and the best news of all, it was weighing in at over 3500 fps. Clearly, my own initial effort was not even holding a torch to the terrain system already available via Blitzwerks Terrain. Perhaps this is not surprising given my one day effort was being compared to a highly optimised module that took months to create. I have thus fired off an email to the author of Blitzwerks Terrain in the hope that the source code survived. If possible, I would like to add some extra features to it including a more sophisticated texturing system which will allow me to shade grasses, apply shader effects and other nice touches. Character Meets Code Another task I added to my daily list was the creation of a quick character prototype for Reloaded. The final character model and animation came through last night and I was itching to get it in code to see what it could do.
As I expected, the character performed really great in code and each time I wanted an animation it was documented and ready for to call up. I added some nice touches to the prototype while I was in there such as automatic smoothing from different animation loops, two modes so I could switch seamlessly between patrol and combat stance, eight directional walking in both modes and a few other subtle tricks to make it all feel natural. I also had a quick play with transitioning the background music sound based on the character mode, so when patrolling the sound is calm and when the character is in combat mode, the sound gets more racy. Proper sounds will be sourced and used for the final product, but the video should give you an idea what it might feel like. I especially like now the combat sound fades out slowly as the heart rate falls after the high octane adrenaline rush. Signing Off It's about half eleven now which gives me a few more hours before I have to turn in. Even though my terrain geometry stuff is a little 'off' there is no reason why I cannot code up the real-time texture transition stuff for the terrain as this is quite separate from the polygons that shape the ground. I can also experiment with some other ideas I had too once my terrain texture prototype is up and running. AGK V2 Kickstarter
Taking a look at the latest score, we have pushed past £21K, hurray! It's great to see people pledging on this project, and my new goal is to see the 570 climb up to 600 pledger's :) If you have any coder friends who don't know about the AGK kickstarter, send them this link if you think they might be interested: http://www.kickstarter.com/projects/tgc/app-game-kit-v2
Let There Be Ground Apart from a bit of distraction from some legacy support work (I still get dragged back when I cannot avoid it) the whole day was a Reloaded terrain day and it was great fun.
What you see above is the current state of my terrain prototype, which attempts to do three things. First represent some terrain height reference data 500x500 in size which is the maximum size of a Reloaded level. The second thing is that the terrain employs LOD for polygon creation so that distant terrain features are not given hundreds of thousands of polygons. The third is the whole thing is super fast and using as few polygons as possible and we'll add a forth feature which is that the terrain is completely dynamic, centered on the player and can be scrolled in all directions. The only thing I have not done yet, and planning to do it after writing the blog is to stitch the LOD polygons together so they create a perfectly seamless wireframe from highest LOD to lowest LOD in the far far distance. I've already done this successfully with the first two LOD levels, and it works nicely, so I just need to crack on and add more code. A nice benefit of this technique is that once you reach the end of the defined terrain reference data, the terrain renderer just uses a height of zero for non-reference areas but continues to render the terrain to the distance. Effectively, the terrain does not end from the players perspective, you just run out of meaningful 'designed' terrain heights. A Higher Perspective I noticed when I finished the initial prototype coding that the terrain layout looks remarkably like the shadow cascade approach to LOD levels. The player stands in the centre, just above the terrain, and looking outwards. Given this perspective, the detail in the subsequent LOD levels does not have to be that pretty and in fact the stitching only really needs to happen on the edges you can see from the players perspective. Not sure if that observation converts to an optimisation yet. I did a few quick optimisations already, including only updating the terrain segments when an actual scroll-event takes place. This rescued huge amounts of processing and made things very slick. Currently the prototype takes only 8000 polygons and runs at 2300 fps when idle and 850 fps when scrolling super fast. The vast majority of this is the writing of the terrain to the dynamic vertex buffer so in terms of CPU performance it's pretty good. Legacy Fixes and Groundwork As tempting it was to start terrain coding right away, I had to finish off the bits I left to create the 'Decent Demo' for the meeting a few weeks ago. This involved solving the shadow artefact in the map editor and also solving the issue of shadow Z clash when the light was perpendicular to a surface. I solved it by ensuring the regular lighting calculation modulated the shadow strength so that when the light is behind or 'at dot zero' I don't add any illumination so the shadow Z clash could be seen. The result is a shadow that looks 'just normal' which is exactly what I wanted to see.
As you can see above, by adding the latest shadow work to the instance stamp engine and map editor, as you add segment walls the lighting is applied instantly. Given the satisfaction of editing in this way, I am strongly considering leaving the 'real time light map baker' on standby and go for 100% real-time dynamic everything :) Anyhoo, with those fixes dealt with, when the time comes to integrate the terrain stuff into the map editor, we should be able to see instance stamps and shadows as well without too much fuss. I also need to re-enable things like entities and other markers to the map editor, but that will come soon enough. Right now terrains are the only thing my brain is allowed to imagine. Signing Off A great day all told. A little more coding to stitch up the terrain mesh segments and then I think it's early enough for a nice supper and watch a Netflix movie. That's my developer tip for the day, and probably a good life tip too, always treat yourself from time to time for a job well done, it will encourage a productive life and re-enforce positive action! AGK V2 Kickstarter I'm on the edge of my seat waiting for the total to push past £21K. It's great logging on every day and seeing it creep up :) Here are the stats at midnight:
Back In The Driving Seat Freshly returned from the baking sun and rolling waves, I find my computer desk exactly where I left it, and the Reloaded project crying out for attention. Hopefully I can make up some time in the next few weeks, produce lots of cool things and restore the balance of Reloaded news in the universe. Naturally the weekend was spent dealing with no less than 260 emails, half of which I could not delete. I then had a small mountain of paper mail, lots of personal life to catch up on and of course the final task this morning was to go through the progress of the last two weeks from our Reloaded team. While I was sunning myself, our esteemed artists Mark and Peter have been producing juicy little nibbles for the Reloaded banquette and I can reveal some of the eye candy right for your delectation.
In the above video, you can see the final elements of the character animation being crafted to give a more realistic reaction to getting shot from multiple directions and from multiple stances. Arguably better than ragdoll, it will really help improve the dynamic feeling when in the midst of a Reloaded combat situation. It seems Google Blogger cannot find Google YouTube videos, even when you give it the full exact title of the video so I had to resort to the above. This video is hosted on our parent YouTube Channel in case you wondered.
Here is a shot from our new Quest system which will allow you to place images and text on screen when you chat with a character. It should be some helpful advice or a map to lost treasure ideal for a wondering hero.
The explosions also got a LOT bigger in my absence, with dark cloud flumes punctuated by orange blasts of heat. It's still work in progress, but I can't wait to blow something up when it finally integrates with the main engine! The Two Weeks Ahead My schedule for the next two weeks are pretty gruelling, in that I have to write a complete terrain engine and editing system which integrates with the main map editor system we have so far. My short list is to be able to dynamically create terrain in real-time, with real-time vegetation, edit for terrain height, texture and foliage plus add the ability to walk on the terrain either through physics or a simple ray cast. All this needs to culminate in a demo at the end of the month which demonstrates the ability to select a terrain genre, randomise said terrain, edit it and then run around it. In addition, as the water table system is key to the terrain texturing system, a basic shaded water plane will also be introduced so bodies of water can be simulated too. It's a tall order, but with some careful planning and single minded focus, I should get plenty done by the time I have to show anyone. Signing Off Naturally all I have to report so far is that my backlog is cleared, all Reloaded emails have been sent out and side tasks delegated. I have a few emails to send out after writing this blog, and then I dive deep into the world of generating real-time terrain. Thanks for your patience and allowing me to swan off for two weeks, but I am back now and ready to report the rip roaring rodeo that is Reloaded. AGK V2 Kickstarter Another amazing bit of news that greeted me on my return was the huge success of the AGK kickstarter which we ran before I left for my holiday.
Fear not sports fans, my focus will remain on FPSC Reloaded entirely. This project is in the safe and skilled hands of Paul who has already spearheaded the V108 update and brings amazing new features to this product daily. The kickstarter has already passed three stretch goals and is set to hit a forth with over 3 weeks still to go. The next stretch goal is the inclusion of Bullet Physics in the main command set, which will compliment the augmented 3D commands perfectly. I am personally hoping we hit that one as the combination of 3D and Physics in a cross platform easy to use language would be an almighty boon!
