Large Levels
I have done some DBP Object tests and the way memory is created and deleted, and it is clear to me that whatever scheme I choose the slow fragmentation of memory in the heap is a critical limitation of the system I am currently using to build segment levels.
Even if I use instancing and shared parent geometry, the size of the memory required for the Vertex and Index buffers, plus the reduction in memory over time means there will be a problem sooner or later when it comes to the capabilities of creating 'very large levels'.
I am pretty sure I can get larger levels with the system as it is, but a small voice is telling me that squeezing a little more from the current system might not be the best plan. I don't have a solution, but I have been sketching out some ideas of possible implementations that allows 'extremely large levels' to be constructed.
Even as I contemplate the possibilities, I am acutely aware that such a detour will affect the project timetable. Even though the existing system will allow you to create 200x200x20 terrain levels with lots of buildings and objects, at some point it will 'run out'. And when it does that, it will most likely crash out. I have run four or five different ways to monitor and track the memory usage of the current system (last night), but in each case I had to reject the solution due to either in-accuracy or memory monitored out of context.
I also gained a great insight into all the different elements within the engine that reserve memory and free it, and when they do this. It was clear that operating within this finite world of just 1.8GB to store everything was quite restricting.
The New Approach
If I decide to take a detour, the new system will be based on a system that has a single aim of serving data to the DirectX buffers so you can see what you are looking at. Everything else will be about generating that data only when needed, and avoid the creation of new pockets of memory during the editing and game play processes. The ideal mechanism would occupy memory only when the camera needed to render a view, and use LOD (level of detail) to reduce polygon quantities down for wide angle or high views.
I don't have the whole picture in my mind right now, but the upshot of such a system would mean you would no longer have the traditional problem of 'stopping editing when the memory ran out'. Instead, the memory is used as a workspace in which to bring things in when you are using them, and remove them when you no longer require their services. This means the limitation is no longer the memory size, but the local storage system which I think we can all agree is plentiful.
I am curious what the voting might be on this possible new approach. Keep with the current plan and be ready for October 2013 or implement this more far reaching technology and maybe add two months onto the project.
Signing Off
As you can see, big thoughts on Friday and some coding to back-up my concern that memory management in it's current form is a little too wasteful. There are also a few solutions that sit between these two approaches, and I need to finish thinking about the many assets required to populate a typical level before I know how big the change might be. I do know that when users buy and start using Reloaded, they should be able to paint segments for as long as they want and as far as they want. It's not everything, but I think it is one of the important ones to get right and I don't mind sacrificing a deadline if it means turning a 'good' result into a 'great' result.
I'm ok with two months on top as long as it makes a better product.
ReplyDeleteMy own personal opinion which may not be shared is that is if the opportunity is presenting itself to overcome perhaps the most significant and important core issues of the past FPSC classic and remove them in FPSCR I would say the theres no question or hesitation to be considered. Two months or two years makes no difference this end - the bottom line is its needed and of such great importance that it such an opportunity should be not missed and grasped with open arms.
ReplyDeleteAt the end of the day the difference to the betterment of the FPSCR product promises to be significant in the opposite way as it detracted in classic and will be of great benefit to FPSCR as a product and to end users in their game making. It should significantly help more sales and the games that are made with it.
Waiting for two months is little compared to the added benefit I would think for all of us. What a big difference that would make.
Missing the opportunity now at this stage seems like is the worst scenario to me and we would most likely be sorry in October and ongoing :-)
As they saying goes "better to have tried and fail than not to have tried at all"
Thanks for looking at that anyway and hopefully it will get through to FPSCR.
Go for it I say.
Had a good think in the bath and have figured out how it can all work now. In theory, it means you could CRAM content into every corner of the 200x200x20 area and it would not even creep over 1GB. It all stems from the concept that most of the segment and entity art are copies and you cannot see the entire world in high resolution all at the same time. Given these two precepts, and a bit of nifty background loading on a second core, you could have a map several thousand tiles across, but I think we will start with 200 :)
ReplyDeleteI think if you take 6 months longer than you thought but create a product whose only limit is the imagination of the developer, rather than a 'Green Bar of Frugality' (GBF), that's what you should do.
ReplyDeletelol "Green Bar of Frugality" I like that :D :D
ReplyDeleteI agree with everyone so far. There's no point in creating this amazing new FPSC with wonderful graphics if we can only have a wardrobe and a sink in each room. I want a wardrobe, a sink, a bed, a pile of rubble in the corner, a chandelier, a few suitcases and a pink fluffy cat.
I agree as well. Take the additional time to do it right; otherwise, I'm afraid you'll paint yourself into a corner again.
ReplyDeleteWhile I'm fairly certain even the hermit in the cave out the back has a multi-core computer, I'm curious: What happens if you design a program to utilise a second (or third, or fourth) core to load stuff in the background and then run the program on a single-core PC? Does it just result in big hitches in the framerate or does it crash?
ReplyDeleteI think two months extra is nothing to worry about providing it gives us all a great end product. The memory issues are a real show stopper for many, many users and anytime spent solving it is time well spent :)
ReplyDeleteI've added a poll in the Reloaded forum. :)
ReplyDeleteReloaded forum? Where would that be?
ReplyDeleteOh, found it. So you're NickyDude, huh? Cool :)
Lee, take note that every single answer in both the comments here and the poll by Andy has said that you must spend the extra couple (or so) of months.
Not sure if this bears any relation though I guess obviously it does to the extent of requiring and or using resources :
ReplyDeleteWould it be possible or feasible without significant additional time involved (amount of time is broad statement I know) to consider Characters and their animation sets being managed by the end user and only those required by a character loaded. i.e. not much point in a character has a pile of animations taking up mem or space if the character just stands at idle or sits in a chair and speaks to the player.
Perhaps the size of characters files and their animations dont have a great bearing in the first instance and or if there are more than one of a specific character type I dont know how FPSC manages all of that but some engines have I believe that ability to load only whats needed for characters as well as other game elements to help with engine efficiencies. Not trying to go off on a tangent here - just seeing it as something associated to only calling whats necessary at to the engine to manage or utilise. IN FPSC classic at least Characters do have a bearing in as much as the more you have the more strain on the engine and fps - again being a major limiting factor.
Just a thought - ignore me - I am sure thats probably for another time anyway if ever. Best to concentrate on whats at hand with the development and not move to another tangent, but perhaps something to think about and bin it or whatever as to any validity or relevance.
Nice to see more people coming along and taking interest in FPSCR and posting hopefully we will see more users at the end of the day so its all for the good.
:-)
Anyone who ever met a deadline either had their product crash on them and/or had to release a patch almost immediately to fix their incomplete game or whatever.I don't mind waiting :( if it means the finished product will be complete.
ReplyDeleteA resounding yes so far in the poll to extend development to produce bigger levels. :)
ReplyDeleteI'd loveee to have a bigger skybox or skydome. Games like fallout and skyrim the skys looks so bloody gorgeous...
ReplyDelete