Wednesday, 23 October 2013

Wednesday Battle: LUA vs FPI

Performance Day

Thursday I will be out of the office getting my eyeballs replaced, but today was all about performance profiling and answering emails, and then some more performance profiling.

The good news is that I have gone from 21fps to 45fps. The bad news is that I had to switch off all shadows to do it ;)  Better news is that I discovered that some file map communication code written to obtain Windows mouse coordinates from the parent IDE application was a serious drain, and upon removing it (only needed for actual TAB page views) rescued some substantial performance back from (or should that be for) the engine.

I have also extended the division of metrics within the main engine so I can see at a glance more fine grain detail of what the engine cycle is doing.

As you can see, apart from Misc2 (which I am still investigating but suspected to be a rogue value), the main energy draws are from the rendering step which is the GPU stall while it produces the visual you see above, and a little bit of terrain and AI processing. When I include the shadow rendering, all other performance considerations shrink and it seems the performance is entirely dictated by the amount of 'rendering and shadow rendering' being done by the engine.

In a way this is good, as I can focus plenty of effort on a single module for instant speed-up. I also have some great tools to assist me, and I know for a fact I can create multiple techniques within the shader to suit different system specifications (for LOW, MEDIUM and HIGH).  It's pretty good fun performance profiling as you have nothing to lose and everything to gain, and can be quite a buzz when you gain a few FPS.  Much more to come!

The Big Debate: FPI vs LUA

Ever since the idea was proposed, the topic of which scripting language is better for Reloaded has been pretty hot of late and I have been asked to step in and clarify the situation as to where we are.

I wrote the FPI (FPSC Instructions) language to quickly allow for simple conditions and actions within the game world. It was never meant to be a language as such, and a million light years away from something like DBP. It was ideal for quick one liners, such as turn on a light when you enter a zone, but became almost laughable when you asked it to coordinate the mental abstractions of an ally character following you around a combat scene. I would even cringe at having to apply a state value half way through the script, so that the subsequent conditions where skipped, only to set he state value to the intended value at the end of the script. If you read that in pseudo-code, you'd severely admonish the student responsible:


The FPI language scores marks for being one of the smallest language on earth, it's syntax almost entirely comprising of a colon and semi-colon. It is perhaps this simple format that has earned it such popularity, and when you start to study the built-in scripts, you're guaranteed to learn the logic within a few minutes.

The basic problem with FPI is that the underlying 'interpreter engine' which processes all the commands is written in DBP, and so each command is actually a trigger point for a multitude of other commands to fire before returning to the main thread. Multiply this with the many commands found in the script, and then by the hundreds of entities that are firing off said scripts and you have a substantial drain on overall performance. Indeed, one of the complaints levied against the classic version of FPSC was that too many scripts in a level would start to depreciate frame rate and force the the creator to clear their level of heavy scripts, divide up their levels or worse, scrap their ambitions for more attainable goals.

When we arrived at the decision to scrap the segment system and re-imagine a new way to construct large buildings and structures, it gave us the confidence to ask similar questions about other parts of the system. As fond as we are about FPI, we also know it is the silent assassin of many large and ambitious levels, and if we are to aim high we must look critically at each part of the whole.

Understand that we have not decided unilaterally to drop FPI and instate LUA. We still have to do some prototyping and profiling to see whether the integration of LUA is viable and improves performance of the engine. There is also the possibility of moving FPI into the core engine, executing the instructions outside of DBP. Therefore, you can rest uneasy in the knowledge a decision has not yet been made, and every effort will be made to work out which solution is best for Reloaded. To that end, you are welcome to continue this discussion, and hopefully arrive at a consensus as to ease of use and suitability.

A Question Of Ease

FPI is easier than LUA. LUA is much nicer than FPI. FPI is simpler than LUA. LUA is easier to read then FPI. The mystery is that all these sentences are correct. Based on who you are, you will find a sentence you can agree with. I think each has their charms. Unfortunately I have been in the situation of supporting two parallel paths of near identical functionality before and I must confess that supporting both can be agonizing and demoralizing. It was this past experience that lead me to conclude there should be only one. It was this decision that fired up the debate we are all enjoying.

I was fortunate enough to have spent a few years teaching at Masters degree level, and I learned a few things about 'ease of use' or more accurately 'ease of understanding'. Just as secret knowledge becomes commonplace once explained, so to does a script or language. If you learned FPI, you can learn LUA. If you know LUA, you can learn FPI. The underlying lessons are identical! If THIS happens, do THAT. I'm afraid that's what programming boils down to. If THIS, then do THAT.

From a personal point of view, and this is from a coder who's regrettably forgotten more languages that he presently knows, the syntax of LUA is very reminiscent of BASIC, which remains my all time favorite language. The idea of IF X THEN Y ENDIF really makes sense to me, and is the principal building block of any large electronic edifice. My prediction is that there will be more scripters for Reloaded than we ever had with FPI, not least because the scripts we will provide will become readable!

A final note addressing Reloaded ease of use. We are designing the software so that scripting by no means a requirement of the creative process. In fact, I expect only around 5% of users should ever need to break out into the world of scripting. The remaining 95% will find the combination of entity properties, sliders for in-game adjustment and level configuration more than sufficient to create a unique experience for your users. The scripting is there for those brave souls brazen enough to think they can create better scripts than TGC :) Rest assured that we will provide a wide variety of LUA scripts, assigned by default to relevant assets, dealing with all the game functionalties you require from opening a locked door, to commanding a battalion of soldiers to follow you into hell. You only have to look at our history of products to know that 'ease of use' is one of the goals we constantly strive for, and we understand that many of you have no desire to program, and why you use FPSC in the first place.

What's Next

You will want to know what happens now. What is the decision! For this I will need to create one or two prototypes, principally to test how the integration into the main Reloaded engine will be, and also to check whether the performance benefits are worth the transition. I also want to study the existing FPI language (which has doubled in size since the modders took over) and understand the scope of maintaining and/or moving it to a more internal engine that can process at higher speeds. 

Aside from the professional reasons, I also want time to play with the LUA syntax and get a feel for how it might work inside the Reloaded universe, and whether it is a good fit. LUA is not the only scripting language out there, and my researches might turn up something newer and more shiny! I also promise to try and resist the urge to create a new language for the occasion :)

Signing Off

Hopefully I have brought everything up to date, and hopefully I can bring you more performance improvement news Thursday evening, when I explore the deeper GPU realms and return with some worthy finds.

A Bit Of Fun

Rick decided to film an impromptu video as we concluded our meeting on Tuesday, and then put it on YouTube (bless him). Just in case you missed it, I have hosted it here so you don't miss out on any of my mindless waffle.

Once Reloaded is in a state where I can show my face in public, I am pretty sure there will be plenty more videos for you (and me) to cringe over.


  1. I'm really happy you've addressed the LUA debate, it gives us an idea of your views on the subject.

    Shadows eating performance, well that's always a given. Though they are chomping through an awful lot for very little on scene. I really hope you can find some ways to optimize the engine and improve the performance.

  2. Lee, I am very happy for your hard work to push through FPSC Reloaded, my question is this, Classic FPSC model packs are fully compatible with the Reloaded version?, I have really wanted to try my soldiers of the "world war II" (model pack 68 and 69) in the new graphics engine, I do not want to take the surprise that are not compatible with reloaded, greetings and congratulations on your progress in the project.

    Grettings from unimansoftent. thank you!

  3. Do we have the option to bake lighting?

  4. Sounds like a lot of work there lee,nice work.

  5. Well that's me satisfied. And glad of statement. As long as a multitude of template script I am happy and if not then CLONKEX! Can you?

    anyway all good top banana.

    1. lol :) There was always going to be heaps and heaps of example and pre-made scripts, even if we didn't switch to LUA. But sure, I can still write that converter once we know more about how Lee implements LUA (assuming he does, of course) :)

  6. Oh dear, sounds like that old programmer's paradox:
    "Boss, I've increased performance tenfold. We're running at 999fps!"
    "What!? How!?"
    "I just turned everything off."

    Oh, and I agree on LUA.

  7. I noticed while playing Skyrim even at a high setting.. the shadows are not very detailed. I expect thats why Skyrim plays so well is because there isn't so much focus on shadows?

    1. Of course. Shadows are probably the second most performance taxing thing for games. The first quite possibly being depth of field or SSAO.

  8. Finally finally finally it appears as though you are rendering at the correct resolution! The gun looks waaay better now! You only have to compare the crosshair in the following two images to see what I mean:

    Excellent news that you're going to do in-depth tests between FPI and LUA.

  9. I agree with Lee totally.
    LUA is the right way into the future, but I do understand the FPI-fans. But Lee got some really important points!

    And yes.. BASIC that wonderful old language, reminds me when I ran out of memory on my 32kb homecomputer when programming a BASIC-game. Those were the days :)

  10. I had a commodore 64. Basic is
    1 print " ooh err missus"
    2 goto 1

    TYpe run and it loops

    IT got more complex after that with get a $ and pokes etc.
    Them were the days. Bruce Lee and lazy Jones and baggit man :)

  11. The closest to basic i ever got was QBasic in Dos lol

    When looking at LUA it takes me back.. reminds me actually a lot of old coding languages i used to use.

  12. I dabbled with LibertyBASIC on Windows 98SE and 2000, and since I started programming properly 3 years ago, between DBPro and AGK I haven't stopped using BASIC.

  13. Some of you might have seen my comments in the TGC forums and as Terry Cox in the FPSC-R forum.

    I just want to state here, that I actually favor LUA for reloaded. I am a professional programmer and so I would be biased, because using FPI I missed the real If/then/else statements and such statements as a select/case. I also miss the iterations or loops such as for/next and the while and do loops. I'm not sure if they are in LUA but LUA I believe will still be a much more powerful scripting language.FPI was a real headache to to complex scripting but much everything I needed to get done I was able to do.

    So, Go LUA.

  14. I opted for both, FPI is the basis of the basic models of FPSC, LUA is for me more than an extravagance, many choose to LUA, why you can not use both programming systems (FPI and LUA)?.

  15. Because some bugger has to maintain them both :)

  16. LUA is much, much better. I started with first versions of FPSCx9 and I know: the only way into the future for FPSCR is LUA. It's great and awesome :)