Happy early Saturday to all!
Seemed like last month's format worked out well for you guys, so we've kept the same format this month. Except that we have eye candy this time :)
'Twas a long month. So much happened that it's hard to remember everything, but that's definitely better than the alternative. This month was all about big pushes toward the completion of big systems: assets, entities, and the physics engine. Major progress was made on all fronts.
I said last month that I'd be a happy camper if I had my primary assets 100% back, and I'm pleased to announce that I am indeed a happy camper! All my old mesh tech is back and accessible from Lua. Some of it even got upgraded; BoxMeshes (previously PlateMeshes) can take on cylindrical and other anisotropic shapes that weren’t possible previously, lending more power to ship & station algorithms. With all of this functionality re-packed into the LT Core library, I’ve ported the PCG algorithms for asteroids, planets, ships and stations. Seeing the ship and station algorithms again reminds me that I still need to put more time into them, but it was still glorious to have my assets back!
I uploaded a full gallery of screenshots for that week, so have a look:
Sadly, while trying to delve into some gameplay programming in Lua, I ran into the same old problem...you guessed it: performance. Turns out I underestimated how badly LuaJIT has been hurting me (that is, in comparison to native C or C++ code). It became more evident once I had my assets back and was trying to push gameplay logic into fully-populated systems. Achieving the classic LT silky-smooth 60FPS was quickly becoming an uphill battle.
After much thought, a cascade of obvious insights befell me. Moving some of the entity logic into C wouldn’t be the end of the world; some things modders will never need to touch. That line of thinking led me to more critical realizations: bottlenecks are always in the computations that happen every frame (especially if the computations happen across a large number of objects). Motion calculations and physics in general are obvious culprits. In fact, I realized that the most expensive, frequent computations are in the low-level logic...much of which need not be accessible in Lua. Most gameplay logic is far higher-level and need not run every frame. I knew this previously -- that costly code should be moved into C -- but never realized how well it could be done without hampering the ability to write gameplay quickly and elegantly in script. To make the high-level logic as fast as possible, I’m now working on an ultra-fast, event-driven entity system that will ensure per-frame processing is cut to the bare minimum.
Running performance tests for the first time even with only some of the above implemented shocked me. Moving low-level entity logic into C yielded absurd gains over Lua, but, perhaps even more surprising, it yielded gains over the old C++ engine. This I can only attribute to the fact that I've done a better job of structuring my entity code for performance! Suffice it to say, as of this month, I am completely convinced that our performance problems can be overcome with the right mixture of C and LuaJIT. 50K moving asteroids will back me up on that:
Once more, I couldn't resist and uploaded another full gallery of eye-candy for the week's work.
Physics (collision detection, in particular) was the secondary focus of the month’s work. LT’s collision detection used to break down quite badly in cases that are traditionally ‘difficult’...unfortunately, with the scale of entities in LT, such cases are not at all infrequent. A complex ship navigating in close proximity to a complex station that’s 1000x larger in scale is, for example, a difficult case that occurs all the time. With several new collision detection structures implemented in the core this month, I’m well on my way to obliterating this bottleneck. Initial tests are showing great performance, although I've yet to finish the whole system. No big gallery for this work, but a little gratuitous ocular edification:
Toward the end of the month, I began implementing a full-blown entity component system (ECS) in the C core. The ECS is the part of the engine that controls how the data that underpins objects like ships, stations, colonies, asteroids, etc. is defined, managed, and altered over time in response to gameplay. From the point-of-view of the would-be LT modder (as well as Gameplay Josh), it's the most important part of the engine. I'm excited to see where this work takes me, since I believe that the new ECS along with an event system is going to simplify gameplay constructs dramatically, while providing superior performance to anything we saw in previous incarnations of LT!
On a side note, I got an opportunity this month to exhibit LT in New Orleans at the 'New Orleans Entrepreneurship Week' event. Since it was less amenable to interaction than previous events, I wrote a quick 'cinematic camera' that flies around a system, smoothly moving and panning while hunting for interesting things to look at. My demo was, essentially, 'Limit Theory Screensaver' :P It was quite lovely and I got very positive feedback. One guy even mentioned that he used to love playing Freelancer, and was excited because LT looked like a modern version of it. Naturally this made my day :) I made yet another gallery for the event, although this one is smaller and has less variety since not much was different, content-wise:
Big thanks to Nathan for managing to summarize what apparently added up to 7K words worth of devlog this month! Remember to hop on over to the forums if you're up for digging into the details, and be sure to check out the three screenshot galleries linked above!
I don’t think I can stress how big of a month it’s been for cold, hard progress. I’ve got assets back, I’m well on the way to defeating the performance issues that ended up killing the old LT, and, most importantly, I've got my eye on the gameplay ball. April should prove to be very interesting indeed.
Thank you all for caring :)
~Josh & Nathan