Limit Theory Development Summary: July & August 2017
Happy Friday all ye (future) fellow explorers of the infinite!
In the last update, I spoke of super-fast new code for powering our game objects, the push toward release-quality code with Lil'T, and, of course, the dawn of the 'multi-programmer' era of LT development. I'm really pleased to be able to report back with great progress on all fronts over the past months.
At this point, we're working *almost* exclusively in Lua -- meaning we're dedicating nearly all of our time to game code rather than engine code. Exciting times, to be sure.
Building the 'Real Stuff'
A great deal of effort over the past two months was put into setting up the scaffolding surrounding gameplay code, and, as a continuation of the work referenced in the last update, pressing further into 'real game code' territory, stepping back to see if the scaffolding held in place, and repeating.
Concretely, we began using our much-touted ECS (which, having come of age, now goes by the name 'CTypes') to create some basic game entities, run basic logic on said entities, and analyze the resulting LuaJIT output & performance to ensure that everything was behaving as expected (-- which is to say, blazingly-fast). CTypes is a really, really exciting system, because it allows us to use fast and size-efficient native C memory layouts (backed by custom allocators just to push it over-the-top!), while being able to write logic in Lua, all the while retaining the benefit of having it dynamically compile down to good-quality assembly thanks to LuaJIT and our architecture. Moreover, it's not just a toy anymore -- CTypes is battle-tested and has benched some really impressive numbers, notably, the ability to perform physics calculations on tens of thousands of asteroids at 60 FPS! I claimed last time that the performance problems inherent in a game like Limit Theory had been solved, and the tests are continuing to back me up on that! Speaking of physics...
Getting Physical with Newton
In implementing game logic, one of the most basic first steps is, of course, having objects that can move and respond to forces and torques. Newtonian dynamics. Previously (and I'll be honest here since it's now fixed...), physics has been fudged in LT pretty badly. Especially angular dynamics (torques, angular momentum, rotating objects). Thankfully, I finally took the time to learn the real math behind said dynamics, and our basic motion engine is running beautifully! We set up a sandbox where we could use the mouse to apply forces to asteroids, and proceeded to hurl massive rocks aimlessly across the stars.
Perhaps it's also worth mentioning that I made a very simple multiplayer version of that particular sandbox, so that we could all fight for control of the same asteroids, and we played it over LAN in the office. This served as a nice little test that the basic networking functionality I promised to include in LT (so as to make a 3rd-party multiplayer mod feasible) is indeed working, and such a thing is indeed feasible!
More Procedural Power
Way back in July, I set Adam to the task of studying distance field surface extraction techniques and finding out if there were any good, open-source implementations out there that would allow us to upgrade our distance-field-based mesh pipeline. I'm sure I could find at least a dozen mentions in past devlogs of 'what ifs' with respect to having a really good surface extraction algorithm, and how everything would be golden if we did. Thanks to Adam's work, it's no longer a 'what if' situation!
Adam succeeded in finding an open-source library that has the functionality we need, and also went through the struggle of figuring out how to build, test, use, and evaluate the quality of the code therein. He was able to show concrete timings and outputs that proved the library capable of performing adaptive, high-quality surface extraction with great speed. At the end of the day, it basically boils down to both a lot of dev time saved, a substantial performance gain, and way more flexibility for our PCG.
Here's a shot of a simple test CSG meshes, demonstrating how well the algorithm works (notice the really nice adaptivity!):
Although I haven't yet gotten around to taking another crack at the ship algorithm with this power in-hand, it's really only a matter of time!
Adam and the Magical BSP Trees
Several times over the past two months I've had to turn around and tell Adam that his BSPs are just grossly over-performant. In our current LT sandbox, we can click objects to select them, view data about them, orbit them with the camera, etc. Selection requires raycasting the scene, which, in turn, requires raycasts against the BSP trees that provide the acceleration structures for meshes. Currently, we're literally raycasting against every single asteroid / ship at once. Really, it shouldn't even work. It's disgustingly inefficient. It should outright melt the computer when you have thousands of complex objects in the scene. But it doesn't. In fact, the fps hit is so low on my machine that I forget how negligent I'm being.
Ultimately, what this means is that we'll probably be able to support like 10000000 million projectiles in the world at once. Or something like that. Basically, REALLY BIG BATTLES. 100 ships is going to be a cakewalk, apparently! We've already established that CTypes can handle massive data/logic throughput with ease, and now we've got these magical BSPs that can handle stupid numbers of queries per frame without batting an eye...yeah, I'm looking forward to seeing 'new LT' battles ._.
In August, Adam began spearheading the campaign to improve our debugging & development efficiency with an in-game developer UI. We decided that the real UI should be one of the last things we do (this was guided by the realization of how much time I've sunk into UI that later became useless when underlying game mechanics changed). Until then, though, we've got some nice, programmer-artsy widgets to see what's going on.
Nebula Algorithm v. N+1
I spent some time after-hours one night making the nebula generator roughly 10x faster while also improving the visual fidelity. I'm mentioning this because nebula generation was one of the most expensive parts of system generation (by quite a lot). It's now much more reasonable, especially on less-powerful GPUs. And more beautiful. Win-win-win.
I put some eye candy shots in an album for the end-of-August devlog. Nothing you haven't seen before, but as you can see I'm trying to be more proactive about pursuing the whole "we want screenshots, it can be of anything!" sentiment that seems to be the prevailing one.
Sadly, Friday, August 11th was Sean's final day here as an intern, as he's now off at college studying CS and no doubt becoming an unstoppable force of programming. We'll miss him for sure...his time here seemed way too short.
In the weeks before he left, Sean set up a combat sandbox that can run really fast combat simulations, provide us with information about the performance of different combat AIs, and rapidly simulate entire 'faction vs. faction' battles. At one point, Sean simulated a 100-faction battle with thousands of ships. It was entertaining but terribly slow since his sandbox isn't using our optimized engine as a backend.
In his final act of glory, Sean finished implementing a fully-generic, hierarchical, fleet-based AI maneuvering system, capable of representing fleets with any number of sub-fleets, and intelligently directing their motion. To demonstrate it, he showed us a group of several fleets flying together in a V formation, with each fleet comprised of squadrons in a V formation, each of which was comprised of fighters in a V formation...you get the idea! All of the math required to make it happen is rather difficult due to this arbitrary nesting requiring scale-invariance. Sean got it all hammered out and now we have proper, hierarchical fleets.
I'm really eager to get Sean's work integrated into the main LT project and have it run under our performance-crazed systems. I look forward to watching hundreds of ships fly in recursively-nested formations :D
In the latter part of August, Adam and I have cumulatively had more of those 'wooohoo!' excitement moments than I can remember having in the past year. It's undeniable. LT development is finally where it needs to be: on a rock-solid foundation and headed steadily toward the finish line. Of course, between here and there we've still got a lot of progress to see. But it's progress of a different nature -- a significantly more enjoyable one!
As always, but perhaps more now than ever, I look forward to the coming months and sharing the results thereof :)
Thank you all again for giving us the opportunity to build Limit Theory.
PS ~ Expect another double-rolled update in Octember (hey, remember from back in the day? That was a thing!) -- 2 months is turning out to be the natural KS update stride for our dev pace.