Share this project

Done

Share this project

Done
FRONTIERS - Explore ● Discover ● Survive's video poster
Play

FRONTIERS - Explore, Discover & Survive in a massive, relaxing open world that emulates the tone of classic first person RPGs. Read more

Seattle, WA Video Games
Share this project
4,574
backers
$157,381
pledged of $50,000 goal
0
seconds to go

Funded!

This project was successfully funded on July 17, 2013.

FRONTIERS - Explore, Discover & Survive in a massive, relaxing open world that emulates the tone of classic first person RPGs.

Seattle, WA Video Games
Share this project

March Update - Case Study: Textures

26 likes

Hello, everyone! Some of your suggestions for update topics back in February were:

  • Architecture / iteration
  • Multiplayer
  • More tech stuff in general
  • Interface
  • Tools

I'm dealing with a problem right now that falls into the tools / tech stuff categories: Texture management. (This is exactly the kind of thing I would assume is too boring to write about, but hey you asked for it!)

Arg, Unity

Unity is a great tool. An amazing tool, really. But it has limitations. The biggest is that it assumes you'll be using 'levels' in your game. This doesn't rule out open world games, but if you were playing by their rules it would be something like Metro: Last Light where the world is carved up into discrete regions with maybe two or three entrance/exit points, plus loading screens in between them. You load a level and everything in it when you enter an area, then unload everything when you leave. Sure some minor stuff like enemies might spawn and despawn while you're there but the general rule is: load all the things / unload all the things. (Including any textures used in the level.)

FRONTIERS isn't like that. There aren't levels - there's just a world. I have exactly one 'level' that's loaded on startup. After that I load and unload individual objects - characters, rocks, terrain tiles - into whatever area the player needs to see.

This was the root of the problem I faced last year - how do I organize content so that I can load and unload it as I move through the world? How do I avoid loading too many objects at once, or too few? I came up with some decent solutions. It'll never be as seamless as a truly streaming open world like Grand Theft Auto, but it works.

Unfortunately this approach prevents also me from using a lot of Unity's built-in tools - stuff like pathfinding, occlusion culling and (apparently) texture memory management. (Note: I keep hedging when making statements about Unity's texture memory because the truth is, I have no idea what's going on under the hood, and no one else seems to either. Google this problem and all you'll find is lots of unanswered questions & educated guesses.)

So while I may have the number of objects under control, I'm now dealing with a related problem: textures. Load too many textures at once and blammo: crash to desktop. These are the main issues:

  • Unity doesn't have a useful, well-documented way to manually unload a texture that has been loaded at runtime. Yes, you read that right. The only method they offer that actually works makes it impossible to load the texture again, which is useless. I could write an entire blog entry about this issue but I'll spare you the colorful language.
  • The foliage and structures in FRONTIERS have a lot of variety. If I was dealing with a mono-climate I'd be fine, but we're talking a whole new set of plantlife and new structure textures for every region. Which means more textures, which means more delightful crashes.
  • Unity doesn't have a useful, well-documented way to manually unload a texture that has been loaded at runtime. Oh did I already mention that? Well take a moment to let it sink in.

What to do?

Use fewer textures, duh. There are lots of ways to do this.

There are a lot of redundant materials in any game world. Got a wood chair, wood table, wood axe handle, wood cupboard? Use the same wood texture for all of them. You lose a little visual flair but I'll happily sacrifice dedicated oak, cherry and pine textures for more plant varieties.

What about color variations? For characters and for some objects I use shaders that apply colors to areas of a texture based on a red/green/blue mask:

Same two textures, thousands of variations
Same two textures, thousands of variations

Overlay textures are awesome. A cliff face might need a massive 2048 x 2048 texture to look good up close. A unique texture that enhances its geometric contours, mind you - you can't get away with a generic texture like you could with the non-natural wood objects. But if you can only spare 256 x 256 pixels per major rock variety - and I can barely spare that - the result is going to be a blurry mess. So you overlay another smaller, tiling texture on top. Now you've got something that looks OK up close while still preserving the broad details of the unique texture. Different overlay textures can produce different rock varieties, too:

What about snow? We used to have unique snow textures for all of the rocks on Willowpeak, but when I started adding structures to that region we ran out of memory in a hurry. So I axed all the snow textures and swapped it out for a snow shader that applies a tiling snow texture based on the surface normal. (A side benefit being that the snow can be added or removed based on season.)

Then there's terrain. When Given first started working on the terrain he told me 'I want to use four textures - dirt, grass, sand, rock.' I told him he was crazy, and I meant it. How could we possible achieve the variety I wanted with only four textures? His solution was to use a color overlay shader. The shader cleverly identifies the average color of the texture it's being applied to, then only changes that 'major' color while leaving the rest alone. I had planned to use 16 2048 x 2048 textures plus local variations, so even with the addition of a 512 x 512 color overlay texture for every terrain tile we still came out ahead. It's tough to describe so I'll just show you a before / after:

There are other methods like texture atlasing and clever UV mapping - this is especially relevant for the interface.

Still not enough

This accomplishes a lot but it's still not enough. The best way to keep texture memory in check is to unload the textures you aren't using. If you're nowhere near a palm tree - and I have ways to tell - do you really need that palm tree texture taking up 256 x 256 precious pixels? No. But as I mentioned before, Unity doesn't have a (useful) way to manually unload a texture that's been loaded at runtime. So until I hear differently - and I'm working on getting directly in touch with someone at Unity who can address this issue - I'm forced to assume that FRONTIERS must be capable of loading all of its textures at once.

I'm shaking my head as I type this because that kind of restriction is ludicrous, but the alternative is to subject players to eventual out of memory errors as I continue to load (and not effectively unload) textures as they move from one part of the world to another.

So the next week or so will be spent aggressively pruning every texture that we don't absolutely, positively need.

Oh Unity, I can't stay mad at you

This texture unloading business is a major headache and kind of inexcusable but it doesn't diminish Unity's overall usefulness. For every hour I've spent wasting time wrangling textures I've probably saved ten with their shader tools or rapid compiling. And to be fair, I am the one that broke the rules - if I had stuck with their level paradigm I wouldn't be facing this issue. (Curse you, stubborn streak!)

AND there's always the chance that someone, somewhere has figured out a useful workaround for this problem, or that the latest version of Unity will address it outright. I'm keeping my fingers crossed for that outcome.

Alright, back to work

I'll go into multiplayer in the next update. Matthew is hard at work on it right now, so by then I'll have some more in-depth information. In the meantime if you're still in the mood for some tech stuff, here are two devlogs you might have missed that touch on Creature AI and randomized character generation.

Cheers,
- L

Hans Christian Böttcher, BdB, and 24 more people like this update.

Comments

    1. Creator Stefan Wimmer on October 8

      How did you solve this issue in the end?

    2. Creator Daryl Putman on April 22, 2014

      I'm curious if there's been a response from Unity? This seems like quite the odd technical limitation.

    3. Creator Lars Simkins on April 15, 2014

      >You could break down the world into regions you could load on the fly
      Good thought - that's how I handled loading regions at first. But storing data in Unity scenes isn't very mod friendly so I've got my own scene format now. Plus since it's an additive load you still have to deal with destroying objects that are no longer needed.

      >Sounds like you changed to the same system Skyrim uses for snow on objects.
      Theirs is way cooler / more sophisticated but overall it's the same idea.

      >That doesn't sound too good :(
      It's not great, but there's always a solution. I know someone who's putting me in touch with Unity directly so I'm sure it'll get cleared up soon.

      And it's forcing me to be more efficient with textures, which I should be doing anyway.

    4. Creator Victor Tombs on April 15, 2014

      Errm....I like the images of the rocks Lars. Another non-programmer who wishes you good luck with finding answers to your Unity problem. :)

    5. Creator Stefan Wimmer on April 15, 2014

      Have you looked into this?
      http://docs.unity3d.com/Documentation/ScriptReference/Application.LoadLevelAdditiveAsync.html

      You could break down the world into regions you could load on the fly (like in Morrowind).

    6. Creator Technotica on April 15, 2014

      That doesn't sound too good :(

      Is there some way to contact Unity directly and see if they have any plans to fix this in future releases? Or maybe they can point to some workaround?

    7. Creator XJDHDR on April 14, 2014

      >"So I axed all the snow textures and swapped it out for a snow shader that applies a tiling snow texture based on the surface normal. (A side benefit being that the snow can be added or removed based on season.)"
      Sounds like you changed to the same system Skyrim uses for snow on objects.

    8. Creator Lars Simkins on April 14, 2014

      >can you edit an already loaded texture that is not being used so that it takes up less memory?
      As a matter of fact I can - some folks have set textures to a white 1 x 1 pixel as a workaround for this problem. If I'm loading from Resources this actually changes the asset 'permanently' so it's not a useful fix in that scenario, but if I loaded my textures exclusively with the WWW class (which is used for streaming assets) I could set the 'unloaded' texture to 1 x 1 pixels, then retrieve the image from an external file again the next time it's needed. But as far as my experience goes, re-loading the texture with the WWW class creates a new texture object,* so every time that's done I ensure that I will eventually have another 1 x 1 pixel texture to deal with. So the question becomes: do I want hundreds of little 1 x 1 pixel textures hanging around? Not really, because those WILL add up over time. But maybe that's what I'm stuck with.

      (*This may not necessarily be true, I may be able to load a PNG file as a binary blob and stuff that into the existing texture.)

      >could you then put a check in place that checks to see if the asset is null and then re-instantiate the asset if it is?
      Resources returns null the next time I try to load the asset because the asset has actually been destroyed by the unloading process. It can't instantiated again until the program is restarted. As for why it's possible to 'destroy' an asset in the Resources folder in the first place - I have no idea. It hasn't been destroyed on disk, so what's the problem? I really wish Resource's inner workings were less opaque; maybe its behavior would seem less unintuitive. From the outside looking in it just seems weird.

      The UnloadAsset function was supposedly created to handle all this nonsense:
      http://docs.unity3d.com/Documentation/ScriptReference/Resources.UnloadAsset.html

      In theory you load a resource using Resources.Load, then unload it with this function when you no longer need it. In practice its behavior is unpredictable and no one seems to know how to use it properly. It doesn't seem to work unless all references to the object being unloaded are null, but if that's the case what's with the automatic reload the next time it's accessed? How can it possibly be accessed when all references are toast? Apparently it can't be used on GameObjects either, and it may or may not need to be called on the same frame as Destroy... who the hell knows. Totally opaque.

      Meanwhile, I'm not working on gameplay. >:[

    9. Creator Jesse Dylan Watson on April 14, 2014

      I found it really interesting, too. :) Fun.

      Seems bizarre that Unity would be designed with such limitations when open world games have been around since GTA III and Morrowind and perhaps even earlier. Did they not anticipate people might not want to create their game segmented into "levels?"

    10. Creator Myles C. Allen on April 14, 2014

      I am a programmer but I know nothing of Unity and have a couple questions.
      1. If you can't unload the textures, can you edit an already loaded texture that is not being used so that it takes up less memory? (then when it is called upon again, just reload it or change it back to the texture you want?)
      2. In regards to the Unity's unload method discussion, if you unload the asset, you know it's more than likely going to be null, could you then put a check in place that checks to see if the asset is null and then re-instantiate the asset if it is? (That way you could use the fact that the asset is null to your advantage.)

    11. Creator Jito463 - WoOS, LofUT on April 14, 2014

      Though I'm not a programmer, I am a bit of a geek, and this is interesting stuff. It's nice to look "behind the scenes" and see some of the technical difficulties faced by the programmers.

    12. Creator Lars Simkins on April 14, 2014

      And now I'm feeling better. I really can't stay mad at Unity. :)

    13. Creator Lars Simkins on April 14, 2014

      Now I'm mad again. Grr.

    14. Creator Lars Simkins on April 14, 2014

      >The unload method does then clean them up.
      In theory yes - in practice leaks still happen. And when it does work, it actually works too well - after a texture that has been loaded from Resources is destroyed / unloaded it's no longer available - using Resources.Load to retrieve that asset in the future will return null.

      They say that you're supposed to instantiate assets retrieved from Resources to prevent this, which makes perfect sense when dealing with objects like prefabs - load the resource, instantiate a billion times, then destroy each instantiated object.

      With textures this makes no sense at all - if I load the texture from Resources, then instantiate it and use that instance, then destroy and collect the instance, I still have the original resource hanging out in memory. Which means I have to be capable of loading my entire texture footprint all at once anyway, which defeats the purpose of loading a texture from Resources in the first place. Which is insane.

    15. Creator Howling Moon Software on April 14, 2014

      Minor addendum: there's no indication that Resouces.UnloadUnusedAssets is limited to stuff loaded by Resources.Load. In fact, the usual method of cleanup is the implicit UnloadUnusedAssets that happens when you change scenes. (And for obvious reasons, you can't change scenes).

    16. Creator Howling Moon Software on April 14, 2014

      Now imagine making mobile games in Unity without a way of unloading textures ;)

      Not to get too in-depth in a blog post comment, but I'm assuming you did look at http://docs.unity3d.com/Documentation/ScriptReference/Resources.UnloadUnusedAssets.html which does a garbage collection style walk of the loaded scene and marks all your "assets" as currently referenced or not, then does cleanup some resources. Like you said, it's mostly hearsay, but in our experience this will find textures, materials, and mesh objects that aren't in the scene and remove them. There's a lot of potential to leak meshes or materials in Unity - *if* you are generating them yourself at runtime. The unload method does then clean them up. I suspect this will fix your issue if you've instantiated game objects in one region of the game and later destroyed them - if nothing else is referencing the asset, it should be cleaned up by UnloadUnusedAssets. It is a long operation (at least on mobile devices), but it runs asynchronously.

      Keep up the great work!

    17. Creator Revisor on April 14, 2014

      Would it help if Unity opened its source code, like Unreal did?

    18. Creator BorisKourt on April 14, 2014

      Amazing update, thanks so much for putting it together.

    19. Creator cordell melgaard on April 14, 2014

      This was an excellent post and it shows just how hard it is to make a video game. Even when you have a game engine helping. Thanks!!

    20. Creator Jason Janse van Rensburg on April 14, 2014

      thanks for writing this in a way that gives someone like me with no in-depth knowledge of software design the impression of understanding what's going on ;)

    21. Creator Jennifer on April 14, 2014

      I absolutely love the detail and cadance of this post. I've been hearing about Unity for forever, but getting to hear about it from the perspective of somebody struggling with it's limitations is fascinating.

    22. Creator Neo on April 14, 2014

      Yeah this is interesting to read! :)

    23. Creator Hugo B. on April 14, 2014

      Awesome update! Cool to read how you push the boundaries of Unity. Hope they will start supporting unloading of textures soon though.. ;-)