Category Archives: World Machine Pro Development

All about WM Pro

Odds and Ends

It’s always amazing to me to realize just how much the 80/20 rule applies to software development. 80% of the code takes 20% of the time; 20% of the code takes 80%. Spent most of this week fiddling with performance issues, bugs and other development gremlins.

The Tiled File Input device now has a basic multi-resolution cache; More work needs to be done to get the performance to where it should be on very large datasets though. Right now, as soon as you actually NEED the cache, the performance will start to degrade somewhat severely.

The Spline Path tool in Layout mode is also giving some issues. It comes down to how in the heck I want to rasterize a spline path with falloff anyways. The current method is quite possibly the slowest I can envision, performing root-finding for every pixel to determine distance to the spline path. Needs to be improved on.
Lots of other tweaks happening too, all aimed at hopefully having the first testing by the alpha team begin prompty.

Inching towards Big Things

The Tiled File Input Device is pretty much done now. All that remains is adding the multiresolution cache, which will be a big help for huge datasets but is not strictly necessary. Speaking of which…
I just got done building a 1.6GB dataset for testing tiled input/output. Clocking in at around 5.5hrs of build time on my Athlon64 3000, it’s a good test for both the tiled output and input systems. Spanning a region from the “Explore 3 – Nice World” tmd file, it consists of a seamless 20×20 tileset of 1024×1024 tiles (that’s 20,480 x 20,480 if you’re counting). Note that to facilitate inter-tile blending the actual temporary dataset on disk sits needed up to 2GB (each tile is built at a higher resolution of 1250×1250 across a larger geometric region and then blended across the shared region).

Getting very close to the first alpha test release. I’m going to backload alot of the remaining work in order to get the first Alpha version out ASAP. Mostly to be able to get feedback about the tiled system / multithreading / layout mode of Pro. Everything besides those is pretty much still in progress, but the feedback and bugs can start on those already. 🙂

Layout mode in particular is going to need a lot of feedback and refinement, as it is an entirely different and extremely powerful direction; but with that potential lies the necessity of alot of streamlining/UI work/thought to make sure that it works as well as it should.

Tiled File Input

The basic functionality of the Tiled File Input is completed.

A bit more explanation of what it does might be in order.

As you know, the normal File Input allows you to load a single file from disk, and place it at any size and location in worldspace. Thus, you can create a low-res guidemap, stretch it across a huge expanse of world space, and then use it to guide the fractal generators to create a high-detail terrain.

With Pro edition and Tiled Output, you can optionally output many small slices of terrain instead of a single large heightfield. In this way you can create practically unlimited-size and detail output — instead of being capped at 8192×8192 (and that only in relatively simple cases), like in the Standard edition.

The Tiled File Input allows you to input a composite terrain that is made up of many small tiles as well. This composite might have been made by World Machine itself (bringing the results of a previous Tiled Output run back into World Machine), by a bitmap-slicing program, or some other source of tiled terrain/image information.

You still define an expanse of world space to map this tileset into — but now the source is many heightfields instead of a single one. Since there’s no way WM can have them all in memory at once (one of the reasons for tiling the terrain in the first place!), WM maintains a cache in memory of the tiles, keeping only the tiles/resolutions necessary in memory at a given time.

What does this let you do?

  • Visualization: You can pan around the (huge) custom terrain Google-Maps style in Layout mode, or fly through the terrain in Explorer mode, making it easy to actually visualize the contents of your massive map.
  • Flexibility: There are no limits placed on the number of streams of tiled data. You could have multiple Tiled File Inputs defined at different locations in world space, providing insanely high-detail terrain at certain locations. You can then set the render quad to whatever location and resolution you want and image as much or as little of these tilesets as you want.
  • Seamless iterative editing: The tiled nature of the terrain is invisible to the other devices of World Machine. You could, for example, input a 100×100 size tileset, each tile of which is 512×512. You can then perform whatever editing operations you want on the terrain — shaping, erosion, etc. Simply wire the output of all of your edits to a File Output device, set the Tiled Output options to match, and run the Tiled Output. All erosion, etc will be performed across tiles* and in a seamless manner, and your output is a new set of tiles that contains all of your edits.

*Obviously, there are some limits to this owing to the tiled nature of the output. When applied to the extreme, effects will look different tiled versus only as a single heightfield. Some comparisons will come later..

There’s quite a bit of coding left to do to finish up the device and make it robust. In addition, the caching mechanism also needs to be improved. Right now it does a simple LRU scheme on the tiles of the set; Future caches will keep full-detail LRUs in addition to containing low-resolultion full-tileset data, which greatly speeds up exploring/layout viewing the tiled data. Hopefully these changes and other things will be completed this week.

Overlay Output Example

The Overlay Output device and all of its attendant code in the core and visualization modules is finished; just wanted to show people an example of the kind of visualizations it provides:

This is showing a terrain coverage map draped atop the heightfield. A coverage map shows what type of terrain is at that location. It’s NOT a terrain texture by itself; rather it’s a type of terrain metadata that specifies WHICH texture to use at a given location.
The coverage map was created by the Coverage Maker device, which produces an RGB-encoded image, where each color is linked to a type of coverage. This allows you to setup your terrain surface texturing for the game or 3D app. You would then (in the engine or the next tool in the content pipeline) map each RGB triplet to a terrain texture to use, and you’re set!

The coverage map seen here is pretty simple, using essentially three input maps: 1) A Hi-Slopes mask 2) A randomness map (basic perlin noise device 3) The Flow map from the erosion device. More details about the coverage maker later.
Anyways, back to the Overlay Output. Since you can also now load RGB images into WM as well, you could, for example, load in an RGB image that you’ve sketched out quickly, and then use the Overlay output to combine it with a heightfield you’re working on to match the sketch. Or lots of other uses — you can probably imagine more than I can.

One more image:

Example 2

Using a different map and coverage schema, and viewing in Explorer mode. Although it isn’t final texturing at all, it actually looks pretty good. It’s amazing how having some actual coverage-based coloration rather than just the height-based colormap gives a much stronger sense of realism for the terrain imho. Having this kind of coverage data available during heightfield creation, if taken advantage of, will allow for much faster terrain texturing on game maps and other such endeavors.

I think focus is very important for tools, and although adding RGB support to WM is certainly an extension of its grasp., I  think terrain type coverage is a closely integrated problem with terrain modelling, and that puts it right into the purview of WM. If you need the ability to output Normal Maps/Coverage Maps/etc, then you have it. Otherwise, it doesn’t slow anything down.
Back to work for now.

Usability versus Features

It’s always interesting as I develop World Machine, as there are always two distinct areas to add content in:

  1. Usability. Things that make the interface cleaner, that allow you to do things quicker, that are less confusing, or that provide additional power. This makes the experience faster/better/more enjoyable for the content creator.
  2. Output Quality. Things that provide a better result. These provide a better finished product, and are the ultimate judge of what a software program can produce. Doing a neat and clean job of producing some output isn’t much use if the output itself isn’t jaw-dropping.

I’ve always tried to put in new features in each of these categories. Since I’m taking a slight break from layout mode development for a little bit now that things are nearing feature-complete in that area. It’s always good to get some breathing room and a little bit of perspective — I’ve found that you can come up with some really interesting insights that you would otherwise miss if you stayed nose to the grindstone on that feature alone. In the meantime, the things that I’ve been looking at over the last few days and am currently researching, designing, or implementing are:

  1. Tabbed Toolbars. Sort your parts toolbar by Device Function (Generator, Output, Filter..), or Data Type (IE, a page for all HF operators, a page for all RGB operators, for Parameter ops, etc). This is something that I REALLY want to implement at some point for Pro, as it cleans up the parts toolbar enormously. The number of icons is really starting to otherwise get unwieldy. Status: Researching. This is something I want to implement, but haven’t actually started mucking with yet.
  2. Overlay Output. Not an output so much as an instrument or a view, it allows you to have one heightfield provide the geometry of the output, and another HF or a RGB image for the color. So in other words, you can superimpose a mask atop the terrain to see where everything fits into it, or drape your RGB texturing coverage map atop the terrain to easily see what types of terrain have gotten placed where. What’s really cool is that this works in Layout and Explorer mode too, letting you see your entire world with, for example, texturing placement maps put atop it, is incredibly useful. Status: Nearly Complete.
  3. Tiled File Input. This device stands to provide for some terrific workflow enhancements for alot of projects. Essentially, after setting up the input file stream (SomeTileXXX_YYY.bmp or whatever it may be), the device will load tiles from disk as appropriate to create a heightfield in the current render area. This allows you to seamlessly work with extremely large worlds — not just for export, but to continuously refine them by being able to load back in your tiled output. You can assign each set of tiles to any region in world space, at which point it’s totally transparent to the WM system– you can “zoom in” on a smaller subregion that spans only a few tiles, combine multiple tile sets in any configuration in the world, etc. For those working with paged terrains, this feature along with the tiled output will be worth its weight in gold. Status: In Progress

There’s other things that are also recieving work time right now — hell, since very little of the Pro code is 100% yet except for the multithreaded building code, many things are. The above are just a smattering of the interesting features being worked on this week that will make their appearance.

Lots of exciting things are afoot.

Layout Shapes

The shape distortion code for Layout mode is pretty much done now. Although with the optional override inputs you can certainly take specific control to achieve some special effects, it was important I think to have a minimal-fuss, built-in noise generator that would provide basic shape distortion without having to plug anything together.

Example Layout Shape

Above is an example of the output that WM might produce with the shape distortion when I’ve drawn a simple polygonal “U” shape. Of course you would then combine that with fractal noises and erosion to produce a terrain:

Final Terrain

For the next few days I’m going to be working on polishing off the last options for the Tiled Output code, and then things might well be ready to go to the Alpha team for first testing. There’s lots of code left to be written, to be sure, but testing work can start long before its 100% complete in that regard.