Inching towards Big Things

The Tiled File Input Device is pretty much done now. All that remains is adding the multiresolution cache, which will be a big help for huge datasets but is not strictly necessary. Speaking of which…
I just got done building a 1.6GB dataset for testing tiled input/output. Clocking in at around 5.5hrs of build time on my Athlon64 3000, it’s a good test for both the tiled output and input systems. Spanning a region from the “Explore 3 – Nice World” tmd file, it consists of a seamless 20×20 tileset of 1024×1024 tiles (that’s 20,480 x 20,480 if you’re counting). Note that to facilitate inter-tile blending the actual temporary dataset on disk sits needed up to 2GB (each tile is built at a higher resolution of 1250×1250 across a larger geometric region and then blended across the shared region).

Getting very close to the first alpha test release. I’m going to backload alot of the remaining work in order to get the first Alpha version out ASAP. Mostly to be able to get feedback about the tiled system / multithreading / layout mode of Pro. Everything besides those is pretty much still in progress, but the feedback and bugs can start on those already. 🙂

Layout mode in particular is going to need a lot of feedback and refinement, as it is an entirely different and extremely powerful direction; but with that potential lies the necessity of alot of streamlining/UI work/thought to make sure that it works as well as it should.

Tiled File Input

The basic functionality of the Tiled File Input is completed.

A bit more explanation of what it does might be in order.

As you know, the normal File Input allows you to load a single file from disk, and place it at any size and location in worldspace. Thus, you can create a low-res guidemap, stretch it across a huge expanse of world space, and then use it to guide the fractal generators to create a high-detail terrain.

With Pro edition and Tiled Output, you can optionally output many small slices of terrain instead of a single large heightfield. In this way you can create practically unlimited-size and detail output — instead of being capped at 8192×8192 (and that only in relatively simple cases), like in the Standard edition.

The Tiled File Input allows you to input a composite terrain that is made up of many small tiles as well. This composite might have been made by World Machine itself (bringing the results of a previous Tiled Output run back into World Machine), by a bitmap-slicing program, or some other source of tiled terrain/image information.

You still define an expanse of world space to map this tileset into — but now the source is many heightfields instead of a single one. Since there’s no way WM can have them all in memory at once (one of the reasons for tiling the terrain in the first place!), WM maintains a cache in memory of the tiles, keeping only the tiles/resolutions necessary in memory at a given time.

What does this let you do?

  • Visualization: You can pan around the (huge) custom terrain Google-Maps style in Layout mode, or fly through the terrain in Explorer mode, making it easy to actually visualize the contents of your massive map.
  • Flexibility: There are no limits placed on the number of streams of tiled data. You could have multiple Tiled File Inputs defined at different locations in world space, providing insanely high-detail terrain at certain locations. You can then set the render quad to whatever location and resolution you want and image as much or as little of these tilesets as you want.
  • Seamless iterative editing: The tiled nature of the terrain is invisible to the other devices of World Machine. You could, for example, input a 100×100 size tileset, each tile of which is 512×512. You can then perform whatever editing operations you want on the terrain — shaping, erosion, etc. Simply wire the output of all of your edits to a File Output device, set the Tiled Output options to match, and run the Tiled Output. All erosion, etc will be performed across tiles* and in a seamless manner, and your output is a new set of tiles that contains all of your edits.

*Obviously, there are some limits to this owing to the tiled nature of the output. When applied to the extreme, effects will look different tiled versus only as a single heightfield. Some comparisons will come later..

There’s quite a bit of coding left to do to finish up the device and make it robust. In addition, the caching mechanism also needs to be improved. Right now it does a simple LRU scheme on the tiles of the set; Future caches will keep full-detail LRUs in addition to containing low-resolultion full-tileset data, which greatly speeds up exploring/layout viewing the tiled data. Hopefully these changes and other things will be completed this week.

The latest 'Behind the Scenes' news from Stephen, the author of World Machine