The Tiled File Input device is complete.

Lots of little tricky cases had to be tracked down and accounted for, but its now working the way I envisioned it, and its quite nice to have!

A little more detail: It now manages a multiresolution tile cache on the fly. Depending on the amount of memory you specify for the TFI device to set aside as a cache, certain resolution versions of the tile data are kept at all times in memory so that the entire tilespan can be very quickly imaged at coarse resolutions (such as when in explorer mode, layout mode, or zoomed out a long ways) without having to go to disk. The rest of the cache space is kept for if higher or full-res data is needed from a tile, such as when you zoom in to full detail level across a small area of a large highly detailed tileset. The end result is (after a somewhat length and optional pre-caching of the tiles), you can manage/view/explore the high detail tilesets with very little slowdown.

It’s pretty awesome to be able to pull in that 1.5GB dataset that I mentioned earlier, and then explore/modify it in realtime!

The only remaining issue with the TFI device is how best to handle caching multiple devices loading the same file stream. This actually occurs pretty often; Layout and Explorer modes spawn threads that actually work on copies of the devices to prevent cross-thread pollution of the data. This is fine for normal devices, but for devices that might have up to 400MB of data associated with them — this is undesirable. We’ll see if it ends up being a big enough issue to work around or not.

Things are just about ready for the first alpha of the tiled functionality and the layout mode. Tiled processing is very very close to finish polish level, but layout mode is just in the very opening stages. I expect that it will change and gain new features many times throughout the alpha and then the beta testing.

Odds and Ends

It’s always amazing to me to realize just how much the 80/20 rule applies to software development. 80% of the code takes 20% of the time; 20% of the code takes 80%. Spent most of this week fiddling with performance issues, bugs and other development gremlins.

The Tiled File Input device now has a basic multi-resolution cache; More work needs to be done to get the performance to where it should be on very large datasets though. Right now, as soon as you actually NEED the cache, the performance will start to degrade somewhat severely.

The Spline Path tool in Layout mode is also giving some issues. It comes down to how in the heck I want to rasterize a spline path with falloff anyways. The current method is quite possibly the slowest I can envision, performing root-finding for every pixel to determine distance to the spline path. Needs to be improved on.
Lots of other tweaks happening too, all aimed at hopefully having the first testing by the alpha team begin prompty.

The latest 'Behind the Scenes' news from Stephen, the author of World Machine