I mentioned in the last post that there are several improvements to Erosion coming, including a hardness input.
But that’s not the only improvement..
The mask input on Erosion previously did a simple alpha-blend between the results and the input. This is.. adequate. One consequence is that the mask is not actually used to guide the erosion itself, ie the masked area doesn’t “sit” well against the rest of the eroded terrain.
Under the next version, the Mask input is an “active” mask. That is, it actually participates in the erosion process:
Fully masked areas will be completely unaffected, as with previous.
Areas outside the mask will now be eroded accounting for the masked area. This is probably what you figured it should do in the first place 🙂 The difference is hard to describe, but profound; for example, a masked area in a low basin that would previously have been filled quickly with sediment will instead stay unfilled, acting as a “sediment sink” for surrounding areas.
Areas that are completely masked will no longer generate any waterflow. This means that masked erosion will be much faster. Fully masked-off areas take relatively negligible time to process.
The main thing that prompted this change was to make terrain with rivers work better with erosion. Now you can simply take the river channel mask, pipe it (inverted) into the erosion mask, and boom!
The river now “sits” much nicer into an eroded terrain. As a nice bonus, the hero-river systems will effectively transport away sediment for the hillslope erosion, just like in real life…
In the last picture above, you can see the effect of the river transporting sediment; the surrounding valley becomes steeply carved with gullies as the river effectively participates in the erosion process! Not bad for a simple mask input 😉
Just figured I’d write a quick note about what’s happening with the latest dev channel release.
A current version with the river tool has dropped to the alpha testing team a week ago; changes and additions are still being added at this point before the Dev Channel release.
Beyond the river tool itself, I wanted to take a quick look at a few other things that will be showing up soon:
Scene View Device
The Scene View device is an extension of the Overlay View, that accepts additional inputs over and above a simple terrain/bitmap combo. The first additional input is what is shown above: you can supply a water depth or watertable map which will be superimposed upon the terrain, allowing you to preview the water surface of your rivers instead of just the riverbed.
This is important because the water level slider presently available works fine for simulating an ocean height but lousy for rivers that have gradients! This ability to display arbitrary water levels is pretty critical and useful; and if it gets you excited about other devices that could export water levels, it should.. 🙂
Another key eventual purpose is to help with workflows using splat maps, where it’s hard to visualize the final result within WM itself. The Scene View will eventually accept your splat (guide) map and also the set of textures you intend to use in-engine and display them appropriately; this should give you less round-trips to the game engine when doing basic map design.
The rough control possible in the Curves device has been a complaint of everyone (including myself!) for a long time now. The Layout curve editor was better but still only let you have linear segments.
I finally took the time to improve the Curve Editor; crucially, the editor is shared between the Layout Generator and the Curves device now, and drastically improved:
This change makes the Curve Filter device about 10x more useful! Presets are shared universally, letting you use the same curve for a layout shape that you might apply to a heightfield.
And oh, did i mention that Curves are now a separate datatype that can be wired to devices? 🙂 Eventually, devices will be retrofitted to accept curves to specify their behavior in all the various ways you can imagine that would be useful…
The Curve Filter is also the first device to show off the new way that we’ll handle device improvement.
In the past, when device functionality changed, an import path was attempted to be created, but sometimes it was impossible to capture the old behavior while still allowing the new.
The new method is to preserve old behavior in a deprecated version that will load by default in older worlds, which you can switch to a new version when you like.
Note that I can’t guarantee that a deprecated version won’t go away at some point; but you’ll at least have fair warning that you’re using an old version now.
The erosion device has gained two new inputs: A rock hardness map input, and a sediment capture input. Combined, they work to provide more control and nuance than the simple mask input provides. Here’s how they work:
Rock hardness does what you expect, and modulates the resistance of the underlying substrate to erosion. Here’s an example image, where the hardness varies from very hard to very soft as you move left-to-right across the heightfield:
This is pretty simple, but useful! I originally didn’t include this input because I didn’t think it made a large enough difference — but especially with geologic time enabled, it can certainly be a very powerful thing to control.
Here’s another example. In this one, a simple gradient slope is being extremely eroded. Guess what area contains a big dollop of resistant rock 😉
So, that’s some exciting new stuff. The big question is, when does it drop to everyone not on the alpha team?
I’m inclined to say “sooner than later”; the whole point after all of the Dev Channel is that things can be in an unpolished state with some rough edges and weird behavior still. But there’s still at least some minor things that should be done, like adding icons still for all the new devices…
But once that’s done and a few other loose ends are wrapped up, it’s probably time to drop the very first version of the river tool to the Dev Channel — I’m sure there’s going to be a million questions and suggestions once that happens, and it’s better to make that sooner than later 🙂
So, it’s definitely time to talk about the River device in the next dev version of WM.
The river device is a major expansion of World Machine’s path tool, providing a huge amount of new flexibility in terms of creating “hero” rivers; that is, major rivers that should be under artist control, as opposed to those created by an automatic simulation-type process.
The best way to describe the new river tool is “semi-automatic”. That is, you will draw in the essential path you want the river to take, much like you might currently do with the Layout Generator’s path tool…
From there however, things become different.
You’ll set some additional basic parameters associated with the river, for example the typical river channel width. The real power is in the river variation model; The river device accepts a new datatype called a “Geomorphic Covariance Structure”, or GCS, which is a description of how the river is shaped in different scenarios. These can be as simple or as complicated as you like; they are created by new devices and/or macros that you or others create to describe river behavior.
The hope is to ship WM with a collection of macros that allow you to plug-and-play GCSes, for archetypal rivers. For example, a lowland valley river vs a fast moving mountain stream.
Some River Geology
To describe what parameters are under the control of the GCS, we’ll have to cover the basic geologic features of a river valley. This includes things like:
(All geology pictures below are wikimedia creative commons sourced)
Meanders are an emergent behavior of the fluid dynamics of a river, causing the river’s path to oscillate back and forth across the general downstream direction.
From your specified river path, meanders are added. These can be specified in all manner of realistic (and non-) ways. Although the model is deterministic rather than simulating hydrodynamics, it does an admirable job of recreating river meanders.
The thalweg of a river is the deepest part of the channel. Our river model captures the repetitive pool and rapids behavior of a river as well as the lateral movement of the thalweg back and forth across the channel due to meander bends of the river.
This includes cut-banks (eroded areas on the outer sweep of a meander), slip-off slopes (shallow areas of deposition at the inside of the river bend), and point bars.
Although you specify the average river width as part of the creation process, the GCS will vary the width of the river in a pattern relative to the meanders and thalweg.
This is very important, as wide/narrow points of the river are often strongly correlated with deep or shallow areas of the river.
Finally, the GCS can also describe oscillations of the valley walls relative to the meanders/etc. Although the shape of the valley is more influenced by factors not specifically related to the river itself (such as hillslope erosion), including this in the GCS lets you correlate things like narrow points in the valley to the behavior of the river channel itself.
So, that’s the geology primer, now let’s look at the actual implementation in World Machine!
The River Model
Here’s a quick walkthrough in pictures of the River Model in action:
Above, you can see a basic straight channel with a simple but powerful River GCS applied; the river moderately meanders back and forth across the gently rising floodplain, with all of the other parameters (depth, width, riverbank cut, etc) all varying in concert to model a basic river.
Here’s a slightly more complicated river that also includes a surrounding valley:
The valley walls are included in this GCS; also, note that a distinct meander belt is present at the base of the floodplain (the shelf present near the river channel itself that, in reality, represents the area that the river continuously migrates within).
Finally, unlike the last few images that feature the river by itself, here’s a more complete example that shows the river embedded within an actual terrain:
This example uses a number of additional river model mask outputs useful for texturing, including masks indicates where the channel, floodplains, and valley exist, as well as a depthmap that can be used to identify deep pools or shallow riffles. This example also includes using the new sedimentation mask input of the erosion device to erode the terrain while preserving the river channel area itself.
The last thing I’ll mention about this upcoming release is that it represents a starting point; Hero rivers were the logical place to start this process, but I can certainly envision extensions to allow a “click to spawn a river from here” method within the Layout View, and perhaps even an eventual fully-automatic river mode that can turn the basic “river networks” created by flowmaps into fully detailed individual rivers. But that’s a post for another day.
I realize things have been pretty slow/uncommunicative lately, for which I do apologize. It’s probably well overdue for me to lift the veil a little bit and show what’s in progress.
If there’s a theme to the next release of WM, it’s this:
Framework Changes: Generalize to Specialize
Cryptic? Here’s what it means.
All of the changes below are changes to the way World Machine works on the inside. These changes have been made to allow for new planned devices and features. Many of those new features have not been written yet, but I will discuss them anyways to provide the context for why the change was made.
Goodbye Layout Generator?
One of the major framework changes is allowing many more “Layout-type devices”, not just the Layout Generator. In WM 2.x the layout generator and the view are intimately linked.
The Layout Generator will be renamed “Layout Shapes”, reflecting that many more Layout devices will be on the way. A new “Layout” tab will be added to the toolbar, and any device (including plugins by other authors!) can now potentially expose a layout UI.
This will immediately be useful in the River Generator device described later. But in the future, other exciting abilities would easily include:
Layout Faults (create specific terraces/fault lines at a location)
Layout Roads (an enhancement of the path tool that could better model roads by including concepts like banked turns, etc)
Layout Segments (let you create segment maps interactively — for example, quickly create a mask for a particular valley by clicking anywhere in it)
I’m sure you can think of many more specialized layout devices that would be useful! And I’d love to hear your ideas. The main thing that has held this back in the past was that the layout interface was only available to the layout generator; that will no longer be true.
This is a simple but powerful extension — there are now a variety of devices that produce and manipulate 1D curves. More will appear with time. Eventually the existing devices will be retrofitted to accept them — making the existing Curves device much more useful since it won’t just be hand-sketching curves anymore.
More Macro-like things?
The macro device is also quite special in WM 2.x. It’s special-ness has been extracted, and can now be extended into new devices. Why would you want to do this?
A Repetition (or Loop) Device would apply its internal network multiple times, looping the result of the first application back into the input for the next time around. This lets you do things like apply a “little bit” of erosion of one type, then a bit of another, repeated over and over to create novel effects.
Devices can now have “internal networks.” This is like a mini-macro where you can override and provide more sophisticated effects than are built in to the device — for example, the Scatter device is planned to have a “User Specified” compositing type that would let you create the merge operation with an internal network rather than than using the default alpha blended/max/etc options.
I can already imagine the head scratching on this one — Not exactly the kind of headline that gets you racing out of bed to use World Machine. And yet, this is a really powerful framework feature, and I’ll explain why below.
There are lots of places within World Machine where you really need more than one piece of data to make sense of something.
Essentially, it’s now possible to create custom datatypes that are a bunch of existing World Machine datatypes (heightfield, RGB, parameter, etc) glued together.
This is roughly equivalent to creating a “struct” in a regular programming language. What’s more, the same code allows for treating an arbitrary number of the same datatype as one thing — roughly an”array” in a programming language.
What does this enable?
Common things you might want to specify that previously would take two or more wires can now be carried around as a single thing. Some examples include:
Distortion amounts (specified as either an X and Y distortion amount map, or angle and amount maps — what’s more, you could throw in a parameter as well to let you tell apart which one you have on your hands!)
RGBA (RGB and an alpha mask)
Texture weightmaps (rather than a bunch of individual wires, you can treat textures and their weights as one unit)
Collections of Things
“Collections” of heightfields/textures/etc that can be treated as a single entity. This lets you do things like:
“Execute for All” macro device that could perform all of its internal operations on each element of the collection. For example, if you imported a whole library of textures and want to color-correct all of them the same way (or, have each one automatically adjusted according to the network you’ve set internally). I haven’t pushed in this direction yet much at all, but it certainly would enable “batch processing” type workflows..
Provide bulk variations to the Instance Scatter device — if you have a whole library of individual things you want to scatter, you can just connect the library to the scatter device rather than 20 individual wires…
How does a complex feature like this actually make your life easier? In lots of ways. For example:
Remove clutter from the workview by having groups collapse all of the wires from devices in one group heading to another group into a single “cable bundle”.
Macro and plugin authors can transfer large numbers of settings between several different devices without exposing any of them to the user, making things much cleaner and intuitive for the end user — keep all the spaghetti inside the black box where it belongs.
So far, I’ve talked about framework changes. I should emphasize again that at least half of what I’ve talked about here is still in the future. What’s been made so far is the framework to enable them.
The Loop Device doesn’t exist yet, for example, but with the new framework in place it is trivial to create. The focus so far has been in designing the generalizations that will allow better specialization in the future.
In the next post, I’ll talk about the major new feature that inspired and leverages these changes, the River Generator…
The next development release is going to be released very soon, and with it the first in a major change of how World Machine is developed and released.
I’m going to go into some detail about something a little different — the business side of software development.
World Machine originally was released on a “big release schedule” — every ~3 years there would be a version packing a bunch of changes together and being made available, usually for an upgrade fee. Earlier this year, my frustration with this kind of development model prompted me to put effort into streamlining and adopting a new model that is becoming very popular; that of frequent updates.
Having frequent and rapid feature releases is:
Good for you: You get to use new features and fixes immediately rather than having things be finished but not used by anyone for months or years until a Big Version is released.
Good for me: I receive satisfaction when the things I’m creating make an immediate impact on how you can do your work.
Good for World Machine: Being able to quickly get feedback and iterate on ideas makes every feature significantly better.
What’s not to love? The downside is figuring out how to deal with everyone’s favorite bugaboo…
charging folks for software.
I’m going to broach this subject for a bit as the business side of software is incredibly important — it’s what puts food on the table for me personally, and allows World Machine Software to continue as a small business.
The traditional development model relies on collecting your new features into a glittering pile of goodies, which people are then asked to pay for. This works pretty well; and I have been able to devote myself full time to World Machine for a while now as a positive consequence.
However, it is fundamentally incompatible with the new feature release schedule.
If every new feature is immediately made available to current customers, there will never be a World Machine 3, since all features are instead added to the current version. That’s not quite what I had in mind! Making it even trickier is how to deal with payment for updates and upgrades. We’ve not charged anyone for an upgrade since 2008. Many of the changes since then have been bug fixes that I feel should always be made available.. but the new features since then certainly could have been part of a new release.
I’ve spent much time over the past several months thinking about ways around this dilemma, including options like feature packs, subscriptions, and more. Each route was evaluated by these two guiding criteria :
Strive to provide the most value to you the customer as possible
Produce enough revenue to allow World Machine to prosper as a business
With those in mind…
here is the outline of the direction we are pursuing. Some of these are new, and others are simply codifying the way we’ve always done things:
World Machine purchases come with one year of free upgrades. During this time every new feature or release created is available to you for free. I plan on quarterly releases of useful improvements.
Your software never expires. After your first year of upgrades is over, you will no longer receive the latest and greatest features. But unlike a subscription, nothing stops functioning. You can also access and re-download any versions you’re entitled to, whenever you like.
You can pay a small fee to keep upgrading. You can buy another years’ worth of free upgrades any time you like. The renewal price is basically the cost of a major upgrade, but spread out over a number of years.
You don’t have to stay “caught up” if you don’t want to. People using World Machine constantly for their business or pleasure will likely want to always have an active update plan, but for others, you can come and go as you please. If none of the new features being produced are compelling to you, you can wait to pay until we produce something that is.
The upgrade purchase is valid for both major and minor releases. Whether its minor improvements or something big enough to cause me to change the Big Number (ie World Machine 3), while your upgrade license is active you will receive it for no additional charge.
I believe that this represents a favorable blend of the best parts of traditional and the more modern “software as subscription” business models. I firmly believe it provides the most benefit to both you as customer, and me as developer. I hope all of you agree!
When does this start?
The first development release of World Machine 3 will be available this monday under the new model.
WM3.0.dev1 is not the end-state of World Machine 3; rather, it marks the beginning of its development. Over the next year, many new and exciting features will be added to WM3 on the dev channel.
What’s even more exciting is that you, the terrain artist, will be instrumental in guiding what features you most want to see appear in WM3. The major version number will be incremented much more quickly than before; I would hope that within another few years we will see World Machine 4 start the same development process…
As I mentioned in my last post, I’m currently doing a pass through the Instance Scatter device, making sure that it works as intended before release.
Why Fractal Distribution Rocks
Someone asked me in email what I meant about the placement device accepting a height object as a fractal basis function. Here’s a great example to illustrate:
We’re going to take this linear gradient and circular alpha mask:
And use the Instance Scatter device with its fractal distribution turned on (persistence : .5, lacunarity: .5, octaves: 8) to produce this:
When you get to examine this yourself in the next release, you’ll see it has all kinds of interesting features reminiscent of a tilted sedimentary type terrain.
This is a super quick example with pretty basic inputs, but it shows how the scatter device is useful for so much more than just placing craters and volcanos.
Here’s one more, which is just slightly modified from the above:
To be honest, I didn’t plan this custom-fractal ability originally; but once the idea to allow fractal distribution happened, it emerged. You’ll find yourself creating all kinds of interesting rock types and textures using the scatter device. Another nice perk is that these fractals are stable across space, and thus are fully tile-able and explore-able.
I fully expect to see lots of user-created custom terrain type macros grow out of this!
Some folks on the forums were asking the other day about how development progress is going. Things are getting very close now for the next Dev Channel Release. I’m doing a final methodical trek through each new feature, and writing example files to help folks learn how to use them.
Something to whet your appetite
Here’s an example image from within World Machine itself demonstrating the “Instance Scattering” device, that takes a single impact crater (defined in localspace), and scatters it across the terrain.
Most powerfully, the instance scatter device lets you treat the local object to be scattered as the basis function for a fractal; in the image above you can see that the craters exist at many scales throughout the terrain, as you’d expect from impact with space debris.
Why it takes so darn long
I hoped to have Localspace support out about 3 months ago. There are many reasons why this release has been delayed, but part of it is the following process:
Each device slowly accumulates a wishlist as I both play around with it myself, and also while I write example worlds using it. Oft times, the wishlist items would be nice to have but are not fundamental, so they go on the backburner. Other times, I’ve realized that I’ve missed a required fundamental ability. When I realize that, instead of putting it on the wishlist I dig in and add it to the feature 😉
For example, what I noticed while building example files showing how to use the Instance Scatterer is that for many uses, it is extremely important to be able to specify an elevation offset for the placement object; This offset lets you control what parts of the object will be considered above the destination terrain level, and what is below. Without offset, the instance scatterer could either build up or cut away terrain, but not both at once. In the crater example above, we’re building up the crater rim but also carving away the inside impact area. You can do that now; but a week ago, not so.
That’s it for now. just wanted to write to show that things are still happening!
Time for a sneak peek at another new WM 2.9 device: The Mimic Device.
This has been in R&D stasis for at least the last six months; I finally got a chance to finish it up this week, and I am confident in its polish level enough to slate it for inclusion in this upcoming dev channel release.
What is the Mimic Device?
The Mimic device allows you to reshape a terrain to mimic the elevation profile curve of a model reference terrain.
What does that mean? A few images might make it more clear. Let’s say we have a basic perlin noise terrain:
And we have another terrain as a model. (Edit: I’ve updated this example to be more obvious!)
Quite different from basic Perlin Noise! Nonetheless, the mimic device transforms the input terrain shown in the first image using the above as a reference… into this:
You can see that it follows the same general shape as the original input, but it matches the elevation range, and more importantly, matches the elevation profile of the model input! Note that the Mimic device cannot introduce detail that wasn’t there to start with –But it does make what is there look the same.
Here’s another example. Let’s say our model terrain was terraced (in this case, by adding a Terrace device to a Voronoi:
The terraced voronoi looks like this:
While the input terrain looks like this:
And sure enough, the terraces are replicated on the mimic!
Hopefully you can start to see how how powerful this ability could be…
The immediate use cases that I developed the device for include:
Matching disparate terrain types together so that they “fit” and transition to each other better.
Shaping terrain by exemplar (For example, provide a reference terrain from a DEM that has the kind of elevation profile character you want).
Making changes and modifications to an existing terrain that have to fit its characteristic shape.
An interesting side effect of the Mimic is that it can actually be used to “erase” changes made to a source terrain, as long as those changes don’t reverse the profile curve of the terrain. For example: In the above example world, if we moved the terrace device from the Voronoi chain to the Perlin Noise chain, we could take that resulting terraced perlin noise input and actually remove the terraces from it by using the smooth-sided terrain as a mimic reference!
This surprising power works because the Terrace device doesn’t create completely flat ledges, and so can be reversed. If that was not the case (for example, if you were trying to reverse the terraces from an 8bit input file) then the Mimic would not be effective as-is — you would have to blur the input terrain first.
Of course, nothing is perfect. What can’t the Mimic do?
The mimic device cannot replace or insert surface detail. For example, providing an eroded model reference will not introduce erosion gullies to the target terrain.
The mimic device doesn’t play especially well with Tiled Builds. It will be flagged as such in the UI (with a !). With high blending percentages it should be acceptable however.
Balanced against those relatively few caveats, however, is a lot of utility.
It’s been a little quiet here, so let me shine some light on what’s cooking at WM!
The next Dev Channel release contains major fundamental changes to the underlying device infrastructure in WM. Those changes started with multiple-res support…. but it wouldn’t be a bad idea to start giving some tours of the new features that will be in the next dev build.
What are they?
A whole previous blog post deals with localspace, but in a nutshell this allows you to create “local” packets that are not referenced to the world.
Why is this useful?
Read the previous post for more details, but this really enabled powerful (and completely new!) workflows for WM. Things like creating hero volcanos or craters in isolation and then splattering them across your terrain randomly; or creating textures for your terrain; etc etc.
Lots more will be written about this as it comes nearer. In my playing with it, it really is a game changer, opening up novel workflows in World Machine for creating terrains in a way that you really couldn’t before.
But besides localspace, what else has been added?
User Interface Improvements
Since the UI needed lots more indicators to work well with Localspace, I’ve been taking the opportunity to fix a few UI issues that have been bothering me…
Groups can set resolution/space
It can be a bit laborious to set resolutions on each relevant device by hand. Now you can set them by group:
This ability also works well with the next item…
Groups are a World Machine feature that I’m really happy with; they help you organize and make sense of your node world. I’ve seen a number of world files “in the wild” that have tried to use hierarchies of groups; unfortunately in WM 2.x it doesn’t work very well. Groups would double-move devices, not move their children, and all kinds of other headaches.
This has been corrected:
Groups now behave as you would expect when nested; moving a group moves its nested groups, etc. In general I’ve just given a second pass through group behavior to make it less likely to mess things up.
More Context Menus
The right-click behavior in the device view has been changed. Now r-click brings up a context menu for whatever you have selected; you can still flip views with the first menu entry, but you can also do other things like quickly add a device, access overrides, etc etc. I find this very useful!
“Secret” New Devices…
They are here. They are secret for now 😉
Update: I’ve never been very good with secrets. Here’s a teaser image of the Scatter device distributing a simple localspace heightfield as a random fractal:
News, Ideas, and Random Musings by the author of World Machine