Category Archives: World Machine Development News

Development News concerning World Machine

Build 3022 ‘Mt Daniel’

Build 3022 became available a few days ago on the Development Channel!

The full changelist is available at the Update Center.

Taken together with all of the pretty major improvements from Build 3021, within a few weeks they will together drop to the Release Channel and be known as the ‘Mt Daniel’ release.

Mt Daniel is one of the highest mountains in the Central Cascades.  Heavily laden with snow until well into the summer, it is a popular mountain climb here in the Pacific Northwest.

There are two main focus points in Build 3022:

Snow Device

For a release named after a snowy mountain, it’s perhaps unsurprising that ‘Mt Daniel’ introduces a completely new version of the Snow device. You can now model snowy scenes in all conditions… be that be a mid-winter snowscape, or lingering snowfields dwindling away in the hot summer months.

One of the included example worlds

There are several new example worlds featuring the Snow device (including the above one) as well as the Snowmelt macro and a helpful blueprint that lets you quickly and easily add snow to any scene.

All the full details about the new Snow device are in the last blog post, check it out there! The new device is dramatically faster, more controllable, and produces better results.

Build Improvements

The second major focus of 3022 was improving build ergonomics in World Machine, especially ultra-high resolution (>16k x 16k in size).

Here’s what new with Building:

Improved Reporting

  • You now have a build log that mentions any warnings or errors that occur during a build. This includes things like : a device failing to build, out of memory errors, or performance warnings when WM has to page memory contents to disk. This should help you track down issues where a build is failing to complete / not performing as expected.
  • The build status dialog also now has an extra digit of progress precision to help give you some reassurance on those really long builds.
  • All of the slow-to-build devices have had their progress reporting reviewed and modified so that you get more consistent progress reporting… no more sitting at “0%” for 20 minutes, only to jump immediately to 100%.

Fully Cancellable

Many devices in World Machine didn’t internally cancel previously — you’d have to wait for the device to finish before the build would cancel. This was a huge problem with large and slow builds. With 3022, all devices will cancel more or less immediately.

In addition, cancelling no longer locks up the UI thread, so it won’t appear that WM has crashed while it waits for the device to cancel.

Fewer Errors, Better Handled

With very large resolution builds that pushed the resources of the machine to the max, some people would run into a fairly critical error: Device failure due to running out of memory would be silently ignored by the build process. This is bad!

The last thing you want to have happen is wait forever for that 64k x 64k  build to complete, only to find out it didn’t produce a result.

Exacerbating this, the previous Layout Generator would build shapes in parallel, which was a fine choice 5 years ago with only a few cores and moderate resolutions, but a disaster with hundreds of cores and high resolutions — with each shape generating into its own heightfield, the amount of memory required spirals out of control, often leading to the failures listed above.

This is now fixed, along with a few other issues that crop up with very-large-sized builds. For some of your ultra high res builds, this change alone will conserve dozens of gigabytes of memory!

It’s worth noting one last thing: Many of the available export file formats don’t actually allow file sizes larger than 2GB. There’s unfortunately nothing that can be done about this, except having WM warn that it cannot export to certain formats in that case. This does not happen right now — but it will in the future.

That’s it for the major changes. There are also some smaller features (like additional macro creator options, speeded-up erosion masking, etc) and of course some bugfixes listed in the full changelist below.

Happy World Machining!

 


Full Changelist:

Major improvements:
————-

* Reworked Snow Device
The new version of the Snow device offers an improved Snow model that is more intuitive, dramatically faster, consistent across different resolution builds, and produces higher quality results. It works in tandem with the new “Snowmelt” macro to easily accumulate and melt snow. See the associated blog post for more details, as well as the new example worlds and blueprint.

* Improved Build Diagnosis and Stability
– The Build report now logs any warning conditions such as out of memory, paging, etc. This will allow you to diagnose any build failures much easier.
– Build errors now stop the build instead of allowing it to proceed with incorrect results.
– Canceling a build is now MUCH more responsive. Erosion, Thermal Erosion, Snow, Blur, Lightmap, Layout Generator, and Macros all now can cancel immediately.

Minor improvements:
——————-

* The Layout generator could previously consume large amounts of memory under some circumstances. The memory requires has been reduced dramatically and additionally progress is now reported much more usefully.
* Macro creators can now specify ranges and also optional exp scaling for scalar, distance, and elevation parameters.
* Erosion now has an early-out mechanism to speed up processing when large amounts of the terrain are masked out.
* Masked mesh output is now smoother, producing additional corner triangles

Bugfixes:
———

* Fixed crash bug on loading certain worlds/macros
* Fixed crash bug that could occur with large sized builds >2GB in size
* Fixed bug in Layout view when dealing with data that is of a different sized from the world
* Fixed bug where WM would claim diskspace failure when plenty was available for builds >2GB in size.
* Fixed bug where Undo disconnected macro connections
* Fixed minor issue where panning in Device workview sometimes didn’t track properly

 

The New Snow Device

Living in the Pacific Northwest of the US is one of the biggest reasons World Machine exists. From deserts to high alpine to rain forests, we have it all – and hiking and traveling those areas provides a lot of inspiration for new devices and functionality within World Machine.

A few weeks ago I took a quick weekend backpack to Colchuck Lake, a beautiful alpine lake nestled in the central cascades. At around 6000ft of elevation, Colchuck gets buried each year under many dozens of feet of snow, and then slowly melts out well into summer.

This is what it looks like at Colchuck Lake in Real Life

Spending time camping on the snowy shore and soaking in the stunning surroundings gave me a lot of interesting ideas, but most importantly made me determined to improve the Snow device in WM.

The snow device has always been tricky to use right. Not only are the exposed parameters fiddly, but the results were often inconsistent between low and high res builds, necessitating even more tweaking and frustration. Most damning, the solution method scaled poorly, making the snow device impossibly slow in high res builds. All of these factors conspired to make it less useful than I had always hoped.

Meet the New Snow Device

The trip to Colchuck was the inspiration needed to overhaul the Snow Device.

Let’s start from the top. It’s simpler to use:

Only a few non-tweaky parameters are needed

 

But can simulate snow accumulation pretty stunningly:

Heavy snowfall in the alpine

 

Unlike the old snow device, it’s also very consistent across resolution of build:

 

Finally, and very importantly, it uses a more efficient solution to the PDEs governing the snow behavior to get some massive performance improvements:

Timing Comparison for a 2k terrain.  33x Faster!

The new Snow device is faster on small worlds and massively (hundreds-to-thousands) faster for very large worlds, where the old Snow device would simply blow up in terms of processing time required.

 

Melt your Heart Out

 

One of the major bits of inspiration I drew from the Colchuck trip was realizing that the melt-out process produces a fundamentally different looking landscape than the same amount of snow freshly fallen. Let’s look at that photo again:

Colchuck Melting (Real Life)

 

During the accumulation process, snow accumulates and softens the terrain, creating a relatively uniform and even snow surface.

During snow melt, the snow surface melts out quasi-evenly (given factors of sun exposure, temperature, etc). But since snow has accumulated to different depths on the terrain, the result is the photo above: The deep gullies and accumulation zones retain snow, while the steep, exposed, or highest areas of the terrain clear of snow early.

How WM does it

You can still specify masks governing accumulation and melt areas during the accumulation simulation – these allow you to have things like snowy peaks with temperate lowlands.

But to account for this “Winter” vs “Melt-Out” dichotomy, the melt process has been split out of the snow device itself. This is also handy because it allows you to tweak the melt levels without having to re-run the accumulation simulation.

Here’s a handy animated GIF of the melt out process, simply moving the “Melt” slider to the right:

Animation of the Melt Process

You can see how, much like the photo from Colchuck above, during meltout snow is retained in the gullies and deep snowfields while melting away from the ridges and exposed surfaces. This fast and convenient method of melting the snow lets you dial in exactly the kind of snowpack you want for your scene.

 

I’m pretty excited by the new Snow device, and the possibilities it opens up. It is not available yet on the Dev Channel, but it will certainly be in the next Dev release. That should happen in June.

Until then, enjoy this sneak peek at a revamped part of WM!

 

Build 3021

Build 3021 is available now and focuses on improving workflows in a a variety of ways.

I mentioned in the last post being interested in what would help your workflow and general usability, and in this release attempted to fix a few common pain points. The full changelist is available on the Update Center (and at the bottom of this post).

Let’s take a short tour through a few of the improvements.

EXR, JPEG Support

This one is simple but long awaited.

JPEG is now supported as a format for input and output. Even though JPEG is a lossy format, it is nonetheless a fairly common one for texture assets, reference maps, etc so it’s very useful to be able to read or write it.

OpenEXR is a different beast altogether. Besides being a long-overdue format, there are a ton of useful features that WM does not yet use, including the ability to potentially put many channels/layers into a single file. If there’s demand, the ability to do this could be added.

Perhaps more importantly, OpenEXR is a HDR format. Right now, WM just clips out of band values to 0..1. In a future release WM will support PBR workflows and HDR color values, and this will come into its own…

Output Manager and Templates

The Output Manager has been dramatically improved to be more useful:

Output Manager and File Templates

There’s a few things happening above:

  • The new Output Manager dialog is easy to use and shows every file path that will be exported
  • Filenames can now use template substitutions within Outputs. The filename for each device is being dynamically created from the device name and the world resolution.
  • Any part of the template path that doesn’t exist will be created. In this example, a “<project> Assets” folder is created to place the “<name>.r16” file within.
  • Outputs within Macros work. You can see the “Unity Export” macro contains two outputs that are exposed to the Output Manager.

Viewport Navigation

A significant amount of work went into making navigation more consistent and easier in all of the various windows in WM.

For the most part, you can continue using World Machine views the same way as you have before — the additional navigation methods are supplementary.

3D View

The new Track command makes it easy to examine areas closely
  • Orbit and Free-look modes now work together as you’d expect. Previously you more or less had to use one or the other as they interpreted the camera differently.
  • The Middle Mouse button now tracks (pans) the terrain. When viewing from the side, this allows you to raise or lower the Z value of the orbit point, something that wasn’t possible before and was sorely needed.
  • The Right Mouse button  zooms as before. However, if you zoom into the orbit point, it pushes the orbit point forward, allowing you to naturally explore the terrain this way.
  • Ctrl and Shift are now global modifiers that speed up / slow down the rate of each action.
  • All of these conventions are basically Maya-style navigation. ALT is not required however 😉

I think you’ll find its easier than ever to just navigate and not have the camera controls get in the way.

Layout View

Layout View navigation has been changed to make it consistent with the 3D view. There is a long-term plan to merge these two views together and let all layout actions happen in the 3D View, so this is an important first step to make that comfortable.

In particular:

  • Left and right click-dragging still pans the terrain.
  • The Middle mouse button now also pans the terrain.
  • ALT + RMB now zooms the same as the 3D View
  • ALT + LMB rotates the view

Device Workview

You got it — the Device workview now also follows the same convention as above (except with no rotation possible).

A few other details were ironed out, including the different roles for CTRL and SHIFT and adding a preference option for preferring to pan or box-select on left-drag — see the changelist for details.

Slope Selector

The Slope Selector has always been a little weird. Many a person has been confused by what exactly the units represent. That’s because, in a decision dating back to the very beginning of World Machine, they were “normalized slope units” — basically a number that represented a slope between 0 and the maximum that could be expressed based on the project settings.

However, that’s useful to approximately no one. So the Slope Selector has been changed to work directly in degrees.

Direct enter in degrees!

You can now also directly enter the numeric values into the boxes. Huzzah! You may also find that the resulting mask is subtly different from before — this mostly doesn’t matter, but this change is versioned because of this.

Little Things

There’s lots of little things I really enjoy personally, such as being able to Copy-Paste (ctrl-shift-V) settings into already existing devices, along with a subtle indicator to let you know if the operation succeeded or failed.

Or finally fixing a couple bugs, like Undo sometimes seemingly not responding, or devices not being rebuilt inside of macros if they weren’t connected to an output…

What’s Next

That’s all for now! This release focused on common workflow complaints; the next release sometime early this summer will be a “named” release including some major new features that I can’t wait to share with you all…

Stephen

 

 


 

The Details…

Build 3021 Changelog

New Features:
————-

* Added OpenEXR support for all inputs and outputs
32bit output for heightfields, 16bit for RGBA bitmaps.

* Added JPG support for bitmap input and output

* Slope Selector now operates in degrees and allows you to directly input numeric values.
Versioned change, right click and choose “Device Version…” to update old selectors.

* Output Manager revamped and dramatically improved.
– Edit all outputs in a grid
– Easily see the file paths for all outputs
– Outputs within Macros are supported and easy to access in normal and tiled builds

* Template system added for output filenames
– You can use template names such as <name> or <res> to name your files. For example, “<name><res>.png” is a valid template name.
– Create folders as part of the output name. For example, “<project> Assets\<name> <res>.exr” is a valid name and will create a “ProjectName Assets” folder that it outputs the file to.

* Copy/Paste Settings
You can now use ctrl-shift-V to paste settings into a device. Useful when you need to clone settings into a device already hooked into the network.

* Revamped viewport navigation, added Maya-style controls
– Maya-style (LMB=Orbit, MMB=Track, RMB=Zoom) supported.
– Reworked 3D View Orbit vs Free Look to easily be able to switch back and forth
– 3D View, Layout View, and even the Device Workview now all support the same consistent navigation system.
– Previous navigation methods continue to be supported.

Small improvements:
——————-

* Curves device now shows a histogram of the input behind the curve
* Added option for prefer pan or prefer select in workview
* Cleaned up shift vs ctrl in device workview. They now have distinct roles: shift means to add instead of replace the current selection, while ctrl accesses the alternate LMB role (either panning or box selecting)
* Improved quick-connect mode (ctrl-drag on a device to start wiring from its primary output; keep ctrl held down and the same will occur for the next device in the chain, etc)
* Fastwire now works by ctrl-dragging on a device
* Autosave now preserves the original filename
* Tiled builds do additional validation of sufficient disk space, writeable folders, etc before building, and reporting also improved.

Bugfixes:
———

* Meshes now always show at their correct size regardless of export sizing setting
* Mesh reduction during tiled builds no longer causes gaps in the edges
* Macros now build their internals properly even when those devices are not connected to an output.
* Fixed garbage character bug for Macro Parameter device
* Fixed bug where Undo would seem to have no affect for several attempts