Category Archives: Software Design and Programming Topics

Dev and Release: An Unexpected Tragedy

 Forwards and Backwards Compatibility

The Dev and Release branches have brought about a complication that in retrospect should not have been unexpected.

World files and macros created in the dev branch are pretty much incompatible with the release branch.

This is not good. It fragments the potential users of a macro or TMD file; Dev Branch users are unable to share any of their TMDs with normal users. We’ve already been seeing these situations crop up on the forums; what’s worse is that WM doesn’t give very good error messages when this happens, so the user doesn’t know what’s wrong — just that WM is having some kind of error for no good reason. frustrating.

This is the crowning annoyance of a longstanding problem — WM’s file-format is quite brittle. Missing a device? Can’t load the world. Saved from a new version? Can’t load it. And sometimes “can’t load it” actually means “stomp on memory and crash the program.”

I’ve been working on a solution to this problem this week,, and I’m happy to say that the solution is, in fact, already complete.

Dance of the Tags (Implementation Details)

The easiest (and most logical) way to solve this problem is to move from a versioned binary blob file to a tag-based file format, such as used by tiff or xml in text. The new file format is still a binary format, but adopts a tag-based structure that will be a bit familiar to anyone who has done html or xml document model work. Key to WM’s new file IO tags are that each tag is a hierarchical container; it contains a size field allowing us to skip to the end of the tag if we can’t parse it. WM’s old file format had absolutely no way to recover from this kind of lack of knowledge.

Anyways. What does this change buy us?

The Dev and Release builds will be able to open each others’ files and macros

Obviously any specific new features won’t work, but files that don’t use any new features will be 100% compatible with the old version.

‘Old’ World Machine versions will be able to open future versions successfully

Once this is in place, even if you are using an older version that can’t understand some new functionality you will be able to load the file. Of course, you’ll receive a warning that you’re missing some things…

Missing device plugins will never cause the world to fail to load

Instead of failing to load the world file or just plain crashing, a placeholder “Unknown Device” will be created and wired into the network.

Missing devices are marked in the workview

Of course, it won’t actually do anything and your world probably won’t work without the missing devices. But…

You may even be able to ROUND TRIP between different versions with most information intact

Devices and maybe features loaded that are not known to the current version will be kept and re-saved to disk; so that you should be able to successfully:

  1. Open a “too new” file that uses some new features. You can’t use em, but you are able to make some other changes anyways and…
  2. Save the changes to disk from your older edition. Then
  3. Open your changed file in the newer version … and have all of its original “too new to work” data still there!
Re-opened in a version with the missing plugin
Re-opened in a version with the missing plugin

That last point is really quite rare — and hard to pull off! Especially when devices, etc change their parameters and functionality. I can’t guarantee that this will always be possible for all features, but at least with current anticipated changes it’s looking like it will be.

The new file format will roll out to the Dev Build first for testing; once it’s been tested for a bit, after that it will also be incorporated into the Release build, at which point all of the above will become possible.


There’s lots of other exciting new changes afoot.

Some of them are in R&D stage so I’m not writing much about them for now, others (like rolling out localspace support to the dev build) will be coming in the next month!

The Taste of Progress

Yesterday I released 2.3.3, which marks the third revision to WM since the 2.3 release… which itself occurred only 2 months ago. 

I’m very satisfied with the new pace of progress! There are many reasons for it, a few of which I will detail below. But more than anything else, this characterizes how I really would like to develop World Machine — relatively frequent releases that bring new abilities and fix issues.

I’ve always been frustrated by the lengthy and slow release cycle that WM suffered from before. It’s not good for anyone to have the fixes for issues languishing for months or years(!) while the product is still in a beta phase before release. I think that 2.3 marked the end of the traditional development model for WM — more details to come.

The next section talks a little bit about what I’ve been able to change post-2.3 to make development faster. 

Development Details:

What did I change to be able to iterate versions much quicker? A few things:

  1. I changed to Mercurial for source control. The ability to branch and merge effortlessly has allowed me to keep feature and release branches seperate, something that was always far tougher than it should have been using the old style SVN (or nothing!).
  2. I automated much of the build process, so that kicking out a new version to the website takes maybe a half hour rather than most of a day.

Not a lot of differences, but enough to remove the friction from the process and make it easy to get changes out rapidly.

On MSVC 2010 and Sales Text

I upgraded to MSVC 2010 yesterday. Primarily for compatibility reasons — MSVC 2005 is not as common anymore and it is getting harder for people to write plugins, etc with it.

The back of the box had this beautiful feature list:

This is interesting to me as one of the things I’ve been slowly working on over the last year is doing a better job on the marketing and sales side of World Machine. There’s a common “golden rule” for writing convincing text: Tell how your product enables the user, do not just list a feature.

This is a great idea actually, as unless you know exactly what kinds of features you’re looking for, a feature list is pointless. However, the above makes me realize something:

When selling to technical people, throw out your normal rules.

I’m going to pick out a couple particularily glaring examples from above:


  1. Accelerate the coding process using your existing skills.
  2. Now practically anything is possible, virtually anywhere.
  3. Be more creative to build richer experiences for Windows
  4. Spend more time imagining the possibilities with [powerful editing tools]


None of these tell me anything at all about the product! I have no idea off the top of my head if MSVC 2010 actually has dramatically improved the coding process with an innovative new IDE, better design tools, etc…. or if they’re simply having  a creative writing session on the back of the box. And I’m inclined to naturally think the latter.

To me, this is a good example of terribly misapplied marketing. They could have sold me with C++0X standards support.. or multimonitor support, or any of the other new things. Instead I got a list of fluff.

The Lesson

If I were to try and distill down the self inflicted marketing wounds above into guidelines for myself as I pursue better marketing techniques, it would look something like this:

  1. Explain how you’ve made your users’ life easier by enabling them to do [blank] but…
  2. Also show them how/why they can achieve this result

You must satisfy both of the above at the same time to convince a technical user. Failure to follow this advice will make your carefully tweaked and sweated over sales text be simply bypassed and ignored as a “content-free zone”.

With that food for thought, I know I can certainly improve the presentation of World Machine on the website when I next have some time to devote to wearing that hat!