The latest release of Mercurial and TortoiseHg, a set of Windows shell extensions and GUI tools for Mercurial, was just released. This latest version (v1.5 and v1.0, respectively) includes both tools integrated into an MSI installer built with WiX. Previous versions were built with (the excellent but non-MSI-based) Inno Setup toolset. The WiX-based installer has the advantage of using a merge module for TortoiseOverlays (login as guest with an empty password), an icon overlay handler shared by shell extensions for several different version control systems: TortoiseHg for Mercurial, TortoiseSVN for Subversion, TortoiseCVS for CVS, TortoiseBZR for Bazaar, TortoiseGit for Git, and probably others.

As usual, I opened the TortoiseHg .msi in Orca before installing it on my workstation. I was pleasantly surprised by the low number of custom actions. I ran it through WiX v3.0’s Smoke.exe to run ICE validation; there were lots of errors and warnings but all but a few were from TortoiseHg’s use of the Visual C++ runtime libraries merge modules. (Is it ironic or just plain sad that they have so many ICE errors? Maybe both.)

Though we don’t use Mercurial for WiX version control—yet, anyway—both SourceForge.net and Codeplex support Mercurial for a project’s VCS.

• Read the Mercurial v1.5 release notes.
• Read the TortoiseHg v1.0 release notes.
• Download the combined Mercurial and TortoiseHg installer.

Last October, the Giving Campaign auction had several regional offerings. One of them was the use of the “NERD treehouse” for a month as a private office. The treehouse is a small conference room on the 11th floor, above another small conference room on the 10th floor nicknamed the Jungle Room for its décor. Though I appreciate the collaborative effects of working in a bunch of cubicles, I…oh forget it, cubes are evil. No limits and I won. Even if just for a month, it’s worth it. Today was my first day in the treehouse. David, the co-creator of Lux, dubbed it The Gazebo of Solitude and the name stuck.

Here’s what it looks like from the outside:

Here’s one angle looking out on a snow-free but cloudy afternoon:

• App-V 4.6 is now available, plus some new Resource Kit tools
• App-V 4.6 RTM and MED-V 1.0 SP1 RC are here!!
• Office 2010 and AppV 4.6

Also released is the Microsoft Application Virtualization SFT View tool that is part of the App-V Resource Kit that might or might not have my fingerprints all over it.

MDOP is also available for MSDN and Technet subscribers for development and evaluation.

As always, seeing software I worked on in users’ hands is the best part of coding. Until it ships, it’s just source code.

Unit-testing custom actions with Lux

Custom actions are a frequent cause of installation failures so it’s important to test them thoroughly. Custom actions themselves usually aren’t tested. The traditional testing approach is to run functional tests on an entire installer and to cover as many scenarios and platform combinations as possible.

Custom action patterns

WiX compiler extensions provide one way of improving custom action quality: Because compiler extensions run at build time instead of install time, they can perform all sorts of data validation and conversion on strongly-typed authoring before converting it to rows and columns of custom tables in the MSI package.

Immediate custom actions then read those custom tables, check current state (for example, component action state, the state of the machine itself), and serialize the resulting data into a custom action data property. Immediate custom actions are the place to do the logic that needs live state and cannot be determined at build time by a compiler extension. Because immediate custom actions run in the security context of the installing user and outside an installation transaction, they generally do not have permissions to modify the machine and if they fail, the installation simply ends without the need to do any cleanup or rollback.

Deferred custom actions read the custom action data property set by immediate custom actions to know what to do. One way to improve custom action reliability is to make as few decisions as possible in deferred custom actions; instead, implement all the logic in compiler extensions and immediate custom actions and have deferred custom actions simply read the custom action data property in a loop to modify the machine.

The WiX custom actions that modify the machine use this pattern. For example, XmlConfig authoring is validated by the WixUtilExtension compiler extension and translated to rows and columns in the XmlConfig table. The SchedXmlConfig immediate custom action reads the XmlConfig table, constructs a custom action data property based on the XmlConfig table and machine’s state (including checking component state and storing existing file data to support rollback), then schedules the ExecXmlConfig deferred custom action to execute the XML changes and the ExecXmlConfigRollback rollback custom action to roll back the changes.

Testing with Lux

Lux is a WiX extension (and associated tools) that let you write data-driven unit tests for your custom actions. The executive summary: Lux runs your immediate custom actions then validates they set properties to the values you expect.

While it’s a simple approach, if your custom actions are factored as discussed above, validating the properties set by immediate custom actions can validate all the interaction between your custom actions, the MSI package, and MSI itself.

If your custom actions aren’t factored as discussed–for example, if your deferred custom actions expect only an installation directory and have logic to construct file paths from it–then it’s likely that your immediate custom actions don’t have a lot of logic that’s useful to test.

Lux does not help you test the custom action code that actually modifies the machine; for that, continue to use other unit-test frameworks and automated tests. By working only with immediate custom actions, Lux can let MSI run the custom actions as-is, eliminating the need to write custom test doubles for the MSI API. Lux runs from a per-user package so unless you run the tests from an elevated command prompt, none of the custom actions get elevated privileges and therefore cannot modify the machine.

Here’s how Lux works:

1. You write your unit tests using XML in WiX source files.
2. The Lux extension converts the XML to a table in a test .msi package.
3. The Lux custom action runs after all other immediate custom actions and evaluates your unit tests.

Authoring unit tests

Lux supports the following unit tests:

• Property values
• Expressions
• Multi-value properties
• Name/value-pair properties

Note that you should always author unit tests in fragments separate from your custom action authoring or any other product authoring. If you mix unit tests with other authoring, WiX includes the unit-test data in your "real" installers.

Property value tests

A simple test lets you specify a property to test, a value to test against, and the operator to compare with (which defaults to "equal").

<Fragment>
  <lux:UnitTest CustomAction="TestCustomActionSimple" Property="SIMPLE" Value="[INSTALLLOCATION]" Operator="equal" />
</Fragment>

When the test runs, Lux compares the value of the SIMPLE property against the (formatted) value [INSTALLLOCATION]. If the two match (because the operator is "equal"), the test passes. Legal values of the Operator attribute are:

equal

(Default) Compares Property to Value and succeeds if they are equal.

notEqual

Compares Property to Value and succeeds if they are NOT equal.

caseInsensitiveEqual

Compares Property to Value and succeeds if they are equal (ignoring case).

caseInsensitiveNotEqual

Compares Property to Value and succeeds if they are NOT equal (ignoring case).

Test conditions

Conditions let you validate code paths in your custom action. For example, if your custom action behaves differently on Windows XP than it does on Windows Vista and later, you can create two tests with mutually exclusive conditions:

<Fragment>
  <lux:UnitTest CustomAction="TestCustomActionSimple" Property="SIMPLE" Value="[INSTALLLOCATION]">
    <lux:Condition><![CDATA[VersionNT < 600]]></lux:Condition>
  </lux:UnitTest>
  <lux:UnitTest CustomAction="TestCustomActionSimple" Property="SIMPLE" Value="[INSTALLLOCATION]">
    <lux:Condition><![CDATA[VersionNT >= 600]]></lux:Condition>
  </lux:UnitTest>
</Fragment>

If a test has a condition, the test runs only if its condition is true.

Expression tests

Expression tests let you test any valid MSI expression. If the expression is true, the test passes. If the expression is false or invalid, the test fails.

<Fragment>
  <lux:UnitTest CustomAction="TestCustomActionSimple">
    <lux:Expression>NOT MsiSystemRebootPending AND SIMPLE</lux:Expression>
  </lux:UnitTest>
</Fragment>

Multi-value property tests

Because deferred custom actions can access only a single custom-action data property, custom actions that need more than one piece of data encode it in a single string. One way is to have the immediate custom action separate multiple elements with a known separator character, then have the deferred custom action split the string at those separate characters. Lux supports such separators using the ValueSeparator and Index attributes.

<Fragment>
  <lux:UnitTest CustomAction="TestCustomActionMultiValue" Property="MULTIVALUE" ValueSeparator="*">
    <lux:Condition>VersionNT</lux:Condition>
    <lux:UnitTest Index="0" Value="1" />
    <lux:UnitTest Index="1" Value="[INSTALLLOCATION]">
      <lux:Condition>NOT Installed</lux:Condition>
    </lux:UnitTest>
    <lux:UnitTest Index="2" Value="WIXEAST" />
  </lux:UnitTest>
</Fragment>

A condition under the parent UnitTest element applies to all individual unit tests. Override it with a Condition child element.

Name/value-pair property tests

Another way of providing multiple values to a deferred custom action is to combine name/value pairs into a single string. Lux supports name/value-pair properties using the NameValueSeparator and Index attributes.

<Fragment>
  <lux:UnitTest CustomAction="TestCustomActionNameValuePairs" Property="NAMEVALUEPAIRS" NameValueSeparator="#">
    <lux:UnitTest Index="InstallationRoot" Value="[INSTALLLOCATION]" />
    <lux:UnitTest Index="Developers" Operator="caseInsensitiveNotEqual" Value="WIXEAST" />
  </lux:UnitTest>
</Fragment>

Building test packages

Lux unit tests run from a minimal package that includes just your unit tests and the resources they need to run. Because Lux runs only immediate custom actions, it doesn’t need a full, per-machine package that includes all the files and other resources to be installed. Such a minimal package saves build time but does require that your WiX source code be well modularized with fragments. For example, you should always author unit tests in fragments separate from any other authoring. If you mix unit tests with other authoring, WiX includes the unit-test data in your "real" installers. Likewise, any other WiX authoring included in unit-test fragments is included in test packages.

Lux comes with a tool that simplifies the creation of test packages. Its name is lux.exe. To use lux.exe:

1. Compile the source file containing your unit tests.
2. Run lux.exe on the .wixobj file and specify a source file for the test package.
3. Compile the test package source.
4. Link the test package .wixobj with the unit tests .wixobj.

For example:

candle -ext WixLuxExtension CustomActions.wxs
lux CustomActions.wixobj -out LuxSample1_test.wxs
candle -ext WixLuxExtension LuxSample1_test.wxs
light -ext WixLuxExtension LuxSample1_test.wixobj CustomActions.wixobj -out LuxSample1_test.msi

Lux also includes an MSBuild task and .targets file to let you build test packages from the same .wixproj you use to build your installers. To build a test package, build the BuildTestPackage target using MSBuild 3.5:

%WINDIR%\Microsoft.NET\Framework\v3.5\MSBuild.exe /t:BuildTestPackage

Running unit tests

After building the test package, you can run it with logging enabled to capture test results:

msiexec /l test1.log /i bin\Debug\LuxSample1_test.msi

Search the log for WixRunImmediateUnitTests to see test results and other logging from the Lux custom action.

Nit: The Lux test runner

Lux also includes Nit, a console program that monitors the logging messages emitted by unit tests and reports success or failure. To use Nit on your test packages, just specify their filenames as arguments to nit.exe. For example:

nit LuxSample1_test.msi

Lux also lets you run Nit on your test packages from the same .wixproj you use to build your installers. To run a test package under Nit, build the Test target using MSBuild 3.5:

%WINDIR%\Microsoft.NET\Framework\v3.5\MSBuild.exe /t:Test

The test package will be built before the tests are run, if necessary. The output looks like the following, with failing tests highlighted in red as build errors:

Test:
  Microsoft (R) Windows Installer Xml Unit Test Runner version 3.5.1204.0
  Copyright (C) Microsoft Corporation. All rights reserved.

  Test luxB21F0D12E0701DBA30FFB92A532A5390 passed: Property 'SIMPLE' matched expected value '[INSTALLLOCATION]'.
  Test TestConditionBeforeVista passed: Property 'SIMPLE' matched expected value '[INSTALLLOCATION]'.
  Test TestConditionVistaOrLater passed: Property 'SIMPLE' matched expected value '[INSTALLLOCATION]'.
  Test TestExpressionTruth passed: Expression 'NOT MsiSystemRebootPending AND SIMPLE' evaluated to true.
nit.exe : error NIT8103: Test luxA6D27EC5903612D7F3786FF71952E314 failed: Property 'MULTIVALUE' expected value '2' but actual value was '1'.
  Test lux210257649C16AFA33793F1CDDF575505 passed: Property 'MULTIVALUE' matched expected value '[INSTALLLOCATION]'.
nit.exe : error NIT8103: Test lux402940A90D3ADAD181D599AB8C260FA0 failed: Property 'MULTIVALUE' expected value 'xxxWIXEAST' but actual value was 'WIXEAST'.
  Test lux453EC8DB458A8F66F0D22970CFF2AE99 passed: Property 'NAMEVALUEPAIRS' matched expected value '[INSTALLLOCATION]'.
  Test lux20CB4F88795F22D15631FD60BA03AFEB passed: Property 'NAMEVALUEPAIRS' matched expected value 'WIXWEST'.
nit.exe : error NIT8102: 2 tests failed. 7 tests passed.
Done Building Project "C:\Delivery\Dev\wix35\src\lux\samples\LuxSample1\LuxSample1.wixproj" (Test target(s)) -- FAILED.

Build FAILED.

"C:\Delivery\Dev\wix35\src\lux\samples\LuxSample1\LuxSample1.wixproj" (Test target) (1) ->
(Test target) ->
  nit.exe : error NIT8103: Test luxA6D27EC5903612D7F3786FF71952E314 failed: Property 'MULTIVALUE' expected value '2' but actual value was '1'.
  nit.exe : error NIT8103: Test lux402940A90D3ADAD181D599AB8C260FA0 failed: Property 'MULTIVALUE' expected value 'xxxWIXEAST' but actual value was 'WIXEAST'.
  nit.exe : error NIT8102: 2 tests failed. 7 tests passed.

    0 Warning(s)
    3 Error(s)

Time Elapsed 00:00:07.87

FAQ

Are these really unit tests? They look a lot like Fit tests.

Fit tests are tabular and data-driven, so they have a lot in common with Lux’s unit tests. But fit tests are focused on high-level outputs, whereas unit tests are low-level developer tests.

Using the custom action code as-is sounds good, but are there any limitations with that approach?

Yes. Because you are running the actual custom action, any code paths that rely on machine state reflect the state of the machine you run the tests on. For example, code that has different behavior on different versions of Windows runs only one way, just like it does in a normal installer. You can add debug code that looks for the presence of the WIXLUXTESTPACKAGE property; it’s set to 1 in a test package.

I have unit tests that fail because directory properties are being returned as empty strings. Why?

The most likely cause is that your directories are defined as children of your installer’s Product element. Lux.exe builds its own Product element to product a minimal test package, so none of the resources defined in your Product are available to the unit tests. The simplest solution is to move those resources to their own Fragment.

Do I have to write my custom actions in C++?

No, Lux works with any immediate custom actions, regardless of the language they’re written in, including MSI type 51 property-setting custom actions.

<!– Major upgrade –>
<Upgrade Id="$(var.UpgradeCode)">
    <UpgradeVersion Minimum="$(var.ProductVersion)" OnlyDetect="yes" Property="NEWERVERSIONDETECTED" />
    <UpgradeVersion Minimum="1.0.0" IncludeMinimum="yes" Maximum="$(var.ProductVersion)" IncludeMaximum="no" Property="OLDERVERSIONBEINGUPGRADED" />
</Upgrade>

<InstallExecuteSequence>
    <RemoveExistingProducts After="InstallValidate" />
</InstallExecuteSequence>

<Condition Message="!(loc.NewerVersionDetected)">
    NOT NEWERVERSIONDETECTED
</Condition>

Easy enough to cut and paste but it isn’t exactly what I would call expressive—the intent behind the code isn’t obvious, though it can be deduced fairly easily.

WiX can do better.

So, like the last time I tweaked the WiX language, it’s time for a new feature: The MajorUpgrade element encapsulates the most common options for major upgrades and creates the appropriate rows in the Upgrade and LaunchCondition tables and provides a simpler way of specifying the scheduling of the RemoveExistingProducts action. For example, here’s the simplest use:

<MajorUpgrade DowngradeErrorMessage="Can’t downgrade." />

Downgrades are blocked by default, which requires you to specify a message for the launch condition message.

MajorUpgrade is a child of the Product element and automatically picks up the UpgradeCode attribute from its parent. That avoids the common duplication of the upgrade code in the Upgrade element.

MajorUpgrade has the following attributes:

For details, see the MajorUpgrade element documentation. The feature ships in WiX v3.5.1315.0 and later.

• The Component/@Guid attribute value defaults to “*” so that if you don’t specify it, WiX generates a stable component GUID at link time. Generated component GUIDs are available as long as your component:
  • Does not contain an ODBCDataSource element. (ODBwhat?)
  • Has only one file.
  • Has only registry values.
• The Component/@Id attribute value defaults to the id of the keypath resource. The component itself cannot be the keypath for this to work (obviously, there’s no id for it to default to).

These changes will appear in the first weekly release of WiX v3.5 in 2010.

Combine the two features and a single-file component can now be as simple as:

<Component>
  <File Source=”foo.exe” />
</Component>

In this case, the component’s id will be “foo.exe” because the File element’s default for its Id is the filename portion of the Source attribute. Call it hygenic double-dip defaulting.

A multi-file component isn’t suitable for generated GUIDs, so it requires an explicit GUID but can still take advantage of default component ids:

<Component Guid=”{A5B56773-5E26-4C5F-AC51-C2470C3658AF}”>
  <File Source=”foo.dll” />
  <File Source=”bar.dll” />
  <File Source=”bob.exe” KeyPath=”yes” />
</Component>

In this case, the component’s id will be “bob.exe” from the keypath File element’s default Id. Note that unless you use the generated GUID default and live within its rules, you must specify an explicit KeyPath attribute value of “yes” on the resource whose id you want to be the component id.

We can’t eliminate the dread component rules but we can make it simpler to live within them.

Candy, Visual Studio program manager and WiX Working Group babysitter/cat-herder, had this to say about the features in this release:

Visual Studio 2010 Beta shipped on 10/21. With the team’s hard work and agility, we were able to release the WiX support on VS2010 Beta 2 today! Here are some highlights for this release:

• Extension Manager Integration – Now you can download and install WiX directly from within the VS IDE by launching Extension Manager
• Major performance improvement during project building – No more IDE freezing during build
• Automatic upgrade from VS2008 to VS2010 – no more manual editing of wixproj files to change the tools version
• Improved IDE experience including drag/drop of nodes, copy/paste of items, and project references
• Added Visual Studio 2010 Express products detection in the WiX VS Extension – New in Beta
• Over 40 + bug fixes

I’m pleased we were able to get a release supporting beta 2 out so quickly and I’m very excited about the “no more IDE freezing during build” improvement. There are several other smaller enhancements that will make WiX v3.5 a no-brainer upgrade when it ships next year.

In the meantime, if you’re using Visual Studio 2010, please grab this build and be on the lookout for future weekly releases. Problems, questions? Send mail to wix-users, file bug reports, or submit feature requests.

OK, so driving was not in the picture for my daily commute to the Fortress of NERDitude. Currently, my daily commute takes about half an hour of waiting for a bus or a train and sitting/standing while someone else does the driving. Doing nothing is much more relaxing than driving in Cambridge traffic, of course, but even with good music, doing nothing gets boring pretty quickly.

I could just bring along my trusty Nintendo DS but I figured that would get boring eventually too. Plus, I figured I could be a bit more mature than that and do something, I dunno, productive like read mail or blogs…

I considered just browsing the Web on my phone, but rejected that idea:

• Web latency over 3G networks is painful.
• 3G networks aren’t terribly reliable underground.
• Web UIs are painful on small screens.

A laptop or even UMPC is out, because I knew I’d frequently need to work one-handed, keeping the other handy for not falling over during the less-than-smooth parts of the Red Line.

No, I needed a small, handheld device that could sync mail and blogs and let me read them offline.

No, it’s not an iPhone

After some research, I bought an iPod touch. (Perhaps in a couple of weeks, my implant will kick in and I’ll do this all again on a Zune HD.) I also bought Phantom Fish’s Byline RSS reader, which syncs to Google Reader. Once I supplied my account name and password, reading blogs using Byline is easy:

• Tap the sync button to download the latest blog posts.
• Read posts and mark some for later follow-up once I’m back online.
• Before heading home, sync to update Google Reader with the posts I read and to download the latest blog posts.
• Once I’m back in range of my home WiFi, sync again.

Please, sir, may I have some more?

Buying apps from Apple’s App Store is simple; the hardest part is typing a strong password into the on-screen keyboard:

• Start the App Store app.
• Search or browse for an app.
• Tap the price and it turns into a BUY NOW button.
• Tap BUY NOW.
• Enter your iTunes password.
• The app is downloaded and installed.

Updates are handled in much the same way:

• Start the App Store app.
• Tap the Updates button.
• If there are any updates, tap the Update All button to download and install the updates in the background.

Because I have an iPod touch instead of iPhone, all my interaction with the App Store happens over WiFi at respectable speeds. (Even better, it doesn’t require interaction with iTunes, which continues to amply demonstrate that Apple has yet to master setup development on Windows.)

I’ve purchased many apps since buying my iPod touch and “purchased” many more free apps. In all cases, downloading and installing those apps was painless.

Knew you’d get to installation eventually

Well, yeah. Here are my takeaways; yours might differ:

• Mobile != connected. Even though 3G coverage is decent from the major carriers, there will always be dead zones, slow zones, and underground zones.
• Rich clients + local processing power == goodness. Sure, Web 2.0 and HTML5 will finally kill the idea of running local apps. Bah. Latency, especially problematic with 3G networks, kills the user experience (see, for example, About Face and other UX books). Rich clients can largely avoid the problem.
• Sync data that lives in the cloud. If your data lives in the cloud, rich clients can sync it, keeping multiple devices up to date. In my case, I prefer to use a laptop or desktop PC to read blogs and mail when I can because, not surprisingly, a big screen is better for reading (or scanning) lots of text.
• Bulletproof installations rock. Would anyone disagree it’s better to have bullet-ridden installation?
• Making it easy to buy apps makes it easy to make money. The App Store had “over 1 billion downloads in just nine months” and “1.5 billion in First Year.” Naturally, with so many free and almost-free apps in the App Store, it doesn’t necessarily follow that we’re talking about a lot of money, depending on your definition of “a lot,” as estimates put the revenue at $2.4 billion a year. But imagine extending the App Store experience to more expensive apps, say on your PC.
• Making it easy to service apps makes it easy to keep customers happy. Servicing deliverables themselves aren’t too hard to get right but making users aware of them and getting them to the user still is. It’s led to way too many auto-start local-system updater services one vulnerability away from total system p0wnage.

Best of both worlds

One of the big reasons Web apps became so popular, for both consumers and the enterprise, is that they don’t suffer from the reliability problems common to installing client apps. Naturally, these aren’t inherent problems with client app installation but are complexities that easily turn into problems—especially without engineering attention to setup.

Web apps avoid the problem by not installing on the client. (Note that I’m skipping the complexities of setting up Web servers and the pseudo-installation provided by Silverlight and Flash.)

iPhone/iPod touch apps avoid the problem by:

• Working in a limited number of well-known environments. (Though the iPhone 3GS adds a second hardware base and there are many rumors about a so-called “Apple tablet.”)
• Working in a locked-down environment.
• Requiring a particular deployment model.
• Supporting a simple update model.
• Requiring review and approval before getting published.

These might seem pretty limiting to most Windows developers. But it’s hard to argue against the success the App Store has achieved because of—probably not in spite of—those limitations.

• Microsoft Visual C++ 2008 Service Pack 1 Redistributable Package ATL Security Update, which contains upgrades for the Visual C++ runtime assemblies. There are three, one for each platform (x86, x64, and IA64), each under 5MB.
• Visual Studio 2008 Service Pack 1 ATL Security Update, which contains a 365MB patch for SP1.
• Wait.
• 365MB? As in megabytes?
• Oh yes.

I’m downloading this once and sharing it on my network rather than downloading it from Microsoft Update n times.