MSIX: Slightly more detail about what might be something huge but probably won’t be

Over on his blog, Rob wrote about his guesses for what the still-somewhat-mysterious MSIX means for Windows deployment. We got some details at the Build conference last week. The two most-detailful were

Here are my key takeaways from these presentations.

MSIX: One package format to rule them all

Appx packages today are Open Packaging Conventions Zip files with a manifest. With the (sometimes painful) magic of XML namespaces, a single manifest can cover many different types of apps: Witness the sometimes-significant differences in the manifests of UWP apps and Desktop Bridge apps. And, of course, Zip files let you easily include different sets of files, like the registry hives in Desktop Bridge app packages.

In MSI terms, the tables that make up the MSI database are represented in special “footprint” files, like AppxManifest.xml and additional well-known files, like Registry.dat. Installed files for an MSI package are typically stored in a .cab file, either embedded or stored next to the .msi file. In Appx, the installed files are stored as normal Zip file entries in the .appx file.

The MSIX Packaging SDK — sometimes named just the “MSIX SDK” — contains MIT-licensed code that implements the Appx packaging APIs to build .appx and .appxbundle packages. If you’ve spent any time building Appx packages, you understand that Microsoft open-sourcing this code is a pretty big deal, if only because the error codes from the API functions are truly terrible and now we can debug the code to better understand error states.

On a slightly more serious note, the MSIX packaging API has several interesting features, like supporting signing and the package “block map,” which is what makes it possible to do differential downloads of package contents.

So that’s all cool but at the end of the day, what we heard about the MSIX format — being cross-platform and available on Windows 7, for example — is essentially saying that Zip files make a good package format because they can contain files (including manifests) and they work everywhere. Well, yes, that’s true but not exactly an earth-shattering pronouncement, is it? A file format is necessary, sure, but what matters is the engine the interprets the contents.

MSIX: One engine to rule them all

MSIX packages, like UWP Appx packages, Desktop Bridge Appx packages, and even MSI packages, are just data. It takes an engine to open a package, read its metadata (manifests or tables and rows), and apply it to the machine.

At Windows Developer Day, MSIX was described as “.appx + MSI.” The mention of “MSI,” combined with repeated mentions of “containers” and tidbits like “Supports all Windows applications” led me almost immediately to assume that MSIX included the use of App-V.

Nine years ago, I joined the App-V team, then in Cambridge (the one in the United States, not the original. One of the big reasons was that I saw a lot of potential in the App-V container model of delivering software. Bringing that capability to ISVs would be a great way to make better installers for everybody. Unfortunately, that didn’t happen while I was there and hasn’t since. But maybe that was about to change…

App-V already supports Win32 features like services that Desktop Bridge does not support (today). App-V already has extensive support for enterprise IT management. App-V has a long history of supporting virtualization of apps that weren’t designed for it.

I’d so thoroughly convinced myself App-V was part of MSIX that I started writing a blog post to parallel Rob’s. Luckily, I didn’t finish it because MSIX is not App-V. There was no mention of App-V at Build except for how it wasn’t going to be deprecated…yet. Naturally, I was disappointed…mostly at having guessed wrong.

Everything discussed at Build points to the engine underlying MSIX on Windows being Desktop Bridge. However, remember that Microsoft has enhanced Desktop Bridge functionality with every release of Windows 10 since the Bridge was introduced in Anniversary Update. So even if today Desktop Bridge doesn’t support some Win32 feature doesn’t mean it never will.

But it also means that MSIX isn’t a revolutionary new engine (or old engine, for that matter) with unexpected power. And unless someone writes such an engine, MSIX isn’t going to be a “thing” on Windows 7. You can build MSIX packages on Windows 7 (and Windows 8.1 — did everyone forget about that one?) but with no runtime engine, they’re just Zip files.

MSIX: Welcome to the evolution

If you found the idea of “.appx + MSI” compelling, well, congratulations, you’re quite the nerd. (Join the club. We have jackets.) The reality, at least as described in Build talks, paints MSIX mostly as a rebranding of Desktop Bridge. That’s not a bad thing but I was expecting more given the buildup.

We’ll likely see more details — maybe we’ll see some exciting ones? — leading up to the next release of Windows 10 later this year.

Easy pure-WiX patching with Melt

“Pure WiX patching” is a godsend, if you’ve ever wrestled with the painfully useless errors common with MsiMsp.exe/PatchWiz.dll. However, pure WiX patching follows a different model than MsiMsp. One wrinkle a lot of people run into is that the loose files for the target and upgrade packages are expected to be available at the paths as written into the .wixpdb files you’re using to build the patches from. That works without additional effort only if the paths in the .wixpdbs differ between the builds and are available from the machine building the patch, such as if the build output is available in a versioned directory on a file server. Typically, the paths are instead local to a build machine and aren’t valid months later, even on the same build machine.

MsiMsp/PatchWiz requires the use of administrative installations, which are essentially an .msi package “unzipped” into a layout. On the plus side, it means you simply need the .msi package (and any external .cab files), which hopefully you kept a copy of, after shipping it to customers. One negative is that even though creating an admin installation just writes files to a directory, it requires elevated permissions, which is unpleasant in a build tool.

In WiX v3.6, I added functionality that lets you get the best of both worlds: Pure WiX patching without having to keep whole build trees around forever. All you need is the .msi package you shipped, the .wixpdb that goes with it, and Melt.exe.

Isn’t Melt for merge modules?

The Melt tool originally had one purpose: To decompile a merge module (.msm file) into the equivalent WiX authoring as a ComponentGroup. Melt lets you accept merge modules if they’re the only way you can get some bit of setup but build them as normal WiX fragments instead of forcing WiX to use the black box of mergemod.dll from Windows.

During the WiX v3.6 development cycle, I was working on patching and ran into the problem I talked about a couple paragraphs ago: Even though I had access to the full build tree, the .wixpdbs contained paths that were local to the original machine that built the product .msi packages, not the tree copied to the drop server.

It turns out that these two problems are very similar, at least in implementing a solution. In addition to Melt.exe supporting an input of a merge module and an output of a WiX source file, I added a new mode: inputs of the .msi and its matching .wixpdb files and as output, the directory where the .msi contents are extracted and a new .wixpdb that’s a copy of the input .wixpdb updated to point to the extracted .msi contents.

Here’s a typical command line:

Melt.exe ProductV1.msi -out %INTERMEDIATE_DIRECTORY%\ProductV1corrected.wixpdb -pdb ProductV1.wixpdb -x %INTERMEDIATE_DIRECTORY%\ProductV1bits

Repeat for ProductV2.msi and pass ProductV1corrected.wixpdb and ProductV2corrected.wixpdb to Torch.exe on its way to Pyro.exe.

Melt.exe doesn’t require elevated permissions. The only extra thing you need is the .wixpdb created by Light.exe when the original .msi package was built.

Getting shelf space in the Store

Reading the Windows Store for developers blog recently, I was pleasantly surprised to see that desktop (i.e., non-Metro-style) apps would be allowed to show up in the Windows Store. Desktop apps won’t get the same treatment as Metro-style apps, of course; instead of being able to install apps right from the Windows Store, desktop apps will have a similar listing page with a link to the developer’s site to handle download.

Still, it’s better than nothing — the Windows Store is likely to have a lot of window shoppers looking to buy apps.

However, a hurdle to get an app listing page is that the app must pass “certification.” In the past, this was known as the Windows Logo program and had many pages of technical requirements. In Windows 8, the list is much smaller and less prescriptive. For example, the Logo program for previous versions required the use of MSI, then required MSI or ClickOnce; for Windows 8, there are no technology restrictions.

Some requirements that struck me as interesting:

1.2 Your app must not take a dependency on the VB6 runtime

VB6, IE6 — anything v6 must go away. 🙂

5.1 Your app must properly implement a clean, reversible installation

If the installation fails, the app should be able to roll it back and restore the machine to its previous state.

If Windows 8 certification is anything like previous versions’, it will involve automated test suites that will highlight rollback that’s less than perfect.

5.4 Your app must never block silent install/uninstall

So don’t throw UI from custom actions and don’t rely on being able to prompt the user (i.e., also don’t break requirement #5.1).

6.1 All executable files (.exe, .dll, .ocx, .sys, .cpl, .drv, .scr) must be signed with an Authenticode certificate

MSI packages aren’t mentioned. But if you ship a Burn bundle or other bootstrapper .exe, that’s going to require signing.

10.1 Your app must be installed in the Program Files folder by default

There’s nothing mentioned that would exempt per-user apps in general from this requirement, though you can always request exceptions. Given the emphasis Metro places on per-user, restricted apps, perhaps this requirement will be extended/relaxed for per-user desktop apps too.

10.6 Your app must write user data at first run and not during the installation in “per-machine” installations

Long-time readers of wix-users will recognize this as advice given again and again. Good to see Windows catching on. 🙂

10.7 Exceptions and Waivers

A waiver is required for apps that write to the global assembly cache (GAC) .NET apps should keep assembly dependencies private, and store it in the app directory unless sharing an assembly is explicitly required.

I suspect it’s a response to the GAC getting used too often when it’s not needed or very useful. (See also Rico Mariani’s blog post on using NGen, which falls into the same boat.) Still, I’m mildly surprised to see it a requirement.

12.5 App running under the WoW64 emulator should not attempt to subvert or bypass Wow64 virtualization mechanisms

This one’s interesting, depending how struct “subvert” turns out to be. For example, several WiX custom actions make fairly trivial “subversions” of WoW64 to avoid the need for both 32-bit and 64-bit custom actions just to write to the right version of Program Files.

All in all, there’s nothing terribly surprising in this set of requirements. A lot of them just codify what’s already known as best practice for Windows apps in the age of UAC. Of course, this is just the first version of the certification requirements and Windows 8 isn’t yet shipping; there’s plenty of time for Microsoft to add, drop, or change requirements. In the end, you get to decide whether the effort of certification is worth having a presence in the Windows Store. Remember the Metro-style app folks, who have no choice in the matter: The Windows Store is the only way they can ship.

Same as it ever was

Microsoft’s BUILD conference is going on this week and it’s chock full of new information about Windows 8 (or whatever they end up calling it). The biggest news is the new Metro-style application, which includes the AppX package format for installing them.

There are three primary buckets applications fall into in the brave new Windows 8 world:

  1. Metro-style
  2. Windows 8 non-Metro-style
  3. “Retro-style” (aka legacy, anything created before 2011)

Metro-style apps get all the goodies Windows 8 offers, including AppX packaging I’ll blog about. As Rob describes, deployment is a first-class citizen in Windows 8. Apps in the other two buckets have the exact same options they have in Windows 7. And by exact, I mean “really, almost exactly 100 percent.”

No soup for you

As is its wont, rather than fixing the weaknesses of its existing platform, Microsoft replaced it with an all-new platform with a different set of weaknesses. Naturally, Windows 8 still includes Windows Installer. But if you were hoping for some of the same deployment love for your non-Metro-style apps that the Windows team showered on Metro-style apps, I’m sorry to disappoint: For better or worse, Windows 8’s MSI is functionally identical to Windows 7’s MSI v5.0.

A quick spelunking in the Windows 8 SDK available to MSDN subscribers reveals mostly structual changes to the MSI header files. For example, the _WIN32_MSI macro is defined as “500” for both Windows 7 and Windows 8:

#if (_WIN32_WINNT >= 0x0601 || (defined(NTDDI_VERSION) && NTDDI_VERSION >= NTDDI_WIN8))
 #define _WIN32_MSI   500
#elif (_WIN32_WINNT >= 0x0601 || (defined(NTDDI_VERSION) && NTDDI_VERSION >= NTDDI_WIN7))
 #define _WIN32_MSI   500

The only addition is a new MSIARCHITECTUREFLAGS value in Msi.h:

 MSIARCHITECTUREFLAGS_X86   = 0x00000001L, // set if creating the script for i386 platform
 MSIARCHITECTUREFLAGS_IA64  = 0x00000002L, // set if creating the script for IA64 platform
 MSIARCHITECTUREFLAGS_AMD64 = 0x00000004L, // set if creating the script for AMD64 platform
 MSIARCHITECTUREFLAGS_ARM   = 0x00000008L //set if creating the script for ARM platform

and a new property name in MsiDefs.h:

#define IPROPNAME_ARM              TEXT("Arm")

As we’ve known for a while that Windows 8 will support the ARM architecture, its presence isn’t too surprising.

There are no other new enumeration values. There are no new MSI API functions. While it’s possible there are, for example, new standard actions that wouldn’t have visible impact in the header files, I don’t find it too likely.

That yields two tiny bits of good news:

  1. Lack of visible interface changes probably means a lack of visible behavior changes. That likely means that your Retro-style installers will probably work without change, or at least as well as the classic Windows 8 desktop supports Retro-style apps.
  2. WiX and Burn won’t need to do anything to support Windows 8. OK, that’s something only a few of us need to worry about…

Coming up: More about Metro-style deployment.

Experimental results part II

It’s been four-plus months since Experimental results part I. All reports point to The Experiment being a success so it’s time to make it permanent. Today I checked in a change that defaults to opting-out of the troublesome disk-costing dialog box. It’s still present but by default will not be shown. If you’re using a stock dialog set, there are no changes you need to make. If you’ve built a customized dialog set, remove the WixUICostingPopupOptOut WiX variable definition from your dialog set fragment or you’re likely to get an error message at link time:

The Windows Installer XML variable ‘WixUICostingPopupOptOut’ is declared in more than one location.  Please remove one of the declarations.

You can opt back in to the disk-costing dialog box by providing a value of 0:

<WixVariable Id=”WixUICostingPopupOptOut” Value=”0″ />

Look for this change in next Friday’s builds of WiX v3.5 and v3.6. (Any build more recent than yesterday’s v3.5.2208.0 and v3.6.1008.0 will have it.)