Microsoft: No Driver Updates Allowed for Win7 and Win8

Microsoft has announced that it is ending the ability to cross-sign drivers, effective 1 July 2021. This will effectively make it impossible to release new or updated drivers for Windows 7, Windows 8, and Windows 8.1 systems, including Server 2012 R2. This is not an exaggeration.

The only option that will remain available to devs who want to release drivers for versions of Windows other than Windows 10 will be to have those drivers pass HLK/WHQL testing. Unfortunately, not all drivers are even eligible for HLK/WHQL testing, and even for those that are eligible, getting some drivers to pass the HLK/WHQL tests is effectively impossible.

I know this sounds like I’m exaggerating. But it’s actually the current plan of record. Read on.

Let’s Review: Driver Signing Options Available in 2020

I think we all know that Windows drivers need to be digitally signed in order to allow them to be installed and used. As of today, October 2020, there are three options for this signing:

1. Attestation Signing — This applies to Windows 10 only. You create an account on the Microsoft Partner Center Developer Dashboard. When you have a driver package you want to release to the world, you sign it with your Code Signing Certificate, you upload it to the Dashboard, and Microsoft signs the package for you. You can then download the package and release it however you wish.
   
   This “works” because your Developer Dashboard account unambiguously identifies you to Microsoft, and Microsoft’s signature allows your driver to be installed.
   
   
2. Cross Signing — This applies only to versions of Windows prior to Windows 10. You sign your driver package using your Code Signing Certificate and a “Cross-Certificate.”
   
   This “works” because you have proved your identify to the satisfaction of your Certification Authority (a level of proof that varies widely) that issued your Code Signing Certificate. The Cross-Certificate identifies the Certification Authority as one that Microsoft trusts. Together, your cert and the cross-cert allow your driver to be installed.
   
   
3. Passing the HLK (historically called the WHQL) Tests — This applies to all versions of Windows. You install and run the tests defined by the Windows Hardware Lab Kit (HLK), which is easier said than done. The HLK produces a log file, which you sign and upload to the Microsoft Partner Center Developer Dashboard along with your driver package. The Dashboard reviews the HLK results, and if all goes well you download your signed driver package.
   
   This “works” for the same reason that Attestation Signing works: Your Developer Dashboard account unambiguously identifies you to Microsoft, and Microsoft’s signature on your driver package allows your driver to be installed.

Microsoft’s Plan: Cross-Signing is Dead

As of 1 July 2021, Microsoft is eliminating option #2, Cross-Signing. This will leave passing the HLK the only option for releasing drivers for Windows versions other than Windows 10.

Microsoft first announced this plan back in the July 2019. And, quite frankly, we didn’t believe it. Like so many such misbegotten ideas (remember “All drivers must pass WHQL or they won’t load on Windows Server”) we hoped that the doc writers were misinformed and/or that the policy folks at Microsoft were floating this idea to gauge the community’s reaction.

Since this announcement we here at OSR have consistently engaged with Microsoft in an attempt to determine the true plan for going forward. As I wrote to one my colleagues at Microsoft back in September 2019:

I am certainly hoping I fundamentally misunderstand something here.

But IF this is correct, this is a huge mistake.  It’s basically Armageddon.  And that’s no exaggeration.  The vast majority of the drivers written by third parties out here in the world must install on production Win7 systems or later.  If we can’t cross-sign them, and we can’t attestation sign them… how do we write new drivers that will load on non-Win10 systems?

While our friends have certainly been both concerned and collaborative they have not been able to offer a definitive statement that says anything other than what’s been published.

At one point I did hear a rumor that Attestation Signing would be extended to support Windows versions prior to Windows 10 (recall, that Attestation Signing only works for Windows 10 today). And now I’m not hearing anything about that possibility at all.

So… Just Pass The HLKs?

Some of you might be thinking: We can still just run and pass the HLKs, so we’ll be OK! I can only guess that the only people thinking this are those who either (a) already run/pass the HLKs, (b) have never before bothered to try to run and pass the HLKs.

Installing and running the HLKs is a heavyweight, time-consuming, and often arduous, frustrating, and often annoyingly arbitrary process. It is primarily designed for testing hardware components for compatibility — and not really aimed at validating the vast eco-system of potential Windows drivers.

Many types of Windows drivers are not eligible for testing by the HLKs. Many common types of Windows drivers — such a many filter drivers — fall into no specific testing category and basically force a test team to choose an arbitrary set of tests to run and attempt to pass. This is the HLK equivalent of forcing a square peg into a round hole. For example, let’s say you have a set of drivers that implement a feature that prevents permanent writes to a disk drive, like Windows Unified Write Filter. You test this entire complex of drivers as a disk drive, right? Obviously. Or not.

Some types of drivers are theoretically eligible for HLK testing, but because of the specifics of the HLK tests have no chance of ever passing. An excellent example of drivers in this category are File System Isolation Minifilters. There are existing HLK tests that are inherently incompatible with the purpose of many such filters. As just one example, consider the case of an Isolation Minifilter that provides transparent encryption. A common feature of such drivers is that encrypted files often include some sort of metadata (groups, and keys, and such). An HLK test might write a file, close the file, and then query the file’s allocation size. Because the test “knows” the allocation policy of the underlying file system, it thinks it “knows” exactly how big the file should be on disk. But our encrypting Minifilter, in doing the work for which it is designed, changes the allocated file size by allocating space for metadata. The test therefore fails. There is no way to change this behavior in a way that will make both applications and the test happy.

Another example of drivers that could theoretically — but not realistically — pass HLK testing, are drivers for devices that run on special-purpose systems that themselves will never work as an HLK client. When the hardware can’t be separated from the system, and the system does not support running the HLKs, there’s no chance for such devices to ever pass the tests.

There are also devices that do not support the full range of behaviors that the HLKs enforce. For example, it is not the least bit uncommon for devices in certain embedded systems to expect to never be powered off. It might be possible, maybe, to get some of these devices to pass the HLKs with a massive amount of effort. But that’s still only some, and not all.

The HLKs were never intended to be a quality bar for drivers for special purpose devices on purpose-built systems. The HLKs are compatibility tests to ensure that common devices work well with Windows, and provide users with the experience they expect, on common servers and desktop machines. And while passing the HLKs is certainly a best practice it’s just not realistic for every device, on every system.

If This Doesn’t Change Lots of Folks Are Screwed

Make no mistake: If this policy doesn’t change, the Windows driver ecosystem — and many big Windows users all over the world — are going to be in serious trouble.

Windows 8.1 and Windows Server 2012 R2 are still under Extended Support by Microsoft, and will remain supported until January 2023. Windows Embedded 8 and Windows Embedded 8.1 are supported by Microsoft through July of 2023.

So, Microsoft is still supporting these versions of Windows but IHVs, ISVs, and OEMs will not be able to release updated drivers for these supported OS versions under this plan.

If this proposal doesn’t change, customers using these OS versions will be positively screwed. How? Let me provide some examples.

In the past few years, here at OSR we have written brand new drivers for special-purpose systems that have been targeted a versions of Windows other than Windows 10. These are embedded-type systems used in medical equipment, homeland security hardware, and lab equipment.

If this proposal doesn’t change, we will not be able to support or update these drivers. So… “Sorry, big government contractor… that big, multi-million dollar piece of equipment that you build for the government? We know you found a bug, but we can’t update the drivers anymore, because of Microsoft’s policy.”

We regularly enhance and support products that are installed by commercial clients on (no exaggeration) hundreds of thousands of systems worldwide. Products that provide intellectual property protection and document security for companies and governments in the US, Canada, Europe, and all throughout Asia.

If this proposal doesn’t change, we will not be able to update these drivers. “Sorry, healthcare providers and big financial institutions! Sorry high-tech firms! We can’t add features to or fix any bugs that get found in your document security system. Microsoft won’t let us update the drivers.”

We have heard loudly, clearly and persistently, from the community at large and from our own clients, that the vast majority of drivers written today need to work on Windows versions dating back to Windows 7.

If this proposal doesn’t change, we will not be able to update the vast majority of these drivers.

We Can Change This — It’s Not Too Late

One thing with which I have historically credited Microsoft is their willingness to listen to the community, and change their course when necessary.

Two examples from recent history illustrate this very clearly.

The first example was Microsoft’s plan that required an EV Certificate to be used to submit HLK results and driver packages to the Developer Dashboard. While it probably sounded like a reasonable requirement, here at OSR we noted that this would make life exceptionally difficult (if not impossible) for distributed organizations. Let’s say your dev team is in the UK and your test team is in the US. If your dev team’s management arranged for your Developer Dashboard account, how do they get the EV Certificate (which is irrevocably locked onto a particular hardware token) to the test team, so the test team can submit the HLK results? It just doesn’t work.

When we discovered this, we brought the problem to the attention of the community. We asked OEMs, IHVs, and ISVs to have their executives raise this issue with Microsoft. They did this, and the policy was changed.

Another example, mentioned earlier, was Microsoft’s plan to require that all drivers installed on Windows Server systems to pass the HLKs. Again, the OEM, IHV and ISV community spoke-up loudly and clearly about why this plan was perhaps well-intentioned but unrealistic. The policy was changed.

Now Is The Time To Speak Up

Requiring drivers to pass the HLKs in order to load on versions of Windows prior to Windows 10 is effectively no option at all.

If you agree with this, it’s critically important that you alert your colleagues to this problem. Explain to them the pain this will cause. Encourage your management team to raise this issue to their Microsoft contacts. We know from past experience that Microsoft values the opinions of OEMs, IHVs, and ISV. We know from experience that Microsoft will listen to pushback on plans that may sound good in theory, but that the community recognizes are not workable in practice. We know from experience that Microsoft will change these plans, when the flaws are made clear.

But you must act. You’ve got to raise this issue to your colleagues and your managers. Help them understand the criticality of this issue. You cannot wait until July 2021, when this policy goes info full effect, to act. You must act now.

---

Updates: 29 October 2020. Added a couple of paragraphs about why passing the HLKs is not realistic for many special-purpose hardware devices, in response to several comments across the web.