Open Source Windows Drivers

EditHint: RefactorMe until some signal emerges from the ThreadMess. (This may not even be the right name for the page, but it's better than what it was called before.) In process...

MicroSoft don't write most of their drivers. Hardware vendors will write drivers for any operating system they see the need for, and no others. The main problem is not writing them, but getting the low-level information needed to do this without reverse engineering. Consider, for example, linux. Linux has better driver support than windows *for old hardware*. Why? Because the information is well known for many bits of gear. New hardware is a totally different story: until a few years ago, pretty much all graphics card drivers (esp 3D accel) were reverse engineered for linux. Slow process. These days, many companies provide docs, code, or more usually, drivers (Nvidia, ATI, matrox, etc). They perceived enough demand to make it worth their while. Another example, Intel dragged its feet on centrino drivers, with the result that linux on modern high-end x86 laptops has suffered (Intel is now addressing this). The linux community has plenty of people capable of writing high-quality drivers, but without documentation this is a problem. It is generally the vendors *not* microsoft who are withholding this information.

The interface Windows uses to talk to the drivers IS standardized (pretty much) [internal to Microsoft]. What needs to be done is hack that set of interfaces so that the drivers can be inserted as is into a different OS. Yeah, yeah, likely to be some incompatibilities with the way OSs are implemented, but this could facilitate a great leap in alternate OS potentials. Reinstating BIOS extensions the way IBM intended (original PC line) would be great. Just need to adapt those techniques to 32/64-bit architectures. Need a protected-mode pre-emptive multitasking core and go from there (Linux would probably do). -- gl (thx. anti-mickey-bias fo sho)

There's no need to hack the driver interface. It's widely published (see http://www.cmkrnl.com/faq04.html#4.1). You are free to write bridges between Linux and Windows device drivers. Good luck. -- EricHodges

Not what I'm talking about here. Not writing driver's for Windows... or for Linux... But having drivers (bios extensions) that were OS independent. Am I misunderstanding here? All dependence on Windows functions need to be weeded out. Write say a COM interface that describes the interface to the hardware at the lowest level. Just twiddling ports and memory-mapped IO/DMA/etc. Need to just stub out dependence on Windows. Transparently as possible, emulate/wrap calls to Windows functions. Windows returns (Quick return with UNSUPPORTED or FAILURE, for example). Why should the low-level driver depend on higher level OS functionality? Bassackwards... The key thing is an interface to the HW model. Sure, HL stuff can be added on top, but don't protect privileged accesses that are "assured" of being legit (manufacturer provided driver, or somebody trying to get their own machine to do something that 'they' want it to do. PO's me good when WinNT/2000+ refuses to do something. F-that. Carry out my order, subservient machine!). Ring 0 if ya need it. GD mickey has been putting the screws to us for a long time. Screwing around with interface versions just so we gotta rewrite stuff. Fed up! Protection/Smecshun, Phhbt!... Can't trust in house ppl? Ditch'em. Can't trust mckey? Ditch'em. Configurable firewall against outside intrusion is a different story. But the machines are OURS, not mickey's. Screw how they think we wanna do something. Or where they think we wanna go. We wanna do it right. They just simply don't. Our money... Their passion. -- GeraldLindsly

You said: "What needs to be done is hack that set of interfaces so that the drivers can be inserted as is into a different OS." All you need is a bridge that lets the different OS call the driver's routines and convinces the driver it's running in Windows. You can get all the info you need from Microsoft's published interfaces. You don't need to hack anything. It's a horrendously complicated task, but it isn't prevented by secrecy. As to why drivers depend on OS services, the answer is because it's easier than writing them yourself and if you don't use the OS services you can't integrate with some aspects of the OS. This isn't something Microsoft invented to screw the end user. -- EricHodges

Not to mention that someone is in fact working on an open-source OS that is intended to be compatible with Windows NT drivers and applications: http://reactos.org/ -- MikeSmith

The real COM interface is a generally transparent native part of C++. (Non-portable technically but not in practice) [COM can be emulated in C with a less elegant syntax]. Perhaps thinking of COM+? Naw, not all that crud. Just the pointer to virtual base class function pointers. As a binary interface to HL codes. -- gl

No, I'm not thinking of COM+. You don't need COM to host Windows drivers on other OSs. -- EH

[Why would anyone want to tie *drivers* to an *object* model, let alone such a constrained one, is beyond me. A much better protocol for the task could be proposed (not that I am convinced it is needed)]

Object model is constrained? By what?! I am looking at a lowest common denominator (GreatestCommonFactor??) deal here. A binary level interface must be agreed upon as protocol. C++'s virtual base pointers, or tables of pointers to functions... same thing... Wanna get fancy? Introduce a InterfaceQuery? by GUID.

[Ever try to put a MOP ontop of COM? multiple dispatch? The c++/c#/Java way of looking at OO is constraining. But the point is, there is no need for an object model at all, just a protocol. Objects don't map particularly well to driver tasks anyway.]

What protocol needs to be agreed upon? You have the Windows driver interface on one side and the non-Windows OS driver interface on the other side. The Windows driver will expect to be able to use the Win32 interface, so those calls will have to be translated to calls into the non-Windows OS. All of these interfaces are already defined. You just have to write the pieces that convert non-Windows OS driver calls to Windows driver calls and Win32 calls to non-Windows OS calls. -- EH

[Errrm, I think we are talking at cross purposes. Scenario 1: everyone else emulates windows (as you describe). Scenario 2: OS agnostic layer agreed on. Whichever, I was objecting to the introduction of COM.]

I'm responding to what GL said above: "What needs to be done is hack that set of interfaces so that the (Windows) drivers can be inserted as is into a different OS." Any way you slice it this requires emulating Windows, since Windows drivers expect to run on Windows. -- EH

Ok. ^5 A protocol or binary image format that would be portable. Sort of an .OBJ standard. Link the drivers in at run time. Cache if desired, but don't load the entire memory map with all sorts of drivers that won't get called at all. I am fairly certain we could agree on a flat memory model. Mixed code/data is a must. Wanna execute my data. Offset 0: jump to master interface enquiring routine. Would probably best be implemented as a property list.

Got a jewel for ya. See GeraldLindsly for a fresh post of PropertyList?. -- gl

I saw it, but I don't see what it has to do with using Windows device drivers on non-Windows operating systems. -- EH

He seems to be talking about handling the loading of device drivers via a property list approach, which is no doubt workable but rather tangential to the topic; that's not the difficult aspect of handling OS-independent device drivers. -- dm

ok. Pessimist in every crowd I guess. Interfaces could be queried by asking the root property list for the interface by name. It returns you a RefPtr? that must be cast to a pointer to the appropriate interface instantiation. Note that PropertyList is also a Referenced. PropertyLists can be nested to any depth. The binary description of the interface is not changed after publication/global acceptance. Need version control here too. But bug fix installments need to be clearly indicated as such (and fixes enumerated) and require no different binary interface. -- gl

That rather seems to confirm my impression (and I did say it was doubtless workable), without at all addressing my point that this is tangential, and the easy part of portable device drivers. -- dm

Right, so now all you have to do is write the code to convert some other OS's device driver calls to Windows device driver calls and the code to convert the Windows device driver kernel mode and Win32 calls to some other OS. Piece of cake. -- EH

Many years ago I actually began such a project one weekend...but regained my sanity by Monday. -- dm

Actually what I was pointing my finger at was a new generation of Hardware Friendly Software. Minimal support required from the host OS and machine then the higher level codes can CRANK! Tangential? Howabout right on the mark if I may say so. Start with PropertyList C++ as standard [not really there yet, still needs work (young piece)]. Decide on a binary file format to implement it (really should include a list of dependent interfaces for starters on PropertyListConventions). -- gl

Eh? What are you talking about? -- EH

Check my understanding: The problem isn't creating a model that works, but rather creating a model that will be used, or grabbing a model that is already used and making it work.

The major issue with the 'is already used' bit is that those drivers may exist in more than one location in their native os: i.e., a printer driver that also is a normal application (HP drivers are bad for this). To make these drivers cross-platform, one either needs to split the driver in to 'driver' and 'other gunk' (and rewrite any 'other gunk' which is necessary for the driver to be functional), or you have to (in the win32 case) reimplement win32 so that you don't violate any assumptions made by arbitrary driver code written by arbitrarily intelligent programmers.

The issue with the 'will be used' bit is the typical one of standard; there's so many to chose from. For this to work, (1)you need manufacturers to think that support your model is a good idea for their business, (2) you need OS writers to think that it's a good idea for theirs, (3) and you need users to think that this all matters enough to not chose operating systems that don't support the model, even if they already support every device under the sun (otherwise (2) becomes false for the important case, therefore (1) becomes false, and the whole thing falls apart).

-- WilliamUnderwood

The only issue OS buyers consider is "Will this OS talk to my X?" They don't care how the drivers are implemented. Most of them don't know that drivers are involved. If I don't understand what GL is talking about, it's a good bet that my mom will not choose an OS based on this criteria. (Can someone explain what GL is talking about? What does he mean by "Hardware Friendly Software"?) -- EricHodges

Further clarification: I am not asking Gerald to explain the issues involved. I do professional OS kernel/driver work, among other things, and when I said that years ago I briefly started a project to solve this (by making Windows drivers usable under Linux), I knew what was involved at the time (well, more on the Linux side, but I had a some idea about the Windows issues). I stopped because the problem is inherently insanely difficult, not because I didn't know what I was getting into. That's also why thousands of other people didn't do this long ago.

I'm asking Gerald to explain his comments only, which make no sense to me even though I understand the topic.

The ideal solution is just for each manufacturer to support Linux directly; it's just an extra expense many still don't want to bother with, although this has been gradually changing. (Well, actually even more ideal would be if they did that and also open sourced their drivers, but many are reluctant to do so because they're afraid they'd lose competitive advantage). -- DougMerritt

I find it telling that no one brought up the UniformDriverInterface standard. While occasionally a hobbyist OS developer will talk about the possibility of using UDI, no one in the business seems at all interested in it. Part of this is because there is already a de facto standard driver interface - that being the Windows model - but also because no one wants to have to deal with making fundamental changes to an existing OS which are as likely to help your competition as it will you. Furthermore, there is a general perception that the UDI is too heavyweight to be practical. Any thoughts?

Looking at the above after several years, I can see that it was confusing without more context. Reminds me of studying mathematics... You read something a dozen times and it still doesn't make sense. Go read something else, figure something out, go back, read the first again and it's understandable. Besides the PropertyList code I published, I only dropped hints of the full concept. I'll see about formulating more of it. -- gl