Re: hard drive scrubbing utility
- From: Joseph M. Newcomer <newcomer@xxxxxxxxxxxx>
- Date: Fri, 12 Jan 2007 14:23:33 -0500
See below...
On Fri, 12 Jan 2007 15:44:05 GMT, MrAsm <invalid@xxxxxxxxxxx> wrote:
On Thu, 11 Jan 2007 21:52:08 -0500, Joseph M. Newcomer****
<newcomer@xxxxxxxxxxxx> wrote:
Note that I
already know how to write device drivers. You have to get to at least this level of skill
before you can even *start* to work on the problem. Note that I also teach courses on this
and have co-authored a book on the subject.
Even if it is OT with respect of original post and of the newsgroup
(your previous answer whetted my programming appetite :) ... :
In your opinion, how long does it take to learn how to write device
drivers for Windows, if a programmer already knows C/C++ Win32
programming and has a basic knowledge of Intel assembly?
Depends on what you mean by "learn" and "device drivers". I teach a 5-day course on basic
principles of driver development, but you can spend months mastering that material. And
we only teach the basics relative to PCI-style bus devices. USB isn't that much harder (I
do a walkthrough of one of Microsoft's USB drivers on Friday), but it can also take months
to learn the intricacies of the USB protocols for a given device.
The type of device driver you are writing makes a lot of difference also. If you are
writing a driver for a PCI, or simple USB device that uses bulk transfer, you pretty much
come out of the course (which is based on our book) able to do that. But if you want to
write network drivers, Bluetooth drivers, video drivers, printer drivers, or anything
related to the file system, you can spent another several months just learning the
intricacies of the protocols for those drivers (which have to interface to rather
rigidly-specified drivers that live above them, and in some cases, to drivers that live
below them). Video and printer drivers actually spend most of the effort completely
outside the device driver model, interfacing to the device so that it renders pixels
correctly (do you know how to compute the miter angle parameters for a mitered line join?
Yeah, all that stuff. On a good day, your video card will do it for you if you can only
program the 22 control registers that tell it what to draw).
Assembly code is never used to *write* device drivers, but knowlege of it sure is valuable
for *debugging* device drivers.
As I demonstrated in one class I taught, you also need to know a lot about hardware (there
are, fortunately, really good books on this subject). I knew I was in trouble when my
students were giving *me* homework! They gave me a crash dump to analyze. I then
improvised a 2-hour lecture the next day on how I analyzed the crash and what I learned.
My conclusion was that it was a hardware problem, because it was a sporadic--once every
few months--access fault which bluescreened the system. I diagnosed it as a failure at
the PCI bus level. Whether it was the motherboard, chip set, or the device plugged into
the backplane, or *some other* device in the backplane was indeterminate, but I gave about
six possible hardware scenarios that could have caused it. For example, you have to know
the PCI bus is not a digital bus, but an analog bus; that waveform timings are critical;
that the wrong inductive/capacitive loading can disrupt the bus; that it is a
positive-acknowledgement bus; that an access fault reading a memory-mapped device register
that has been read tens of times per second for several months has to be a sporadic error;
and that adding an exception handler might solve the problem by allowing for a graceful
recovery.
****
****
Do you have a list of books you would suggest to learn device driver
programming (including the book(s) you wrote on this subject)?
Our book, of course; Mason & Viscarola's book (from the OSR Web site); if you want to deal
with file systems, Navindra Jain's book (also available from OSR); the DDK documentation;
Walter Oney's books (but you have to ignore the C++ aspects of his style--they are even
less likely to work with the newer compilers than they were when he wrote the books);
reading the source code of existing Windows drivers and making sure you understand exactly
what they are doing at each point; and a serious mastery of multiprocessor concurrency
issues as well as OS-level issues. I always warn my students that every driver has a
*minimum* of three concurrent threads of control, which can be executed truly
concurrently; that there is essentially no upper bound on concurrency; that if you ask for
operation A(), then A could be completed *before* you return from the call to invoke it;
little things like that.
There are some new books coming out from Microsoft Press on the new Kernel Mode Driver
Framework (think: MFC for drivers, but without the C++). And one forthcoming from OSR,
Real Soon Now (I paid for it months ago, but it hasn't shipped yet).
Avoid Kant's book. There's a game driver writers play, which involves opening it to any
random pair of pages, and timing how long it takes to find the bug that will certainly be
there. I've not played it, but others have, and have told me how much fun it is. (Maybe
it's the fact that I don't go out drinking means I miss these sessions).
The toughest class I ever taught had 9 students who had spent 3-6 months each trying to
learn drivers on their own. I couldn't handwave *anything* in that course, and had to
explain every example, sometimes six or seven times, until they were satisfied they
understood each of them. It was a *great* class. We ran until 7pm almost every day!
Once you start, hanging out in the sibling microsoft.public.programmer.win32.kernel or the
OSR newsgroups will be very informative.
****
Is the job-market for Windows device-driver developers good?****
Overall, it is very poor. At least part of that is a philosophy that goes "Gee, we now
have the hardware ready to go. All we need to do now is write a device driver. Well,
it's a month before product launch; that should be enough time".
I can write 200 lines/hour of MFC code; I can do so for extended periods (months). If I'm
writing driver code at a tenth of that speed, I writing too fast and making mistakes. So
they don't realize that a program of the complexity of a device driver can take many
months to write, and therefore that the cost is going to be very high.
No company wants to believe that they should pay $100,000 for a driver. But that's about
what one costs, whether you do it in-house or outsource it. Since the companies rarely
have in-house talent, they send the work offshore. I've seen some of those outsourced
drivers. They won't pass Driver Verifier; they crash and burn with frightening
regularity, and in one case the only way the entire product line could be salvaged was to
write a new driver from scratch; their $25,000 "bargain" driver was so badly written that
I filled up three pages of fine print with the number of beginner errors it contained. It
went to market a year behind schedule, but the drivers (they hired in-house talent)
apparently work very well. The original driver's architecture was a disaster and the
coding was erroneous, and it could never have been fixed to pass the required WHQL testing
to get signed by Microsoft. But manufacturers simply don't want to put out the expense a
device driver actually requires.
I've not had any driver work in three years...or is it four? Anyway, a very long time.
I've done code reviews of drivers (as recently as a few months ago) and found problems for
customers, anything from "this line here" to "throw it out and have it rewritten,
preferrably by someone who can read the documentation", but no "new" work in 3-4 years. So
while it is an interesting skill, the market is not overwhelming.
Perhaps there is a better market for in-house driver skills (a good line on the resume for
a full-time position), but for those of us who are outside contractors, the current level
of business, at least in the driver world, is pretty poor.
*****
****
What are the software tools required to develop a device-driver?
The Microsoft DDK. Also, tools like the Driver Verifier, PreFast, the Call Usage
Verifier, and the Static Driver Verifier. I have a 2-hour lecture on the tools. They are
mostly free from Microsoft.
Avoid any "tool" that has a point-and-grunt interface to let you build drivers quickly,
especially any tool that uses C++. Most of these have died the deaths they so richly
deserved, but some continue to pop up, sort of like the zombies in "Return of the Living
Dead". Bottom line: you can't use C++ in the kernel. If you believe otherwise, you are
living in a delusional world. I spent a half-hour with the C/C++ compiler lead at the
last WinHEC, and while he went very deeply into the reasons the Microsoft C++ compiler
cannot generate code that can live in the kernel, I took away only one bit that is
important: don't use it. [Yes, streaming audio and video drivers are written in C++.
There's a story I can't tell that proves this was a real technical blunder].
The tools are not exactly "fraudware" (in the Russinovich sense), but are certainly
"delusionware". They let you live under the delusion that you have a driver. You almost
certainly don't. You just haven't realized it yet. You won't until you discover (a)
everything it doesn't do that it needs to and (b) that none of these things can be done in
the framework you have.
****
Visual C++ 6 or 2003 are good for this purpose?****
Just about totally useless. Drivers live in their own world, and Microsoft does not
support the building of drivers in VS at all.
****
Or does Microsoft provide device-driver specific compilers?****
Yes. Little-known fact: for the price of a DDK, about $13, you get a full C/C++ compiler.
You can download the Platform SDK and WinDbg free, get a freeware editor, and you have a
Win32 development environment for $13. No MFC, no ATL, but raw Win32 works fine.
****
Are these tools freely available for learning purposes from Microsoft****
web site?
Everything is free except the DDK; for reasons nobody understands, you have to buy it, for
the aforementioned $13 media cost. Or, if you have some level of MSDN (I'm not sure it
comes with the lowest levels) it is included. All other tools from Microsoft are free.
They don't have a code coverage tool, but Bullseye has one for about $800. For a serious
driver project, this is such a small cost that it is easily justified. But it isn't
justified for an individual who is in learning mode, but it also isn't necessary.
joe
****
Joseph M. Newcomer [MVP]
Thanks very much,
MrAsm
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
.
- Follow-Ups:
- Re: hard drive scrubbing utility
- From: MrAsm
- Re: hard drive scrubbing utility
- References:
- hard drive scrubbing utility
- From: avatar70
- Re: hard drive scrubbing utility
- From: Joseph M . Newcomer
- Re: hard drive scrubbing utility
- From: MrAsm
- hard drive scrubbing utility
- Prev by Date: Re: Official name of File Folder Drop Down
- Next by Date: Re: hard drive scrubbing utility
- Previous by thread: Re: hard drive scrubbing utility
- Next by thread: Re: hard drive scrubbing utility
- Index(es):
Relevant Pages
|
