GenEnvironmentVariable and CreateProcess failures under XP

From: LarryH (BigLar_at_nospam.nospam)
Date: 04/22/04 

Date: Thu, 22 Apr 2004 12:41:03 -0700

I am having truble figuring out how to debug a problem with VB6 program that shells other Win32 executables. The program has been running for years and was just recompiled under Windows XP Pro SP1. The problem is with calls to the Win32 API not returning results even though the exit code indicates success. The program calls GenEnvironmentVariable to get the sytem PATH, SetEnvironmentVariable to prepend a value to the system PATH, then calls CreateProcess to launch Notepad or other Win32 executables. I can launch Notepad 6 or 7 times in a row with no problem, then I start getting one of three unexpected results. The unexpected results are:

(1) GetEnvironmentVariable fails to return anything, but still returns "1" to indicate success,
(2) CreateProcess fails to launch Notepad (or whataver is called), but still returns "1" to indicate success,
(3) Createprocess fails to launch Notepad (or whataver is called) and returns "0" to indicate failure.

Once an unexpected result happens, it will happen on all subsequent attempts to launch that program until I shut the program down (when debugging, I have to shut VB 6 down). Only the third result makes sense. The Event log is not updated by any of these errors.

I cannot find any information on this problem in the MSDN library or through Internet searches, including some of the more popular VB sites.

I have also been told that this problem ocurrs infrequently on Windows NT, but only happens once, not continuously. On XP, of the two PCs we are testing on, one computer produces the unexpected results a lot more frequently that the other. All of our PCs are locked down and all software is remotely installed. Our workstation people looked at the computers and say the same patches have been applied to each. I can replicate this problem at will.

Is this a known problem? What can I do about it? If it is unfamiliar, what tools do I need to debug it?