Re: PB 4.1 - how to FTP without WinInet library ?

From: SergeiR (anonymous_at_discussions.microsoft.com)
Date: 02/17/04 

Date: Tue, 17 Feb 2004 02:52:29 -0800

Hi John,

thanks for the info. Regarding the limit of stack size in
functions, is 4 kb the limit due to compiler switch used
in build process ?

Actually, I just encreased the amount of memory in my test
from yours, not changing the scope or type of buffer.
Let's assume it overflowed stack, so ... In another
experiment, memory buffer was allocated as new() and yet
in another one it was made of LocalAlloc(). The problem
was reproduced using LocalAlloc() of 4kb same way as with
stack memory buffer of 4kb.

So I guess there may be something else in there.

Anyway, I had decreased the buffer size to 1 kb and our
product started to work normally with Httplite DLL, at
least customer using Unix FTP server didn't get a problem
so far, and it is good to know where the problem
originates from.

Thanks again

Sergei

>-----Original Message-----
>Though it's not related to this problem, you want to be
careful with 'CHAR
>pBuffer[4000]' in any event. Individual functions with
stacks > 4KB on
>WinCE can cause real problems if you exhaust the stack
space. Normally if
>you run out of stack the system generates an exception,
but if it's >4KB
>system may not catch it and you can really trounce things
badly and have no
>idea what went wrong.
>
>With regards to the dwBytesRead not being equal to the
bytes that you set, I
>don't know what Wininet's behavior is in this case. For
httplite, however,
>I simply call Winsock recv() and read in as much data as
I can - up to the
>size of the buffer passed me. I do not call recv()
multiple times if there
>is more buffer left to fill up. It's possible that on a
slow link that <
>4KB of data was available to be read in this scenario.
So for httplite at
>least, checking when the bytes returned = 0 is the safe
approach.
>
>There is not a max buffer size that httplite supports.
It will take what
>you give it and pretty much pass that directly to recv().
>
>--
>John Spaith
>Software Design Engineer, Windows CE
>Microsoft Corporation
>
>This posting is provided "AS IS" with no warranties, and
confers no rights.
>You assume all risk for your use. © 2003 Microsoft
Corporation. All rights
>reserved.
>
>"sergei" <anonymous@discussions.microsoft.com> wrote in
message
>news:163301c3f49a$fc25c320$3501280a@phx.gbl...
>Hi John,
>
>I've done some testing using your example using our CE 4.1
>image, which is equipped with Httplite dll. Your exact
>example works as it was designed, though I have some
>concern, which may help to nail down the problem.
>
>In the code you check for success of function
>InternetReadFile() and then also check for another
>condition
>....
>if (dwBytesRead == 0) {
>....
>
>comparing it with 0. I have made slight modification of
>this test condition to compare actual bytes read with
>bytes requested, and not with zero. Like in below
>
>....
>if(dwBytesRead!=cBufferSize)
>{
>// all done ??
> break;
>};
>....
>
>what I have found then is the situation when while
>InternetReadFile() returns success, the dwBytesRead!
>=cBufferSize. The key in my tests was the buffer size used
>with Httplite function InternetReadFile().
>
>It works when CHAR pBuffer[1000] and it does not when it
>is CHAR pBuffer[4000].
>
>Maybe this will not confirm the problem of or a feature of
>Httplite DLL in your test scenario.
>
>Please let me know if this would be confirmed or not, so I
>could change the code to be sure what the maximum buffer
>size supported is.
>
>Best regards
>Sergei
>
>>-----Original Message-----
>>Sergei,
>>
>>Even though httplite isn't officially supported, I do fix
>bugs in it when we
>>find out about them. I wrote a small program to simulate
>what you were
>>doing here to try and reproduce and fix this problem.
>However I could not
>>reproduce the issues you were seeing. I am running my
>program on a CEPC x86
>>debug, CE 5.0. FTP Server I am communicating with is IIS
>5.1 on WinXP.
>>Httplite hasn't changed that much between CE 5.0 & 4.1
>and I don't think
>>that anything in underlying net stack would be this
>broken in CE 4.1 to
>>cause you problems.
>>
>>Here is my code, if it helps out any. I couldn't figure
>out what you were
>>doing the FindFirstFile on so I guessed, though this
>shouldn't be the root
>>of the problems.
>>
>>If you cannot get this code to work on your platform I'm
>afraid I can't help
>>you much more. If you can get it going enjoy. Also in
>the event you get it
>>working, if you can track down what the difference
>between my code and yours
>>that is causing problems and let me know what it is, I
>may be able to fix it
>>in a future httplite.
>>
>>Good luck!
>>
>>int
>>WINAPI
>>WinMain(HINSTANCE hInstance,
>> HINSTANCE hPrevInstance,
>> LPWSTR lpCmdLine,
>> int nCmdShow)
>>{
>> HINTERNET hFtp = InternetOpenW
>(NULL,INTERNET_OPEN_TYPE_DIRECT,NULL,NULL,0);
>> if (hFtp == 0) {
>> wprintf(L"InternetOpen failed, GLE=0x%
>08x\r\n",GetLastError());
>> return 0;
>> }
>>
>> HINTERNET hConn =
>>InternetConnectW
>(hFtp,cszHostName,INTERNET_INVALID_PORT_NUMBER,
>>
>>L"anonymous",L"abcd",INTERNET_SERVICE_FTP,
>>
>INTERNET_FLAG_PASSIVE,0);
>> if (hConn == 0) {
>> wprintf(L"InternetConnect failed, GLE=0x%
>08x\r\n",GetLastError());
>> return 0;
>> }
>>
>> WIN32_FIND_DATA findData;
>>
>> HINTERNET hFindFile = FtpFindFirstFileW
>(hConn,L"temp.txt",&findData,0,0);
>> if (hFindFile == 0) {
>> wprintf(L"FtpFindFirstFileW failed, GLE=0x%
>08x\r\n",GetLastError());
>> return 0;
>> }
>>
>> HINTERNET hOpen =
>>FtpOpenFileW
>(hConn,L"temp.txt",GENERIC_READ,FTP_TRANSFER_TYPE_BINARY |
>>INTERNET_FLAG_RELOAD,0);
>> if (hFindFile == 0) {
>> wprintf(L"FtpOpenFileW failed, GLE=0x%
>08x\r\n",GetLastError());
>> return 0;
>> }
>>
>> DWORD dwTotalBytes = 0;
>>
>> while (1) {
>> CHAR pBuffer[1000];
>> DWORD cBufferSize = sizeof(pBuffer);
>> DWORD dwBytesRead = 0;
>>
>> if (! InternetReadFile(hOpen, pBuffer, cBufferSize,
>&dwBytesRead)) {
>> wprintf(L"InternetReadFile failed, GLE=0x%
>08x\r\n",GetLastError());
>> return 0;
>> }
>>
>> dwTotalBytes += dwBytesRead;
>>
>> if (dwBytesRead == 0) {
>> wprintf(L"InternetReadFile returned 0 bytes. Total
>Bytes read =
>>%d\r\n",dwTotalBytes);
>> return 0;
>> }
>> }
>>}
>>
>>
>>--
>>John Spaith
>>Software Design Engineer, Windows CE
>>Microsoft Corporation
>>
>>This posting is provided "AS IS" with no warranties, and
>confers no rights.
>>You assume all risk for your use. © 2003 Microsoft
>Corporation. All rights
>>reserved.
>>
>>"SergeiR" <anonymous@discussions.microsoft.com> wrote in
>message
>>news:bb2c01c3ecb9$77e706c0$a401280a@phx.gbl...
>>Hi John!
>>
>>Thanks for the prompt reply. The source code in the
>>suggested ftpupdate.cpp file is quite similar to mine,
but
>>with the difference in creating of new, temporary file on
>>disk first.
>>
>>We can't do that due to the fact that there is lack of
>>that disk space on CF on that small device. So I went
with
>>the version similar to what ftpudate.cpp does, but with
no
>>temporary file, just read block after block from FTP
>>server into a memory buffer, and then, if all went okay,
>>write file to CF (or overwrite is that's the case).
>>
>>And this my approach works fine with WinInet. So maybe
>>Httplite is somewhat different and this doesn't work.
>>
>>At this moment we don't know yet if management will
choose
>>to pursue this direction further and troubleshoot the
>>problem with Httplite dll, or divert to another direction
>>and choose an alternative method ( without FTP).
>>
>>Best regards
>>Sergei
>>
>>>-----Original Message-----
>>>Hi,
>>>
>>>I've looked over your code below and I can't see what's
>>causing the
>>>problems. As Mark said, unfortunately HttpLite is
>>officially unsupported.
>>>In the future we do planning on actually supporting and
>>documenting it, but
>>>we can't presently.
>>>
>>>However, we have an application named FtpUpdate that
>>downloads new images to
>>>CE devices that uses the FTP functionality of httplite.
>>It is shipped in
>>>Platform Builder in %_WINCEROOT%
>>\public\wceshellfe\oak\FTPUpdate. The only
>>>suggestion I can give you would be to check out the FTP
>>specific source code
>>>in this dir and try to and copy and paste it into your
>>program and hope that
>>>that code path makes everything just work.
>>>
>>>Hope this helps and good luck!
>>>
>>>--
>>>John Spaith
>>>Software Design Engineer, Windows CE
>>>Microsoft Corporation
>>>
>>>This posting is provided "AS IS" with no warranties, and
>>confers no rights.
>>>You assume all risk for your use. © 2003 Microsoft
>>Corporation. All rights
>>>reserved.
>>>
>>><anonymous@discussions.microsoft.com> wrote in message
>>>news:a95f01c3ebe2$f128b920$a501280a@phx.gbl...
>>>> Hi Mark !
>>>>
>>>> Thanks a lot for the tip! It really saves lots of
work.
>>>>
>>>> I have followed your directions, and quite succeded,
>>>> but ...
>>>>
>>>> So let me check with you whether or not I have made it
>>>> through.
>>>>
>>>> a) my code comments out #include "wininet.h" and
>replace
>>>> it with #include "dubinet.h"
>>>>
>>>> b)inside dubinet.h file, located at
>>>> C:\WINCE410\PUBLIC\COMMON\OAK\INC
>>>>
>>>> I look for a line with "CESYSGEN IF DUBINET_DUBFTP"
and
>>>> for another line with " "
>>>>
>>>> and they are
>>>>
>>>> 1173 : // @CESYSGEN IF DUBINET_DUBFTP
>>>> 1437 : // @CESYSGEN ENDIF
>>>>
>>>> so they are both commented out ! Well, delete them
>>anyway
>>>> as you direct.
>>>>
>>>> c) go to the cesysgen.bat file for my project in
>>>> C:\WINCE410\PUBLIC\EmCORE_minimal\WINCE410
>>\Geode\oak\MISC
>>>> and insert the two lines in the beginning of it, i.e.
>>line
>>>> numbers are 54 and 55
>>>>
>>>> d)do a full command line rebuild, followed by a
>>>> RebuildPlatform command from PB 4.1
>>>>
>>>> e) observe the appearance of the httplite.dll in the
>>>> Release build of the project !
>>>>
>>>> Looks great ! It is just 69kbytes, very good !
>>>>
>>>> f) test the newly build dll.
>>>>
>>>> Well, things went wrong here ...
>>>>
>>>> LoadLibrary(httplite.dll) - success
>>>> InternetOpenW()- success
>>>> InternetConnectW(hInet, v_szServerName,
>>>>
>>>> INTERNET_INVALID_PORT_NUMBER,
>>>> m_tszLoginName, // LPCTSTR lpszUserName
>>>>
>>>> m_tszPassword, // LPCTSTR lpszPassword
>>>> INTERNET_SERVICE_FTP,
>>>>
>>>> INTERNET_FLAG_PASSIVE, 0)) - sucess
>>>>
>>>> FtpFindFirstFileW(hFTP, v_szSourceName, &FindFileData,
>>>>
>>>> INTERNET_FLAG_NO_CACHE_WRITE, 0) - success
>>>> FtpOpenFileW(hFTP, v_szSourceName, GENERIC_READ,
>>>> FTP_TRANSFER_TYPE_BINARY |
>>>> INTERNET_FLAG_RELOAD,
>>>> 0) - susccess
>>>> InternetReadFile(hOpen, pBuffer, cBufferSize,
>>>>
>>>> &dwBytesRead) - returns success several
>>>> times, but after several blocks downloaded the number
>of
>>>> bytes read is not what was requested. GetLastError()
>>>> returns 0 ... It only gets a small portion of a
>file ...
>>>>
>>>> can't find where the problem is. Mark, did all the
>>>> functions from Httplite.dll work well for you ?
>>>>
>>>> I have no problems using exact same code with
>>Wininet.dll.
>>>> Should the code be different for httplite.dll ??
>>>>
>>>> Best regards
>>>> Sergei
>>>>
>>>> >-----Original Message-----
>>>> >Hey Sergei. Don't go implementing a whole new FTP
>>client
>>>> just yet.
>>>> >
>>>> >We had the same problem. We have a headless
>>>> configuration, and we made
>>>> >heavy use of WinInet in a 2.11 platform. When we
>moved
>>>> to 2.12, WinInet
>>>> >became part of IE. Luckily, we found that the
>>>> functionality was still
>>>> >available through "dubinet."
Unfortunately, "dubinet"
>>>> went away with CE
>>>> >3.0.
>>>> >
>>>> >Luckily, it looks like all of that functionality came
>>>> back in httplite in CE
>>>> >4.0, and we're actively using it in a product running
>>CE
>>>> 4.20. We haven't
>>>> >had any problems with it yet. The only thing we had
>to
>>>> do, at the
>>>> >recommendation of a Microsoft developer, was make a
>>>> slight modification to
>>>> >the file "\public\common\oak\inc\dubinet.h." You
need
>>to
>>>> remove the two
>>>> >"CESYSGEN IF DUBINET_DUBFTP" tags, or you won't get
>>>> proper prototypes for
>>>> >the FTP functionality. Then, we just added the two
>>>> following lines to our
>>>> >cesysgen.bat file:
>>>> >
>>>> >set __SYSGEN_HTTPLITE=1
>>>> >set __SYSGEN_HTTPLITE_FTP=1
>>>> >
>>>> >Rebuild, link the "httplite.lib" import library to
our
>>>> app, include
>>>> >"dubinet.h", and we were rolling.
>>>> >
>>>> >I know the modifications to the public\common file
>>suck,
>>>> but we couldn't
>>>> >find a better way, short of redefining all of those
>>>> prototypes ourselves.
>>>> >Try it. It works.
>>>> >
>>>> >Mark Murawski
>>>> >Vocollect, Inc.
>>>> >
>>>> >"sergeir" <anonymous@discussions.microsoft.com> wrote
>>in
>>>> message
>>>> >news:945401c3eaf2$f6b3ce70$a101280a@phx.gbl...
>>>> >> Thanks Maxim for the idea. Unless there is another
>>>> >> solution found ( either some handcraft play around
>PB
>>>> or a
>>>> >> third party library), we could think about it.
>>>> >>
>>>> >> You mean by using ws2.dll ( and .lib), right ?
>>>> >>
>>>> >> Anyone who are using CE httplite library please
>>advise !
>>>> >>
>>>> >>
>>>> >> Sincerely
>>>> >> Sergei
>>>> >>
>>>> >> >Implement the FTP protocol yourself on top of
>>sockets -
>>>> >> not this huge a task,
>>>> >> >provided you will need a subset only.
>>>> >> >Get the appropriate RFC and start coding.
>>>> >> >
>>>> >> >--
>>>> >> >Maxim Shatskih, Windows DDK MVP
>>>> >> >StorageCraft Corporation
>>>> >> >maxim@storagecraft.com
>>>> >> >http://www.storagecraft.com
>>>> >>
>>>> >>
>>>> >> >-----Original Message-----
>>>> >> >> Is there another way to still use FTP simple
>>>> functions
>>>> >> >> available but do not include this huge
>>WinInet.DLL in
>>>> >> the
>>>> >> >> image ?
>>>> >>
>>>> >
>>>> >
>>>> >.
>>>> >
>>>
>>>
>>>.
>>>
>>
>>
>>.
>>
>
>
>
>.
>

Relevant Pages

  • Re: PB 4.1 - how to FTP without WinInet library ?
    ... It has to do with how Windows CE catches stack overruns. ... © 2003 Microsoft Corporation. ... not changing the scope or type of buffer. ... product started to work normally with Httplite DLL, ...
    (microsoft.public.windowsce.platbuilder)
  • [NEWS] How to Remotely and Automatically Exploit a Format Bug
    ... Exploiting a format bug remotely is not as difficult as one would think. ... We will use very minimalist server along this paper. ... Since the buffer is directly available to a malicious user, ... Guessing the address of the shellcode in the stack ...
    (Securiteam)
  • [NEWS] Multiple ValiCert Security Problems
    ... * Enterprise VA Host Server for processing validation requests VA API ... Multiple buffer overflows exist in the CGI script, forms.exe, which is ... Analysis of the code and stack contents reveals that the unchecked buffer ...
    (Securiteam)
  • [REVS] Buffer Overflow Exploitation and Prevention
    ... The following security advisory is sent to the securiteam mailing list, and can be found at the SecuriTeam web site: http://www.securiteam.com ... you have to know what a buffer overflow is (we highly recommend reading ... to by `str' is too long, we smash, outside of the buffer, EBP (not always ... We'll have to appropriately prepare the stack in order for our ...
    (Securiteam)
  • [REVS] Writing Buffer Overflow Exploits - a Tutorial for Beginners
    ... The following security advisory is sent to the securiteam mailing list, and can be found at the SecuriTeam web site: http://www.securiteam.com ... Buffer overflows in user input dependent buffers have become one of the ... The bottom of the stack ... To keep it simple, shellcode is simply assembler commands, which we write ...
    (Securiteam)