Re: FTPFindFirstFile unicode

Tech-Archive recommends: Fix windows errors by optimizing your registry

It looks like you might have hit some buffer limitation on the other side of the
connection. It has occurred to me that the buffer might be MAX_PATH but the ftp system
might choose to use UTF-8 encoding. Therfore, a 260-character Japanese name would need to
be encoded using 520 characters in UTF-8, and this might be where the problem is.

You can check for surrogates (although I suspect this is now NOT the problem!) by getting
the file name as a string and printing out the bytes of the string. The look to see if
any of the bytes are in the surrogate range. But I suspect that a UTF-8 encoding might be
the cuplrit.

Try encoding the filename in UTF-8. Note that you can do this by using my Locale
Explorer, choosing the MultiByte tab, pasting the Japanese filename in the top window, and
using WideCharToMultiByte with the UTF-8 locale selected. Then look at the result and see
what would happen if you accepted, say, the first 260 UTF-8 characters. Since some UTF-8
encodings take more than 2 characters, there is a possibility that this is what you are
seeing that creates the even smaller limit (128 vs. 130)

This is still just guesswork on my part, but I would rate it high in the list of probable
causes.
joe
On Tue, 29 Sep 2009 09:20:23 -0700 (PDT), ksr <sujatha.kokkirala@xxxxxxxxx> wrote:

On Sep 29, 1:38 am, "Mihai N." <nmihai_year_2...@xxxxxxxxx> wrote:
I am using WinInet API FtpFindFirstFile to enumerate files and folders
on FTP server. It works fine for filenames that have english
characters and filepath upto 260 characters. But for filenames that
have Japanese characters it fails.
For Japanese filenames it works fine upto 128 characters, but fails on
longer filenames. It is a unicode compiled project, my question is,
why is it failing to read upto 260 characters for japanese filenames.
I tried by explicitly using FtpFindFirstFileW, but it does not work.
Please help.

I would try to connect with a telnet to the ftp server and see if
it supports RFC 2640 ("Internationalization of the File Transfer Protocol")
Most servers don't.

If it is supported, then I would do some digging to see if FtpFindFirstFile
understands it. It is possible that it is not.

If it works for short Japanese file names, but not for longer ones,
I would suspect some buffer lenght parameter is wrong.

--
Mihai Nita [Microsoft MVP, Visual C++]http://www.mihai-nita.net
------------------------------------------
Replace _year_ with _ to get the real email


Thank you for your responses.

Joe, to your questions:

On a non-Japanese windows, using characters A-Z0-9, I can read
filenames upto 260 characters (path+filename+ext), explorer limits the
length to 260 characters and FTP can read this path.
On a Japanese windows, using characters A-Z0-9, again I can read
filenames upto 260 characters (path+filename+ext)

However on a Japanese windows, using Japanese characters I can
consistently see that it can read filenames upto 128 characters
(excluding extension). This is irrespective of path length (ie, path
could contain Japanese or A-Z0-9 characters). The number of characters
in the filename is 128 but byte count is 256, if you include file
extension, number of characters is 132 and byte count is 260.

It looks like there is a bug.

Can you explain how I can check this?
"Also, check whether or not your Japanese characters require Unicode
surrogates for UTF-16 encoding. "

Let me know.

Thanks,
ksr

Joseph M. Newcomer [MVP]
email: newcomer@xxxxxxxxxxxx
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
.

Relevant Pages

  • Re: Input Character Set Handling
    ... two 8-bit encoded characters matching 208 and 144. ... You wrote that you deal with say Japanese and Korean 'legacy' encodings ... neither for Russian in UTF-8 nor for Japanese ... I don't know why you did not know before about say .HTML files ...
    (comp.lang.javascript)
  • Re: Fast UTF-8 strlen function
    ... > How do I print a UTF-8 string to the console in Linux? ... technically able to render all those characters... ... Did you see my "Japanese Hello World" example? ...
    (alt.lang.asm)
  • Re: FTPFindFirstFile unicode
    ... be encoded using 520 characters in UTF-8, and this might be where the problem is. ... Explorer, choosing the MultiByte tab, pasting the Japanese filename in the top window, and ... It works fine for filenames that have english ...
    (microsoft.public.vc.mfc)
  • Re: Writing to the newsgroup?
    ... You should be able to read the Japanese I'll ... able to read each other's Japanese characters. ... However, if a UTF-8 post ... all show the same character encoding info: ...
    (sci.lang.japan)
  • Re: FTPFindFirstFile unicode
    ... It works fine for filenames that have english ... characters and filepath upto 260 characters. ... For Japanese filenames it works fine upto 128 characters, ...
    (microsoft.public.vc.mfc)