Re: get wide character and multibyte character value


On Wed, 23 Jan 2008 23:13:01 -0800, George
<George@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote:

Hello everyone,


I need to know the wide character (unicode) and multibyte (UTF-8) values of
a character string of czech. I personally know nothing about czech. Is the
following approach correct?

1. I use L on the character string and watch memory to get the wide
character representation of the character string in little endian form;

2. I change the computer region/language to czech, and use function
WideCharToMultiByte, and use CP_ACP as input code page and use the L
character string as input to get the output multibyte character string output
from parameter lpMultiByteStr.

Is (1) and (2) correct? Any more efficient and smart ways?


thanks in advance,
George

The others already answered this better than I could, but I think I
might be able to clarify a couple things (I have been working on
converting an application to unicode for weeks, so this is all very
fresh in my head)

With the precompiler option UNICODE defined and you have a string
which is wchar_t (a string with L"" at the start will be wchar_t),
then what you see viewing memory in Visual Studio is that string in
16-bit unicode.

If you use WideCharToMultiByte with CP_ACP as the CodePage argument,
what you are doing is converting the 16-bit unicode to the current
Windows code page on your computer.

Code Pages are pretty much obsolete. Code pages were a way of
representing different characters not in the 0-127 ASCII range (like
Czech characters). Unicode is designed to represent all characters
that exist, where code pages are only designed to represent enough
characters for you to work with on your computer. So, converting to
your computer's code page, you are actually going from a more
versatile character set to a less versatile character set.

To get the UTF-8 version of the string, use UTF-8 instead of CP_ACP.

Which version you want depends on what you're doing. Within Windows,
generally the wide char version is what you want. UTF-8 is more
generally used for unicode in general, if you are interacting with
third-party software and they refer to unicode, they will frequently
mean UTF-8. If the software is halfway-decently documented though,
they should tell you explicitly which encoding they mean.

So there are three character encodings in what you're talking about,
UTF-16 (not completely identical with wchar_t but basically the same
in Windows) which is the Windows internal Unicode, UTF-8, and your
local code page.

(I know I am not explaining everything perfectly ... like the
difference between wchar_t and UTF-16 ... but I think as far as your
question goes you should get the idea.)

It can be a little confusing, because when people say "unicode", it
can mean a number of different things, and it isn't always clear which
they mean.

--
Joe Cosby
http://joecosby.com/
My cat seems to produce enough extraneous hair to create another whole
cat about twice a year. They actually seem to be trying to form under
various tables and bookshelves, but only in uncarpeted areas. They
never get much beyond a sort of misty transparent stage and then they
get blown out into the middle of the room.
- nenslo

:: Currently listening to John Shirley Reads 1996 1997 1998, by John Shirley, from "3-Fisted Tales of "Bob" Audio Book"
.

Relevant Pages

  • Re: Unicode Support
    ... > Not knowing much about UTF-8 (my Unicode knowledge extends as far as ... > literal strings of this form as long as the character code for quote ... > can never appear in a MBCS (multibyte character sequence). ... then XP Notepad directly understands UNICODE and you can ...
    (alt.lang.asm)
  • Re: Attention: European C/C++/C#/Java Programmers-Call for Input
    ... Simply make a straight decision now - you will use UTF-8. ... character format) much like UTF-8 which itself ... I would have little more than UNICODE left. ... generator is assembly language. ...
    (comp.arch.embedded)
  • Re: Posting with XHR and ISO-8859-15
    ... UTF-8 code units can be byte values ... Latin-9, and Unicode are the same, so there wouldn't be any troubles ... URIs, I can't use encodeURIComponent. ... ISO-8859-xx in the sense that not every character that can be encoded ...
    (comp.lang.javascript)
  • Re: Attention: European C/C++/C#/Java Programmers-Call for Input
    ... No other encodings - no Latin-1, no UTF-16, no home-made character sets, no extra fonts. ... Look at existing tools and source code that supports UTF-8, and see how it can make your work easier and give a result that users might actually be able to *use*. ... A couple of days work here is a drop in the ocean compared to the man-years it will take to work with your home-made encoding, and you will at least have the benefit of a better understanding of your problem. ... I would have little more than UNICODE left. ...
    (comp.arch.embedded)
  • Re: Unicode Delphi Win32 - which approach
    ... UTF-8 encoding is different from ANSI,... ... The first 256 Unicode characters map to the ANSI character set. ... Delphi supports UCS-2 on both platforms. ...
    (borland.public.delphi.non-technical)
Loading