Re: Help on sppeding up a 15,000 line batch process
From: VM (vonchi_m_at_yahoo.com)
Date: 04/15/04
- Next message: Charlie Williams: "RE: How to run a function with parameters by delegate?"
- Previous message: Dave Bailey: "Re: Invalid Cast"
- In reply to: Gary Hunt: "Re: Help on sppeding up a 15,000 line batch process"
- Next in thread: VM: "Re: Help on sppeding up a 15,000 line batch process"
- Messages sorted by: [ date ] [ thread ]
Date: Thu, 15 Apr 2004 12:19:10 -0400
Thanks for your suggestion. Unfortunately, the "database" are a bunch of
proprietary files. Since everything's so prorietary, the only interface I
have to search for data is the DLL call and the struct that I use to pick up
the data.
But thanks anyway,
Vaughn
"Gary Hunt" <betaplace@codequest.co.uk> wrote in message
news:ewIBBssIEHA.3220@TK2MSFTNGP12.phx.gbl...
> Looking back at your original code - I think in this circumstance there
may
> be a "better" way.
>
> Am I right in saying that you are taking data from an SQL server or
> equivalent, putting it into a datatable and processing each row?
>
> Well if so - and if we've identified that it is the .Net String -> ANSI
> conversion that is taking the time then maybe we can get the data out of
the
> SQL server in ANSI format and never use unicode...
>
> It would be possible - and stop me if I'm butchering things too much
(hacker
> at heart!) - to use something like the following from SQL server which
would
> result in binary data we could put straight into the struct....
>
> SELECT CAST([fbu] AS varbinary) AS fbu, CAST([del] AS varbinary) AS del,
> CAST([cty] AS varbinary) AS cty, CAST([z4] AS varbinary) AS z4 FROM
[table]
>
> This will result in a datatable / sqldatareader that now has ANSI data in
> the columns. They will be presented as byte arrays which is exactly what
an
> ANSI string is.
>
> What do you think - too much of a fudge or an acceptable performance
> compromise...
>
> There are other items that would need to be changed as you are marshalling
> as TStr's which have their length before the data itself - so the class
> would look like
> byte iprubLen;
> byte[] iprubLen;
>
> Dunno - just think out loud I guess...
>
> g
>
>
> "Michael Giagnocavo [MVP]" <mggUNSPAM@Atrevido.net> wrote in message
> news:OgsffhlIEHA.3128@TK2MSFTNGP10.phx.gbl...
> > Are you only using ASCII (7-bits)? If so, perhaps you can do your own
> > marshalling and speed it up (assuming that the marshaller is converting
> > unicode to some different codepage). The reason being that going to
ascii
> is
> > simply taking the first 7bits, while true encoding conversion is more
> > involved.
> >
> > Perhaps you should summarize the problem now that you've narrowed it
down
> > (right?) and post to ms.public.dotnet.framework.performance.
> >
> > -Michael
> > MVP
> >
> > "VM" <vonchi_m@yahoo.com> wrote in message
> > news:%23wGVvlkIEHA.2164@TK2MSFTNGP12.phx.gbl...
> > > So assuming it just accepts ANSI, would it be possible that calling
this
> > > DLL
> > > function (made in C) from within a C application is so much faster
than
> > > converting the C-struct to C# and marshalling the data for the
interop?
> > > The previous version (made in C) did the whole process in less than 4
> > > mins.
> > > while the C# version takes over 30 mins.
> > >
> > > Vaughn
> > >
> > > "Michael Giagnocavo [MVP]" <mggUNSPAM@Atrevido.net> wrote in message
> > > news:uClF4SkIEHA.2236@TK2MSFTNGP10.phx.gbl...
> > >> If the unmanaged code is expecting ANSI, and you pass Unicode, it's
> just
> > >> going to mess up :(.
> > >> -mike
> > >> MVP
> > >>
> > >> "VM" <vonchi_m@yahoo.com> wrote in message
> > >> news:OC5GHFjIEHA.1220@tk2msftngp13.phx.gbl...
> > >> > If I use Unicode and the struct doesn't have legible data (after
> being
> > >> > modified), then I must use ANSI? Isn't there some way to cast the
> data
> > > so
> >
>
>
- Next message: Charlie Williams: "RE: How to run a function with parameters by delegate?"
- Previous message: Dave Bailey: "Re: Invalid Cast"
- In reply to: Gary Hunt: "Re: Help on sppeding up a 15,000 line batch process"
- Next in thread: VM: "Re: Help on sppeding up a 15,000 line batch process"
- Messages sorted by: [ date ] [ thread ]
Relevant Pages
|
