Re: Avoiding dupes when merging files

From: Lucas Tam (REMOVEnntp_at_rogers.com)
Date: 11/25/04 

Date: Thu, 25 Nov 2004 19:55:40 GMT

"Bob Hollness" <bob@blockbuster.com> wrote in
news:#UQFGZy0EHA.3616@TK2MSFTNGP11.phx.gbl:

>>
>> Take a look at the Microsoft Text Driver - you can run SQL queries on
>> the text file. Perhaps you can just query each file checking for
>> dupes?
>>
>> Or you could load the data into a datatable (or hash table type
>> object?), with the PK set as the filename... if a duplicate shows up,
>> the datatable should throw a duplicate PK exception which you would
>> catch and ignore.
>>
>> Or lastly... perhaps you should think of a different method of
>> storing the data? Maybe a database is a better idea than text files?
>>
>> --
>> Lucas Tam (REMOVEnntp@rogers.com)
>> Please delete "REMOVE" from the e-mail address when replying.
>> http://members.ebay.com/aboutme/coolspot18/
>
> Thanks for the fast reply. I have to use text files so that really is
> not an option. Any pointers or some sample code on how to use the
> datatable? I like the idea of being able to trap a dupicate OK error.

I replied to your message a bit early in the day, but I'm not sure if
you received it:

Here's the example from MSDN (particularly the SetPrimaryKeys Sub):

http://msdn.microsoft.com/library/default.asp?url=/library/en-
us/cpref/html/frlrfsystemdatadatatableclassprimarykeytopic.asp

I've used it a couple of times and it works fine.

Here is what you do in short:

1. Add your columns to a datatable.
2. Add the same column from step 2 into a primary key array.
3. Add the primary key array to the DataTable.PrimaryKey property. 

-- 
Lucas Tam (REMOVEnntp@rogers.com)
Please delete "REMOVE" from the e-mail address when replying.
http://members.ebay.com/aboutme/coolspot18/

Relevant Pages

  • Re: Avoiding dupes when merging files
    ... > I currently have 2 text files which contain lists of file names. ... Perhaps you can just query each file checking for dupes? ... Or you could load the data into a datatable, ... should throw a duplicate PK exception which you would catch and ignore. ...
    (microsoft.public.dotnet.languages.vb)
  • Command line to remove duplicate files?
    ... I have a Fedora 6 system and want to remove some duplicate files. ... It worked great and found all duplicates but did not remove the dupes. ... ~Ohmster | ohmster /a/t/ ohmster dot com ... Put "messageforohmster" in message body ...
    (alt.os.linux)
  • Re: creating view to select unique records
    ... and load it into oracle via sqlldr. ... The source table has duplicate rows in it. ...
    (comp.databases.sybase)
  • Re: Duplicate Record Mess
    ... If you have a match between NAMES and HuskyTemp, ... instance, if you have more than one (duplicate), but NO record in HuskyTemp, ... If it doesn't matter how many dupes in NAMES, ... By the way, if John D. and his son John both live at the same address, ...
    (microsoft.public.access.queries)
  • Re: Extracting duplicate records
    ... daniel wrote: ... > of the duplicate records. ... > dupes and view the information in non-indexed columns to make my ... return the dupes. ...
    (microsoft.public.sqlserver.server)