Re: SQL Server Import CSV data and skip duplicate rows

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

Hi
I would create a linked server , import all the data into SQL Server and
then deal with duplicates

http://dimantdatabasesolutions.blogspot.com/2007/02/dealing-with-duplicates.html






<Michael Slater> wrote in message news:2009102933547slaterm61@xxxxxxxxxxxx
Hi,
I have a SQL Server 2005 Database, and want to import data into the
Database from a series of .csv files that get generated on a nightly
basis.

It is likely that the csv files will contain duplicate rows, so i need to
be able to skip the duplicate rows.

This is with SQL Server 2005 Express edition.

Thanks,
Michael


EggHeadCafe - Software Developer Portal of Choice
.NET 2.0 Application Restart When it Goes "KABOOM"
http://www.eggheadcafe.com/tutorials/aspnet/1bad01a6-ebbb-4cb2-96c0-abf68699dcd7/net-20-application-rest.aspx


.

Relevant Pages

  • Re: DTS Speed Issues
    ... Allan Mitchell MCSE,MCDBA, (Microsoft SQL Server MVP) www.allisonmitchell.com - Expert SQL Server Consultancy. ... My plan is to write an app in .Net as you suggested to monitor> the filesystem from what I can tell via MSDN documentation it will watch all> the subdirectories.. ... don't know if I'll go as far as to create a DFS because> its always uploading to the same network directory. ... My next Question is this,> there are duplicate filenames but in the original directory structure they> are in specific folders so that isn't a problem. ...
    (microsoft.public.sqlserver.dts)
  • Re: SQL Server Import CSV data and skip duplicate rows
    ... I have a SQL Server 2005 Database, and want to import data into the Database from a series of .csv files that get generated on a nightly basis. ... It is likely that the csv files will contain duplicate rows, so i need to be able to skip the duplicate rows. ...
    (microsoft.public.sqlserver.programming)
  • SQL Server Import CSV data and skip duplicate rows
    ... I have a SQL Server 2005 Database, and want to import data into the Database from a series of .csv files that get generated on a nightly basis. ... It is likely that the csv files will contain duplicate rows, so i need to be able to skip the duplicate rows. ...
    (microsoft.public.sqlserver.programming)
  • Re: Out of Memory When using COM Objects
    ... we use SQL Server 2000 database. ... The strange thing is when I try to duplicate the problem, ...
    (microsoft.public.inetserver.iis)
  • Re: BULK INSERT ignores UNIQUE index with IGNORE_DUP_KEY set?
    ... you get a warning and not an error - is that correct? ... SQL Server only emits a warning. ... CREATE TABLE blamblam ... Duplicate key was ignored. ...
    (comp.databases.ms-sqlserver)