Re: What is the optimum way to insert data into database every 10 seconds?

Thank you very much Rick.

I have only a Hyper-V Windows Web Server 2008 machine. So, your suggestion is appropirate for two machines solution, as far as I understand, right?

Burak

"Rick Sawtell" <r_sawtell@xxxxxxxxxxxxxxxxxx> wrote in message news:OgcETpp6JHA.1568@xxxxxxxxxxxxxxxxxxxxxxx

"buran" <buran@xxxxxxxxx> wrote in message news:eQgnoSp6JHA.1372@xxxxxxxxxxxxxxxxxxxxxxx
Hello,

I am working on a web application that is supposed to get data from a data source on the net and update the database on the server every 10 seconds 24/7. I had worked with ADO.NET years ago, and know a bit about the disconnected structure, but I'm not sure what technology and method to use for optimum performance. Any suggestions are welcome...


Burak

That is a pretty wide open set of requirements there.

If it were me, I would architect in a few more requirements like:

1. The gathering of the data from the web should not impact the insertion of the data into SQL Server
2. The insertion of data into SQL Server should not impact the gathering of data from the web.

That said, I would approach this application asynchronously rather than synchronously.

I would use .NET message queues and a couple of windows services.

WindowsService1 would get the data from wherever on the net and deposit that data into the .NET message queue. The advantage of .NET queues is that if you have machine that is hosting the message queue (that is separate from your windows service machine), the windows service machine gets it's own .NET queue as well. The synchronization between the service machine and the queue machine is handled for you and basically guarantees transactional integrity. That means if the queue machine is down, you can still gather your data from the net and deposit it to your local message queue. When the queue machine is back online, everything sync's back up.

The second Windows Service would read the queue and insert/update the database appropriately. Using this approach, if the database is down for a while, but your Service1 is running, then when the db is back online, it can catch back up without missing anything.

As for the database structure, it depends on the nature of the data you are gathering, that said, in general, a 3NF structure would probably work best and be the most efficient.


Just my $0.02


Rick Sawtell
MCT, MCSD, MCDBA, MCITPro




.

Relevant Pages

  • [HPADM] QUESTION: Print Queue Issues
    ... This windows print server has Unix Printing Services ... If I pause the print queue on the windows server, ... based print jobs stuck in the queue using "lpstat". ...
    (HP-UX-Admin)
  • Re: Solaris 8 accessing shared NT print queues
    ... > using access method L to access the windows 2000 print queues without ... > creating a local queue on SAP solaris servers. ... SAPLPD as a service on the Windows server and use ...
    (comp.unix.solaris)
  • Re: Windows Service - Accessing Users Desktop
    ... Why do you need to start a "Windows" application on the server in the first ... what if there is no interactice logon or what if there is no ... the Windows Service is not able to find it. ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Microsoft Message Queue Server?
    ... Microsoft Message Queue comes as part of the NT Server 4 Option Pack, ... If using NT Server 4, ... If using Windows 2000 Server, it's included (install it from the control ... As far as I understand, if you want to install a queue controller, you need ...
    (microsoft.public.vc.mfc)
  • Re: broadcast 140 faxes all at the same time
    ... A fax message in server's queue is picked for sending only when a device is ... How are you saying that all faxes are being dialed at the ... > updates onto the windows 2000 server and the xp ... xp or microsoft fax or windows 2000 server. ...
    (microsoft.public.win2000.fax)
Loading