Re: Max number of documents / performance

First off, SQL 2000 supports something called change tracking which will
index new documents almost real time, you don't need to completely rebuild
your index.

The rest of the answers are inline.

--
Hilary Cotter

Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html

Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com



"William F. Kinsley" <bacardi@xxxxxxxxxxxxxx> wrote in message
news:FA71C7A1-36A7-45FB-9A79-6653A3D1DA0C@xxxxxxxxxxxxxxxx
I have a large database that support 1,500+ concurrent users running an
intense DB aplication. It currently supports scanning on documents(Using
OCR) and images as LOB in the DB. These table are configured for their own
filegroup so they can be on a different array then the relational data.
There are over a million scanned documents and about 3,000 - 5,000 are
added daily.

I now need to add the ability to perform fulltext searchs on these
documents and perforamce is a major requirement. (most of the servers are
high end 8 way 20-30 gigs of memory, fiber SANS, etc)

Some of my servers are running SQL2000, and a few new ones are running
SQL2005. Now I am aware that SQL2005 allows for adding the new documents
to the index without a complete index rebuild, but I must support both.
One of my issues is that the servers are used almost 24x7.

My question is:
1) Should the documents and fultext index service be moved to their own
server?

Possibly, it will reduce the contention on the tables if there is heavy
workload activity on that box. Then the complexity will be in keeping both
servers in sync.
2) What is a realistic number of documents that can be indexed per
server?

We are are 500,000,000. There are complexities with growing this large

3) What are my performance options? Thrid party tools?

Whenever you scale you run into problems. For example I know a large media
outfit with a 2 terrabyte database which has 100 index servers - they are
using one of the leading search vendors application. Whenever you scale you
need more machines. With the exception of SQL FTS which doesn't allow you to
query index properties by name.

4) How can index the new documents on a SQL2000 server without doing a
complete rebuild of the index?

Change tracking.
Thank you,
Bill


.

Relevant Pages

  • Re: sql 2005 active/passive cluster options
    ... that is storing data on a seperate SQL 2005 Standard box. ... SQL server had an hardware fault causing application downtime for ... their product is an active/passive sql cluster. ... terms of active/passive failover I need two identical servers (since ...
    (microsoft.public.sqlserver.clustering)
  • Re: Performance Monitor / Database Storage using ODBC
    ... This solution would be just working temporarily (a few servers), ... Andrew J. Kelly SQL MVP ...
    (microsoft.public.sqlserver.tools)
  • RE: permissions compatible with pre-Win2000 servers
    ... Based on your reply, the NT machine which running SQL 7 is not a PDC, BDC, ... With regards to the anonymouse connection to SQL, ... >Our concern is for our NT4 servers that are dedicated to running SQL7 ...
    (microsoft.public.windows.server.migration)
  • Re: Web App Security Model.
    ... SQL permissions are correctly restrictive (so worse case the allowed ... If these machines are standalone the threats posed by them are ... applications / implementation and whether their design has ... My company wants to have a few Windows Servers running web app's (ASPX ...
    (microsoft.public.security)
  • Rebuilding a license server
    ... I have two Win2000 servers to rebuild ... Both machines are AD domain controllers, ... Talent hits a target no one else can hit. ...
    (microsoft.public.windows.terminal_services)