scaling out data access
- From: tin <tin@xxxxxxxxxxxxxxxxxxxxxxxxx>
- Date: Sat, 17 Sep 2005 15:33:01 -0700
requirment: scaling out data access to increase overall system capacity while
decreasing individual request execution time, without server clusters <-- is
that possible??
a bit trade off in idv. request execution time is acceptable if the whole
system would scale. expensive server cluster is out of the equation. i can
think of 2 solutions:
#1 - data replication (transactional model), 2 concerns:
a) all testing i have done, shows that there's at least 10 seconds delay in
replicating the data? would sql 2005 improves the latency?
b) the data is still accessed from hard disk, so it doesn't really improve
performance? i know in linux/unix, you can mount the memory to a drive, and
use the drive for data access?? is there something like this in windows
2003??
#2 - use ado.net (most likely 2.0) to load all the data from the database to
a dataset, this way data can be accessed from the memory, and i can load the
data to multiple servers to balance load? a few concerns:
a) is ado.net 2.0 designed for heavy massive concurrent read/write?
b) would the DataTable.Select() really selects faster than the SQL Select
against the database?? i have half a million business records in a table,
let's say to match a business name? i heard in 2.0, the indexing engine is
improved? does it also benefits the datatable.select method?
c) i know i need to handle synchonizing the data between the different
servers holding the dataset, and the sql database itself, is there anything
in ado.net 2.0 and sql 2005 that would help, or is there anything i should
pay more attention to? i guess the query notification and ado.net row change
events would help?
d) this is a big problem. the dataset would be loading half a million
business data, which is a few hundreds MB big. and the dataset is going to
be hosted in an asp.net web application/service. that means any file change
in the appl, would cause a recompile/restart, and the dataset needs to be
loaded again, is there a way to avoid that??
.
- Follow-Ups:
- RE: scaling out data access
- From: Kerry Moorman
- RE: scaling out data access
- Prev by Date: Generating parameter names in ADO.NET 2.0
- Next by Date: Casting Untyped DataSet to Typed DataSet
- Previous by thread: XML Schema Generated DataSet Persistence Pb
- Next by thread: RE: scaling out data access
- Index(es):
Relevant Pages
|
