Re: Capacity Planning / Hardware limits
- From: Steve Dz <Steve Dz@xxxxxxxxxxxxxxxxxxxxxxxxx>
- Date: Fri, 4 May 2007 14:24:01 -0700
Thanks for the reply!
First of all, you need to tell us if we are discussing live playoutThe large scale events will be live, then rolling to traditional on-demand
webcasting or streaming static video's on demand.
usage. The actual mix will probably be 95% live stream but during parts of
the year dip down to maybe 70% live stream, the rest being on-demand.
Getting 30,000 concurrent viewers on a livestream will be alot easierAs expected because disk I/O isn't an issue.
compared to serving 30,000 concurrent ondemand video views.
Getting 30,000 concurrent livestream viewers will be a matter ofIf we were friends, I'd make a sarcastic remark here, like "duh" ; )
determining maximum serverload and then add servers till you get where
you want to be.
Determining the maximum load of a server will be the hard part, I wasIt is intended for the live stream events.
not able to find much accurate benchmarks on recent hardware.
Modern hardware has made a big leaps since the P2 CPU's used in the
old benchmarks though.
1) Caching 'on demand' video files will be an interesting topic of
discussion. Basically disk I/O performance will be your first
bottleneck when serving on demand video files.
As far as I know Windows Media Services has no out-of-the-box cachingI was refering to setting server A as the source of the live stream,
or distribution mechanism built-in. You should be aware of the fact
that WMS will utilize 512MB of RAM at most for caching. Adding alot of
RAM to your servers is a waste of resources from what I have seen. I
would really like to know why Microsoft chose to only use 512MB of
system memory. WMS does provide support for custom 'caching' plugins,
I would love to hear what people have produced in this area. The SDK
supposedly has source examples availble.
assumingly fed from a Windows Media Encoder running at the event. Servers B,
C, D, etc. would then publish a "remote publishing pont" so that server A was
never overloaded, and servers B -> ? would handle individual client
connections. This way I would be able to scale up and keep server A's
performance stable. Access ing servers B -> ? would be via a hardware load
balancer. In Longhorn, there seems to be built in support for this (Proxy)
I would really like to know if 64bit WMS can cache more than 512MB in
memory!
http://www.microsoft.com/windows/windowsmedia/forpros/serve/features.aspx
infers that 64 bit increases performance by 40%, and Longhorn an additional
40%.
For performance and redundancy sake you will need to run multiple
servers. Are you going to use central storage (SAN) or direct attached
storage (DAS) to store your video files? You will either need a really
fast, large and stable (single point of failure) SAN or face the
problem of file replication to multiple Windows Media servers.
Depending on the pool of video's and how it is used, at any time 99%
of the requests could be for 1% of your data. Succesfully implementing
somesort of caching mechanism will greatly offload your spindles and
vastly increasy your capacity. You will want to run a minimum of 2
networks on your servers, one for server to server communication and
one to talk to your end-user.
SAN Storage will most likely be used for the video files.
I am really curious as how other people are handling caching.Great information!
2) Modern clocked multicore Xeon CPU's are fast and likely not a
bottleneck in your system. A modern server should be able to fully
saturate a Gigabit ethernet channel easily. We use dynamic link
aggregation on Intel multiport Gigabit Nics to create multi-gigabit
trunks to our core router (802.3AD).
We are still not sure where the limit is here but we have been able toAlso great information!
succesfully saturate nearly 2 Gbit per server during live events.
Microsoft has an application called 'Windows Media Load simulator'
available but stress testing multi gigabit links isnt that easy.
3) One of the downfalls of 32bit numbers is the limit to 4,294,967,296
in size (2^32). Windows stores the amount of memory in bytes, and
since a 32bit number can only hold 4,294,967,296, there is a limit of
roughly 4 billion bytes (or 4 gigabytes). Basically 32bit Windows can
not address more than 4Gigabyte of memory. Windows will segment the
available 4GB into 2GB of userspace memory and 2GB kernelspace memory.
I have seen WMS will not use over 512MB for file caching so this is no
problem. However, for each client connection WMS will need to reserver
some memory. If you want to run alot of concurrent connections it will
be possible for WMS to run out of available 2GB user space memory.
Some tweaks are possible like using Physical Address Extension and
dividing the kernelspace memory differently but my advice would be to
always use 64bit Windows. Driver or 3rd party application
incompatibility may be a problem though but this is easily checked.
4) In a 30,000 concurrent viewer setup you will want at least one loadI was expecting to use either a Foundry ServerIron or F5 Big-IP. Thanks for
balancer. Traditional loadbalancers are bad for streaming media since
they will want to route return traffic through the loadbalancer. On
high volume traffic this is either impossible or will require very
expensive network equipment. Foundry networks is known for its DSR or
Direct Server Return / Response but Cisco or other vendors will have a
similar solution.
the heads up on the DSR/R method. I'll look into that as well.
Incoming requests are handled by the loadbalancer to a real server but
return traffic will flow directly back to the client requesting the
data.
More servers, bigger pipes!
It is hard to get some absolute numbers and limits in high volume
setups because alot of variables are involved.
You will want your platform scalable because tomorrow the 300Kbit
stream will be a 800Kbit stream served to 60,000 people.
.
- References:
- Re: Capacity Planning / Hardware limits
- From: Kokkers
- Re: Capacity Planning / Hardware limits
- Prev by Date: Re: Preventing other encoders from streaming to Win Media Server
- Next by Date: Re: Help! (Repeat Post) Possible to bypass security for Windows Media files?
- Previous by thread: Re: Capacity Planning / Hardware limits
- Next by thread: WMS9 server hang when using Unicast and Multicast is not work
- Index(es):
Relevant Pages
|
