Re: File Compression (native NTFS) - help settle an argument

From: Lanwench [MVP - Exchange] (lanwench_at_heybuddy.donotsendme.unsolicitedmail.atyahoo.com)
Date: 12/18/04 

Date: Sat, 18 Dec 2004 14:29:22 -0500

Davie Robertson wrote:
> Hi Lanwench,
>
> I know this post was originally as a question, but don't you think it
> warrants discussion from a business perspective?
>
> I have also researched this in addition to having experience from a
> technical and business perspective (I've been using NT since the beta
> Win32 build), and for the use I have outlined I believe it's a valid
> option.

Sure - it exists, and you can use it if you wish. It just may not be the
best solution. I don't think it is.

> I am not against investing in storage for servers, but I
> believe it should be the right type of storage - throwing money at a
> problem is not always the right solution.

Yes, but throwing hardware at an issue of insufficient disk space is usually
the best solution from a technical standpoint.
>
> I wanted to know if anyone on here has any more technical reasons
> than a performance hit - all online measurable research I have found
> is based on NT4 running on 8 years old hardware (and even that had a
> 'negligible' performance hit).
>
> Unfortunately, I cannot just "try it and see" as that decision lies
> with our infrastructure team - and some of their KPIs are based on
> the amount storage they manage, i.e. the more storage they manage the
> better job they are doing = the more they cross-charge = the more
> profit they make = the more budget they get each year

I can't speak to that as it isn't a technical issue.

I'd think performance/speed would be a very big consideration for me -
probably the biggest one. Not just in terms of what the users experience,
but in terms of processor/I/O/general resource load on the server(s) in
question.

>From http://support.microsoft.com/kb/251186/EN-US/:
"While NTFS file system compression can save disk space, compressing data
can adversely affect performance. NTFS compression has the following
performance characteristics. When you copy or move a compressed NTFS file to
a different folder, NTFS decompresses the file, copies or moves the file to
the new location, and then recompresses the file. This behavior occurs even
when the file is copied or moved between folders on the same computer.
Compressed files are also expanded before copying over the network, so NTFS
compression does not save network bandwidth.

"Because NTFS compression is processor-intensive, the performance cost is
more noticeable on servers, which are frequently processor-bound. Heavily
loaded servers with a lot of write traffic are poor candidates for data
compression. However, you may not experience significant performance
degradation with read-only, read-mostly, or lightly loaded servers."

And from
http://www.microsoft.com/resources/documentation/Windows/2000/server/reskit/en-us/Default.asp?url=/resources/documentation/Windows/2000/server/reskit/en-us/core/fnee_exa_ewyg.asp -

"Limit your use of file compression or encryption. These features can add
some overhead and you should use them sparingly if not required by your
enterprise and if performance is critical. "

>
>
>
> "Lanwench [MVP - Exchange]" wrote:
>
>> Davie Robertson wrote:
>>> Thanks Troy (and Lanwench).
>>>
>>> However I don't agree with you both.
>>
>> OK, so try it and see. My advice is not a direct order - it's just
>> based on my experience and research. I think you'll find you take a
>> performance hit if do this.
>>
>>>
>>> We've got about 18,000 users and about 100 departments. Each Dept.
>>> has it's own share for people to exchange (and archive) the odd
>>> report, PowerPoint, spread***, etc (within the dept.). I know all
>>> about sharepoint etc (but I have to win each battle at a time).
>>>
>>> There are no application datastores on these shares, just office
>>> files that might get accessed up to 10 times a day. I cannot
>>> believe we should just (continue) to invest in more storage when
>>> we're not getting full use of the technology already in place - in
>>> this case NTFS compression.
>>
>> I advise against this, but then, I already said that.
>>>
>>> Further, with the introduction of XML based files formats the
>>> wastage of HD space will increase over time. There must be a point
>>> where it is more efficient to transfer xMb from a HD and uncompress
>>> it in memory before transferring it (x times 2) across the Lan than
>>> to load xMb direct from the HD.
>>>
>>> It's also worth pointing out that the cost of physical storage has a
>>> very impact on the cost to the end user in a large enterprise - most
>>> IT dept. charge based on the cost of managing the capacity - of
>>> which the capital cost is a small %
>>>
>>> I guess I'm too 'old school' :)
>>>
>>>
>>> "troy" wrote:
>>>
>>>> You would be better served to you purchase more disk space. The use
>>>> of compresed files is going to be slower. Disk Space is cheap, time
>>>> to upgrade :)

Loading