I recently encountered a situation where I needed to provide a compressed byte stream to Java’s Inflator class. This probably isn’t a situation that you run into a lot as a .NET developer, but to make a long story short, my FluentCassandra library had to send a compressed byte stream to the database server in order to execute a query.

The part that got me scratching my head is the fact that nothing in .NET Framework mentions ZLIB as a supportable compression type. So I had to start hunting down the specification and found that you can actually generate ZLIB compatible compression with the DeflateStream. However the format that Java and other libraries expect has a header in the stream as well as an adler-32 checksum at the end.

Here is the solution that I came up with that can be seen below and also on GitHub’s Gist.

public static byte[] ZlibCompress(byte[] data)
{
	using (MemoryStream outStream = new MemoryStream())
	{
		// zlib header
		outStream.WriteByte(0x58);
		outStream.WriteByte(0x85);

		// zlib body
		using (var compressor = new DeflateStream(outStream, CompressionMode.Compress, true))
			compressor.Write(data, 0, data.Length);

		// zlib checksum - a naive implementation of adler-32 checksum
		const uint A32Mod = 65521;
		uint s1 = 1, s2 = 0;
		foreach (byte b in data)
		{
			s1 = (s1 + b) % A32Mod;
			s2 = (s2 + s1) % A32Mod;
		}

		int adler32 = unchecked((int)((s2 << 16) + s1));
		outStream.Write(BitConverter.GetBytes(IPAddress.HostToNetworkOrder(adler32)), 0, sizeof(uint));

		// zlib compatible compressed query
		var bytes = outStream.ToArray();
		outStream.Close();

		return bytes;
	}
}

I am putting this code out there, so nobody else has to hunt for hours trying to find a solution that should be built into .NET given the popularity of the ZLIB compression format.

One of the many recommendations that Yahoo makes on optimizing your web site for high amounts of traffic, and to make the response time speedier to your user is GZip encoding all your static content. I usually do this as a standard for setting up any of my Web Servers, in addition to setting expiration headers on my static content, to ensure that I am serving as little content as possible.

IIS 7 has improved and simplified the compression and serving of static files by making it easier to setup and configure than previously in IIS 6.0. The IIS 7.0 compression works perfectly for CSS, HTML, and Text files, however JavaScript is another story. The JavaScript files on my IIS 7.0 server were not being compressed and served with GZip encoding, which is a major problem for any Web 2.0 site where 75% of your content severed per request is JavaScript. (I just made that number up, but it sounds right!)

I found Rick Strahl’s post on this very subject that he wrote up about a 9 months ago. It was helpful in diagnosing my problem, however it didn’t solve it. The HTTP compression is configured in IIS 7.0′s ApplicationHost.config file (c:\windows\system32\inetsrv\config\applicationhost.config), see below for the default settings:

<httpCompression directory="%SystemDrive%\websites\_compressed" minFileSizeForComp="0">
    <scheme name="gzip" dll="%Windir%\system32\inetsrv\gzip.dll" />
    <staticTypes>
        <add mimeType="text/*" enabled="true" />
        <add mimeType="message/*" enabled="true" />
        <add mimeType="application/javascript" enabled="true" />
        <add mimeType="*/*" enabled="false" />
    </staticTypes>
</httpCompression>

As you can see anything that starts with the MIME type of text or message is GZip encoded just fine. However there is also application/javascript as a compressible MIME type, there is nothing wrong with that, because there are 3 accepted ways to set a JavaScript MIME type.

1. text/javascript
2. application/x-javascript
3. application/javascript

However the problem comes in when you look at the default MIME type mappings setup, in the same ApplicationHost.config file, a little further down.

...
    <mimeMap fileExtension=".jpg" mimeType="image/jpeg" />
    <mimeMap fileExtension=".js" mimeType="application/x-javascript" />
    <mimeMap fileExtension=".jsx" mimeType="text/jscript" /
...

As you may notice the MIME type for JavaScript files is set to application/x-javascript, which is not the same as the default in the compression section above. So I added the following MIME type, application/javascript, to my Web.config thinking I had the problem licked, and all that I had to do was change the default MIME type for JavaScript files.

<system.webServer>
    <staticContent>
        <remove fileExtension=".js" />
        <mimeMap fileExtension=".js" mimeType="application/javascript" />
    </staticContent>
</system.webServer>

However that didn’t work either, and it should have because the MIME type now matched my compression MIME type. I even verified the MIME type in fiddler. So I then tried my last option to change the MIME type to text/javascript, which is the defacto standard on the internet for JavaScript MIME types.

<system.webServer>
    <staticContent>
        <remove fileExtension=".js" />
        <mimeMap fileExtension=".js" mimeType="text/javascript" />
    </staticContent>
</system.webServer>

Finally, this was the key to getting the JavaScript GZip Compression working IIS 7.0. And this didn’t require me to modify the ApplicationHost.config file get it done. Which is something I love about the new IIS 7.0, I can do my whole server configuration through FTP and my Web.config file.