As anybody who has run a growing website or blog knows, response time is going to get worse with the more users you have visiting your site. The users come from all angles, RSS feeds, homepage visits, search engine visits, people sealing your static files that you host, and pretty much anything else that can be served over HTTP. The solution to this problem is to off load your static content on to a Content Delivery Network or CDN. CDN providers cost a lot of money though, so it is nothing for us mere mortals with one server can afford.

But thanks to Google anyone can now run their own CDN for free on Googles servers. Lucky for you and me Google has made the process really painless and you can even have the CDN under you own domain name. In my case static.coderjournal.com.

What Is A Content Delivery Network?

According to Wikipedia:

A content delivery network or content distribution network (CDN) is a system of computers networked together across the Internet that cooperate transparently to deliver content most often for the purpose of improving performance, scalability, and cost efficiency, to end users. The first web content based CDNs were Speedera, Sandpiper, Mirror Image and Skycache, followed by Akamai and Digital Island.

Basically it is a network of computers around the world that serves your content to the end user closest to one of those many servers around the world. This method of delivery cuts down on server overload, DNS hops, and delivery time.

When sites like Microsoft, Yahoo, Google, or Amazon delivery content they use Content Delivery Networks (CDN’s) to host most of their content, especially static files such as images, stylesheets, downloads and anything else you can think of. The reason they do this is to reduce load on their application servers, that serve dynamic content, such as PHP or ASP.NET pages.

What Is Google App Engine?

So you may ask what is Google App Engine:

Google App Engine lets you run your web applications on Google’s infrastructure. App Engine applications are easy to build, easy to maintain, and easy to scale as your traffic and data storage needs grow. With App Engine, there are no servers to maintain: You just upload your application, and it’s ready to serve your users.

You can serve your app using a free domain name on the appspot.com domain, or use Google Apps to serve it from your own domain. You can share your application with the world, or limit access to members of your organization.

App Engine costs nothing to get started. Sign up for a free account, and you can develop and publish your application for the world to see, at no charge and with no obligation. A free account can use up to 500MB of persistent storage and enough CPU and bandwidth for about 5 million page views a month.

Google has also announced a very very affordable price plan that any mere mortal can afford. They are not ready to start charging people yet, but here are the details:

• $0.10 – $0.12 per CPU core-hour
• $0.15 – $0.18 per GB-month of storage
• $0.11 – $0.13 per GB outgoing bandwidth
• $0.09 – $0.11 per GB incoming bandwidth

How do I setup my own CDN using Google App Engine?

To use Google App Engine you need to do a couple things that readies you computer to publish your static content to Google. Please take note that my setup is for Windows, but you can easily modify the process for any other OS.

Setup

1. You need to download and install Python on your computer. You may already have it if you are using a Unix environment (i.e. Linux or Mac OS X). If you need to download it or would just like to check to see if it is up to date, please visit http://www.python.org/download/ and download the correct version for you operating system.
2. Install Python to c:\Program Files\ (all my scripts that I have designed to make the publishing to Google are going to be using this path).
3. You will also need Google App Engine SDK which is available at http://code.google.com/appengine/downloads.html. Download the version that is for you OS. Note that the SDK will check for the Python install, so make sure you install it before the SDK.
4. Sign up for Google App Engine at http://appengine.google.com/, you will need a valid Google account. I suggest you sign up for a Google Apps account and use that as your Google account. Why I suggest this will become apparent later on.
5. Once you are done with the setup process you need to create an application. Click the “Create an Application” and give your application a name (called “application identifier”). This is a unique name for all Google App Engine applications. For example I set my application identifier to “coderjournal”. Click though to the next part of the application, if this is your first time registering an application you need to specify your cell phone number and confirm your account with a SMS code that Google sends you.

Publish To Your CDN

1. Download my publishing files, hosted on my CDN, at http://static.coderjournal.com/downloads/coderjournal-cdn.zip
2. Create a directory on your computer specifically for you CDN files. My directory is c:\websites\static.coderjournal.com. Fill this directory with all your static files you want hosted on your CDN. Fill it full of all your css, downloads, flash, images, javascripts, videos, and anything else you want hosted.
3. Unzip the files I provided to you in step 1 into the directory you created in step 2.
4. Next we need to edit the YAML configuration file. Open the app.yaml file in your favorite text editor and change application: coderjournal to application: {your application identifier}.
5. Next go down and edit your static directories, in mine I have css, downloads, flash, images, and js. You can create your own by just modifying the ones I put in the file.
6. If you installed the Google App Engine SDK in the default directory and Python in c:\Program Files\ then skip to step 7. The next part is also required if you are using the x64 version of Windows, because Google App Engine SDK installs in c:\Program Files (x86)\. So change the paths in publish-cdn-coderjournal.bat to your actual paths.
7. Now double click on publish-cdn-coderjournal.bat and a command window will display. Fill in your Google account and password that you used to sign up for the Google App Engine account. And you content will start to publish.
8. You now have you own private CDN that can be accessed at http://application-identifier.appspot.com.

Using Your Own Domain (Optional)

1. If you created your own Google App as suggested up in Setup step 4, you can create your own custom domain for your CDN. If you didn’t, don’t worry just create one, and follow the steps below.
2. Go to the dashboard of your Google Apps and click “Add more services”.
3. Under other services you will see Google App Engine and a place to enter your application identifier. Enter you application identifier and click “Add It Now”.
4. It will take you to the next page where you enter in the domain you want for your CDN, I suggest something simple like static.yoursite.com.
5. Then you just need to follow the steps for adding a CNAME to your DNS and you are ready to go with you custom domain.

How do I use my own CDN?

Well this is the cool part! You just use the absolute path to your files. For example if you wanted to host the image to your right you would just use the following in your HTML:

Potential Gotcha: I forgot to mention that currently the files hosted statically are case-sensitive. I have reported this issue to Google, hopefully they will correct it soon. http://code.google.com/p/googleappengine/issues/detail?id=466

<img src="http://static.coderjournal.com/images/ideapipe-logo.png" />

It is really that simple. Now comes the cool part that I need your help with, and proof that this is really a true CDN. I would like to see how many different IP Addresses my CDN points to. So far I was able to find the following IP addresses:

• 72.14.207.121
• 64.233.179.121
• 66.249.91.121

That point to:

static.coderjournal.com

To see what IP Address you get on your local machine just pull up the command prompt and type:

ping static.coderjournal.com

Please report your findings in the comments below. I am sure everybody would love to see how big Google’s CDN really is.

Today I had the need to create a custom event using jQuery, in order to launch a customized form validation event from a global submit event. I did this so I could focus in on the first form field that had an error. My event from the global.js script, that is included on every page of IdeaPipe, looks like this:

$("form").submit(function () {
	var valid = $(this).validate();
	
	// if the form didn't validate then focus the input on the first error
	if (!valid) 
		$(this).find(":input[error]:first").focus();
		
	return valid;
});

This is pretty standard jQuery. What this code above does is set a custom function for the submit event for any <form /> tag on the page. The submit event will only be allowed to continue if a return value of true is returned from the function.

I was able to create this custom jQuery event with the following code:

jQuery.fn.extend({
	validate: function (fn) {
		if (fn) {
			return jQuery.event.add(this[0], "validate", fn, null);
		} else {
			var ret = jQuery.event.trigger("validate", null, this[0], false, null);
			
			// if there was no return value then the even validated correctly
			if (ret === undefined)
				ret = true;
			
			return ret;
		}
	}
});

There are two different states to this method. Primarily because in JavaScript all parameters are optional for functions. So the two states of this function are:

• validate(fn) – sets the event
• validate() – fires the event

An example of setting the event is:

$("form.user-login").validate(function () {
	var userNameValid = ValidateLoginUserName();
	var passwordValid = ValidateLoginPassword();

	return userNameValid && passwordValid;
});

In this example the form is valid if both the login user name and password validate.

An example of using the event is the same as the method above.

$("form").submit(function () {
	var valid = $(this).validate();
	// do some stuff	
	return valid;
});

This may not be the standard bind() and trigger() that most jQuery programmers are use to, but I needed an event that would return a value of true or false, so that I my submit event handler knows if it should focus on errors or continue the submit process.

Hope everybody finds this useful.

Recently for IdeaPipe I have been looking for ways to deliver my content more quickly and reduce unnecessary bandwidth use.

According to Yahoo’s Performance Team more than half of the viewers of the Yahoo websites start with an empty cache, which means the browser has to download all the resources for the first time. This combined with a high traffic website and unneeded white space and comments can really add up to a significant bandwidth use. There are many popular ways to minify your static content tax on your bandwidth, using many popular tools, as described in this excerpt from Yahoo:

In terms of code minification, the most widely used tools to minify JavaScript code are Douglas Crockford’s JSMIN, the Dojo compressor and Dean Edwards’ Packer. Each of these tools, however, has drawbacks. JSMIN, for example, does not yield optimal savings (due to its simple algorithm, it must leave many line feed characters in the code in order not to introduce any new bugs).

The goal of JavaScript and CSS minification is always to preserve the operational qualities of the code while reducing its overall byte footprint (both in raw terms and after gzipping, as most JavaScript and CSS served from production web servers is gzipped as part of the HTTP protocol).

The cream of the crop seems to be a tool Yahoo developed to deliver its own static text content scripts and styles, the YUI Compressor:

The YUI Compressor is JavaScript minifier designed to be 100% safe and yield a higher compression ratio than most other tools. Tests on the YUI Library have shown savings of over 20% compared to JSMin (becoming 10% after HTTP compression). Starting with version 2.0, the YUI Compressor is also able to compress CSS files by using a port of Isaac Schlueter‘s regular-expression-based CSS minifier.

The YUI Compressor is a Java JAR file that can be download from Julien Lecomte Blog.

The YUI Compressor yielded exceptional results, however it was missing one thing. Integration in to my build and deployment process. In IdeaPipe I use a MSBuild script to compile, manipulate, and prepare for publishing. So naturally I built a MSBuild Task to minimize my JavaScript and CSS files.

The magic actually happens by invoking Java in an external process for each file passed in to the task.

Process process = new Process();
process.StartInfo = new ProcessStartInfo {
	FileName = @"c:\program files\java\jdk1.6.0_06\bin\java.exe",
	Arguments = String.Format(
		@"-jar ""C:\development\tools\yuicompressor-2.3.5.jar"" --type {0} --charset utf8 {1} -o ""{2}"" ""{3}""",
		type,
		ShowWarnings ? "--verbose" : String.Empty,
		newFile,
		oldFile
		),
	UseShellExecute = false,
	CreateNoWindow = true,
	RedirectStandardOutput = true,
	RedirectStandardError = true
};
process.Start();
process.WaitForExit(5000);

Then I read the warning from the standard error output and send them back to Visual Studio as a compile warning if the ShowWarning property is true.

string[] warnings = process.StandardError.ReadToEnd()
	.Replace("\r", String.Empty)
	.Split(new string[] { "\n\n" }, StringSplitOptions.RemoveEmptyEntries);

foreach(string warning in warnings)
	Log.LogWarning(null, null, null, oldFile, 1, 1, 1, 1, FormatWarning(warning), null);

To integrate this in to my MSBuild script I had to first register my task:

<UsingTask TaskName="ManagedFusion.Build.YuiCompress" AssemblyFile="$(ProjectDir)..\ManagedFusion.Build\bin\$(ConfigurationName)\ManagedFusion.Build.dll"/>

Then setup my ItemGroup for the files:

<ItemGroup>
	<JavaScriptContent Include="$(SourceWebPhysicalPath)\**\*.js" />
	<CssContent Include="$(SourceWebPhysicalPath)\**\*.css" />
</ItemGroup>

Then finally I setup my task to perform the minimization against the JavaScript and CSS files seperately:

<Target Name="AfterBuild">
	<!-- do other stuff to prepare for publishing -->
	<YuiCompress Files="@(JavaScriptContent)" Type="JS" />
	<YuiCompress Files="@(CssContent)" Type="CSS" />
</Target>

You can easily incorporate this in to your own MSBuild scripts or even your Visual Studio Project which is just an MSBuild file for compiling your source code for the project. I have included my source code below:

Download: YUI Compressor MSBuild Task Source

Note: There are a couple of static paths to be on the look out for and modify as necessary for your own code. In my code the Java runtime is loaded at c:\program files\java\jdk1.6.0_06\bin\java.exe and the YUI JAR is located at C:\development\tools\yuicompressor-2.3.5.jar.

Update (2008-5-21): Thanks George, apparently IIS doesn’t like serving straight C# files. So I added the code to my Coder Journal Source Control, so that it can be downloaded from there.