03 Dec 2009

Sometimes you just need to CodingHorror it!

4 Comments Uncategorized

The title of this post is a tongue-in-cheek reference to SubSonic’s inline query by the same name, which in turn is a reference to the blogger Jeff Atwood’s blog who you should all know.  Rob Conery, the SubSonic project leader, named the inline query class CodingHorror after he allegedly read Jeff Atwood’s post titled Embracing Languages Inside Languages, in which he bestowed the virtues of inline SQL inside your code, instead of the standard bequeathed statement that I bet you all have heard “We do all database work in stored procedures.”.  Jeff outlined his though process on ad-hoc vs stored procs as follows:

The Subsonic project attempts to do something similar for SQL. Consider this SQL query:

SELECT * from Customers WHERE Country = "USA"
ORDER BY CompanyName

Here’s how we would express that same SQL query in SubSonic’s fluent interface:

CustomerCollection c = new CustomerCollection();
c.Where(Customer.Columns.Country, "USA");
c.OrderByAsc(Customer.Columns.CompanyName);
c.Load();

I’ve mentioned before that I’m no fan of object-oriented rendering when a simple string will suffice. That’s exactly the reaction I had here; why in the world would I want to use four lines of code instead of one? This seems like a particularly egregious example. The SQL is harder to write and more difficult to understand when it’s wrapped in all that proprietary SubSonic object noise. Furthermore, if you don’t learn the underlying SQL– and how databases work– you’re in serious trouble as a software developer.

Enough of the History lesson, why are you writing this post

Well the reason for this post is two fold, I haven’t written a post in a couple of weeks, and I wanted to write about my own CodingHorror moment, while working with the Entity Framework, that solved a big problem I was having with getting the LIKE statement to work in LINQ.

I am working with a client of mine trying to expose their data to the web via a simple REST service.  Their whole database model has been constructed in the Entity Framework in a simple active record format.  One of the requirements, I was given, was to support wildcards on a couple of the input fields.  I thought this was no big deal, because I knew that the StartWith, EndsWith, and Contains methods of the System.String object, when used in LINQ, translated down to the SQL LIKE operator. 

However at the time I didn’t anticipate how much of a pain in the butt it was to actually figure out which of these three methods I should use depending on where the wildcard was placed in the string.  And how I would support a wildcard that was placed in the middle of a string.

Enter Entity Frameworks’s parameterized Where method

The parameterized Where method, of the ObjectQuery<T> object, really saved my butt.  It allowed me to produce the exact effect of the query I was looking for, and with out a lot of hacking.  Here is what I did:

if (!String.IsNullOrEmpty(toAddress))
	table = table.Where(
		"it.ToAddress LIKE @toAddress", 
		new ObjectParameter("toAddress", toAddress)
	);

var results = 
	from result in table
	where result.Account.AccountId == accountId
		&& result.DateTime >= startDate

return View(results.ToList());

The best part of the above code is that I can still use LINQ, and jump in to standard SQL syntax when the LINQ syntax fell short. 

This is my opinion gives the Entity Framework a step above the rest, because it means that I don’t have to complicate my life and my program by moving to a stored proc or making the whole select statement inline-SQL (which I wouldn’t ever do). 

Features like this that make developing software easier, and obviously have the programmer in mind, are what we should all strive for.  I like this for the very fact that the Entity Framework team has acknowledged that LINQ, while a wonderful addition to .NET, doesn’t meet all the needs for pulling data from the database and they sought to minimize the short comings by allowing developers to jump back in to the natural SQL language.  This is a wonderful addition in my book.

17 Feb 2009

TF30042: The database is full. Contact your Team Foundation Server administrator.

5 Comments Uncategorized

Today I received the following error while trying to check in some code after a marathon night of coding:

TF30042: The database is full. Contact your Team Foundation Server administrator.

I got one of those “oh crap” sinking feelings, that some how my TFS server had decided to just die.  After doing a little research on this error, which there is very little (read close to none) information about on the internet.  So I gave up searching and decided to do a little trial and error adhock testing, and I found out that this error has nothing to do with the database but everything to do with the size of the database’s log file.  I came up with the following solution, that you will want to run in Microsoft SQL Server Management Studio:

WARNING!!! My TFS server is in a non-production environment and I am basically the only one who uses it.  Make sure to check with your network administrator and make a back up before you run the following code.

USE [master]

ALTER DATABASE [ReportServer] SET RECOVERY SIMPLE WITH NO_WAIT
ALTER DATABASE [ReportServerTempDB] SET RECOVERY SIMPLE WITH NO_WAIT
ALTER DATABASE [TfsWorkItemTracking] SET RECOVERY SIMPLE WITH NO_WAIT
ALTER DATABASE [TfsIntegration] SET RECOVERY SIMPLE WITH NO_WAIT
ALTER DATABASE [TfsVersionControl] SET RECOVERY SIMPLE WITH NO_WAIT
ALTER DATABASE [TfsWorkItemTrackingAttachments] SET RECOVERY SIMPLE WITH NO_WAIT
ALTER DATABASE [TfsActivityLogging] SET RECOVERY SIMPLE WITH NO_WAIT
ALTER DATABASE [TfsBuild] SET RECOVERY SIMPLE WITH NO_WAIT
ALTER DATABASE [STS_Config_TFS] SET RECOVERY SIMPLE WITH NO_WAIT
ALTER DATABASE [STS_Content_TFS] SET RECOVERY SIMPLE WITH NO_WAIT
ALTER DATABASE [TFSWarehouse] SET RECOVERY SIMPLE WITH NO_WAIT

ALTER DATABASE [ReportServer] SET RECOVERY SIMPLE 
ALTER DATABASE [ReportServerTempDB] SET RECOVERY SIMPLE 
ALTER DATABASE [TfsWorkItemTracking] SET RECOVERY SIMPLE 
ALTER DATABASE [TfsIntegration] SET RECOVERY SIMPLE 
ALTER DATABASE [TfsVersionControl] SET RECOVERY SIMPLE 
ALTER DATABASE [TfsWorkItemTrackingAttachments] SET RECOVERY SIMPLE 
ALTER DATABASE [TfsActivityLogging] SET RECOVERY SIMPLE 
ALTER DATABASE [TfsBuild] SET RECOVERY SIMPLE 
ALTER DATABASE [STS_Config_TFS] SET RECOVERY SIMPLE 
ALTER DATABASE [STS_Content_TFS] SET RECOVERY SIMPLE 
ALTER DATABASE [TFSWarehouse] SET RECOVERY SIMPLE 

DBCC SHRINKDATABASE(N'ReportServer')
DBCC SHRINKDATABASE(N'ReportServerTempDB')
DBCC SHRINKDATABASE(N'TfsWorkItemTracking')
DBCC SHRINKDATABASE(N'TfsIntegration')
DBCC SHRINKDATABASE(N'TfsVersionControl')
DBCC SHRINKDATABASE(N'TfsWorkItemTrackingAttachments')
DBCC SHRINKDATABASE(N'TfsActivityLogging')
DBCC SHRINKDATABASE(N'TfsBuild')
DBCC SHRINKDATABASE(N'STS_Config_TFS')
DBCC SHRINKDATABASE(N'STS_Content_TFS')
DBCC SHRINKDATABASE(N'TFSWarehouse')

The above code will actually put all the TFS databases in Simple Recovery mode, which basically means no log file, and then shrinks all the log files that were previously in use. After you run this script in Microsoft SQL Server Management Studio you should not get this error message anymore, when you try to check in your files.

09 Dec 2008

Creating an extension module for .NET URL Rewriter and Reverse Proxy

4 Comments Uncategorized

Wow that is a long title. Recently I have been looking for quick posts that I can put out each day to keep my blog relevant and also so I don’t feel like I am slacking off too much. Today I want to post about a little known feature in my .NET URL Rewriter and Reverse Proxy (aka. Managed Fusion URL Rewriter) that I have developed in my spare time, mostly out of necessity for this blog and other projects I have worked on.  Here is a quick run through of what it does.

Managed Fusion URL Rewriter is a powerful URL manipulation engine based on the Apache mod_rewrite extension. It is designed, from the ground up to bring all the features of Apache mod_rewrite to IIS 6.0 and IIS 7.0. Managed Fusion Url Rewriter works with ASP.NET on Microsoft’s Internet Information Server (IIS) 6.0 and Mono XPS Server and is fully supported, for all languages, in IIS 7.0, including ASP.NET and PHP. Managed Fusion Url Rewriter gives you the freedom to go beyond the standard URL schemes and develop your own scheme.

But one feature that I added that is not part of the official Apache mod_rewrite documentation is the ability to add custom modules to extend the use of the URL rewriter in non-traditional ways.  One great example of this was born out of wanting to clean up the SEO mess I created in the early days of this blog.  I had to support the following different types of URL patterns:

  1. http://www.coderjournal.com/?p=23
  2. http://www.coderjournal.com/2008/03/14/some-post.html
  3. http://www.coderjournal.com/2008/03/14/some-post

to transform them in to the URL pattern that I finally settled on today:

  • http://www.coderjournal.com/2008/03/some-post

In the above list #2 and #3 were pretty easy to transform using the following rules:

RewriteRule ^(/[0-9]{4}/.*).html$    $1/ [NC,R=301]
RewriteRule ^(/[0-9]{4}/[0-9]{1,2}/)[0-9]{1,2}/(.*)$    $1$2 [R=301]

Because they contained all of the elements that make up my current URL.  As you can imagine problems arose when I had to support links that used #1′s syntax.  It contains zero elements that I can use to create my current URL.  Being a programmer who beleives that each part of a system should handle gracefully the domain it was designed to support, in this case a URL rewriter should be able to handle any senario that has to do with URL rewriting.  I added in support that allowed developers to naturally extend the URL rewriter to accomplish any type of URL rewriting task they could think of.

Setting Up the URL Rewriter Rules

In my case I needed to handle the following SQL query everytime I saw a URL that matched #1.

select concat('http://www.coderjournal.com/',year(post_date),'/',month(post_date),'/',post_name,'/') from wp_posts where ID = $1;

What this query does is query the WordPress database table that contains all the posts by the post ID and have it return the actual absolute path to the post, that should be displayed in the URL.  To do this I created a new directive for the mod_rewrite syntax called RewriteModule.  I also had to extend the RewriteRule and RewriteCond directives to support these new module extensions.  The RewriteModule, RewriteRule, and RewriteCond are defined by the following syntax:

RewriteModule <Reference Name> <Namespace>,<Assembly>
RewriteRule[([<Left Module>],[<Right Module>])] <Pattern> <Substitution>
RewriteCond[([<Left Module>],[<Right Module>])] <Test String> <Condition Pattern>

The parts in light blue parts above are optional to creating the rule.  In my case for this blog the rewriter directives looked like the following:

RewriteModule PostQueryString CoderJournal.Rewriter.Rules.PostQueryStringRuleAction, CoderJournal.Rewriter.Rules
RewriteRule(,PostQueryString)   ^/\?p=([0-9]+)$    "select guid from wp_posts where ID = $1;" [R=301]

I have highlighted in red the important parts of the syntax that indicate the custom module processor that should be used on the RewriteRule directive and how it relates back to the class defined in the RewriteModule

Creating the Module

I have to warn you that I am not going to demonstrate and show all the properties and methods on the interface that are important for creating a custom module, but I am going to show you the actual meat of the module that is involved in the lookup of the URL from the database.

public Uri Execute(int logLevel, string logCategory, HttpContext context, 
                   Pattern pattern, Uri url, string[] conditionValues, 
                   IDictionary<string, string> flags)
{
	string inputUrl = url.GetComponents(UriComponents.PathAndQuery, UriFormat.UriEscaped);
	string sqlCommand = pattern.Replace(inputUrl, Text, conditionValues);
	string substituedUrl = String.Empty;

	using (MySqlConnection connection = new MySqlConnection(Properties.Settings.Default.DatabaseConnection)) {
		using (MySqlCommand command = connection.CreateCommand()) {
			command.CommandText = sqlCommand;
			command.CommandType = CommandType.Text;

			try {
				connection.Open();
				substituedUrl = command.ExecuteScalar() as string;
			} finally {
				connection.Close();
			}
		}
	}

	return new Uri(url, substituedUrl);
}

It may not be clear right away what is going on, but on line 6, I am replacing the defined value in the regular expression (^/\?p=([0-9]+)$) with the SQL query (from above) to produce a query that will be run against the database. So if the following URL came in to my server:

It would produce a SQL query that looked like this:

select concat('http://www.coderjournal.com/',year(post_date),'/',month(post_date),'/',post_name,'/') from wp_posts where ID = 372;

Notice that the ID, 372, shows up in both the URL and the query, that is because this is the part I am most interested in, in the URL, because it is the only part of the URL that I need to query the database to find the actual path of the post.

Now that we have the query we can execute it on the database, using lines 9 through 21, and create the resulting URL on line 23. The resulting URL is then passed back through the URL rewriter, and processed using the flags defined. In my case [R=301], actually indicates that I want to do a 301 Permanent Redirect on the URL, which tells the browser and search engines, a like, that they need to update their URL for this page.

You can test out the above conditions by using the following URL’s that all redirect back to this page:

  1. http://www.coderjournal.com/?p=372
  2. http://www.coderjournal.com/2008/12/9/creating-extension-module-net-url-rewriter-reverse-proxy.html
  3. http://www.coderjournal.com/2008/12/9/creating-extension-module-net-url-rewriter-reverse-proxy/

The code as always is available on my SVN server at Google Code.

I hope this comes in handy to some of you developers that have to support legacy URL’s in your own product or a project that you are working on. As always if you have any questions or need anything clarified please feel free to contact me or leave a comment below.