19 May 2010

.gitignore Config File For .NET Projects

7 Comments Uncategorized

I wanted to post this mostly for my future reference.  But I think it is also equally as useful to anybody else with a .NET project that is using the Git as their source control, and want to make sure non-code extras that come with .NET projects don’t get checked in.

.gitignore File

This file specifies the paths and files to ignore in your project.  Each line constitutes a new path, and each one can use basic RegEx to generalize the ignore checking.

To create the file run the following commands from your Git command prompt for windows use msysgit, or Linux and Mac OSX you can use the native command line for this.

touch .gitignore
{{ Add The Text In The Next Part }}
git add .gitignore
git commit -am "adding ignore checking paths for this project"
git push origin master

Or you can just create a file named “.gitignore” in notepad and save it and use your favorite Git client.

[Oo]bj/
[Bb]in/
*.suo
*.user
/TestResults
*.vspscc
*.vssscc

If you have any ignore statements that you find useful for your projects, I am interested in growing this file, so leave them down in the comments.

16 May 2010

TimeUUID only makes sense with version 1 UUIDs

2 Comments FluentCassandra

In a world where we are all use to dealing with objects we often forget that everything gets reduced to ones and zeros before being transmitted over the wire to the destination.  Most times the destination easily handles converting this object back in to an object on the other side that is easily understood and consumed.  The frustration comes when we run in to a situation where the other side doesn’t understand our transmitted data.  This can often cause us to pull our hair out, become irritable, and throw out hands up in disgust.  Well recently I have been doing all that when trying to solve what sounds like simple problem on the surface.  Sending the bytes of a Type 1 UUID, or GUID, over the wire from .NET to a server running on Java.

The Problem

It is a simple one, I need to transmit a Guid from .NET over to a UUID in Java.  Mind you Guid and UUID solve the same thing in the differently languages so this isn’t a case of mis-mapped different objects.

TimeUUID only makes sense with version 1 UUIDs

This error is the error that came up each and every time I sent the GUID object over to the Java Server using the following method.

Guid g = new Guid("38400000-8cf0-11bd-b23e-10b96e4ef00d");
var bytes = g.ToByteArray();

Server.Transmit(bytes); // this method call is an example

The part of this that really made me pull my hair out was the fact that the GUID that I was sending over was a valid Type 1 UUID.  If you don’t believe me check it out here.  So after writing a bunch of unit tests and double and triple checking the fact that the GUID I was sending over was a Type 1 GUID, it finally dawned on me that maybe the problem was the fact that Java handled the byte order of the UUID different than .NET.

So I wrote a quick Java program to verify this thesis.  And this is what I found.

UUID x = UUID.fromString("38400000-8cf0-11bd-b23e-10b96e4ef00d");
System.out.println(x.toString());
System.out.println(x.version());

// results:
// 38400000-8cf0-11bd-b23e-10b96e4ef00d
// 1

byte[] byteArray = { ... the bytes here ... };
ByteBuffer bb = ByteBuffer.warp(byteArray);
UUID y = new UUID (bb.getLong(), bb.getLong());
System.out.println(y.toString());
System.out.println(y.version());

// results:
// 00003840-f08c-bd11-b23e-10b96e4ef00d
// 11

You might have noticed that the byte order for the first 8 bytes of the UUID have been reversed for each section.  So I reversed the bytes in my byteArray and here is the output I received.

// results:
// 38400000-8cf0-11bd-b23e-10b96e4ef00d
// 1

I jumped for joy when I saw this come out correctly, because it meant that I had solved the problem.

But Not So Fast

I still needed to write the code on the .NET side to handle reversing and unreversing these bytes when sending over to and receiving from the Cassandra server.  So to correct this I just created a simple byte array converter for my Cassandra type TypeConverter object for TimeUUID. 

private byte[] ReverseLowFieldTimestamp(byte[] guid)
{
    return guid.Skip(0).Take(4).Reverse().ToArray();
}

private byte[] ReverseMiddleFieldTimestamp(byte[] guid)
{
    return guid.Skip(4).Take(2).Reverse().ToArray();
}

private byte[] ReverseHighFieldTimestamp(byte[] guid)
{
    return guid.Skip(6).Take(2).Reverse().ToArray();
}

public override object ConvertTo(ITypeDescriptorContext context, System.Globalization.CultureInfo culture, object value, Type destinationType)
{
    if (!(value is Guid))
        return null;

    if (destinationType == typeof(byte[]))
    {
        var oldArray = ((Guid)value).ToByteArray();
        var newArray = new byte[16];
        Array.Copy(ReverseLowFieldTimestamp(oldArray), 0, newArray, 0, 4);
        Array.Copy(ReverseMiddleFieldTimestamp(oldArray), 0, newArray, 4, 2);
        Array.Copy(ReverseHighFieldTimestamp(oldArray), 0, newArray, 6, 2);
        Array.Copy(oldArray, 8, newArray, 8, 8);
        return newArray;
    }

    if (destinationType == typeof(Guid))
        return (Guid)value;

    return null;
}

And a little coding and some unit tests later, I was able to verify that the UUID (or GUID) would be correctly transmitted to Java and read back from Java correctly with the changes I made above.  I don’t completely understand why the byte order is different, because UUID is a pretty solid standard.  But I am sure it has something to do with the Endianness of the two languages.

In The End

This turned out to more of an adventure than I thought it would be, and I had to break out some Java code to do it, but after about an hour coding and testing, everything seems to be working and TimeUUID is now supported under .NET. 

Note: By the way I really didn’t want to install Eclipse to test out Java snippet, luckily Mono Develop supports Java development.

05 Apr 2010

Creating a Time UUID (GUID) in .NET

4 Comments FluentCassandra

Previously I had written about how to setup Cassandra as a database on your Windows machine.  As I was diving in deeper to learn more about the subject, I realized that .NET lacks a critical type to Cassandra, for column comparison and sorting, called TimeUUIDType. TimeUUIDType is a Version 1 UUID used in the CompareWith attribute of the storage config file.  A Version 1 UUID is defined as the following:

Conceptually, the original (version 1) generation scheme for UUIDs was to concatenate the UUID version with the MAC address of the computer that is generating the UUID, and with the number of 100-nanosecond intervals since the adoption of the Gregorian calendar in the West. In practice, the actual algorithm is more complicated. This scheme has been criticized in that it is not sufficiently “opaque”; it reveals both the identity of the computer that generated the UUID and the time at which it did so.

What are the Cassandra CompareWith Types

The CompareWith attribute tells Cassandra how to sort the columns for slicing operations.  The default is BytesType, which is a straightforward lexical comparison of the bytes in each column.  Other options are AsciiType, UTF8Type, LexicalUUIDType, TimeUUIDType, and LongType.  You can also specify the fully-qualified class name to a class of your choice extending org.apache.cassandra.db.marshal.AbstractType.

SuperColumns have a similar CompareSubcolumnsWith attribute.

  • BytesType: Simple sort by byte value.  No validation is performed.
  • AsciiType: Like BytesType, but validates that the input can be parsed as US-ASCII.
  • UTF8Type: A string encoded as UTF8
  • LongType: A 64bit long
  • LexicalUUIDType: A 128bit UUID, compared lexically (by byte value)
  • TimeUUIDType: a 128bit version 1 UUID, compared by timestamp

NOTE: The CompareWith types are what we in the relation database world call Table keys.

How do these Types Relate to .NET Types

These types listed above have the following mapping to .NET types:

  • BytesType: Byte[]
  • AsciiType: String (generated by Encoding.ASCII)
  • UTF8Type: String (generated by Encoding.UTF8)
  • LongType: Int64
  • LexicalUUIDType: Guid (generated by Guid.NewGuid())
  • TimeUUIDType: Guid (no native way to generate from .NET Framework)

As you can see from above all of the types can easily be generated by .NET, except for the TimeUUIDType.

What is the point of the TimeUUIDType

I will let Arin Sarkissian describe why you should care about TimeUUIDType:

Since we’re going to want to display lists of entries in chronological order we’ll make sure each Columns name is a time UUID and set the ColumnFamilys CompareWith to TimeUUIDType. This will sort the Columns by time satisfying our “chronological order” requirement. So doing stuff like “get the latest 10 entries tagged ‘foo’” is going to be a super efficient operation.

As Arin’ says TimeUUIDTYpe is a “super efficient” way to perform chronological sorting and pulling of data from the Cassandra database.  And since our needs as developers to store chronological data in a database don’t really differ by programming language, I have created a Time UUID generator that can fit the data in to a standard Guid object.

The Time UUID generator was pretty easy to create after I figured out the byte array structure and the differences between how Java and .NET generate byte arrays.  Below is all the code you need to generate a Time UUID or Time-Based Guid object in .NET.

public static Guid GenerateTimeBasedGuid(DateTime dateTime)
{
	long ticks = dateTime.Ticks - GregorianCalendarStart.Ticks;

	byte[] guid = new byte[ByteArraySize];
	byte[] clockSequenceBytes = BitConverter.GetBytes(Convert.ToInt16(Environment.TickCount % Int16.MaxValue));
	byte[] timestamp = BitConverter.GetBytes(ticks);

	// copy node
	Array.Copy(Node, 0, guid, NodeByte, Node.Length);

	// copy clock sequence
	Array.Copy(clockSequenceBytes, 0, guid, GuidClockSequenceByte, clockSequenceBytes.Length);

	// copy timestamp
	Array.Copy(timestamp, 0, guid, 0, timestamp.Length);

	// set the variant
	guid[VariantByte] &= (byte)VariantByteMask;
	guid[VariantByte] |= (byte)VariantByteShift;

	// set the version
	guid[VersionByte] &= (byte)VersionByteMask;
	guid[VersionByte] |= (byte)((int)GuidVersion.TimeBased << VersionByteShift);

	return new Guid(guid);
}

You can find the actual code as part of the FluentCassandra project which contains useful type generators and serialization types for working with Cassandra.  The actual file for generating the time-based Guid is located here.  https://github.com/managedfusion/fluentcassandra/blob/master/src/GuidGenerator.cs

I hope this little tidbit helps in the adoption of Cassandra in the .NET community, because I believe NoSQL databases like Cassandra are where all the cool jobs you probably want to be working in are moving to.

NOTE: To use the above code to generate a time specific dates which can be used for pull data out of the database or putting current dates in to the database, you just need to pass in a valid .NET DateTime object like this:

// generate a date/time for now
Guid nowGuid = GenerateTimeBasedGuid(DateTime.Now);

// generate a date/time for a specific date
Guid thenGuid = GenerateTimeBasedGuid(new DateTime(1980, 3. 14));