19 Jan 2012

RavenDB Sequential Number Generator

1 Comment Uncategorized

RavenDB has a great identity generator that is very fast to generate a new identity when you are inserting a ton of records at once. You can read more about it here: http://ravendb.net/documentation/docs-api-key-generation and http://codeofrob.com/archive/2010/05/16/ravendb-hilo-what-how-and-why.aspx

However if you start and stop the RavenDB service, by restarting your computer, or recycling your app pool, the numbering of the identity generator skips, which for most people it really doesn’t matter. However sometimes sequential numbers really matter, for example in an invoice number you don’t want to skip a number because it can really screw up your accounting office?

So faced with this problem I created a method that made sure the sequence of numbers being returned was always in order, regardless of restarts to the RavenDB service. Before I get into the code lets first discuss the downsides to this code I am going to present.

  • First and fore most this is going to be a lot slower than the Hilo system, because with each increment it syncs the number back to the database
  • This code is only designed for one application to generate this number from RavenDB, this is because the lock that makes the sequential number possible is in the application, not RavenDB
  • Last thing, and I am saying this because it needs to be repeated, this code is not designed if you need to insert a 100,000 records at once. If you need to do that you might want to come up with a better solution
private static readonly object GeneratorLock = new object();

private int NextAccountNumber()
{
	lock (GeneratorLock)
	{
		using (new TransactionScope(TransactionScopeOption.Suppress))
		{
			while (true)
			{
				try
				{
					var document = GetDocument();
					if (document == null)
					{
						PutDocument(new JsonDocument {
							Etag = Guid.Empty, // sending empty guid means - ensure the that the document does NOT exists
							Metadata = new RavenJObject(),
							DataAsJson = RavenJObject.FromObject(new { Current = 1}),
							Key = "Raven/InvoiceNumber"
						});
						return 1;
					}

					int current;
					current = document.DataAsJson.Value("Current");
					current++;

					document.DataAsJson["Current"] = current;
					PutDocument(document);

					return current;
				}
				catch (ConcurrencyException)
				{
					// expected, we need to retry
				}
			}
		}
	}
}

private void PutDocument(JsonDocument document)
{
	DocumentStore.DatabaseCommands.Put(
		"Raven/InvoiceNumber", 
		document.Etag,
		document.DataAsJson,
		document.Metadata);
}

private JsonDocument GetDocument()
{
	return DocumentStore.DatabaseCommands.Get("Raven/InvoiceNumber");
}

If you see any changes that should be made, I have created a Gist with this code. https://gist.github.com/1641384 Branch it and make your changes and share below in the comments.

13 Dec 2011

Cleaning Up Your Git Repository For NuGet 1.6

10 Comments Uncategorized

NuGet 1.6 was released today. And with it came some great new features, one that I am particularly excited about is.

Using NuGet Without Checking In Packages (Package Restore)

NuGet 1.6 now has first class support for the workflow in which NuGet packages are not added to source control, but instead are restored at build time if missing. For more details, read the Using NuGet without committing packages to source control topic.

The reason for this is that it finally allows us to remove the packages directory and all the packages from our repository.  Which I always thought created a mess of source control systems, and required unnecessary check-ins. It also created a huge amount of bloat especially since some projects (cough… NUnit) included so much extra crap that nobody needed and required all that extra storage space in the repository and commit history.

Setup

Let’s assume that you have a solution that is either already using NuGet, or planning to use it, and that you want to set up the no-commit workflow.

Right click on the Solution node in Solution Explorer and select Enable NuGet Package Restore.

Enable NuGet Package Restore Context Menu item

That’s it! You’re all set.

(taken from http://docs.nuget.org/docs/Workflows/Using-NuGet-without-committing-packages)

.gitignore

Next we will want to add the following to your .gitignore file. (create one if you don’t already have one) And add the following to the top of it.

# Don't include NuGet Packages
packages/

This will make sure any changes in the packages directory will be ignored when adding to the repository.

Make sure to commit these changes before the next step.

$ git add .
$ git commit -am "updating project for NuGet 1.6"
$ git push origin master

Cleaning up your repository

This next step is optional, but the purpose of it is to reduce the size of your repository. And for the more OCD among us, remove all reference to the packages directory so it is like it never existed.

$ git filter-branch --index-filter 'git rm --cached --ignore-unmatch -r ./packages/*'
Rewrite 48dc599c80e20527ed902928085e7861e6b3cbe6 (266/266)
Ref 'refs/heads/master' was rewritten

$ git push origin master --force
Counting objects: 1074, done.
Delta compression using 2 threads.
Compressing objects: 100% (677/677), done.
Writing objects: 100% (1058/1058), 148.85 KiB, done.
Total 1058 (delta 590), reused 602 (delta 378)
To git @ github.com:nberardi/some-project.git
 + 48dc599...051452f master -> master (forced update)

Cleanup and reclaiming space

While git filter-branch rewrites the history for you, the objects will remain in your local repo until they’ve been dereferenced and garbage collected. If you are working in your main repo you might want to force these objects to be purged.

$ rm -rf .git/refs/original/

$ git reflog expire --expire=now --all

$ git gc --prune=now
Counting objects: 2437, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (1378/1378), done.
Writing objects: 100% (2437/2437), done.
Total 2437 (delta 1461), reused 1802 (delta 1048)

$ git gc --aggressive --prune=now
Counting objects: 2437, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (2426/2426), done.
Writing objects: 100% (2437/2437), done.
Total 2437 (delta 1483), reused 0 (delta 0)

Note that pushing the branch to a new or empty GitHub repo and then making a fresh clone from GitHub will have the same effect.

(taken from http://help.github.com/remove-sensitive-data/)

Happy Coding

05 Oct 2011

Stay Hungry, Stay Foolish

No Comments Uncategorized

Steve Jobs

February 24, 1955 – October 5, 2011

My first computer was the Apple II, my second computer was a Macintosh Classic, my most useful computer to date has been my iPhone. The iPhone has been the perfect convergence of technology and life for me, just the way that I imagine Steve Jobs envisioned it.

His vision of what the world could and can be will be sorely missed in the generations to follow. But his legacy will live on, so rest in peace.

Posted from my iPad