October 2006 - Posts - Karl Seguin

You aren't gonna need it .... But maybe you will...

There are three overlapping high level design goals I always try to follow:

Be explicit,
Keep it simple and
You aren't gonna need it

Building something simple that's easy to understand and change is the best way to build extensibility and flexibility. Actually, it isn't only the best way to meet future demands, it's also the cheapest - you can have your cake and eat it too!

Lately, I've been running up against YAGNI more and more...

For example, we are at the very start of a new project. There are specific requirements for performance and availability. I'm interested in leveraging .NET's in-memory caching capabilities, but need something that'll scale. Specifically, the application servers will sit behind load-balanced web services and in front of a clustered database.

Over the weekend I researched distributed caching solutions. The only thing I found was nCache, but the cost wasn't in our initial budget (the software + the extra caching servers), so I'm not sure if it'll go or not. Worse, I probably can't make call (or push for it) until I'm further along - maybe we'll meet our performance requirements without it. I need to build a solution that can adapt to different caching solutions (from none to a full blown layer), but don't want to make my code more complicated.

In this particular case I'm lucky. The solution is lightweight. All I really need is a single layer of abstraction. I'll write an ICache interface and do an implementation that does nothing except implement my members. Later on I can write an InMemory implementation that uses .NET's cache...and even later on I can write one for nCache. It'll be testable, simple and flexible. But in the end, I really might not need it.

Posted Oct 30 2006, 07:41 AM by karl with 6 comment(s)

Sidetracked with MySQL & SQL Server Performance

Here's the short version:

This is only for inserting 500 rows into a 3 column table
InnoDB really seems screwed on Windows platforms running 5.x - Certainly don't have anything conclusive, but I'd watch out
For my very simple test, MySQL on Linux blew SQL Server 2005 on windows out of the water.
On windows, ASP.NET's connection to MySQL seems as fast as PHP's

Here's the long version
For a large upcoming project, we've pretty much decided on a .NET application and MySQL database. We'll be using a database cluster for our primary work and smaller database servers to handle secondary tasks (i.e., set up logging as a simpler replication solution).

Before jumping head-first, I wanted to play with the 5.0 beta connector/NET drivers from MySQL. Previously I've only used the ODBC drivers to connect to MySQL and wasn't impressed. I pretty much wanted to make sure it worked like I thought it should (which it does) and that there weren't gonna be any major performance pitfalls. Everything was running locally on my machine - I know, not ideal, but I figured it'd be good enough to catch anything specific I might need to look at more closely (and I was right).

I created a simple table with 3 columns : autoincrement int as the PK, varchar column, an datetime column. My first test was simple - loop 500 times and insert a record ala:

VALUES ("Random Name 34", Now())

I didn't see much of a difference between opening/closing the connection for each insert (thank you connection pooling), keeping the connection open outside of the loop or using a stored procedure. In all cases, the code ran....REALLY REALLY slowly - 13 seconds to be exact!.

I knew there was no way for 500 inserts to take 13 seconds, so I wrote the same code for SqlClient, set up SQL Server 2000 and the test ran in 0.3 seconds (again, all locally).

At this point, my Linux-friendly boss got very interested. I was really worried that the connector might be a POS, so I wrote the same code in PHP and got the same result - good news for .NET, but still baffled.

I finally noticed the table type was InnoDB. I switched it to MyISAM and all the problems went away. All in all, it's safe to say that locally on my windows machine, SQL Server ran a bit faster than MySQL (say 0.28 seconds vs 0.3)

We decided to test this in a linux environment. My boss wrote the same code for our powerful linux development server and got things rolling using PHP. MyISAM was running at a blazing 0.05, InnoDB at 0.2. Instead of being about 35x slower, the InnoDB storage type was only 4 times slower.

We were really interested to see how the .NET / SQL Server solution on the same hardware would compare. Luckily, that was easily possible on our Windows development server. Would we see the same 6x improvement in performance from the better hardware? Sadly not. The .NET / SQL Server code ran at 0.25 seconds...a very marginal increase.

What made MySQL run so fast? Was it the hardware? Why couldn't SQL Server (2005 Express on the dev machine) take advantage of the hardware as well?Was it because it was running on Linux (that certainly seems to be the case with InnoDB).

Given the same hardware, it would be nice to get SQL Server running in the 0.0X range like MySQL. Even if this was an accurate test (you know, for all I know it's 1 checkbox somewhere in SQL Server and zooom), each database will handle different tasks differently. Tomorrow we'll try connecting .NET with MySQL on the linux box and hopefully get the same results. So far so good for the Connector/NET :)

Posted Oct 24 2006, 06:50 PM by karl with 6 comment(s)

Filed under: Grab a coffee before reading

Wait a couple more days before getting a new computer

I overheard our IT Manager talking about ordering 4 laptops, and told him he should wait until friday - that's the day Microsoft is supposed to be offering upgrade coupons for Vista.

You can read more at:
http://www.crn.com/showArticle.jhtml?articleID=193104349

I'd hate to buy one on thursday and find out about this later..so I figured I'd pass it along :)

Posted Oct 24 2006, 06:28 AM by karl with no comments

Posted a job on jobs.codebetter.com (web dev ottawa, ont)

In case you missed the announcement, Eric and Brendan released jobs.codebetter.com not too long ago. I've decided to take advantage of their promotion of having free postings and put up a job for a full time (with benefits) position at Fuel Industries (where I work) in Ottawa Ontario (canada)

It's for a junior/intermediate web developer who's familiar with PHP / .NET / JSP (don't need all three). Check out the actual posting for more details.

Hats off to Eric and Brendan for the fine work.

Posted Oct 17 2006, 01:40 PM by karl with 3 comment(s)

does www.asp.net really need ads?

Call this mindless rambling on a sunday, but does www.asp.net really need to have banner ads all over the place? Aside from the homepage, almost every page has 1 or 2 banner ads. There in the forums, all over the tutorials and cleverly placed on the blogs. I find it funny that Microsoft is in need of generating a bit of extra revenue, while:

php.net and any of it's mirrors, has no ads.

Sun's JSP homepage, has no ads Ruby

Ruby on Rails homepage, has no ads Perl

Perl was the only competitor's site where I found a few ads.

The ads make the site less professional looking and make it feel a little dishonest - like a used car salesman.

Posted Oct 15 2006, 09:07 AM by karl with 7 comment(s)

don't use string.format ?!

In response to my post evangilizing string.format, Travis commented that my advice was wrong, and that it was "pathetic" to give priority to mere code cosmetics rather than performance. It's obvious that Travis feels strongly about the topic - as do I.

In some cases I think Travis is right. For example, if you're building a missile guidance system, or a video game, or maybe software that drives a heart pump. But for _any_ piece of software where you adopt a general-purpose managed language (like C#, VB.NET or Java), well, I think Travis is dead wrong.

I'll be the first to admit that I've never actually done performance testing on String.Format - but I have looked inside the class at how it works. True enough, there's a lot going on in there.

My point though is that in this day and age, the primary cost of developing software is maintenance. I've seen some studies pinning maintenance at over 100% of the total cost of ownership. In other words, the money you'll save by making code readable and maintainable is going to eclipse your hardware cost. But even that isn't really my main issue with what Travis said, because, in my humble opinion, Travis is guilty of massive premature micro-optimization. I'm pretty sure that even on outdated hardware (say a 5 year old computer), the difference in the majority of cases is going to be completely unnoticeable.

Of course, in cases where it is noticeable, you create nice code first, profile, and optimize. Travis seems to be suggesting you skip the first two step. I'm sure someone has a clever saying about this. (Both Red-Gate and JetBrains offer good .NET profiler, I highly suggest one or the other).

It’s one of those 80/20 things. Optimizing the first 80% tends to be relatively easy, but it quickly gets a lot harder to find anything else. Once you've identified and implemented the two or three big hits, things become a lot harder to tweak. If you're having performance issues, look at table indexes and caching opportunities. I've seen single indexes improve performance by orders of magnitude.

The fairly linear evolution of computer languages is directly related to hardware advances. C was possible as an abstraction above assembly because of it. Same goes for OO, garbage collectors, exceptions, webcontrols and advanced formatting.

Posted Oct 04 2006, 06:42 PM by karl with 14 comment(s)

Give your website more context

A pattern I've used over the years, when applicable, is to create an XXXContext class within my web project. The scope of the class is for a request; the purpose is to make available data and methods relevant to most requests the site will deal with.

Let's pretend we're building some type of reporting website. Almost every page and control is going to need to know three very important things: StartDate, EndDate and ReportType. For simplicity, let's pretend those parameters are passed as querystring values, i.e.: index.aspx?s=20061001&e=20061005&t=pie. A bad solution would be to have every page retrieve the information and validate it - since we are dealing with a querystring, we'll also need to convert the data. Rather, we'll create a ReportContext class and let it take care of it. It'll look something like:

internal class ReportContext
{

   #region Fields and Properties

   private DateTime _startDate;

   private DateTime _endDate;

   public DateTime StartDate

      get { return _startDate; }

   public DateTime EndDate

      get { return _endDate; }

   #endregion

   #region Constructors

   public ReportContext(HttpContext context)

      if (context == null)

         throw new ArgumentNullException("context");

      bool hasStartDate = GetStartTime(request.QueryString["s"], out _startDate);

      _endDate = GetEndTime(request.QueryString["e"], hasStartDate);

   #endregion

   #region Singleton

   //this is a pseudo singleton, but since it's per request, there should never be a problem

   public static ReportContext Current

get

         HttpContext context = HttpContext.Current;

         if (context == null)

            throw new InvalidOperationException("ReportContext can only be used from within a http context");

         ReportContext reportContext = (ReportContext)HttpContext.Current.Items["ReportContext"];

         if (reportContext == null)

            reportContext = new ReportContext(context);

            HttpContext.Current.Items["ReportContext"] = reportContext;

         return reportContext;

   #endregion

   #region Private Methods

   private bool GetStartTime(string startDateString, out DateTime startDate)

      if (string.IsNullOrEmpty(startDateString) || !DateTime.TryParseExact(startDateString, "yyyyMMdd", CultureInfo.InvariantCulture, DateTimeStyles.None, out startDate))

         startDate = DateTime.Now.AddDays(-5);

         return false;

      return true;

   private DateTime? GetEndTime(string endDateString, bool hasStartDate)

      DateTime endDate;

      if (string.IsNullOrEmpty(endDateString) || !DateTime.TryParseExact(endDateString, "yyyyMMdd", CultureInfo.InvariantCulture, DateTimeStyles.None, out endDate))

         if (hasStartDate)

            return null;

         return DateTime.Now;

      return endDate;

   #endregion

It can then be consumed from anywhere in our web project via:
ReportContext.Current.StartDate

This greatly helps reduce code duplication. It's also possibly to cleanly encapsulate input validation and custom rules. If you followed the code above, you'll notice that if the "s" and "e" parameters are invalid, then it'll automatically set a StartDate of 5 days ago and an EndDate of today. But, if there's a valid "s" without a valid "e", well, we'll assume we want a report for a single day and leave EndDate null. Granted, that might business logic which should be located in your business layer, but in this case it's a presentation specific rule ('cuz we said so). Also worth noting is the use of the HttpContext.Items collection, which provides us exactly with the scope we want - 1 request. We're using a lazy-load, but you could just as easily place the code in BeginRequest.

It'd be possible to achieve the same thing using a base class for your pages. Either way works, but I do prefer this method.