CodeBetter.Com
CodeBetter.Com
RSS 2.0 via Feedburner
           Do you Twitter? Follow us @CodeBetter

Matthew Podwysocki

Life of a Functional Programmer

April 2008 - Posts

  • Side Effecting Functions are Code Smells

    I know the title might catch a few people off guard, but let me explain.  Side effecting functions, for the most part, are code smells.  This is a very important concept in Domain Driven Design (DDD) that's often overlooked.  For those who are deep in DDD, this should sound rather familiar.  And in the end, I think Spec# and some Design by Contract (DbC) constructs can mitigate this, or you can go the functional route as well.

    What Is A Side Effect?

    When you think of the word side effect in most languages, you tend to think of any unintended consequence.  Instead, what we mean by it is having any effect on the system from an outside force.  What do I mean by that?  Well, think of these scenarios, reading and writing to a database, reading or writing to the console, or even modifying the state of your current object.  Haskell and other functional languages take a pretty dim view of side effects, hence why they are not allowed, unless through monads.  F# also takes this stance, as "variables" are immutable unless otherwise specified.

    Why Is It A Smell?

    Well, let's look at it this way.  Most of our operations call other operations which call even more operations.  This deep nesting is then created.  From this deep nesting, it becomes quite difficult to predict the behaviors and consequences of calling all of those nested operations.  You, the developer might not have intended for all of those operations to occur because A modified B modified C modified D.  Without any safe form of abstraction, it's pretty hard to test as well.  Imagine that any mock objects that you create would have to suddenly know in 5 levels deep that it is modified in some function.  Not necessarily the best thing to do.

    Also, when it comes to multi-threaded processing, this becomes even more of an issue.  If multiple threads have a reference to the same mutable object, and one thread changes something on the reference, then all other threads were just side effected.  This may not be something that you'd want to do.  Then again, if working on shared memory applications, that might be.  But, for the most part, the unpredictability of it can be a bad thing.

    Let's take a quick example of a side effecting an object like implementation of a 2 dimensional Point.  We're going to go ahead and allow ourselves to add another Point to the system.

    public class Point2D
    {
        public double X { get; set; }

        public double Y { get; set; }

        public void Add(Size2D other)
        {
            X += other.Height;
            Y += other.Width;
        }
    }

    public class Size2D
    {
        public double Height { get; set; }

        public double Width { get; set; }
    }

    What's wrong with the above sample is that I just side effected the X, and Y.  Why is this bad?  Well, like I said, most objects like these are fire and forget.  Anyone who had a reference to this Point now has a side effected one, that they might not have wanted.  Instead, I should probably focus on retrieving a new one at this point, since this is pretty much a value object.

    What Can You Do About It?

    Operations that return results without side effects are considered to be pure functions.   These pure functions when called any number of times will return the same result given the same parameters time and time again.  Pure functions are much easier to unit test and overall a pretty low risk.

    There are several approaches to being able to fix the above samples.  First, you can keep your modifiers and queries separated.  Make sure you keep the methods that make changes to your object separate from those that return your domain data.  Perform those queries and associated calculations in methods that don't change your object state in any way.  So, think of a method that calculates price and then another method that actually sets the price on the particular object. 

    Secondly, you could also just not modify the object at all.  Instead, you could return a value object that is created as an answer to a calculation or a query.  Since value objects are immutable, you can feel free to hand them off and forget about them, unlike entities which are entirely mutable.  Let's take the above example of the Coordinate and switch it around.  Think of the DateTime structure.  When you want to add x number of minutes, do you side effect the DateTime, or do you get a new one?  The answer is, you get a new one?  Why, well, because it's a structure, and they are immutable, but not only that, it solves a lot of those side effecting problems.

    public class Point2D
    {      
        private readonly double x;
        private readonly double y;
       
        public Point2D() {}
       
        public Point2D(double x, double y)
        {
            this.x = x;
            this.y = y;
        }

        public double X { get { return x; } }
       
        public double Y { get { return y; } }
       
        public Point2D Add(Size2D other)
        {
            double newX = x + other.Height;
            double newY = y + other.Width;
           
            return new Point2D(newX, newY);
        }
    }

    Spec# is a tool that can help in this matter.  Previously I stated why Spec# matters, well, let's get into more detail why.  We can mark our side effect free methods as being pure with the [Pure] attribute.  This allows the system to verify that indeed we are not side-effecting the system, and any time I call that with the same parameters, I will get the same result.  It's an extra insurance policy that makes it well known to the caller that I'm not going to side effect myself when you call me.  So, let's go ahead and add some Spec# goodness to the equation. 

    [Pure]
    public Point2D Add(Size2D other)
    {
        double newX = x + other.Height;
        double newY = y + other.Width;
           
        return new Point2D(newX, newY);
    }

    But, now Spec# will warn us that our other might be null, and that could be bad....  So, let's fix that to add some real constraints for the preconditions.

    [Pure]
    public Point2D Add(Size2D other)
        requires other != null
    {
        double newX = x + other.Height;
        double newY = y + other.Width;
           
        return new Point2D(newX, newY);
    }

    Of course I could have put some ensures as well to ensure the result will be the addition, but you get the point.

    Turning To Design by Contract

    Now of course we have to be a pragmatist about things.  At no point did I say that we can't have side effects ever.  That would be Haskell and they put themselves into a nasty corner with that and the only way around it was with monads, that can be a bit clumsy.  Instead, I want to refocus where we do them and be more aware of what you're modifying.

    In our previous examples, we cut down on the number of places where we had our side effects.  But, this does not eliminate them, instead gather them in the appropriate places.  Now when we deal with entities, they are very much mutable, and so we need to be aware when and how side effects get introduced.  To really get to the heart of the matter, we need to verify the preconditions, the postconditions and mostly our invariants.  In a traditional application written in C#, we could throw all sorts of assertions into our code to make sure that we are in fact conforming to our contract.  Or we can write our unit tests to ensure that they conform to them.  This is an important point in Eric Evans' book when talking about assertions in the Supple Design chapter.

    Once again, Spec# enters again as a possible savior to our issue.  This allows us in our code, to model our preconditions and our postconditions as part of our method signature.  Invariants as well can be modeled as well into our code as well.  These ideas came from Eiffel but are very powerful when used for good.

    Let's make a quick example to show how invariants and preconditions and postconditions work.  Let's create an inventory class, and keep in mind it's just a sample and not anything I'd ever use, but it proves a point.  So let's lay out the inventory class and we'll set some constraints.  First, we'll have the number of items remaining.  That number of course can never go below zero.  Therefore, we need an invariant that enforces that.  Also, when we remove items from the inventory, we need to make sure that we're not going to dip below zero.  Very important things to keep in mind.

    public class Inventory
    {
        private int itemsRemaining;
        private int reorderPoint;
       
        invariant itemsRemaining >= 0;
       
        public Inventory()
        {
            itemsRemaining = 200;
            reorderPoint = 50;
            base();
        }
       
        public void RemoveItems(int items)
            requires items <= ItemsRemaining;
            ensures ItemsRemaining == old(ItemsRemaining) - items;
        {
            expose(this)
            {
                itemsRemaining -= items;
            }

            // Check reorder point
        }
       
        public int ItemsRemaining { get { return itemsRemaining; } }

        // More stuff here in class
    }

    What I was able to express is that I set up my invariants in the constructor.  You cannot continue in a Spec# program unless you set the member variable that's included in the invariant.  Also, look at the RemoveItems method.  We set one precondition that states that number of items requested must be less than or equal to the number left.  And we set the postcondition which states that the items remaining must be the difference between the old items remaining and the items requested.  Pretty simple, yet powerful.  We had to expose our invariant while modifying it so that it could be verified, however.  But, doesn't it feel good to get rid of unit tests that prove what I already did in my method signature?

    Wrapping Things Up

    So, I hope after reading this, you've thought more about your design, and where you are modifying state and that you have intention revealing interfaces to tell the coder what exactly you are going to do.  The Design by Contract features of Spec# also play a role in this to state in no uncertain terms what exactly the method can do with the preconditions and postconditions and through my class with my invariants.  Of course you can use your regular C#, or language of choice to model the same kind of things, yet not as intention revealing.

    So, where to go from here?  Well, if you've found Spec# interesting, let Microsoft know about it.  Join the campaign that Greg and I are harping on and say, "I Want Spec#!"
  • Upcoming Functional Programming/F# Talks

    Well, I certainly have an ambitious May schedule ahead of me.  Most of course will be revolving around functional programming and F# as it seems to be finally catching on.  I've been noticing a bunch from the Java and Ruby communities becoming interested in such things as Scala, Haskell, OCaml, Erlang and F#.  I was rather heartened by this as some in the Ruby world like here and here coming back to the static world for ways of representing data and functions in different ways.  Of course Lisp and Scheme (IronLisp and IronScheme) still manages to eek in the rebirth, but still remains on the outside.

    I will be speaking at the Northern Virginia Code Camp on May 17th for a total of two topics:

    • Improve your C# with Functional Programming and F# concepts
      Learn how .NET 3.5 takes ideas from Functional Programming and how you can apply lessons learned from it and F# respectively.

    • Introduction to Functional Programming and F#
      Come learn about functional programming, an older paradigm that Object Oriented Programming, and the ideas around it.  This talk will cover the basics including high-order functions, functions as values, immutability, currying, pattern matching and more.  Learn how to mesh ideas from functional programming with imperative programming in F# and .NET.

    So, if you're in the DC area, go ahead and register here and show your support for the community.

    Also, I will be taking some time to spend up in Philadelphia this month at the next Philly ALT.NET meeting to also talk about F#.  Still ironing out the details on that one in regards to the DC ALT.NET meeting in May.  Either way, should be a good time!
  • Making Spec# a Priority

    During ALT.NET Open Spaces, Seattle, I spent a bit of time with Rustan Leino and Mike Barnett from the Spec# team at Microsoft Research.  This was to help introduce Design by Contract (DbC) and Spec# to the ALT.NET audience who may not have seen it before through me or Greg Young.  I covered it in detail on my old blog here.

    Spec# at ALT.NET Open Spaces, Seattle

    As I said before I took a bit of time during Saturday to spend some time with the Spec# guys.  I spent much of the morning with them in the IronRuby session explaining dynamic languages versus static ones.  They had the session at 11:30, the second session of the day, in direct competition with the Functional Programming talk I had planned with Dustin Campbell.  Greg was nice enough to record much of the session on a handheld camera and you can find that here.  It's not the best quality, but you can understand most of it, so I'm pretty happy. 

    The things that were covered in this session were:
    • Spec# overview
    • Non-null Types
    • Preconditions
    • Postconditions
    • Invariants
    • Compile-Time checking versus Runtime checking
    • Visual Studio Integration
    All in all, I thought it was one of the best sessions and I'm glad they came out.  Hopefully we'll see more from them in the future.

    Scott Hanselman also recorded a session with the Spec# guys for Episode 128.  This is a much better interview than on DotNetRocks Episode 237 that Rustan did last year. This actually gets into the technical guts of the matter in a much better way, so go ahead and give it a listen.  I was fortunate enough to be in the room at the time to listen.

    The New Release

    Mike and Rustan recently released a new version of Spec# back on April 11th so now Visual Studio 2008 is supported.  You must remember though, this is still using the Spec# compiler that's only C# 2.0 compliant.  So, anything using lambdas, extension methods, LINQ or anything like that is not supported.  As always, you can find the installs here

    As with before, both the Spec# mode (stripped down mode) and C# mode are supported.  What's really interesting is the inferred contracts.  From an algorithm that Mike and Rustan worked on, they have the ability to scan a method to determine its preconditions and postconditions.  It's not perfect, but to have that kind of Intellisense is really powerful.



    What you can see is that the GetEnumerator method ensures that the result is new.  Keep in mind, result is a keyword which states what the return value is for a method.  It also says that the owner of IEnumerator will be the same as before.  Object ownership is one of the more difficult things to comprehend with Spec# but equally powerful.

    Another concept that's pretty interesting is the ability to make all reference types non-null by default in C# or in Spec# modes.  Instead of having to mark your non-null types with an exclamation mark (!), instead you can mark your nullable types with a question mark (?) much as you would with the System.Nullable<T> generic class.  All it takes is the flip of a switch in Spec#:



    Or in the C# mode:



    And then you have all the Spec# goodness.

    Why It's Important

    So, why have I been harping on this?  To be able to express DbC as part of my method signature is extremely important to me.  To be able to express my preconditions (what I require), postconditions (what I ensure), my invariants (what state will change) is a pretty powerful concept.  Not to mention, to enforce immutability and method purity is also a pretty strong concept, especially in the times of multi-core processing.  More on that subject later.

    Focus on Behaviors

    What Spec# can bring to the table is the ability to knock out a bit of your unit tests.  Now, I don't mean all of them, but what about the ones that check for null values?  Are they valid if you already put in your method signature to require a non-null value or use the ! symbol to denote a non-null type?  Those edge cases aren't really valid anymore.  The ability to track your invariants is the same as well as your postconditions.  Instead, what that does is frees you up to consider the behaviors of your code, what you should have been testing anyways.

    Immutability

    Immutability plays a big part in Spec# as well.  To some extent, I'll cover more in a Domain Driven Design post, but instead will get some things out of the way here.  Eric Lippert, C# team member, has stated that immutable data structures are the way of the future in C# going forward.  Spec# can make that move a bit more painless?  How you might ask?  Well, the ImmutableAttribute lays out that explicitly.  Let's do a simple ReadOnlyDictionary in Spec#, taking full advantage of Spec#'s attributes, preconditions and postconditions:

    using System;
    using System.Collections;
    using System.Collections.Generic;
    using Microsoft.Contracts;

    namespace Microsoft.Samples
    {
        [Immutable]
        public class ReadOnlyDictionary<TKey, TValue> : ICollection<KeyValuePair<TKey!, TValue>> where TKey : class
        {
            private readonly IDictionary<TKey!, TValue>! dictionary;
       
            public ReadOnlyDictionary(IDictionary<TKey!, TValue>! dictionary)
            {
                this.dictionary = dictionary;
            }
           
            public TValue this[TKey! key]
            {
                get
                    requires ContainsKey(key);
                { return dictionary[key]; }
            }
           
            [Pure]
            public bool ContainsKey(TKey! key)
            {
                return dictionary.ContainsKey(key);
            }

            void ICollection<KeyValuePair<TKey!,TValue>>.Add(KeyValuePair<TKey!, TValue> item)
            {
                throw new NotImplementedException();
            }

            void ICollection<KeyValuePair<TKey!,TValue>>.Clear()
            {
                throw new NotImplementedException();
            }

            [Pure]
            public bool Contains(KeyValuePair<TKey!, TValue> item)
            {
                return dictionary.Contains(item);
            }

            [Pure]
            public void CopyTo(KeyValuePair<TKey!, TValue>[]! array, int arrayIndex)
                requires arrayIndex >=0 && arrayIndex < Count;
            {
                dictionary.CopyTo(array, arrayIndex);
            }

            [Pure]
            public int Count
            {
                get
                    ensures result >= 0;
                    { return dictionary.Count; }
            }

            [Pure]
            public bool IsReadOnly
            {
                get
                    ensures result == true;
                    { return true; }
            }

            [Pure]
            bool ICollection<KeyValuePair<TKey!,TValue>>.Remove(KeyValuePair<TKey!, TValue> item)
            {
                throw new NotImplementedException();
            }

            [Pure]
            public IEnumerator<KeyValuePair<TKey!, TValue>>! GetEnumerator()
            {
                return dictionary.GetEnumerator();
            }

            [Pure]
            IEnumerator! System.Collections.IEnumerable.GetEnumerator()
            {
                return dictionary.GetEnumerator();
            }
        }
    }

    As you can see, I marked the class itself as immutable.  But as well, I removed anything that might change the state of our dictionary, as well as mark things with non-null values.  That's a little extra on top, but still very readable.  I'll be covering more in the near future as it applies to Domain Driven Design.

    Call to Action

    So, the call to action is clear, make Spec# a priority to get it in C# going forward.  Greg Young has started the campaign, so we need to get it moving!
  • xUnit.net Goes 1.0 and Unit Testing F#

    As I've said before on my previous blogs, I'm very much into F# and functional programming lately.  With that, I'm still in the mode of TDD.  Just because you enter a new programming paradigm, doesn't mean you throw away your XP and TDD roots.  Instead, I find it just as valuable if not even more so when switching to a new model.

    xUnit.net 1.0 Goes Live

    As Brad Wilson said earlier this week, xUnit.net released version 1.0.  You can read more about the announcement here.  Since the RC3 timeframe, it was pretty much feature complete, so there hasn't been much change since that time.  Instead, the focus was on polishing the existing functionality and including the integration with ASP.NET MVC, Resharper 3.1 and TestDriven.NET.  The GUI runner, such as it is has been pretty limited, but ok for most purposes. 

    Many questions used to arise, why xUnit.net?  Why do we need yet another framework out there?  Why not just add onto the existing ones in the market?  I think with the 1.0 release, the critics in this release to do things a bit differently than MbUnit and NUnit have approached things.  For example, the Assert.Throws<TException>, not having to decorate the classes with [TestFixture] and a few other things come to mind as well as being very extensible.  It's a smaller framework, but I really don't have a problem with that.  With most of the other frameworks, I don't use half of it anyways.

    So, why do I care as much as I do about this one over say others at the moment?  Well, it's great that we have such choices in the market now.  As Scott Hanselman said at ALT.NET Open Spaces, Seattle, he's a StructureMap, Moq and xUnit.net guy.   I'll get into the reason shortly enough.

    The Traditional Way

    When you think of doing your unit or behavior tests, you often need a class and decorate with attributes, have a initialize and teardown and all that goodness.  Since F# is considered a multi-purpose language, this works just fine.  That's the beauty of F# and hopefully will drive its adoption into the marketplace.  So, consider the following functions with the appropriate tests in a more traditional way such as NUnit within Gallio.  I am trying out Gallio so that I can see how well it reacts to F# as well as just kicking the tires.  I highly recommend you at least check it out.

    Anyhow, back to the code.  Let's take a simple example of a naive routing table through pattern matching, to see whether a call is allowed or not.  Like I said, naive, but proves a point with pattern matching.

    #light

    #R @"D:\Program Files\Gallio\bin\NUnit\nunit.core.dll"
    #R @"D:\Program Files\Gallio\bin\NUnit\nunit.framework.dll"

    let FilterCall protocol port =
      match(protocol, port) with
      | "tcp", _ when port = 21 || port = 23 || port = 25 -> true
      | "http", _ when port = 80 || port = 8080 -> true
      | "https", 443 -> true
      | _ -> false
     
    open NUnit.Framework

    [<TestFixture>]
    type PatternMatchingFixture = class
      [<Test>]
      member x.FilterCall_HttpWithPort8888_ShouldReturnFalse() =
        Assert.IsFalse(FilterCall "http" 8888)
       
      [<Test>]
      member x.FilterCall_TcpWithPort23_ShouldReturnTruee() =
        Assert.IsTrue(FilterCall "tcp" 23) 
    end

    Unfortunately of course for me, the Gallio Icarus Runner doesn't really work well for me at the moment with F# integration.  Instead, I get all sorts of issues when doing so.  This is where I get the large FAIL.



    This seems to repeat itself unfortunately for the xUnit.net and MbUnit integration as well, so it's not quite ready for primetime in the F# space.  Also, when I exit the application, there is a Gallio session runner that keeps running in memory and therefore I can't perform any builds.  So, I have to manually go into Task Manager and kill the process.  Not the best experience I've ever had...  So, for now, the limited functionality in the xUnit.net GUI Runner works for me.

    The More Functional Way

    Instead, we see a lot of pomp and circumstance that we just don't need.  In the functional world, a lot of the time, we don't want or need to create these classes just to test our functions.  After all, most of what we do has no side effects or at least should be (and are code smells mostly if they are not, but that's another post for another time).

    At the request of Harry Pierson, another F# aficionado and IronPython PM, talked to Brad and Jim Newkirk about adding static function unit test capabilities to xUnit.net.  And sure enough, we now have them, so let's compress the above code into something that looks more like F#.

    #light

    #R @"D:\Tools\xunit-1.0\xunit.dll"

    open Xunit

    let FilterCall protocol port =
      match(protocol, port) with
      | "tcp", _ when port = 21 || port = 23 || port = 25 -> true
      | "http", _ when port = 80 || port = 8080 -> true
      | "https", 443 -> true
      | _ -> false
     
    [<Fact>]
    let FilterCall_TcpWithPort23_ShouldReturnTrue () =
      Assert.True(FilterCall "tcp" 23)

    [<Fact>]
    let FilterCall_HttpWithPort8888_ShouldReturnFalse () =
      Assert.False(FilterCall "http" 8888)

    So, as you can see, I compressed the code quite a bit from here.  Since I'm doing functions and nothing more with just some basic pattern matching, this approach works perfectly.  That's why I am a fan of this.  Open up the GUI runner or just the ever popular console runner, and run it through and sure enough, we get some positive results.



    The interesting thing to see upcoming is how well the TDD space will play in the functional programming space.  I don't think it should be any different of an experience, but time will tell.

    Where to Go?

    From here, where do we go?  Well, I'm sure the GUI Runner of xUnit.net will get better over time, but the Gallio folks are pushing for the Icarus Runner.  Right now, only the xUnit.net runner works for me, so that's what I'm going to stick with at the moment. 

    An interesting thought occurred to me though.  Are the unit tests we're doing mostly nowadays purely functional anyways?  Would it make sense to test some C# code in F# to produce cleaner code?  Not sure, but I like the idea of having that choice.  Or even for that matter, writing my unit tests in Ruby for my staticly typed C# code.  Within the .NET framework space, the possibilities are vast.  And that's the really cool thing about it.  But will we see an IronPython or IronRuby testing framework within the .NET space?
  • let Matt = CodeBetter + 1

    Hello CodeBetter community! 

    #light

    type FullName = string * string

    let FullNameToString (name : FullName) =
      let first, last = name in
      first + " " + last
     
    let blogger = FullNameToString("Matthew", "Podwysocki")

    I'm pretty excited to be joining the CodeBetter gang after following for so many years.  I want to thank Jeremy Miller, Brendan Tompkins, Dave Laribee, Greg Young and others for welcoming me to the fold. 

    So Who Are You and Why Are You Here?

    So, just to introduce myself, I work for Microsoft in the Washington DC area.  I'm active in the developer community in whether it be in the .NET space, Ruby, or less mainstream languages (F#, Haskell, OCaml, Lisp, etc).  I also run the DC ALT.NET group since the November timeframe of last year and helped plan the latest incarnation in Seattle. 

    The number one reason I'm here is to help better myself.  Deep down, I'm a language geek with any of the aforementioned languages.  I'm one of those who strives to learn a language every year, but not just learn it, let it sink in.  That's really the key.  Sure, I can learn a certain dialect, but it's not quite being a native speaker.  That's how I can take those practices back to my other languages to try to apply lessons learned such as functional programming paradigms (pattern matching, currying, first order functions, etc).

    I also have a pretty deep interest in TDD/BDD, Domain Driven Design and of course one of Greg Young's topics, messaging.  Right now in the world of messaging, I think we're in a pretty important time in the development world when messaging, multi-threaded processing and such is going to be more mainstream and hopefully less hard than it is now.

    I'm also a tinkerer at heart.  I'm looking at testing frameworks to help make my TDD experiences easier.  I'm looking at IoC containers to help make my system just a bit more configurable.  I'll look at the tests to see how each one does what it is.  That's the fun part about it.

    I'm also on the fringe with such topics as Spec# and Design by Contract.  I'd love nothing more than to see many of the things being done at Microsoft Research become a bit more mainstream and not just seen as a place where we might see something 10 years down the line.  Topics such as Spec#, F# and others have real importance now and it's best to play with them, give them our feedback and such.

    So What Do You Want From Me?

    Here's the thing, since I'm always looking to better myself, I'll need your help along the way.  I value your feedback along the way and hopefully we'll learn from each other.  Now that this is out of the way, time for more serious topics...

More Posts