in Search

Do you Twitter? Follow us @CodeBetter

Raymond Lewallen

Framework Design, Agile Coach, President Oklahoma City Developers Group, Microsoft MVP C#, TDD, Continuous Integration, Patterns and Practices, Domain Driven Design, Speaker, VB.Net, C# and Sql Server

Oh how I miss CruiseControl and NAnt

Here is a screenshot of my TFS build. 20 minutes. The integration builds have been running for a week now. They are triggered on check-in using a tool called Automoton because TFS does not have this functionallity by default. Most builds are taking over 10 minutes. CC + NAnt, this build would take about 20-30 seconds. The codebase is very small and there are only 265 unit tests so far.

Published Apr 13 2007, 11:14 AM by Raymond Lewallen

Filed under: Continuous Integration, Patterns and Practices, .Net Development

Comments

Derik Whittaker said:

Raymond,

I feel your pain. Is is just me, or is it that the entire testing framework for TFS is very slow. just run one test will take 15-30 seconds just to load the framework. Where in NUnit it would take only a few.

Also, what a joke that VSTS was released without a CI component. It is laughable that a new product that is meant to replace tools like CruiseControl and NUnit dont even have the bare minimim feature set to do so.

Derik

# April 13, 2007 12:56 PM

Raymond Lewallen said:

The framework is ridiculously slow for testing. I have a lot of complaints with MSTest aside from being slow, but its TFS that is is the real culprit. Sure, there is a lot that its doing, but OMG opening a project and getting the latest source takes me almost 2 minutes! (Some of that has to do with ReSharper load time). I'm getting a lot of experience with TFS over the last few weeks, and as I get more, I'm going to start posting on what I have found good and bad, from the perspective of someone who has years of experience with tools like CC, NUnit, NAnt, TD.Net, Vault, Subversion, etc.

# April 13, 2007 1:10 PM

Jeremy D. Miller said:

Raymond,

Couldn't you just use CC.Net against TFS & MSBuild? Or is the culprit just that TFS/MSTest is that much slower than SVN/NUnit?

# April 13, 2007 1:21 PM

Derik Whittaker said:

Pretty sure it is just that TFS is real slow. It is to the point where i now have a chance to read/surf alot during the day now because I run my tests 50+ times a day and give me about an extra 30 minutes of down time.

Derik

# April 13, 2007 1:26 PM

Raymond Lewallen said:

Jeremy,

TFS is the culprit. Its not so much MSTest (although it is slow). Because of what all is entailed with TFS; the tracking, bugs, impediments, work tasks, check-ins being associated, the SCRUM regressions and velocity calculations, etc etc there is too much going on to make this a feasible continuous integration solution I fear. Its way too heavy and has too much stuff (valuable stuff, yes, but sloooowwwww) going on. The configurations are a klug. MSTest will fail a test 1 out of 10 times its run for no understandable reason causing the build to fail (getting tests to run on a build required an add-in install from MSFT, btw).

I have lots to say on this subject, and I'll be posting about it as my work with getting integration working in TFS begins to wind down soon.

I just wanted to start by submitting a quick complaint about how 10000 lines of code (including Asp.Net) takes 20 minutes to get source, build, test and report.

# April 13, 2007 1:31 PM

David Starr said:

I gotta say, folks, this smacks of a configuration issue. I am simply not experiencing the pain that I hear youdescribing. We were having these problems initially because during a TFS build, the build was pulling all of the sourc code from the Team Project. We fixed this by defining a workspace at a lower level for a particular project and building against just that workspace.

# April 13, 2007 2:14 PM

Raymond Lewallen said:

David,

I have actually manually edited the WorkspaceMapping.xml and TFSBuild.proj file to reflect what you are talking about back when the build was first setup last week. The builds still take 10-20 minutes. If it was continuously a 16 minute build, that would be one thing, but it can be 10 minutes for one build and 20 minutes for the next build or anything in between, which is indicative of the TFS engine just being slow and inconsistent in behavior. Trust me, I wish I knew how to fix it and it wasn't behaving this way, but I've read through everything I can find and haven't found an answer yet.

Happy to hear its working well for you though!

# April 13, 2007 2:31 PM

Tim Dallmann said:

Hmmm... I've been running some medium sized builds with tests for quite some time now, and have not noticed any slowness (and I did used to use CC.Net, though not for these projects). I have to wonder if your hardware is sufficient. One thing I noticed right away was that TFS can be resource intensive. I have a separate build machine with 1GB RAM and dual processors. TFS has 1GB RAM and dual processors. TFS SQL is also on a separate box with 2GB RAM and dual processors. So perhaps you just don't have enough horsepower? Just a thought...

# April 13, 2007 3:28 PM

Ayende Rahien said:

@Tim,

The specs that you quote are very high.

Three separate machines? I am going to assume that you are not part of a team of several hundreds, that may require it.

The main issue that TFS is _very_ chatty on the network, and the client is always connected and talking to the server.

# April 13, 2007 6:13 PM

Cos said:

MS _____ Version 1... enough said.

If you expected more then....

# April 16, 2007 8:06 AM

Callum said:

I really think you are doing something wrong somewhere. I have set up many continuous integration builds with TFS and I have never hit problems like you are describing. I also find that my build times are pretty consistent for all builds of the same type. The fact that your build times are variable (and variable by 100% according to your numbers) infers there is some sort of issue somewhere.

# April 16, 2007 5:20 PM

Caleb Jenkins said:

Raymond - for a perf diff that bad... I'm guessing that you have code coverage enabled or some other metric that Nant doesn't support OOTB?

# April 28, 2007 10:45 AM

Raymond Lewallen said:

Caleb,

Only thing enabled is compiling and tests. No other metric, code analysis or coverage is enabled. If noticed that what takes the longest is just getting sources. Since that is the issue, then perhaps CC will not solve my performance problems due to the fact the bottleneck exists in pulling source from source control.

# April 28, 2007 10:56 AM

Ognjen Bajic said:

-Does it take so long to get the sources outside of the build process (when you execute get leatest manually) as well? Normally, get is is a rather fast procedure...

Complexity in other areas of the server (work items, statistics etc.) doesn't make the getting of the source slower since it is a read operation.

-Could you publish how long each build step lasted.

# May 2, 2007 11:56 AM

Name (required)

Your URL (optional)

Comments (required)

Remember Me?

Enter the numbers above:

Add

About Raymond Lewallen

Working primarily in the public sector during his career, Raymond has designed and built several high profile enterprise level applications for all levels of the government. Raymond now works as a solutions architect for EMC. Raymond is an agile coach, Microsoft MVP C# and also president of the Oklahoma City Developers Group and Oklahoma Agile Developers Group. Raymond spends a lot of his time learning and teaching such things as Test Driven Development, Domain Driven Design, Design Patterns and Extreme Programming practices and principles, to name a few. Raymond is also an advocate of Alt.Net. Raymond is primarily a framework guy, so don't ask him anything about UI :)

About Me

Raymond Lewallen

Oh how I miss CruiseControl and NAnt

Comments

Derik Whittaker said:

Raymond Lewallen said:

Jeremy D. Miller said:

Derik Whittaker said:

Raymond Lewallen said:

David Starr said:

Raymond Lewallen said:

Tim Dallmann said:

Ayende Rahien said:

Cos said:

Callum said:

Caleb Jenkins said:

Raymond Lewallen said:

Ognjen Bajic said:

Leave a Comment

About Raymond Lewallen

This Blog

Syndication

Recent Posts

Tags

News

MVP

Archives