« Re: “When vendors get nuts”
A son. »

October 5th, 2007

  TDD: The ‘Logans Run’ of Software Development…

Greetings,

I want to start by making it clear that I know why testing is good, and that it’s really important, but I think that the TDD proponents are glossing over the most difficult part of a project.

I would very much like someone to address the issue of modifying code that is not new, and not already perfectly tested (or even completely specified!).  That is to say, the vast majority of actual code out there.

TDD is intensely focused on the early development phase (or at least TDD proponents are), and on writing new code, as opposed to what the majority of software developers actually do; maintain and update existing code.

It’s really straightforward (and fun!) to write entirely new code in the TDD fashion.  I’ve done it for about 3 decent sized projects now (one Java and two Rails), and it can be really pleasant, and a great focusing tool.  No arguments there; when you do it from the start, it’s really wonderful.

On the other hand, when you’re making incremental changes here and there throughout a very large, pre-existing, only partially tested codebase, it’s vastly less pleasant to try and do it test-first.

The Ruby Autotest tool is not such a pleasant tool at that point.  You stop wanting to write failing tests, because fixing it means autotest is going to try to do a full retest, which sucks for developer flow…  Even if your tests take ‘only’ 5 minutes to run, breaking a test makes you wince, and writing a failing test and then fixing it is for masochists only (and ones who want to miss the project milestones at that).

The focus of every presentation (this one included) I’ve seen on TDD being on the start of a project makes me wonder why nobody’s talking about later in projects…

Nearly every developer out there is going to face a large codebase with poor testing coverage, and will have to make changes that aren’t entirely new code, to existing code that isn’t entirely tested.  Does TDD have a solution for the ‘large, crufty codebase’, or is it suited only for 1.0 versions, small projects, and projects that were TDD from the start?

This isn’t really a rhetorical question for me.  I really want to get my organization’s culture more oriented towards testing.  I’ve got buy-in from lots of people that when they’re writing new modules and services, they’ll do it test-first (or at least ‘test around the same time as the code’, which is all I can ask for at this point), and that’s great.  But has anybody developed any tools to make TDD better suited to maintenance and improving existing code?

–  Morgan

p.s.  I’m skipping BDD entirely, because BDD is so hardcore in the ‘only for already very well specified solutions’ camp, that it’s meaningless for this question.  I’m also using ‘TDD’ and ‘test-first’ interchangeably, and I probably shouldn’t be.

p.p.s.  The title refers to how in Logans Run, everybody was destroyed at 30, so there weren’t any old people. In the world of TDD (or at least TDD presentations), there are no old projects, every one is fresh and new, so the issues that come with an old code base are never addressed.

This entry was posted on Friday, October 5th, 2007 at 8:09 pm and is filed under work, coding, TDD, BDD, testing. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.


3 Responses to “TDD: The ‘Logans Run’ of Software Development…”

  1. On October 6th, 2007 at 4:36 am Arnaud M Says:

    ( WOW, sorry for the truncated post, my browser submitted when I unpaused the media player :S )

    Hi, you express here a point I have felt since the early days of “test-first XXXX”.

    I do not think that the Agile universe has anything to propose yet to address the ‘large, crufty codebase’, apart from the obvious:

    - Struggle to identify functionality suitable for modularization (by leveraging protocol boundaries, file format boundaries), this is probably the most painful step
    - Mock modules in the “modern zone” of the code
    - Identify a candidate module well surrounded by mocked modules
    - Write tests for the surrounded module
    - Migrate the candidate module to the “modern zone”
    - Iterate

    I too have yet to read someone about using BDD in the long run but I am pretty confident that the general direction it shows us is the right one, even if the road ahead is still under construction.

    I feel that Gregg has a good point when he says that we have to assimilate that the test part of the code is integral to the code, not just a lateral artifact. Yes there is an increased cost in test infrastructure with BDD, but we must get rid of the depressing sensation that it is a test cost, to embrace the idea that it is THE (normal) cost of developping the code.

    Software agility is a virtuous circle.

    P.S. I love the title of your post !


  • On November 1st, 2007 at 10:07 pm Dan Croak Says:

    Hey Morgan,

    I think the only solution is really just identifying where test coverage is poor (rcov or otherwise), and making sure your client allows you to carve a significant portion of your time systematically improving the test coverage one small piece at a time (in the real world, you’ll almost always have to be continuing to build new features - with tests - at the same time).

    I definitely leave autotest way before I’m seeing 5 minute test runs (I’ll focus by hand on a test file or test method such as ‘ruby test/unit/user_test.rb’).

    I can’t think of any tools that are going to help you fill the gap any quicker than what you’d use regularly … test/unit, shoulda, heckle, rcov, etc.


  • On November 2nd, 2007 at 1:57 am Cyberfox Says:

    Greetings,
    Yeah, I’m using rcov as much as I can (had to hack up custom rcov scripts to get it to work), trying to use Heckle but having some issues… Using flog to find bad code sections… I don’t know what ’shoulda’ is, I’ll look it up.

    It’s not so much my clients, but my company, as I’ve never done the contract thing. I wish I could carve out time explicitly, but instead I have to pad my estimates and use the extra time to improve the test coverage, and do my best to encourage a testing culture in the other devs so we don’t fall too far behind. :)

    I’ve been given a lot of freedom to do deep refactorings that both improve the ability to build future features, _and_ provides points that can be mocked to improve testability.

    But doing TDD at this point in the development cycle is just painful, unfortunately. I get my TDD fix working on my own projects. :)

    – Morgan