Criteria for Selecting a Technical Debt Calculator

In "Calculating Technical Debt", I proposed a set of criteria for selecting a technical debt calculator. This list was:

• Plugin architecture
• Flexible roll-up rules
• Current perspective
• Trend perspective
• Languages supported
• Environments supported
• Supports qualitative data
• Build environments
• Custom dashboards
• Aggregates multiple projects
• Continuous integration servers
• User community

To this list I added a few more criteria relevant to using the tools:

• Tool quality (is the quality calculator buggy?)
• Latest release (is this project alive?)
• Documentation (can I figure out how to use it?)

In "Characteristics of Technical Debt", I came to the conclusion that the McConnell internal quality characteristics were well suited to calculating technical debt. His criteria belong on the list then:

• Calculates maintainability
• Calculates flexibility
• Calculates portability
• Calculates reusability
• Calculates readability
• Calculates testability
• Calculates understandability

I make no claim this list is exhaustive, but it covers enough of the issues important to me that I am comfortable moving forward with it.

In making these evaluations, I need some projects to run through the tools to see how they perform. The obvious choice is some open-source projects. These projects should:

• Be developed in Java
• Have their development history captured in a version control system
• Have high development activity
  
• Have JUnit tests
• Some built with Ant, some with Maven 2

A trip around a few open-source repositories turned up a few likely candidates:

Jena - A framework for building Semantic Web applications. I'll focus on the ARQ component. (Ant)

Jena (ARQ module) has eight releases in its Subversion tags folder:

$> svn ls https://jena.svn.sourceforge.net/svn/root/jena/ARQ/tags

ARQ-2.0/

ARQ-2.0-RC/

ARQ-2.0-beta/

ARQ-2.1/

ARQ-2.1-beta/

ARQ-2.2/

ARQ-2.3/

ARQ-2.4/

Tiles - A Web templating framework.  (Maven 2)

Tiles has eight releases in its Subversion tags folder:

$>svn ls http://svn.apache.org/repos/asf/tiles/framework/tags

tiles-2.0.0/

tiles-2.0.1/

tiles-2.0.2/

tiles-2.0.3/

tiles-2.0.4/

tiles-2.0.5/

tiles-2.0.6/

tiles-2.1.0/

This would seem a trivial matter to pick out some projects to use as a basis for our tool evaluation. It turns out this is not so. I won't bore you with the reasons, but this took WAY longer than I expected.

With criteria and test cases in hand, I'll move on to evaluating the options. Sonar seems like a reasonable place to start.