Setting the mood, VersionOne published its second annual survey on the state of agile development, reporting that 57% of respondents stated that their agile development teams were distributed across the organization. The survey reported that 55% of organizations conducted agile development across multiple sites.

According to Brent Barton, CTO for agile development consulting firm SolutionsIQ, typical agile project teams are starting to grow larger. Whereas the typical team might have topped out at 15 members a year ago, Barton is increasingly seeing agile project teams numbering as many as 25 members.

Scrum, which is a project management methodology that can be applied to agile, is also increasing, according to the Srum Alliance, an industry group. Based on nearly 1200 responses, admittedly from a self-selected audience (people already familiar with Scrum), over 80% began using the methodology only within the past couple years.

And, while Scrum (like agile methodologies in general) are typically associated with new software development, roughly half the respondents said that they were now extending use of Scrum and agile for software maintenance or upgrades as well.

Agile is an umbrella concept that describes development, testing, and project management techniques that are designed to be much lighter weight than conventional approaches.

They stipulate doing "just enough" planning to avoid upfront analysis paralysis, and agile typically encourages intimate collaborations between the business and the different players that are involved in designing, developing, and testing software. As such, agile has been primarily associated with tiny projects and informal, often spontaneous idea exchanges with colleagues down the hall, in addition to daily planning sessions.

For instance, as agile planning tools provider Rally announced this week that it was adding features to improve planning and visibility across multiple projects, a number of consulting firms recounted war stories on how they scaled out agile, or how agile could scale up as an enterprise development strategy.

Admittedly, the VersionOne survey didn't necessarily report that most agile projects themselves were already distributed, but that for most respondents, agile has caught on to the point where it is being used by multiple teams at multiple sites within their organization.

At least one provider recounted results from an agile project using Scrum project management techniques that was distributed, not only across multiple sites, but continents as well. Outsourcing firm Exigen described a project that involved teams spread across three North American sites, plus a main development team in St. Petersburg, Russia, that cranked out over a million lines of code over 14 months.

The secret sauce was encouraging team members to improve their oral communications skills, have at least 3 hours of office hours overlap between North America and Russia (the Russians would have to be in the office between noon and 9pm, local time, to make this work), and frequent instant messaging contact that would supplement the daily Scrum meetings and the paired development, and development for testing XP techniques that were used on the project.

"It wasn't as smooth as face to face meetings, but the process was workable," admitted Doug Mow, a senior vice president with Exigen. But something must have worked, as the project was written up by Scrum methodology guru Jeff Sutherland. In his case study, Sutherland praised it as "the most productive Java project ever documented," based on a documented rate of 15.3 function points (a metric for amount of code) per developer, per month.

So what does it take to institutionalize agile across the enterprise? A team at Digital Focus, now a unit of software development consulting firm Command Information, delivered a presentation stating that you must focus on four key areas that start, not surprisingly, by encouraging significant cross training to make agile cross-functional teams more self-reliant.

Specifically, agile projects typically require cross functional teams that bring together developers, testers, architects, and business representatives in close collaboration. According to Dave McMunn, managing director for Command's Digital Focus unit, it means that related disciplines, such as software development, QA, and database administration, should learn each other's skills so they don't have to constantly call in external specialists each time to put out fires.

To some extent, the recommendations also encourage cross-fertilization between the technology and business sides as well. McMunn explains that by sharing skills and becoming more self-reliant, agile teams stand a better chance of staying together longer through individual or multiple projects.

Other recommendations include making the project or program management office become more hands-on. Instead of simply worrying simply about the execution of tasks and the management of interdependencies between projects (e.g., where different projects may rely on the same database table to be revamped), PMO staff should drop their clipboards and go more native, mingling with project teams to become mentors who can provide the big picture to agile teams busy solving their own little piece of the business problem.

Additionally, changes would need to be made to vendor procurement processes to get them more in sync with agile planning processes. In effect, just as agile dispenses with highly detailed requirements documents up front, the same should apply to technology procurements. Just as the agile team is encouraged to collaborate, McMunn suggests that vendor procurements be structured more as long-term partnerships.

Finally, McMunn urges that incentive pay or onus structures and hiring practices be changed to emphasize team work over narrow technology expertise. "It is easier to teach technical skills than it is to teach teamwork," he commented, adding that nobody has yet figured out what the new pay systems would exactly look like. "That's an area that still needs development," he concedes.

Our View

There's little question that agile methodologies have struck a nerve given decades of dismal track records by even the best software development organizations. There have been countless studies making the same conclusion: that software is typically delivered late, over budget, and lacks full support of original project goals.

Agile advocates like Scott Ambler, now of IBM, have long railed against traditional heavyweight processes that typically bogged software development projects down in analysis paralysis, often resulting in software that's obsolete the day it's delivered. Yet the proposals and ideas delivered at Agile 2007 demonstrate that while agile is starting to scale, it still has along way to go before the methodology becomes robust enough for enterprise deployment.