Devs, not biz leaders, struggle to go Agile

Many developers like to blame business leaders for being demanding and uncertain when specifying IT projects, but companies new methodologies are showing it's sometimes the developers who are struggling the most with process change.

Agile's Adam Boas (Credit: David Braue/ZDNet Australia)

These surprise findings were a common theme throughout a series of introspective presentations at the recent Agile Australia 2010 conference in Melbourne, where the likes of Bankwest, Suncorp and NBN Co shared their trials and tribulations in the increasingly popular method for accelerating the partnership between IT and business project owners.

"One aspect that always frustrated me was the expectation from the technical side that the business don't know what they want," explains Andy Weir, CIO at Bankwest, which initially dabbled in Agile development back in 2008 — unsuccessfully, because it was originally "born, bred and raised" within the IT organisation. "That was a real issue for us."

Under the guidance of Weir, who entered the CIO role early this year, Bankwest has renewed its commitment to Agile and is seeing dramatic results as business and IT stakeholders work together to tackle the "very, very difficult" task of managing expectations across the board.

"The reality is that [business people] are never going to know what they want," Weir adds. "And that's one more reason to go down the Agile route, because they're never going to have all the answers."

In the "scrum" method of Agile development, projects are built through a series of weeks-long "sprints" after which progress is evaluated in group scrums against business requirements, and a plan set for the next sprint. Developers are often paired to encourage sharing of ideas and rapid problem solving. Since it provides shorter iterations between project landmarks, the approach reduces the impact of business indecisiveness by nipping potential conflicts in the bud. Yet solitary-minded developers, Weir added, often had the most problems actually adjusting to this new way of working: "these guys and girls aren't used to dealing with the business face to face, and consequently there were some real challenges," he explains.

"We had to drag some developers kicking and screaming into paired programming, and one of the biggest things for us was getting the developers and testers comfortable in engaging with the business. Whilst it's exciting for some, being ripped out of dark rooms and working with the business directly has been enormously stressful for others."

Tools of the trade

Many developers also resist the unknowns of Agile, fearing that after years of doing things a certain way, they don't have the skills to adapt to a new way of working. Jeff Smith, CIO of Suncorp, calls this the "it-can't-be syndrome": "one of the big things we had to break through were fallacies about Agile, such as that it can't be used for large projects, or for mainframes, or strategic projects, or for infrastructure," he explains.

"Everything we heard was about how it can't be done, and I got frustrated and said 'we have to turn that into what can be done'," he continues. "Belief systems are very hard to change; when you say 'we're going to do something because it's better', people generally don't believe you. Their own past history and wisdom gives them the tools to get their work done, but it's also the biggest constraint about what they can do going forward."

Yet while many of these fears are mental more than real, one development lead noted that many developers are quite able to adapt philosophically but resist the push because business leaders can grab onto the idea of Agile without providing the tools to support it.

"From a developer's point of view, it's not such a pretty sight when Scrum comes in and I don't have any tools for working," says Adam Boas, principal engineer at software-as-a-service provider Aconex, which has embraced Agile as a way to respond more rapidly to the demands of multibillion-dollar projects including the Panama Canal redevelopment.

"Previously, developers had long cycles and protection from the realities of the business; what they wanted; and when they expected delivery," explains Boas.

"They always had something to fall back on — but now Agile is redoing this vast communications and iteration cycle."

"They're getting a lot of exposure and not a lot of help," he adds. "But if they don't have a toolset that's going to allow them to respond to all this rapid change — if they don't have some way to be agile as developers — it's not clear how they are going to respond to all the business' pretty charts. There are differences in the way developers approach problems, and that's got to be accommodated in order to build that trust relationship."

Building such relationships was equally an issue within Telstra, where a recent push towards Agile development delivered strong results — but was, as at Bankwest and Aconex, not without its challenges.

One issue came in convincing developers to become more disciplined with tasks such as documentation. "Overcoming the perception that Agile has less rigour, because it has less documentation and is therefore less of a rigorous and true development process, [caused] massive difficulties," says Michael Bromley, one of several people trying to replicate Telstra's Agile success in his new role as head of portals and online services at NBN Co.

"One of the problems running Agile projects is that generally people are disinterested in documenting the things they need to do to get through their governance gates," explains Bromley. "They're normally left until the very end, and details get dropped out. We probably spent 10 per cent of our time trying to solve the hurdles of staying in the governance model whilst we still executed in an Agile fashion."

Much of the other time was spent, as at other companies, dealing with business indecisiveness. But because Agile lets developers and business leaders address indecisiveness early on — and frees business leaders from pressure to conclusively establish business needs at the project's start — the overall result has been one of gradual acceptance of the need for closer teamwork.

"The organisation is still learning what it needs," Bromley says. "Business leaders very often had no idea what they really wanted; it's common and acceptable, but [in developer-oriented project methodologies] they had never been given the opportunity to say 'I'm going to change my mind'. It was always change requests, three weeks of delays that roll over and impact three other projects, and change was really not viable. But by telling them to tell [developers] what they wanted, it brought them all together to work as a team."