Insight Agile Governance - Article

Tweet

Mail

In an interview with VitAL magazine, James Yoxall, Process Director, IndigoBlue and James Thomas, CTO, IPL reveal how the dynamic management of uncertainty within a clearly defined framework has been successfully used to deliver effective control and manage complexity whilst ensuring the acknowledged benefits of Agile development.

The Challenge of Governance

Agile development has moved into the mainstream, with even the government now encouraging its use. But there is a problem: organisations are keen to attain the benefits of incremental business delivery, rapid return on investment and speed to market, but they also want strong governance. And, in the view of many Agile coaches and practitioners, the variable scope and continuous change inherent within an Agile development are simply incompatible with the principles of traditional governance.

The result of the growing perception that Agile is not effectively governable has been a shift toward its use for non-critical rather than strategic projects. One alternative is to follow the model adopted by many organisations and apply Agile only at the delivery level, whilst using traditional techniques at the management level. The result, however, is effectively a waterfall project with iterations of technical delivery, which is not going to realise Agile’s full benefits.

In some ways, there are aspects of Agile that naturally support governance. It is inherently collaborative, so that key stakeholders have direct visibility of the status of a project at all times; it identifies a business owner with formal authority to make fast, effective decisions; and it emphasises real, tangible progress, rather than simply ticking off lines in a Gantt chart.

Common Issues with Agile and Governance

However, the way in which the majority of Agile projects are run today, there is no doubt that governance is a very valid concern. Without the traditional upfront project specification there is a lack of defined baseline from which to affect governance. The Agile project team is typically focused on internal optimisation, which often results in a reluctance to interrupt progress in order to generate information for the benefit of others in the business. And the reality of complex business organisations is that a single individual cannot truly represent all authorities when making day-to-day decisions.

There are also more extreme interpretations of Agile which further undermine governance objectives, from the insistence on focusing on the near future only and avoiding any form of long-term forecasting, to the avoidance of all change management procedures.

So what is the way forward? Private and public sector organisations want the benefits of Agile; and these organisations need strong governance to ensure strong financial control and minimise the risks associated with larger, more complex developments. Are these two objectives mutually exclusive or is there some element of common ground that will enable organisations to explore the full benefits of Agile without compromise?

Control vs Uncertainty

The key difference between Agile and non-Agile projects is the attitude to and management of uncertainty. On waterfall projects all uncertainty is, theoretically, removed before development starts; while uncertainty is intentionally carried into the development phase on Agile projects.

The traditional assumption that uncertainty is the opposite of control is flawed. With an approved specification there is an assumption that uncertainty has been resolved; yet this leads to unplanned problems late on in the project lifecycle and sub-optimal projects.

Managed uncertainty provides significant benefits. It reduces the up-front investment and time-to-market; ensures that effort is not expended on low-value deliverables ahead of completing high-value deliverables; allows more decisions to be made later when the project is better understood; and reduces the cost of any changes which may occur.

However, delaying some of the uncertainty resolution means the final solution is not definitive at the start. It therefore challenges traditional governance models that assume a clear endpoint, require accurate time estimates to reach that endpoint and recommend senior stakeholder involvement only when exceptions arise.

On the other hand, Agile projects that carry too much uncertainty can fail. Therefore an Agile project should be explicit about the design decisions that are being left until later; it should assess whether specific design work is required outside of the natural flow of work within iterations; and plan the significant design activities. The governance process is then in a position to validate the uncertainty management thinking, risk and planning, and to properly engage in the subsequent, on-going process of uncertainty resolution.

Change Management

Agile is all about change: expecting change to happen, in fact making it more likely to happen through ongoing uncertainty management and reducing the cost of change. However, many implementations of Agile promote the idea that change control is a "bad thing". This is clearly inappropriate where there are formal contractual relationships, but change control of internal implementations is equally essential in most large projects or complex organisations.

The real debate is not whether change control should exist, but how to effect it without restricting the flow of work to the team. Two specific techniques can be used: formally define levels of decision that can be taken independently by specific roles within the project; and formally define levels of change that need to be decided in advance by appropriate authorities, against those that can be made immediately and logged and reviewed afterwards.

This approach has to be underpinned by highly effective management of an agile team that has been set goals that reflect the business needs – not simply the development of software, however innovative. Strong engagement with the wider organisation is key – both on project goals and in providing the information required to support decision making related to significant changes. There is also a need for external agents to validate and confirm the quality of the delivered product, particularly compliance with legal or corporate standards. Projects can and should be explicit about measures of success, which can then be reviewed as part of governance procedures

Conclusion

Agile is no longer the preserve of entrepreneurial organisations willing to accept higher levels of risk to achieve rapid innovation and competitive differentiation. Its proven delivery of fast turnaround, reduced cost and return on investment is compelling for any organisation.

Agile and governance are not mutually exclusive – with the right model, organisations can exploit the benefits of both to transform performance and meet business objectives whilst effectively controlling risk.