3 misconceptions I would like to see removed from the Agile Extension to the BABOK®

Featured
11604 Views
31 Comments
461 Likes

3  misconceptions I would like to see removed from the Agile Extension to the BABOK®The IIBA (International Institute of Business Analysis), in collaboration with the Agile Alliance, recently released a draft for public review of the Agile Extension to the BABOK®. The guide describes “business analysis areas of knowledge, their associated activities and tasks, and the skills necessary to be effective in their execution within the framework of agile software development”.

Below are 3 misconceptions that, in my opinion, the current draft of the Agile Extension is helping perpetuate:

1. There are only two models, agile and waterfall. If you aren’t using the former, you must be following an ineffective plan-driven methodology with a single requirements phase.

The Agile Extension briefly recognizes the existence of non-agile, non-waterfall methodologies, stating, on page 7, that “agile methodologies are a subset of iterative software development”. However, other parts of the document are written in a way that reinforces the prevalent misconception that the only alternative to agile is the failure-prone waterfall method. I’ve extracted two examples to illustrate this issue:

“Analysts are required to approach requirements from a 360 degree perspective. They are required to work with the business sponsor on a strategic level, and define how the proposed product or feature aligns with the organizations portfolio and strategy. They must then work with the business and project team to break this vision down into requirements that support effective and accurate estimation. In an agile project this is done for each iterative release, as opposed to the single requirements phase that exists in plan‐driven methodologies. The analyst delivers just‐in‐time, detailed requirements to the development team so they can build only what is required for a specific iteration.” (p. 5) [emphasis mine]

There are different ways of detailing requirements in plan-driven methodologies that are neither just-in-time nor the monolithic requirements phase typical of the waterfall model. I know many BAs (myself included) who have been performing the exact same tasks described above in non-agile environments, avoiding both extremes of just-in-time requirements and requirements detailed in their entirety prior to the development phase. Many of the methodologies I’ve used in projects in the past 15 years could be classified as “plan-driven”, but rather than proposing a single requirements phase, they prescribe overlapping iterative development cycles in which requirements and design are refined and revised during project execution.

The concept that there are alternatives to the opposing models of JIT and the “big requirements upfront approach” should not be ignored in a guide about agile for business analysts. Such understanding is crucial to allow analysts the opportunity to evaluate the different alternatives in requirements processes that are available to support agile projects with different characteristics and levels of complexity.

“Agile offers the opportunity for business analysis to benefit from the frequent feedback provided by the business. “ (p. 3)

Again, the implied assumption here is that, outside of agile, you are working on a sequential, waterfall model that prevents feedback from happening throughout the course of the project. In the plan-driven, non-agile development efforts I’ve worked on, early and frequent feedback for the analysis work was always available, either as part of a formal process, or in response to an informal request from the BA to the business stakeholders. Such feedback loops were available throughout the stages of requirements discovery, validation, and communication. It’s true that agile methods facilitate frequent feedback, but there is nothing preventing non-agile development approaches from obtaining frequent feedback from business stakeholders, not only for written requirements, but also working software, early models created to test the concepts of the system, or even paper prototypes.

2. The model of just-in-time requirements is superior to all others

“By providing just‐in‐time requirements, there is less rework of requirements because only the requirements required for the current release are defined in detail and developed.” (p. 3)

I don’t see, in real life, any supporting evidence to the conclusion that just-in-time requirements necessarily translate into less rework. Sure, if you prepare requirements 2 years, or even 2 months, before development starts, there is a high probability that changes will be needed in the original requirements to reflect evolving business needs. On the other hand, producing subsets of requirements focused on just what is required for the current release in some cases significantly increases the risks of important requirements being overlooked or misunderstood. Limiting the analysis to what is being produced in the next cycle often prevents analysts from studying requirements in relationship to other requirements, and from seeing each piece of a system as a meaningful component of a working whole. An example of just-in-time requirements leading to substantial amount of rework due to lack of systems thinking can be found in this article:  Agile vs. Traditional Software Development: Why is the debate still going on? - Part I: The dangers of bounded rationality.

3. Documentation should be automated, barely sufficient to meet the needs of the team, and written to fulfill an immediate need

“A general principle with agile is that if a document is valuable enough to be created, it is probably important enough to be automated so that it is part of the living code base. This has lead to the rise of automated acceptance testing and behaviour driven development that allows business analysts to work with quality assurance professionals to create executable specification in the form of examples.

This principle comes directly from the Agile Manifesto which says ‘Working software over extensive documentation’. It is often misinterpreted as meaning no documentation. Instead, documentation should be barely sufficient to meet the needs of the team.” (p.100)

“In agile methodologies, the software is constructed within days of the team agreeing to deliver a set of requirements, and so does not need to include information that the team can remember or agree to during the construction process.” (p.99)

“Ensure that all documentation produced through business analysis is intended to fulfill an immediate need, has clear value for stakeholders, and does not create unnecessary overhead.” (p.99)

“The context plays an important factor in the amount of documentation required. Some projects are mandated to produce documentation by external entities (for example, regulators). Documentation may also be needed to provide a long‐term record of decisions reached by the team or functions implemented in the application. However, this documentation can be written after the software is developed, which ensures that it actually matches what the team delivered.” (p. 100) [emphasis added]

I am yet to find data to back the claim that automated, barely sufficient documentation produced to fulfill an immediate need generates better, faster, or cheaper results in most software projects. The agile concept of “just enough, barely sufficient” documentation is an understandable reaction from the software development community to the exorbitant amount of paperwork often required in software projects (mainly in large companies). However, in my consulting practice I’ve seen many organizations suffer the consequences of waiting to document a complex software product after it was developed, or, even worse, just using code and automated tests as the only documentation of system behavior, and postponing additional documentation until someone asks for it.

A typical example was a project implementing complex business calculations that used multiple variables in various combinations. The developers’ work was based mainly in face-to-face conversations, with minimal documentation. After deployment, remembering exactly what was decided for each possible combination of variables was tricky, even for skilled developers who had worked on this part of the solution. Developers found it hard to determine in detail what the system was doing in some of the exceptional cases, and to remember why the system was made to behave like it did. The time required to recover such knowledge (including the background of some decisions that were not captured anywhere) was significant. The impact included avoidable costs reconstructing the information, the refusal from some customers to use the system until a specification was produced, and uncertainty and delays during the discussion of enhancements to the product. In this case, and in many others in my past experience, producing just enough documentation to allow the software to be built may have generated short-term benefits, but at a high cost to future decision-making and system adoption rates. I see a level of incoherence between the agile concepts of “embracing change” and “barely sufficient documentation”, as the latter often proves to be a hindrance to future change, as people struggle to remember, or are no longer available to share, undocumented implementation details, and the reasons behind them.

The idea of avoiding excessive documentation is clearly beneficial to software projects. Still, while it’s a positive development to get rid of documentation created just to “check a box”, or to ensure that the performance of a team can be measured against some bureaucratic rule, it seems dangerous to start promoting, as a general practice, barely sufficient, preferably automated, just-in-time documentation. Where is the evidence supporting this approach? Perhaps it’s time that we BAs start following Greg Wilson’s suggestion, and adopt higher standards of proof before a concept is accepted as a general recommendation.

- - -

The Agile Extension to the BABOK represents an important contribution to the business analysis community, offering analysts a useful introduction to agile concepts and explanation of the typical working practices used by BAs in agile projects. The document also provides an overview of some of the commercially available agile approaches, recommends BAs to research various agile frameworks to understand their pros and cons, and recognizes that it is “common for project teams to blend characteristics from more than one agile methodology based on unique team composition, skills, (...) and other factors”.

As stated in the guide, many of the concepts and approaches described will prove valuable to business analysis in any methodology and environment. The missing piece, in my opinion, is a clearer acknowledgement that:

1) waterfall is not the only alternative to agile frameworks--there are other models that are being successfully used in IT projects throughout the world, from which BAs can learn and benefit;

2) agile practices (such as JIT requirements) won’t necessarily produce superior results in all projects;

3) avoiding excessive documentation, and any documentation that doesn’t serve a valuable purpose, is a valid goal, but that doesn’t mean that automated documentation, or documents prepared only to fulfill an immediate need, are good replacements to quality documentation that may have to be written long before its needed, as means to ensure the right information about system behavior will be available when it becomes necessary for the organization to retrieve it.

As I wrote in another article,

Agile practices are not intrinsically “value-adding” — they must be aligned to business needs and goals in order to provide true value. Customary practices such as on-site customer, stand-up meetings, or test-driven programming may be suitable for some projects, but should not automatically be deemed supportive of agility just because they are part of a commercially available agile method.

In addition to describing agile practices, and recommending that analysts spend time researching the various methodologies to understand their pros and cons, the Agile Extension should encourage BAs to use their analytical skills, and influence within their project team, to promote the selective borrowing from agile and traditional methods to fit the needs of each individual project. Many of the agile teams I’ve worked with recently have been developing “hybrid models” that incorporate elements from agile and more traditional software development methods in order to increase the odds of the solution delivering real value for stakeholders. A good example is the initiative to identify areas for which producing detailed requirements upfront can mitigate project risks caused by a lack of understanding of the impact a component will have in other parts of the system. Another is the use of comprehensive documentation to ensure compliance with regulatory requirements, while still taking advantage of agile practices such as short development cycles that continuously deliver working software.

By recognizing that many non-agile practices are also capable of producing business value, and that successful results may require blending elements from agile and traditional models to better fit project needs, the Agile Extension would help more teams achieve the benefits of agility while avoiding the “square peg in a round hole” syndrome that often forces agile methods into situations to which they are not suited.

 


Adriana Beal is a native of Brazil living in the U.S. since 2004. For the past 15 years she has been leading requirements discovery efforts and software process improvement initiatives for a diverse client base that includes IT, telecom, and major U.S. institutions. Adriana specializes in business analysis of complex software systems, and frequently uses hybrid software development approaches that combine agile and traditional practices to ensure the product’s strategic alignment to the business need. She has two technical books published in Brazil, and work internationally published by IEEE and IGI Global. Her educational background includes a B.S. in Electrical Engineering and an MBA with emphasis on strategic management of information.

Like this article:
  461 members liked this article
Featured
11604 Views
31 Comments
461 Likes

COMMENTS

kbrennan posted on Tuesday, February 21, 2012 8:45 AM
Hi Adriana,

Thank you for your feedback. The Agile Extension is only a draft at this time and we released it specifically to solicit feedback from the public. I would definitely encourage you or any readers to send us your comments through the feedback form or by email to Agile@iiba.org. We're taking comments through the end of February 2012 and will be revising it following the end of the feedback period.
adrianab posted on Tuesday, February 21, 2012 9:03 AM
@kbrennan:

Thanks for leaving your comment -- I hope many readers will see your suggestion and send their comments to the IIBA.

I did submit my feedback several weeks ago, and recently learned (during a talk I gave to the Austin IIBA Chapter) that at least one other BA from Austin submitted her suggestions as well.

Adrian Marchis asked me to write an article about the current draft, and I agreed to do so as I think it helps the debate.

Regards,

Adriana
ajmarkos posted on Wednesday, February 22, 2012 1:34 PM
Hi:

Per Scott Ambler, Agile is, while maintaining minimal documentation, about quality documentation, including sufficient integration considerations. Quality documentation captures the essential. (Seems most folks do not focus alot on quality and integration when talking about Agile.)

Now a BA's main charge in an Agile project is (or should be) coming up with the bigger picture of behavioral requirements. This is used to both scope out the project and plan the various iteratitions

Putting it all together: The BA's main task in an Agile project is comming up with a properly integrated big picture of the essentials, with minimal documentation.

Guess what the BA's main task in a Waterfall project is (not a bad Waterfall project, but a successful one.)? It is coming up with a properly integrated big picture of the essentials with minimal documentation.

As far as Business Analysis is concerned, Agile is not - or should not - be a big deal.

Tony
lenwhitmore posted on Friday, February 24, 2012 9:26 PM
Hi Adriana,

I had been wondering what your comments were about the extension. I have my own which I have been working on and will be sending in on the form this weekend. I had approached Ellen Gottesdiener with some feedback as she is one of the two people I know that where on the committee that drafted the Aagile extension.

Anway after reading it I came away with a sense the IIBA still doesn't understand what agile software development is.

Let give you some examples of what caused that uneasiness to be reached. From the last paragraph on page 3 it states "Iterative development processes provide opportunities". Iterative, while the most used of methods I think we would agree, is not considered agile. In section 2.1.1 on Backlogs on page 8 "The backlog serves as a wish list for the product." A wish list come on. How about "An ordered list of work required to develop the product." How about section 2.1.3 on Roles and Responsibilities on page 9 the poor choice of wording "The scrum master is responsible for managing the Scrum process and managing any blockers... " The SM responsibility is to manage blockers, sounds like a football game. The responsibility is to remove impediments, plain and simple.

I know you have read this before from me but it is worth saying again. The way I see it, 'Agile' is not a methodology. Agile is not really an approach either, I think that Don Wells in his article "Agile Software Development: A gentle introduction" put it best "The most important thing to know about Agile methods or processes is that there is no such thing. There are only Agile teams. The processes we describe as Agile are environments for a team to learn how to be Agile."

Anyway I liked what you had to say and hope they listen.

Len Whitmore
adrianab posted on Saturday, February 25, 2012 2:10 PM
@Len -- it's great to have your opinion here, and to learn that you are going to send your comments to IIBA as well.

I agree with your comments on iterative developments and backlogs.

The only thing we don't seem to be on the same page is regarding "agile is not an approach". This because every single academic work I've read about agile (and I like academic studies because then tend to be more rigorous and evidence-based than books written by consultants) treat it as a methodology or approach. (I prefer "approach", but it's beside the point here.) You may favor a different use for the term, and based on your mental model, say there is no such thing as agile methods, but I don't think we can ignore the fact that in the IT industry, "agile" is a term adopted to reference a set of methodologies/frameworks that share a common manifesto.
mnicholas1 posted on Tuesday, February 28, 2012 10:33 AM
Adriana, I agree with your comments especially # 3. (Documentation should be automated, barely sufficient to meet the needs of the team, and written to fulfill an immediate need).

The Agile Manifesto begins with the following preamble:
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

What oftentimes is overlooked is the qualifying statement after that preamble: i.e. "That is, while there is value in the items on the right, we value the items on the left more." Notice it states that there is value in the items on the right.

My beef is a lot of folks have come to construe documentation in any form as Not Agile. Certainly there has been a lot of unnecessary requirements documentation. The fact is though, that as the complexity and criticality of a solution increases, there is a place and necessity for good documentation. I do agree with the Agile principal that states: "Simplicity--the art of maximizing the amount of work not done--is essential."

Processes and documentation should be rationalized to factor in the business value of doing them and if so, to what level of granularity. Ultimately we need to ask: If there's no business value in doing something, then why are we doing it. We still need to document requirements to a level of specificity that actually has business value . . . e.g. who are the consumers and what will that documentation be used for.

A lot of people would like to believe it's either not necessary to document requirements or that no formal rigor is really necessary, or that you can do something at the last minute, but I respectfully submit that sentiment is just not true . . . especially if you're working on delivering a moderately complex solution.
ajmarkos posted on Tuesday, February 28, 2012 12:34 PM
mnicholas:

Great comments! For system behavioral-related requirements value is in documenting the essential. That should be the Agile approach: Determine what is essential to document, and then ensure that only such is documented.

The current focus "do just barely enough" is problematic because it fosters just what you are saying: The very prevalent believe that documentation in any form is not agile.

Tony
adrianab posted on Tuesday, February 28, 2012 1:27 PM
I agree, great comments, everyone.

@mnicholas1: "Processes and documentation should be rationalized to factor in the business value of doing them and if so, to what level of granularity." -- great observation.
kbrennan posted on Wednesday, February 29, 2012 8:48 AM
Hi Len (and everyone else),

Thanks again for your comments. I'd like to take the opportunity to clear up what I'm concerned may be misconceptions about our standards development process.

The Agile Extension, like every other IIBA standard (such as the BABOK Guide) is developed primarily by volunteers and practitioners with experience in the topic, rather than written by IIBA staff. We do have a technical writer/editor who worked with the team to ensure consistency of style and voice, as well as structuring the document, and I reviewed it to ensure that it was consistent (where needed) with the content of the BABOK Guide and drafted some sections that covered topics I had personal experience with, but this document is really the work of that volunteer team. I think they did a great job, but ultimately that's something the community has to judge.

The Agile Extension brought some unique challenges for us, as this is an area where business analysis practice is still emerging. We also had participation and support from the Agile Alliance (www.agilealliance.org) to further ensure that the perspective of the Agile community was correctly represented.

However, I do want to stress that this is a DRAFT publication, and not the final product. Past drafts for public feedback (of the BABOK Guide, for instance) have led to significant revisions. Our goal was to put something in front of the community as soon as we had a reasonably complete draft, rather than spend a lot of time polishing it only to find out that it didn't meet the needs of the community.

While I can't specifically comment on the feedback above, because the volunteer team will decide what needs to be changed based on the responses we get, we will soon begin reviewing all of that feedback and incorporating it into the next iteration. At this time I can't tell you when that release might occur because I don't know how significant the revisions will need to be. All I can say is that we anticipate that the Extension will continue to be developed as Agile itself continues to change, and that we plan on integrating Agile further into the next iteration of the BABOK Guide as well (along with other topics).

Thanks once again to everyone for your comments, and I'll remind all of you that today (Feb. 29th) is the last day for feedback so please submit your comments to us through the feedback form or at Agile@iiba.org.
ajmarkos posted on Wednesday, February 29, 2012 9:47 AM
kbrennan:

Heres' some feedback: Is the pattern, per the BABOK, of talking about what is popular to continue, or is the objective to present what what logically makes sense. Agile is an excellent example of how the two are in dissync.

Tony

shaneh@softed.com posted on Friday, March 2, 2012 9:30 PM
Hi Adriana

Thanks for taking the time to put this feedback together.

The current release of the Agile Extension to the BABOK(r) is a draft and we (the team putting it together) really appreciate any and all feedback. I have taken your article and the additional comments above and included them with the other feedback we have received.

Please feel free to send more thoughts and comments to the feedback address: agile@iiba.org

All the best

Shane
(Program Director - Agile Extension to the Business Analysis Body of Knowledge(r))
mark.hunt posted on Tuesday, March 6, 2012 10:05 PM
Adriana, I'm not sure I agree either that the Agile Extension denigrates nor implies specifically that Agile is the only way to go.

To your first objection...
I have been looking forward to this addition to the BABOK since the draft was first being distributed last year. One unfortunate problem with the current BABOLK is that there is really no mention at all regarding Agile nor what types of requirements gathering, business processes or expecations there are for Business Analysts in this world.

The intent of the Agile Extension as I have always understood it (and is clearly stated in several ways starting on the first page) is to provide guidance to Business Analysts on Agile teams.

I don't feel it's the responsibility of a manual or guideline of a new process or guidelines to explore all of the other options available, explain the differences, and tell the reader which one they need to follow and why. If your company or team doesn't follow XP, Scrum or other flavor of Agile then don't read the Agile Extension. However, if you do, why make the user question why they're following Agile processes-- just help explain what to do in order to be successful at it.

Regarding objection two...
Actually, there are a number of studies related to JIT development and requirements gathering; look up Scott Ambler among others to find some of these.

So the objection is being made that the Agile Extension is stating a falsehood, and that the only correct way to gather requirements is up front and in detail. I would argue that's not necessarily true either, but we could all find examples where both approaches have failed.

Removing this statement (“By providing just‐in‐time requirements, there is less rework of requirements because only the requirements required for the current release are defined in detail and developed.”) from the Agile Extension is pretty much suggesting there is no benefit to following Agile processes on a project-- so why even have the Agile Extension in the first place? I think that's an argument that goes beyond this and becomes an argument on the benefits and advanrages of doing agile development.

This is mentioned in the closing arguments as well-- that the Agile Extension shouldn't be able to state that following agile methods won't necessarily produce improved results, and that it should even clearly spell that fact out. Again, we're getting into arguments on agile vs. non-Agile, and in my opinion making all of these apologies, explanations and caveats goes beyond what a BABOK document needs to do. Leave the arguements on what processes are "best" to discussion groups and blogs instead. If as a BA you feel you need to understand what is expected of you as an agile BA, then I think it's a must read. If not, or if you feel that agile is an overblown set of principles that have no real benefit in the real world-- then don't.


Finally, objection three...
That there is no proof that JIT and Lean requirements saves time and can reduce rework. Same response here- read Scott Ambler and others who have done these types of studies, then ask teams who have actually done this successfully. Belive it or not some of us actually have.

In closing, I commend Shane for all of his work in seeing this Extension through and everyone else who provided input; my only complaint at all would be that I wish we had this available to us years ago :-)
shaneh@softed.com posted on Tuesday, March 6, 2012 10:12 PM
Hi Mark

Thanks for the kind comments and for the feedback.

We are now starting to consolidate and process the feedback from various sources about the Agile Extension, and it will take us a while. There are over 300 separate comments to be worked through and consolidated, so please be patient with us.

As soon as we have processed all the feedback we will prepare the next release and get it our to the community.

Thanks to everyone who cares enough to provide us with their thoughts and feedback - we really do want this to be a community product.

All the best

Shane
adrianab posted on Wednesday, March 7, 2012 9:30 AM
@Mark:

" I'm not sure I agree either that the Agile Extension denigrates nor implies specifically that Agile is the only way to go."

I do not agree with that statement either-- not sure where you got the idea that I think the extension implies agile is the only way to go. In item #1, I complain about the document often contrasting agile with waterfall, as if waterfall was the only alternative to agile. (If it were, then I'd conclude agile is the only way to go because waterfall is a terrible way of building software -- with very rare exceptions).

"So the objection is being made that the Agile Extension is stating a falsehood, and that the only correct way to gather requirements is up front and in detail. "

I'm sorry if my writing I wasn't clear enough for you -- definitely not what I was saying and definitely not how some other people interpreted what I wrote. In my entire life as a BA consultant, I only had one waterfall project. A few of my projects have used some sort of agile approach, the majority of them use an incremental/iterative method in which requirements are created and refined throughout the project. I don't even use the term "gather requirements", as I disagree with it, so maybe you were talking about some other article here.

"I don't feel it's the responsibility of a manual or guideline of a new process or guidelines to explore all of the other options available, explain the differences, and tell the reader which one they need to follow and why."

I agree. That's not what I meant either. My point is, if you are going to compare a method/approach/framework to another to say one is better (as the extension does in some points), you should at least acknowledge that the two methods/approaches/frameworks you are comparing are not the only options out there. To me, anything is better than waterfall, so there is not much point in saying that agile is better than waterfall.

"Actually, there are a number of studies related to JIT development and requirements gathering; look up Scott Ambler among others to find some of these."

I read Scott Ambler, and I would suggest that you and everybody involved in reviewing the Agile Extension read also "Making Software: What Really Works, and Why We Believe It" by Andy Oram and Greg Wilson. Lots of good information on why many of the agile studies out there do not pass scientific tests and are simply not verifiable. In any case, if the team working on the Extension thinks Scott Ambler's work can support their statements, I'd like to see the citations there, please.

Having said all that, thank you for all your comments, and thank you Shane, for taking into consideration all the feedback you are getting here and at the IIBA website for your next review.

The more different viewpoints are shared by the BA community, the better for making sure IIBA ends up with a quality Agile Extension.

mark.hunt posted on Wednesday, March 7, 2012 6:01 PM
@Adriana:

My bad in using the term "waterfall"; to clear up any confusion perhaps it's best to replace anywhere I used "waterfall" above with "non-agile"; from discussions we've had before I should have known better than to use that buzzword :-)

And perhaps I do have a misunderstanding in what you are asking to be done in your article. In it you state several times that there is no documentation that agile processes in fact add any value- or using in using JIT requirements or Lean. In your follow up you further stated that if there are in fact any studies purport to show that agile methods improve project delivery that these are flawed and aren't verifiable. So this is where I came to the opinion that you seemed to feel that the Agile Extension is making false claims or else should contain caveats and disclaimers that downplay any advantages to using agile methods.

Rather than devolving into an argument on agile vs. non-agile, I'd rather ask something constructive: in order to help Shane and his group, what in your opinion would allay your concerns in updating and clarifying the Agile Extension to the BABOK?
adrianab posted on Wednesday, March 7, 2012 7:37 PM
@Mark -- thank you for clarifying your use of the term "waterfall".

I do have a hard time equating "waterfall" to "non-agile" because there are plenty of incremental/iterative approaches that cannot be considered agile (even the Extension acknowledges this, just not consistently throughout the text) and are completely different from the waterfall model as well. And problems with terminology can definitely get in the way of understanding each other's points.

In regard to your summary of my article and your last question, I feel like asking you back: are you sure you read it? :-) Because I don't think what you wrote reflects my position, and at the end I took the time to summarize my points in numbered items, starting with "The missing piece, in my opinion, is a clearer acknowledgement that: (...)"

Based on the compliments I received from a couple of thought leaders in our field (either in LinkedIn or by email), I'm left to believe that my message is clear enough to serve as one point of input for IIBA, and that my views are shared by at least a few well-respected authors with connections to the Institute (I certainly hope they will make their position known to them, if not here ;-).

Just to reinforce my main point, what I am expecting from the IIBA is more rigor with terminology and evidence. When you say something like "By providing just‐in‐time requirements, there is less rework of requirements because only the requirements required for the current release are defined in detail and developed.”, where is the evidence for the conclusion of "less rework"? The statement may be right, it may be wrong, (The book I mention in my previous reply, by the way, offers evidence to the contrary, in particular for projects large in size and criticality, so for credibility's sake, that particular statement should be backed by solid argumentation, not to mention an indication of the contexts in which these results were found.)

bobsavage posted on Wednesday, March 14, 2012 2:36 PM
@Adriana, I'm a little late responding to this, but I think your remarks are excellent.

Something that I think catches people up is that they think of Agile versus non-Agile as being a binary proposition. In my opinion there is a spectrum running from Agility to Safety. An example project on the "safety" side of the spectrum might be a nuclear power plant, and an example on the "agile" side might be a blog about your kitty.

The binary view, I think is due to perspectives that arose in two specific historical contexts. When software development methodologies were first being documented, it was only natural for them to focus on the "safety" side, which led to early codification of the Waterfall approach, but soon after that approach was criticized for being unwieldy on most project types. It also shouldn't be surprising that during the '90s a new perspective arose that focused on the "agility" side of the spectrum. But both of those historical moments have passed.

It is time for us to acknowledge the existence of the entire SPECTRUM with "agility" and "safety" being mere poles. Most projects are going to land somewhere in between those poles. Slavish adherence to either "safety" or "agility" is already distrusted by many, but we must also reject the perspective that we must make a binary choice between "plan-driven" and "change-driven". Look at the needs of your organization, the project, and even the phase within the effort, and then find the place on the "Agility-Safety" spectrum that is appropriate for your current needs.
rpannone posted on Thursday, March 15, 2012 1:23 PM
Hi Adriana,

Thank you for sharing your expertise and having the courage to challenge the status quo.

Let me start by stating the first formal description of the waterfall model is often cited as a 1970 article by Winston W. Royce though Royce did not use the term "waterfall" in this article. Royce presented this model as an example of a flawed, non-working model.

Since my system-software development career spans 35 years I once was a staunch supporter and practitioner of the “waterfall” approach to system-software development.

Based on lessons learned I am now a staunch supporter and practitioner of an iterative and incremental approach to system-software development.

The traditional waterfall approach is to document the requirements for the whole system and then freeze the requirements to provide a stable base for implementation to proceed.

Though this may work in some cases, it assumes that we can come to a complete understanding of the system before developing real code. Perhaps this is possible in simple or well-known domains but this is a risky assumption to make for most product (system-software development) projects high in uncertainty and risk.

Fundamentally, agile product (system-software) development is about building product features incrementally by Customers (Business Partners) and IT working collaboratively.

Being agile is built around a key assumption – we continue to learn about the solution during the project. When we recognize that our understanding develops during the project then we realize that trying to document all requirements and freezing requirements during early project phases will either cause rework or the delivery of a system that does not meet the Customer needs.

We need an approach to product (system-software) development and delivery that can embrace change during the project. The iterative and incremental nature of being agile evolves both requirements and the system-software under development.

In order to bust the myths and misconception between waterfall and agile one needs to start with first and foremost an individual, team and the enterprise needs to come to agreement or a common definition for what agile and waterfall are and then take this understanding to their next level of awareness.

The bottom line after all is:
1. Deliver commercial or operational value early and often, giving ourselves the best opportunity to beat the competition to market, realize revenue and discover insights that we can use to help us improve.

2. System-software development should revolve around teamwork, integrity, a spirit of humility, transparency, inspection, adaptation and always doing right by each other, our internal business partners and external customers.

http://www.linkedin.com/in/russellpannone
adrianab posted on Thursday, March 15, 2012 4:52 PM
@bobsavage -- well said, thank you for adding your comments.

@rpannone,

"Based on lessons learned I am now a staunch supporter and practitioner of an iterative and incremental approach to system-software development."

Same with me :-). I hope nobody ever gets the impression that I'm a supporter of the waterfall method.

"Being agile is built around a key assumption – we continue to learn about the solution during the project."

See, there are multiple "non-agile" approaches that are also built under the same assumption. I'm using the word "agile" here the same way it's used in the Agile Extension ("Agile is a term used to describe a number of iterative development methodologies that have developed over time. (...) Agile methodologies are a subset of iterative software development.")

Just to be clear, I have nothing against agile approaches when they are adopted based on being a good fit to project profile. I'm a big fan of "blended" methods designed to meet the needs of individual projects, and when I talk about blended methods, it has nothing to do with waterfall, but rather with accepting that sometimes an iterative/incremental "plan-driven" approach will be more suitable than a "change-driven" agile approach.
rpannone posted on Thursday, March 15, 2012 7:23 PM
In addition to my comments with respect to waterfall versus agile I am all for planning and it is a core practice in iterative and incremental development and being agile.

Calling an acceptable approach to system-software development “plan-driven” is not going to give you the results you are looking for primarily because it inherently conveys the wrong message.

Most projects that are plan-driven quickly turn out to be projects that are managed by the “fuel-tank”; a driver may consume enough fuel to drive from Los Angeles to New York but just because the fuel is spent is no quarantine the car got to New York.

A “plan-driven” approach is followed by tracking time and money spent against your plan. Though this may appear on the surface to be a “key” process indicator it leads to tracking progress based on whether or not we are within the tolerances levels for expended cost and schedule and not an emphasis measuring commercial or value delivered.

My article titled Differentiating between Estimating and Committing should give you additional insights to my thoughts on this subject.
adrianab posted on Thursday, March 15, 2012 8:39 PM
@rpannone: From your article:

"Traditional approaches assert that uncertainty can be defeated by big requirements, big design, and big planning up front, but that is incorrect. "

That is not necessarily true. There are plenty of traditional approaches that have nothing to do with big requirements and big planning up front, or fixed time and budget. I find it hard to advance the debate when people reduce it to a "waterfall vs. agile" discussion, as if we haven't developed many different software development methods that are non-agile and non-waterfall, some of them dating back to the 50s. I've worked for a consulting company in NYC between 2005 and 2010. We never used an agile approach, and never used waterfall or big requirements upfront document either. We used a lot of prototyping and incremental development with lots of customer feedback in our "plan-driven" method, and the proof it worked is that the most clients had been outsourcing software projects to us for 12 years or more.

In any case, I agree with your estimating vs. committing differentiation. Thank you for suggesting the article.
rpannone posted on Thursday, March 15, 2012 10:22 PM
What I see happen time and time again when folks approach system-software development based on a plan-driven approach is they gravitate towards the traditional project management iron triangle.

When applying an iterative and incremental approach to system-software development and being agile there is a new “agile triangle”' that is based not on time, cost, and scope as in a plan-driven approach but instead, based on value, quality, and constraints (time, cost, and scope).

The general idea behind the agile triangle is that we need to take the focus off of delivering to a set schedule, a fixed budget, and some predetermined set of deliverables; and focus more on the value the product is delivering.

This is a concept reinforced in Jim Highsmith’s book Agile Project Management: Creating Innovative Product.
zarfman posted on Friday, March 16, 2012 11:01 PM

Greeting;

Well after all the sound and fury has anything changed,any problems been solved any conclusions arrived at?

I'm reminded of an old joke.

Several philosophers were debating how many teeth does a horse have? One neophyte suggested they actually go find a horse and count its teeth. He was thrown out of the society.

Remind me again why I should hire you?

Regards,

Zarfman
shaneh@softed.com posted on Saturday, March 17, 2012 6:11 AM
@Zarfman - Ouch :-)

I know why I'd hire the group who are contributing to the discussions - they are providing us (the Agile Extension team) with really valuable feedback and helping us to improve that work product, for which my heartfelt thanks!

My personal perspective is that "Agile" is more a state of mind and a collaborative teamwork philosophy than a set of practices or processes. There are many practices which can be used in many different ways in many different "methodologies" and there is no one Agile way. The selection of a set of practices and processes should be based around the context of the project, team and organisation.

The practices and techniques we include in the Agile Extension are intended to be a set that we (the Extension team) and other practitioners have found to be useful in environments where an Agile perspective is being applied. We do generally feel that working in an Agile way will be more effective than working in a non-Agile way, based around our own experiences and studies.

We do all feel that the way of working is not a binary decision, but much more of a continuum, and finding the right terms for the ends of the continuum has been a challenge. "Agile" is possibly a reasonable placeholder for one end, but the best term for the other extreme still escapes me - @bobsavage's term "Safety" doesn't feel quite right to me, as in a safety critical system I would still prefer that the team had a collaborative Agile philosophy about working together. I definitely don't like "change-driven" vs "plan driven" as that implies that planning is somehow bad, which it is not.

Perhaps it's a symptom of the age-old requirements quandary that Barry Boehm identified in the mid-80's: IKIWISI - I don't know what I want but I'll Know It When I See It. :-)

I do feel that an Iterative and Incremental approach is most frequently the best way to build software, and most of the branded Agile approaches do embody iterative and incremental software development. (Feature Driven Development is one of the branded Agile methods that doesn't use iterations, but does recommend incremental development of small chunks of work).

In the Extension our intent is not to align with any specific branded Agile method, and this is something we will be looking at in the revision process. Scrum is one of the brands and has contributed a lot to the lexicon used when discussing the commonly used Agile practices and processes. We will be looking at how we express the ideas without linking them to specific brands (and if that is actually appropriate or if we should simply acknowledge the source of the terminology).

So - again, my thanks to everybody is contributing to the discussion, and I would happily employ you on my team ;-)

All the best

Shane
bobsavage posted on Saturday, March 17, 2012 12:31 PM
@Shane - Good luck on the next steps for the Extension.

On use of the word "Safety": I understand your point, but I still think the reason one formalizes processes and adds to the required documentation is to reduce risks. Sometimes the risk amounts to wasted money, but the most extreme version of risk is clearly endangerment of human health, especially for large numbers of people (thus my nuclear power plant example). If someone DOES come up with a better term than "Safety", I would be happy to hear it.

"The selection of a set of practices and processes should be based around the context of the project, team and organisation. "

I agree, but would add to that 'the phase of development'. For example, all of the Use Cases don't need to be completed ("filled") before work can begin on design, but there is still a waterfall-like imperative, because the design can only practically begin when the problem has been analyzed sufficiently so that *some* requirements (functional and non-functional) are pretty well nailed down. Knowing which requirements are going to drive design generally requires an overview of close to all the requirements, so the key requirements can be picked out.

A concrete example from my own experience was a project to create a brand new online portal for people to access services. Rather than detail all of the functional requirements, I identified them, but only detailed key functional requirements and key non-functional requirements (particularly security, extensibility, and maintainability). Then the team was able to move on to evaluating alternatives for the platform upon which the portal would be built. Only later, after a choice had been made and implemented, did I return to the other requirements. At that point the project was almost "traditional" in that, from my perspective, I just detailed the rest of the requirements (and a few new ones that were found along the way). Periodically I released completed (validated) requirements to Software Engineering's backlog. I completed requirements development LONG before design and implementation was complete. With the exception of the early iteration to find a suitable platform for later development, the "agility" was barely apparent to me as a BA. (Well, some people will point to the lack of a formal Requirements Document, which is a whole conversation in itself).
rpannone posted on Saturday, March 17, 2012 1:09 PM
I do not mean this as a joke or sarcastically.

If we must come up with a label how about we call it common sense.
zarfman posted on Monday, March 19, 2012 4:55 PM


Greetings one and all:

By the way, my original question still stands; After all the sound and fury has anything changed,any problems been solved any conclusions arrived at?

For many years I’ve posed the question how can a BA not trained in a specific art or science be expected to conduct a meaningful analysis of a problem domain within that specific art or science. My assertion is the typical BA can’t.

Moreover, I strongly believe that the argument over waterfall, agile etc. is a symptom of my opening comments. Sounds to me like someone is hoping for a paint by numbers solution, or not.

Perhaps the time has come to let the SME do all the analysis. By SME I mean that individual the actually understands the problem and knows how to solve it. Moreover, the basics concepts of data input, processing, DBMS and reports are not that difficult to learn if not already known. Business people are more computer literate now.

Also, most businesses are required by law to adhere to certain business practices. These are spelled in thousands of rules and regulations. One can hope that the SME is aware of these rules and regulations, if not, is smart enough to get help.

Regards,

Zarfman
ajmarkos posted on Tuesday, March 20, 2012 11:50 AM
Zarfman:

Why not let the SME do the Business Analysis? Answer: Because they are not able to. Individuals with an IT background have a stronger analysis bent that average. Yet most BA's with an IT background hate doing real analysis - real, like in proper decompostion and interface analysis. Trying to get the average SME to perform proper analysis just will not work.

Tony
zarfman posted on Tuesday, March 20, 2012 10:12 PM


Greetings A J Marcos:

Zarfman wrote: By SME I mean that individual the actually understands the problem and knows how to solve it.

A J Marcos wrote: Why not let the SME do the Business Analysis? Answer: Because they are not able to.

Zarfman writes: I would assert that an SME that doesn’t understand the problem or how to solve it is not an SME. Perhaps we have a semantics problem with what is an SME and what does a BA do, or not.

Zarfman writes: Let’s assume that you and I are working on a fair value accounting system. If I give you the following; a mockup of the data input screen and tell you how to validate the input; a mockup of the output and tell you how to calculate each item of output: as well as where to find the necessary and sufficient data other than screen input data if required; other performance data e.g. each input data element must be verified within one second or less; I along with internal audit will personally test the product any time you want us to. The rest is up to you, would you be happy with a situation of this nature if why not?

This assumes the subject is fair value accounting and that I’m competent (SME) in the area of fair value accounting. I guess it also assumes that you have little or no interest in accounting or becoming one. It also assumes that I’m very happy to pay your $250.00/hr rate.

Regards,

Zarfman
ajmarkos posted on Wednesday, March 21, 2012 9:41 AM
Zarfman wrote:

I would assert that an SME that doesn’t understand the problem or how to solve it is not an SME.

AJ Markos writes:

Ya can't fire everybody. Really, it has always been my experience that very few SME's, even at the management level, have more than a siloed understanding of their work. Analysis is discovery. Very few SME's have done significant discovery outside of their narrow sphere of responsibility. What's more, few would know how to effectively, systematically do such.

The example you gives appears to be of a narrow scope of analysis project. I have experienced few projects that did not expand across the boundaries of many SME silos. But, for its scope, it appears to be a very complete Business Analysis. Why would that SME need to give this info to a BA? He could just skip the BA and give it to development.

Zarfman, you bring up a very important point: Does the BA have the desire to learn the business at hand? The vast majority of BA's do not have the initiative to systematically learn the essentials of a larger scale segment of a business. But, that said, charging them with such a responsibility will work much better than assinging such job to one or more SMEs.

Tony
rpannone posted on Wednesday, March 21, 2012 10:45 AM
Gaining consensus on the definition of the roles and responsibilities when being agile is a foundational element to a successful adoption.

In the world of iterative and incremental product (system-software) development and being agile the Business Analyst and/or SME becomes a valuable contributor to the Product Owner and other team memebers.

To ensure close collaboration between the team and the Product Owner (a.k.a. the internal or external customer), being agile ensures that the necessary elements are effectively communicated directly to the team without unnecessary documentation.

However, to ensure continuity of information, we know that functional documentation that is adequate and representative of the system-software to be developed is essential.

In a multi-team context, the Business Analyst may be called upon to play the role of Product Owner. He/she then becomes responsible for core components of the product within the various sub-teams.
Only registered users may post comments.


Upcoming Live Webinars



Latest Articles

End-to-End UML: Class Diagram
Oct 19, 2014
0 Comments
About the Series This series of articles is designed to present the set of core UML diagrams in a way that emphasizes the important relationships bet...

Featured Digital Library Resources 

brought to you by enabling practitioners & organizations to achieve their goals using:

Copyright 2006-2014 by Modern Analyst Media LLC