Recently in Agile Development Category

We're already deep in the holiday season, but it's never too late to add a bit of Xmas spirit to your Web Application!

I don't know about you, but I am getting asked more and more to deliver some type of application that has a mobile front end. Of course mobile is one of the big trends nowadays, and there are good reasons for it. In a recent Forrester survey, 75% of decision makers claim that deploying mobile apps has increased their workforce productivity.

The question I seem to get asked often is, how should you implement mobile? To keep it simple, I have found that there are three implementation strategies you can choose from to build your mobile app:

1. Native Applications
2. Mobile Web Applications
3. Hybrid

I was excited to read a story in PM Network profiling our client, SATA International, an airline headquartered in Portugal. The article highlights how SATA - using OutSystems' Agile Platform - built a custom airport operational management system after the packaged software vendor behind their previous application came to them at renewal time presenting a significant price increase.

"Even having this limited experience, the success of our first agile deployment instilled confidence that both the methodology and the technology would effectively support the project delivery."

Once SATA kicked off this project using the Agile Platform, they were able to deliver the application even quicker than expected. Paulo Ornealas told PM Network:

"We actually issued the final release a month early, with an even higher than- promised level of functionality."

"Financially, we were able to realize significant savings in both capital expenditure and operating expense," Mr. Ornelas says. The final solution came in at only 15 percent of the total cost proposed by the vendor, and it was totally tailored to the organization.

IT professionals in the education space have a difficult task facing them - they have to meet the demands of students, teachers and administrative support staff with what typically amounts to a shoestring budget.

Building a Mobile Web Application is the fastest way to get a mobile app in the hands of your users. You get to build an app at a fraction of the cost of building native, it works on multiple devices from day one, and you don't need to go through the appstore publishing process (if you want to find out why Mobile Web Applications are the way to go, check this free eBook).

Today, more than ever, analytics are the keys to success.  One of the areas that is still very hard to measure is the productivity of your application development teams.

While finding out how productive your teams are might seem scary, it is far better than not knowing. In this post I will provide three steps and some resources to help you start down the path to understanding the productivity of your application development teams. Once you have a means to measure and monitor, then you can iteratively improve your application development productivity.

1. Make the case for a metrics program
Measuring just for the sake of measuring isn't very useful. The real benefit of having metrics is gaining insight on what can be improved. Not only that, metrics also help you set objectives, what I like to call ultimate goals, and then focusing your team on achieving this goal.

Some of the best advice I have found is from the analysts at Forrester.  I know that a lot of you might not have access to their research so I apologize in advance.   This article (even if you don't have full access you can read the overview), by Liz Barnett, discusses why metrics for application development are important and gives some good advice on what you need to measure and why automating the measurement process is a good idea.

2. Measure your productivity
From my own research and experience, function points are one of the best mechanisms to measure application size and team productivity. As stated by the International Function Point User Group (IFPUG) using this metric for application development focuses measuring functionality delivered to the business, rather than how big the application is internally, how complex the code is, or what language was used to write the application.

Function points have been around for a long time and are a tried, proven and standardized approach.  There is a wealth of resources on how to measure function points, and I personally liked this PDF from a SoftwareMetrics training course and this 5 step process for counting function points from devdaily.

3. Compare to a baseline
After measuring your productivity, you need to determine how your team performs compared with other teams: poor, average or better than average.  There is good news - you can find benchmark information in numerous places. The one I like best is ISBSG.  They provide function point metrics from across the IT industry and with their data you can compare your results and act accordingly.

Go for it!
So, what are you waiting for?  Get a small team to spend some time coming up to speed on counting function points for your organization or go find a third party resource to count for you.  The investment will be worth it when you can accurately assess your application team's productivity and make decisions based on this knowledge.

At OutSystems we got our customers bootstrapped on this process by measuring their performance with something we called the OutByNumbers project. You can take a look at the results from that study here and see how we automated function point counting. I have to say the results are pretty surprising!

We recently met with Lawrence Wilkes, Director and Principal Consultant with Everware-CBDi, to talk about how IT departments are addressing their legacy systems. 

We will post Lawrence's response to the following three questions over our next few blog posts:

• What are the top three things you've seen people do to successfully address the competing demands of new build vs. legacy modernization with limited resources?

• Is there a role for model-based tools like the Agile Platform in legacy modernization?

• What are the top three mistakes people make when addressing their legacy?

#1   What are the top 3 things you've seen IT do to successfully address the competing demands of new build vs. legacy modernization when faced with limited or shrinking resources?

LW: My experience is enterprises that are really successful tend to buck the trend. They are more radical and don't just "follow the crowd". Rather, there are usually one or two key people who have an architectural vision that breaks new ground - at least from their organization's perspective. Typically they will:
 

1. Combine application modernization and business improvement programs. They get business stakeholders involved in understanding the options for modernizing, both in terms of business needs and the applications that allow a more radical approach to be taken to requirements.  They avoid the knee-jerk reaction of buying a package or simply outsourcing the entire problem because they are seen as expedient options that, however, often results in something that is no better able to respond to change than the legacy it is replacing. Instead, they are keen to search out economies of scale and cost; capitalizing on existing investments in applications where possible, rather than just throwing them away, and break-down organizational and application silos to create solutions that are genuinely more flexible.
   
    
2. Keep business design and architecture in-house; create strong architecture and Centers of Excellence (CoE) roles that are able to exert strong architectural governance over outsourcing and delivery. They do not allow the outsource parties to gain control, nor do they lose knowledge of their business, applications and environments.
   
   
3. They use agile approaches but are not slavishly following the conventional "agile" manifesto - agile needs to be used within an architected, governed framework, not ad hoc. They establish a formal reference model and framework, which allows rapid delivery and governance of products, delivered by an agile process.
   

What steps has your organization taken to successfully address legacy systems while delivering new systems?

In our next post we will address Q.2 and the role of model-based development tools play with legacy modernization.

Lawrence Wilkes is a Director and Principal Consultant at Everware-CBDI. Lawrence is a frequent speaker, author and consultant on best practice in SOA, Application Modernization and Enterprise Architecture. Via CBDI Forum, the Everware-CBDI research capability and portal, Lawrence has led the development of the CBDI-SAE methodology and supporting Knowledgebase, which is used by both end-user organizations and system integrators around the world. Lawrence has an extensive background both within end-user and vendor organizations having worked both in the business and IT side, which brings particular insight into business/IT convergence.

OutSystems is all about Agile - and the same applies to our internal R&D team which has been using Agile since 2002. Because we've been using Agile for so long all the principles are pretty much entrenched in our culture - but as the company and the department grew, we started to feel the need to share our Agile knowledge with newcomers to the team.

We considered several ways of doing this. We pondered doing a traditional PowerPoint presentation; we thought about assembling a mandatory reading list; we entertained the idea of adding support on our tools for some of the Agile tenets; and so on...but in the end we decided to organize a contest around Agile to help new team members learn about the methodology the fun way!

Let's play: Who wants to be an Agilionaire?

The idea is simple. Each week a question about Agile was sent to the entire team. The question was accompanied by 4 possibilities, and the contestants had to pick the right one. Everyone could reply to the question within 24 hours, and the winner would be the person with the most correct answers. You can check the questions & answers here.

This approach worked really well, and had a lot of advantages over any of the other methods - here are a few:

• People were encouraged to look for an answer. This not only ensures people are paying attention to what you're teaching, it also promotes self learning.
• The questions were picked based on the real issues we witnessed internally. This means that people were learning what they needed most, not every detail on Agile.
• We sent a small justification of the answer with references to sites, blogs, and books on Agile - and provided more bibliographical references than we could ever hope to transmit on a single presentation.
• We got an idea of people's knowledge on Agile. Now we are aware of how much people know about Agile (and the results were very good, I might add). We also have an idea of the areas where specific people need more help, and will use this knowledge to help them get up to speed.
• Everybody participated. If we had done a Power-Point presentation, I'm sure we wouldn't have had everybody raising their hands.
• It was really cheap! There was no time invested in preparing the presentation or in changing the internal tools. The only work required was preparing the questions and answers.
• It was fun! Everybody liked this idea, and we got great feedback from the team! It was surely more fun than going through a 2 hour presentation on Agile...

2 months and 8 questions later, we have a WINNER! Miguel Melo had the brilliant result of 7 out of 8 questions right, proving to everyone that he's a true Agilionaire! To celebrate the victory, Miguel was awarded a priceless handmade trophy (built and designed at OutSystems;) an adventure pack to prove he's also Agile out of office; and he gets to show up in the Agility Blog! Congratulations Miguel!

Last week's Agile2009 in Chicago was a pleasant surprise in a couple of ways.  First, in a down economy the conference attracted nearly 1400 attendees from around the globe - which is outstanding considering many conferences organizers we've talked to are reporting a 30% decline in attendance. According to the organizers 60% of the attendees were first-timers at the conference which is a testament to the quality of the event. And, there was a large international contingent - we met lots of Aussies, Kiwis, Brits and Europeans; in fact it felt like 50% of the Agilists we spoke to were from outside the US - and great to meet them all! 

Second, after four days of interacting with other conference goers we noticed two trends that were very different from last year's conference:
 
a) The majority of attendees are now actively practicing Agile and have participated in multiple projects.  Last year it seemed many people were attending in order to learn about Agile in preparation for their first project.  This year, our informal (and unscientific) survey showed 53% have been personally involved in over 4 Agile projects.

b) Sprint duration among participants has dramatically reduced from four weeks last year - to two week sprints this year.  During one of the conference sessions this year "Death by SCRUM Meetings," a session attendee told us he estimated the majority of the audience who were polled on the average length of their sprints said two weeks.  This prompted the OutSystems team to do an informal survey of the people we talked to, which confirmed the number - two weeks is the norm for sprint durations.

Agile is definitely becoming main stream. Alistair Cockburn's keynote "I Come to Bury Agile, Not to Praise It" made this point - the days of small co-located teams following the Agile Manifesto are dead and gone.  Today we have very large distributed teams tackling big problems which are changing the face of Agile as it was originally conceived.  The term Agile now encompasses so much more than just doing things faster - Agile is dead, long live agile.


I Come to Bury Agile, Not to Praise It - Alistair Cockburn
Main Take Away: Agile has grown up and no longer in the domain of the few; it has now received the attention of large scale enterprise and evolving faster in terms of maturity.  It is taking on a life of its own.  The speaker noted that he observed this to be true because Agilists are now in the "ri" stage of the "shu-ha-ri" concept.  "ri" being where an individual is now capable of inventing and blending concepts and into new approaches but still within the "Agile" umbrella.  ("shu" - learning a methodology, "ha" - collecting techniques, "ri" - inventing and blending techniques).

Introduction to SCRUM - Henrik Kniberg
Main Take Away: Scrum seems to be the most popular of the agile methods because of its ability to incorporate any technique or tool in its use. Interesting quote for us agile & tools people "Do not develop an attachment to any one weapon or any school of fighting." Miyamoto Musashi, 17th Century Samurai.

WANTED: Seeking Single Agile Knowledge Development Tool-set, Brad Appleton, Peter Alfvin  Main Take Away: Companies in very complex environments developing embedded applications, corporate applications and everything in between will continue to struggle with disparate tools.  

Agile in the Enterprise Corporation, Panel: Israel Gat, Stephen Williams, Laureen Knudsen, Scott Killen  Main Take Away: Why Agile? It's about the money!  Do feature budgeting.  Develop and maintain a release model in a business framework.  Get the organization on board within the dynamics of your specific company - collaborate.  Re-organize to Operational and Executive teams that are cross-functional and make their interaction with R&D asynchronous.

Marriott's Agile Turnaround - Jesse Fewell Main Take Away:"Agile can't fix bad strategy"
                          
First, Kill All the Metrics - Niel Nickolaisen , Chris Matts
Main Take Away: Reward and punishment results in meaningless metrics and thus not all metrics we use are meaningful.  For a metric to be meaningful, it has to measure process and not people; they need to be aligned with the objectives and strategy of the organization; and should show trends.  Metrics that measure people tend to result in unhealthy competition between team members especially in a self-regulating / high performing team.

Strategies in Replacing Systems in Agile Projects - Niklas Bjornerstedt
Main Take Away: Presentation discussed a number of patterns in replacing systems and recognizing these patterns can help in the process.  Although there are patterns that can be used to help in transitioning users, migrating data, etc.; it still boils down to a case by case basis.

How to evolve a Product Backlog - Ronica Roth, Mark Kilby
Main Take Away: The main discussion centered on following the user work flow to evolve the product backlog.  Along with that, ask questions on the 'what' and the 'why' primarily.  The 'how' questions need to focus more on the current processes and not necessarily on the future although it would be good to note that.

Agile Metrics - Dan Rawsthorne
Main Take Away: The talk was directed to organizations that are new to agile.  One of the key points is to level-set on metric-related terminology like what "done" means so that there is no confusion between team and customer.  A suggestion was put forth to create a "doneness criteria" which essentially is a checklist.  Other terms were velocity and capacity that accordingly most folks new to agile get confused on.  The speaker suggested that velocity = capacity until proven otherwise.  Capacity is what you estimate you can do and velocity is what you have proven you can do.

Don't think about it - what's the one word that springs to mind when you think of "Agile" in terms of application development? Write it down somewhere.

A recent LinkedIn question asked for advice on how to get a company started down a path to Agile and SCRUM. I thought it might be useful to share some advice and see if the OutSystems community had any ideas to add.

At OutSystems, our roadmap is strictly a guideline because companies have different methods, cultures, and management approaches. When introducing Agile, here are the steps we generally go through. Concepts and activities that are emphasized will vary based on understanding of the customer culture, organizational structure (formal and informal), as well as their prior knowledge of Agile methodologies.

1. Education - first, level-set those involved in the roll out. We found that there are many different levels of understanding of what Agile is even if people say they "get it". Part of this process is to make them actually go through the process before the project begins. At OutSystems, we have "Agile-in-a-Day" training sessions that provide participants with a hands-on introduction to help develop real understanding of key concepts and activities. If you have a project already selected, involve the business sponsor, business manager, and key business users.

2. Project Definition / Selection - Once everyone is clear on the vision and direction, a project can get started. The project may have been identified earlier but how the project is chosen or what criteria was used needs to be understood. Since this is the first Agile project, we need to make sure that risks at this level are addressed to help ensure success for both project and process. The criteria for selecting the project needs to include the solution's level of complexity, visibility, resources, and integrations.

3. Execute Project - Now we start the project by educating the target users if they were not involved in the initial Agile-in-a-day. Go through the project kickoff; explain the methodology, roles, responsibilities, timeline, deliverables, etc.; fairly standard project stuff and along the way - expunge the word "scope" from all project documents, thoughts, and discussions. Work on the backlog, feature negotiations, the sprints, scrum meetings, demos, etc. Once a sprint is done, do a retrospective and refine the processes for the next sprint and the next project. Once the solution is in production, conduct a tuning sprint. This is a special sprint we do at OutSystems to ensure 100% adoption by implementing features that will boost adoption and conducting both solution performance & platform tuning in the production environment.

4. Perform Project Retrospective - apply lessons learned to subsequent projects and refine other processes. Note that this process improvement will involve Support organizations and dynamics between project teams. One of the things we often encounter is that Support organizations cannot move as fast as the project sprints and tend to delay Agile projects. Similarly, non-Agile projects have a difficult time addressing the integrations with Agile projects. As you execute your first project, you will find that you may need to bend or even break some rules to keep your project on track as defined by your timebox. Therefore at the end of the project, you will need to work with Support and other internal organizations to establish new protocols or processes specific to Agile projects.

5. Iterate Steps 1-4 - In our experience, we found it necessary to conduct multiple Agile-in-a-Day sessions to get everyone in the company level-set on the organization's approach to Agile. Agile is a mindset change; expect hurdles and naysayers. Besides, change is always difficult even if the participants are willing, able, and have executive sponsorship.

(Unofficial #6 - Be realistic and good luck)

What do you think of this roadmap? How have you introduced Agile into your companies?

Miguel and I were recently discussing an interesting presentation by Martin Fowler and Rebecca Parsons on engaging the Architecture team into your Agile methodology.  The presentation is titled "Agilists and Architects: Allies not Adversaries" and the summary points out that "As Agile becomes more accepted, concerns from architecture groups are increasing. Traditional ways that architects engage with development groups conflict with Agile methods."
 
With architecture being a core component of the OutSystems approach to Agile application development and the role of Delivery Manager, we thought it would be interesting to share this presentation and ask the OutSystems Community how you have embraced architecture while practicing Agile in an Enterprise setting. 

For example, many of you are successfully delivering a high degree of reuse while focusing on delivering individual applications.  While OutSystems' Agile Platform provides a solid foundation for refactoring code and adjusting architecture in progress, we suspect that there are many instances where your Corporate Architecture team needed to play a role in moving the application delivery forward. 

So, if you are a Delivery Manager or Architect it is time to share your experiences of working together in an Agile application development project.  How much education was required?  Did you feel resistance? What were the lessons learned?

"Scrum brings the customer and the development team together as often as every sprint.  Because the customer is involved throughout the development process and is consulted once per iteration, the team can never deviate very far from the customer's vision." says Laszlo Szalvay in his recent Better Software Magazine article on SCRUM in which the author does an excellent job of providing an overview of SCRUM.

Personally, I couldn't agree more and I think he nails this key point about SCRUM. I suspect that most of you will be aligned with this too.

As an application development practitioner from the "old days" for me, iterative development and continuous business user involvement is the ultimate Agile truth - it's the best way to keep projects on schedule and deliver a high value solution.  

But now for the real question about customer involvement:  What happens when you don't have the right customers involved?  

Last week I was discussing an Agile project with one of our OutSystems Engagement Managers and he described this exact issue. The team was very excited to tackle this project using Agile methodology as it was going to be the first for this particular client's business area.  The business was on board, trained on Agile and they had the full support of the management who were committed to participating in planning user stories, each sprint demo, backlog settlement, etc.  

The problem was that the supervising managers were not the ones who would use the application on a daily basis and were somewhat out of touch with the processes and issues on the ground. What happened? The management team met with resistance from the guys in the trenches about the initial app - and then what? ...they changed their minds about the functionality. Ultimately this proved to be a positive experience as the Agile and SCRUM-based approach they used caught the mismatch early and the resulting app was accepted.   So, the lesson here : beware - management participation is critical but you also need the real end users to make sure you don't get too far off track!   

Have you come across this type of issue on your Agile projects? How is Agile helping your projects deliver better software?

"The uncertainty of outcome is the only certainty" This should ring true to every Agilista out there.  Let me give you a minute to think it through.  When was the last time you worked on an application development project where the outcome was certain?

In my 25 years in the IT industry the outcome has never been a given.  Oh, I tried ... we did formal analysis, with Information Strategy Plans, Entity Relationship Models, Process Models, Use Cases, Flow Diagrams, Class Diagrams and the list goes on.  But in the end, it only got us so far.  The reality was that things changed, we got stuff wrong and the result was not what was expected. 

I recently ran across a couple of great Agile techniques (with crazy acronyms) to help remedy all this outcome uncertainty that I wanted to share:

1. IWKIWISI - "I Will Know It When I See It."  For many of us this is a reality.  We have some general ideas as to what we want but will not be able to articulate it exactly. Agile's "deliver early and often" is key to addressing this issue.

2. GrASBI - "The Great Axiom of Software and Business Indeterminacy".  This term was defined in a recent CIO magazine blog by Michael Hugos titled "Agile Techniques, Agile Hype and the Essence of Agile (in IT and Business.)"  The key idea is that having a general idea of what you will deliver will get you close to the expected result, and that having detailed definitions of what you will deliver will not get you any closer.  The challenge is, of course, getting the business to change habits and recognize this truth.

We all grapple with shifting the traditional development mentality (both IT and the business) to one based on Agile concepts.  One of the hardest behaviors to overcome is this new way of defining direction and refining as we go.  So, let's overcome GrASBI, empower our application of IWKIWISI and get Agile!  

There seems to be a growing amount of discussion around the Kanban approach to software development.  For those of you who, like me, had never heard of Kanban, it has its roots in Japanese lean manufacturing concepts.  I read a great article titled "Kanban Development Oversimplified" by Jeff Patton.  If you are interested in the Kanban way to organize your development then give this a quick read - it is well written and starts off with a good look at the origins of the Agile methodology. 

From my perspective, having now done a little research into Kanban, the approach is very similar to something I practiced back in 2002 while looking after storage products for BMC Software.  In my opinion, Kanban is a great process for software product companies who want to drive efficiency across their product delivery process.  However, it is not ideal for corporate IT shops who are working on improving their application development processes and better alignment with the business.
 
Even though I'm an Agilista at heart, I believe we can learn from the Kanban approach, however, there are some things that just don't seem like a good practice.  For example:

• Lack of Iteration: Kanban doesn't include an iterative approach - which I feel is one of the key Agile tenets that delivers real benefit.  It allows us to continuously align with the business and deliver exactly what the business wants - regardless of the set of requirements we started with.  Invaluable in my opinion.

• No commitment to deliver: Without iteration and time boxed sprints you never have the opportunity to set expectations and then meet them - 'deliver early and often' as we like to say.  This is critical to drive business trust and Agile adoption. 

• Writing Larger Stories:  Kanban proposes writing larger stories around the concept of a minimal marketing feature (MMF).  Once again, a concept that makes a lot of sense for software product companies in the commercial software world.  In my experience, writing larger stories results in longer development cycles for a feature, resulting in the opportunity to wander off the path before a user sees what you have done.  Historically we have never managed to get  the MMFs correctly defined, which results in rework and extra costs.  The approach of developing some of the functionality and then getting feedback is a hallmark of Agile and leads to better business alignment by allowing the team to react to change from the business.  This works better with smaller, role based stories.

Even with these shortcomings, Kanban's lean manufacturing approach is interesting as it should really help a team identify process bottlenecks, and I suspect could be applied in a more Agile manner of development. 

I am sure you all will have lots more comparisons and thoughts on how Kanban could make your Agile approach better. Let us know what you think! 

So, SAP is embracing Agile development for their new SaaS strategy (see Phil Wainewright's post on the recent SAP announcements); About time don't you think?  Most ISVs are way ahead of the hype around Agile and have been applying some form of Agile development since the beginning of this century. Now, for those of you working in corporate IT shops who think SAP's move is a great endorsement of Agile methodology and are going to run to your management shouting "the time is now!" ...let me caution you on a few key points where you might run into some bumps. 
 
1. You probably don't have an ongoing product-based budget. This means that you can't simply keep on developing until the customer is satisfied.  In your corporate IT world you will still have to get funding for each project, which means some form of project scoping and estimating. If this is done the way it has been in the past it will not be very Agile! Also, if you really embrace Agile, those features that get pushed out of scope by the business (due to more important features) will have to be put into another project which will require more scoping, estimating, funding and perhaps even a bid process with your third party contractors - NOT very agile.  All this can defeat the benefit of Agile and leave your business with the Agile blues. (I discussed this issue in a previous About Agility Blog post)

2. You probably don't have a staff of full time Product Managers.  Of course you don't have product managers in a corporate IT shop.  This role is supported by the business users. Thus, you have to work really hard to change corporate culture and get these business users to play in an Agile way.  This will require more of their time, more trust and a shift of application ownership during development, that for many is a real culture shock.  Be prepared.  The good news is within a few sprints you should have them on board as long as you keep your sprints short (2 weeks max) and hit your sprint demo deliverables.  Once they get a good taste of Agile they will not go back!
 
3. Your self-organizing team isn't so well organized. The challenge many of you in the corporate IT world face is a lack of on-staff resources.  Thus, a lot of applications work is co-developed or outsourced to third parties.  This makes the concept of self organizing teams a bit risky as it introduces lots of politics, negotiations, etc.  Once again, not very Agile.  You will have to find the right contractors and get your procurement organization to let them participate in an Agile manner so they don't risk losing their shirt or end up using non-Agile practices. 
 
So, while I am a big proponent of Agile, to be successful in the corporate IT world requires a lot of culture change!  And while SAP may go Agile in one big push, for the rest of you, I say "be Agile - take it one project at a time, think big but start small and learn with your business as you go."  

Calling all corporate IT shops - have you adopted Agile practices? Have you encountered these issues? How have you dealt with them?

This was a great session by John Rymer and Dave West.  They provided some interesting stats:
•    80-90% of IT spend is on software maintenance
•    Business are demanding innovation
•    Time to market is more critical now than ever before.

These three stats should be staring you right in the face - How can you be more innovative in shorter time frames than ever when 80-90% of your budget is spent on maintenance? 

A key message from this session is that software bloat is the biggest issue.  They pointed to five causes for software bloat:

1.    The sheer number of features we attempt to add to an application
2.    Specification and protocol stacks continue to grow
3.    Vendor consolidation leads to large suites which you have to purchase entirely
4.    Middleware continues to get more and more complex
5.    Traditional development processes lead to overly complex applications

Of course the key message is that lean software is the antidote to software bloat.  Agile methods can help us to focus on only the high priority features resulting in leaner software.  The key challenge is how to get your business to support an Agile approach and come to terms with the notion that, in many cases, it is more cost effective to not automate processes that seldom occur - they are better handled manually keeping your applications lean and focused on the key business processes.
 
Feel free to share examples where your business has embraced Agile and successfully de-prioritized application functionality as too low to implement.
  

Last week, I attended Forrester's IT Forum in Las Vegas, Nevada, USA.  In looking back over my notes I wanted to share some of the highlights.  The first keynote started off with the resounding words:  "Never Waste a Good Crisis".  Bobby Cameron, Vice President, Principal Analyst, Forrester  expanded on this notion it appeared that the main point is that during a crisis change tends to be accelerated which should allow those savvy IT organizations to address these key challenges:

•    How do I get leaner and reduce IT waste?
•    How do I protect and promote innovation?
•    How do I reshape my own role?
•    How do I measure and communicate value?

As I listened to the discussion and many of the sessions to follow, I learned that many of the analysts and attendees were becoming aligned with the idea that an Agile approach to application development will help on many of these fronts.  First, it will help IT get leaner and reduce waste by only delivering what the business needs while helping IT with business communication or alignment.  It was also pointed out that if organizations really embrace Agile and let teams be self directed, we should even break down some of the inefficiencies of big, inflexible architecture and allow teams to select the development tools, application stacks, etc to get their jobs done quickly with minimal cost / waste.  What a novel idea!

Bobby also discussed the importance of IT leaders really paying attention to IT Value.  The key point is that value is always perceived.  Some people see value in driving a Mercedes while others see it as a waste of money.  Thus for IT professionals we must make sure that the business sees value in what we deliver.  Thus, we must become more agile and deliver on business objectives in a way that facilitates communication and measures business value - the last key challenge in the list above.  As we all know, an Agile application development approach will dramatically lead to this outcome. 

I found it interesting that Forrester's analysis is showing that many traditional IT roles are being moved into the business - some of the examples given were project management and business analysis roles.  

I am interested to hear if any of you are shifting some roles out of IT and into the business.

Ana Paula of Whatever Consulting spoke this afternoon on "OutPractices: Agile Transformations of any Software Development Process". She talked to us about a subject close to her heart - the Eclipse Process Framework (EPF) which aims at producing a customizable software process engineering framework, with exemplary process content and tools, supporting a broad variety of project types and development styles. Ana spoke in particular on how OutSystems customers can leverage the EPF library to further develop their in-house processes especially when they need to include the use of other non-OutSystems processes or methods.

Our next presentation was by Stefan Meier, Associate Director, Software Development, Information Sciences Department at XDx. Xdx is a biotech company that relies on software and technology for their Molecular Diagnostic Technology. They assess patients who have had a heart transplant and whether they are likely to accept or reject the transplant. They are also working on a clinical study to provide the data to find correlations between genetics and pathology in disease/immune system-related areas.

Miguel Barreto, CEO of Home Energy spoke to us next on "Launching a New Company in a Fast and Agile Way" Home Energy is a new company that's part of the Martifer, a Portuguese renewable group. Home Energy was established in May 2008 with a goal of being a market leader in a new space opened up by new regulations related to Energy Certification and Microgeneration (small photovoltaic panels for home use) for the domestic housing market. January 2009 was when the market was going to take off and Home Energy needed to be ready to support a 200,000 housing market.

Next up at NextStep - Marteijn Mout who is the project managers of the 'New meter market model' at E.ON Benelux. E.ON is one of Europe's largest Energy providers with over 30 million customers in 30 countries.  E.ON Benelux has around 270,000 customers and they have been a customer of OutSystems since 2004.

In this NextStep session Pedro Rosa Santos who leads the consulting services team for Keep it Simple, an OutSystems Partner, addressed how Dynamic Business Applications can play a big part in the Insurance industry.

Pedro described the quantum leap experienced in the Insurance industry from 2005-2008 as it tried to deal with market demand for new products, online self service, the need for real-time information exchange with agent & broker channels and the drive to invest in new technology in order to reduce costs. The problem, Pedro explains, is that this has created an extremely complex IT environment for most Insurance companies around the globe - and the goal of building an infrastructure that is responsive to market needs - actually started to have the opposite result - preventing responsiveness!  

The "Agile Delivery Experiences" track was kicked off by Rosa Mimoso, Director of Global & Support Systems of Caixa Seguros.  Rosa shared how Caixa Seguros e Saude,Portugal's largest insurance group used Service Oriented Architecture (SOA) and OutSystems Agile technologies to address their extremely large and complex business and technology environment (millions of customers & claims, 4 companies, 5 brands and hundreds of legacy apps.)  

Mike Jones and Carlos Alves of OutSystems got the audience's brains thinking with a discussion on "Changing The economics of Application Development" with a set of questions:

"Can a company really innovate in today's economic environment?" - they think "yes", and gave examples of how new ideas are coming up from the business, they are small but they are making a significant difference to business success - however, they may need custom built software (not packages) due to the need for speed and flexibility.

*"Is it all doom and gloom?" *- the guys argued that NOW is the right time to innovate and get Agile. They challenged IT teams to be ready for whatever comes next with the economy -and suggested that the only way for the Audience and their businesses to not only survive but thrive, is to get more Agile.

"WTBWITD?" - Mike joked with the audience that this wasn't really easy to pronounce and needed some marketing spin - but "What The Business Wants IT Delivers" Carlos said, is the essence of Agile which is basically "a systematic approach to accommodate innovation"

"What is Lean?" - the process to making sure IT delivers what the business needs - Agile is the secret to becoming Lean.

"What is Agile and how is OutSystems different?" -Industry agile projects: iterative development with 4-6 week sprints, continuous business participation and deliver vertically integrated solutions at the end of each sprint.

-The OutSystems Agile Approach: able to pre-size the project with high accuracy; time-box principle, smaller up-front analysis and more prototyping; 2 week sprints that deliver vertically integrated solutions at the end of EACH DAY. Leverage SCRUM practices.

A great question came from the audience about how to become MORE agile when your team is on the other side of the world and difficult to reach - they are doing 3 month sprints...which is not Agile, its waterfall.

Answers from the OutSystems team: The first issue is likely to be the lack of commitment from the business side to attend the regular weekly sprints. The second is that they need to have a business sponsor with authority who attends the regular sprints. Final suggestion - to ensure that an Engagement manager is local to the business-team, and that the Delivery Manager and the Engagement Manager are in constant communication.

Wrapping up, Mike and Carlos challenged the audience on "What role will you play in today's economy? You have two choices: put your head in your hands and hope it all goes away or take the opportunity to embrace change and innovate."

Wise words that we can probably apply to all our lives in these uncertain times.

Evergreen applications are business applications that continue to evolve as the business changes and so provide constant value, year after year. At OutSystems virtually any solution that is deployed using Agile methods is required to stay evergreen and subject to a continuous Agile Evolutionary Maintenance process. Part 1 of this article explored the main phases of the Agile Evolutionary Maintenance process and defined the three key project types that are most common.

In Part 2, I will shift my attention to defining the different types of backlog items that typically make up an evolutionary maintenance sprint. In addition, I will define some key technology challenges you must overcome in order to deliver on the promise of evergreen applications. Finally, I will describe how OutSystems can help in this process.

Backlog Items of Evolutionary Maintenance

The typical backlog items that are part of an evolutionary maintenance release include:

1. Defects - fixes to application defects and business processes that are not correct for real life conditions, and therefore impact daily business operations;
2. Application inefficiencies - over time, applications composed of multiple services will experiences changes in performance profiles due to changing business use and shifting service load. It's key to identify and overcome performance inefficiencies such as:
   
   • Slow Web pages;
   • Slow database access;
   • Integration bottlenecks in distributed services architecture.
3. Usability improvements - overcome annoyances that impact user efficiency or adoption;
4. Change Management - Incrementally deliver new valuable features. Either selected from ones left out of previous releases or newly identified during the use of the live application.

 

Challenges of Implementing Agile Evolutionary Maintenance

To achieve an effective Evolutionary Maintenance process it is imperative that the maintenance team has the right tools. They need access to development and project management tools which:

• Allow them to assess application performance;
• Assist with capture and prioritization of change requests;
• Streamline the change, test and deployment process.

Without the proper application delivery and management environment your maintenance teams will not be able to deliver the agility needed to keep your applications evergreen. Let's explore these key challenges in a little more detail: 

1. Automated instrumentation of the application that doesn't require expensive operational support, extra coding, etc. This is important in order to understand where to invest in code optimization when dealing with application performance issues. Areas on which to focus attention:
   
   • Identification of the application web pages visited, their rendering time and isolation of slow screens that should be reengineered;
   • Identification of slow queries that may require optimization;
   • Performance assessment of the web services and APIs being used by the application to provide access to external systems;
   • Review of application error logs;
   • Centralized analysis of all application feedback including performance logs, audit logs, etc.


2. Enhanced capabilities for end-user collaboration to gather feedback on application usage. This capability is critical in order to address incorrect application behavior and improve usability and adoption. Areas on which to focus attention:
   • Provide the end-user with the means to give feedback on the application behavior. This process should be intuitive for users and result in unambiguous feedback for the maintenance team;
   • Streamline feedback management to effectively classify, prioritize and transform the user feedback into manageable backlog items.


3. The availability of enabling technology which supports rapid change and boosts delivery efficiency. Areas on which to focus attention:
   • Provide adequate application documentation when planning for evergreen applications. This is because the person performing maintenance tasks is typically not the same as the one who developed the original application; therefore it is critically important that the application's structure and logic is easy to understand;
   • The development tools must make it easy to respond to change by supporting the change process. In our experience this includes tools that automate impact analysis and isolate change , which in turn, minimize testing and improve overall development team productivity;
   • The application configuration and build environment must be automated to remove errors and support a rapid deployment of changes to test/production environments.

 

How OutSystems Helps with Going Evergreen

OutSystems' Agile Platform and project management tools provide the technology needed to shorten delivery cycles, increase software development agility, project predictability, responsiveness to business change and overall development productivity. For Evolutionary Maintenance teams, OutSystems technology addresses the key challenges faced by maintenance teams when going Evergreen:

• The Agile Platform automates the logic required to collect performance data and audit information. In addition, it supports the automatic logging required to address performance tuning, including monitoring third party integration done via the platforms Integration Studio toolkit;


• The Agile Platform's Service Center tool provides the necessary functionality to analyze performance data and audit logs collected from the application during run-time;


• The Embedded Change Technology (ECT) and the Agile Network's Issue Management tool ensure end-user collaboration :
  • ECT provides an easy way to collect application feedback from testers, end users, etc. ECT provides meaningful information based on feedback directly from the application screen, and all feedback is automatically registered and stored in the Issues Management tool for the project;
  • The Issue Management tool handles end-to-end issue management, from capturing user feedback, to issue classification and prioritization. Issues can then be included in the maintenance release backlog for delivery.


• The Agile Platform's TrueChange™ engine supports automated impact analysis and self-healing. This streamlines the change process and assures that quality applications are delivered. Self-healing is unique to the Agile Platform and is made possible by the fact that development is done in a model-based environment. This allows the platform to understand and manage all changes across all the development artifacts. For example, a simple change request to "add a new field to a Web page form that is only visible to a Sales Manager" might require changes to the User Interface, Data Model, external Service APIs, Access Control rules and Business Rules. With its full reference checking and self-healing capabilities, TrueChange™ safely rebuilds the sections of the application that can be automatically inferred, and provides the Business Developer with impact analysis on any manual changes he or she may still need to make to fully address the requirement;


• The Agile Platform's 1-Click Publishing streamlines the configuration management, build and deployment process - effectively reducing common build errors and the typical build and deploy time to minutes.

 

An Evolutionary approach to Maintenance is key to keeping your applications evergreen and delivering the best business ROI. As you expand your teams' Agile processes, you can apply the Evolutionary Maintenance phases defined in Part 1 of this article across a variety of different project types and architectures. To be successful you will need to overcome the technology challenges described in Part 2.

Be sure to review how OutSystems can help for your web business applications!

Evergreen applications are business applications that continue to evolve as the business changes and therefore provide constant value year after year. At OutSystems, virtually any solution that is deployed using Agile methodologies is required to stay evergreen and subject to a continuous Agile Evolutionary Maintenance process. Part 1 of this article explores the main phases of Agile Evolutionary Maintenance while Part 2 identifies some key technology challenges you must overcome to deliver on the promise of Evergreen Applications.

Setting the Stage

Once your application goes live, it immediately enters the Agile Evolutionary Maintenance phase and will stay in this phase in order to keep up with changing business needs and to address application inefficiencies that are only identified with "real life" demand and conditions. To successfully deliver on this concept, the Agile Maintenance team schedules regular sprint releases focused on delivering the "next most valuable features" for the business.

Through this sequence of regular Evolutionary Maintenance releases, the application will keep up with business needs while minimizing the impact of change on the business. The underlying principle is the same as for general Agile development - fast deployment of the most valuable features, delivered on time and aligned with the business.

The Agile Evolutionary Maintenance Cycle

The Agile Evolutionary Maintenance process is a continuous cycle of application releases. Each cycle includes the following steps:

1. Gather defects, user feedback and new business needs for the live application. This step is actually
   a continuous process that normally occurs in parallel with the applications' release process;
2. Classify and prioritize all the feedback collected;
3. Convert the classified feedback into project backlog items, agreeing on the next release content.
   The focus should be on the most critical defects and highest value change requests;
4. Develop and deploy the Backlog agreed for the next release.

 

Evolutionary Maintenance releases depend on the volume and type of change required. In my experience these releases fall into one of the following three categories:

• Hot Fixes - quick intervention required to fix a critical defect that is dramatically affecting business operation, and deploy it immediately. These interventions are required to be completed in less than 24 hours and in some cases during the next couple of business hours.
• Sprint Releases - made up of one sprint, lasting one or two weeks. In Agile Evolutionary Maintenance it's common to set regular short releases to keep answering accumulated user feedback and correct non critical defects.
• Evolutionary Projects - When there is a more substantial business change, then a specific evolutionary maintenance cycle may be extended to a small/medium project with several 2 week sprints. This is a common approach to deploy a very large application: incrementally release extra functionality that has not made it into the initial agile project. Experience has proven time and time again that waiting for the perfect and complete solution leads to a less than expected return on investment due to constant changes in business scope and delayed benefit from the new application features.

The Agile Evolutionary Maintenance cycle is critical to keeping your applications evergreen. Following this approach will dramatically improve overall return on investment over the life of your application.

In Part 2 of this article I will identify some key technology challenges and how you can overcome them to deliver on the promise of evergreen applications.

What happens when your application software change cycle time shrinks from months to hours or days? Over the past four years, we have overseen the deployment of hundreds of Web business applications all following agile methods. During the course of these projects, we have faced many challenges and found some surprising benefits.

This article describes some of the lessons we have learned and provides advice on how you might overcome some key challenges in your own agile projects.

Introduction

In 2001 when I assembled a team of seasoned Web professionals, we looked back at our frustrated past of trying to release complex Web applications in Extranet and Intranet environments. While we were always able to deliver these projects, the process was painful.

The review of our past performance showed that it was impossible to accurately analyze and document the real business requirements. Following a traditional waterfall method meant that projects were doomed to be late and over budget. We were plagued by changing requirements, unanticipated integration challenges, usability annoyances and ultimately the formidable task of training thousands of users in a short amount of time.

So we decided to look at the problem from a radically different point of view where we embrace and nourish change. Fundamental to this shift were two key project assumptions:

1) Assume that the application scope can never be accurately defined.

2) Let the application scope evolve during the project based on user feedback.

Agile methods align nicely with this approach. When complemented with tools that allowed us to time-compress the change and upgrade processes, we were on our way to dramatically improving our ability to deliver complex Web business applications.

Fast forward to 2008 and 500+ projects later. Today, we have successfully adopted an extreme change mantra where we constantly enhance and re-factor applications as we drive toward meeting the real business requirements.

Agile - A Mantra for Extreme Change

Our mantra of extreme change is well served by agile methods. Over the years we have found that embracing extreme change has helped us meet our project milestones and deliver applications that are aligned with business needs and easily rolled out to large user bases. We have also found that business users actually make pretty good testers and that incorporating them into the process early and often results in a better end product.

On the flip side we have faced several unforeseen challenges. Collecting feedback from end users early and often resulted in a need to find a better way to manage and prioritize the feedback. In particular, help desks and documentation present a big challenge with embracing extreme change and must be addressed up front or you will suffer when you roll-out your application.

Over time, we've developed a tried-and-true Agile approach to help overcome these challenges and continue to work on improving our agile skills and ability to deliver dynamic applications that support extreme change.

Let's look at some of the lessons we have learned.

Listen to Users at Large

Over time, we formalized our mantra of extreme change around an agile method and delivery platform based on Scrum. In the process, we learned how difficult it is to capture timely feedback from a large user base through the help desk and one-to-one conversations.

About two years ago, we started experimenting with embedding feedback functionality in the applications themselves. We added a non-intrusive hovering button to the bottom of every application page that can be clicked to expand into a feedback form. The user points to the relevant area of the page, types the feedback and submits it. The page contents, user name and session information are collected automatically, packaged into a Change Request and, sent to the agile delivery team. The results have been staggering. In one case where an application was launched to 25 users, we collected more than 200 suggestions, bugs and comments in two days. Out of this 200+, 10 were identified as critical by the agile team (IT and the business) and they were implemented in three days. This tuning process skyrocketed adoption of the application and allowed a smooth roll-out for the remaining 1,100 users.

Transform Business Users into Testers

One of the great side benefits of having your users so heavily involved is that they make excellent testers. That is why we make efforts to get more business users participating in the early betas delivered at the end of each sprint. Taking this approach we can spot usability inefficiencies, functional errors and optimize the experience for first time users early in the process. The end result is that final acceptance testing and QA sign-off is simplified because we eliminate the delivery bottle neck.

Prepare for Life after the "Go Live"

Rapidly responding to change is especially valuable in those days after launch where the business users get to use the application and experience the harsh reality of a new way of working. It is important that the project team stay around so they can quickly tune any remaining usability and performance issues. That is why we have restructured our typical project schedule to include a Tuning sprint after the application has been rolled out into production.

The impact on the team is substantial. First, you need to prepare the Product Owners and Business Stakeholders for the influx of post-launch work resulting from business user's first impressions and feedback. This feedback is valuable because it helps increase the usability of the application and decrease the need for a help desk or user manuals. But it requires an extra effort managing a growing backlog, prioritizing features and approving changes.

Secondly, the handover from development teams to maintenance teams needs to happen later on in the application life cycle. Therefore, it is always a good idea to keep the development team in place after the production release to take care of the Tuning sprint.

Manage the Help Desk Challenge

Our extreme change mantra usually extends throughout the life of the application. After the Tuning sprint, the application typically decreases its level of critical change and is then passed over to maintenance teams. Agile maintenance teams keep a steady, continuous release cycle, launching new features every two to four weeks. This short release cycle requires constant retraining of the help desk which is simply not cost effective.

One solution is to release less frequently. In fact, some of our customers have artificially slowed the release cycle for application updates to accommodate the learning curve of the help desk. However, this becomes problematic as the business quickly learns you can now make changes very rapidly as experienced during the Tuning phase of the project. Thus slowing down the release process in maintenance mode requires resetting expectations with the business.

We have found that a better solution--especially effective for Intranet applications--is to spread the help desk function among the business user community. In a specific situation, one of our customers launched a Requisitions Management application to 1,400 employees, most of whom were first time browser users. To distribute the help desk load, they used 45 departmental administrative assistants as first points of contact. The agile delivery and maintenance teams then focused on keeping this smaller number of power users in sync with the new releases.

You might wonder why we did not simply incorporate the help desk team into the delivery process in order to train them all at once. Our experience has taught us to look at the problem in a different way. Because of the large amount of applications supported by this staff, they had no specific domain expertise, and that prohibited them from efficiently participating in the delivery phase.

Dispense with Useless User Manuals

If you think maintaining a help desk is difficult, keeping a standard user manual in sync with a continuously changing application is tougher. Here the best solution has been to embed contextual "Help" into the application itself. This way, changes in the application and in the Help or instructional content are always incremental, keeping everything in sync. We have found that a content management framework embedded within the application simplifies this process of delivering user documentation in sync with each new code base.

In traditional development projects the user documentation task can, and often is, performed fairly independently of the development work. Since in agile projects the scope evolves at each sprint, documentation and development teams must work side by side and contribute to each new release. Expect a substantial impact in your delivery, maintenance and documentation teams as you shift into an agile way of working.

React Fast to Increase User Adoption

Assuming change as part of the software delivery process is crucial to Agile delivery. In addition, we have found that this approach also dramatically increases the adoption of our applications.

Let me give you an example. In a recent project, we had to implement a Web-based invoice approval system that would replace a complex set of manual processes. A particular sub-process involved the final invoice approval by cost center directors. In the manual process, the director's personal assistants (PAs) brought them an explanation of the invoice with an approval form ready for signature.

The new Web invoice approval system involved a "small" re-engineering of the process that assumed a pure self-service model without PA intervention. (Note: all of the cost center directors were part of the steering committee that helped define and approved this innocuous process change.) However, during roll-out we realized that half the directors had shared their passwords with their PAs, so they could do the pre-validation. Suddenly we had administrative assistants with $100,000 approval power.

As you might guess, the capacity to change fast was crucial here. A new PA profile was added to the application, the work flow was changed to include the pre-validation process, and most importantly the PAs provided input on requirements. During the original analysis, the PAs were not even considered as stakeholders.

This change was implemented over a couple of days driving end user adoption and making IT a hero!

During the project postmortem, we met with the consultants responsible for the initial requirements analysis to figure out why they did not foresee this problem. In addition, we wanted to understand why the cost center directors signed off on the process design. We discovered that the directors were embarrassed to state they relied so heavily on their assistants so they approved the process change hoping it would work.

When it comes to predefined business requirements, adopting an extreme change mantra can help you overcome poor requirements, "wishful thinking" and re-engineering challenges. We have dozens of these stories that reinforce the need for continuous change, even after the application has gone into production.

Conclusion

By far, the greatest benefit of pursuing our mantra of extreme change is delivering what the business needs, on time and on budget, both predictably and well. When a company is able to react quickly to unforeseen situations, and overcome them successfully in a rapid manner, there is a direct correlation with user satisfaction--adoption simply increases and IT regains its rightful place, aligned with the business.