Ovum - Technology Evaluation and Comparison Report - Software-Lifecycle-Management-2011-2012

Incorporating
Technology Evaluation and Comparison Report
OVUM Butler Group
Software Lifecycle Management 2011/2012

Managing application development, from Agile and cloud to embedded systems and mobile apps
WWW.OVUM.COM
Enterprise IT Knowledge Centre

At the heart of the new service are more than 150 ICT analysts from the former Ovum and Butler teams. They provide deep insight into both vertical and horizontal business technology, delivered through best-in-class research and analysis. To their insights, we add the expertise of Datamonitors 350 business analysts. It is this combination that makes the new Ovum IT service especially valuable to clients: by integrating the three teams, we can offer unique insight into the opportunities and issues facing you and your customers, and dispense invaluable advice to help you create an effective technology strategy a process that we describe as Collaborative Intelligence. Our comprehensive research agenda spans the full IT investment lifecycle. Our analysis and advice help you to create the optimal technology investment portfolio for the organisation, select and implement the appropriate solutions and services, and manage those investments to realise the desired business benefits. Our coverage ranges from insight into industry-specific business processes and analysis of vendor markets, through to radical opinion on disruptive technologies and best-practice IT implementation guides. Here we present thought-leading research and strong examples of Collaborative Intelligence in action, and we look forward to working in partnership with enterprises globally. For more information, please contact Mike James on +44 1482 608380 or mike.james@ovum.com
Research
Michael Azoff Tony Baer Chandranshu Singh
Important Notice
We have relied on data and information which we reasonably believe to be up-to-date and correct when preparing this Report, but because it comes from a variety of sources outside of our direct control, we cannot guarantee that all of it is entirely accurate or up-to-date.
Acknowledgements
Tony Cripps Maxine Holt Tim Jennings Surya Mukherjee
This Report is of a general nature and not intended to be specific, customised, or relevant to the requirements of any particular set of circumstances. The interpretations contained in the Report are nonunique and you are responsible for carrying out your own interpretation of the data and information upon which this Report was based. Accordingly, Ovum is not responsible for your use of this Report in any specific circumstances, or for your interpretation of this Report. The interpretation of the data and information in this Report is based on
Published by Ovum Published June 2011 Ovum

All rights reserved. This publication, or any part of it, may not be reproduced or adapted, by any method whatsoever, without prior written Ovum consent. Artwork and layout by Karl Duke, Steve Duke, and Jennifer Swallow
generalised assumptions and by its very nature is not intended to produce accurate or specific results. Accordingly, it is your responsibility to use your own relevant professional skill and judgement to interpret the data and information provided for your own purposes and take appropriate decisions based on such interpretations. Ultimate responsibility for all interpretations of the data, information and commentary in this Report and for decisions based on that data, information and commentary remains with you. Ovum shall not be liable for any such interpretations or decisions made by you.
Software Lifecycle Management 2011/2012

Contents
Chapter 1: Executive summary 1.1 Executive summary 1.2 Report objectives and structure Chapter 2: 2011 Trends to watch: application lifecycle management 2.1 Summary 2.2 Application lifecycle management matters 2.3 Key trend one: the Agile impact on ALM continues to ripple 2.4 Key trend two: Software-as-a-Service ALM sees strong growth 2.5 Key trend three: ALM for mobile development 2.6 Key trend four: application and security management 2.7 Key trend five: ALM and product lifecycle management 2.8 Key trend six: emergence of DevOps 2.9 Recommendations Chapter 3: The Ovum SLM 2011 technology evaluation model 3.1 Summary 3.2 IT-business alignment 3.3 Software assurance 3.4 Enterprise agility 3.5 Delivery management 3.6 Product innovation 3.7 Recommendations 3.8 Breadth of lifecycle discipline coverage 3.9 ALM integration framework 3.10 ALM business solutions 11 13 19 21 23 24 26 26 27 29 30 31 32 35 37 38 39 40 41 42 44 44 46 48
CONTENTS SOFTWARE LIFECYCLE MANAGEMENT 2011/2012
Contents Continued
Chapter 4: Agile and Lean process adoption for software development 4.1 Summary 4.2 Match development process to organizational culture and project type 4.3 Kanban adds new flexibility to development workflow management 4.4 Continuous delivery and deployment: a new phase in Agile development 4.5 Agile and lean will embrace large-enterprise development projects 4.6 Recommendations Chapter 5: Enterprise mobile development: trends, platforms, and tools 5.1 Summary 5.2 Mobile application development is fragmented but standard platforms are emerging 5.3 Mobile development is active in open source and web-based technology 5.4 Mobile development environments reduce the complexity of targeting multiple devices 5.5 Recommendations Chapter 6: ALM in systems/product engineering: two case studies 6.1 Summary 6.2 Embedded software transforms the products business
55 57 58 59 60 62 64 65 67 68 72 76 80 83 85 86
6.3 Case study one: requirements management for a diversified european tier-1 automotive supplier 86 6.4 Case study two: Agile processes for a major aerospace contractor 6.5 Recommendations Chapter 7: Technology comparison 7.1 Application Lifecycle Management Features Matrix 7.2 Application Lifecycle Management Decision Matrix 7.3 Vendor analysis 92 94 95 97 118 135
Contents Continued
Chapter 8: Technology Audits Atlassian Atlassian ALM CollabNet CollabNet TeamForge 5.4 HP HP Application Lifecycle Management IBM Rational IBM Rational ALM Solution Micro Focus Micro Focus Caliber, Silk, and StarTeam Products 2010 Microsoft Microsoft Visual Studio 2010 Ultimate MKS MKS Integrity V2009 sp4 Polarion Polarion ALM Enterprise 2011 Rally Software Rally Unlimited Edition Serena Serena ALM products TechExcel TechExcel DevSuite v8.3 ThoughtWorks Studios ThoughtWorks Studios Agile ALM Chapter 9: Vendor profiles Aldon AccuRev Adobe CA Technologies Coverity Digite edgeIPK Electric Cloud Kovair OutSystems Oracle Perforce Software RADTAC Rogue Wave Software
157 159 169 181 191 203 213 221 233 243 253 265 275 285 287 288 289 290 292 293 294 295 296 297 298 299 300 301
Contents Continued
Chapter 9: Vendor profiles (continued) Salesforce.com Sapient Seapine Software Tasktop Technologies Tomos VersionOne Zend Appendix Glossary 303 304 305 307 308 309 310 313 315
Incorporating
OVUM Butler Group
CHAPTER 1: Executive summary
WWW.OVUM.COM
1.1 Executive summary

Catalyst
Application lifecycle management (ALM) is a process and philosophy as well as a software solution to support the process. After a period of relative stability, several factors are beginning to reshape the ALM market. The popularity of Agile processes and methodologies, one of the few growth spots during the recent recession, is driving vendors to change their product plans. The spotlight on the cloud, particularly software-as-a-service (SaaS) ALM, is having a similar impact on ALM vendors as it is on the rest of the enterprise software industry, as software delivery and business models undergo pressure for change. Continued growth of embedded software components in smart products and systems engineering is prompting a minority of ALM vendors to target this market. Finally, the rapid adoption of smart mobile devices is presenting an opportunity that remains open for ALM vendors.
Key findings:
Agile development has reached the mainstream, and new pragmatic approaches focused on mixing and matching appropriate practices have emerged, supported by Agile ALM systems. SaaS-based ALM offerings will continue to be a growth market in the year ahead. With the explosion in mobile application development started by Apples iPhone/iPad and Googles Android platform, smart mobile will become a major target for enterprise applications. IBM Rational and HP are ahead of the market in integrating security testing into the application lifecycle, and rivals will need to acquire or partner with niche players to catch up. Embedded software development for engineered products is on an exponential growth curve and represents a lucrative market for the few ALM providers that have chosen to target it. In 2010, DevOps entered the awareness phase in the ALM market, and 2011 will see automated tooling targeting the needs of DevOps and Agile release management in particular. The introduction of continuous delivery and deployment represents a new phase in Agile development. Large organizations must accommodate tooling diversity when developing enterprise standards for managing the product lifecycle. A federated approach is the only practical option. Businesses must learn lessons from enterprise systems projects such as ERP or CRM migrations: change is painful, but be sure to communicate benefits that are tangible not abstract.
Ovum view
Over the past year, the highlights of the ALM tools market have centered on the continuing adoption of Agile practices across multiple teams in the organization, growing vendor and customer interest in managing the lifecycle of embedded software that is developed for complex engineered products, the blossoming of a broader-based market for ALM tooling in the cloud, and the emergence of a multi-polar market for serious mobile application development. The continued strong presence of the Apple iOS platform, the rapid emergence of Google Android, and the upcoming entry of HTML5 are providing developers with a choice of deployment platforms for native or web-based smart mobile applications. ALM vendor response to these market drivers has been varied.
CHAPTER 1: EXECUTIVE SUMMARY
13
Enterprise Agile: Almost every vendor is paying more than lip service to providing planning and resource management tools that can accommodate enterprise rather than individual team-based Agile development, yet there remain scattered holdouts. Significantly, a perceived lag in support of lightweight processes for managing Agile teams across the enterprise proved to be one of the strongest areas of customer impatience as measured by our customer-sentiment surveys (see Chapter 7). Complex embedded systems development: A minority of ALM vendors have chosen to support embedded systems development, a sector that has high barriers to entry but potentially offers strong rewards to vendors because the engagements carry high dollar value. The cloud: While pioneers such as Rally and CollabNet have offered cloud-based ALM capabilities for years, 2010 marked the year that most of their rivals began broader based rollouts beyond performance or security testing. Here again, there are several vendors that have yet to dip their toe into the water. Mobile development environments (MDEs): Similar to IDEs (integrated development environments), MDEs are specific to mobile developer needs, such as simplifying development across multiple mobile devices. ALM tools providers are not integrating with MDEs because the market is new and there are too many products. Developers will have to switch between their MDE and ALM tools, which remains an opportunity for ALM vendors. This report also sees the first usage of a name change in Ovums coverage of the management of software development. ALM terminology takes the application in application development and attaches it to the concept of lifecycle management. Within the world of enterprise IT these applications were always understood to mean software applications. However, this understanding no longer applies because today we find ALM broadening out in two directions. The systems/product development space is seeing significant software usage, and the terminology of application makes less sense with embedded software as part of a larger system or product. There is also the need for software development teams to improve engagement with the business side of the organization and use a language that is more transparent to the business. On both counts a better name for this activity is software lifecycle management (SLM), which we use in the title of this report and discuss further in Chapter 3 where we tie in SLM to a new business-oriented approach. The rest of the report retains the ALM usage for backward-compatibility purposes.
Business issues
Software development organizations have been justifiably on the defensive. There is little mystery that IT is being called on to do more with less, yet with barely 10%-20% of the budget to fund innovation. In the meantime, most development organizations have been stuck in disconnected process ruts, and not surprisingly have developed bad reputations for failing to deliver on time, on scope, or on budget. Pioneered by vendors including Rational Software prior to its acquisition by IBM, software development tools vendors began to aim wider, with bundled tooling targeted at supporting the lifecycle from requirements and testing through to the development and release-management stages. However, until recently, ALM as a market remained relatively flat. It boiled down to two key problems: Befitting the engineering profession that they really are, software developers view issues as technical problems and are notoriously poorly equipped to justify the business benefits of integrating and streamlining the application lifecycle. On the vendor side, until recently ALM tools were ALM in name only. At best, bundles of disjointed tools connected by point-to-point interfaces, ALM products were not the integrated suites for which they were promoted. Traditional ALM bundles were incapable of providing a coherent end-to-end picture of the progress of translating a business requirement into working software, nor could they provide support for closed-loop decision-making processes regarding the management of application development projects. As a result, most IT organizations found it difficult to make the business case for justifying significant ALM investments.
14
SOFTWARE LIFECYCLE MANAGEMENT 2011/2012
Several key trends and events have changed this, including the emergence of: Agile development methodologies aimed at breaking software development bottlenecks. Although initially codified in 2001-2, Agile practice is emerging, with growing numbers of pockets in enterprise IT organizations. Traditionally confined to small co-located teams on self-contained projects, successful projects have had two impacts on the enterprise: larger distributed teams are adopting Agile from the ground up, while IT organizations are beginning to embrace Agile as an official enterprise methodology. De facto standards for tools integration using loosely coupled serviceoriented architectures that leverage RESTful-style service integration. Cloud-based delivery, which simplifies mass deployments of ALM processes and tooling. Best practices that recognize the need to federate islands of ALM practice, tooling, and artefact repositories while preserving the tooling and process autonomy of local teams of business units.
...the ALM tools industry has followed the patterns of the software industry at large...
Significantly, the ALM tools industry has followed the patterns of the software industry at large, which is characterized by consolidation at the top complemented by the rapid emergence of open service tooling at the periphery. In the past couple of years IBM Rational has completed its acquisition of Telelogic, while Micro Focus is still digesting the dual acquisitions of Borland and the software quality assurance tooling of Compuware. Yet consolidation has not prevented the emergence of new voices in the ALM community, with players such as Rally, ThoughtWorks Studios, and VersionOne capitalizing on the success of Agile methodologies, and CollabNet and Polarion leveraging open source.
Technology issues
There is a huge misconception that ALM is a product not a process. By definition, it is the process by which IT organizations manage the creation, deployment, and operation of software over its full lifecycle. In practice, ALM has been associated with tooling suites aimed at managing the tasks of this lifecycle, but ALM vendors have rarely delivered on the promise of integrating the management of the full application lifecycle. ALM integration has been a perennial problem in the industry. On the one hand, customers generally do not rip out their favorite point solutions and replace them with a single suite from one vendor. Even where the ALM part is replaced with a single suite, there are often other tools to integrate with, such as in systems engineering where there are activities managed by product lifecycle management (PLM) tools. ALM vendors themselves have integration issues as merger and acquisition activities result in the need to integrate tools internally. Attempts by individual vendors to create an ALM API standard have not succeeded. However, the integration problem has been resolved to some extent with the emergence of integration hubs, and the policy of ALM vendors building point-to-point connectors to the leading tools in each ALM segment. Integration hubs have grown around the most popular IDEs: Eclipse and Microsoft Visual Studio. Tasktop is the most well-known Eclipse-based ALM hub, through its Mylyn open source product and premium solution, Tasktop Enterprise ALM. A number of vendors also connect to Visual Studio Team Foundation Server. IBMs Jazz platform was designed as an ALM hub, but while other vendors have not connected to it, Jazz remains an essential hub for IBM Rationals own wide portfolio of ALM solutions, from legacy to modern solutions, from internally built products to acquired solutions. Serena is applying technology developed for the defunct Eclipse Application Lifecycle Foundation (ALF), coupled with process orchestration technology that it had developed for its former mashup business, to deliver a new process orchestration-oriented ALM hub. Other hubs have also appeared. Rally Software, for example, will connect its Agile project management suite to many ALM solutions, and similarly, the leading ALM vendors covered in this report have some level of direct integration to third-party products through connectors and plug-ins.
15
The key change in recent years in ALM has been the need to ensure that the management of the development activity does not constrain the choice of how development is done. Principally this means supporting Agile methodologies, as well as traditional waterfall-like processes. Agile impinges on ALM in a number of key areas. Requirements management in Agile needs to deal with the capturing of business-level stories and allow for tree-like hierarchies of stories to evolve, with end leaves representing technical tasks. From a project-management point of view, these stories need to be made visible and moved across the lifecycle in a ...ALM exists in transparent manner, with traceability of the requirements to code artefacts order to achieve through the stories. In addition, testing needs to be made an integral part business aims... of all lifecycle activities, with test runs and results appropriately managed. Ultimately, ALM exists in order to achieve business aims, including delivering development projects within schedule, within budget, and within acceptable quality standards, and producing value to the business. For IT professionals to present ALM to the business side of an organization it is necessary to use terminology that business people can understand because it is the senior management that approves the budget for ALM solutions. The benefits can be highlighted by structuring ALM in terms of business solutions. Ovum has identified the following five categories: IT-business alignment: ALM helps to ensure that IT projects deliver what the business needs. Specific tools within ALM that help are: IT governance, application portfolio project management (APPM), project management (PM), requirements definition, and requirements management. Software assurance: This concept aims to widen the remit of QA to ensure that the application performs not only with high quality, but that it is secure, and that in production it performs to the level required. From a business perspective, SA ensures that the application does what it promises. Tools segments cover QA, testing, application security, and application performance management. Enterprise Agile: Agile methodologies were created in small co-located teams, but as Agile principles and practices are applied in ever larger projects there is a need to support larger teams that are also globally distributed. Businesses will want to increase the success rate of large-scale projects because these have a history of failure. Specific tools that apply to this category include APPM and Agile PM. Delivery management: The importance of the last mile between completing developer and QA testing to deploying successfully into production has gained increasing attention with the rise of the DevOps movement. As applications become larger and more complex, the task of deployment becomes more challenging and risky. Good (Agile) processes and automation can make a big impact. Tools that apply here are build, release, and deployment management. Product innovation: As software becomes a significant component in products, from cars to mobile phones, there is a need to expand ALM beyond IT and into the systems and product engineering space. ALM and product lifecycle management integration becomes necessary as organizations look to streamline their processes and leverage skills and tools. The tools side of ALM tends to split between the developer-oriented tools that developers need in order to get the job done, and business-oriented management tools that are necessary to achieve business goals. The categorization above brings out the latter, and this is the market for enterprise ALM suites.
Market issues
The ALM market has seen some consolidation in hot areas of activity, with The ALM market has HP and IBM making acquisitions to improve their support for application seen some security, HP through Fortify, and IBM through Ounce Labs. The need to consolidation in hot ensure that Agile and Kanban (incremental process and systems change for organizations) are supported has led to acquisitions, including areas of activity... CollabNet buying Danube software, and Atlassian acquiring GreenHopper from Pyxis. Vendors such as HP have made significant internal investments in building out their own Agile solutions, as well as modernizing and extending their core strengths in testing and QA/testmanagement tools.
16
In 2011, the primary forces shaping the ALM market include: Mainstreaming of Agile. The growth in popularity of Agile across the enterprise has created an appetite for tools that adopt a more strategic approach to managing Agile across multiple teams or entire swaths of the enterprise. Players known for Agile planning, such as Rally and ThoughtWorks Studios, and more recently (through acquisition), CollabNet, have led the way in delivering Agile planning through the cloud. Meanwhile, both IBM Rational and Serena are emphasizing the ability to orchestrate Agile planning with requirements, quality, and release-management offerings. Spreading of the cloud. Almost every ALM vendor now offers cloud delivery for some or all of their tools. The pioneers include Rally and CollabNet, whose tools (Agile planning and SCCM, respectively) originated as cloud-based offerings. At this point, MKS is the only vendor that has not reported demand for tooling delivered through the cloud. Fruition of embedded software development. Smart systems are all around, with embedded software or firmware turning up in products across virtually every durable goods sector. In products such as automobiles and consumer electronics, embedded software is not only supporting the product, but also defining it. Although product engineering firms are in a minority in the software development market, their appetite for tooling is way ahead of their numbers because their need for ALM is far more complex. Product and systems engineering firms have complex needs, making them a good market for ALM tools. The existing product lifecycle management tools market dwarfs the ALM market size. Given the rise of embedded software, the product and systems space is promising to be a significant growth market for ALM, in particular for big-ticket suites. IBM Rational (through its Telelogic acquisition) and MKS are devoting most of their attention to this market, while Polarion and Serena plan to vigorously contest this space. An opportunity for improved support of smart mobile application development. Mobile developers are mostly focusing on the high-end mobile market, targeting a small band of platforms, currently Apple iOS and Google Android. Ovum also expects open web-based technologies (HTML5, CSS3, and JavaScript) to be added. In addition, there is a host of second-tier mobile platforms that may well grow. Given that iOS is a closed proprietary platform, its strong position can only be maintained by a continuing stream of successful Apple products, something that cannot be guaranteed. Given the complexity of building apps for multiple devices, models, and form factors, developers need tools to efficiently manage that complexity, giving rise to a host of new mobile development environments. ALM vendors have the opportunity to either enter this market directly or to integrate with the products. However, no winners have emerged in this new market and ALM vendors have yet to capitalize on the opportunity. ALM tools APIs have become the domain of de facto, rather than formal, standards. Perennial favorites include Microsoft Visual Studio and Team Foundation Server (TFS), which are consensus client and server standards for vendors already in the Microsoft .NET universe. Conversely, the Ovum expects Java community and others have largely settled on the Eclipse plug-in model for the IDE client end, but there is no consensus on the server side. Otherwise, DevOps to become a APIs outside the .NET world remain vendor-driven. stronger market Ovum expects DevOps to become a stronger market force in the future. For force in the future. now, DevOps has entered the agenda as developers adopting continuous testing and release processes from Agile methodologies require repeatable processes that interface with the IT operations world. Most ALM vendors are still in learning mode and are still trying to identify who is the right target market for a stage in the software lifecycle that extends beyond the developer market that is their comfort zone.
The ALM decision matrix

The Ovum decision matrix explores the competitive dynamics within the ALM market and to help organizations make informed choices about a vendor based on its technology strength, reputation among customers, and impact in the market. The decision matrix provides a comprehensive and transparent view of ALM vendor capabilities and presents advice on which vendors should be shortlisted, considered, or explored. Ovum invited the leading ALM vendors to participate in the technology evaluation and comparison part of the report, and we believe that inclusion in the Ovum Decision Matrix is itself recognition that sets these vendors apart from the rest of the market. The results are summarized in the table below; vendors are listed in alphabetical order within each category.
17
Rating
Company/Solution
HP
Ovum Opinion
HP has recently launched a new ALM solution that offers comprehensive end-to-end functionality, with exceptional strengths in enterprise QA and test management. IBM Rational is steadily rolling out its next-generation products that support its Jazz framework, while building on its existing base with managing the lifecycle of software embedded in complex systems of systems. Microsofts Visual Studio 2010 continues to provide .Net developers (and others) a full featured ALM platform. The solution further enhances capabilities with Test Lab management and tie in to the Azure cloud. MKS pioneered development of unified tooling covering SCCM, requirements, quality, and project management; a key focus is product engineering companies managing development of software in complex systems. Micro Focus has taken the first steps in converging the QA products inherited from the Borland and Compuware testing tools acquisitions, but has yet to roll out broader based offerings that reflect its existing tools portfolio. Polarion has steadily built out its ALM solution organically, based on open source solutions, to provide an enterprise level ALM solution, with the systems/product engineering market a particular focus. Serena is evolving into a vendor of ALM solutions that are based on a process orchestration engine. It is also partnering with Nolio to develop a unique release management solution that extends into orchestrating software deployment. TechExcels workflow rooted solution and knowledge base system marks it out as an exceptionally well integrated solution, with increased support for SaaS ALM a 2011 target. Atlassian has built its development management solutions in a unique way, offering some industry leading products such as JIRA. Its cloud based studio and recent acquisition of bitbucket add further diversity. CollabNet has built a growing stable of tools leveraging its popular Subversion SCCM technology. The Danube acquisition has extended its solution covering Agile planning. Having helped pioneer cloud-based ALM, Rally has built an Agile planning solution that takes a lightweight approach to related areas of project, requirements, and quality management. ThoughtWorks Studios offers its Adaptive ALM solution to ensure teams can perform in an Agile way or adopt hybrid practices. A vendor at the forefront of Agile thinking; for example Go addresses DevOps needs.
IBM
Shortlist Consider Explore
Microsoft
MKS
Micro Focus
Polarion
Serena
TechExcel
Atlassian
CollabNet
Rally Software
ThoughtWorks Studios
18
1.2 Report objectives and structure

Chapter 2. 2011 Trends to watch: application lifecycle management
Current trends in ALM are examined. With the economy recovering from the recent downturn, we identify areas in the ALM market that have seen growth. For example, the newer Agile vendors in the market are beginning to have an impact at the enterprise level, the growth of software in products is broadening the ALM market, and the DevOps movement has gained rapid visibility.
Chapter 3. The Ovum SLM 2011 technology evaluation model

This chapter provides a definition and architecture for Ovums approach to SLM. It serves as grounding in the process and tools aspects of SLM and provides an in-depth explanation of Ovums evaluation model used in the Decision Matrix technology assessment: the features matrix.
Chapter 4. Agile and Lean process adoption for software development

Agile and Lean processes and methodologies are having a significant impact on how software development is done. In 2010 it can be said that Agile became mainstream, which means all developers are now at least aware of Agile and will consider Agile practices as a viable option. However, the question of which particular process to adopt is still complex, reflecting the challenge of matching work style to project type and organizational culture. This chapter addresses the trends in Agile and Lean development adoption.
Chapter 5. Enterprise mobile development: trends, platforms, and tools

A number of areas in development have seen significant change, and mobile applications is one of them. The rise of Apples iOS devices and App Store concept has opened a major market for developing consumer apps, with enterprises beginning to exploit the new potential of smart devices. This chapter examines the rival mobile platforms, the trends in adoption, and the rise in mobile development environments (MDEs).
Chapter 6. ALM in systems/product engineering: two case studies

In the engineered products world, software is increasingly defining, rather than simply enhancing products. Todays automobiles and aerospace systems are no longer electromechanical products, but complex intelligent systems that optimize every aspect of operation. Todays consumer electronic devices are differentiated more by their software than their physical features. This section provides two case studies to illustrate the changes taking place in systems and product engineering.
Chapter 7. Technology comparison

This section reviews the leading vendors and solutions found in the ALM space, providing recommendations and market analysis, and presents Ovums ALM Decision Matrix: a 3D bubble chart placing vendors in a matrix covering technology, customer sentiment, and market impact. This section also presents Ovums ALM features matrix which shows a side-by-side view of vendor solution capabilities covering their existing ALM products.
Chapter 8. Technology audits

This section contains the Ovum technology audits of the leading vendors in the ALM market and their solutions.
Chapter 9. Vendor profiles

This section outlines some of the other vendors found in the ALM space. No conclusions should be drawn from the inclusion or exclusion of a vendor in this section.
19
Incorporating
OVUM Butler Group
CHAPTER 2: 2011 Trends to watch: application lifecycle management
WWW.OVUM.COM
2.1 Summary
Application lifecycle management has entered a new phase in its evolution. It has to deal with the impact on software development of Agile and Lean processes and methodologies, as well as the impact of the growth in cloud services and software-as-a-service in particular. In the longer term the vendor community will face the need to renew its relevance by recasting its products to demonstrate the delivery of tangible business value.
Ovum view
The combination of a challenging changing economy and business climate is clearly steering many of the changes that will impact the ALM community in 2011. It is spurring the growing adoption of Agile methodologies that emphasize delivering business value, the attraction of the cloud and SaaS for ALM tool delivery to help reduce costs, the beginnings of a DevOps movement that is seeking to improve the reliability of software delivery to raise quality and drive down costs, and continued growth in embedded software development for high-tech products. In other cases new technologies are driving changes to application development. These include the emergence of smart mobile devices as the next major enterprise software development target, and continued development of rich interactive web applications that raise vulnerabilities and need an integrated application security and testing approach.
Key messages
With Agile now in the mainstream, new pragmatic approaches focused on mixing and matching the right methodologies will emerge and will be supported by ALM tools vendors. Demand for cloud deployment will prompt SaaS-based ALM offerings, not only from new entrants, but also from established players. With the explosion in mobile application development started by Apples iPhone/iPad and Googles Android platform, smart mobile will become a major target for enterprise applications. IBM Rational and HP are ahead of the market in integrating security testing into the application lifecycle, and rivals will need to acquire or partner with niche players to catch up. Embedded software development for engineered products is on an exponential growth curve and represents a lucrative market for the few ALM providers that have chosen to target it. In 2010 DevOps entered the awareness phase in the ALM market, and 2011 will see automated tooling targeting the needs of DevOps and Agile release management in particular.
CHAPTER 2: 2011 TRENDS TO WATCH: APPLICATION LIFECYCLE MANAGEMENT
23
2.2 Application lifecycle management matters

ALM as process and tooling have changed but the concept remains a necessity
There is a huge misconception that ALM is a product, not a process. By definition, it is the process by which IT organizations manage creation, deployment, and operation of software over its full lifecycle. In practice, ALM has been associated with tooling suites aimed at managing the tasks of the lifecycle, but ALM vendors have rarely delivered on the promise of integrating the management of the full application lifecycle. Nonetheless, ALM as a process, and as a technology that manages it, continues to evolve. One aspect, however, remains constant: the need for ALM in anything but the simplest software project. This means that the lifecycle should be managed end-to-end across the following disciplines and phases. Application project and portfolio management Project inception and requirements gathering Requirements management Design and use-case analysis Code construction Testing and QA Build, release, and deploy Application performance
Advanced RD & RM, QA Full ALM stack for traditional development and, increasingly, Agile Sophisticated users Productivity is the #1 issue Will use OSS to premium Mixed patterns major opportunities Reduce costs strong message ALM as a message does not sell to Agile developers OSS ALM tools do well Mass market: SME and freelance/consultants Large businesses Software houses, ISVs, SIs Global enterprises, industrials
Higher-value deals
Addressable market
Figure 1: ALM system market today: Who is buying what? Source: Ovum
24
The activities are linear under the traditional, waterfall development process, whereas iterative processes such as those found in Agile and Lean development methodologies will overlap with these throughout the lifecycle. An ALM system approach provides the means to manage hundreds, even thousands, of requirements, trace their construction and testing, and verify their deployment, managing a project from tens to thousands of individuals working in multiple, often distributed, teams. An ALM system approach is necessary to manage project complexity, provide traceability of requirements throughout the lifecycle, monitor project progress, ensure high quality, and achieve predictable delivery. Once a project size reaches a critical threshold it is virtually impossible to deliver a successful outcome without the essential management tools comprising ALM. Ovum segments the ALM market (see Figure 1) where the mass market may show some resistance to ALM adoption, while at the top end ALM is recognized as essential for project success.
ALM systems are becoming more in tune with user needs

The need for customers to buy the complete ALM stack from one vendor in order to benefit from an integrated platform has not helped the ALM market. First, not all ALM systems are sufficiently integrated to fulfill this requirement, and second, customers do not want to rip and replace favored existing tools, not least because there is waste involved through disruption and retraining. The ALM market has responded with solutions that are better integrated and can also interoperate with other tools, usually the leading tools in the various ALM segments. A number of vendors offer an ALM hub that will allow two-way transfer of data between tools from different vendors, such as between those from Rally Software and Tasktop. The trend is for point solutions from within an ALM suite to be able to work stand-alone without necessitating the user to buy the complete stack, but when other solutions are added from the same vendors portfolio, integration benefits are gained when plugged in to the platform.
ALM systems must offer business value
Ideally, all businesses want their software-development projects to be predictable, on time, on budget, reliable, high-quality, and to deliver value to the business. ALM systems have traditionally been sold to developers and IT directors, but this approach has yielded limited success. Senior management outside IT will have little understanding of why a better approach to software development will help the business. IT organizations, backed by ALM tools vendors, must craft solutions that are designed to deliver tangible business benefits.
ALM systems have traditionally been sold to developers and IT directors...
The key realization is that the way that organizations develop software is often equally as important to the business as the software itself, and the onus is on IT to communicate and back this up. Ovum has identified key software development and delivery solutions that will affect the business, ensuring that reliable and secure software is delivered that is compliant with company policies and regulatory requirements, and performs to requirements. For ALM tools vendors, the difference is not the end of point tools sales or a matter of simply bundling multiple point tools together. The solutions require several pieces that enable traditionally disparate functional tools to operate as a single solution. This includes a means for managing workflow and collaboration, performing version control for all artefacts (not only source code, but also requirements, tests, and build scripts), and providing a means for sharing but not duplicating artefacts. Businesses are increasing their reliance on IT and software applications in particular, and will therefore have a greater need for ALM. The ALM market in turn will appeal to businesses by ensuring that they can address the fundamental value benefits of ALM.
25
2.3 Key trend one: the Agile impact on ALM continues to ripple
Agile development can only succeed in the presence of good automation
Agile software development adoption is continuing its diffusion into the mainstream, but 2011 will see a more pragmatic approach to how Agile is actually practiced. There will be a range of adoption practices from fully orthodox adoption, such as pure Scrum and Extreme Programming, to a scenario where an organization will pick elements from Agile methodologies that fit in best with existing processes and requirements. Agile development has a good degree of awareness at the lower end of the ALM market pyramid in Figure 1, but there is a tendency for ALM systems to be associated with the old style of working and to be too tool-heavy. This is unfortunate because Agile development can only succeed with the presence of good automation. Unit testing, test-driven development, continuous integration, testing and delivery, electronic whiteboard, and Kanban board are examples of how Agile benefits from automated solutions. This paradox will be resolved once Agile developers have a better understanding of the new generation of Agile ALM tools. The term Agile ALM is a recent description for these tools and will help spread the message that Agile and ALM are not contradictory. There is better understanding of the benefits of ALM systems going up the pyramid in Figure 1. Kanban in particular has become notable in 2010. There are three rules in Kanban: visibility of workflow, limits on work in progress, and pull from demand rather than push from requirements. An increasing number of Agile ALM systems can offer electronic Kanban boards.
2.4 Key trend two: Software-as-a-Service ALM sees strong growth

ALM vendors can no longer avoid the SaaS delivery option
ALM vendors are seeing growing demand for Saas-delivered ALM tools. While ALM system sales overall suffer the dual challenges of economic sluggishness and upgrades to support Agile software development (requiring rewriting or creating new solutions), SaaS ALM has bucked the trend. Unlike managed services, which command a premium Ovum believes no for taking on the burden of running the hardware that delivers the software vendor can stay in solution, SaaS is a model based on cloud computing economics.
the ALM market without offering a competitive SaaS solution.
New entrants and enhancements to existing SaaS ALM are emerging. Ovum expects the leading enterprise ALM vendors to follow suit because the appeal of SaaS will reduce their market share if they do not. Micro Focus, the sleeping giant in ALM, has yet to reveal its hand in ALM after the Borland acquisition, and Ovum believes it too will offer a SaaS solution. Given this marked trend, Ovum believes no vendor can stay in the ALM market without offering a competitive SaaS solution.
Developers are gaining faith in SaaS

Three key factors explain the trend toward SaaS ALM. First is the economics argument, second is the confidence issue in adopting a cloud-hosted service, and third is the growth in DVCS which is perfect for SaaS.
26
SaaS economics are ideal for the cost-conscious CIO. Users can self-service their requirements via a website. They pay per-use at competitive rates, and can scale up or down according to actual needs. SaaS therefore lowers costs, particularly with the use of multi-tenant architecture, which offers users the immediate benefits of upgrades and ease of use for the hosting providers, a win-win situation. As Salesforce.com has shown, the purely psychological barrier to SaaS can be overcome. Of course, there will be projects where security issues predominate and code will need to be located behind the local firewall, but as SaaS vendors demonstrate secure and reliable services this will become a minority concern. Ovum advises that enterprises using SaaS should deploy their own encryption systems for sensitive projects. Another attraction of SaaS ALM that also reflects Salesforce.coms success is that developers in smallto-midsize enterprises lacking IT resources and systems admin expertise, or small departments in large corporations that do not wish to rely on IT, can benefit from the hosted aspect of SaaS. With SaaS the burden of managing servers is eliminated. The advantage of DVCS is that the need to be continually online is removed. DVCS users keep master files locally, and these are synchronized to servers, ensuring work runs smoothly when developers swap between online and offline work modes. As ALM solutions increasingly adopt web interfaces, it matters little from a developers perspective whether the central servers are behind the local firewall or in the hosted cloud service.
2.5 Key trend three: ALM for mobile development

Mobile joins the enterprise applications mainstream
The mobile smart device market has been opened up by Apple with its iPhone and iPad, creating a secondary market of application developers supplying the virtually unlimited appetite of device users. The Apple devices introduced a new level of usability with innovative use of touch screens and motion detection. The result has seen Apple, followed by Google with its Android OS, enter a market that the incumbent mobile players Nokia, RIM, and others had believed was theirs. In addition to this new wave of application development, the iPad introduces a tablet form factor that also opens up new application possibilities. For application developers the new devices require a new learning curve in mastering APIs, SDKs, and extensive class libraries. Ovum believes that by the close of 2011, smart mobile devices will start to become a deployment platform for enterprise applications, and beyond that the growth will be significant. The path into the enterprise for mobile will resemble that of PCs a generation ago. As mobile devices gain acceptance by consumers they will bring them into work and they will also find ways in which they can use them at work in the same way that local departmental workgroups began sneaking in PCs 25 years ago. This will force businesses to manage these devices rather than fight them. There are benefits to the business in that this new generation of devices will transform the way the workforce interacts with enterprise systems. For example, while PCs liberated data from mainframes, they also created demand for new departmental applications and further empowered knowledge workers. The trajectory will be similar for mobile, which will not only extend existing enterprise applications, but also create demand for new ones while keeping the workforce more connected than ever. The practice of letting employees choose to have their work mobile device under personal contracts and have the costs reimbursed is spreading, propelling the entry of smartphones into the workplace. However, in the long run, adoption is likely to follow the PC model because many IT organizations will implement policies for standardizing on specific mobile platforms and corporate coverage plans, which in turn will make the challenge of mobile application development and/or enablement more practical.
27
Demand for mobile development environments is emerging

These business applications will require team development, and ALM will be necessary to manage the projects. The challenge with mobile is that deployment targets are far more varied than with conventional rich desktop or web browser clients. Furthermore, mobile devices and platforms are evolving at a much faster rate than desktop platforms, while end users are following consumers in embracing the latest technologies. This has created demand for mobile development environments that allow one model to be developed and automatically deployed multiple times to a wide range of devices. The attraction of mobile development environments for enterprises is that it: Enables strategic investments to be made in mobile device delivery. Ensures that the costs around the complexity issues do not escalate out of control, and helps manage the updating of devices without the expensive task of shipping them in and out. Provides out-of-box connectivity to enterprise applications such as Oracle and SAP.
Mobile platform wars get serious and impact developers

The emergence of HTML5 and Apples rejection of Adobes popular Flash-based rich Internet application (RIA) framework has set the stage for mobile platform rivalries that will force developers to make rich versus reach choices. While HTML5 will raise the bar in the richness that standard web development will deliver, it will never attain the richness offered by Apples native iOS platform and Objective-C native programming language, or other proprietary frameworks such as Flash or Microsoft Silverlight. Nonetheless, HTML5 may be good enough for the needs of enterprise software development, which in most cases wont require the state-of-the-art rendering or video-streaming features promised by the proprietary frameworks. However, HTML5 development may need to contend with different browser or platform implementations. While the iPhone-versus-Android battle has drawn the spotlight, the smart mobile platform market is still forming. On the sidelines are players such as RIM and Nokia/Symbian that are seeking to ride the momentum created by Apple and Google. In 2011 Microsoft will also make a comeback to the field with the launch of Windows Phone 7. Given its absence, Microsoft has one last chance to make credible the competition between it, Apple iOS, and Android. While the initial launch of Phone 7 is aimed at consumer devices, Microsoft promises a follow-up release that will add functionality to support enterprise applications, such as greater support for Microsoft Exchange policies that address security concerns. Each of the vendors in this market supplies free, or has available at cost from third parties, dedicated integrated development environments (IDEs) for developing on the platform. A number are based on Eclipse. Therefore from a purely developer code-centric viewpoint, there are plenty of tools to support the burgeoning application market.
Shades of gray emerge

Although the emergence of HTML5 offers a promising path for building multi-platform mobile applications, there will continue to be variations in different browser or platform implementations. Development with HTML5 will not be as plain vanilla as it seems. At the other end of the spectrum there are cracks emerging in the proprietary frameworks that could make some cross-platform portability thinkable. Apples recently modified strategy, which will permit access to conversion tools that can natively compile Flash or other non-native applications to run on its mobile platform, further blurs the issue for developers because non-native applications may run, but in all likelihood sub-optimally, on the iPhone or iPad. Similarly, devices that accept the Adobe Flash player (a notable exception is Apple) will give developers the benefit of being able to write an application in Flex that is portable across multiple devices, albeit with some necessary changes. The next generation of Flex will specifically feature this capability and will leverage the Open Screen Project that Adobe runs together with various stakeholders in the mobile market to achieve mobile application portability. These accommodations will make the rich-versus-reach decision for enterprise developers less black and white.
28
2.6 Key trend four: application and security management

Improvements in network security have helped deflect hacker attention from infrastructure to software
The issue of application security is hardly new. Security was traditionally the domain of specialists responsible for guarding the perimeter, whether for access, authorization, or authentication, or the physical facility itself. A number of factors have served to shift the point of vulnerability and the nature of the problem from infrastructure to software. Improvements in network security have helped deflect hacker attention from infrastructure to software. Furthermore, the growth of e-commerce and the mainstreaming of Web 2.0 RIAs that are often written by less sophisticated developers using looselystructured scripting languages are giving hackers fatter, more vulnerable, and more lucrative targets. Awareness of the implications for software development has come largely at the impetus of the payment card industry (PCI), which has drafted certification standards for merchants and card issuers. It has been further buttressed by the increasingly routine occurrence of software updates aimed at patching security holes. For example, in October 2010 alone: Oracle issued a critical patch update that fixed 29 security issues for Java SE and Java for Business products. The Mozilla Foundations newly released Firefox 3.6.11 fixed eight major flaws that could enable a remote attacker to install malicious software. Microsoft shipped 16 patches addressing 49 vulnerabilities affecting Windows, Internet Explorer, Office, and the .NET Framework. A variety of niche players providing static tests that parse and analyze code much like debuggers, and dynamic tests that that exercise the application much like ethical hackers, have emerged for helping software development organizations test for security holes. In turn, IBM Rational and HP have over the past 18 months acquired static and dynamic testing software tools companies. As with other forms of software testing, security testing is well suited for delivery through a SaaS model that reduces the need for internal development teams to have all the necessary specialized knowledge for conducting rigorous security tests. Over the coming year the number of players, particularly those that deliver via the cloud, will continue to multiply as standalone status and dynamic security testing becomes a commodity.
Conducting security testing as a standalone process is at best a defensive maneuver that will not in the long run address the root causes of security vulnerabilities...
Conducting security testing as a standalone process is at best a defensive maneuver that will not in the long run address the root causes of security vulnerabilities or reduce the cost of keeping software secure. Nor will it prevent potential reputational damage that vulnerable software can cause. Ultimately, an integrated ALM approach that treats security testing in the same way as all other forms of software testing will become essential for IT organizations to manage rather than simply defend themselves against security vulnerabilities. This requires the ability to: Auto-populate test results to defect tracking and/or integrate with IT service desk trouble-ticketing systems Associate security test strategy with non-functional requirements Integrate to impact analysis and financial analysis tools that help IT organizations prioritize the problem Identify architectural security vulnerabilities in code design Track changes to requirements, code, and configuration files, and assess the impact on security.
29
2.7 Key trend five: ALM and product lifecycle management

As software defines engineered products, software and product engineering converge
It is difficult if not impossible to come across a durable goods product today that lacks some form of intelligence or soft functionality that is the result of embedded software. For example, todays car is comprised of a decentralized network of roughly 80 to 100 specialized computers that control the operation of the vehicle. Drivers no longer directly control the car through mechanical coupling, instead they drive by wire where sensors in the brake With smart pedal, accelerator, gear shift, or other device can detect driver action, and in turn optimize the control of the drive system based on sensory data such as products, the the rate of wheel rotation or the wetness of the road. By contrast, a smart processes of mobile device is essentially a metal or plastic container that houses software electronics. The common thread is that software no longer simply enhances development along the product, but defines it.
with mechanical, electrical, and systems design are becoming increasingly intertwined.
This has created a huge opportunity that has so far been exploited by only a handful of ALM players to deliver tooling that plans and manages the development and testing of software that becomes part of engineered products. With smart products, the processes of software development along with mechanical, electrical, and systems design are becoming increasingly intertwined. Software development tasks such as modeling, code development, and testing can no longer be easily divorced from mechanical and electrical design and design simulation. Similarly, developing requirements and managing change becomes an intersection point between software and physical (for example, mechanical and electrical systems) engineering disciplines. The result is pitting the giants of the ALM and the CADCAM worlds in competition for ownership of overlapping processes. For product companies whose organizations are still functionally siloed, it creates competition between software engineering and the physical engineering disciplines over who owns the product requirements, and who owns the change-management process.
Agile developers and concurrent engineers talk the same language

The waterfall product development process originated in the manufacturing industry and was picked up by the software engineering community in the early days of computing. At the time it seemed a sensible process to adopt. Since then the software industry has had to live with the consequences of an approach riddled with faults. It is only in the last decade that Agile and Lean practices have entered the mainstream and changed matters for the better. But while the software industry grappled with waterfall, the manufacturing industries began moving away from it and created an alternative: concurrent engineering. This approach took form a good decade before Agile, and exemplified many principles in common with Agile. With the increasing use of Agile in software development, the increasing use of embedded software in products, and product engineers being familiar with concurrent engineering, there is a way of closing the loop and introducing Agile embedded software development in a concurrent engineering framework. The professionals working in these separate communities can therefore all speak the same language.
30
2.8 Key trend six: emergence of DevOps

DevOps vendor impact
When issues about operating software are reported to the IT service desk, recriminations all too often result as software development and IT operations (ITO) specialists point fingers at each other regarding who caused the problem. This is attributable to a mutual lack of trust and siloed outlooks where the software development team is focused strictly on the software development lifecycle (SDLC) while ITO is focused on running systems. For example, at a large investment trading company the failure rate in the last mile was as high as one-third of all deployments due to the complexity of deployment scripts. Both groups have begun to attack the divide. With adoption of the ITIL framework for IT service management (ITSM), many ITO organizations have started to codify their processes for handling routine activities such as incident management, problem resolution, or server provisioning. ITIL adoption has often taken root as a result of top-down policies that are the by-product of compliance or cost-reduction initiatives. A more process-focused approach to their work has triggered overtures to software development. For example, it is logical that if you adopt an ITIL problem-resolution process, then part of that process may involve a checklist item for ensuring that service problems attributable to software are automatically fed to QA where they update defect-tracking processes, and so on.
With adoption of the ITIL framework for IT service management (ITSM), many ITO organizations have started to codify their processes for handling routine activities...
In parallel, a grassroots-oriented movement has emerged across all the silos (developers, systems administrators, release managers, DBAs, network engineers, and project and operations managers) with the aim of collaborating to remove the In practice, DevOps barriers. These dialogues are inspired by the Agile movement and emphasize face-to-face communication over documentation and throwing helps manage the over walls. build-scripting
process...
In practice, DevOps helps manage the build-scripting process, embedding into the application APIs that can trigger related infrastructure capacity and/or security change-management processes, and securing more cross-training between software development and IT operations roles. The latter builds on similar multi-disciplinary approaches that have become popular with Agile software development methodologies. Significantly, the term DevOps that describes this movement is fairly new, having emerged only in the past year. In 2011, the DevOps movement will begin to enter the awareness phase, with most ALM vendors starting to add integration points. In 2011, actual practice will be largely confined to early adopters.
There is an Agile connection: continuous delivery

The Agile communitys most recent trend of continuous delivery also ties in with the DevOps movement where the practice of deploying applications in the live environment can be as frequent as daily and even several times a day. Continuous delivery requires the type of collaboration that DevOps promises, making the release process efficient and predictable. For further reading we can suggest Jezz Humble and David Farleys book Continuous Delivery (Addison Wesley 2010): as well as pioneering the practice, the authors have written a valuable source book on the concepts and provide enlightening war stories.
31
2.9 Recommendations
Recommendations for enterprises
Continued pressure on IT budgets is continuing to drive IT organizations to revisit how they manage the application lifecycle. In this sense 2011 will be little different from 2010 because the same budgetary and time-to-delivery pressures will still be there.
Dont get caught up by fads, and adopt balanced approaches to Agile and cloud
Agile methodologies and cloud will be hot concepts in 2011. While Agile methodologies are hardly new, growing mainstream acceptance is also leading to more balanced approaches that mix and match the right principles to the environment. While the cloud is a newer phenomenon, it can also help software development organizations to take surgical approaches to improving and reducing many of the costs and lead time when automating critical aspects of the application lifecycle. IT organizations should approach innovations such as Agile and cloud selectively. The good news is that growing maturity with Agile adoption across the software development environment is yielding a new trove of best practices for mixing and matching the right methodologies for the problem. Although there is less experience with the cloud, the SaaS model is well suited for incremental adoption. Developers can get straight to project startup, avoiding the need for IT support or the threat of procurement delays. Costs are better managed, and the tool delivery mechanism via a web interface makes little difference to whether the host is on the cloud or behind the firewall. The only inhibitor is confidence.
IT organizations should approach innovations such as Agile and cloud selectively.
Plan for mobile development

As iPhone, Android, and other smart devices grow commonplace in 2011, there should be planning for development or extension of enterprise applications to mobile devices. Unfortunately, part of that process will dictate classic rich versus reach platform choices. The good news is that there are options for developing reach applications that address multiple devices, the bad news is that none of these is optimal. Development should be restricted to one or two platforms, or should accommodate the type of complexities to which web developers have been long accustomed.
Bake security into the application lifecycle

Grow more vigilant in incorporating security testing into the application lifecycle. Integrated solutions spanning static and dynamic testing, combined with analysis and reporting, are just beginning to emerge. For now, encourage adoption of a security-first mentality to any applications that are exposed to the Internet and make sure to get some developers properly trained in security testing. The next step, integrating with risk management, is still at least a year away as few if any tools are available to support it.
Approach ALM on its contribution to the business

Review your approach to ALM with the goal of being able to identify how software development contributes to your business, and how issues in managing the lifecycle of applications pose obstacles to meeting those goals. For example, communicate the need for managing Agile development as a way to reduce time-to-benefit or for managing software quality to provide more assurance of reliable software delivery and service. Senior management outside IT will be more receptive if ALM tool implementations are conceived as projects to help the business rather than simply improve the efficiency of software development.
32
Encourage DevOps collaboration

All too often, software that has made it through the gauntlet of development and testing can encounter delays when migrated to production, or can deliver unexpected performance or reliability issues once it is in production. The DevOps movement represents the tip of the iceberg in addressing bottlenecks in the release process. Encourage operations staff and developers to start a dialogue and help one another overcome last-mile bottlenecks. In particular, developers can help operators manage release and deployment phases and overcome the divergence of developers embracing change, and operators embracing stability. Exploring script-less release automation will have a huge impact on DevOps. If your organizations operations groups have active ITIL implementation efforts, explore how these processes can map to the application lifecycle.
Recommendations for vendors

The strongest ALM market trends are in the areas of supporting Agile and Lean software development (from Lean and, increasingly, the Kanban concept) and SaaS delivery. If you are an ALM vendor review your support for Agile and Lean, and if you are a consulting service in ALM, ensure you have the skills to support Agile and Lean development to help customers deal with the numerous issues that arise when, for example, interfacing Agile in non-Agile environments (a common scenario). Agile is a market where the traditional preference has been for low-cost or no-cost tools, and where highly functional tools are often associated with excessive bureaucracy and management overheads. However, enterprises that scale out their Agile efforts across multiple teams SaaS ALM delivery will require effective management, and for ALM vendors this will be the sweet spot of the market. is seeing rapid
Customers are demanding cloud delivery options
uptake...
SaaS ALM delivery is seeing rapid uptake and if you are an ALM vendor then review your options to compete effectively in this new market. Ovum believes that SaaS ALM will match on-premise purchases over next three years, particularly at the lower end of the market.
Accommodate mobile development

The market for write-once, deploy multiple times mobile development environments (MDEs) will be a growth area. Currently a distinct sub-market, the need for ALM will grow as the applications become more complex and the size of teams grows. MDEs are development-focused solutions that are related to traditional integrated development environments rather than ALM systems. As the activity in mobile enterprise application development gains pace, ALM vendors should consider their strategy. Ovum believes this sector will become significant, and the MDE players may well expand to ALM functionality that will compete with established ALM players. Conversely, for ALM players, developing automated testing capabilities that account for the varying client footprints and application consumption patterns should become one of the first areas for product extension.
Add or enhance your security-testing portfolio

The race is on to accommodate security testing. While a handful of vendors have acquired static and dynamic testing tools, most have not. The first step is to either acquire or form partnerships with the growing body of niche tool providers in this space. However, this is just the beginning. In the long run, a robust security-management offering will also require reporting, analysis, and integration with areas such as PPM, IT financial management, and enterprise GRC (governance, risk, and compliance) solutions.
33
Recast your offerings as business solutions

ALM is not simply a matter of managing software development. Because non-technology companies are not in the software business per se, software itself is only a means to a business end. Traditionally, software development organizations have not commanded the large budgets of their business counterparts for off-the-shelf software, and the lack of budget is even more of a constraint during challenging economic periods. Instead, by delivering solutions that address business problems, not siloed software development tasks, ALM vendors should find better success. Ovum has identified the following areas as ripe for ALM business solutions. IT-business alignment: ensuring projects deliver what the business needs by applying project portfolio management and requirements management. Software assurance: quality assurance and beyond, ensuring that reliable and secure software is delivered that is compliant with company policies and regulatory requirements, and performs to requirements. Enterprise agility: ALM needs to support the right process and methodology for the project type and team experience and skill set. This includes support that spans traditional to Agile processes, with hybrid and custom variations, on a large scale and with globally distributed teams. Development-operations alignment: The DevOps movement became noticeable in 2010 and aims to break the barrier between development and operations, ensuring that the last mile in application delivery does not become a sticking point. Product innovation: ALM and PLM are overlapping with the increasing use of embedded software in products. There is a need to manage embedded software as part of an ALM process, and this includes the latest generation of mobile smart devices exemplified by the iPhone and iPad.
34
Incorporating
OVUM Butler Group
CHAPTER 3: The Ovum SLM 2011 technology evaluation model
WWW.OVUM.COM
3.1 Summary
Catalyst
The goal of software lifecycle management (SLM) is to deliver development projects on time, on budget, and within acceptable quality standards, while also ensuring they add value to the business. An organizations senior management approves the budget for SLM solutions, so IT professionals need to communicate the reasons for SLM on the basis of how it delivers value to the business. Ovum has created an SLM framework that identifies the benefits of SLM by structuring it in terms of five business solutions.
Ovum view
Ovum sees several disrupters to traditional notions of managing the lifecycle of software development. First, there is the misconception that software development is synonymous with application development. Although the appetite for new applications has grown thanks to the emergence of a mass consumer market for software applications through the Internet and especially mobile devices, software development today is about more than applications. The role of embedded software in engineered products has grown to the point where it no longer simply enhances them, but defines them. As a result, the development lifecycle is becoming broader, extending beyond conventional applications as we know them. Embedded software is growing exponentially, from smartphones and tablets to complex systems-of-systems in engineering and manufactured products like automobiles. Secondly, IT departments and product manufacturers need to engage with the business side of an organization: this is essential if these two groups are to succeed in delivering what the business needs. The development of this increasingly complex software needs to be managed, whether it is for realtime systems or multitiered Web applications, and SLM is the means of managing that complexity and avoiding project chaos. Ovum provides a new business-oriented view of the management of software development, or SLM as we now to prefer to call it, that is in contrast to traditional application lifecycle management (ALM) thinking. It is based on five pillars that reflect a message that business executives can recognize as offering value to the business and forms a basis on which IT departments can have a dialogue with the business side of the organization. We expand upon these five pillars in our key messages below (also see Figure 2).
Key messages
ITBusiness alignment: SLM helps to ensure IT projects deliver what the business needs by improving collaboration. Software assurance: Businesses want software that is reliable, secure, and does what it claims to do an SLM approach can ensure these targets are achieved. Enterprise Agility: As Agile adoption grows, teams are using it in larger, more complex projects. SLM is necessary to support multiple, distributed teams in large enterprise projects. Delivery management: The last mile in the software development lifecycle can be the showstopper; SLM and deployment automation help resolve operations obstacles. Product innovation: Embedded software in products and systems has grown exponentially; SLM helps manage the complexity of embedded software projects.
CHAPTER 3: THE OVUM SLM 2011 TECHNOLOGY EVALUATION MODEL
37
3.2 IT-business alignment

SLM helps to ensure that IT projects deliver what the business needs by improving collaboration
SLM helps to ensure that IT projects deliver what the business needs. The following SLM tools help ensure that IT and the business are aligned: IT governance, portfolio project management (PPM), project management (PM), requirements definition, and requirements management. The traditional vendor approach in the ALM market was based on improving the software development lifecycle (SDLC). This was perceived as enough for customers to buy the product; customers would let SDLC improvements speak for themselves. The problem with this approach is that the business, which controls the IT budget, perceives no business value in SLM. The IT department has a different worldview from that of business people. Unless the IT manager has a business background, the IT department is illprepared to explain the business value of SLM. These issues are compounded by misconceptions, for example, that SLM is a product and not a process or philosophy, and that the product is a monolithic enterprise suite. The reality is that SLM is first of all a philosophy that believes software development is a complex activity performed by both humans and machines and needs to be managed. In the Todays SLM 1970s, the top computer scientists believed that all you needed to write good products are software was an exact and detailed specification once you had the spec designed with nailed down, everything flowed from that. The philosophy and understanding has improved a great deal since then. Todays SLM thinking embraces the improved third inevitable and unavoidable change that occurs continually through the SDLC; party tool one of the strengths of Agile methodologies, for example, is dealing effectively interoperability, with changes in requirements.
recognizing that customers have diverse tooling needs.
Secondly, the market rejects the rip-and-replace required in buying end-to-end ALM suites. Todays SLM products are designed with improved third party tool interoperability, recognizing that customers have diverse tooling needs. For instance, the development team will want tools that address immediate tasks (issue/defect tracking, Agile story creation and tracking, automated build and integration), while IT managers want tools to aid planning and execution. These customers have tool preferences and existing investments: SLM tools need to coexist in the same ecosystem; the idea of each vendor creating its own ALM ecosystem has not gone down well in the market.
IT-Business Alignment
PPM, PM, RD, RM
Software Assurance
QA, Testing, Security, APM
Enterprise Agility
PPM, PM
Delivery Management
Build, Realise, Deploy, DevOps
Product Innovation
SLM-PLM Integration
SLM Integration Framework

Workflow, Repository, SCCM, Collaboration, SDK/API
IDEs, Development productivity tools Cloud hosting, Test lab management

Figure 2: Ovum SLM solution model: business perspective and technology stack Source: Ovum
38
Thus, IT organizations must take a fresh look at SLM. First of all, the way the IT department, or system or product engineers, develop software matters: it impacts how and when value is delivered to the business the way the software team and the business communicate and collaborate and the alignment and adaptability of software to the business. The challenge for IT professionals and system engineers is how to communicate the above impacts to the business estimate value delivered to the business keep practitioners focused on delivering value. Therefore, SLM needs to be anchored to the business, while also serving the needs of developers. The market is changing: cloud services are creating new opportunities, which help to reduce costs, and SLM vendors are now offering SaaS options. Supporting Agile methodologies is now essential; however, SLM needs to be independent from any one development methodology.
...SLM needs to be anchored to the business, while also serving the needs of developers.
3.3 Software assurance

An SLM approach can ensure that software products are reliable, secure, and do what they claim to do
The software assurance (SA) concept aims to widen the remit of QA to ensure that the application performs not only reliably, but that it is secure and performs in production to the required level. From a business perspective, SA ensures the application does what it promises. The tool segments in SA include: requirements gathering and management, QA, testing, software security, and application performance management (APM). QA, the traditional quality management practice, is a subset of SA: QA focuses on ensuring the whole SDLC adheres to quality standards and that software testing is performed at various stages in the SDLC to monitor defect rates and performance reliability. Many aspects of QA are now rooted in statistical process control and have associated methodologies like six sigma and Lean thinking. SA expands on QA by taking responsibility for ensuring the product performs to expectations and business needs. It also extends into the production phase through APM. Closing the loop between development and APM has been sought for a long time but the silo nature of development and operations has made this difficult. SA is the means to providing APM feedback to developers.
Ovum believes that software security should be an integral part of development.
SA also adds a new concept: quality assurance of SLM. SA has the responsibility of ensuring the right process and methodology are adopted for any particular project to ensure business and development needs are met, and that the process is being implemented correctly.
Finally, SA adds application security as a front rank activity: Ovum believes that software security should be an integral part of development. For many organizations involved in development projects, security experts are not permanent members of the team and are typically brought in at the project start. However, software changes continually throughout the SDLC and so their expertise is often unavailable when needed. Security skills need to be available within the team, security concerns need to be built in to the architecture, and security issues need to be included in the done list at the end of an Agile iteration when the various quality checks are assessed. The security competency of the development team needs to be raised a notch. SA is a holistic approach that ensures delivered software will work as required without any surprises; it also raises the level of QA and covers security and production performance as part of the overall assurance.
39
3.4 Enterprise agility

As Agile adoption grows and teams apply it to larger projects, SLM is necessary to support distributed teams in large enterprise projects
Enterprise development processes span the processes and methodologies for developing software. Since the early days of computing and the waterfall process (a concept borrowed from manufacturing), the software community has matured its understanding of what works best for software: today we have Agile methodologies like Scrum and Extreme Programming, and Lean thinking influencing software development with techniques like kanban and value stream analysis. The critical phase today is on scaling up Agile so that it can benefit a wider range of projects. Agile methodologies were created in small co-located teams, but as Agile principles and practices are applied in ever-larger projects there is a need to support larger, globally distributed teams. Businesses will want to increase the success rate of large-scale projects as these have a history of failure using traditional processes. Specific tools that apply in this category include PPM and Agile PM. The challenges in large-scale, distributed development can be mitigated through automation (this is one area where automated tools are indispensable). The Agile methodologies must also be adapted to fit the needs of enterprise projects globally distributed stakeholders and daily Scrum stand-up meetings that involve multiple time zones, use electronic whiteboards, and so forth.
The need to interface Agile with non-Agile teams and groups is also necessary in largescale projects...
Agile processes are also increasingly being adopted in systems engineering and manufacturing companies, where the approach is showing benefit in bringing stakeholders across systems, product, and software groups closer together and is reducing the communications gap. Such environments may already be highly regulated with compliance laws in place, or have stringent health and safety controls these aspects need to be made a part of the requirements feeding into the Agile stories.
The need to interface Agile with non-Agile teams and groups is also necessary in large-scale projects and this means that all teams will need to adjust their planning and processes. For example, Agile teams expect or need much greater communication with the other teams engaged in a project and where this is not possible, alternative arrangements, such as the use of proxies, need to be made. The role for SLM in large-scale Agile projects is critical: the complexity of the project and the greater process discipline demanded by Agile methodologies require the support of automated tools, e.g. distributed change and configuration management systems, automated build, and system integration testing. In addition, architecture and modeling is normally not associated with small-scale Agile projects in IT departments, whereas in systems engineering, modeling is a core activity, and in large-scale projects an architecture blueprint for the whole project is essential in providing a high-level view. Therefore, addressing how architecture is to be approached in the project is a vital part of scaling up Agile to enterprise projects. Figure 3 presents a view of enterprise Agile practices in SLM. The central activity (or heartbeat) is the Agile iteration (also called Sprint in Scrum), converting stories into code, and the activities and tools supporting the iteration are identified and surround it. This circular process is in contrast to the traditional waterfall process.
40
Proje c m a n t portfolio agem ent Proje c man t agem ent
tion Issue track ing Colla bora
Know l base e d g e
Architecture Design
Application performance management Whiteboard/kanban IDE
Modeling
Story
Iteration
Code
Software change and configuration management Workflow engine Cloud lab services
Security Metrics & analytical intelligence

Requ ir m a n ements agem ent irem e defin nts ition Prog ram and ming testin g QA DevO ps
Figure 3: Enterprise Agile SLM
Requ
Source: Ovum
3.5 Delivery management

SLM and deployment automation help resolve last-mile obstacles
Businesses rely on IT more than ever before, with distributed Web-based applications being the norm. These applications raise the complexity of deployment, with multiple touch points to existing systems and service layers. At the same time, the churn rate of requirements is greater than before, with Agile processes rapidly introducing changes. With these trends we see developers dealing with change, and the higher risks associated with them, at continually faster rates. Conversely, operations staff, with their focus on reducing risk, want stability in the production environment. Developers, QA, database administrators, operations staff, network engineers, website designers and administrators, and managers commonly work in silos, which impedes communication this likelihood is greatest in large enterprises where job roles are more narrowly defined. For operations staff to deal effectively with the deluge of changes in the form of patches and deployments, they need closer collaboration with all the stakeholders, and they need better tools to help manage the complexity.
41
DevOps has grown to address these issues: operations staff who have to manage thousands of lines of deployment script codes are effectively turning into part-time programmers, and the Agile message and philosophy of closer collaboration and communication has got through to them. DevOps sits in the middle of Agile, cloud services, ITIL version 3, and ITSM in terms of process. The DevOps ethos adheres to Agile values as expounded in Extreme Programming: Communication: Talk face to face instead of relying on documentation. Simplicity: Keep it simple. Feedback: Give feedback frequently and rapidly. Courage: Be bold! Take initiative, innovate. Respect: The people on the other side of the wall are just like you. Trust: Cultivate mutual trust. Transparency: Elevate problems through visibility. The challenge that DevOps addresses is the last mile the transition from software e/merging from developers and QA and successful deployment into production. The last mile is also replete with pitfalls; some of the areas where problems may occur include: configuration file changes third-party binary files and class libraries database schema scripts build scripts test harnesses production environment OS changes difficulty understanding script when the key operator who wrote it is away One useful deployment test (attributed to Mary and Tom Poppendieck) is to ask: If you change one line of code, how long does it take to deploy? Is the operation repeatable? Is it reliable? Answering these questions and managing the above pitfalls is where DevOps has the most to offer. Delivery management is a pillar in Ovums SLM in recognition of the importance of DevOps to the successful deployment of software and to the recognition that the lifecycle extends into production.
3.6 Product innovation

Embedded software in products and systems has grown exponentially; SLM helps manage the complexity of embedded software projects
Embedded software today forms a substantial part of many products, including those not historically associated with software, from automobiles to washing machines. Some estimates put the growth rate of embedded software between 10% and 30% (BITKOM, Germany, 2008). Figure 4 shows the growth rate of software embedded in a variety of products since the computer era began. Not only is the range of products using software increasing but the extent and way in which software advantages are harnessed is evolving. Many mechanical and hardware-fixed elements are found to be replaceable by software, offering greater flexibility and fine tuning. The size of these components is leading to OSs (typically Linux) to be embedded with the systems, and this allows the software to be more easily programmed in and updated. A defining characteristic of embedded systems is that the user interacts with a hardware/mechanical object and not directly with the software. However, the trends essentially place a computer in a box with some defining added components that make a mobile phone different from an aircraft navigation system: this phenomenon of greater computational capability in smallersized devices is also propelling the embedded software revolution.
42
Size in object instructions
10
Space flight control Switching systems Automotive embedded SW Linux kernel
10
107
10
1950
1960
1970
1980
1990
2000
2010
Years
Figure 4: Growth of embedded software Source: Christof Ebert and Capers Jones, Computer, IEEE, April 2009
Key aspects that differentiate embedded software from other software include: the software interacts with the system or user in realtime and therefore needs to manage events; the stringency on reliability and safety is greater than with typical business applications (there is no opportunity to reboot when the software is controlling the safety of a human life); some of the emphasis on reliability concerns security (the system must be proofed against security breaches which in turn can impact safety); resources (the surrounding environment may constrain the degree of computational hardware that can be fitted in to run the software); and robustness to change (the software needs to be proofed against a changing environment). The greatest benefit of embedded software is in product innovation. As The greatest benefit software usage increases, new creative opportunities arise for improving of embedded the product performance or human-product interface experience. For software is in example, augmented reality (adding to reality) techniques are being developed to move the car dashboard to the windscreen in a head-up product innovation. display. First of all, such developments improve car safety as the drivers eyes continually stay on the road, and the head-up display can present only the essential information, minimizing distraction. Such developments also improve safety by helping navigate the vehicle during periods of poor visibility using radar sensors and GPS. Routes can be displayed on the windscreen replacing the satnav device screen. Software creates these possibilities and innovation is the hallmark of embedded software. With the rise of embedded software use, there is a need to manage its greater complexity during development; SLM becomes essential in that capacity. SLM automates the mundane and tedious activities that by their nature make developers prone to errors. The problem now is that as SLM makes inroads into systems and product spaces, the occurrence of manual steps is transferred to the interface between SLM and product lifecycle management (PLM). PLM systems are typically used to manage complex systems and engineered products, managing the exchange of artifacts and people between the different groups: software, systems, and product/hardware. The ideal is an integration between SLM and PLM, as SLM expands beyond enterprise IT and into the systems and product-engineering space. Integration between SLM and PLM helps organizations streamline their processes and leverage skills and tools. Multiple teams are involved in product development and manufacturing; the mock-ups used rely extensively on software modeling; and software teams need to exchange project artifacts with systems and product engineers and algorithm experts. Thus, integrating SLM and PLM systems will be increasingly important.
43
A likely scenario in Ovums view is that a PLM vendor will acquire an SLM vendor to create a seamless integration path for product and software developers. Within the SLM market a select number of vendors are active in the systems and product-engineering space: IBM, which has a strong presence in the systems space after its Telelogic acquisition, is better placed than other SLM vendors to exploit this growing market segment, but other vendors such as MKS, Polarion, and Serena are also making inroads.
3.7 Recommendations
Ovums new SLM framework provides a basis for evaluating the tools that can deliver value to the business and reduce development complexity. Some framework elements flesh out important themes that have been on the agenda for a long time, such as IT-Business alignment, while other themes are new, for example the concept of SA and its assurance of security being built-in during development, and applying quality control to the development process selected for a project. The need to scale up Agile to large-scale enterprise projects is another area where SLM can offer benefits. Enterprises should implement SLM as a philosophy and support their IT professionals with appropriate SLM solutions: choices available today span open source for the more developer-oriented tools, to enterprise solutions, whether delivered on-premise or as SaaS. The SLM tools market is increasingly offering integration points with a host of third-party tools: this is what customers are demanding, and tools that offer a high degree of compatibility in a best-of-breed environment will offer the best return on investment.

Ovums name change of ALM to SLM reflects an emphasis on the business value in developing software. We believe that the name SLM resonates better with the business side of an organization. We also recognize that in the embedded software world the software is no longer an application in itself, but is a part of a product or larger system. The growth rate of embedded software is exponential and is therefore an important market for vendors; this market needs SLM to manage complexity. Thus, rather than be tied to applications, the management of software development and its lifecycle needs to grow broader, hence a transition in our terminology to SLM and to encompass all types of development work.
Alternate view
There is a parallel in Agile processes, and Lean kanban in particular, where software continually flows in as stories/requirements, and flows out as code. The focus on any one distinct application gives way to a Lean software factory where production lines run in parallel, delivering software pulled by demand. The name change to SLM ties in to that emphasis on software that is required for multiple types of uses, from applications to embedded components. The name change also marks a transition from the old ALM that promoted monolithic suites to the modern SLM products where solutions can better coexist with other tools, whether they be offered on the cloud or on premise.
3.8 Breadth of lifecycle discipline coverage

Below are descriptions of the first section of entries in the Ovum Features Matrix evaluation.
IT governance
Corporate governance applied to IT has risen in profile as a result of regulatory compliance requirements. In the case of distributed loosely coupled enterprise architecture such as SOA the IT governance provides a services and components registry, policy management, and compliance monitoring.
Application project portfolio management

This involves the management and tracking of progress of resources and finances for multiple projects. It is business-focused for prioritizing projects and aligning with business needs, while keeping track of risk and rationalizing demand.
44
Project management
This includes functionality for managing the resources, plans, and schedules of a project, and might include budget management. For Agile projects this will involve managing user stories and developer tasks, traditionally a requirements-management task.
Requirements definition
This functionality enables business users and analysts to elicit requirements with the help of techniques such as stakeholder interviews, use cases, and scenario modeling, using text and graphical interfaces.
Requirements management
This is a tool for storing and defining requirements interdependencies, communicating requirements to stakeholders, and tracking and managing requirements change through the lifecycle.
Model-driven development
This involves the use of visual models, usually based on UML or DSL. Some tools allow models to be directly executed for validation purposes or used to auto-generate code.
Database modeling and design

Capability to model entity relationships, and manage schemas and other functionality to design and implement back-end data repositories for applications.
Change and configuration management

CCM functionality enables users to work on their own version of source code and work artefacts, track changes, and manage different versions, facilitating overall change management in a development process. Advanced tools provide configuration management, managing the documents and files related to project operations, security, test environments, and other associated artefacts.
Build management
Build management covers the complexity of managing many files and dependencies and their compilation needs, as well as build tool scripts, but not the actual build tools.
Deploy and release management

Deployment management is concerned with managing the target environments from test rigs to production rollouts, including managing many different operating systems and versions. Release management covers managing the different releases, versions, updates, and patches.
QA management
QA management involves managing different types of tests (manual/code review, unit, integration, and systems), collecting execution logs, providing automated continuous testing, managing test metrics, test planning, resource allocation, and test result reporting. It also provides a repository for test cases and other test data storage, and increasingly covers risk-based testing and risk analysis.
Defect and issue management

This provides workflow, ideally integrated through the ALM suite with PM, RM, and TM tools, for defect communication and reporting, as well as interfaces for defect capture and tracking through the lifecycle. It may be integrated with enterprise help desk and ITSM systems.
Application performance management

This helps IT departments meet SLAs by collecting live application performance data through monitoring application servers and using agents to monitor applications, networks, and databases. Enduser experience monitoring is emphasized, as are virtualized environments. It includes reporting to facilitate trend analysis and performance tacking, and helps troubleshooting and root-cause analysis during failures.
45
Software asset management

Tools for policy-based software governance enable organization-wide software asset tracking, controlling, monitoring, and reporting usage, and also provide software contract management capabilities.
3.9 ALM integration framework

This describes the second section of entries in the Ovum Features Matrix evaluation.
General
Does the suite use a common repository? A common suite repository ensures that all artefacts are centrally stored, to enable better integration between component modules and therefore between various development lifecycle phases. Does the common suite repository support external federation? The repository should be able to connect to, and retrieve data from, external data sources for management or reporting purposes. Is there any workflow engine used within the suite? The qualitative question asks Specify where it applies in lifecycle, degree of automation, and is it visual? Key areas for this are requirements, project process, and change and configuration management. Analytic reporting on ALM project health is a reporting facility with built-in reports on project status and progress. Customizable reports and BI analytics capability is even better. Native IDE integration with Eclipse and/or Microsoft Visual Studio are the most popular IDEs and native integration is a minimum. Developers can invoke ALM tools and functionality from within their preferred development environment. Integration with other IDEs and support for additional IDEs. OS platform support of tool operation where the ALM solution is deployed on any operating system platform of choice. OS platform support of application target covers whether the solution can be used to develop applications for any target platform. Are mainframe and distributed platforms covered by same tools? If not, then describe which tools apply to which category. Suites that provide heterogeneous environment support from the same tools reduce administration overheads. Cloud readiness/support. Answer separately in case of hosted (SaaS/PaaS) tools, and development for the cloud. Web-based client access. Is the entire suite functionality accessible through a web browser to support distributed developers and teams? Agile support: Three-part question: N Continuous integration: continuous build and integration of modules and systems. N Continuous testing (automated build, deploy, and regression/unit tests). This advanced functionality performs regression testing and has two contexts: continuous unit testing uses idle machine time to take a snapshot of work in progress, builds, and automatically runs unit tests. Continuous mainline testing runs regression tests against an integration of mainline code. N Continuous deployment performs continuous integration and testing and also continuous deployment into production environment Can the process guide be accessed within ALM system at point of application? Process/methodology guides can be accessed directly from ALM tools and opened at the relevant sections. Can the user switch between the relevant point in a process guide and tool in either direction; e.g. clicking in the process guide opens the relevant ALM tool at the relevant function.
46
Product Lifecycle Management integration. Is there any integration with PLM products and what is the extent of the integration? Is there support for CAE/CAD systems? Mobile Development Environment support: What integrations are there with platforms dedicated to mobile application development.
Internal tool interoperability (within ALM suite and across other your products)
Does the application PPM tool read the project management tool? Indicate if these are the same or separate products. Specify the nature of integration. Does the project management tool read the requirements management tool? Specify the mode of integration (SOA, data replication, file sharing, UI-level integration). Does the project management tool read the software change and configuration management tool? Specify the mode of integration (SOA, data replication, file sharing, UI-level integration). Does the project management tool read the quality management tool? Specify the mode of integration (SOA, data replication, file sharing, UI-level integration). Does the requirements management tool read the quality management repository? Specify the mode of integration (SOA, data replication, file sharing, UI-level integration). Is there two-way read and write between the Quality Mgmt. tool and Defect Tracking tool? Specify the mode of integration (SOA, data replication, file sharing, UI-level integration). Is there a tightly integrated platform? 1: with entities and artefacts traced across whole lifecycle; 2: can new events and workflow automation be triggered by artefact relationship events. Were looking to see whether this solution has actually been built from ground up as an integrated whole rather than loosely coupled system so that 1 and 2 are possible.
External tool interoperability (within third-party tools)

Is the API published? A published API enables developers to integrate existing enterprise applications with the ALM suite. Is the API royalty-free? The solution API is free of charge or royalty fees. Is the API web services-based? A web services-based API makes a good fit with a local SOA IT environment. List direct connectors to third-party ALM tools. Any direct connectors to third-party ALM solutions are to be listed at the interface as well as repository level to facilitate interoperability and data exchange; the connectors should provide read-and-write access. Is there an SDK? An SDK facilitates client-side custom development and application scope and functionality extension.
Collaboration
Is there built-in instant messaging (IM)? Is IM built into the solution for real-time team collaboration? Can work artefacts be linked to documents? Allowing any stakeholder to attach notes and documents to work items makes information and documentation available where it is needed and easily found. Can source code, scripts, change requests, and tests be annotated/commented? Tracking changes or providing feedback is best if code, scripts, change requests, and tests can be annotated with relevant stakeholder comments. Is collaboration wiki-based with read-and-write capability? The level of collaboration can be enhanced with the help of an in-built wiki that allows users to create, read, and edit pages. It also serves as a knowledge repository. Is process/methodology authoring and publishing wiki-based? Are the author(s) of artefacts displayed? Information about the author of an artefact helps in team work.
47
Change and configuration management (CCM)

CCM: is it native or via a third party? If native: N Does the tool go beyond just software code artefacts and is it applicable to other artefacts and documents? N Workflow for access management involves customizable visual workflow for regulating and authorizing access to artefacts contained within the repository. What is the maximum number of concurrent users? N Automated branch inheritance is where new child branches auto-inherit from parents. N Explore tool following branch inheritance can navigate visually through branches along inheritance tracks for ease of use. N Move and rename branches, can edit branches with ease. N Visual branch and merge enables users to create and merge code branches using drag-and-drop techniques, simplifying the process of managing different versions and the subsequent integration. N Visual file history analysis and differencing provides a visual change history comparison tool. N Apply policies at check-in including tests must be done and code passed, or code review done. N Audit tracking with audit trails of every change committed, and change history for artefacts and documents to support organizations regulatory compliance efforts. N File and folder versioning to provide easy management of workspace. Tool availability. State whether on-premise or cloud (SaaS) offering.
Architecture
Graphical architecture modeling may be proprietary or Object Management Group MDA standard, describe which. Is the model executable? List modeling languages supported (Unified Modeling Language, Domain Specific Language, other). Describe languages supported. Synchronization between models and code. Is there any model to code transformation? Provide further details if a comprehensive synchronization between code and model is available.
3.10 ALM business solutions

This describes the final entries in the Ovum Features Matrix evaluation.
IT/Business Alignment
Application Project Portfolio Management
Multiple projects management includes tools for managing multiple projects, helping prioritisze projects in the pipeline and creating business cases for proposed projects. Offers monitoring project health through dashboards and portfolio-wide resource management. State whether it supports 10s, 100s, or 1000s of projects. Critical path analysis identifies key resources, how long they require to complete a task, and how they connect in a process. It uses a mathematical algorithm to identify the critical path, defined as the longest path of planned activities, and which helps define the longest possible project completion time. Can Pert chart be created? Financials Management includes capabilities for tracking and communicating costs, time, and investments for overall delivery of projects in the portfolio, as well as cost versus benefit analysis features.
48
Planning and scheduling. Describe whether single or multiple projects are coordinated, and key features, including creating Gantt chart. What-if modeling provides features for modelling scenarios and analyzing outcomes. Resource management where financial, human, and system resources are allocated to the portfolio so that projects in the portfolio achieve their objectives on schedule. Demand Management includes features for managing demand or requests for projects. Risk management involves managing risk by the analysis of tasks/jobs, the relative risk of damage, and the likelihood of occurrence. Is a risk graph created? Can risk priority numbers (RPN) be used? Reporting provides portfolio-wide visibility to business stakeholders through reporting and metrics tracking features such as pre-built, customizable template-based status reports and dashboards. State whether the solution may also include a report designer tool for creating custom reports, ad hoc reports, and/or has pre-built reports. Tool availability. State whether on-premise or cloud (SaaS) offering.
Project management
Supports a variety of processes/methodologies. List process templates and guides supported. Describe the extent of support. For example, template-based support for creating, modifying, implementing, and monitoring process instances within a development project. Use-case and task estimation. Tools for estimating effort and resource requirements of tasks; such tools may make use of statistically-collected data to forecast the level of effort required. Describe whether Agile effort estimating is supported with points voting system. Flexible tracking of work effort. Can work effort be tracked as points, lines of code, time, function points, or other method? Resource allocation and scheduling. Tools and interfaces for allocating resources to and scheduling execution of project-specific tasks and activities. Manage backlogs: product, release, project, Sprint/iteration. In Agile development, backlogs are lists of requirements (or user stories), and depending on the level, lesser or greater detail is provided. Hierarchical stories/tasks structure. For supporting Agile development, can user stories be structured in hierarchies and how deep can these be? Are they unlimited? Prioritization of user stories/tasks provides an Agile development support feature for easily changing the prioritization of user stories and tasks. Customized cards allowing comments/annotations/flags, supporting Agile workflow board or Kanban, with electronic cards that allow developers to annotate or comment on the task/user story and also add flags to indicate various conditions such as late, needs rework, and high priority. Electronic workflow whiteboard is the electronic equivalent of a whiteboard with workflow columns and swimlane features. Specify whether Kanban limits for work in progress is supported. Gather metrics. State whether from own plugged-in tools or third-party tools. Specify which metrics. Agile charts. State whether velocity and burn-down/up charts can be created out-of-the-box. Any other charts? Project intelligence (BI for application development: real-time analytics and reporting). This is essentially BI for application development, offering real-time reporting and deep analytics with usercustomizable reporting. Project reporting: metrics, statistics, charts, and dashboards. Project-specific reporting capabilities include data collection through metrics tracking and statistical techniques, reported through status charts and dashboards with drill-down capabilities for self-service investigation. Specify if standard, template-based reports can be created. The solution may also provide real-time metrics-tracking, analysis, and reporting capabilities.
49
Report generation supports custom report generation mining the ALM repository by a team or individual. Tool availability. State whether on-premise or cloud (SaaS) offering.
Semantic analysis of text for ambiguity, omissions, and compliance. Supporting good requirements documentation standards by identifying commonly used vague and ambiguous terms, omissions of known terms, and aspects related to compliance, such as requirements to industry quality standards. Specify what quality metrics, if any, are provided. Customizable user interface labeling. Can the product be used for ALM and PLM applications and allow labeling of the interface to be customized, such as through a glossary of terms. Glossary. An agreed set of language use, terms, and standards to be defined and used by all project stakeholders for consistency and supporting collaboration. Application domain experts may define application attributes using terms from the application domain. Convert unstructured input data to structured formats. Automated conversion from Visio, Word, Excel, and other documents to requirements structure. Output formats. Specify whether supports structured and semi-structured requirements outputs and what type. Generate workflows: a) logic and application process workflow and specify how this provided; and b) workflow for UI rendering. Traceability (for audit trails) supported. Can the origination and authorship of requirements be recorded? Is this feature protected for audit purposes? Multiple definitions versions supported. Holds multiple versions and histories of requirements definitions. Specify what multi-lingual support exists. Collaboration capabilities. Allows users to add comments and feedback, and specify what is provided, such as wiki, email, or IM. Tool availability. State whether on-premise or cloud (SaaS) offering.
Requirements interdependency mapping. Tool capability to create links between related requirements and define the nature of the relationship. Requirements tracking throughout the lifecycle. Capability to track and maintain requirements throughout the application lifecycle, mapped to downstream artefacts such as models, code, tests, and released applications. Baselining. A state preservation and change management technique that involves taking a snapshot of a projects requirements and storing it as a baseline. Stakeholders and change managers can compare different baselines to measure progress or change. Change impact analysis. Enables users to analyze requirements through impact analysis, monitoring the effect of changes on interdependent requirements and identifying which artefacts are affected. Coverage analysis. To ensure that all stakeholder requirements are met, coverage analysis shows that all requirements are linked to delivered software and that tests have been conducted and passed. Requirements-based test case generation. Enables automated test case generation from requirements for testing code against the actual requirements. Advanced solutions take the user through steps to produce complete tests usable in automated testing. Less advanced solutions create template test cases that the users fill in manually.
50
Visual requirements representation. The graphical representation enables users to easily understand the most complex set of requirements. Visual representation lends itself to further use by model-driven development techniques. Is the visual representation synchronized with the text requirements? Is there automatic cross-generation between textual and graphically captured requirements? Reporting. Reporting capabilities at multiple levels from executive visibility into the requirements engineering process to day-to-day management purposes. The solution should provide pre-built reports for metrics and attributes such as the percentage completed, tested, and implemented. Tool availability. State whether on-premise or cloud (SaaS) offering.
Software assurance
Software assurance integration
Electronic signature support supporting auditing and sign-off processes. Requirements/QA integration: N Are requirements and test plans maintained in same or federated repository? N Test cases auto-generated by requirements. N Automated validation of test plan coverage of requirements. N Conduct gap analysis of test plans through requirements tool. Application performance management integration. Does the tool use performance testing scripts to do synthetic tests of actual performance? IT service desk integration auto-populates defect/issue-tracking with code-related trouble ticket incidents/problems. PPM/QA integration. Can test results and test coverage be viewed by project within the PPM tool? Defect-tracking/task-management integration: N Can defect-tracking tool autopopulate task lists for specific team members? N Automatic generation of log entries from tests automatically generates log entries from test execution, and creates defect entries for failed test cases. N End-user access for log creation provides an interface where developers and testers can log defects and attach related documents, or links to the build, test environment, and requirements. N Link back to related resources where Log entries are link back to the origin of issue. N Issue workflow vetting provides a workflow for dealing with issues and allows validation points to be created such as authorization to carry out certain steps.
Quality management
Track tests by requirement supports requirements-based testing Test-case branching and reuse is essential for the best use of resources when managing large testcase sets. Automation in use-case test authoring and refactoring provides features for creating test cases from application use cases. Ideally, the solution has auto-generation of test cases from use-cases. Lesser tools only automate template test-case creation that needs to be manually populated with test details. If the project requirements change or new requirements are added then the use-cases change, and linked tests need to be refactored. This feature automates the refactoring of tests linked to use-cases. Historical-based analytics and reporting provides functionality for historical analysis and trend reporting, as well as current test performance, supported by a repository for storing test-cases and results. Can product features/tasks be represented on a risk chart? This provides support for risk analysis. Manage combinatorial testing supports pair-wise test management, and higher tuples.
51
Scheduling and automated execution of tests. Manage test executions for batches and continuous integration and testing. Integration with web services testing tools. Describe which tools are supported. Test data management. Specify whether data can be generated and how private data can be masked. Unit test management. Is test-driven development supported and how are unit tests managed, such as changes to unit tests? Manage test lab environment includes features for setting up a test environment and its management. Users must be able to create and provision, modify, and delete test environments as appropriate. Describe on-premise and cloud-supported options. Third-party QA and requirements managment tool integration. Which third-party tool integrations are supported (list)?
Enterprise Agile
APPM/Agile planning integration. Are Agile project KPIs visible inside APPM tool? Multi-project Agile planning. Do Agile planning tools have visibility to multiple Agile projects? Multi-project Agile resource balancing/allocation. Can Agile planning tools rebalance resources across projects? Compare burndown/up rates by team? Multi-project dependency management. Does Agile planning track relationships between projects? Can projects and/or iterations be prioritized? Multi-project risk management. Assess risks for completing specific stories within project iterations/timelines. Support for process impediments. Explain how issues/impediments are addressed with tooling support. Support for hybrid Agile and other processes. Explain how process-agnostic the tooling is, and how optimum the tool support is for any distinct process. Agile/QA management integration. Compare defect rates and other KPI across teams/projects.
Application security
Integrate with static testing (code-scanning) tools. Describe support for static testing tools. Integrate with dynamic testing tools. Describe support for dynamic testing tools. Security issues auto-populate defect-tracking system. Does the tool integrate with the defect tracking tool to highlight security issues? Security tracking integrated with trouble-ticketing system. Does the security tool track security issues in the ticketing system? Security issue impact analysis. Can an impact analysis be performed on security issues? Risk prioritization of security issues. Can tasks be categorized or characterized by security risk? Reporting: security assessment. Is security reporting bundled as core functionality?
Delivery management
Continuous integration: N Private workspace: automated, continuous builds, integrating a copy of mainline code in the private workspace. Continuous integration techniques help developers overcome code change integration pitfalls. N Public workspace: automated continuous builds working on the mainline code ensure that the mainline is always working code and free from defects.
52
Build rollback. Functionality that allows automated rollback of failed or incorrect builds. Dependency tracking. Dependency-tracking feature enables users to view the dependencies for a particular build. During builds the dependencies are automatically managed so that components are ready for integration in the correct order, avoiding failure or an incorrect version being integrated. Intelligent build. Only needs to build files that have been changed Build validation. Provides a feature that enables users to validate code before a build is commenced to ensure all libraries, modules, and components are valid. It is particularly useful for large project builds that can take significant time, avoiding failure due to a dependent part not being ready. Integration with Ant or Maven. Popular build automation tools such as Ant and Maven are directly callable from within the build-management tool. Deployment management. Management of automated deployment of releases to various environments including developer, test, production, and their variations. Release management. Management of releases for multiple version streams, for multiple machine builds, and multiple developer, test, and production releases.
53
Incorporating
OVUM Butler Group
CHAPTER 4: Agile and Lean process adoption for software development
WWW.OVUM.COM
4.1 Summary
Impact
Agile software development methodologies and processes are now found in the mainstream. However, the question of which particular process to adopt is still complex, reflecting the challenge of matching work style to project type and organizational culture.
Ovum view
Branded Agile methodologies are useful for focusing on the core principles and knowledge of Agile. The most popular methodology, Scrum, has a Scrum Master certification program, which has become a mini-industry, but this has a downside as it fixes the methodology and restrains its evolution. In practice, organizations adapt Agile to their needs and this is in keeping with Agile principles. At large scale Agile adoption is also a challenge that most branded Agile methodologies do not address. Agile culture is distinct, as it is highly team oriented and collaborative, with a macro-management style. This does not often match the organizational culture, which can be categorized into four types: controlling, cultivating, competency, and collaborative. Actual organizations will have all four characteristics, but there tends to be one dominant type. Changing the organizational culture to fit the software development teams needs is unrealistic and, even if it were attempted, could take many years. Therefore, we advise a more pragmatic approach that works with the strengths of the organizational culture. Kanban is a Lean development practice that can be used in Agile projects or in combination with traditional work processes as a way of adding visibility to work and controlling the workflow. As there are only three rules to Kanban, there is no training necessary; the board itself becomes a learning ground as the team modifies it to fit the project needs. Adopting Kanban can be a first step to Agile process adoption. The first principle associated with the Agile manifesto (created in 2001) says: Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. The idea of continuous delivery has only become a reality recently, with some estimating 5% of Agile projects to be practicing the technique. Continuous delivery means continuous build, test, and deploy in one automated step. Of course, if the testing shows faults, the process iterates around fix-build-test, but the concept is that working software is rapidly pushed into production. This approach will not suit all projects, but where speed of correction is of the essence, the automation available today can make this approach a reality.
Key messages
Match development to organizational culture and project type. Kanban adds new flexibility to development workflow management. The introduction of continuous delivery and deployment represents a new phase in Agile development. Agile and Lean practices will embrace large-enterprise development projects.
CHAPTER 4: AGILE AND LEAN PROCESS ADOPTION FOR SOFTWARE DEVELOPMENT
57
4.2 Match development process to organizational culture and project type

A pragmatic approach to finding a matching methodology and process
Agile software development methodologies and processes are now found in the mainstream. However, the question of which particular process to adopt is still complex, reflecting the challenge of matching work style to project type and organizational culture. There are many types of projects with varying characteristics; for example: Custom in-house rapid delivery for small jobs to long-term projects. Maintenance and support operations random demands, from helpdesk to scheduled deployment tasks. Independent software vendors (ISVs) the whole business is dedicated to software production. Embedded software increasing use of software to replace hardware in products, from vehicles to white goods, but experience with software system management is varied. Mission-/life-critical systems safety-first software for medical devices to support life systems, with testing and risk analysis major efforts. Small-scale to large- and mega-scale many projects are now global, with various functions performed by outsourcing partners. Internal/external customers expectations, service-level agreements, chargeback, and other contractual differences play a part, plus the ease of access to the customer and end users. Public sector/nonprofit the need to fit into established procurement processes and governance controls. The process selection is also dependent on factors such as the skills range of the team, the resistance to change where plans to roll out new processes apply, and the culture of the business and IT department. Therefore, there is no simple formula for matching project type to development process. A rough rule to apply is that where user requirements are known and understood (often where the business and IT people are one and the same), it is suitable to adopt a waterfall-like development process. Waterfall-like processes are simpler to implement, require less Ovum believes Agile overhead, and are easier to monitor. However, the reality is that most software projects have too many unknowns in the requirements (even and Agile-like where the customer is convinced they are known), and are liable to change working styles will because of external business or market forces. Therefore, Ovum believes predominate. Agile and Agile-like working styles will predominate. We use the term waterfall-like in this report as we do not recommend ever adopting a purely waterfall process it has too many intrinsic faults to be justified. Thus, a waterfall-like approach may use the basic stage-gated approach for requirements, design, and construction but adopt an Agile-like testing regime, in which testing is taken upstream and performed at all stages (such as requirements testing and design testing) in order to mitigate against the disadvantages of late-stage testing in orthodox waterfall. Having less reliance on documents and more face-to-face discussions between stakeholders is another Agile practice that can be adopted in waterfall-like approaches. Agile processes and methodologies represent a body of knowledge from which many Agile practices can be chosen, fine-tuned, and customized to the needs of particular projects. Various branded methodologies focus on particular segments of the development stack so, for example, Scrum is good for project management, while Extreme Programming (XP) is good for core development. There is thus a need to build up a set of approaches to create a complete process. Branded Agile methodologies are useful for focusing on core principles and knowledge, and Scrum Master certification has become a mini-industry, but the downside is that this fixes the methodology and restrains its evolution. In practice, organizations adapt Agile to their needs and this, in itself, is being Agile and agile. At large scale, Agile is also a challenge that most branded Agile methodologies do not address.
58
Working with the strengths of the organizational culture

Agile culture is distinct, as it is highly team oriented and collaborative, with a macro-management style. This does not often match organizational culture, which can be categorized into four types: collaborative, controlling, cultivating, and competency. Actual organizations will have all four characteristics, but there tends to be one dominant type. Agile culture is Changing the organizational culture to fit the software development teams distinct, as it is needs is unrealistic, and even if it were attempted, it could take many years.
highly team
A pragmatic approach is to better understand the culture as it is, with both oriented and its weaknesses and strengths. Work with the strengths so the Agile collaborative, with a development team is gaining optimum benefit from what the organization can offer, and try to reduce the cultural barriers by attempting to understand macro-management style. how other groups in the organization see their world (and the outside world). Every group has an in-world and out-world view; to some extent these views will not match reality. It helps to attempt to understand another groups in-world and outworld views in order to reduce the mismatch in ones own groups in-world and out-world views. For example, if you are in development, then in practice this exercise amounts to listing ones assumptions and prejudices toward the people with whom you engage in the business (such as business analysts and management), end users, and operations and services staff. You would then attempt to see how these groups see themselves and you, and finally meet with these people to see how to break down barriers and improve work processes.
4.3 Kanban adds new flexibility to development workflow management

Pulling work Just in Time
The Kanban system of signaling originated at Toyota and became a core part of the Toyota production system. It was later named Lean production and Just-in-Time (JIT) manufacturing. The essential principle behind Kanban is to pull work on a need basis, starting from the downstream end, so demand ripples upstream and the parts produced have a ready customer. The JIT approach ensures that parts or work items arrive at precisely the time they are needed and in the quantity needed. Applied to software development, Kanban manifests as three features: a visual workflow (for example, a physical or electronic whiteboard), pull by demand, and use of work-in-progress (WIP) phase limits. An example of a Kanban workflow board is shown in Figure 5. In a typical Agile whiteboard (for example, as used in Scrum), the columns represent the stages of a single iteration, or Sprint, so work comes in from the left and is complete when it reaches the right-most column. In Scrum, no new work is allowed to enter from the left while the Sprint is in progress; in Kanban, work items are allowed to continually flow in from the left. What stops the board from rapidly clogging up is that columns are limited as to how many items are accepted at each stage. These WIP limits control the flow of work items and allow it to be matched to the teams work rate. Sophisticated Kanban boards have been created, and the design is often evolutionary and unique to the teams involved. For a development team new to Kanban, one common approach is to start with a basic board (such as in Figure 5) and adapt it over time to fit the needs of the projects and organization. Issues involved are: how to distinguish between work items requiring different amounts of time to complete how to distinguish between work items from different projects (for example, use a different board or one board with swim lanes) whether all actual work is represented on the board how tasks that have hit delays or problems are identified how work items can be prioritized whether they are given an express swim lane or color-coded.
59
Kanban can be used in combination with traditional work processes as a way of adding visibility to work and controlling the workflow. As there are only three rules to Kanban, there is no training necessary; the board itself becomes a learning ground as the team modifies it to fit the project needs. Kanban can also be used as a complement to Agile practices for example, Kanban and Scrum are a typical match. The Sprint element is the main Scrum feature affected, with the iterations replaced by continuous delivery.
Phase
To Do 6
. .
Dev 2
Q 1
Test 2
Release Done
3 E B
J I G H K L
F C D
Figure 5: Kanban workflow management
Source: Ovum
4.4 Continuous delivery and deployment: a new phase in Agile development

Continuous build, test, and deploy in one automated step
The Agile manifesto created in 2001 has 12 principles associated with it. The first says: Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. It can be debated whether this was meant literally then, but today the idea of continuous delivery is becoming a reality. The continuous delivery pipeline can include a manual/exploratory testing stage before final automated deployment. A stronger variant is continuous deployment whereby the whole pipeline from source control check-in to deployment is automated. A 2010 webinar run by SD Times and hosted by Kent Beck ran a poll on the question how often do you deploy to production? (See Figure 6.) The webinar was aimed at the more advanced Agile developers and the results show that deploying once a day or less represents about 10% of cases, and therefore this high deployment frequency is a noticeable practice within the Agile community.
60
How often do you deploy production?
Once per year Once per quarter Once per month Once per week Once per day Once per hour On every commit
Figure 6: Straw poll on continuous deployment based on 200 responses
Source: Kent Beck www.threeriversinstitute.org/blog/?p=516
Continuous delivery means continuous build, test, and deploy, in one automated step. Of course, if the testing shows faults, then the process iterates around fix-build-test, but the concept is that working software is rapidly pushed into production. This approach raises a lot of issues. Not all end users welcome continuous delivery/deployment of updates to their mission-critical software. There may be issues of update compatibility with other production environment systems into which the application integrates, which may have slower update cycles. Of course, these issues can be made part of the systems and performance testing. However, there are also issues around end-user training and readiness. Not all end users welcome frequent disruption to their work days for updates to features that may not affect them. It is best for the business/customer to pull such continuous delivery by demand, rather then let developers push these out. There are circumstances in which continuous deployment offers a competitive edge. Financial traders depend on speed to beat the market and may value a correction or update to a trading system over the inconvenience of having a change disrupt work. The very act of continuous deployment means the disruptive effects of a change are minimal, as the amount of change from the previous version is likely to be relatively small compared with, for example, a quarterly update. Thus, continuous deployment mitigates the problems that can mount up and lay hidden until exposed when major releases are made. Another aspect of continuous deployment that alters the usual reality of introducing releases into a live environment is the practice of live updating or zero-downtime releases: changes are instantaneous, with users hardly aware that the change has taken place. The key aspect here is the capability of instantaneous rollback. In well-managed sites, it is possible to run multiple live production environments, some of which are purely for testing and, in all other respects, identical to the working environment. Some of these may be for a select group working as a canary warning system, so continuous deployment is staged across these environments to give sufficient warning time in case of unexpected problems.
61
The focus of continuous delivery on the operations activity is recognition that the bottleneck is moving away from development. The operations community is also aware of this change and, especially those involved in sysadmin programming, is beginning to embrace Agile values and work patterns. DevOps is the name given to a movement currently in early stages that promises to spread wider, and which is aimed at improving communication across development, operations, QA, and others. DevOps aims to break the old silo way of working and achieve quicker deployments with higher success rates. We expect to hear a lot more about DevOps in 2011.
4.5 Agile and lean will embrace large-enterprise development projects

Customize and adapt practices to the needs of particular teams
The adoption of various Agile methodologies has reached one-third of all practices used in software development, with an equal measure falling into the ad hoc and custom category, and the final one-third including waterfall-like, Rational Unified Process, Capability Maturity Model Integration, Team Software Process/Personal Software Process, and others. With Agile adoption there is no single authoritative Agile methodology. It is a mixed bag ranging between Scrum, Extreme Programming, feature-driven development, Lean development, and test-driven development. Furthermore, for many it is a pick-and-mix style of Agile process creation, and this may not always be sufficiently Agile for the purists. However, compared with the preceding age of waterfall development, we are witnessing a step change in the Agile direction. Scrum and Scrum-like leads the field for Agile project management.
With Agile adoption there is no single authoritative Agile methodology.
The practice of Agile development continues to face challenges as, for example, large-scale projects look to adopt it and practices that work well in small projects are impractical in the large ones, or distributed teams face the challenges of time zones and distances.
An architecture blueprint is essential for large-scale Agile development projects

As Agile projects face challenges of scaling up, two fault lines have emerged. One has to do with architecture and upfront design, and the other revolves around when user stories are converted into technical tasks in the popular Agile methodology, Scrum. These fault lines betray the small-scale project beginnings of Agile and reflect its growing pains. Resolutions to these pitfalls exist, and are best explored as part of (a Lean philosophy of) continuous improvement.
Creating an architecture is necessary for larger or more complex Agile architecture is projects. The belief that evolving design is sufficient and that architecture can be factored into the project later is a rather risky approach to take for necessary for larger or more complex large-scale projects, but unfortunately advocated by some of the original Agile Alliance gurus. This debate has continued within the Agile community Agile projects. since its early days, and lines tend to be drawn, with Agile developers on one side and those with more traditional backgrounds on the other side. However, this is not an issue about the broader question of Agile versus traditional that debate has been largely won by Agile. Rather, it is a question for Agile developers to address when engaged in large-scale projects. Professional Agile developers involved in complex real-world projects all find the need for creating a software architecture, often supported by modeling and testing. The reason architecture is such a contentious Agile issue is that it runs against the grain of no or little upfront design. Just as requirements are not all signed off at the start of a project, the architecture is likely to need evolving, and too much upfront design results in waste.
Creating an
62
One resolution is to apply Agile iterations to architecture, just as Agile iterations apply to code. Think of architecture development as a layer running parallel with code development. Architects and designers (roles that can be taken up by the development team itself) apply Agile thinking to architecture: start with what is known (and being architecture, these are the large or hard issues) and build on it. Where the focus of code development is on delivering customer value, the focus of architecture is on delivering clarity and vision. (The coders are now the architects customers.) The question of how much architecture to do upfront will depend on the scale and complexity of the project, the skills and experience available, and the teams domain familiarity.
Agile stories into technical tasks: a serious crunch point in the Scrum process that is often neglected
The Scrum methodology is the most popular Agile practice, but its main emphasis is on project management. As a consequence, this leaves hanging the question of when Agile stories are translated into technical tasks: is it during the Sprint (iteration) planning or during the Sprint itself? One resolution is the Agile development practiced at BT, in which user stories are broken into smaller and smaller bites prior to a Sprint. This, in effect, makes the translation from business-level story to technical task a series of small steps rather than one large step to surmount. Agile will focus on the simplest way to get the value: the value of task breakdown is to increase the confidence of the Sprint. Once this is clear to the team, it can make its own decision about task delineation, often on a story-by-story basis. Complex or unfamiliar stories will have more risk than familiar or simple stories, and the team may want to perform a team task breakdown on these (or can designate this job as a story), while leaving the simpler stories as they are with technical delineation performed during the Sprint. This question is closely linked with the one about architecture and design: the decision of how to transform a requirement into a technical task will be made vastly easier if it can draw understanding from architecture.
Embedded software in products: an opportunity for Agile in engineering and manufacturing

A history of Iterative and Incremental Development (IID) is given in a paper by Craig Larman and Victor R Basili (Larman and Basili, June 2003). It charts the introduction of IID in the 1970s to the present day and the birth of Agile development. There have been many projects in the software engineering communities going back to the 1970s; many are aerospace and defense examples in which techniques such as time boxing in short iterations and regular feedback to improve design were practiced. The pioneers who created these techniques understood that the waterfall process was limiting. The paper by Winston Royce in 1970 that first described waterfall was aimed at identifying its weaknesses. Royce described the advantages of a fully iterative development process before shying away from it for practical purposes for example, lack of good automation in development tools hindered adoption of advanced process techniques. A full historical analysis also needs to take into account the influence of Lean techniques and Rapid Application Development on Agile. However, what is curious is that while waterfall was adopted by the software community in the early days of computing, the manufacturing industry from whence it came moved on. The limitations of waterfall were also apparent to those practicing it in industry, and their solution was concurrent engineering (CE), which started in the 1980s and came to fruition in the 1990s. Significant influencers were the Defense Advanced Research Projects Agency and the Department of Defense Continuous Acquisition and Lifecycle Support (CALS) initiative. CE has nothing to do with concurrency as in parallel programming; rather, the concurrency is in the way in which multiple aspects of a product development lifecycle can be addressed concurrently in order to bring a product faster to market and at lower cost and market risk. (See Figure 7.) There are strong parallels with Agile development, with CE addressing product lifecycle management (PLM) and Agile addressing application lifecycle management (ALM).
63
The growth of embedded software in manufactured product industries including automotive, telco, consumer goods, household goods, and so forth, necessitates an ALM approach to software development. With CE practiced in PLM and Agile in ALM, it is a natural next step for Agile to enter product industries through the CE door. A combined Agile CE approach would bring out compatible development techniques for the embedded software components in smart products.
Requirements
Software design
Electrical & electronic design
Safety design
Microelectronics design
Reliability engineering
Hardware design
Documentation
Mechanical design
Figure 7: Concurrent design elements in concurrent engineering
Source: Ovum
4.6 Recommendations
We believe Agile methodologies and processes have significant advantages in the majority of software projects over traditional waterfall and waterfall-like processes. We advise Every organization undertaking pilot projects and identifying Agile champions, advocates, and role models within your organization who can spread the practice. Giving is unique and has sufficient attention to an Agile rollout strategy, including providing training different needs and mentoring to development teams and using change management, will Agile practices can yield benefits in the long run.
be adapted to fit those needs.
We also suggest giving sufficient attention to how Agile practices can be best customized to the organization, including addressing issues around architecture, large-scale projects, and fitting into non-Agile business processes. Every organization is unique and has different needs Agile practices can be adapted to fit those needs.

Support for Agile practices is the biggest driver in the software development tools market (including ALM). One important success factor will hinge on how well your tools integrate with other vendor tools and open-source tools in the ALM space, as users will have a mix of tools present and not buy into ripand-replace. Ease of use is another critical factor; a steep learning curve to figure out how to use features will not go down well less is more. Useful differentiation in the ALM tools market includes out-of-the-box integration with best-of-breed solutions and the ability to track work artifacts across lifecycle segments for example, trace a requirement through code implementation, tests, test results, and deployment. For Agile support, the ability to stream project metrics to popular application project portfolio management tools as used by senior management is a significant differentiator.
64
Incorporating
OVUM Butler Group
CHAPTER 5: Enterprise mobile development: trends, platforms, and tools
WWW.OVUM.COM
5.1 Summary
Impact
The new-generation smartphones and tablets are satisfying a sea change in consumer demand for mobile devices a market that Apples iconic iPhone initially created. Businesses are beginning to react to this change but it is not always clear how best to harness the opportunity for productivity benefit. The market is fragmented among a number of players, making the task for application developers more complicated if they are to cover multiple mobile platforms and form factors. The challenge for developers further increases as the devices themselves get more sophisticated, with embedded multi-core chips, graphics processing units (GPUs), and other specialized processors needing to be programmed. This situation offers an opportunity for mobile development environment (MDE) vendors providing a degree of re-use with write once, deploy multiple times features, albeit after user-interface (UI) tweaking.
Ovum view
The feature phone has the current mass market at over 3 billion users, but the future is in smart devices, from new-generation phones to tablets. At the time of writing smartphones are estimated to represent 20% of the mobile market, and the top mobile phone manufacturers are increasingly focused on smartphones for their latest models: the shift is plainly visible. The microchips powering new-generation mobile devices are also leaping ahead in computational capability (see Nvidias Tegra 2 for example), making these small devices equivalent to the computers of only a few years ago. Such computational capability is bringing a mass of developers to smart devices. Just as the rush for land grab for e-commerce development took place in the dotcom boom era, a similar rush is taking place on mobile platforms. While for consumers the focus is on games, social networking and other Web 2.0-type applications, for enterprises it is currently about transferring what already exists on and via the Web to be accessible on smart devices. These enterprise apps are therefore mostly free and face either towards customers and suppliers or to their internal staff. The enablement of staff mobility is designed to ensure organizations are not left behind as patterns of device and Internet use change. Paid-for enterprise apps will appear but the bulk of activity will be about ensuring that all stakeholders and customers can be reached and connected. Satisfying developer needs, as this switch to the mobile platform takes place, would seem to be natural, and many vendors have taken up this challenge. However, Ovum finds that the degree of fragmentation in the mobile world has created many barriers for developers to overcome, and the development tool arena is equally fragmented. Even if you take the current market leaders Apple iOS and Google Android, there is variation in devices and form factors to cover, particularly for Android. To cater for this new developer activity a new breed of integrated development environments for mobile apps, MDEs, have appeared on the market. One group is those that target the needs of large enterprises, with advanced management capabilities overlapping with application lifecycle management, and in some products provide 4GL development for non-programmers. Then there are a host of developer-oriented MDEs where apps are written in open web-based technology and are converted to run natively in usually the most popular devices. Currently, the traditional ALM vendors do not see a need to especially cater to the mobile market, though the growth of MDEs will start to eat into some ALM segments. While smart devices have brought the capability for sophisticated apps to be run in the mobile world, it is the app store concept that has transformed the market by removing the network and OEM bottleneck through which apps can be loaded onto devices. The app store connects mobile owners directly with developers, and this has brought thousands of developers to the mobile market. The app store provides a single portal through which developers can channel their products and consumers select them. The store manages the administration and economics from submissions to earnings. An app store is a must for any smart device today. However, browser-based apps open up further options for users.
CHAPTER 5: ENTERPRISE MOBILE DEVELOPMENT: TRENDS, PLATFORMS, AND TOOLS
67
The rich Internet application (RIA) plug-ins from Adobe, Microsoft, and Oracle offer the high-end mobile app options. However, apps built on HTML5 (incorporating Ajax) and other open standards and technologies (CSS3, JavaScript) will increase their market share as these standards gain improved acceptance. Many new development tools are available on the market to address the needs of mobile developers wanting to target multiple leading market devices. While at present most developers rely on the SDKs provided by the individual platform makers, given the fragmentation in the market the value of an alternative approach will begin to be recognized. Another aspect of importance to developers is the long-term viability of the mobile platform vendor/device/language: presence in the market, ability to execute, and having the tools to create the applications are all important factors. In Ovums view open web-based technology choices will become more popular given the market fragmentation and the need to address multiple devices. Finally, understanding how the mobile app changes the Internet is of concern to all stakeholders, including businesses wanting to keep abreast of the changes taking place. The app store is being hailed (hyped) as the next marketing channel beyond the Internet, but the Internet will remain the backbone and the Web remains the key outlet whether through integration with apps or not. So the office worker of the near future will most likely be using a tablet device, enjoy the convenience of being able to switch apps on instantly rather than wait for machine boot-up, be connected everywhere, and have installed in their devices powerful system-on-chip microprocessors. With greater use of SaaS and other cloud services, these new-generation portable devices will be the front human-machine interface to an array of apps, tools, and systems.
Key messages
There is an unmistakable shift from feature phone to smartphone as apps make the latter more desirable and this in turn drives more developers to the platform. Developers are pivotal to the mobile apps explosion, driving demand for MDEs that can cover the leading smart devices. Mobile application development is fragmented but de facto industry standard platforms are rapidly emerging. Mobile development is highly active in open source and web-based technology. MDEs reduce the complexity in managing app development on multiple devices. Online app stores are providing new channels that are opening the gates for developers to deliver their applications to market.
5.2 Mobile application development is fragmented but standard platforms are emerging
Smartphones and tablets are where the mobile development action is
A number of trends indicate where the mobile market is heading. First of all, consumers are buying smartphones over older generation feature phones. Although this trend is greatest in the Western world, it is expected that smartphones will become globally dominant.
68
Where consumers are leading, businesses will follow, as the workforce becomes enabled by small devices that improve mobility and productivity. Wholesale swap-over to smartphones over feature phones is taking place in the enterprise, and examples of work forces being Where consumers given tablets to ensure staff are not being left behind on the technology curve are being reported. There may be an element of herd action in this are leading, but these trends also reflect a realization that gains in productivity and businesses will convenience merit these changes.
follow...
For the mass consumer market the mobile platform and its associated app ecosystem are seen as a challenge to the Web, Internet search, and associated click advertising. Google, which has the most to lose with this change in search patterns, has staked its presence in the mobile market with its Android OS. Sales of Android devices have now overtaken Apple, at least in the US. The most likely outcome of consumer Internet behavior is a continuation and expansion of existing patterns rather than complete shifts; the Web and search activity will continue, but presence on mobile will increase. Given that Facebook access from mobile devices outpaces access from fixed computers, the pattern changes are already in evidence. It is worth considering that these trends have not been predicted and have caught many mobile industry incumbents by surprise.
Development perspective
Switching application development from desktop or web to mobile is not a simple task; mobile devices have constraints and user usability expectations will be high. This will place added demands on application developers. Among the early warning signs for mobile developers are the growing numbers of end customer enquiries to call centers for help in performing simple tasks on their mobile phone. For developers the challenge is to understand that smart device users expect a higher level of usability than offered in older generation devices and software; Apple set the standard in usability. While enterprises with field workers have been early adopters of handheld devices, scanners, laptops, and now smart devices, the trend is towards enabling office workers who are increasingly becoming mobile or work remotely. Most large enterprises now run some form of mobile app programme and they will need to address the same usability issues. Disadvantages with the mobile platform do exist, such as connectivity issues around bad lines or overloaded networks. There is the limitation of the device form factor itself, governing ease of use, the power of the device to match user expectations of app speed, and ability to work in offline and online modes. Of course improvements in technology are taking place the ideal tablet size is currently part of the market wars, and mobile processors are making astonishing strides, such as the Nvidia Tegra 2. This system-on-chip (SoC) contains eight processors, including a dual-core ARM CPU and Nvidias own GPU offering console-screen quality with HD video on handheld devices. Such devices open up the scope for traditional application developers to switch attention to mobile devices. However, these new microprocessors also pose a challenge to developers to program them efficiently the GPU, for example. Developers need to address a host of issues, such as planning for the following: the workflows governing how mobile applications are used, as they may differ from traditional desktop applications the different push or pull data provision models available, which affect whether the user needs to be active in downloading data/updates, or whether the server pushes information whenever the user goes live the extent to which the application fully exploits local computational power and native APIs and devices features (touchscreen, geo-location, compass, accelerometer, gyroscope etc) whether the application will be custom built from scratch, a custom extension of an existing application, or supplied by a third-party supplier. The client-side platform options include: a rich browser application, a thin client tunneling through to the server, or an app that can run offline and use local storage whether the app will be built on open standards, a proprietary solution, or just be an extension to an existing system with backend in place.
69
Native versus web: the new fault line for mobile application development
The extent of market penetration and where it is trending are key to how developers choose their development platform. By far the most popular development platforms are Apple and Android. Microsoft is taking a new mobile initiative with Windows Phone 7, aimed at the consumer, and is currently a runner up in adoption. The Nokia tie-up may well make it a third pillar in the market. With development activity for the Apple iPhone/iPad and Android devices the highest across the platforms, it may appear that native apps is where the future lies: for Apple that means Objective-C, the iOS development language. Android native apps are built on Java, which is an open language and platform, but Android Java runs on the Dalvik virtual machine, not the Java Virtual Machine, and its associated class libraries are designed to exploit smartphone features. Incidentally, this difference is the cause of contention between Oracle and Google and is still playing out. Nevertheless, the two key players, Apple and Google, have emphasized native application development over using the open Web HTML browser platforms that both use. This also means that native vendors control the apps through their app store, shutting out rivals from app purchase commissions. However, there are two vectors driving developers towards open web-based apps and away from purely native development. Developers themselves prefer the openness of the Web and do not like being locked in to native environments or dealing with the pain of deployment across multiple devices. Although Ovums survey on developer activity shows that there is currently a willingness to cover three platforms in parallel, as the market matures the maintenance burden will begin to tell. The Web, with the increasing support for HTML5, and spread of WebKit offer developers sufficiently rich functionality to create apps almost as good as native ones. These apps are run via the browser, either as Ajax rich apps, or via plug-in players (Adobe Flash, Microsoft Silverlight, JavaFX) that enhance the user experience. There is browser support variability across the various feature standards that make up HTML5 and CSS3, and this will affect how universal such apps can be, but convergence will occur in time as HTML5 grows in acceptance. The second vector is those players other than Apple and Google who wish to see a slice of the market. One initiative is the Wholesale Applications Community (WAC), created to unify activity around an open platform. WAC encourages use of HTML/HTML5, and has JavaScript APIs transferred from a number of open standards initiatives, including BONDI and Joint Innovation Lab (JIL). WAC at time of writing has 48 corporate members spanning mobile network operators to mobile manufacturers, including Alcatel-Lucent, Fujitsu, LG, Oracle, Accenture, Ericsson, Huawei, Intel, Qualcomm, and Samsung. In addition there are vendors who straddle the native and web approaches, such as Adobe, Oracle, and Microsoft, plus the community pushing for an open web-standards approach:
Adobe Flash
Adobe offers an open platform around Adobe Flex and AIR, offering a write once model with deployment wherever the Flash player can run (Flash 10.2 for Android and other smartphones, offering the highest quality, and Flash Lite aimed at computation and memory constrained feature phones). These models do need some tweaking for tailoring to individual device screen resolutions and size; the emphasis of the Adobe message is on app logic reuse. Adobe has had muted impact in the mobile space. However, with Flash 10.2 designed to run on smart devices, Ovum expects it will gain ground as developers see the benefit of re-use of applications on multiple platforms. Currently three key devices are in Adobe Flexs scope: Google Android, RIM BlackBerry Tablet, and now Apple iOS (iPhone and iPad). The BlackBerry Tablet is notable for being architected itself on AIR, and the long-running saga between Adobe and Apple has taken a positive turn with Apple relaxing its rules on third-party applications supplied from its App Store, so that Adobe Packager for iPhone created apps will now be permitted. The Packager allows Adobes model of write once, reuse multiple times to apply to the iPhone as well as other devices. Adobes model allows application logic to be reused while the UI needs to be designed for specific devices. Packager also supports the iPad, though currently does not fully exploit the iPads higher screen resolution. This may well change in future releases.
70
Oracle Java
Oracles plans for Java on the client side are finally beginning to take shape. Java ME (Micro Edition), together with the Java Lightweight User Interface Toolkit (LWUIT) is the solution for feature phones, which Oracle is happy to point out has a market size of more than 2 billion phones with Java ME installed, and which vastly outnumbers smart devices. Especially for countries like India where 97% of mobile phones are feature phones, the market for Java ME is important. Ovums research indicates adoption of feature phones in emerging countries will continue growing due to economic and infrastructure reasons. Java ME is also evolving; Java ME.next (as the proposal for next-generation ME is called) is in progress and a highlight includes better integration between JavaScript and Java to support different models for mobile apps combining Java and web technologies functionality that Oracle has already starting implementing in its mobile runtimes. However, the notion that Java ME apps are write once, run anywhere has not become a reality. The market is too fragmented and apps need to be fine-tuned to particular devices. Although the market size for Java ME-compatible devices looks healthy, the question is whether developers choose to write apps for that platform. Developers will follow the money and Java ME does not offer that incentive. The app store concept helps monetize developer effort and so far the feature phone manufacturers have been slow to follow Apples lead. Certainly in the developed economies where smart device adoption has leapt up, developers have been attracted to build smart device apps because of the monetary opportunity. As Figure 8 shows, while feature phones vastly outnumber smart devices, apps available on smart devices vastly outnumber feature phone apps. Midlets (Java ME apps) are fine for feature phones, but a richer client experience is expected for smart devices. Sun initiated JavaFX as a rich client platform and Oracle has made one significant correction since taking over Java: it announced in September 2010 that JavaFX 2.0 will be a re-architected pure Java edition. The earlier version is based on JavaFX Script, yet another scripting language which many developers bemoaned. The advantage of JavaFX 2.0 is that it will run any JVM language. It is exactly what Java developers need. At time of writing the beta release has been scheduled for the first half of 2011 and general availability is targeted for the second half. Oracle has not announced any plans for JavaFX to run on mobile phones and has said its current focus is for the desktop and full browsers. Although it is early days, we believe the growth of smart mobile device usage combined with the evolving computational capability of such devices will blur the boundary and make it inevitable for Oracle to make JavaFX mobile compatible. It has taken time to correct the course client-side Java has taken and some Java developers have moved to other platforms, such as Adobe Flex for a richer client side with use of Adobe BlazeDS (or its enterprise parent Adobe LiveCycle ES) to connect to server-side Java. However, when JavaFX 2.0 goes live, Ovum expects it will pull back some of these developers. JavaFX 2.0 has the additional merit of providing native interoperability with JavaScript and HTML5. In Ovums opinion if a future release of JavaFX is not made smart mobile compatible then it will be a missed opportunity.
Microsoft Silverlight
Microsoft has Silverlight as its rich Internet application (RIA) browser plug-in player. Silverlight received attention recently as Microsoft stressed its dual approach of supporting HTML5 for the broad sweep of RIAs, while Silverlight was the option for higher-end web applications and the prime development platform for Windows Phone 7. This initially created shock waves in the Microsoft camp, but the alignment makes sense given that Microsoft does not want to lock itself out of web development as Silverlight has relatively low uptake. A significant announcement made by Microsoft recently is that Windows Phone 7 (WP7) will only run on ARM and the upcoming Windows 8 will also run on ARM (as well as Intel chips), opening up the huge mobile market where ARM runs on 95% of mobiles. Microsoft also emphasized its support for HTML5. The company is not making the mistake of cornering itself, but ensuring that it is present where developer activity is highest. The announcement at the Mobile World Congress 2011 that Nokias smartphones will run WP7 is a huge boost for Microsoft. So far Microsoft has failed to slow down the success of Apple and Google in the mobile space.
71
The technology Microsoft has to offer developers is first rate (Visual Studio Tools for WP7) and though it is early days in the burgeoning smart device market, the Nokia deal may well prove a turnaround in Microsofts mobile fortunes. Ultimately success for Silverlight on mobile apps will depend on the size of the market for WP7. Developers will not commit to a proprietary development environment unless the market exists to justify the investment.
HTML5 and WebKit

HTML5 is the next evolution of the Webs markup language, HTML, and a major incremental step. While the W3C Recommendation status for the standard is set for 2022 or later, the Candidate Recommendation is set for 2012. It is already applied to some degree in many top-end browsers including Firefox, Safari, Google Chrome, and the upcoming Microsoft IE9. For developers the crucial improvements in HTML5 are the new APIs such as Canvas for 2D drawing, support for offline (local) storage, document editing, and drag and drop. Currently the lack of agreement (due to competing vendor interests) over video streaming is a weakness. When combined with JavaScript and Ajax widgets, the standard allows for RIA development superior to previous standards. HTML5 is also significant for specifying a standard for how the HTML is written and how the browser is to render it. The result is an open standard for web apps that can run in any compliant browser. There are two open source web layout engines that support HTML5: the most popular is WebKit and is at the root of many mobile browsers, including Android, BlackBerry, Nokia (Series 60), Apple Safari, and webOS. The other one is Mozilla Gecko used in Firefox for mobile. Over 90% of mobile web traffic in the US is estimated to run on WebKit browsers. Note: we use the term open web-based technologies throughout the report and it essentially means (today) the combination of HTML5, CSS3, and JavaScript.
5.3 Mobile development is active in open source and web-based technology

Developers make platform decisions for the long term
Developers, IT departments, and businesses have to make their choices now as to which platforms and languages to use to build their mobile apps. See Figure 9 for where developers are currently focusing their efforts. This is a critical time as investments are made in time and resources, Developers, IT in training and tools. Once these decisions are made it takes significant reasons to cause a switch to alternative solutions. In this formative phase of departments, and mobile development, it is as much about winning hearts and minds as about businesses have to solid technical solutions. In deciding where to focus effort, the size and growth make their choices of a particular market is important, as is the possibility of opting for a solution now as to which that spans multiple device markets.
platforms and languages to use to build their mobile apps.
Developers are concerned about how difficult a mobile app language is to learn, what is the likely longevity of that language, and how efficient that language is comparing lines of code with other languages. Programming languages today are expected to support object orientation and have frameworks that follow the model-view-control (MVC) pattern. Specifically for mobile development the issues are around having tools to build and prototype apps, and the availability of fast-to-load device emulators with debugger features. For example Apple requires its iOS developers to run the Apple development tools on Macs. Many developers who work on PCs are unhappy at the additional costs involved; for these developers, tools that can transform solutions to run natively on iOS are therefore attractive.
72
Common concerns with development tools include: how rich can the GUIs be, what technical support is available, is support resourced by the vendor or reliant on open community forums, and are the APIs open and documented? Finally, developers building paid-for apps want a reasonably quick and stressfree path to market and monetize their apps. This last point is a major barrier for mobile devices strongly controlled by the mobile networks; these typically offer pre-installed apps. It can take more than six months to get through the hurdles, and the costs involved prohibit small developers from entering this market. Apple swept these barriers aside, offered its App Store which emulated the iTunes model, and in doing so changed the mobile industry. Access to an app store with micropayment facility, where users can download apps at will, is therefore a App stores enable must. Adobe has thus created the Adobe InMarket, which is a one-stop device users to shop for its AIR developers, who only need to register an app on this connect directly market and it becomes automatically distributed on multiple device app with developers, stores. The rise of the consumer-oriented app store has taken control and income operators and away from the network operators, hence the interest within the industry in OEMs. WAC. App stores enable device users to connect directly with developers, bypassing network operators and OEMs. These developers can compete more effectively with their small and cheap apps as compared to pre-installed and network-approved apps. It remains to be seen how successful WAC becomes. it will need to offer services that are as efficient as Apples App Store, and as attractive to mass developers.
bypassing network
Open web-based standards are a sound long-term choice for mobile development
Open source solutions will have a large part to play in the mobile world. Figure 8 shows the state of play in apps installed a year ago at the time of writing. The market has moved rapidly since and, in terms of OS, Googles open source Android has now surpassed Apple iPhones market size, and its growth rate is higher than any other smartphone. The size of the app market is highly relevant to developers: Java MEs impressive 2 billion device market size trails behind Apple and Android in apps available.
There are a number of attractions for developers of open web-based standards...
There are a number of attractions for developers of open web-based standards compared to learning the Apple iOSs Objective-C or the Android Java library: the learning curve is faster especially for developers already familiar with RIA; the skills gained have greater currency and applicability across multiple types of applications beyond just mobile; the longevity of the skills is likely to be greater as HTML5 becomes universally supported while proprietary/native systems will likely have shorter lifetimes; and the ability to write for multiple devices must be the top attraction. The fact that many MDEs have appeared that cater for open web-based developers and provide native app conversion lends weight to opting for open standards. MDEs remove the burden of managing and maintaining apps across multiple devices and form factors, and Ovum believes these tools take the pain out of the fragmented nature of the mobile market. Although it is early days for HTML5, and enterprises in particular are concerned that consumer-facing apps based on open web-standards will have patchy uniformity across the many browsers on the market at present, this will improve, and choosing open web-based technologies will be a solid bet for the future. For high-end client-side apps there is Adobe Flash and Ovum does expect Java FX 2.0 to make a difference to the large, especially corporate, base of Java users. The following mobile OSs represent the leading platforms on the market at time of writing and is not an exhaustive listing of what is available.
73
Device Installed Base (est. Q2 2010)

390 60 20 Symbian iPhone (iOS) Android Java ME BlackBerry Flash Lite Windows Phone 0 0
Number of apps available in app stores (Q2 2010)

6,900 225,000 72,000
3,000
45,000 7,800 5,000
110 1,400 75 2500 2000 1500 1000 500
million units 3500 3000
13,500 50,000 100,000 150,000 200,000
apps 250,000
Source: Mobile Developer Economics 2010 and Beyond. Produced by VisionMobile. Sponsored by Telefonica Developer Communities. June 2010. Licensed under Creative Commons Attribution 3.0 Licence. Any use or remix of this work must retain this notice.
Figure 8: Mobile devices installed and number of apps available in app stores, Q2 2010
Source: Mobile Developer Economics 2010 and Beyond
Development characteristics of key smart device OS platforms

Apple iOS Proprietary OS for iPhone, iPad, and other Apple devices. iOS has origins in NeXT, Unix, and the current Mac OS X. Apple apps are written in Objective-C. Apple offers extensive developer support such as Xcode, an IDE, for designing, testing, emulating and debugging, and Cocoa Touch, a model-view-control design framework that provides the runtime, APIs, and libraries. The SDK is required to be installed on a Mac machine. The Safari web browser is based on WebKit and will run apps built on open web technologies. The browser will not support third-party plug-in players such as Flash. However, a number of products exist on the market to convert non-iOS apps to run natively on iOS. Apples App Store pioneered the concept and is leading the industry in app volume. Google Android Open source OS based on Linux. Applications are written in Java and run on the Dalvik virtual machine. Scripting languages such as Python and Ruby are supported through the Android Scripting Environment. Browser is based on WebKit and Google Chromes V8 JavaScript engine. It supports open web technology built apps. Googles Android application framework includes support for UI widgets, 2D and 3D graphics based on OpenGL, storage using SQLite, a unit testing framework, and easy integration with maps, email, messaging, and more. Apps are distributed via Googles Android Market. HP webOS HP has released the HP Slate 500 tablet PC, running Windows Phone 7. However, the latest news concerns a trio of products based on its own (Palm acquired) webOS: a credit-card sized, consumertargeted smartphone called Veer, a larger enterprise targeted smartphone called Pre3, and a tablet called TouchPad. webOS is a native mobile OS based on Linux, proprietary to HP devices, and runs the open source WebKit browser. HP provides an Eclipse-based SDK for application development based on open Web technologies. HP offers device-specific App Catalogs for app distribution.
74
% of developers 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% iPhone OS BlackBerry OS Windows Phone Android OS Symbian OS Other OS
Figure 9: Smartphone platforms that developers are currently supporting or plan to support
Source: Ovum Mobile Application Developer Survey
Microsoft Windows Phone 7 This mobile OS replaces Windows Mobile and is currently consumer market focused but with increasing enterprise applicability. Although WP7 has a small footprint in the market, its new tie-up with Nokia is expected to have an impact. The app store is called Windows Phone Marketplace and covers all distributed digital content. Development languages run on XNA for games and Silverlight for other applications, using development tools Visual Studio 2010 and Microsoft Expression. The UI design language for Phone 7 is new and called Metro. Nokia Nokia has offered Symbian and MeeGo OSs but at the Mobile World Congress 2011 announced a new partnership with Microsoft and will support Windows Phone 7 for its high-end phones. Nokia will likely drop MeeGo altogether by 2012 and Symbian will continue to be supported as legacy and in exploiting existing markets. Symbian An open source mobile OS maintained by Nokia that runs on ARM-based mobile devices. Applications can be written in a number of languages. The Symbian SDK uses the Qt framework and supports standard C++. Apps can also be programmed in Symbian C++. The OS supports Java ME, Flash Lite, and programming languages such as Python, Ruby, and .NET. Qt supports HTML5 and Qt Quick offers rapid RIA development. MeeGo A Linux-based open source mobile OS originated by Intel and Nokia and now hosted by the Linux Foundation with multi-vendor support including AMD. Applications are written in C++ using the Qt framework.
75
Qualcomm Brew Brew MP is an application development environment for mobile devices running the Brew RTOS on Qualcomm chips. Applications are built in C/C++ and Java using Java ME. Flash Lite is also supported. Brew has stringent certification requirements for apps and the minimum cost is $400 for 100 app submissions. RIM BlackBerry OS and BlackBerry Tablet OS BlackBerry OS and BlackBerry Tablet OS, two distinct OSs, are proprietary to RIM. The latter is based on the QNX Neutrino realtime OS, and BlackBerry has plans to unify its devices around it. The BlackBerry Tablet OS has been built from WebKit and Adobe Flash (Adobe AIR). Applications can be built with Adobe tools and open web technologies. There is a BlackBerry WebWorks SDK for Tablet OS. The vendor has made strides recently in emulating Apple in the type of support offered to the developer community. The BlackBerry OS will become legacy; development supports Java ME. RIM runs the BlackBerry App World app store for developer app distribution.
5.4 Mobile development environments reduce the complexity of targeting multiple devices
A new tools market has grown to support mobile development
The market for mobile development environments (MDEs) and tools is blossoming in tandem with the opportunity to tap into the exponentially increasing mobile app market size. Developers can write apps on the SDKs provided by device and OS manufacturers, and the runaway leader in this aspect is Apple, but there is a downside: as that market size increases, visibility of individual apps decreases. So developers look to run their apps on multiple devices, perhaps where there is less competition from other apps and where the device market size is still significant. This is where the maintenance burden becomes an issue. While developers may consider opting for two or three devices that cover their main market targets (consumer, enterprise etc), this nevertheless covers a multitude of model variations covering form factors and versions of devices. The way forward may well be to opt for open web-based technologies, which essentially means HTML5 and JavaScript on the client side. There are new tools appearing on the market that will help build open standards based apps and automatically create native executables. We provide a brief overview of products on the market, first looking at enterprise-strength mobile app development studios designed for connecting to corporate data sources and with all the security and management features provided, then looking at mobile development environments simply focused at building apps.
Development tools for smart devices enterprise-level MDE market

This category contains vendors offering tools and services that would satisfy the needs of large organizations that are developing apps for smart devices, whether for their external customers, partners/suppliers, or for their internal staff. The level of tooling provided covers the app lifecycle and overlaps with ALM tools, particularly on design, core development, build, testing, deployment, and app upgrading. Some also provide app performance monitoring.
Adobe Flash Professional, Flex, AIR

Adobe has a number of tools to build applications on the Flash Player. Flash will run on most smart devices except on Apple iOS. The Flex framework is available as a free SDK, but Adobe offers a number of tools: Flash Professional for the visual design aspects of mobile apps and Flex Builder for core application development. AIR is the framework for offline and online apps. Flash 10.1 is the first product of the Open Screen Project, which is supported by over 70 industry players.
76
For connecting to and managing the backend there is Adobe LiveCycle Enterprise Suite, of which BlazeDS is a cut-down free version. Adobe does not offer any ALM features, which need to be supplied by other tools.
Antenna Software AMP

AMP Studio is hosted on Antennas cloud Platform as a Service and provides a full service for development, integration, management, and security. Studio features design and editing tools, uses a scrip language, with visual drag and drop, an integrated test environment, and back-end integration tools through the AMP Gateway. AMP Management Center manages users, deployment, updates, device profiles, app configuration, and security. AMP Client is a device-side client software that will run on multiple device platforms and connects with the AMP Gateway for online enterprise applications. Covers Android, BlackBerry, iPhone/iPad, and Windows Mobile/Phone devices.
edge IPK edgeConnect

An enterprise user experience platform to enable companies to develop business applications using a write once publish multiple times model. edgeConnect comprises a development tool and a runtime environment server and is targeted at business users with some domain knowledge, with no programming required, and offers full lifecycle management capabilities. Separation of the logical process (model) from the presentation (view); changes in the process can be rippled out to multiple presentations. Target devices include mobile platforms for online apps. Offline apps will be supported in future, based on HTML5.
IBM Rational Rhapsody

Android apps can be built using IBM Rationals model-driven development tool geared towards embedded software and realtime system development: Rhapsody. Rhapsody offers a visual representation of the Android framework API that developers can link to in models. The model is then used to generate Java code. Rational Rhapsody can read the AndroidManifest.xml file to visualize activities, services, and broadcast receivers. IBM Rational offers a comprehensive ALM suite that integrates with Rational Rhapsody, but it has no unified strategy for mobile support. For example, IBM Rational Application Developer and the BlackBerry Web Plug-in for Eclipse can be used to develop JavaServer Faces apps for the BlackBerry OS but not the new BlackBerry Tablet OS.
Microsoft Visual Studio 2010 Express for Windows Phone

Visual Studio 2010 is Microsofts trump card for enabling developers to build applications for its Windows Phone 7 mobile OS. The Express edition is free to use and comes with Windows Phone Developer Tools. Integrated with the IDE, it offers design, testing, and integration with Microsoft Expression Blend to help build rich Windows Phone apps. The package includes the Windows Phone Emulator and supports development with Silverlight and XNA Game Studio programming.
Pyxis Mobile Application Studio

The Application Studio offers software design with a configure once and run anywhere approach, uses drag and drop tools to design workflow and connect to backend systems, with no custom coding. Once configured, applications run natively on Apple iPhone and iPad, Android, BlackBerry, and Windows Phone. Has a management tool to manage deployment and updates.
77
Service2Media M2Active
A development and runtime platform from which a single script can create applications and services to run natively on multiple mobile devices. Devices covered include iPhone, BlackBerry, Android, Windows Phone, Symbian, Java ME, Bada, and MeeGo. Complete lifecycle management from mobile design, development, testing, website access and app store management, and update management. Server side of apps are hosted on the M2Active Platform-as-a-Service offering monitoring and support services.
Sybase Unwired Mobile

Sybase (owned by SAP) offers a development platform to build mobile solutions that connect to back-end business data sources. Allows one model to be deployed to multiple mobile devices. Manages a good deal of the application lifecycle management functions. Devices covered include: Apple iPhone, iPad, BlackBerry, and Microsoft Windows 32/64 and Windows Mobile and Phone 7. Uses the Eclipse IDE and Sybase SQL Anywhere embedded mobile database. Development is with a 4GL environment.
Syclo AGENTRY
Eclipse-based development environment uses a model-driven approach with 4GL requiring no coding. Provides many aspects of ALM features for development, testing, and deployment management. Applicable for a broad range of devices including Windows/Windows Mobile, iOS, and BlackBerry for offline and online apps, and integration with back-end enterprise applications.
Development tools for smart devices general MDE market

This category contains many vendors offering tools and services, so what follows should be considered a representative sample rather than an exhaustive list. Furthermore, these are relatively new players with currently small customer bases. Ovum lists these vendors without any implication as to their longterm viability. Many will likely remain small, fall by the wayside, or be involved in acquisitions, and a few may grow significantly:
Appelerator Titanium
Free and open source, the Titanium development platform offers desktop and mobile editions. Source files are created in HTML5, CSS, and other web technologies (JavaScript, Python, Ruby, PHP) and solutions run natively on iOS (in Objective-C) and Android (in Java) devices. Appelerator recently acquired open source IDE Aptana Studio.
FlexyCore iSpectrum
An Eclipse plug-in development environment for creating mobile apps in Java and automatically generating Objective-C for running on the iPhone. Will also boost Android and Java ME apps. A free Open Edition includes a debugger, an application certificate and App Store distribution, but this is for open source apps only. The commercial editions provide improved features such as enterprise distribution, media player support, and storage.
Genuitec MobiOne Studio

A development environment that uses HTML5, JavaScript, and other web technologies to create and test mobile apps. Currently supports the iPhone; in the pipeline is support for Android, BlackBerry, and Nokia.
78
Mosync
An open source mobile phone development environment. It offers a free Community edition for open source app development. Basic Pro is the same as Community but for commercial apps. Gold Pro adds services such as technical support. MoSync integrates with Eclipse for C/C++ app development. Supports MoSync emulator and the following devices: iOS (iPhone, iPad, iPod Touch) 3.x +, Android Cupcake 1.5, 1.6, 2.1, Java ME MIDP 2 (J2ME), Moblin 2.x, Pocket PC 2003, Smartphone 2003, Symbian S60 2nd, 3rd, and 5th ed, and Windows Mobile 5.06.5.
jQuery
Free and open source, jQuery is a JavaScript library for creating Ajax apps and simplifying web development. It has wide popularity including support from Microsoft and Nokia. The mobile version is supported by Adobe, HP, Mozilla, Nokia, and others. jQuery Mobile is designed to run on multiple devices: Android, BlackBerry, HP webOS, iOS, MeeGo, Symbian, and Windows Phone. At the time of writing the framework is in alpha release.
PhoneGap
Free and open source mobile app development environment using web standards. Apps are written in HTML5, CSS3, and JavaScript, wrapped in PhoneGap to access native APIs, then deployed to multiple devices. The PhoneGap build is performed as a cloud service: the developer supplies the web standards based app and the service returns a native app. Supported devices include Android, BlackBerry, HP Palm, and Symbian, with Apple iOS, Windows Phone, MeeGo, and Samsung bada in the pipeline. The PhoneGap Web site hosts many tools for building apps using open web standards.
Sencha Touch
Free and open source, an Ajax framework provider now positioned as an HTML5 and web-standards JavaScript application framework provider. Supports Android and Apple iOS devices. Offers extensive libraries for other type of development including web, Java, and animation.
Squace
Squace offers a universal mobile interface that makes it possible to access the whole Internet and related services effectively from any mobile phone, eliminating many of the problems caused by proprietary platforms and closed communities. The interface efficiently utilizes the small mobile screen, making it very suitable for low-end phones, as well as more advanced ones. It uses a very thin client downloaded once from the Web and is completely independent of the operator, the operating system, or the device manufacturer, providing a consistent interface on all devices.
Volantis
Mobile app development environment Volantis Framework using HTML5, CSS3, and JavaScript to generate native and web apps. The Framework has an app creator with support for device databases. Supports 7,400 devices including Apple iPhone, Android, BlackBerry, Symbian, Windows Mobile, and HP Palm, and over 100 browsers. There is also a free and open source version available called Volantis Mobility Server but with most of the Framework features reduced or turned off.
79
WorkLight
WorkLight Platform offers an environment for developing smartphone apps using XML, CSS, HTML and JavaScript, allowing access to native device APIs, including iPhone, Android, and BlackBerry. The WorkLight Platform supports development on smartphones, tablets, desktop, and web solutions.
Mobile development: additional services

Testing mobile apps is particularly tricky as there are so many different devices to test, and emulators, when available, are far from perfect, can be slow to run, and should never be the sole test target anyway. A number of mobile app testing services have emerged to satisfy the demand for mobile testing; the following is representative of the market. BSQUARE is a One-Stop Resource for Everything Mobile service for testing apps on Android, Windows Mobile, Windows Phone 7 or Windows Embedded mobile devices. The vendor offers many additional services. DeviceAnywhere offers realtime interaction with handsets that are connected to live global networks. Built on the companys patented device interaction technology, DeviceAnywhere Test Center enables developers to connect to and control mobile devices via the Internet for application and content testing. Offering an original, non-simulated, realtime platform, developers can remotely press buttons, view LCD displays, listen to ringtones, and play videos etc, just as if they held the device in their hands. Jamo Solutions links to development environments such as Eclipse, HP QuickTest Professional, and Microsoft Visual Studio using M-eux Test, providing a test script writing environment that then downloads the executable to the mobile device. The Run-on-Device tool executes the scripts using a local scheduler, and generates an execution report. Perfecto Mobile offers MobileCloud, a software-as-a-service (SaaS) platform for testers and developers to access real mobile handsets and tablets connected to live mobile networks spread over different locations. Mob4Hire and uTest offer mobile app crowd testing and market research services. A good resource for mobile development is the Mobiforge website, run by dotMobi (the mobile domain registry), which provides resources for developers to build web apps.
5.5 Recommendations
Choosing the technology for building mobile apps will be at the top of the agenda. Given the maintenance and support burden for managing multiple devices, spanning across different mobile vendors, and across the range of models any single vendor offers, the use of a MDE becomes paramount. Large organizations will want to consider the enterprise-strength MDE that offers a host of management benefits on top of basic core development tools.
80
Some of these enterprise MDE vendors also offer pre-packaged enterprise app solutions. A number of these MDEs allow apps to be built on open web-based technologies. However, some may lock the customer in to a proprietary 4GL. The choice of which solution to opt for will depend on what in-house staff and skills profiles are available.
For any organization making their first steps in mobile app development Ovum recommends opting For any organization making their first steps in mobile app development for open standards Ovum recommends opting for open standards based solutions. For Ajax based solutions.
The choice for the rest of the market is from a range of relatively new players, many offering free and open source tools, and many based on development using the combination of HTML5, CSS3, and JavaScript. These tools generate native app solutions to run on the most popular smart mobile devices. like apps the use of HTML5 and JavaScript is fine, and for higher-end apps there is the Adobe Flash platform. By the end of 2011 we expect JavaFX 2.0 to provide a seamless end-to-end Java solution from mobile device to the corporate backend. Microsoft Silverlight, while gaining ground for .Net based web applications, is a natural choice for Microsoft houses. However, the Windows Phone 7 is still playing catch up in the market, and this needs to be weighed when choosing a development platform.
Recommendations for tools vendors

Most ALM vendors have not considered mobile development as a special case and believe their tools are fine as they are to support mobile development. This is the case for ALM activities such as requirements gathering, requirements management, and project management. The exceptions are IBM and Microsoft: currently IBM Rational views the mobile device in the context of realtime systems and embedded software and has support for the Android OS; and Microsoft is ahead of the other ALM vendors in gearing Visual Studio to support WP7 development (but not other mobile OSs). One can understand Microsofts position with its vested interest in WP7, but for IBM Rational there is an opportunity for a unified mobile development strategy. ALM has been a tough market for vendors in recent years, with Agile and SaaS ALM being the two exceptions that have shown growth. At the same time the smart mobile development market has leapt forwards and new MDE tools are appearing, some of which overlap ALM functionality. While MDE can be considered the mobile equivalent of an IDE with additional features such as testing and deployment, the enterprise MDEs are already displacing the need for traditional ALM systems to cover many management aspects of mobile development. Part of the change taking place is that whereas mobile apps up to now have been the activity of mainly individuals and small teams, the size of teams are growing as apps become more sophisticated and also join larger projects with significant server-side activity. In the longer term, as mobile development activity continues to grow and become comparable to other application development, Ovum expects that ALM vendors will need to address the needs of mobile developers. This will mean integration with MDEs; as many MDEs are Eclipse based this task will be made easier. However, the fragmented nature of the mobile market will challenge ALM vendors. Supporting Agile methodologies in mobile development tools is where Ovum sees market differentiation and opportunities to occur as the market matures.
81
Incorporating
OVUM Butler Group
CHAPTER 6: ALM in systems/product engineering: two case studies
WWW.OVUM.COM
6.1 Summary
Catalyst
In the world of engineered products, software is increasingly defining products, rather than simply enhancing them. Todays automobiles and aerospace systems are no longer electromechanical products, but complex, intelligent systems that optimize every aspect of operation. Todays consumer electronic devices are differentiated more by the software than by the physical features of the product. Ovum finds that the growing importance of embedded software includes: changing the roles of people; eroding the barriers between software and traditional mechanical, electrical, and systems engineering disciplines; and changing the market dynamics between ERP, PLM, and ALM vendors that support this space. (See the Ovum report Software Development in the Product Lifecycle.) Our research also reveals that lifecycles for embedded software development vary by industry. This report provides dual case studies. The firstst pinpoints how the role of embedded software is impacting not only the product, but also the requirements and change management processes, for a major European tier-1 automotive supplier. The second discusses how software ALM and Agile processes are making a difference at a leading American aerospace contractor.
Ovum view
Todays automobile carries roughly the same degree of intelligence as an iPhone or Android smartphone, which is provided through a network of 80100 controllers that govern all the major functions of the car. For the major tier-1 European automotive supplier profiled in this report, the importance of software has changed the organization itself and the way product requirements and definitions are managed. With software engineers now outnumbering mechanical and electrical engineers by a 2:1 margin, the main challenge has not been over who owns the product definition, but instead, which process and tooling to adopt for managing critical artifacts of the product lifecycle. The battles should be familiar to veterans of ERP adoption. For the aerospace firm, progress is at an earlier stage; application lifecycle management practices and tooling are being phased in.
Key messages
ALM is crucial to drive down the cost of process management: as the entire lifecycle is managed in a cohesive and collaborative manner, the process interface and interaction costs will decrease. From a change management standpoint, software engineering is far more volatile than mechanical or electrical engineering. ALM consolidation in a systems engineering company enables project transparency. Agile processes can work, even within lifecycles for products whose structures are extremely complex. Large organizations must accommodate organizational, process, and tooling diversity when developing enterprise standards for managing the product lifecycle. A federated approach is the only option that is practical. Systems and software engineers aim to improve their collaboration as a result of Agile adoption on the software side; Agile software engineers can work with non-Agile systems engineers. Lessons can be learned from enterprise systems projects such as ERP or CRM migrations: change is painful, so implementers should be sure to communicate benefits that are tangible, not abstract, to their organizations.
CHAPTER 6: ALM IN SYSTEMS/PRODUCT ENGINEERING: TWO CASE STUDIES
85
6.2 Embedded software transforms the products business

Key industry forces are reshaping embedded software development
Durable goods of all kinds are getting smarter, with prime examples being smartphones that are as powerful as microcomputers were not so long ago, aircrafts that are flown by wire rather than hydraulic controls, and of course automobiles. In the automotive sector, several trends have driven the growth of embedded software: Moores Law, which has made it economical and feasible to develop and deploy microcontrollers through the driving machine consumer demand for advanced features, such as high-tech entertainment and convenience systems (multimedia for passengers, GPS navigation for the driver, etc.), or advanced operational features such as anti-lock braking or rearview video for parking and maneuvering around tight parking spaces regulatory compliance pressures for fuel economy, safety, and environmental protection, which have created a need for advanced electronics governing fuel efficiency, emissions control, and safety features such as air bags and smart bumpers. The result is that software development itself has become central to product designers; smart controllers are no longer designed as afterthoughts. The Ovum report Software Development in the Product Lifecycle reviews in detail the degree to which embedded software is becoming ubiquitous and how it reshapes products (such as automobiles): Changes by product sector There is no single software or product engineering process that applies to all industries. Impact on people The growing role of embedded software in products is redefining software engineering and the relationships of software engineers with mechanical, electrical, and systems engineers. It is redefining who owns the product, as software engineering plays a more prominent role in shaping and defining the product. Impact on processes Growing interaction between software, mechanical, electrical, and systems engineering disciplines changes the rules of engagement for managing product requirements, product definition and the bills of materials (engineering and manufacturing), design, quality assurance, and release and change management. As more players and constituencies are involved, synchronizing change cycles grows more challenging. Impact on technology and solutions The delineation (and overlap) of responsibilities impacts how organizations use ALM, PLM, and ERP systems in the product lifecycle. Overlap is significant in areas such as design, requirements, quality, configuration, and release management.
6.3 Case study one: requirements management for a diversified european tier-1 automotive supplier
Organizational background
Ovum interviewed the director of IT for engineered systems of a large, diversified European tier-1 automotive supplier. The organization uses requirements, change, and configuration management tooling from MKS and IBM Rational.
86
The company has mirrored the rest of the automotive industry, having undergone significant mergers, acquisitions, and restructuring. Over the past decade, the company has acquired nearly one dozen companies, especially in automotive components, doubling in size to over $20bn. It has dozens of groups, which fall under several divisions that make everything from chassis and powertrain to interior (passenger compartment) components. Together, the automotive components business employs 20,000 product engineers, of whom significantly two-thirds are software engineers. That is in response to the reality that automotive components are designed differently today. A decade or more ago, these were controlled by hydraulic systems coupled with pedals, but today the hydraulics have been replaced with electronic sensors and actuators, which are coupled with pedals and microcontrollers.
Modularity is reshaping the role of automotive suppliers

With few exceptions, almost every manufacturing sector has gone virtual. Long gone are the days of total vertical integration, in which product design, components and parts production, and final assembly were all performed inside the same company. This trend has been ongoing since the 1950s. The rationale is flexibility and lower cost. When a manufacturer relies on suppliers, changes in the design of a product can be accomplished by switching suppliers rather than having to retool and restock existing facilities and retrain internal staff. Consequently, in the automotive industry, supplier networks are well established and typically comprised of multiple tiers of suppliers that provide raw materials, fabricate them into parts, and then assemble them into major components such as batteries, transmissions, wheel axles, tires, brakes, windows, and so on. In recent years the trend has accelerated, and tier-1 suppliers are now designing and assembling entire subsystems or modules, such as the passenger compartment (or cockpit, which contains the passenger quarters, dashboard, GPS, entertainment, and other systems), chassis (which includes braking and underbody, plus active and passive safety controls), and powertrain (which manages the engine, transmission, and fuel system). What has happened in automotives has occurred in almost every other durable goods manufacturing sector: Electronics product makers outsource chip fabrication, circuit and processor manufacturing, component production and, in many cases, management of the supply chain. For instance, Apple relies on Chinese supplier Foxconn to manufacture iPhones and iPads, rather than make its iconic products itself. Aerospace manufacturers have long operated highly complex supplier networks to assure supply of components and, in some countries, in response to political pressures to distribute production (and contract revenues) across multiple local jurisdictions. Appliance manufacturer supply chains increasingly mirror those of automakers as leading brands outsource some or all of their manufacturing and supply. The impact of the trend towards greater diversification of the product value chain is that tier-1 suppliers are becoming, in effect, mini-automakers, in that they are assuming broader responsibility for product development. The industry has restructured accordingly, with mergers and acquisitions creating tier-1 suppliers, not like the specialized suppliers of specific parts and systems of old, but as varied and complex as the automakers that they serve. Not surprisingly, when it comes to rationalizing roles, processes, and tools, they are encountering similar levels of difficulty to large global enterprises trying to harmonize their ERP implementations.
Requirements and change management process workflow

Software functionality is growing more sophisticated
In most cases, automakers still specify the requirements for the systems that suppliers produce. However, that script is beginning to change. In more cases, roles are changing; the supplier in this case study is increasingly innovating on its own and selling its designs to automakers, a scenario that occurs approximately 20% of the time. For instance, it recently created an enhancement to make diesel engines more fuel efficient, and licensed that design to different automakers using high-precision engineered components that are sensitive down to the micrometer. These, in turn, are combined with sophisticated software.
87
In part, such innovations require new degrees of precision engineering that increase the sophistication of the associated software. However, the primary impact is that when significant investment is made in designing such precision parts, the investment must be leveraged with a design that is flexible enough to fulfill a range of requirements. That reinforces the need to apply disciplines such as Software Product Families (SPL), which is based on the notion that parts should be designed to accommodate products that include multiple models or variations. For requirements, this drives the need for the ability to adapt and reuse. That practice is common in the automotive field, and even more critical as software is embedded into highly sophisticated parts designs.
Software does not get special treatment

Given that two-thirds of this suppliers product engineering staff write or work with software, software development is intertwined with mechanical, electrical, and systems development. Although the activities may be discrete defining the mechanical, electrical, and software designs are typically separate concurrent streams they are all managed under a common project umbrella for the specific component. As a result, software requirements are part of product requirements; changes to software are product-specified engineering changes, and therefore not managed as a separate process. Nonetheless, as shown in Figure 10, there are variations in patterns of activity. Although the streams are concurrent, changes to mechanical or electrical specifications need to occur earlier because the impact will cascade back into the supply chain, requiring suppliers to produce different parts or deliver different raw materials. For instance, upgrading a braking system may require redesigned parts and electronics from suppliers and, of course, with it, the need to develop new or modified software. However, once the braking system specification is set, there may be numerous downstream modifications to the embedded software that further optimize its operation. By contrast, changes to software usually do not require changes to physical Ideally, the organization strives components for several reasons. First, given the growing power and miniaturization of processors and memory, it is rare for more powerful to make major software to require larger control modules that impact the design of a part product engineering or circuit. Second, introducing engineering changes to the physical design changes as early in late in the process could jeopardize or delay release of the product.
the process as possible...
However, managing the engineering change request process always involves a delicate balance. Ideally, the organization strives to make major product engineering changes as early in the process as possible; however, it must also avoid unnecessarily delaying a project that could, in turn, delay production and delivery of the product to market. Furthermore, the supplier needs to retain the flexibility to make some late changes that tangibly make the product better aligned to its market, or respond to significant changes in customer demand, government regulation, or technology innovation. The automotive supplier described here reports a greater incidence of software-related engineering changes. It estimates that for every change to a mechanical design, there are likely to be ten engineering changes to electronic circuitry and 100 software changes. Furthermore, the timing of such changes vary, with mechanical design changes coming earlier because of the need for suppliers to respond and for designing the necessary factory production tooling; at the other end of the scale, some software changes may actually occur after the start of production, as they may involve refinements that will not affect parts fabrication or product assembly. The company concluded that it is simpler to handle change requests for software and hardware in the ALM system than it is in the PLM system. This was one of the factors that drove the companys decision to use ALM tools from MKS and IBM Rational (which, in turn, is being phased out as part of a transition to MKS, on the strength of MKSs relational database-oriented repository). MKS Integrity will be integrated with a PLM-related toolchain to manage requirements and engineering change.
88
ECR activity
Rate of changes/ defect issues
Software
(not to scale)
Electrical Mechanical
Product development timeline
Figure 10: Rate of engineering change requests (ECRs)
Source: Ovum
Automobiles are typically designed as product families. Because they encompass multiple models or feature variations for each product line (typically called platforms), configuration management becomes a major activity in the lifecycle. For instance, a higher-level model of the same car may substitute an automatic for a manual transmission, a more powerful engine (for example, six cylinders instead of four), and modifications to the brakes to accommodate a heavier, more powerful car. In most cases, the controllers containing the software will be the same, but the software will have to be modified to fit the models configuration. Much of the activity in product design becomes a mix-and-match rather than write-fromscratch exercise, ensuring the right mechanical, electrical, and software components are designed into the specific model; changes to these components, in turn, impact the test regimen. This places a premium on robust configuration management processes and tooling that supports unified change management.
The requirements process uses the well-established V model

As the products are highly complex, the organization uses the well-established V model common in systems engineering disciplines. (See Figure 11.) Although there are numerous variations on this model, the common thread is a sequential approach that initially defines business requirements (what the product is supposed to do and how it supports the goals of the business such as the market need that the product addresses). This is followed by a hierarchical process for assessing how those requirements are realized in a system and then decomposed to subsystem and component level. The guiding notion is to define the product and drill down to the sum of its parts, and then ensure that there is a test associated with each requirement at each level of the systems hierarchy. The V is formed as product requirements are defined from the top down, but validated and tested from the bottom up. This attention to bottom-up analysis is essential for specifying the right physical components, the configuration of software to support them, and the tests used to validate the code followed by integration testing of code with components, components with subsystems, and subsystems with systems.
Start of production
89
There are similarities and differences with the classic waterfall development model. Like waterfall, the V model is sequential, but unlike waterfall, there is room for recursive iteration as requirements and design are fleshed out through the more detailed systems, subsystems, and component steps. There is a niche for Agile processes Ever since Japanese manufacturers shook up the automotive industry with their adoption of Lean manufacturing practices and, with it, concepts for reducing inventory and other wasteful practices, emphasis on total quality management, and reduction in product lead and manufacturing/delivery cycle time the automotive industry has been challenged to reduce its traditional three- to four-year lead times from concept to rollout of new models. That prompts a tantalizing question regarding software development could Agile development methodologies that have become popular with IT organizations also apply to product companies that rely heavily on complex, multi-tiered supply chains?
Business requirements
User acceptance tests
System requirements
System tests
Sub-system requirements
Integration tests
Component requirements
Component and unit tests
CONSTRUCTION
Figure 11: Requirements V model Source: Ovum
At the organization described in this case study, Agile processes are primarily used for entertainment features developed as part of the dashboard package. The rationale is that, once consumer electronics devices are specified, there are few supply-chain dependencies, and this creates an opportunity for product designers to craft new services that are mostly software (rather than hardware) driven. The Ovum report Requirements Management for Development Projects addressed whether the V model could be adapted for Agile; forthcoming research will study how Agile methodologies could be adapted for the product lifecycle.
Tooling and process

The large preponderance of software engineers in this company reflects the fact that software development has become intrinsic to its product lifecycle; as a result, tools associated with software development, rather than CADCAM/PLM, are used for tracking product (not just software) requirements, quality, change, and configuration management. MKS Integrity was chosen as the system of record enterprise-wide because of its architecture as a single product. The system uses the same repository for all functions; the user can click on a link and retrieve the product configuration, plus the requirements and tests associated with it. MKS Integrity will be accessed by all engineers across the three divisions.
90
As the organization is diverse, a federated approach to managing the product lifecycle was the only feasible option. Although MKS Integrity became the core system for managing the product lifecycle, it would operate alongside multiple systems. On the systems level, MKS coexists with PLM systems such as PTC Windchill for managing materials, physical design, and the engineering bill of materials (BOM), which consists of a hierarchical listing of mechanical design features; and SAP for the manufacturing BOM, which details the materials and parts that make up the product. There are also groups using rival requirements, change, and configuration management tools from IBM Rational. Given the diverse organization, it should not be surprising that there are different levels of process maturity from one operating unit to the next. The challenge of speaking in the same process language is magnified when dealing with clients; even when they use the same tooling, variations in document formats and schema are commonplace. There are also different levels of tool adoption. Although the first unit to implement MKS Integrity did so in 2000, none of the 14 business units has finished the job. (The same is likely true for implementations of Windchill and SAP.) In one unit, implementation has yet to begin. The companys long-term plan is to migrate remaining instances of IBM Rational DOORS to MKS Integrity, although no firm date has been set.
Coexistence is a fact of life

In most product engineering companies, there is a battle between physical engineering and software engineering over who owns the product definition and, with it, product requirements. Yet, for this automotive supplier, that issue was answered with brute numbers: at least two-thirds of the companys engineering force are software engineers. Therefore, instead of turf battles between physical engineering, which uses PLM tools, and software engineering, which uses tooling from ALM providers, here the battle has been largely waged between factions that ...context is typically have used MKS Integrity versus IBM Rational DOORS (for requirements the big casualty management), Synergy (for configuration management), and Change (for when data is change management). Furthermore, many users and customers use exchanged between Microsoft Word or Excel to document requirements. The organization uses Requirements Interchange Format (RIF) to interface processes being between DOORS and MKS Integrity. Like any interchange format, there are integrated. always issues about whether semantics or other design intent information gets lost in the translation, but the organization has not found this to be an issue so far. Admittedly, context is typically the big casualty when data is exchanged between systems instead of processes being integrated. At this point, the ALM and PLM vendor communities have not developed associative changes that automate the change process across systems (for example, automatically updating the counterpart systems when artifacts are changed). The primary strategy for limiting semantic losses comes down to people; even if the interchange format is imperfect, the goal is to get people across business units and external partners to collaborate closely so there is human knowledge of what requirement the design is to address, and how it is to address it. There are further variations when dealing with the companys manufacturing plants around the globe. Different manufacturing plants in different countries will require different manufacturing BOMs (from SAP), based on a maze of local material sourcing, product safety, or other mandates.
systems instead of
Change means change

MKS was selected because of its single-product format. (By comparison, IBM Rationals offerings are separate products that must be integrated.) However, the continued attraction of different parts of the company to DOORS was attributable to familiarity people do not like changing from tools they are accustomed to. In the case of DOORS, it was the underlying document-centric file system that made requirements documents behave much like the Microsoft Word or Excel files with which people were familiar. A change to a DOORS requirements document would not be immediately reflected in downstream tools.
91
Although MKS can render requirements documents in Microsoft Office-like formats, the underlying single repository architecture meant that any change to requirements would be immediately reflected (or flagged) in associated change, quality, and configuration management artifacts. In the grand scheme of things, such features help prevent errors, but still require adjustment to user habits. Migrating from IBM Rational to MKS tools required changes to how product engineering handled change they could no longer take for granted that there would be enough lag time to make tweaks before the changes would be replicated to other systems.
6.4 Case study two: Agile processes for a major aerospace contractor
In most cases software engineering is still a separate organization
Our second case study focuses on a major, highly diversified US aerospace company. Compared with the automotive supplier, the aerospace company makes a much wider variety of products. Organizationally, it is also far more diverse, and in most business units software engineering remains in a separate organizational silo from the product engineering side. The company is in a long-term transition to implementing application lifecycle management practices to improve the reliability and predictability of product development.
Application lifecycle practices serve as process glue

ALM is regarded as a means of enforcing standard practices within the organization. One of the challenges is that there are many product lines, with different programs in those product lines. Given the diversity of product lines, it should not be surprising that the lifecycles are managed differently from each other; as a result, there is little leverage when engineers move across to different projects. For instance, some teams are using Agile methodologies, while others are using classic waterfall methodologies. Ideally, ALM tools can help that process; for this organization, unifying processes through common tooling will be a long slog, as there is as yet no single solution across the organization. The company hopes to consolidate its tools by making IBM Rational Team Concert (RTC) the central ALM glue governing software change and configuration management (SCCM). It plans to start in one part of the company and gradually widen the rollout. SCCM migrations are like Rome they cannot be built in a day. The organization selected RTC as its next-generation SCCM solution on the strength of its interfaces to the wide range of third-party tools currently in use. For instance, there is a substantial Microsoft Visual Studio 2010 installed base to support.
Agile methodologies are starting to make an impact

For many teams, ALM means an Agile process with continuous integration, enabling code reuse and establishing a set of core values; for the company, Agile means Scrum. Yet, in the product engineering world, there have been unique challenges in Agile adoption; at first glance, Agile methodologies that emphasize modest, bite-size incremental deliverables might be at odds with the highly complex product structures that are especially prevalent in aerospace engineering. Yet, this company has seen noted successes with Agile, with some teams realizing tangible benefits. With approximately three years of experience, many teams that have embraced Agile have been able to make product delivery times and product content more predictable. Furthermore, they have improved their ability to deliver what customers really want. The company has approximately 100 Scrum masters across a number of software divisions, with several thousand engineers using scrum. However, although the methodology is similar, there are many variations in the way scrum has been implemented. And, of course, many parts of the organization have not adopted any form of Agile methodology, such as product and systems engineering. The process migration has had philosophical hurdles, with engineers asking why Agile is better.
92
There are speed bumps

Given that some parts of the organization practice Agile while others dont, there are often communications issues. For example, with hardware there will be an understanding or assumption about the order of delivery of some 87 electronic boards. Therefore, during scrum planning, that hardware sequence feeds into the software line priorities. The scrum product owner answering the most important priority has more than just software to consider. The inclusion of the supply side (in this example, the electronic boards) gets into the scope of the software planning. Scrum at the company means that the end of each sprint produces a deliverable that works, has value, is measureable, or is an algorithm something that can be demonstrated and is quantifiable. Sometimes the item is urgent and needed for a hardware component. Therefore the advantage of scrum is asking the right questions and being able to adjust priorities and plans so each sprint produces a work that is deliverable and maximizes value. Overall, Agile adoption has benefited projects at the company because it is causing the engineers to ask the right questions, probe the systems teams to get the answers, and pull data and reconcile differences.
A federated approach to ALM tools integration

Like any diversified organization with diversified tools, getting all to shake hands and communicate is an issue. In this respect, this organization has problems no different from any organization in the midst of enterprise systems implementation, such as ERP or CRM on the business side. ALM integration has similar issues when it comes to Traditionally, integrating processes, data, and tooling. manual import of Traditionally, manual import of data into general-purpose tools such as data into generalMicrosoft Excel has been the default option. In the long run, such manual purpose tools such approaches are time consuming and error prone. Therefore, the company as Microsoft Excel needs ways to pull metrics out of tools and correlate them automatically. Replacing all the legacy ALM products to achieve this automation is cost has been the default option. prohibitive. The organization decided that RTC was the best solution because it could connect with IBM Rational products DOORS, Rhapsody, Synergy, and ClearCase, as well as other defect-tracking solutions. The company is not trying to interface its financial part; rather, it is looking at purely the software development space interfaces with systems engineering, planning, implementing, reviewing, SCCM, and so on. Integration is not yet complete. For Agile planning the company has used Version One, but that entailed transferring data manually from the ALM tools into the product, creating overheads. RTC automates these activities.
There are hurdles to integrating with PLM

ALM and product lifecycle management (PLM) tools at the company do not integrate well together. They can share some data through import and export features, and some parts, such as the financials, are run separately in Microsoft Project. The PLM tools provide a data warehouse for archiving documents such as official releases and customer interface specifications. In addition, PLM is a much broader area of activity, encompassing the software and ALM activities. The company is consolidating its PLM implementations on PTC Windchill and hopes to interface directly with ALM in the future. The majority of activity is through the Windchill data warehouse, and its process scope covers the various mechanical, electrical, and software subsystems. Projects at the company usually start with a request for proposal. The system and software engineers get involved at the start of the proposal solution stage. Systems engineers have the overall product view and system design ownership, while software components are the responsibility of the software engineers. The two groups continue to work together throughout the project lifetime. Some parts, such as algorithm design, are owned by specialist teams or parts of the organization. The systems engineers use PLM tools and need manual steps to interface with ALM tools. However, within the software groups, there is more of a closed-loop process: IBM Rational DOORS can generate work items or updates for RTC, and this change request will automatically go into the next sprintplanning meeting and be tracked throughout the lifecycle.
93
The various engineers make extensive use of models for: architecture, emulation, and simulation. However, a significant hurdle is that moving models down the project lifecycle cannot be done transparently. There are often separate systems and software models, which are different from each other: the company has not seen any vendor addressing this model continuity problem.
QA and change control have been adapted to fit in with Scrum

QA and testing have been impacted by adopting the Scrum Agile methodology. Testing will be pulled into the Agile process going forward. IBM Rationals ALM tools include quality management software that will be more widely integrated as part of future activities. QA is a lifecycle activity QA and testing have that is made up of QA staff sitting with engineers. Engineers peer review each others work with design reviews and gates in place, and refer large decisions been impacted by adopting the Scrum to change control boards (CCBs). However, the way CCBs interact with the Agile teams has not been standardized across all teams; this is evolving as Agile methodology. stakeholders improve their collaboration. There are several layers of CCBs: the product CCB is outside software and has the potential to be a bottleneck; however, software has its own CCB with major deliverables gated. Where Agile and ALM processes are practiced, there is an effort to pull in CCB stakeholders as part of the product owner scrum role. Therefore, decisions are made both during and prior to scrum planning. The company is learning how to minimize the need to wait for CCB approval by incorporating CCB members as part of the normal Agile workflow, such as the sprint-planning meetings. CCBs own the major project gates, such as release to customer. In the past the CCB managed every change; now a backlog of items is approved to work on without additional CCB checking. Thus, the CCB has a major role in gates for release, security, sign-offs, and QA. In turn, QA is involved at a story level as part of the approval flow. The goal is to integrate these stakeholders during the course of the sprint, rather than waiting until a formal CCB meeting to communicate needs.
6.5 Recommendations
Recommendations for enterprises: learn from past enterprise system migrations
Expect cultural resistance to changing entrenched tools, processes, and attitudes. On the business side, you probably learned this lesson the hard way when you implemented ERP, CRM, or even BI systems. Any ERP project veteran can show the scars from battles to get different business units to converge on basic matters, such as the definition of what constitutes a customer and what a customer record should contain. The same applies to defining processes and artifacts in the application lifecycle, especially when it comes to overcoming misinformation on what constitutes a software (or product) requirement. What is interesting with the automotive company profiled in this report is that the brunt of the battles did not happen where one would have expected: between traditional physical engineering (such as mechanical and electrical) and software engineering over who owns product definition. That was already dealt with through brute force, and two-thirds of the engineering staff are now software engineers. Instead, the battles were fought in areas that would look familiar to any ERP veteran, namely which vendor product should be used to manage the lifecycle. When an enterprise standard is adopted, communication is essential to drive home the value proposition of moving people out of their comfort zones. Communicate the big picture on why the change will make a difference, help the business, and (ideally) improve life down in the trenches. (Of course, no matter how good the tool or intentions, bad implementation will only make matters worse.) In these cases, the need to promote the advantages of working not only in a new template, but with fully interactive systems, could bring more immediacy and transparency to the process. To sell people on change, acknowledge that short-term costs and adjustment pains will impact productivity. The obvious example is that data cannot be tweaked in the same way before the change is committed. Promote change by describing tangible benefits: there will be less reworking because you are always working with the most current design update. That, in turn, will contribute to more strategic goals such as improved product quality and faster time to market, which will have an obvious personal impact: better career security.
94
Incorporating
OVUM Butler Group
CHAPTER 7: Technology comparison
WWW.OVUM.COM
7.1 Application Lifecycle Management Features Matrix

Ovums features matrix methdology
The key vendor products evaluated in this report are scored against a features matrix. The entries in the matrix are described in Chapter 3 under the software lifecycle management evaluation model. The entries represent features available out-of-the-box and are scored Y if available, Y* if partially available, and N if not available. Because a row may represent a combination of more detailed features, the Y* indicates that the capability is only partially delivered. Ovum applies a weighting system in its evaluation of the matrix according to its view of what is valued in the market. However, these are removed here, allowing readers and users of the Decision Manager (the interactive version of the matrix) to apply their own criteria. The completion of the features matrix was conducted in parallel with the qualitative interviews for the technology audit, where each vendor was asked to complete a questionnaire posing detailed questions about product capabilities. These were answered with respect to features that were present in the latest, or where applicable soon-to-be-released, versions of each product.
Features Matrix
Atlassian SECTION A: ALM Tools Coverage Breadth of lifecycle discipline coverage Application Project Portfolio Management Project Management Requirements Definition Requirements Management Model-Driven Development Database modeling and design Change & Configuration Management Build Management Deploy and Release Management QA Management Defect/Issue Management Application Performance Management Software Asset Management SECTION B: ALM Integration Framework General Repository support common repository federated repositories Workflow engine Analytic reporting on ALM project health Native IDE integration with Eclipse and/or MS Visual Studio Y Y Y Y Y Y Y Y* Y Y Y Y Y Y Y N Y Y Y Y Y Y Y Y N N Y Y Y Y Y N N Y Y Y Y N N Y Y Y N Y N N Y Y Y Y N N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y CollabNet HP IBM Rational
Feature Key: Y = feature is included out-of-the-box, Y* = partial yes, N = no capability
CHAPTER 7: TECHNOLOGY COMPARISON
97
Micro Focus SECTION A: ALM Tools Coverage Breadth of lifecycle discipline coverage Application Project Portfolio Management Project Management Requirements Definition Requirements Management Model-Driven Development Database modeling and design Change & Configuration Management Build Management Deploy and Release Management QA Management Defect/Issue Management Application Performance Management Software Asset Management SECTION B: ALM Integration Framework General Repository support common repository federated repositories Workflow engine Analytic reporting on ALM project health Native IDE integration with Eclipse and/or MS Visual Studio Y* Y Y* Y Y Y* Y Y Y Y Y Y Y Y Y Y Y N
Microsoft
MKS
Polarion
Y Y N Y* Y Y Y Y Y Y Y Y N
Y Y Y Y N N Y N Y Y Y N N
Y Y Y Y N N Y Y Y Y Y N N
Y Y Y Y Y
Y Y Y Y Y
Y Y Y Y Y
98
Rally Software SECTION A: ALM Tools Coverage Breadth of lifecycle discipline coverage Application Project Portfolio Management Project Management Requirements Definition Requirements Management Model-Driven Development Database modeling and design Change & Configuration Management Build Management Deploy and Release Management QA Management Defect/Issue Management Application Performance Management Software Asset Management SECTION B: ALM Integration Framework General Repository support common repository federated repositories Workflow engine Analytic reporting on ALM project health Native IDE integration with Eclipse and/or MS Visual Studio Y Y Y Y Y Y Y N Y N N N N Y Y Y N N
Serena
TechExcel
Y Y Y Y Y Y N Y Y Y Y N N
Y Y Y Y Y
Y Y Y Y Y
N N Y Y Y
99
Atlassian SECTION B: ALM Integration Framework (continued) General (continued) Integraton with other IDEs Are mainframe & distributed platforms covered by same tools hosted (SaaS/PaaS) tools Cloud readiness/support development for the cloud Web-based client access Agile suppport: continuous integration continuous testing continuous deployment can process guide be accessed within ALM system at point of application Product Lifecycle Management integration: PLM solutions supported CAD, CAE systems supported Y Y Y Y N Y N Y
CollabNet
HP
IBM Rational
Y N Y Y Y Y Y N
Y Y Y Y Y Y Y Y
Y Y Y Y Y Y Y Y
N N
N N
N N
Y Y
Internal Tool Interoperability (within ALM Suite and across other your products) Does the Application PPM tool read the Project Management tool Does the Proj. Mgmt. tool read the Requirements Management tool Does the Proj. Mgmt. tool read the Software Change and Configuration Mgmt. tool Does the Proj. Mgmt. tool read the Quality Mgmt. tool Is there two-way read and write between the Quality Mgmt. tool and Defect Tracking tool Is there a tightly integrated platform: with entities and artefacts traced across whole lifecycle can new events and workflow automation be triggered by artefact relationship events External Tool Interoperability (with third party tools) APIs: published royalty-free web services based Is there an SDK? Y Y Y Y Y Y Y Y Y Y Y Y Y N N Y Y Y Y N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y
100
Micro Focus SECTION B: ALM Integration Framework (continued) General (continued) Integraton with other IDEs Are mainframe & distributed platforms covered by same tools hosted (SaaS/PaaS) tools Cloud readiness/support development for the cloud Web-based client access Agile suppport: continuous integration continuous testing continuous deployment can process guide be accessed within ALM system at point of application Product Lifecycle Management integration: PLM solutions supported CAD, CAE systems supported Y Y Y Y Y Y N Y*
Microsoft
MKS
Polarion
Y Y Y Y Y Y Y Y
N Y N N Y Y Y Y
Y Y Y Y Y Y Y Y
N N
N N
Y N
Y Y
Internal Tool Interoperability (within ALM Suite and across other your products) Does the Application PPM tool read the Project Management tool Does the Proj. Mgmt. tool read the Requirements Management tool Does the Proj. Mgmt. tool read the Software Change and Configuration Mgmt. tool Does the Proj. Mgmt. tool read the Quality Mgmt. tool Is there two-way read and write between the Quality Mgmt. tool and Defect Tracking tool Is there a tightly integrated platform: with entities and artefacts traced across whole lifecycle can new events and workflow automation be triggered by artefact relationship events External Tool Interoperability (with third party tools) APIs: published royalty-free web services based Is there an SDK? Y Y N Y Y Y N Y Y Y Y Y Y Y Y Y Y N N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y
Y*
101
Rally Software SECTION B: ALM Integration Framework (continued) General (continued) Integraton with other IDEs Are mainframe & distributed platforms covered by same tools hosted (SaaS/PaaS) tools Cloud readiness/support development for the cloud Web-based client access Agile suppport: continuous integration continuous testing continuous deployment can process guide be accessed within ALM system at point of application Product Lifecycle Management integration: PLM solutions supported CAD, CAE systems supported Y Y N N Y Y Y Y
Serena
TechExcel
Y Y Y N Y N N Y
N Y Y Y Y Y Y Y
N Y Y Y Y Y Y Y
N N
N N
Y N
N N
Internal Tool Interoperability (within ALM Suite and across other your products) Does the Application PPM tool read the Project Management tool Does the Proj. Mgmt. tool read the Requirements Management tool Does the Proj. Mgmt. tool read the Software Change and Configuration Mgmt. tool Does the Proj. Mgmt. tool read the Quality Mgmt. tool Is there two-way read and write between the Quality Mgmt. tool and Defect Tracking tool Is there a tightly integrated platform: with entities and artefacts traced across whole lifecycle can new events and workflow automation be triggered by artefact relationship events External Tool Interoperability (with third party tools) APIs: published royalty-free web services based Is there an SDK? Y Y Y Y Y Y Y Y Y Y Y Y Y Y N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y
102
Atlassian SECTION B: ALM Integration Framework (continued) Collaboration Is there built-in Instant Messaging? Can work artefacts be linked to documents? Can the following be annotated/commented: source code scripts change requests tests Collaboration is Wiki-based with read and write capability Process/methodology authoring and publishing Author(s) of artefacts visible Change and Configuration Management Change & configuration Management/SCCM provided natively (vendor has tool) via integration to 3rd party tool If provided natively: applies beyond just software CCM visual workflow with automation and enforcement automated branch inheritance explore tool following branch inheritance move and rename branches visual branch & merge visual file history analysis and differencing apply policies at check-in audit tracking file and folder versioning Tool availability on premises cloud (SaaS/PaaS) Architecture Graphical architecture modelling Synchronization between models and code Y N Y Y Y Y Y Y Y Y Y Y N Y
CollabNet
HP
IBM Rational
N Y Y Y Y Y Y Y Y
N Y N Y Y Y N Y Y
Y Y Y Y Y Y Y Y Y
Y Y Y
N Y N
Y Y Y
Y Y Y Y Y Y Y Y Y Y
N N Y Y Y Y Y Y Y Y
N N N N N N N N Y Y
Y Y Y Y Y Y Y Y Y Y
N N
Y N
Y Y
103
Micro Focus SECTION B: ALM Integration Framework (continued) Collaboration Is there built-in Instant Messaging? Can work artefacts be linked to documents? Can the following be annotated/commented: source code scripts change requests tests Collaboration is Wiki-based with read and write capability Process/methodology authoring and publishing Author(s) of artefacts visible Change and Configuration Management Change & configuration Management/SCCM provided natively (vendor has tool) via integration to 3rd party tool If provided natively: applies beyond just software CCM visual workflow with automation and enforcement automated branch inheritance explore tool following branch inheritance move and rename branches visual branch & merge visual file history analysis and differencing apply policies at check-in audit tracking file and folder versioning Tool availability on premises cloud (SaaS/PaaS) Architecture Graphical architecture modelling Synchronization between models and code Y Y Y Y Y N Y Y Y Y Y N Y Y
Microsoft
MKS
Polarion
Y Y Y Y Y Y N Y Y
N Y Y Y Y Y Y Y Y
N Y Y Y Y Y Y Y Y
Y Y Y
Y Y Y
Y Y Y
Y Y N N Y Y Y Y Y N
Y Y Y Y Y Y Y Y Y Y
Y Y Y Y Y Y Y Y Y N
Y Y Y Y Y Y Y Y Y Y
Y Y
N N
N N
104
Rally Software SECTION B: ALM Integration Framework (continued) Collaboration Is there built-in Instant Messaging? Can work artefacts be linked to documents? Can the following be annotated/commented: source code scripts change requests tests Collaboration is Wiki-based with read and write capability Process/methodology authoring and publishing Author(s) of artefacts visible Change and Configuration Management Change & configuration Management/SCCM provided natively (vendor has tool) via integration to 3rd party tool If provided natively: applies beyond just software CCM visual workflow with automation and enforcement automated branch inheritance explore tool following branch inheritance move and rename branches visual branch & merge visual file history analysis and differencing apply policies at check-in audit tracking file and folder versioning Tool availability on premises cloud (SaaS/PaaS) Architecture Graphical architecture modelling Synchronization between models and code N N N Y N Y Y N N Y Y Y N Y
Serena
TechExcel
N Y Y Y Y Y N Y Y
Y Y Y Y Y Y Y Y N
Y Y Y Y Y Y Y Y Y
Y Y Y
N Y N
N Y N
N N N N N N N N Y Y
Y Y Y Y Y Y Y Y Y Y
N N N N N N N N N N
N N N N N N N N Y Y
N N
Y Y
N N
105
Atlassian SECTION C: ALM Business Solutions IT-Business Alignment Application Project Portfolio Management capability Multiple projects management Critical path analysis Financials management Planning and scheduling What-if modelling Resource management Demand management Risk management: analysis tracking using risk priority numbers Reporting: ad hoc pre-built Tool availability: on premises cloud (SaaS/PaaS) Project Management capability Use case and task estimation Flexible tracking of work effort Resource allocation and scheduling Manage backlogs: product, release, project, Sprint/iteration Hierarchical stories/tasks structure Prioritization of stories/tasks Customisable cards allowing comments/annotations and flags Electronic workflow whiteboard Gather metrics from: own plugged-in tools 3rd party plugged-in tools Agile charts: velocity and burndown/up Project Intelligence (BI for app dev real-time analytics and reporting) Project reporting: metrics, stats, charts, dashboard report generation Tool availability on premises cloud (SaaS/PaaS) Y Y N Y Y Y Y Y Y N Y N Y Y Y Y Y N N Y N N N N N Y Y Y Y
CollabNet
HP
IBM Rational
Y N N Y N Y N N N Y Y Y Y
Y Y Y Y Y Y Y Y N Y Y Y Y
Y N Y Y Y Y Y Y Y Y Y Y Y
Y Y Y Y Y Y Y Y Y N Y Y Y Y Y Y
Y Y Y Y Y Y Y N Y Y Y N Y Y Y Y
Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y
106
Micro Focus SECTION C: ALM Business Solutions IT-Business Alignment Application Project Portfolio Management capability Multiple projects management Critical path analysis Financials management Planning and scheduling What-if modelling Resource management Demand management Risk management: analysis tracking using risk priority numbers Reporting: ad hoc pre-built Tool availability: on premises cloud (SaaS/PaaS) Project Management capability Use case and task estimation Flexible tracking of work effort Resource allocation and scheduling Manage backlogs: product, release, project, Sprint/iteration Hierarchical stories/tasks structure Prioritization of stories/tasks Customisable cards allowing comments/annotations and flags Electronic workflow whiteboard Gather metrics from: own plugged-in tools 3rd party plugged-in tools Agile charts: velocity and burndown/up Project Intelligence (BI for app dev real-time analytics and reporting) Project reporting: metrics, stats, charts, dashboard report generation Tool availability on premises cloud (SaaS/PaaS) Y Y Y Y Y Y Y N Y Y N Y Y Y Y N N N Y* Y* N Y* N Y* Y* Y* Y* Y N
Microsoft
MKS
Polarion
Y Y Y Y Y Y Y Y Y Y Y Y Y
Y N Y Y Y N Y Y Y Y Y Y N
Y N Y Y N Y Y Y Y Y Y Y Y
Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N
107
Rally Software SECTION C: ALM Business Solutions IT-Business Alignment Application Project Portfolio Management capability Multiple projects management Critical path analysis Financials management Planning and scheduling What-if modelling Resource management Demand management Risk management: analysis tracking using risk priority numbers Reporting: ad hoc pre-built Tool availability: on premises cloud (SaaS/PaaS) Project Management capability Use case and task estimation Flexible tracking of work effort Resource allocation and scheduling Manage backlogs: product, release, project, Sprint/iteration Hierarchical stories/tasks structure Prioritization of stories/tasks Customisable cards allowing comments/annotations and flags Electronic workflow whiteboard Gather metrics from: own plugged-in tools 3rd party plugged-in tools Agile charts: velocity and burndown/up Project Intelligence (BI for app dev real-time analytics and reporting) Project reporting: metrics, stats, charts, dashboard report generation Tool availability on premises cloud (SaaS/PaaS) Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y N N Y Y Y Y
Serena
TechExcel
Y Y Y Y Y Y Y Y Y Y Y Y Y
Y Y N Y N Y N N N Y Y Y Y
Y N Y Y N Y N Y Y Y Y Y Y
Y Y Y Y Y Y Y Y Y N Y Y Y Y Y Y
108
Atlassian SECTION C: ALM Business Solutions (continued) IT-Business Alignment (continued) Requirements Definition capability Semantic analysis of text for ambiguity, omissions, and compliance Customisable user interface labelling Glossary Convert unstructured input data to structured formats Output formats: structured semi-structured Generate work flows: logic/application process UI renderings/screen Traceability (for audit trails) supported Multiple definitions versions supported Collaboration capabilities Tool availability: on premises cloud (SaaS/PaaS) Requirements Management capability Requirements interdependency mapping Requirements tracking through lifecycle Baselining Change impact analysis Coverage analysis Requirements-based test case generation Visual requirements representation modeling synchronized text and models Y N N N N N N N Y on premises cloud (SaaS/PaaS) Y Y N Y Y Y N N N N N N Y Y Y
CollabNet
HP
IBM Rational
N Y Y N Y Y N N Y N Y Y Y
N Y Y Y N Y N N Y Y Y Y Y
N Y Y Y Y Y Y Y Y Y Y Y Y
Y Y Y N N N N N Y Y Y
Y Y Y Y Y Y N N Y Y Y
Y Y Y Y Y Y Y Y Y Y Y
Reporting Tool availability:
109
Micro Focus SECTION C: ALM Business Solutions (continued) IT-Business Alignment (continued) Requirements Definition capability Semantic analysis of text for ambiguity, omissions, and compliance Customisable user interface labelling Glossary Convert unstructured input data to structured formats Output formats: structured semi-structured Generate work flows: logic/application process UI renderings/screen Traceability (for audit trails) supported Multiple definitions versions supported Collaboration capabilities Tool availability: on premises cloud (SaaS/PaaS) Requirements Management capability Requirements interdependency mapping Requirements tracking through lifecycle Baselining Change impact analysis Coverage analysis Requirements-based test case generation Visual requirements representation modeling synchronized text and models Y Y Y Y Y Y Y Y Y on premises cloud (SaaS/PaaS) Y N Y Y Y Y Y Y Y Y Y Y Y Y N
Microsoft
MKS
Polarion
N N N N N N Y Y N N Y N N
N Y N Y Y Y Y N Y Y Y Y N
N N Y Y Y Y N N Y Y Y Y Y
N Y Y Y N N N N Y Y Y
Y Y Y Y Y N Y Y Y Y N
Y Y Y Y Y Y N N Y Y Y
110
Rally Software SECTION C: ALM Business Solutions (continued) IT-Business Alignment (continued) Requirements Definition capability Semantic analysis of text for ambiguity, omissions, and compliance Customisable user interface labelling Glossary Convert unstructured input data to structured formats Output formats: structured semi-structured Generate work flows: logic/application process UI renderings/screen Traceability (for audit trails) supported Multiple definitions versions supported Collaboration capabilities Tool availability: on premises cloud (SaaS/PaaS) Requirements Management capability Requirements interdependency mapping Requirements tracking through lifecycle Baselining Change impact analysis Coverage analysis Requirements-based test case generation Visual requirements representation modeling synchronized text and models Y Y Y Y Y Y Y N Y on premises cloud (SaaS/PaaS) Y Y N Y N Y N Y N N Y Y Y Y Y
Serena
TechExcel
N Y Y Y Y Y Y Y Y Y Y Y N
N Y Y N Y Y Y Y Y Y Y Y Y
N Y Y Y N Y Y Y Y Y Y Y Y
Y Y Y Y Y Y Y Y Y Y N
Y Y Y Y Y Y Y N Y Y Y
Y Y Y Y Y N N N Y Y Y
111
Atlassian SECTION C: ALM Business Solutions (continued) Software Assurance Software assurance integrations Electronic signature support Requirements/QA integration: are requirements and test plans maintained in same or federated repository? automated validation of test plan coverage of requirements conduct gap analysis of test plans through requirements tool App. Performance Mgmt. Integration: IT Service Desk integration: use performance testing scripts to do synthetic tests of actual performance autopopulate defect/issue tracking with code-related trouble ticket incidents/problems view test results & test coverage by project within the PPM tool? can defect tracking tool autopopulate task lists for specific team members? automatic generation of log entries from tests end user access for log creation link back to related resources (build, VM, etc) customisable visual workflow Quality Management capability Track tests by requirement Test case branching and reuse Automation in use case test authoring and refactoring Historical based analytics & reporting Can product features / tasks be represented on a risk chart? Manage combinatorial testing Scheduling & automated execution of tests Test data management Unit test management Manage test lab environment: on premise on cloud Third-party QA & requirements mgmt. tool integration N N N N N N Y N N N N N N Y
CollabNet
HP
IBM Rational
N Y
N Y
N N
PPM/QA integration:
Defect tracking/task management integration:
N N Y Y
Y Y Y N
Y Y Y Y
Y Y Y Y
N N N N N N N N N Y Y Y
Y Y Y Y Y N Y Y Y Y N Y
Y Y Y Y Y Y Y Y Y Y Y Y
112
Micro Focus SECTION C: ALM Business Solutions (continued) Software Assurance Software assurance integrations Electronic signature support Requirements/QA integration: are requirements and test plans maintained in same or federated repository? automated validation of test plan coverage of requirements conduct gap analysis of test plans through requirements tool App. Performance Mgmt. Integration: IT Service Desk integration: use performance testing scripts to do synthetic tests of actual performance autopopulate defect/issue tracking with code-related trouble ticket incidents/problems view test results & test coverage by project within the PPM tool? can defect tracking tool autopopulate task lists for specific team members? automatic generation of log entries from tests end user access for log creation link back to related resources (build, VM, etc) customisable visual workflow Quality Management capability Track tests by requirement Test case branching and reuse Automation in use case test authoring and refactoring Historical based analytics & reporting Can product features / tasks be represented on a risk chart? Manage combinatorial testing Scheduling & automated execution of tests Test data management Unit test management Manage test lab environment: on premise on cloud Third-party QA & requirements mgmt. tool integration Y Y Y Y Y Y Y Y Y N Y Y Y Y
Microsoft
MKS
Polarion
N Y
Y Y
Y Y
PPM/QA integration:
Y Y Y Y
Y Y Y Y
Y Y Y Y
Y Y Y Y
Y N N Y Y N Y Y Y Y N Y
Y Y Y Y Y N Y N Y N N Y
Y Y Y Y Y N Y N Y Y Y N
113
Rally Software SECTION C: ALM Business Solutions (continued) Software Assurance Software assurance integrations Electronic signature support Requirements/QA integration: are requirements and test plans maintained in same or federated repository? automated validation of test plan coverage of requirements conduct gap analysis of test plans through requirements tool App. Performance Mgmt. Integration: IT Service Desk integration: use performance testing scripts to do synthetic tests of actual performance autopopulate defect/issue tracking with code-related trouble ticket incidents/problems view test results & test coverage by project within the PPM tool? can defect tracking tool autopopulate task lists for specific team members? automatic generation of log entries from tests end user access for log creation link back to related resources (build, VM, etc) customisable visual workflow Quality Management capability Track tests by requirement Test case branching and reuse Automation in use case test authoring and refactoring Historical based analytics & reporting Can product features / tasks be represented on a risk chart? Manage combinatorial testing Scheduling & automated execution of tests Test data management Unit test management Manage test lab environment: on premise on cloud Third-party QA & requirements mgmt. tool integration Y N N Y Y N N Y N N N Y N Y
Serena
TechExcel
Y Y
Y Y
N N
PPM/QA integration:
N N Y Y
Y Y Y Y
Y Y Y Y
Y N N N
Y Y N Y Y N Y N N Y Y Y
Y Y Y Y Y Y N Y Y Y Y Y
Y Y N N N N Y Y N N N N
114
Atlassian SECTION C: ALM Business Solutions (continued) Software Assurance (continued) Application Security Integrate with static testing (code scanning) tools Integrate with dynamic testing tools Security issues autopopulate defect tracking system Security issue impact analysis Risk prioritisation of security issues Reporting: security assessment Enterprise Agile APPM/Agile planning integration Multi-project Agile planning Multi-project Agile resource balancing/allocation are Agile proj KPIs visible inside APPM tool? do Agile planning tools have visibility to mult. Agile projs? can Agile planning tools rebalance resources across projs? compare burndown rates by team Multi-project dependency management does Agile planning track relationships between projects? can projects and/or iterations be prioritized? Multi-project risk management assess risks for completing specific stories within proj. iterations/timelines Y is reporting bundled as core functionality N N N N N N
CollabNet
HP
IBM Rational
Y Y N N N N
Y Y Y Y Y Y
Y Y Y Y Y Y
Support for process impediments Support for hybrid Agile and other processes Agile/QA Mgmt. integration Delivery Management Continuous integration private workspace public streams Build rollback Dependency tracking Intelligent build (e.g., only needs to build files that have been changed) Build validation Integration with Ant OR Maven Deployment Management Release Management compare defect rates and other KPI across teams/projects
Y Y Y
Y Y N
Y Y Y
Y Y Y
N Y N Y Y Y Y Y Y
Y Y N Y N Y Y N Y
Y Y N Y N Y Y Y Y
Y Y Y Y Y Y N Y Y
115
Micro Focus SECTION C: ALM Business Solutions (continued) Software Assurance (continued) Application Security Integrate with static testing (code scanning) tools Integrate with dynamic testing tools Security issues autopopulate defect tracking system Security issue impact analysis Risk prioritisation of security issues Reporting: security assessment Enterprise Agile APPM/Agile planning integration Multi-project Agile planning Multi-project Agile resource balancing/allocation are Agile proj KPIs visible inside APPM tool? do Agile planning tools have visibility to mult. Agile projs? can Agile planning tools rebalance resources across projs? compare burndown rates by team Multi-project dependency management does Agile planning track relationships between projects? can projects and/or iterations be prioritized? Multi-project risk management assess risks for completing specific stories within proj. iterations/timelines N is reporting bundled as core functionality Y Y N N N N
Microsoft
MKS
Polarion
Y Y Y Y Y Y
Y Y Y Y Y Y
N N Y Y N Y
N Y Y
Y Y Y
Y Y Y
Y Y Y
Y Y Y Y Y Y Y Y Y
Y Y Y Y Y Y Y Y N
Y Y Y Y Y Y Y Y Y
N Y N Y Y Y Y Y Y
116
Rally Software SECTION C: ALM Business Solutions (continued) Software Assurance (continued) Application Security Integrate with static testing (code scanning) tools Integrate with dynamic testing tools Security issues autopopulate defect tracking system Security issue impact analysis Risk prioritisation of security issues Reporting: security assessment Enterprise Agile APPM/Agile planning integration Multi-project Agile planning Multi-project Agile resource balancing/allocation are Agile proj KPIs visible inside APPM tool? do Agile planning tools have visibility to mult. Agile projs? can Agile planning tools rebalance resources across projs? compare burndown rates by team Multi-project dependency management does Agile planning track relationships between projects? can projects and/or iterations be prioritized? Multi-project risk management assess risks for completing specific stories within proj. iterations/timelines Y is reporting bundled as core functionality N N N N N N
Serena
TechExcel
N N N N N N
N N N N N N
Y Y N N Y N
Y Y Y
Y Y Y
Y Y Y
Y Y Y
N N N N N N N N Y
N N Y Y Y Y Y Y Y
N N N N N N N N N
Y Y Y Y N Y Y Y Y
117
Heat Map
The features matrix scores were aggregated in each section, suitably weighted for importance of feature points and feature section, placed in a table, and the values heat-mapped. Figure 12 shows the Ovum ALM Vendor Heat Map. The scale below the map indicates how score percentage values map to colors: a light shade indicates high capability, a dark shade low capability. The heat map provides an at a glance view of the features matrix and how individual vendor products compare against each other.
Th ou gh tW Stu ork dio s s
Mic ro Fo cus
Co llab Ne t
ian
oso f
Po lar ion
ren a
ass
Ra lly
Sections
Breadth of lifecycle discipline coverage ALM integration framework: general Internal tool interoperability External tool interoperability Application Project Portfolio Mngt. Application Security Architecture Change and Configuration Mngt. Collaboration Delivery Management Enterprise Agile Project Management Quality management Requirements Definition Requirements Management Software assurance integrations
Color code Feature %: 83.3-100 66.7-83.3 50-66.7 33.3-50 16.7-33.3 0-16.7 0
Figure 12: Ovum ALM Vendor Heat Map
Source: Ovum
7.2 Application Lifecycle Management Decision Matrix

Catalyst
As software development environments become increasingly complex and IT organizations continue to face budget pressures, application lifecycle management (ALM) solutions become part of the toolset necessary to manage development environments and distributed teams, and to ensure timely delivery of increasingly complex application software in a resource-efficient way. Ovum has developed this Decision Matrix report to help enterprises select the most appropriate ALM vendors based on technology strength, reputation among the ALM user community, and presence in the market. Ovum aims to provide a complete view of vendor capabilities, and advises on which vendors enterprises should shortlist, consider, and explore.
118
Tec h
IBM
Mic r
MK
Atl
HP
Se
Ex
cel
Ovum view
Delivery Management is increasingly gaining prominence to address growing software reliability concerns. HP, IBM, Microsoft, and MKS occupy the high end of the market and are the top overall performers. HP and MKS have moved to the shortlist category since Ovums last Decision Matrix: Ovum believes HPs latest offering, HP ALM 11.0, with the leading position in the software quality management market, makes the company a formidable force in the ALM market. MKS advanced on the strength of its integrated platform and reach into the specialized, highly lucrative segment of the ALM market: embedded software in product automation and systems engineering. Serena has a renewed focus and taken the initiative in the application delivery space. Polarion has impressed with its organic growth. Atlassian, CollabNet, and ThoughtWorks Studios are smaller vendors with impressive ALM suites. Rallys, Polarions, and TechExcels respective market positions have improved since the 2009 Decision Matrix report. Micro Focus is the sleeping giant of the ALM space.
Key messages
With economic recovery under way, organizations look for additional value from ALM investments. Regulatory Compliance and Agile adoption remain major drivers of the fragmented ALM market. SaaS, as a business model and an application delivery model, has had pockets of success, and ALM is one of these. Software development for mobile devices and embedded software development are red-hot areas. Application security, previously an overlooked area, has been prioritized by vendors such as HP, IBM, and Microsoft. ALM vendors are struggling to respond to the DevOps movement. Change and configuration management (CCM) functionality has been commoditized by popular open source tools.
Market developments
With economic recovery under way, organizations look for additional value from ALM investments
Since Ovum published its last application lifecycle management (ALM) Decision Matrix report in June 2009, the largest geographic markets for enterprise IT have gone through a severe economic downturn. Predictably, the downturn led to large-scale IT initiatives being shelved, and ALM was no exception. This downturn has again brought the age-old problem of better management of software development processes to the fore. It is common knowledge that IT projects often suffer with budget and schedule overruns and delivered functionality not meeting business requirements.
119
Although the overall ALM market has not grown appreciably in the past year, there are hot spots in areas such as Agile planning, complex/embedded systems development, and expanded take-up of tool offerings through the cloud. Spend rationalization has led organizations to look at ways to both maximize the benefits of existing ALM tool investments by re-examining their development processes, and to explore less capitalintensive options such as open source, cloud-based ALM. Additionally, organizations are keen to break up the siloed IT environment and bring the development function closer to the customer- or business-facing disciplines (or both) such as IT service management, with the overall goal of reliable and secure IT services and applications. On the supply side there is an overall trend towards broader integration, driven by the opportunities to upsell and to be the strategic tools provider that becomes the customers focal point for third-party tool integration. There is also growing support for enhanced collaboration and globally distributed development.
Although the overall ALM market has not grown appreciably in the past year, there are hot spots in areas such as Agile planning, complex/embedded systems development, and expanded take-up of tool offerings through the cloud.
Given the frequency of project failures, organizations have done well to look at alternative approaches to managing software development. The focus is not just on better execution of the traditional software development lifecycle (SDLC) from requirements to release, but also on areas such as application performance management, application security, and bridging the gap between development and operations. Furthermore, emergent solution delivery channels such as SaaS ALM have gained traction in the market.
Regulatory compliance and Agile adoption remain major drivers for the fragmented ALM market
Research reveals that regulatory compliance across industry verticals is a leading driver for the ALM market. Compliance efforts lead to ALM adoption in two ways. Firstly, such initiatives necessitate changes to various business processes and application software that need to be managed at enterprise level. This could mean hundreds or even thousands of application change requests for the development team. Projects of this scale require concerted effort across the organization and involve multiple stakeholders. The need for end-to-end visibility into the lifecycle that spans multiple projects and departments creates a strong case for ALM adoption. Secondly, organizations view ALM as an effective tool for strengthening the governance of development processes themselves. This is because ALM enables organizations to trace business requirements throughout the development lifecycle and further on to application code running in production environments, which creates audit trails for all process artifacts, tasks, and development team members. The software development landscape has shifted in favor of Agile development practices. However, the hype around Agile has died down, and organizations now take a pragmatic approach to Agile there is no single, pure Agile implementation in practice. Therefore, Ovum now sees hybrid environments in which the development methodology adoption is governed by the constraints of the project. Meanwhile, Agile practices also continue to evolve as organizations pick Agile elements that best suit their requirements, such as continuous integration, test-driven development, and (electronic) whiteboards. Agile has also given impetus to related management practices such as Lean and Kanban. These techniques, Kanban in particular, have resonated well with the market, and vendors have followed up with tools to support these processes. Another crucial factor is the availability of skilled and experienced developers. Since its inception, the Agile movement has been led by developers who have wanted lightweight processes that facilitate software development without hindering developer creativity. Thus, organizations faced with skills shortages have not had much success with Agile. Agile ALM vendors have seen significant traction in the market; however, this is not based solely on the rapid rate of Agile adoption. It also has to do with the vendors being agile themselves and offering holistic solutions that couple products supporting Agile with superior customer support and end-user training at an attractive price. On a different note, Ovum expects that the recent consolidation activity in the Agile ALM market will continue, and the functionality offered by various Agile ALM tools will converge.
120
Overall, the market for ALM tools remains fragmented along several fault lines. (See Figure 13.) On the demand side, factors such as the organizations size and the complexity of the IT environment play an important role in determining buying behavior. The bottom end of the market includes many small and medium-sized enterprises (SMEs) and freelance developers, development teams with 10 members or less, and consultants. This segment typically has highly skilled developers who can work alone or in small teams, and prefers Agile methodologies and open source software. The next stratum is comprised of medium to large organizations that often look at ALM from a cost containment perspective. ISVs are a special case, with presence across all size bands large, medium, and small; here developer productivity and tool interoperability remain key issues and, predictably, open source software is used heavily. Large, globally distributed enterprises are at the top of the pyramid; these organizations have existing and ongoing investments in ALM tools and rely heavily on advanced requirements engineering processes. This continues to be the most lucrative segment for ALM vendors in general and Agile ALM vendors in particular as Agile practices permeate into traditional waterfall process territory.
Advanced RD & RM, QA Full ALM stack for traditional development and, increasingly, Agile Sophisticated users Productivity is the #1 issue Will use OSS to premium Mixed patterns major opportunities Reduce costs strong message ALM as a message does not sell to Agile developers OSS ALM tools do well Mass market: SME and freelance/consultants Large businesses Software houses, ISVs, SIs Global enterprises, industrials
Higher-value deals
Addressable market
Figure 13: ALM system market today: Who is buying what? Source: Ovum
SaaS ALM: the big switch

Industry experts have made the business case for SaaS many times over, with arguments typically revolving around terms such as total cost of ownership, enterprise data security, pre-integrated applications, freedom from managing application infrastructure, and so forth. As a business model and application delivery model, SaaS has had pockets of success, and the ALM market is one enterprise software segment where it has done well. ALM tools buyers usually have a software engineering background and can better appreciate the benefits of SaaS; predictably, SaaS ALM vendors Ovum views ALM as have benefited from this aspect of the ALM market, registering strong doublean ideal candidate digit growth both terms of in revenues and installed base. The average revenue for SaaS, and growth rate for ALM vendors whose primary delivery model is SaaS (namely believes SaaS Rally Software, Atlassian, CollabNet, and TechExcel) is 32.5%. Although this growth is impressive, one must also keep in mind that it has come off a penetration will relatively small base compared with larger ALM vendors; whether such highgrow further... growth figures can be sustained year-on-year remains to be seen. Ovum views ALM as an ideal candidate for SaaS, and believes SaaS penetration will grow further, reinforced by other major growth drivers such as Agile development and mobile application development. SaaS has the majority share in the mass market for ALM, and its presence in other segments is bound to increase with time.
121
Notably, the bottom of the ALM pyramid is also the hub for technological innovation in this market; for instance, these are the organizations that have been willing to extend the envelope by working with innovative, lightweight open source tooling. As these new technologies make their way into enterprise environments, the case for SaaS is bound to get stronger. Although technology adoption will largely depend on software refresh cycles, the ...the premise of premise of SaaS ALM can no longer be ignored by large enterprises and SaaS ALM can no large ALM vendors alike. To clarify the point, consider the recent longer be ignored developments in the software configuration management segment of ALM by large enterprises distributed version control systems (DVCSs) represent the next wave of and large ALM innovation in SCM, and the key word is distributed. Any software application or process that is distributed in nature typically spans multiple vendors alike. persons, systems, and geographical regions. This makes such applications and processes ideal candidates for SaaS. Although DVCS adoption at present is greatest at the bottom of the pyramid segment, it is only a matter of time before it penetrates into other segments of the ALM market. Application development is an extremely collaborative activity, and increasingly distributed. Ovum believes organizations are already overcoming their psychological barriers to SaaS, and with increasing DVCS adoption there will be a compelling case for moving the rest of the application lifecycle stack to the cloud. Vendor analysis Rally Software and VersionOne have both set out to deliver hosted ALM services from inception, as their owners recognized the potential for SaaS early on. Not surprisingly, competitors have been drawn in. CollabNet was a predominantly SaaS-based vendor from inception. Its latest acquisition, Codesion, brings into its fold a market-leading hosting provider for CollabNets Subversion software versioncontrol solutions. Codesion (formerly CVSdude) expanded from offering CVS to Subversion, its most popular offering, as well as up-and-coming DVCSs such as Git and Mercurial. The CollabNet portfolio now spans small-scale to enterprise-scale projects, the target market for TeamForge. TechExcel has announced that its next major release in 2011 will offer expanded SaaS capabilities. The company expects growing demand for its hosted solutions and says customers will benefit from faster ramp-up capabilities. Atlassian has been improving the capabilities of its SaaS offering, while continuing to offer desktop solutions. It recently announced the acquisition of Bitbucket.org, a rival to Codesion and principally a hosting provider for Git and Mercurial. Bitbucket provides another step in expanding Atlassians SaaS capabilities and, by offering DVCSs, it fits well in a SaaS context. Although Atlassian shies away from calling its platform an ALM system, it is one in everything other than name. Although IBM Rational lagged in providing access to ALM tooling in the cloud, in 2010 it embraced the cloud with a vengeance. The vendor began with a modest base of offering AppScan security testing and Rational Policy Manager. In 2010, IBM Rational followed up with a multifaceted strategy for offering public and private cloud services. The initial rollout comprised Rational Team Concert, Rational Requirements Composer, Rational Quality Manager, Rational Build Forge, and Rational Asset Manager, hosted by IBM or business partners on IBM or third-party infrastructure, such as Amazon EC2. Serena delivers some of its offerings via the cloud, including the Serena Business Manager orchestration engine, Serena Agile, and Serena PPM. However, it has yet to host any of the core Dimensions requirements or SCCM products through the cloud. ThoughtWorks Studios provides subscription-based offerings through a third-party hosting provider. Other more recent vendors in ALM SaaS are also visible in the market. Tomos, Digite, and Kovair all have offerings, and the overall picture is a market going through a significant transition to SaaS ALM. For now, neither MKS nor Polarion have been proactive. Polarion allows its tooling to be hosted, but does not offer the subscription-based licensing that often accompanies cloud-based deployment. MKS does not yet have active plans as it has not yet encountered sufficient demand from its customer base.
122
Mobile development the new gold rush

Mobile devices computing power and performance is increasing at a breakneck pace. With the increasing popularity of Apple iOS- and Google Android-based devices, the market for mobile application development is heating up, and Ovum expects that it will only grow as businesses go about extending their customer-facing applications to mobile devices. Ovum Ovum believes the believes the next few years will see a great spike in mobile application next few years will development, led by target platforms such as Apple iPhone and iPad, Google Android-powered smartphones and tablets, and Windows Phones. see a great spike in However, application development for mobile devices, or even extending mobile application enterprise applications to such devices, places new constraints with development... respect to the devices form factor, user behavior, and so forth. While mobile applications emerge in the consumer market, the need for extending development to mobile targets is also starting to surface in the enterprise for employee-facing applications. Although in the short term the transition will occur relatively slowly, Ovum believes enterprise adoption of mobile devices as application targets will grow in a fashion similar to that of PCs a few decades ago. Vendor analysis As a group, ALM vendors are still figuring out how to support this emerging area of need in the software development market. So far ALM vendors do not recognize a special need to support mobile development; the main tool activity is in supporting core application development, especially around open web-based technology and translating such models into native apps. None currently provide mobile development environments (MDEs) or partner with third parties that offer them. For now, MDE support is the domain of niche players such as: Rhomobile, with its Ruby-based development framework open source offerings such as Appcelerator Titanium platform, which supports development through multiple common web languages, and mobile deployment management platforms such as GPXS Sybase, which at enterprise level maintains the most complete offering, spanning from application development and asset and security management to delivering business applications and messaging services. Among ALM vendors, only IBM Rational has addressed mobile targets; it has done so to a limited extent. For instance, its Rhapsody realtime UML and SysML modeling tool can deliver preconfigured API libraries for Android platforms, plus a more generic capability for targeting other mobile clients; Rational Application Developer offers code assist, emulation, and validation capabilities for Android and RIM Blackberry devices only. Across IBM Software Group, there are numerous products that address some aspect of mobile application development or deployment. However, IBM has yet to unify its mobile application development tool strategy. As mobile development support tools gain popularity and prevalence, ALM vendors must be able to connect to these and draw out metrics and track work items. In 2011, Ovum expects some ALM vendors to ramp up mobile support; watch for partnerships and acquisitions in this space.
Application security is an area of concern

While software quality has always been a key factor driving ALM adoption in organizations, recent developments suggest that software reliability and application security are the new problem areas. As organizations have proceeded to secure their perimeters, networks, and infrastructure from attacks, application security has been left largely unaddressed and therefore vulnerable. In our view, application security should be factored into the development lifecycle itself, rather than left for third-party tools to address. Security should be an integral part of the application lifecycle, and organizations should conduct security risk assessments at various stages of the lifecycle, such as requirements, design, development, and integration testing. Features such as integration with static and dynamic code testing tools, security issue impact analysis, defect management tool integration, risk prioritization of security issues, and security assessment reporting are no longer simply nice to have from an ALM functionality perspective.
123
Vendor analysis Application security is an area that most ALM vendors have overlooked. Of all the vendors profiled in this Decision Matrix report, most integrate with static and dynamic source code testing and verification tools. However, in our view, this is a basic feature of ALM products; ALM solutions offering application security management functionality must also offer other required features. Thus, in our assessment, only IBM and HP offer functionality to manage application security issues during the lifecycle; they have already taken the first steps toward integrating security testing into the application lifecycle, but have not yet completed the job. Both offer static and dynamic security testing in their portfolios; IBM Rational has integrated static and dynamic testing capabilities, which came from separate acquisitions two years apart, in a consolidated offering that is available in role-based editions. For HP, the big news in 2010 was its acquisition of long-time partner Fortify, which provided static and dynamic testing capabilities plus a new risk management offering. By contrast, MKS integrates with several automated testing tools that only provide spot coverage of security testing issues. Of note, Compuware began developing security testing capabilities within the automated testing tooling that it subsequently sold to Micro Focus; Ovum awaits Micro Focuss forthcoming next-generation release, which will consolidate the quality management assets that came with the Compuware and Borland tool acquisitions. Microsoft has also tackled application security through introducing a security framework. Ovum believes that in 2011 other ALM vendors will need to acquire or partner with niche players in order to start offering their own roundtrips between security monitoring and defect tracking and requirements management (RM) tools.
Software plays a greater role in product engineering

Leading vendors such as IBM Rational and MKS are already seeing the brunt of growth in their businesses in supporting development of software that is embedded in complex systems such as engineered products or civil infrastructure. Additionally, Polarion and Serena are targeting this segment for growth. This market segment has grown in spite of the recession because of continued demand from consumers for high-tech products, and from segments such as public sector and biomedical devices for smarter equipment. Revenue growth in this segment of the ALM market is significant in spite of the fact that it is a niche market in which the number of potential customers is dwarfed by those in the mainstream enterprise sector. However, these customers have a need for large implementations addressing requirements, quality, design, change, and release management implementations that often number in the hundreds or thousands of seats. The average size of deals in this segment is vastly outweighing those of mainstream enterprise IT development. Vendor analysis In this segment, there is entrenched rivalry with large CADCAM players such as PTC, Siemens, and Dassault Systemes. The main attraction for ALM players is that the PLM space is huge compared with their traditional enterprise software development space. However, only a few ALM vendors are rising to the challenge because targeting product lifecycle management (PLM) dictates highly complex project requirements and quality management functionality that might be excessive for enterprise software development. IBM Rational and MKS have taken the most forward positions in targeting this space. Through acquisition of Telelogic, IBM Rational added products such as DOORS for requirements management, Focal Point for portfolio management, and Rhapsody for SysML modeling, which are targeted at complex systems. MKS has refocused its entire ALM business around this fast-growing segment. Additionally, Polarion and Serena have customers in this segment, and plan to ramp up in this space in 2011.
ALM vendors are struggling to respond to the emerging DevOps movement

Like mobile, this is a segment that has caught many ALM vendors off guard. That is because software development and IT operations have traditionally been separate end markets served by different technology vendors.
124
DevOps is the consequence of two trends: top-down adoption of ITIL process initiatives among IT operations organizations, especially for incident, problem, and change management more recently, grassroots movements, especially in the Agile software development community, to make the release management aspect of continuous integration a more predictable activity. Vendor analysis Today, only a handful of vendors with software development and operations/service management/ application performance management tools have built feedback loops between service-desk and defect-tracking tools. Most ALM vendors now include DevOps in the future development slides of their product roadmaps. The logical starting point is for software-related trouble tickets reported by the IT service desk (help desk) to be automatically submitted to defect-tracking systems. Most ALM vendors with defect-tracking tools currently support this capability, some with formally supported integrations to popular support desks such as BMC Remedy, and others with more generalized APIs. The next logical step is to enable repurposing of performance test scripts to be used for conducting synthetic tests on the operational side; a few ALM vendors offer this capability. The more advanced vendors in this space will begin to add links between source code and change management (SCCM) and the configuration management databases (CMDBs) that are emerging for more ambitious ITIL implementations. Currently: IBM Rational, MKS, Serena, and HP have productized links to specific service-desk systems, and Polarion has released a general-purpose API. IBM Rational and HP can repurpose performance tests scripts as synthetic operational performance tests, and IBM also supports integration of change management through interoperation of Rational Asset Manager and Tivoli CCMDB. Serenas new partnership with Nolio offers possibilities for integration of change management processes as part of release automation; at this point, both vendors need to set their priorities for the direction of their relationship. Remaining ALM vendors have yet to formulate their DevOps product strategies. The primary challenge is marketing, not technology, as ALM and IT service and infrastructure management tools providers have the systems in place that could form the building blocks of combined solutions. Ovum expects a virtuous cycle: vendors with solutions will benefit from the DevOps movement, and the DevOps movement in turn will gain momentum as automation support reinforces good DevOps practices. This field is ripe for innovation; for instance, Nolios unique release automation opens possibilities for linking deployment scripts to classic runbook automation.
The Application Lifecycle Management Decision Matrix

The Ovum Decision Matrix provides a summary of the market standing of the major ALM vendors based on a quantitative and objective assessment of their technology capabilities, market impact, and customer sentiment. The Decision Matrix provides vendor selection guidance for enterprises looking to deploy ALM solutions and advises them on whether a vendor should be shortlisted, considered, or explored further. In this release of the ALM Decision Matrix, the clustered technology assessment and insufficient customer sentiment survey scores for some vendors created difficulties in classification. Ovums ALM Decision Matrix therefore offers two angles of analysis: The Decision Matrix: Figure 14 includes the vendors that have significant responses in all three assessment categories, namely Customer Sentiment, Technology, and Market Impact. The core decision matrix prioritizes Customer Sentiment and Technology assessment scores, placing them on the X and Y axis, with Market Impact represented by the size of the bubble. Technology forms the bedrock of a successful ALM offering, and Ovum finds that high Customer Sentiment scores are indicative of successful go-to-market strategies.
125
100
Atlassian
95
Polarion
User Sentiment (scale - % figures)
90 85 80 75 70 65 60 55 50 50 60
Rally
Serena
Microsoft CollabNet MKS
HP IBM Rational
70 80 90 100
Technology assessment (Scale - % figures)

Shortlist Consider Explore Impact = 1 Impact = 10
Bubble size represents market impact:
Figure 14: ALM Decision Matrix
Source: Ovum
The Extended Decision Matrix: This includes the vendors that have registered considerable Market Impact and Technology assessment scores but do not have significant Customer Sentiment data to be rated objectively. Due to the high Technology and Market Impact assessment scores garnered by these vendors, Ovum explores an alternative means of representing all vendors studied: an extended ALM decision matrix (Figure 15). This scatter diagram represents each profiled vendor, with the X axis reflecting a vendors Technology assessment score and the Y axis representing its Market Impact score. A summary of the assessments for the all vendors is given in Table 2. Shortlist HP IBM Rational Microsoft MKS
Table 2: ALM Extended Decision Matrix ranking of all key vendors
Consider Micro Focus Polarion Serena Software TechExcel
Explore Atlassian CollabNet Rally Software ThoughtWorks Studios
Source: Ovum
126
IBM Rational
Market Impact (Relative % Figures)
Microsoft
HP
Rally Atlassian CollabNet TWS
MFocus MKS Serena TechExcel Polarion
50
55
60
65
70
75
80
85
90
95
100
Technology (% Figures)
Explore
Figure 15: Extended ALM Decision Matrix
Consider
Shortlist
Source: Ovum
As discussed in the market developments section, the ALM market has undergone many changes. The Ovum ALM Decision Matrix also reflects this shift in market direction. Vendors that were part of Ovums 2009 ALM Decision Matrix report but have not been included in this report are Aldon, Compuware, and Borland. Aldon was excluded because its coverage of ALM was not broad enough to meet Ovums definition. Similarly, having sold its software quality business to Micro Focus, Compuwares resultant portfolio was too narrow for it to be called an ALM vendor. Borland was excluded because it had been acquired by Micro Focus.
Atlassian, ThoughtWorks Studios, Rally Software, and CollabNet are smaller vendors with impressive ALM suites
New entrants to 2010s Decision Matrix include Atlassian, CollabNet, ThoughtWorks Studios, and Micro Focus. Atlassian, CollabNet, and ThoughtWorks Studios all have strong propositions for the ALM market backed by strong offerings, and Ovum expects that these vendors will continue to impact the market in the years to come.
Polarions and TechExcels respective market positions have improved

A significant aspect worth noting is that Polarion and TechExcel fall under the Consider category in this edition of the ALM Decision Matrix. In the previous report these vendors were included in the Explore category. The Consider category is the name Ovum has given to the set of vendors a tier below overall market leaders. The market traction these vendors have gained is impressive, given that only two years have passed since Ovum published its previous ALM Decision Matrix report and that we take numerous factors into account before arriving at respective market positions for leading vendors in any enterprise software segment. In addition, Polarion and TechExcel do not have the overall scale enjoyed by the market leaders (HP, IBM, and Microsoft).
127
Micro Focus is the sleeping giant of the ALM market

In the previous ALM Decision Matrix report, Borland was positioned in the Shortlist category, and Compuware in the Consider category. Although it seems counter-intuitive that Micro Focus, which acquired Borland and Compuwares software quality business, should be positioned in the Consider category in the present edition of the Decision Matrix, at time of writing Micro Focus has not released an ALM suite. It is only making available a select number of tools from its acquired portfolio. Ovum characterizes Micro Focuss latent potential as that of a sleeping giant: we will wait to see what emerges, but in our opinion the vendors message post acquisition was not communicated clearly to the market. In contrast, Micro Focuss Application Management and Quality segment has registered double-digit revenue growth, and Ovum expects that the vendors impact on the ALM market will grow further in 2011 and beyond.
HP and MKS emerge as ALM market leaders

HP was placed in the Consider category in the previous ALM Decision Matrix report. This years report ranks HP third in the Shortlist category. This reflects HPs expansion of its tools portfolio from quality management, application performance management, and project performance management to a more well-rounded stack. The missing elements in HPs newly released ALM bundle include CCM, modeldriven development, database modeling and design, build management, and more advanced Agile project management features; significantly, some of these capabilities are also covered by CollabNet, with which HP maintains a strategic alliance. With the launch of HP Software ALM 11.0 in late 2010, the vendor has brought to market a significantly broader offering. HP has communicated clearly its product roadmap, although Ovum has questions about the ultimate role that CollabNets products will or will not play in HPs future. Thanks to its long-time dominance of quality management and test automation, HP also brings a well-developed partner network and a broad base of third-party integrations. Many of these include rival ALM products such as IBM Rational. HPs prime weaknesses include the newness of its ALM offerings, of which several portions are only starting to be market tested, and its low score on Customer Sentiment. (It scored second to last just above IBM Rational some of which can be attributed to the newness of the offering.) Similarly, MKS also advanced from Consider to Shortlist this time around. As the vendor that pioneered ALM as a single unified product, MKS earned the second-highest Technology rating in the Decision Matrix. Its product, MKS Integrity, ties RM, SCCM, project planning, and quality management (although not test automation) into the same platform. Along with IBM Rational, MKS has taken a forward position in the fast-growing, potentially lucrative embedded software development space. That is both MKSs strength as an ALM solution provider, and its weakness when positioned in the overall market. As a modest-sized company, MKS has been smart to focus its guns and not be all things to all people; however, the flip side is that the company has low recognition in the broader ALM market, which is dominated by enterprise IT software development.
IBM and Microsoft maintain their leadership positions

IBM and Microsoft have maintained their respective leading positions in the market. Since the last ALM Decision Matrix report, IBM has released the Jazz platform, launched new Jazz-based ALM products, migrated some of its existing ALM products to Jazz, and used Jazz as a connection bridge for the remaining ALM portfolio. This goes to show that IBM as a market leader recognizes that tool interoperability is a key issue faced by organizations, and has moved quickly to address this, even though it involved re-architecting most of the companys legacy ALM portfolio. Microsofts ALM message has got stronger in the past two years; in addition to ALM, the vendor has a complete software development stack as well as an interactive development environment (IDE), and a large developer community. Microsoft has also made its ALM branding stronger, leveraging the popularity of the Visual Studio IDE; however, requirements definition (RD) and management and software quality management remain areas of concern for the IT behemoth. In this Decision Matrix report, IBM has the top score in the Technology and Market Impact dimensions; yet the company has not done well in the Customer Sentiment dimension. From a technology perspective, IBM has the broadest and arguably the deepest portfolio among all vendors profiled in this report. IBMs Technology score is 94.3%.
128
Furthermore, the vendor has ensured that it stays ahead of the pack by having a good grasp of market demand and meeting it through tool support. IBM has the highest Market Impact score (10), and all other vendors have been ranked relative to IBM. The Market Impact dimension takes into account parameters such as: revenues from the ALM market and its spread in terms of organization size, vertical presence, and geography; year-on-year ALM revenue growth in percentage terms; and market recognition. IBM has the top score in all these parameters except revenue growth, which is expected due to IBMs large base. Customer sentiment is one area in which IBMs performance is significantly lower than other vendors. It also has the lowest Customer Sentiment score among ALM market leaders, whereas in the previous ALM Decision Matrix report IBM was one of the top scorers in this category. Ovum believes that the survey results provide many significant takeaways for IBM. Notably, IBM has scored low on product usage, service levels and response times, client engagement, and customer support. Customers arent always clear on IBMs ALM roadmap for several reasons. It has come about through multiple acquisitions, and IBMs Jazz initiative has led to re-architecting and rebranding of several Rational products. Microsoft is second only to IBM in the ALM market. Microsoft has the third-highest Technology score (83.4%), bettered only by IBM and MKS. Microsoft has a broad ALM portfolio, though certain gaps exist. On the whole, Microsofts performance on the Technology scale is commendable, reflecting the vendors transition from primarily IDE vendor, to ALM vendor with heavy dependence on partners (VSTS 2008), to ALM vendor with a well-rounded portfolio (VSTS 2010), to ALM market leader (Visual Studio 2010) with a strong brand and consistent performance. Microsofts Market Impact score is second only to IBM; this is largely due to Microsofts high recognition and ALM revenues. On the Customer Sentiment scale Microsoft has performed better than IBM, the overall market leader; however, it is not the top scorer in this category even though it has the highest Customer Sentiment score among the market leaders. There is room for improvement, primarily in customer support.
Market Leaders
Breadth of lifecycle discipline coverage
10
MF
9
Architecture modeling support

T
MF I M
8
P I T
Solution Architecture
M MK P T
I P TS C TS MK S R S MK A H C P R T I H C
Collaboration
Internal tool interoperability
External tool interoperability
A M T
Altassian Microsoft TechExcel
C MK TS
CollabNet MKS
H P
HP Polarian
I R
IBM Rational Rally
MF S
Micro Focus Serena
Figure 16: Market leaders analysis: technology (suite architecture)
Source: Ovum
129
Overview
The application lifecycle management market is broad and includes many subsets. Vendors vary in focus and market strength. In this section, Ovum takes a deeper look into the underlying patterns in the market by comparing the top vendors in each of the criteria that contribute to the Technology, Market Impact, and Customer Sentiment ratings.
Technology
The state of technology leadership reflects ALM market maturity The Market leaders analysis: Technology (suite architecture) schematic (see Figure 16) reflects market maturity in terms of technological capabilities. A cursory glance at the schematic shows that most vendors are sensitive to key issues such as internal tool interoperability, integration with third-party ALM tools, and the collaborative nature of software development activities. The market has also converged in terms of overall solution architectures nearly all vendors now offer common repository-based solutions with support for repository federation, as well as other essential pieces such as workflow, analytics, web-based client access, and support for Agile development methodologies. Having figured out the general market direction, vendors are keen on plugging the gaps Although no other vendor offers an ALM portfolio as comprehensive as IBMs, the intent of filling in the missing pieces through either product development or third-party integrations is apparent. The ability to fit seamlessly into customers IT environments and interoperability with incumbent tools is no longer a key differentiator, as is clear from vendors scores in this category. It is interesting to note that IBM has the lowest score in the external tool interoperability dimension; given that IBM offers all pieces of the ALM puzzle in its large installed base, it is apparent that the companys strategy is to facilitate and incentivize customers further reliance on IBM technology. In addition, IBM is working towards closing this gap through its Jazz platform and the OSLC initiative; as OSLC has failed to attract other mainstream ALM players so far, its industry impact cannot be qualified at this point.
10.0
M H
S M TS I I P R S
Delivery management
MF
MK
9.5
Project management
I TS I MK
9.0
8.5
H M
8.0
TS T
7.5
MF I
T S
S MF MK H P M I MK P R MF T M S I I R MF
Enterprise agile
Quality management
Software assurance integrations
H P
HP Polarian
I R
IBM Rational Rally
MF S
Micro Focus Serena
M T
Microsoft TechExcel
MK TS
MKS Thoughtworks Studios
Figure 17: Market leaders analysis: technology (suite components)
Source: Ovum
130
Project management is the gateway to ALM Six vendors out of 12 have perfect scores in the project management dimension, and nine have scores over 90%. This goes to show that lifecycle visibility, traceability of artifacts, and the ability to act or take corrective measures based on near-realtime information is a top priority for most ALM buyers. More importantly, executing project management correctly is the key to better fortunes in the ALM market, as demonstrated by emerging vendors such as Rally Software, ThoughtWorks Studios, Polarion, TechExcel, and CollabNet. A commonly held view is that application project portfolio management (APPM) follows naturally from project management. This is true, but only with respect to managing multiple projects or project hierarchies. Most vendors provide a hierarchical or tree structure with rolled-up reporting capabilities. However, other aspects of APPM, such as critical-path analysis, financials management, what-if scenario modeling, risk management, and demand management, are not natural extensions of project management capabilities. Therefore, if a vendor performs project management well, it likely will also offer multiple project management functionalities; however, adding other capabilities will require substantial development effort. Hence, the APPM dimension is not as clustered as the project management dimension; only two vendors have perfect scores, and only four vendors have scores of more than 90%. Requirements definition, requirements management, and change and configuration management remain specialist businesses No vendor has a perfect score in the RD category and, on the whole, vendors scores are lower than corresponding project management or APPM scores. This goes to show that RD is a specialist area insofar as vendors with established RD products have been able to protect their businesses, and emerging ALM vendors have not devoted significant product development resources in this direction. Interestingly, Microsoft has the lowest RD score of all vendors. Only IBM has a perfect score in the RM dimension; other leading vendors include Micro Focus, Serena, Rally, and TechExcel. For most vendors, RM amounts to requirements tracking through lifecycle, dependency mapping, requirements baselining, and reporting. Requirements-based test case generation and visual requirements modeling do not seem to be focus areas for most vendors. CCM is a specialist area of ALM, and the same can be inferred from the clustering of vendors in this dimension. However, it must also be noted that popular open source CCM tools such as Subversion have created a strong undercurrent in this ALM discipline, and led to the commoditization of this segment. Some vendors that scored high marks in the CCM dimension, such as Atlassian and Polarion, embed Subversion into their ALM solutions. IBM, Microsoft, Polarion, and Serena share the top spot with perfect scores; MKS and Atlassian are tied for the second spot with scores marginally lower than those of category leaders; CollabNet and Micro Focus share the third spot with scores just under 80%. The gulf between these eight vendors and the remaining four is extremely wide, with the next-best scores (those of HP, Rally, and ThoughtWorks Studios) just over 20%. It is interesting to note that CollabNet, the lead vendor on the Apache Subversion project, scores lower than Polarion and Atlassian (vendors that embed the open source CCM tool into their ALM suites). In the main, this is due to CollabNets lack of visual workflow with automation and access management, visual exploration of branch inheritance, and automated branch inheritance features that Polarions version of Subversion provides. Although Atlassian does not offer visual workflow with automation and access management, it offers automated branch inheritance, and visual exploration capabilities for branch inheritance. Software assurance and application security remain the stronghold of established ALM vendors The longer a product has been on the market, the more third-party integrations it is likely to have; this is largely true for the software assurance space, which encompasses requirements management, defect management, quality management, application performance management, and IT service desk, with established ALM vendors scoring higher on software assurance integrations than their recent competitors.
The longer a product has been on the market, the more third-party integrations it is likely to have...
131
Application performance management is an interesting case in point: only IBM, HP, Micro Focus, and Microsoft have APM integrations. Micro Focus, HP, Microsoft, Polarion, and MKS are the leading vendors in the software assurance integration dimension. IBM and Polarion are exceptions, and the rationale behind IBMs limited number of third-party integrations has already been discussed. (The vendor is working towards an industry-wide ALM integration framework and is not keen on developing integrations with point products.) Polarion and TechExcel although TechExcel does not feature in the software assurance dimension due to weak requirements management and QA integration are the only emerging players in the ALM space, with significant investments in quality management and software assurance integrations. Quality management scores are largely along expected lines, with HP, IBM, TechExcel, and Micro Focus scoring well. TechExcel and Micro Focus are tied for the second spot ahead of HP, and that may come as a surprise to many; its primarily due to TechExcels test lab management capability for cloud based test environments that HP lacks. HP offers test lab management only for on premise environments. Micro Focus misses out on a perfect score due to the lack of test lab management capability for on premise environments. Application security has largely been overlooked by ALM vendors. Hence, in the application security dimension, established ALM vendors with existing investments in application security testing scored well, and a wide gulf exists between these vendors and the rest of the pack. IBM, Microsoft, HP, and MKS each have a perfect score. Vendors were evaluated on parameters such as static and dynamic testing tool integration, automated defect creation for security issues, security issue impact analysis, security risk prioritization, and security assessment reporting. Established ALM vendors are now aboard the Agile train Many emerging ALM vendors have been championing Agile practices for many years, for which they have reaped commensurate benefits as the market has now largely adopted the Agile mantra. There have been ups and downs and learning opportunities for both the Agile vendor community and its early adopters. Not all organizations have had smooth sailing with Agile adoption, and tool support for Agile methodologies has evolved over the years as vendors learned what worked and what did not. However, this market is now hot and, in Ovums view, is moving towards feature set convergence. Ovum finds all leading ALM vendors back up their Agile marketing communication with actual tool support, with the exception of Micro Focus. Established ALM vendors such HP, IBM, Microsoft, MKS, and Serena scored well in this dimension and, overall, nine vendors scored more than 80%. Micro Focuss Agile support is limited. Micro Focus StarTeams capabilities can be extended using the SDK for multi-project Agile planning. StarTeam is process agnostic, thus users can configure process templates to Agile or Agile hybrid variants. SilkCentral Test Manager and SilkTest support API-based integration with Agile project management and planning tools. Delivery management is increasingly gaining prominence to address growing software reliability concerns Delivery management encompasses build, release, and deployment management disciplines. As the Technology (suite components) schematic illustrates, most ALM vendors realize that downstream processes of the software lifecycle are as important as their upstream A concerted counterparts, if not more so. Build, release, and deployment management approach towards precede production deployment of applications. A concerted approach towards delivery management is the result of the numerous production delivery application failures, and it aims to enhance the reliability of delivered management is the software by closing the gaps between build, deployment, and release result of the processes, thereby making software releases more predictable.
numerous production application failures...
Micro Focus and MKS are the leaders in this category; however, they are just marginally ahead of IBM and ThoughtWorks Studios. Other vendors that scored well in this dimension include Microsoft and HP, followed by Atlassian, CollabNet, and Polarion.
132
Customer Sentiment
Product quality Product usage
R C P
A
S M H
10
Product integration
8
H P R M I S C
C
Client engagement
R P
MK S C A M
R S A P H C
R A P S
Customer support
R I MK A
Financial stability
M I
MKM P S R C
A P H
R S M C
MK I P A M S R
P S
Service capabilities
Service levels Portfolio depth
Vertical specialization
A MK
Altassian MKS
C P
CollabNet Polarian
H R
HP Rally
I S
IBM Rational Serena
Microsoft
Figure 18: Market leaders analysis: Customer Sentiment
Source: Ovum
Emerging ALM vendors are overall leaders in Customer Sentiment Emerging ALM players Atlassian, Polarion, and Rally are the overall leaders in Customer Sentiment. Market leaders IBM, Microsoft, MKS, and HP performed relatively poorly on the Customer Sentiment scale in this edition of the ALM Decision Matrix. IBM and Microsoft were rated by their customers as leaders in the financial stability dimension. IBM also makes an appearance in the vertical specialization, service capabilities, and product integration dimensions; however, its score is a notch lower than category leaders in all three cases. In addition to financial stability, Microsoft was rated a joint leader in the product usage and portfolio depth dimensions. Microsofts scores in other Customer Sentiment dimensions, such as product integration, vertical specialization, client engagement, and service levels, are a notch lower than respective Customer Sentiment category leaders. Microsoft also has the highest overall Customer Sentiment score among the Shortlist vendors. Surprisingly, IBM is not among the leaders in the portfolio depth dimension, even though it has arguably the most feature-complete toolset in the market and is the undisputed leader in Technology ratings. This anomaly is likely due to the surveyed customers dissatisfaction with IBM, and is reflected in IBMs low scores in dimensions that have direct correlation with customer satisfaction, such as customer support, service levels, and client engagement. MKS finds a mention in the product usage dimension as a joint category leader; the vendor is also present in the client engagement, service levels, service capabilities, and vertical specialization dimensions. HP is present in the product integration, vertical specialization, portfolio depth, and financial stability dimensions. The vendor is the category leader in the vertical specialization dimension and a joint leader in the portfolio depth dimension.
133
The key factor behind market leaders poor performance on the Customer Sentiment scale appears to be low customer satisfaction, an aspect in which the smaller vendors have excelled. The dimensions that indicate customer satisfaction with the vendors quality of service (such as customer support, service capabilities, service levels, and client engagement) collectively contribute 35% to the overall Customer Sentiment score. Each of these dimensions has an emerging ALM vendor as the category leader, with all four Shortlist vendors out of reckoning. Ovum believes the following factors have led to this result: As they are smaller in both company size and customer base, the emerging ALM vendors are able to have more intimate relationships with a larger cross-section of customers. Customers probably do not have as grand of expectations of smaller companies as they would of the larger, more diversified vendors.
Market Impact
Recognition
10
9 8 M 7 6 H 5 4 3
Regional presence
I M
Revenues
2 1 0
H I M M R H MF A I
A R P
Size band presence
Revenue growth
Vertical presence
A M
Altassian Microsoft
H P
HP Polarian
I R
IBM Rational Rally
MF
Micro Focus
Figure 19: Market leaders analysis: Market Impact
Source: Ovum
Shortlist vendors are the overall leaders in Market Impact HP, IBM, and Microsoft are the top-three vendors in all dimensions of Market Impact, except revenue growth and vertical presence. These three vendors, together with Rally Software and Atlassian, take the second spot in vertical presence. The vertical presence dimension indicates the degree of presence a vendor has across verticals with respect to its own total revenues, not against absolute revenue earnings. This metric therefore measures how well a vendors income is spread across verticals a high score indicates that the vendor has customers across many verticals. Interestingly, MKS, a Shortlist vendor, finds no mention in any Market Impact dimension. This is due to the fact that MKS is not among the top-three vendors in any Market Impact dimension, and makes the Shortlist category purely on the strength of its ALM offering. Polarion, Rally, and Atlassian are the leaders in the revenue growth dimension; as with vertical presence, this dimension does not take into account the absolute value of earnings.
134
The Market leaders: Market Impact schematic (Figure 19) shows the gulf between market leaders and the rest of the pack in sheer revenue terms. This divide is also apparent in the Extended Decision Matrix chart. It is also worth noting that relative positions of the market leaders in each of the dimensions (except revenue growth) are unchanged, with IBM as the overall leader, ...it is unlikely that followed by Microsoft and HP. The smaller vendors are growing faster than the ALM giants
As the list of leaders in terms of revenue growth indicates, the smaller vendors are growing faster than the largest ALM vendors. That said, the difference in scale is of two orders of magnitude, and it is unlikely that the market structure will change in any meaningful way any time soon. However, the good showing by smaller ALM vendors on the Technology and Customer Sentiment scales, incorporating factors such as technological capabilities, future product roadmap, customer perception of product quality, and customer perception of their quality of service, suggests that they will remain strong contenders in the ALM market.
the market structure will change in any meaningful way any time soon.
7.3 Vendor analysis

Atlassian
User sentiment radar
Product quality
10
Impact radar
Recognition
10
Product usage
8 6
Product integration Regional presence
8 6 4 2 0
Client engagement
4 2 0
Revenue
Customer support
Financial stability Service levels
Service capabilities Vertical specialization
Size-band presence Vertical presence
Revenue growth
Portfolio depth
Technology radar
Breadth of lifecycle discipline
10 8
6 4 2 0
Solution architecture
Collaboration
Atlassian
Figure 20: Atlassian ALM radars
Maximum category score
Average across vendors

Source: Ovum
Atlassian is an emerging ALM vendor, with an impressive ALM portfolio. Atlassians Technology and Market Impact scores reflect the vendors relative position in the ALM market. On the Technology front, Atlassians scores are close to the average on dimensions such as breadth of ALM coverage (joint fourth), solution architecture (joint fourth), external tool interoperability (joint leader), and collaboration (shares the second spot).
135
In terms of lifecycle disciplines Atlassians offering merits a mention only in change and configuration management (joint second), due in great measure to the Subversion repository that Atlassian embeds in its hosted ALM solution, JIRA Studio. Atlassians overall score on the Technology scale is the lowest among all vendors profiled in this report. Atlassians Market Impact is two orders of magnitude lower than category leader IBM. Although these Market Impact comparisons suggest that Atlassian has not made a dent in the ALM market, which is dominated by mega-vendors such as IBM, HP, and Microsoft, Atlassians Customer Sentiment scores suggest otherwise. Ovums end-user survey results have placed Atlassian as the leader on the Customer Sentiment scale. This also reflects the popularity of Atlassians offerings and the market perception surrounding them. Ovum believes Atlassian will benefit from the flux in the ALM space, especially from trends such as SaaS ALM and enterprise adoption of distributed version-control systems.
10
Delivery management
8 6
Project management
4 2 0
Enterprise Agile
Quality management
Atlassian
Figure 21: Atlassian ALM components radar
Source: Ovum
Atlassians ALM portfolio comprises Atlassian JIRA, GreenHopper, Confluence, FishEye, Crucible, Bamboo, Clover, Bitbucket, and JIRA Studio (the hosted Agile development suite made up of JIRA, GreenHopper, FishEye, Crucible, Confluence, and Bamboo with Subversion as the embedded suite repository). Salient aspects of Atlassians ALM offering include single shared repository-based architecture, wiki-based collaboration and knowledge management, and strong defect and issue management capabilities. The offering lacks native analytics functionality, and requirements definition and management are other improvement areas. However, Atlassians offering has a strong developer focus insofar as most of the tools listed above are aimed at practitioners. Furthermore, Atlassian has shifted direction from being a developer tools vendor to offering a third-party hosted/SaaS ALM solution; although many functionality gaps remain in Atlassians solution, overall the vendors strategy on greater Agile adoption, enterprise adoption of DVCS, and SaaS ALM is likely to pay off. Atlassian has embarked on the next phase of growth. The privately held Australian ALM vendor recently attracted 42,000,000 investment from Accel Partners. The company plans to use these funds to expand its presence in Europe. The EMEA region currently accounts for 40% of Atlassians revenue. Atlassian currently has over 8,000 customers in Europe. With a global customer base of 22,000 across 138 countries, Atlassian has a strong presence in industry verticals such as manufacturing, financial services, professional services, public sector and education, and healthcare. Nearly 70% of Atlassian customers are small to medium-sized enterprises (SMEs); however, the vendor aims to penetrate the large-enterprise segment further in the near term. In the past two years, Atlassian has acquired GreenHopper (June 2009), and Bitbucket (September 2010). The GreenHopper acquisition was Atlassians foray into the Agile project management space; GreenHopper was a JIRA add-on offered by GreenPepper Software. Atlassian acquired Bitbucket in September 2010 to widen its support for the Mercurial distributed version control system (DVCS).
136
Recommendation: Explore
Atlassian earns an Explore rating primarily due to its moderate Technology score and high Customer Sentiment score. Although Atlassians products are not the best in class in either breadth of coverage or feature depth, the vendor has been able to carve out a niche for itself in the ALM market, which is evident from its large globally distributed installed base and impressive vertical presence. Notably, the vendor has created a large third-party developer ecosystem, and the list of its third-party plug-ins is impressive. Atlassians ALM portfolio has evolved in line with what Ovum believes is the way forward for Agile and globally distributed software development teams in SMEs. Atlassians tool set will also find favor with teams of 10 members or less.
CollabNet
Product quality
10
Impact radar
Recognition
10
Product usage
8 6
8 6 4 2 0
Client engagement
4 2 0
Revenue
Customer support
Revenue growth
Portfolio depth
Technology radar
10 8
6 4 2 0
Collaboration
CollabNet
Figure 22: CollabNet ALM radars

Source: Ovum
CollabNet is an emerging ALM vendor best known as the creator of Subversion, the popular open source repository. CollabNets Technology scores are above average In dimensions such as collaboration and internal and external tool interoperability; however, the vendors performance has been below par on parameters such as solution architecture, breadth of lifecycle discipline coverage, and architecture modeling support. CollabNets Market Impact score is predictably low in a market dominated by heavyweights such as IBM, Microsoft, and HP. However, the vendor has recorded doubledigit revenue growth and still has ample room for expansion. CollabNets performance on the Customer Sentiment radar is close to the group average on most counts, and slightly below par on product quality, customer support, and portfolio depth. CollabNets ALM strategy has always revolved around the cloud to support globally distributed development efforts. CollabNet Subversion was the first major step in this direction, a centralized artifacts and source code repository with a web interface. CollabNets ALM portfolio now includes TeamForge, the hosted planning and lifecycle management solution; ScrumWorks, the Agile project planning tool; Subversion Edge, for managing globally distributed Subversion instances; and Lab Management, for managing development and test environments in the data center or in public clouds.
137

10
Delivery management
8 6
Project management
4 2 0
Enterprise Agile
Quality management
CollabNet
Figure 23: CollabNet ALM components radar
Source: Ovum
CollabNets offering would benefit from enhanced reporting and analytics capabilities. The vendor has also sought to fill key gaps in functionality through partnerships, a recent example being its partnership with HP Software to round out QA and requirements definition and management functionality. CollabNet has made some key acquisitions at various stages of its history. The flagship offering, CollabNet TeamForge, came about after SourceForge Enterprise (an on-premise solution acquired from VA Software in 2007) and CollabNet Enterprise (a hosted planning and lifecycle management tool) were merged. The vendor retired CollabNet Enterprise after migrating existing customers to TeamForge. The acquisition of Danube in early 2010 was another significant milestone; it brought the ScrumWorks Agile project planning and management tool to CollabNet, addressing a critical gap in the vendors Agile strategy. Furthermore, in late 2010 CollabNet acquired Codesion, the largest third-party hosting service for Subversion. CollabNet has an installed base of over 6,000 organizations, of which 700 are TeamForge clients. CollabNet has a significant presence in the large-enterprise segment, and many of those deployments are globally distributed and enterprise wide, with 10,000 or more users in each case. This is noteworthy as CollabNet is an emerging ALM vendor and lacks the scale that market leaders such as IBM, Microsoft, and HP enjoy. CollabNets primary market in terms of geography is consists of the two Americas; it has few clients in EMEA and Asia-Pacific. Most CollabNet clients come from manufacturing and financial services verticals. The vendor also has significant presence among ISVs.
CollabNet gets an Explore rating due to its moderate Technology score. CollabNet is a strong contender in areas such as hosted and SaaS ALM, project planning and lifecycle management, and delivery management. However, its less-than-average score across important pieces of the ALM portfolio, such as requirements definition, requirements management, software assurance integrations, and quality management, led Ovum to assign this rating. However, CollabNet deserves a closer look from enterprises that struggle to manage globally distributed development efforts, as well as from ISVs.
138
HP
Product quality
10
Impact radar
Recognition
10
Product usage
8 6
8 6 4 2 0
Client engagement
4 2 0
Revenue
Customer support
Revenue growth
Portfolio depth
Technology radar
10 8
6 4 2 0
Collaboration
HP
Figure 24: HP ALM radars

Source: Ovum
HP is one of the leaders of the ALM market. HPs Technology and Market Impact radars reflect the vendors relative position in this market. On the Technology front, HP has secured the fourth position overall, behind IBM, MKS, and Microsoft. On the Customer Sentiment scale, HPs performance is close to the average in some dimensions, and below average in dimensions such as product usage, client engagement, service levels, and customer support. On the Technology scale HP has below-par scores in dimensions such as collaboration, change and configuration management, and project management. Ovum expects that HPs recent partnership with CollabNet will help the vendor enhance capabilities in the CCM space. The Market Impact radar shows vendors positions relative to market leader IBM, and HPs performance is above average in all dimensions except revenue growth. The vendors overall position on this scale is third, behind IBM and Microsoft. HPs suite of application lifecycle products falls under its business technology optimization portfolio. The ALM suite contains many point tools, which are strung together by HPs latest ALM platform, HP ALM v11.0, to create an end-to-end offering that not only caters to IT practitioner roles such as developers and testers, but also provides project and portfolio management, planning capabilities, and lifecycle visibility and traceability to managers.
139

10
Delivery management
8 6
Project management
4 2 0
Enterprise Agile
Quality management
HP
Figure 25: HP ALM components radar
Source: Ovum
In addition, it provides integration with enterprise systems such as project portfolio management, business process management, and SOA governance on one end, and with DevOps tools on the other end. Key products in HPs ALM portfolio (in addition to HP ALM 11) include HP Quality Center 11, HP Performance Center 11, HP Sprinter, HP Unified Functional Testing 11, HP QC Agile Accelerator, and HP Test Data Management. In addition to quality management, HP QC 11 serves as the solution repository and offers the requirements management, defect management, and software delivery management aspects of HP ALM 11. HP Sprinter, a new manual testing tool, is also part of this release. HP Unified Functional Testing 11 is a new product that combines HP QTP with HP Service Test (integration testing) through a single interface and offers unified reporting capabilities. HP Performance Center 11 provides performance testing capabilities for web-based and legacy applications and, as part of the ALM solution, enables end-to-end traceability of artifacts. HP QC Agile Accelerator is an Agile project management tool built from the ground up. This recent addition to HPs ALM portfolio has earned the vendor a nearly perfect score in the enterprise Agile Technology dimension. Overall, HPs ALM strategy seems to center around consolidation of existing leading positions in key areas including quality management, application security, and moving quickly to capture market share in ALM hot spots such as Agile development. The vendor has also focused on filling functionality gaps through in-house development and partnerships. HP is a mega-IT vendor and, as such, the company has made quite a few big-ticket acquisitions in the past few years. The acquisition of Fortify in September 2010 is relevant from an ALM perspective. The Fortify acquisition provided HP with tools to address application security concerns in the development lifecycle. HP has a keen sense of the rapidly changing software development landscape, as well as applications. Many new capabilities added to existing tools such as QTP and HP Service Test, and new products such as HP LoadRunner TruClient, HP Sprinter, and HP QC Agile Accelerator, led us to believe that HP aims to stay abreast with the complex, ever-changing nature of modern applications.
Recommendation: Shortlist
HP has advanced on Ovums ranking from the Consider category in 2009s Decision Matrix to the Shortlist category this year. The vendors good scores in the Technology dimension (marginally lower than Microsofts), near-average Customer Sentiment score, and third-highest Market Impact score have led to its Shortlist rating. HPs ALM offering merits attention from organizations across size bands, geographies, and industry verticals. The offerings support for a wide range of processes and project types, coupled with HPs comprehensive QA functionality, make it a good fit for enterprises.
140
IBM Rational
Product quality
10
Impact radar
Recognition
10
Product usage
8 6
8 6 4 2 0
Client engagement
4 2 0
Revenue
Customer support
Revenue growth
Portfolio depth
Technology radar
10 8
6 4 2 0
Collaboration
IBM Rational
Figure 26: IBM Rational ALM radars

Source: Ovum
IBM Rational is among the largest vendors in the ALM space, and its Market Impact scores reflect its status as an application lifecycle management heavyweight. It registers the highest Technology score, ahead of MKS, Microsoft, HP, and Polarion. IBM scores the highest or close to the highest among the 12 ALM vendors profiled in this report across most Technology dimensions, including breadth of lifecycle discipline coverage, solution architecture, collaboration, application project portfolio management, project management, requirements management, change and configuration management, quality management, and delivery management among others. In terms of its Market Impact, IBM is the undisputed leader, much ahead of closest competitors Microsoft, and HP. In this research exercise the Customer Sentiment scores of the large ALM vendors have mostly been unimpressive, and same is the case with IBM. Its score is above average on just the financial stability dimension of Customer Sentiment. Predictably, IBM has the largest ALM portfolio of all vendors included in this report, insofar as it has multiple products for lifecycle disciplines such as requirements management and change and configuration management. The following is a partial list of IBM Rational products: Rational Team Concert project management and lifecycle visibility Rational DOORS requirements management Rational Quality Manager test management Rational Test Lab Manager test environment provisioning and management Rational ClearCase CCM Rational ClearQuest defects and issue management Rational Build Forge build and release management Rational AppScan application security
141

10
Delivery management
8 6
Project management
4 2 0
Enterprise Agile
Quality management
IBM Rational

Source: Ovum
Figure 27: IBM Rational ALM components radar
A key aspect of IBMs ALM strategy is Jazz. Jazz is an architecture framework that aims to address tool interoperability, one of the biggest concerns regarding ALM. A few years ago IBM announced the Jazz Initiative, based on the understanding that organizations IT environments typically have tools from multiple vendors, and that making these tools work well together, orchestrated by a common framework, is an unsolved challenge given that often even tools from the same vendor have poor interoperability. Jazz supports a wide range of operating systems (including mainframes), and has a web services (REST)-based federated architecture. Jazzs associated industry body is the Open Services for Lifecycle Collaboration (OSLC), an IBM-led community that is working towards developing a common architecture framework to enable better integration among lifecycle tools. However, this initiative is still at an early stage, and industry adoption of OSLC specifications will depend on many factors other than technical. So far, OSLC has not attracted any other mainstream ALM vendors, and the success of this initiative cannot be ascertained at this point. IBM is in the process of migrating legacy Rational products to the Jazz platform. (Rational here refers to all products that are part of IBM Rational, not to be confused with products that came to IBM through the Rational Software acquisition in 2002.) The company has also developed new products native to Jazz. These new products include Rational Team Concert, Rational Requirements Composer, and Rational Quality Manager. IBM is a large IT vendor, and acquisitions play a major role in shaping its product portfolio and growth in the markets in which it operates. Significant acquisitions from an ALM standpoint include Rational Software (2003), BuildForge (2006), WatchFire (2007), Telelogic AB (2008), and Ounce Labs (2009).
High scores on the Technology and Market Impact scales have put IBM Rational in the Shortlist category in this Decision Matrix. IBMs ALM portfolio offers all-around functionality in virtually all ALM disciplines. The solution set supports the entire range of use cases, project types, and processes found in enterprise organizations with mature software development functions. Although IBM also sells to small enterprises and small development teams, the feature set will likely be expensive and overkill for their needs.
142
Micro Focus
Technology radars
10

10 8
Delivery management Application security
8 6 4 2 0
Project management Requirements definition Architecture modeling support
6 4 2 0
Enterprise agile Quality management
Requirements management Configuration management
Collaboration
Internal tool interoperability External tool interoperability
Impact radar
Recognition
10 8
Regional presence
6 4 2 0
Revenue
Size-band presence
Revenue growth
Vertigal presence
Micro Focus
Figure 28: Micro Focus ALM radars

Source: Ovum
Micro Focus, an enterprise applications management and legacy modernization vendor, entered the ALM space by acquiring Borland and the software quality management product line from Compuware in 2009. Micro Focus scored well in the Technology dimensions, such as breadth of lifecycle discipline coverage, architecture modeling support, delivery management, requirements definition, requirements management, and quality management, among others. Micro Focuss overall position on the Technology scale is eighth, mainly due to below-average performance in several ALM disciplines. For instance, Micro Focus received below-par scores in key areas such as application project portfolio management, project management, enterprise Agile, and application security. Micro Focuss Market Impact scores reflect its position in the market and the mindshare that the vendor has managed to capture. Micro Focus is the category leader in the Market Impact vertical presence dimension, due the fact that it has the most uniform vertical spread of revenues among all profiled vendors. Micro Focus placed fourth jointly with Rally Software on the Market Impact scale, behind IBM, Microsoft, and HP. Ovum anticipates Micro Focus launching its acquired Borland Agile ALM products, but its prolonged withdrawal from this part of the market is a weakness. For this Decision Matrix, Micro Focus was not rated by enough customers for Ovum to aggregate and present statistically significant Customer Sentiment scores.
143
Micro Focuss ALM portfolio includes Silk Suite (software quality management), StarTeam (change and configuration management), Caliber suite (requirements definition and requirements management), and Together (model-driven development). The vendor has a global presence, with the Americas being the primary geography. Micro Focuss installed base is comprised mostly of large and medium-sized enterprises, with a relatively small percentage being small businesses. Significant customer verticals include media and communications, healthcare, professional services, financial services, and public sector, among others.
Recommendation: Consider
Moderate Technology and Market Impact scores earn Micro Focus a Consider rating. Micro Focus is a leader in categories such as requirements management, software quality management, and delivery management; however, less-than-average performance in other key ALM disciplines led us to assign this rating. Micro Focuss offering must be considered by enterprises, in particular by those that wish to approach ALM from a software quality and reliability perspective.
Microsoft
Product quality
10
Impact radar
Recognition
10
Product usage
8 6
8 6 4 2 0
Client engagement
4 2 0
Revenue
Customer support
Revenue growth
Portfolio depth
Technology radar
10 8
6 4 2 0
Collaboration
Microsoft
Figure 29: Microsoft ALM radars

Source: Ovum
Research for the ALM Decision Matrix report places Microsoft in the leaders bracket. Microsoft is second only to IBM in terms of ALM market leadership. On the Technology scale, Microsoft placed third, behind IBM and MKS. The vendor scored well in all Technology dimensions except requirements definition and requirements management.
144
Microsoft is the second-place vendor on the Market Impact scale; Microsofts Market Impact score is predictably behind IBM, mainly due to the difference in revenues these two vendors earn from the ALM market. Microsofts performance on the Customer Sentiment radar is close to the average in most dimensions, and better than average in product integrations and financial stability. Notably, Microsofts product quality rating is below par. Overall, Microsofts performance on the Customer Sentiment scale is better than that of the other three Shortlist vendors (IBM, MKS, and HP). This goes to show that Microsoft has managed customer expectations better than the other market leaders. Microsofts ALM portfolio revolves around the Visual Studio IDE; with the current version, Microsoft has done away with the various role-based editions, and now offers Visual Studio 2010 in Ultimate, Premium, and Professional editions. There is a separate VS2010 Test Professional edition for QA teams; it offers a non-programmer environment for QA professionals. Over the past few years, Microsoft has not only managed to create space for itself in the ALM market, but also emerged as a market leader. Ovums 2009 edition of the ALM Decision Matrix also recognized Microsoft as an ALM market leader. Microsofts quick rise to the top in the ALM domain is not surprising. Microsoft Visual Studio IDE is one of the leading development environments in the market, and has earned Microsoft a large and globally distributed developer community. Given Microsofts leading position in the operating systems business (both client and server) and its ownership of the .Net technology stack, it was apparent that Microsoft would be an ALM market leader as soon as it focused on this market. Microsofts ALM offering, previously known as Visual Studio Team System 2010, is now simply called Visual Studio 2010. Additionally, all editions of the VS2010 offering now include the MSDN subscription. Microsofts ALM strategy has clearly moved away from selling point tools to selling an integrated ALM platform.
10
Delivery management
8 6
Project management
4 2 0
Enterprise Agile
Quality management
Microsoft

Source: Ovum
Figure 30: Microsoft ALM components radar
Good performance on all three scales (Technology, Market Impact, and Customer Sentiment) led to Microsofts Shortlist rating. The vendors ALM offering is comprehensive, supporting all roles, project types, and processes found in an enterprise-class development function. The solution merits closest attention by organizations that have standardized on the .Net stack; however, the Visual Studio IDE also supports non-.Net development. Organizations developing on and for the Microsoft cloud platform should also consider VS2010 for its deep integration with Azure.
145
MKS
Product quality
10
Impact radar
Recognition
10
Product usage
8 6
8 6 4 2 0
Client engagement
4 2 0
Revenue
Customer support
Revenue growth
Portfolio depth
Technology radar
10 8
6 4 2 0
Collaboration
MKS
Figure 31: MKS ALM radars

Source: Ovum
MKS secured the second-highest score on the Technology scale in this edition of the ALM Decision Matrix, and that led to the vendor being placed in the Shortlist category. However, the vendors performance on the Customer Sentiment radar is less than satisfactory. Significantly, like other market leaders, MKS received less-than-average scores in a few Customer Sentiment dimensions, such as portfolio depth, financial stability, product integration, and customer support. However, MKS managed a par score in the client engagement dimension. MKSs overall Customer Sentiment position is relatively poor compared with the leaders in this dimension, but the vendor did better than two other Shortlist vendors, IBM and HP. Its score is marginally behind Microsoft, the Customer Sentiment leader among Shortlist vendors. Thus, Ovum is inclined to believe that MKS customers perception of the vendor is largely favorable.
146

10
Delivery management
8 6
Project management
4 2 0
Enterprise Agile
Quality management
MKS
Figure 32: MKS ALM components radar

Source: Ovum
MKSs performance on the Market Impact scale is also less than satisfactory. This is attributable to MKSs highly specialized target market, organic product development, and growth. MKS targets organizations with complex IT environments and, consequently, highly specialized business requirements that MKS addresses through its ALM offering. MKS customers are typically large enterprises in the Americas and EMEA, from industry verticals such as automotive, finance, electronics and high technology, medical devices, aerospace and defense, healthcare, and government. This led to MKS losing points in Market Impact dimensions size-band presence and regional presence, which typically favor vendors with equitable distribution in terms of customer size and geographical presence. MKS also lost points in the revenue and revenue growth dimensions compared with other vendors included in this report. MKSs ALM offering is MKS Integrity, a monolithic platform that covers all ALM disciplines. MKSs solution scored less-than-average marks only in the architecture modeling support dimension. In all other Technology categories the vendors score is at least average, if not better.
MKSs close-to-highest score on the Technology scale and moderate score on the Customer Sentiment scale placed the vendor in the Shortlist category. The Customer Sentiment and Market Impact scores are lower than would be expected of an ALM vendor of MKSs stature, largely attributable to MKSs poor level of customer awareness. However, other leaders such as HP and IBM Rational scored even lower. There is little to doubt the comprehensive nature of MKSs offering and its relevance to diverse ALM requirements, including Agile development.
147
Polarion
Product quality
10
Impact radar
Recognition
10
Product usage
8 6
8 6 4 2 0
Client engagement
4 2 0
Revenue
Customer support
Revenue growth
Portfolio depth
Technology radar
10 8
6 4 2 0
Collaboration
Polarion
Figure 33: Polarion ALM radar

Source: Ovum
Polarion is the only emerging ALM vendor with an impressive Technology score. Polarions score is marginally behind Shortlist vendors Microsoft and HP, and the vendor placed fifth overall on the Technology scale. This is mainly due to Polarions wide coverage of the ALM disciplines and aboveaverage scores in all ALM disciplines covered. Polarion offers a well-rounded ALM solution, and the only missing element is tool support for architecture modeling. Polarions performance on the Technology scale is impressive, particularly in change and configuration management, enterprise Agile, project management, solution architecture, and application security. The vendor scored highly on the Customer Sentiment scale as well, an area in which many established ALM vendors have faltered. Polarion received above-average scores in all Customer Sentiment dimensions, and is the category leader in dimensions such as client engagement, service capabilities, product quality, and customer support. On the whole, Polarion placed second on the Customer Sentiment scale, marginally behind Atlassian. On the Market Impact scale, Polarions performance is predictably poor. Polarion is the leader in terms of revenue growth; however, the growth came off a small base, hence the overall impact is low. Polarion does not have much history behind it; however, if the vendor continues to grow at a high double-digit rate (as a percentage of revenue growth year-on-year), it will soon be in a position to challenge bigger, established ALM vendors and have a greater impact in the market. Polarions ALM offering, Polarion ALM Enterprise, was first shipped in 2005. In this short time the vendor has put together an impressive solution, underpinned by a single solution-wide repository (Subversion). The offering is competent in most areas, as can be seen from Polarions ALM radars; however, certain gaps exist, such as lack of requirements modeling capability and lack of feature depth in application project portfolio management. Notably, the solution was developed from the ground up, caters well to the needs of SMEs, and is currently targeting the enterprise market.
148

10
Delivery management
8 6
Project management
4 2 0
Enterprise Agile
Quality management
Polarion

Source: Ovum
Figure 34: Polarion ALM components radar
Polarions impressive performance on the Technology and Customer Sentiment scales led Ovum to assign the Consider rating. The vendors ALM revenues are miniscule compared with the market leaders; however, that does not take anything away from Polarions offering. Polarions ALM solution merits closer attention from organizations that use Subversion as a repository and are yet to deploy ALM, as well as from organizations engaged in product engineering.
Rally Software
Rally Software was placed in the Explore category in the previous edition of the ALM Decision Matrix; the vendor managed to retain its position in this edition of the report. This is in line with the growing market adoption that Rally Software has enjoyed over the past couple of years. Rallys performance on the Technology scale is along expected lines; the vendor lost points in areas that it does not cover, such as application security, delivery management, change and configuration management, and architecture modeling support. It lost points in other Technology dimensions in which its scores were below par. Rally scored well in dimensions that it does cover, such as project management, requirements management, and enterprise Agile. On the whole, Rallys position on the Technology axis is second to last, slightly better than Atlassian. Ovum looks at breadth of ALM discipline coverage and feature depth in each discipline when evaluating vendors on the Technology scale; hence, vendors that operate in niche areas or offer only a subset of overall ALM functionality lose points. Rally is primarily an Agile ALM vendor, but with lifecycle traceability, Agile planning, and project management as major functionality areas, its offering addresses a broad and growing market. Ovum is also of the view that Rally Software is a leader in the ALM sub-segment (Agile ALM) in which it operates, and its impact on the ALM market is bound to grow in the near to medium term. Rally has executed its market strategy well; hence, the vendor finds itself a leader in the Agile ALM space. Rallys score on the Market Impact scale is predictably low compared with market leaders IBM, Microsoft, and HP. However, Rally has the highest impact among vendors placed in the Explore category, and is tied with Micro Focus for fourth place. In Ovums view, Rallys Market Impact is also comparable to other established ALM vendors such as MKS and Serena Software. Rally performed well on the Customer Sentiment scale; this goes to show that Rally has gauged the market well and captured mind share in its target market. The vendor has par or above-par scores in all Customer Sentiment dimensions, and leads the pack on dimensions such as product quality, client engagement, and service levels. This is significant because Rally has an installed base of 2,500 organizations in 60 countries.
149

Product quality
10
Impact radar
Recognition
10
Product usage
8 6
8 6 4 2 0
Client engagement
4 2 0
Revenue
Customer support
Revenue growth
Portfolio depth
Technology radar
10 8
6 4 2 0
Collaboration
Rally Software
Figure 35: Rally Software ALM radars

Source: Ovum
Rallys ALM solution is offered in many editions; the most comprehensive version is aptly named Rally Ultimate Edition. The vendor started shipping its Agile planning and project management solution in 2004; the solution has been on the market for nearly seven years, and this has enabled Rally to fine tune its product strategy and leverage the trends in the market to its advantage. Rally has also been able to rapidly integrate functionality from acquired solutions such as 6th Sense Analytics and AgileZen into the core platform. Having accomplished Agile planning and project management, the vendor now plans to expand into adjacent upstream areas such as application project portfolio management, and deepen its vertical expertise by offering vertical-specific functionality on top of its core Agile ALM platform. How the market reacts to Agile ALM offerings with vertical-specific functionality remains to be seen.
150

10
Delivery management
8 6
Project management
4 2 0
Enterprise Agile
Quality management
Rally Software
Figure 36: Rally Software ALM components radar
Source: Ovum
Rallys low score on the Technology scale led us to assign the Explore rating. However, as the discussion above suggests, Rallys relevance to the organizations practicing Agile development is on the rise. The vendor now seeks to help organizations scale their Agile practices from team or department level to enterprise wide. Ovum expects that Rallys influence in the Agile ALM market will grow and the vendor will break into the Consider category in the next edition of the ALM Decision Matrix.
Serena Software
Serena Software is an established ALM vendor with significant market impact and a large installed base. Serenas main business is developing and selling tools that support the application lifecycle, from an automation perspective and from lifecycle visibility and artifact/work-item traceability perspective. The vendor offers tools for distributed and mainframe environments. Serena was placed in the Consider category in the previous edition of the ALM Decision Matrix; the vendor has managed to retain its position in this edition of the report. On the Technology scale Serena performed better than most vendors in the areas that it covers, such as application project portfolio management, project management, requirements management, change and configuration management, and enterprise Agile.
151

Product quality
10
Impact radar
Recognition
10
Product usage
8 6
8 6 4 2 0
Client engagement
4 2 0
Revenue
Customer support
Revenue growth
Portfolio depth
Technology radar
10 8
6 4 2 0
Collaboration
Serena Software
Figure 37: Serena Software ALM radars

Source: Ovum
However, the vendor lost points in key areas such as delivery management and solution architecture due to lack of support for continuous integration, continuous testing, and product lifecycle management integration. Serena also scores below par in the collaboration dimension due to lack of wiki-based knowledge management capability. In the quality management dimension Serena lost points due to lack of support for test data management and unit test management. In all other ALM disciplines that Serena covers, its score is above par. Overall, Serena placed sixth on the Technology scale, behind IBM, MKS, Microsoft, HP, and Polarion. Serenas performance on the Market Impact scale is along expected lines; the vendor is predictably behind market share leaders IBM, Microsoft, and HP. That Serena has managed to hold onto its market share in a regressive economic climate is commendable. On the Customer Sentiment scale, Serena has on-par scores in most dimensions and a below-par score in product quality. Notably, Serena Software has managed customer expectations well; this is also reflected in the vendors high scores on customer support and service capabilities dimensions. In the previous edition of the ALM Decision Matrix, Ovum opined that Serena could pose a serious threat to the ALM market leaders due to its strong market presence and technology leadership; however, the vendor needed to improve its Customer Sentiment ratings. In this edition of the report, Serena has come down a few notches on the Market Impact scale; however, it managed to better its Customer Sentiment rating and placed fourth on the Sentiment scale behind Atlassian, Polarion, and Rally Software. Over the past few years, Serenas ALM strategy has undergone some changes. Until late 2010, the vendor was focused on composite applications, commonly known as mashups, and had a product called Serena Business Mashups, which aimed to facilitate cloud-based composite application development. Serena has shifted focus back to its main business application lifecycle management. Serenas ALM message is focused on demand management, development management, and release management, with overall governance and automation of the application lifecycle offered through Serena Business Manager.
152
It is interesting to note that Serena takes a process-oriented view of the application lifecycle and considers ALM a set of processes and activities that need to be orchestrated for delivering applications that meet business requirements on budget and on schedule. Key to this approach is Serena Business Manager, a product that not only acts as the process management and integration platform, but also provides the bulk of the reporting functionality inherent in Serenas ALM offering. Other key products include Serena Dimensions CM, Serena Dimensions RM, Serena PPM, Serena Prototype Composer, Serena PVCS, and Serena Agile.
10
Delivery management
8 6
Project management
4 2 0
Enterprise Agile
Quality management
Serena Software
Figure 38: Serena Software ALM component radar
Source: Ovum
Serenas moderate Technology score coupled with its significant Customer Sentiment score earned the vendor a Consider rating. As opined in the previous edition of the ALM Decision Matrix, Serena is well placed to challenge the Shortlist vendors for market leadership. Serenas large installed base is comprised mainly of large and medium-sized enterprises from financial services, professional services, public sector, manufacturing, and retail/wholesale. In our view, Serenas process-oriented approach to ALM will resonate well with its target market, and may well elevate the vendor to the ALM market leaders category in future.
TechExcel
TechExcel was placed in the Explore category in the previous ALM Decision Matrix; the vendor has been placed in the Consider category in this edition of the report. This is mainly due to TechExcels above-average Technology scores in all dimensions that the vendor covers through its ALM offering. TechExcel has below-par scores in Technology dimensions such as change and configuration management, application security, and delivery management; however, the vendor is not focused on these areas and does not offer native tool support for these disciplines. Ovum rates TechExcels ALM offering highly, and this is reflected in the vendors scores in individual Technology dimensions. However, the Decision Matrix report looks at both feature depth and breadth of functionality offered; hence, TechExcel lost some crucial points on the Technology scale. Overall, the vendor placed seventh on the Technology axis, behind IBM, MKS, Microsoft, HP, Polarion, and Serena. For this Decision Matrix, TechExcel was not rated by enough customers for Ovum to aggregate and present statistically significant Customer Sentiment scores. TechExcels Market Impact is two orders of magnitude lower than that of market leaders IBM, Microsoft, and HP. Although the gulf between TechExcel and the market leaders is wide and will not be bridged in the near to medium term, the ALM tools market has weathered the economic slump and TechExcel has ample room for growth.
153
Technology radars
10

10 8
8 6 4 2 0
6 4 2 0
Collaboration
Impact radar
Recognition
10 8
Regional presence
6 4 2 0
Revenue
Size-band presence
Revenue growth
Vertigal presence
TechExcel
Figure 39: TechExcel ALM radars

Source: Ovum
Moreover, the vendor registered high double-digit revenue growth over the past year, and Ovum expects this growth to continue. On the whole, Ovum expects TechExcel to grow at a rapid pace for the next few years. This growth will likely come from TechExcels investments in third-party hosted and SaaS variants of its ALM offering, DevSuite. DevSuite comprises four modules: DevSpec, DevPlan, DevTrack, and DevTest. In addition, TechExcel offers Agile Studio, which comprises DevPlan and DevTrack, an offering customized for teams practicing Agile development. DevTest and DevTrack are bundled into a DevTest Studio offering aimed at QA teams. Overall, TechExcels solution provides tight integration among individual modules, offers repository support for unstructured data, and scales well to support a large number of concurrent users.
TechExcels impressive scores in the Technology dimensions that it covers earned the vendor a Consider rating. In Ovums view, TechExcels solution will be relevant to and merits close attention from mid-market enterprises and independent software vendors. The solution has been proven in many ISVs environments, particularly with PC and console games developers.
154
Technology radars
10

10 8
8 6 4 2 0
6 4 2 0
Collaboration
Impact radar
Recognition
10 8
Regional presence
6 4 2 0
Revenue
Size-band presence
Revenue growth
Vertigal presence
Figure 40: ThoughtWorks Studios ALM radars
Source: Ovum
ThoughtWorks Studios is a new entrant to the Ovum ALM Decision Matrix, having entered the ALM market with a comprehensive ALM solution. ThoughtWorks Studios offers an Agile ALM solution that not only covers Agile planning, project management, and reporting, but also provides tool support for delivery management in Agile environments. ThoughtWorks Studios logged above-par scores in most of the Technology categories that it covers, namely project management, delivery management, requirements definition, and enterprise Agile. In high-level Technology dimensions such as breadth of lifecycle discipline coverage, solution architecture, collaboration, and internal tool interoperability, ThoughtWorks scores are close to, if not better than the average across vendors. ThoughtWorks Studios lost points in the requirements management dimension as the vendor does not yet support key features such as requirements-based test case generation and visual requirements modeling. Overall, ThoughtWorks Studios placed ninth on the Technology scale, ahead of the other three vendors with the Explore rating (Atlassian, CollabNet, and Rally Software).
155
ThoughtWorks Studios ALM offering comprises: Mingle Agile project management, requirements management, release planning, defect and issue tracking, and collaboration Go configuration management, build management, and release management Twist test management and functional testing For this Decision Matrix, ThoughtWorks Studios was not rated by enough customers for Ovum to aggregate and present statistically significant Customer Sentiment scores. On the Market Impact scale, a wide gulf exists between ThoughtWorks Studios and the market leaders, and it is unlikely that ThoughtWorks Studios will cover the gap in the near term. However, The vendor has executed its marketing strategy well, and been successful in capturing mindshare in this expanding market. This is due in no small measure to the strength of ThoughtWorks Studios Agile ALM offering, which was developed from scratch to support Agile development practices. Ovum expects that the vendor will continue on the growth path and be at the forefront of the challenger league by the time Ovum publishes the next edition of the ALM Decision Matrix. Furthermore, ThoughtWorks Studios has been acquiring customers in the highly competitive ALM market at a rapid rate, showing how traditional ALM vendors have been slow to deliver on the Agile front.
ThoughtWorks Studios moderate score on the Technology scale have led us to assign an Explore rating. However, Ovum believes ThoughtWorks Studios offering is a comprehensive ALM solution for organizations practicing Agile, and merits special attention from such enterprises. The solution will be especially relevant to enterprises that have failed or hit roadblocks in their Agile initiative, as its tools can be adopted incrementally with demonstrable value at each step.
156
Incorporating
OVUM Butler Group
CHAPTER 8: Technology Audits
WWW.OVUM.COM
Incorporating
OVUM Butler Group
ATLASSIAN: Atlassian ALM
WWW.OVUM.COM
TECHNOLOGY AUDIT
Atlassian
Atlassian ALM
SUMMARY IMPACT
Atlassians Application Lifecycle Management (ALM) offering differs from its closest competitors products in three significant ways. Firstly, it is an integrated toolset as opposed to a single monolithic product. Secondly, it has a strong developer focus, with tools that emphasize developer productivity enhancement through ease of use and simplicity rather than through extensive reporting. Thirdly, Atlassians low pricing business model is highly competitive. The key aspects of the solution are: JIRA Studio. Atlassians integrated toolset that leverages Subversion as the repository.available as a third-party hosted (dedicated) service or as a software-as-a-service (SaaS) solution. Atlassian Bamboo. Provides continuous integration, deployment, and release management for both on-premise and Amazon EC2 hosted environments.
KEY FINDINGS
Strengths: Single shared repository-based architecture and pre-integrated toolset. Wiki-based collaboration and knowledge management. Strong support for Agile methodologies including Scrum, XP, Kanban, and Lean. Superior defect and issue management capabilities. Weaknesses: Should look to improve requirements definition and management functionality. Lacks native analytics functionality. Key Facts:
i Completely web-based interface. Supports various integrated development

environments (IDEs) including Eclipse, Visual Studio, and IDEA.
i Integrates with HP Quality Center for QA management.
OVUM VIEW
Software development environments usually have many point products implemented to manage the different phases of software lifecycle. Apart from the tool interoperability and integration issues, this also results in the development artifacts being scattered, making it difficult to derive useful information from these systems. Atlassian is aware of the complexity of IT environments and offers a set of tools that not only integrate well with each other but also support various commonly used ALM solution repositories. Atlassians sales operation differs significantly from other IT vendors. The company has no proactive sales team, products can be bought directly off the website, and technical support/maintenance costs are included free in the first year as part of the perpetual license, and then charged at 50% of the license fee in subsequent years. These licenses are purchased for individual Atlassian products as there is no ALM suite. However, when combined, the point products form an integrated suite.
CHAPTER 8: ATLASSIAN ATLASSIAN ALM
161
In addition, Atlassian offers a range of hosted options for its products, including JIRA Studio which combines a portfolio of Atlassian tools an ALM suite in anything but name. Customers can see the licensing prices on the vendors website and can calculate exactly how much their proposed ALM solution will cost as there is full pricing transparency and no additional charges. Atlassian began offering development tools in 2002, and has an installed base of over 20,000 organizations across 134 countries. In less than a decade, Atlassian has established itself as a significant player in the ALM market, through its strong developer focus and by offering flexible support for multiple development processes (ranging from waterfall to Agile and hybrid variants), development platforms, and deployment options. In addition, Atlassians low starting price points and short implementation phases enable organizations to derive business value from the outset. In Ovums opinion, Atlassian offers a compelling proposition for development teams in the small to medium range, and also for some larger sized projects. The solution provides optimal functionality at a price point that is attractive not only to software shops, but also to department level development teams in large organizations. The price advantage also implies that line of business managers can sanction tool procurement without seeking approval from the CIO.
Recommendations
Atlassian offers highly flexible support for Agile development methodologies and globally distributed development teams. Organizations with extensive requirements for engineering processes will have to supplement Atlassians offering with a dedicated requirements definition and management solution. Where a more lightweight requirements approach is possible, this can be delivered with Atlassians Confluence wiki. Teams with ten developers or less should consider Atlassians tools for their ease of use, developer focus, and extremely low entry costs. For teams this size, the cost is $10 per tool, and all proceeds go to charity.
FUNCTIONALITY SOLUTION OVERVIEW

The Atlassian ALM portfolio contains a wide range of different tools. JIRA. A defect and issue management tool that also offers project management capabilities and can be used to manage Agile stories. GreenHopper. For Agile project management. Confluence. A wiki solution for requirements gathering, documentation, and project collaboration. Figure 41 shows a range of ways in which this product is used. FishEye. A source code exploration and reporting tool. Crucible. A tool for source code review. Bamboo. A continuous integration tool for automating build and testing. Agents can run on-premise or in the cloud using Amazon EC2. This product includes a builder, but can also connect with other popular builders such as Ant and Maven. Clover. A code coverage and test optimization tool for Java. Bitbucket. A code hosting site for Mercurial, the distributed version control system. JIRA Studio. A hosted Agile development suite comprising Subversion for the repository, JIRA, GreenHopper, FishEye, Crucible, Confluence, and Bamboo. Additionally, the vendor offers connectors for commonly used IDEs such as Microsoft Visual Studio, Eclipse, and IDEA.
162
Confluence is for technical teams...

Requirements Brainstorming Feedback
Documentation Launch plans
Track & Discuss
Specifications
Project Status
Figure 41: Atlassian confluence Source: Atlassian
Atlassian acquired Bitbucket in 2010 to widen its support for the Mercurial distributed version control systems. Quality assurance (QA) is not covered by Atlassian, but it does provide integration with third-party QA management solutions, including HP Quality Center, EMC LabManager, Catch Enterprise Tester, and Zephyr. It also integrates with other defect and issue management solutions besides its own JIRA tool. While Atlassian doesnt offer native software change and configuration management (SCCM) capabilities, JIRA Studio has an integrated Subversion repository. Furthermore, FishEye can read and index multiple repositories such as Subversion, Git, CVS, Mercurial, Accurev, Perforce, and Rational ClearCase, and it also offers support for source code level reporting and code exploration. Atlassians offerings are web browser-based with no client side footprint, the tools offer multiple third-party integrations out of the box, and they provide a much shorter time to value compared to traditional onpremise ALM solutions. Atlassian offers strong support for Agile development practices through developer tools such as Bamboo, and Agile planning and project management capabilities through GreenHopper. From a development process perspective, Atlassian offers support for waterfall development, as well as Agile methodologies such as Scrum, XP, Kanban, and Agile hybrid processes. In Ovums view, all-round process support is crucial for an ALM solution as it enables the tools to better fit the way that development teams work, leading to greater tool adoption and delivering value to the client organization. A key aspect of Atlassians offering is its developer focus, and that most of the tools in its portfolio are practitioner tools, as opposed to the heavy tracking/reporting focus of other new generation ALM tools. However, Atlassian also offers end-to-end traceability for lifecycle artifacts through multiple repository integrations, and provides superior tool support for source code analysis and peer review. Collaboration is another strong point of Atlassians ALM suite, and its key features include in-built instant messaging (IM), a wiki for document based collaboration, the ability to link documents to development process artifacts, and functions to annotate source code, scripts, change requests, and tests. Atlassians JIRA Studio is a single repository solution leveraging Subversion as the store for development process artifacts, as well as for source code version control. The solution also supports environments with multiple repositories, which is a key differentiator as it allows the solution to fit easily into the client organizations development environment. This shows that despite being a recent entrant to the ALM market, Atlassian has its priorities correct. It has brought to market a solution that not only reflects the new trends in software development but also has the ease of use, sensible design, and ability to integrate with existing point solutions that customers may have.
163
JIRA includes an embedded workflow engine that can be customized to suit different environments, and a free visual workflow plug-in for all JIRA users is now also available following a recent acquisition. While project management analytics is a missing component, Atlassian provides support for custom reports, and repository data can be made available for analysis through third party plug-ins. Ovum expects that the vendor will cover this gap in a future release. In addition, Atlassian offers support for visual architecture modeling through a third-party plug-in. Other key features of the solution include Google Apps and Microsoft Office integration, which allows users to import and attach documents, spreadsheets, or presentations to process artifacts, as well as leverage Google Apps. Atlassian also offers a service-oriented architecture-based application programming interface (API) and a software development kit to extend the capabilities of its solution.
SOLUTION ANALYSIS
Requirements definition and management
Atlassians requirements definition and management functionality is spread over JIRA and Confluence, and certain capabilities are available through third-party plug-ins. Confluence acts as a requirements gathering solution, while JIRA provides tracking and reporting features. Confluence enables users to capture requirements from structured and unstructured sources, and by leveraging a third-party plugin, users can import and convert unstructured input from external sources to a structured format, which also lends itself well to semantic analysis. Available features include a glossary, Microsoft Office integration, a customizable user interface, and collaboration features such as annotation of process artifacts. Requirements management features include requirements dependency mapping and reporting. Its important to note that JIRA can be adapted for requirements management, and can be used to hold large numbers of Agile story hierarchies. However, the product wasnt designed to be a requirements management solution. Therefore, requirements are modeled as issues that can be linked together, and further dependencies can be created as sub-tasks or child nodes in a tree-like structure. Reporting features include pre-built and custom reports, as well as graphical dashboards. Gaps in functionality from an enterprise perspective include lifecycle traceability, requirements baselining, change impact analysis, requirements coverage analysis, automated test case generation from requirements, visual representation of requirements, and requirements versioning. In Ovums opinion, Atlassians solution will benefit from covering these functionality gaps, especially features such as lifecycle wide traceability, requirements versioning, and change impact analysis. Organizations that follow the waterfall development methodology or depend heavily on requirements engineering processes should look to supplement the available functionality through a dedicated requirements management tool.
Project and portfolio management

JIRA and JIRA Studio (the integrated ALM offering) both offer project and portfolio management capabilities. However, portfolio management functionality is geared more towards managing multiple projects than managing an organizations application portfolio. Organizations typically manage multiple projects with one JIRA instance, and they can bring more projects under JIRA in time, as well as set up additional instances if required. Project portfolio management features include planning and scheduling with support for Agile processes and pre-built and custom reports. Other functionalities such as critical path analysis, scenario modeling, what-if analysis, resource management, and risk management are offered through the partner ecosystem. JIRAs integration with Microsoft Project helps round out the project and portfolio management functionality, providing access to critical path and what-if analysis features. In addition, financial management (features such as billing, expenses, and time tracking) and risk management functionality are offered through third-party plug-ins. Native project management features include work-item traceability, backlog management (including support for product, project, sprint, and release backlogs) for Agile projects, user story and task prioritization, a workflow whiteboard, metrics tracking, and reporting. Features such as resource allocation and scheduling are offered through the Microsoft Project integration and third-party plug-ins. Metrics tracked by JIRA include workload, user workload, version workload, issue resolution time, resolved issues against total issues created, average defect/issue age, and time since last issue.
164
Enterprise Agile capabilities

GreenHopper provides Agile project management capabilities for JIRA projects. A card wall is provided, which enables users to visualize and quickly assess the status of work-items, as well as plan iterations. While users can create projects from scratch, GreenHopper offers Agile project templates to help them get started quickly. Atlassians Agile functionality is best suited to the Scrum methodology, but it also lends itself well to Kanban, Lean, and XP. Furthermore, the solution can adapt to custom development processes. The key strength of this offering is its ease of use. The card wall is made up of planning and task boards that allow users to edit, prioritize, and organize cards as they see fit. Visibility into the development lifecycle is provided through features such as product, project, Sprint, and release backlogs out of the box, and time tracking can be done either through developers IDE integration or by modifying the task card directly. GreenHoppers reporting features include dynamic configurable burnup and burndown charts for Sprints and releases, velocity charts, and cumulative flow diagrams. GreenHopper leverages OpenSocial gadgets and JIRAs reporting features to deliver graphical dashboard functionality.
Continuous integration, deployment, and release management

Bamboo provides continuous integration functionality for both on-premise and Amazon EC2 hosted environments. The tool offers native support for Subversion, Mercurial, Perforce, and CVS repositories, while GIT and ClearCase are supported through third-party plug-ins. Supported build automation tools include Ant, Maven, Maven2, make, NAnt, Visual Studio, and Sonar. Bamboo can integrate with any testing tool that supports JUnit XML output. Supported languages include Java, .NET, Ruby, and PHP. Available functionality includes build process definition and workflow, build planning, dependency mapping, build validation, dashboards for quick visibility into build process status, configurable alerts and notifications through email/IM/RSS, IDE integration through Atlassian IDE connectors, bi-directional integration with JIRA to close the loop between failed builds and issues/defects, distributed builds with the help of remote agents, and detailed reporting that not only provides visibility into the status of builds but also facilitates root cause analysis for build failures. Bamboo also provides third-party integrated deployment and release management functionality. Integration with tools such as Maven and Nexus enables automated deployments. Release management is supported through a plug-in that has recently been acquired by Atlassian and will be made available as a free add-on to Bamboo.

JIRA is essentially a defect and issue tracking tool. It enables users to create, view, edit, and track issues, attach files/screenshots/URLs as supporting documentation, estimate the problem resolution time, and create sub tasks. The tool gives administrators complete control over end-user functionality and access permissions, issue types, issue fields and labels, mapping issue management workflow to business processes, and email notifications. All of these parameters can be configured at both solutionwide and project levels. Developers can access JIRA from their IDEs, and commonly used IDEs are supported out of the box, while others such as Adobe Flash Builder, Zend Studio, and Oracle JDeveloper can be integrated through third-party connectors. JIRA allows users to search issues through a search toolbar that offers free-text search, issue keywordbased search, and automated task listing by user ID. Other search features include complex searches through a navigator interface, saving search options, and search results output in multiple formats such as browser, printable, XML, HTML, Microsoft Word, and Excel. Other available features include multi-dimensional and project specific reporting, customizable dashboards, global/project/issue/comment level privileges, email notifications of issues, and search results delivered through email. The solution also provides customizable workflow, allows different issues to be linked together, and even provides stakeholders with voting rights. JIRA is extendable through plug-ins, and the tool integrates with version control tools such as CVS, Subversion, Perforce, and Accurev. In addition, it facilitates integration with custom home-grown applications through XMLRPC and SOAP interfaces.
165
PRODUCT STRATEGY MARKET OPPORTUNITY

Atlassian should benefit from the constant changes in the ALM market, especially from trends such as globally distributed development, greater Agile adoption, application development in the cloud and for the cloud, and the need for greater collaboration between distributed development teams. In Ovums view, Atlassians investment in distributed version control systems is likely to pay off as the vendor offers superior support for distributed version control tools. On the other hand, trends such as greater Agile adoption will be beneficial to the new generation of ALM vendors. Ovum believes that Atlassian stands to gain from the current trends, and will grow with the market in the near to medium term. Atlassian has a policy of delivering source code to each customer with its sales, which, together with a wide set of documented APIs, enables many enterprise customers to integrate JIRA and Confluence with other internal applications and create tight data transfer integrations. The vendor has found this to be a key lynchpin to JIRAs adoption in large enterprises.
GO-TO-MARKET STRATEGY
Though the target market segment for the solution is horizontal, companies from manufacturing, financial services, professional services, healthcare, and public sector and education segments are significant in Atlassians installed base. Atlassian has a strong presence in the Americas, with the region accounting for approximately half of the vendors annual revenues. Europe, the Middle East, and Africa accounts for 40% of its revenues, while Asia-Pacific accounts for the remaining 10%. The majority of Atlassians revenues come from direct sales. Web-based sales (directly from the companys website) account for 85% of the companys revenues, while sales through partners account for the remaining 15%. Atlassians business partners include Customware, GoToGroup, and Pix Software. Globally, there are over 400 Atlassian partners, which provide services such as deployment, customization, integrations, and plug-in development. The company also has technology partnerships with Google, CollabNet (Subversion), and Contegix. The toolset is sold for a perpetual or subscription license on a named-user basis. Bamboo is sold for a perpetual license based on the number of remote agents. Atlassian estimates that the average project value for an installation (behind the firewall) is approximately $12,000, and that it includes 25 named user perpetual licenses for JIRA, GreenHopper, Confluence, FishEye, Crucible, and Bamboo. The hosted version of the offering with all other parameters unchanged is available for $5,000. Large deployments typically cost approximately $56,000 and include enterprise-wide perpetual licenses for JIRA, GreenHopper, Confluence, FishEye, Crucible, and Bamboo. The hosted equivalent of JIRA Studio for 1,000 users is available for $15,000 per year. There is no professional services component to these indicative figures given above.
IMPLEMENTATION
According to the vendor, a department level or enterprise-wide implementation for 30 to 500 users can be set up and configured for use in one day. Third-party (Contegix) hosted instances can be set up in less than five minutes, and solution customization such as adding connectors to other lifecycle tools and enterprise applications as well as plug-ins takes approximately seven to eight hours. Atlassian doesnt have a professional services team, and its implementation services and consulting are provided by Atlassians system integrator partners. The vendor offers standard support during business hours for all regions globally. Support in the first year is bundled with the perpetual license price. For subsequent years, Atlassian charges 50% of license costs for technical support. End-user training isnt bundled with the solution as Atlassian believes that most users dont require training as there is no associated learning curve. However, client organizations can purchase onsite or web-based training classes from the vendors website. All tools except JIRA Studio can be deployed on-premise, and other deployment options include thirdparty hosted and SaaS. The toolset is supported on Windows, Linux, and Mac OS platforms. Atlassian has over 20,000 customers of which 14,250 use JIRA, 9,600 use Confluence, and 3,000 use GreenHopper for Agile planning and tracking. JIRA Studio has 1,200 customers.
166
DEPLOYMENT EXAMPLES
Zynga. The social game development company uses Atlassians JIRA, GreenHopper, Crucible, and FishEye. It uses Perforce as its solution repository. Atlassians solution is central to Zyngas development environment and touches on many other facets of Zyngas IT environment. The complete case study on Zyngas usage of Atlassian tools can be found on Atlassians website. hCentive. hCentive healthcare enables users to share their healthcare expenses anonymously, thereby allowing consumers to better manage their healthcare expenditure. hCentive uses the hosted solution JIRA Studio for managing their application development function. JIRA Studio enables the company to better manage its geographically distributed development efforts. The complete case study on hCentives usage of Atlassian tools can be found on Atlassians website. John Hopkins University. John Hopkins University uses JIRA as a workflow tool and knowledge management solution. JIRA is central to the universitys change management, defect tracking, and task management processes. Issues are submitted JIRA, and through its use as a defect tracking and issue management system, it has also come to be the institutions primary knowledge base for IT related issues. Atlassian Pty Ltd 375 Alabama Street #325 San Francisco, CA 94110 USA Tel: +1 (0)415 701 1110 Fax: +1 (0)415 449 6222 www.atlassian.com Atlassian Pty Ltd Herengracht 124128 1015BT Amsterdam Netherlands Tel: +31 20 796 0060 Fax: +31 20 524 8360
167
Incorporating
OVUM Butler Group
COLLABNET: CollabNet TeamForge 5.4
WWW.OVUM.COM
TECHNOLOGY AUDIT
CollabNet
CollabNet TeamForge 5.4
SUMMARY CATALYST
Originally known as the creator of the popular open source Subversion source code control system, CollabNet has steadily built and acquired its way to a broader portfolio of application lifecycle management (ALM) solutions that cover the planning and lifecycle management of software development, Agile program management, and lab management in the cloud. CollabNet recently acquired Codesion, the largest third-party cloud service provider for Subversion. TeamForge is CollabNets ALM hub and core planning engine. Although TeamForge supports Agile project management, the recently acquired ScrumWorks product remains CollabNets primary Agile tool at the team level. CollabNet has energetically promoted source code change and configuration management (SCCM) in the cloud. CollabNet has recently entered a strategic partnership with HP Software that addresses its major gap: quality and requirements management.
KEY FINDINGS
Strengths: Enables planning of features and release timelines in the same view. Deeper integrations with third-party applications through the CollabNet. Visual workflow and repository browsing supports remote repository management. Weaknesses: Would benefit from enhancements to reporting functionality. Lacks analytics capabilities. Key Facts:
i ScrumWorks Pro features can be accessed through TeamForge. i Users can track ScrumWorks artifacts through CollabNet Tracker.
OVUM VIEW
CollabNet pioneered software development in the cloud with Subversion, an open-source source code management (SCM) system with a web interface that was designed as a central resource for globally distributed development teams. Subversion not only opened the way for development in the cloud, but it was one of the first of the modern wave of lighter weight, low cost or (if used under the open source distribution) no-cost SCM systems that helped to commoditize this market. CollabNet subsequently developed collaborative planning and workflow tools wrapping around Subversion. The genesis of todays TeamForge offering is in the convergence of the cloud-based CollabNet Enterprise with the onpremises offering SourceForge Enterprise, which was acquired from VA Software in 2007. Today, most customers of the original CollabNet and SourceForge products have migrated to TeamForge.
CHAPTER 8: COLLABNET COLLABNET TEAMFORGE 5.4
171
CollabNet has a large and growing installed based of over 6,000 organizations, over 700 of which are TeamForge customers. Notably, some of the largest software development shops, including independent software vendors (ISVs), use the CollabNet TeamForge platform. Many CollabNet customers are large enterprise organizations that have deployed the platform for tens of thousands of users across the globe. CollabNet has a direct presence in all major geographies and a strong partner network, and its hosted and software-as-a-service (SaaS) deployment options help to ensure a steady and predictable revenue stream. It has buttressed its SaaS strategy with Lab Management, an offering that provisions virtual servers in the cloud, plus the recent acquisition of Codesion, the largest thirdparty hoster of Subversion. Ironically, while CollabNet rode the post-2000 wave of global development, it largely missed the emergence of Agile development. Although TeamForge supported Agile planning templates, the large global team planning functionality was considered too rich by many Agile development shops. That gap was addressed with the early 2010 acquisition of Danube, which brought with it the ScrumWorks Agile planning product. Significantly, CollabNet chose to keep the ScrumWorks product separate, as it addresses a distinct market and use case: smaller teams that do not require the planning features of TeamForge. Nonetheless, it has provided bi-directional integration that can be configured for interactive or periodic refresh. Staying true to its roots, CollabNet recently unveiled Subversion Edge, a free superset of Subversion that adds some of the most frequently downloaded third-party open source components, accompanied by a web-based management console for governing code repositories. The major gap in CollabNets ALM tooling used to be in the quality and requirements management space, but its recent alliance with HP for integrating Quality Center fills the missing links for both parties. As the alliance is still fairly new, both companies are still getting to know each other. However, because the two are such a complementary pair, Ovum believes that CollabNet could make a good acquisition target for HP. CollabNets current product portfolio is shown in Figure 42.
CollabNet Agile Development

Agile Program Management Subversion Management Agile Software Development Dev/Test Cloud Provisioning
CollabNe
t Develop
ment Platf
orm
Figure 42: CollabNet product portfolio
Source: CollabNet
172
Recommendations
Enterprises with multiple distributed development teams TeamForge will benefit organizations that are struggling to manage their globally distributed development teams. Small and medium enterprises and ISVs CollabNets subscription-based pricing model enables small teams to benefit from its ALM solution. The solution has been deployed by multiple ISVs. For complex multi-stage product development Organizations that require advanced requirements management functionality for complex, multi-stage projects can deploy CollabNet TeamForge ALM. However, they should augment requirements and quality management functionality through thirdparty tools.

The CollabNet ALM suite comprises CollabNet TeamForge, a web-based platform for managing the software application lifecycle; CollabNet Subversion Edge, an enterprise-grade distribution of the Subversion version control system; CollabNet Lab Management, a tool for provisioning and managing hardware assets in virtual and cloud environments; CollabNet ScrumWorks Pro, an Agile project and program management tool; desktop clients for Eclipse, Windows, Microsoft Visual Studio, and a dedicated client for ScrumWorks Pro; numerous connectors to third-party ALM tools; and a Simple Object Access Protocol (SOAP) application programming interface (API) for creating custom integrations. CollabNet products are available onsite or as a hosted service via Codesion cloud services. CollabNets approach to ALM is collaborative and community-oriented, with all functionality provided through the community interface (see Figure 43). The community enables access to user and project administration, task creation and management, SCCM, defect management, and lifecycle visibility and traceability through CollabNet Tracker. It also enables access to integrated build and test management (functionality is provided through integration with Hudson), release management, alerts, notifications, and realtime reports. In addition, the solution offers a project wiki, discussion forums, and document management capabilities. While TeamForge supports multiple methodologies and offers a template for Agile planning, teams following pure Agile can rely on ScrumWorks as a standalone tool, with TeamForge as the platform for coordinating Agile projects with other development projects. The TeamForge platform offers Web 2.0 interfaces and each user is provided with a personal workspace within the community. Users can view project dashboards, track task and artifact status, view projects with which they are involved, and manage artifacts per project. The TeamForge community can be hosted over a single/shared or a federated repository structure, depending on the number of geographical locations. User access is facilitated through the web-based interface. In addition, the solution allows developers to work from within an integrated development environment (IDE) (it supports Microsoft Visual Studio and Eclipse IDEs). CollabNet also provides a Windows desktop client for other user categories such as testers, business analysts, and project managers. TeamForges support for globally distributed teams is also evident through the embedded useradministration functionality. In addition to internal users being authenticated against an Lightweight Directory Access Protocol (LDAP) server, TeamForge allows secure access to users from external entities such as partners or outsourcing providers. Access permissions are granular and include resource-specific access to users, path-specific access to source code, and project-level restrictions. CollabNet has significantly enhanced the capabilities of the Subversion version control system with the present release. Dubbed as Subversion Edge, the distribution includes Apache Web Server and ViewVC, along with Subversion, preconfigured to work together. As a result, the repository scales better, offers graphical repository browsing, and simplified management and administration features. Another key aspect of the present release is the addition of ScrumWorks Pro. In the previous Technology Audit, Ovum had identified the lack of dedicated Agile planning and project management functionality as a major weakness of CollabNets offering. With the addition of ScrumWorks Pro, however, that gap has been plugged. Additionally, TeamForge now includes an Integrated Application Framework, which not only facilitates user interface (UI)-level integration, but also enables external applications to access TeamForge objects directly, as well as the TeamForge event handler, thus fostering deeper integrations.
173
TeamForge offers support for a range of development processes, methodologies, and tools, and enables centralized management of the entire development function.
The CollabNet Platform: Structure & Features

My Workspace Community & Projects
User and Project Admin Task hierarchy, Alert mechanism Browse SCM, tracker SCM integration Bugs, Artifacts Requirements Integrated Hudson
Build & Test
Project Admin
Tasks
Source Code
Tracker
My Page
Users personal workspace Per-project: task, tracker & resource status List existing, and create new, projects Central Repository
Dashboard
CVS
SVN
Lab Management
On-demand servers in public or private clouds Collect, archive & release packages Real-time reports & Status Discussion forums
Projects
File Releases
Monitoring
Manage monitored artifacts and tools by project Personalization settings
Non-Developers
CollabNet Desktop: - Microsoft Windows Edition
Developers
CollabNet Desktop: - Eclipse Edition - Visual Studio Edition
Report
My Settings
Discussions
Project-based Wiki
Wiki
Documents
Indexed objects 300+ File types
Figure 43: TeamForge development platform
Source: CollabNet
SOLUTION ANALYSIS
Architecture and platform support
CollabNet TeamForge is a web-based platform for ALM that runs on top of a Java Enterprise Edition application server (see Figure 44). TeamForge has a three-layer architecture comprising the core platform in the middle, the repository layer at the bottom, and the web-based user interface at the top. Every artifact, from code fragment to related wiki pages, test requirement, user story, planning milestone, and so on, is an object that can be associated with other objects. Subversion is a key component of TeamForge, providing core application services for software configuration management. The TeamForge solution repository is implemented in a PostgreSQL database, but TeamForge also supports Oracle. Lab Management is an optional module. TeamForge interacts with desktop-based IDEs such as Eclipse and Visual Studio, and the Windows desktop through a web services interface. Third-party tool interactions are enabled through an out-of-thebox web services (SOAP) API. CollabNet supports HP Quality Center for test management, and Subversion, IBM Rational ClearCase, CVS, and Perforce for version control and configuration management. It supports Microsoft Project for task management, SourceIQ Analytics and Hudson for continuous integration, and Electric Clouds Electric Commander for automated build, test, and deployment.
174
TeamForges document control system integrates Microsoft Office with its own proprietary underlying documenting versioning system. This allows people who work in Office and to have their files automatically saved in TeamForge each time they save their Word files. Besides Microsoft office, the TeamForge document version control technology can index and search over 250 file types. A complete list of CollabNet TeamForges integrations is available on the vendors website. Client organizations can also create custom integrations with the help of the SOAP API.
Process and methodology support

CollabNet supports a range of development processes including Agile, waterfall, and hybrid approaches. CollabNet has recently begun promoting TeamForges Agile support, which includes a project template with pre-configured trackers for user stories, epics, development tasks, defects, and test cases. New templates can also be created from scratch, and multiple templates can be saved. CollabNets Dynamic Planning feature enables management for Agile projects (time/scope) by using a configurable planning tree hierarchy that consists of products, releases, and iterations. These objects are mapped to epics, stories, and task lists maintained in the feature tree. The planning tree may also contain links to other process and project artifacts, and individual nodes can be linked to work items at any level in the feature tree. Ironically, the richness of TeamForges planning and lifecycle management functionality was long considered a drawback among many Agile practitioners. To better reach this market, CollabNet acquired Danube so as to offer a simpler planning tool geared towards Agile workgroups. In the year since the acquisition, both tools have been kept separate as they appeal to different target audiences. However, CollabNet has developed two-way integration, so work items from ScrumWorks (which encompass user stories and development tasks) can populate TeamForge, which is used for planning at the enterprise level. Recognizing that both tools serve separate markets, CollabNet does not plan to unify them.
Reporting
Users can generate burn-down reports for any planning tree object, which helps TeamForge to combine project management with Agile requirements management, defect management, and test management. In addition, it offers a rich set of reports and charts that help stakeholders track project status. TeamForge leverages the open source Flex 3.0 framework for creating rich UIs and reports that display project status information. The graphical reports are limited to open versus closed artifacts, number of open artifacts by priority, burn-down chart with velocity, and a capacity utilization report for any level within the planning tree. It would be good to see additional report-customization capability in a future release. The solution would also benefit from native or third-party integrated analytics functionality.
Workflow and lifecycle visibility

TeamForges planning and workflow-modeling capabilities enable users to link and track artifacts and work items according to their convenience. Project team members can use associations to link defects, issues, documents, discussions, wiki pages, release plans, feature requests, user stories, and support cases with the source code, to provide full traceability of all assets throughout the software development lifecycle. Project managers provide role-based access to project team members, create dependencies, and enforce rule-based transitions for artifacts that are used as the hub of the software development workflow. Business users can create multi-step processes with a large degree of control over when the transition to the next step is to be made, and which users can make it. Automated email notifications can be set up to inform stakeholders of state changes and progress, and they can add comments, attachments, or new artifacts via email. Existing projects and workflows can also be reused as templates for new projects. Project managers have realtime visibility across the lifecycle, and with the help of CollabNets flexible release-planning functionality, they can reorient and reprioritize deliverables based on realtime status reports.
Software change and configuration management

The Subversion SCCM system is part of the core platform architecture. This enables greater intrinsic control and leverage over the project artifacts. In addition, all artifacts are objects, and can be discovered, linked to any other object, and monitored in realtime with the help of CollabNet Tracker.
175
The web-based single centralized repository architecture of the solution enables source-code sharing, change tracking, Internet Protocol (IP) protection, and better governance. It also provides low-cost SCM for geographically distributed development teams. Features include versioning of files, directories, and file metadata, as well as branching/tagging support including trunk-based branching and merging. There is also support for continuous integration and non-exclusive file locking, atomic commits, handling of binary files, and support for merge tracking. Users can create workflows that enforce traceability of code-commits to artifacts such as requirements and issues, and can generate realtime notifications via email. Subversion Edge, available with the current release (TeamForge 5.4), adds remote management/administration capabilities, as well as visual repository browsing functionality.
Eclipse
Browser In
dependen
t User Inte
rface
Web Services
Applicatio
ns
Collaborative APIs
Windows &
CVS
VS
HP QC
Subvers
Perforce
ion Core Se r vices
Hudson
J2EE Ap plicatio
n Ser ver
Figure 44: CollabNet ALM platform architecture
Source: CollabNet
Distributed lab management for virtual and cloud environments

CollabNets Lab Management offering, previously known as CUBiT, is an optional add-on to TeamForge that provides cloud management capabilities for distributed development teams. The module supports hosted build and test tools, and maintains extensible pools of standardized build and test resources that can be accessed by developers. It enables organizations to leverage private clouds, as well as public clouds such as Amazon EC2 or CollabNet OnDemand, and contains a profile library that enables users to centrally manage build/test configurations and software. The module provides self-service access to users so that they can create custom profiles as well as use pre-built templates for commonly used build and test automation tools. The tool provides visibility into continuous integration processes through a reporting engine. Status reports cover key parameters such as machine allocation and capacity, and reports grouped by users, roles, and functions are also available. The governance framework enables compliance reporting by securing access to artifacts and maintaining audit trails. The module also provides financial management features such as project-cost tracking, and cost accounting of machine time needed to facilitate project/user reporting.
176
PRODUCT STRATEGY
CollabNets target market is horizontal in terms of both industry segment and company size. The vendor has customers across multiple verticals and revenue bands. Key customer verticals include high-tech, financial services, government, and healthcare. CollabNet also has a presence in automotive, consumer goods, media and entertainment, and semi-conductors. CollabNets customers range from very small development shops to large enterprises with thousands of developers. Most of CollabNets customers are based in North America, Europe, and India. CollabNet has sales offices in North America, the UK, Europe, and Asia Pacific. It sells in South America through a reseller. CollabNets route to market is a mix of direct and indirect sales, skewed in favor of direct sales. CollabNet derives about 90% of its revenues through direct sales across industry verticals, geographies, and revenue bands. Indirect sales are primarily oriented towards bolstering the direct channel and extending CollabNets reach in geographies such as Europe, Asia Pacific, and South America. CollabNets system integrator and distribution partners include Strategic Systems Solutions, Aservo, Elego, B-Vision, CMI, Innerbus, and Three Pillar. CollabNet also has partnerships with key technology vendors including Microsoft, VMware, HP, TaskTop, Hudson, SourceIQ, and PushToTest. As a result of its partnership with Microsoft, CollabNet supports Visual Studio IDE, and also offers a Windows Desktop client for TeamForge. CollabNet also has a partnership with HP for HP Quality Center test management integration, and with VMware to provide support for Open Virtualization Format, which enables code to be configured and packaged as a virtual appliance for release. CollabNet TeamForge is sold via a per-user subscription license, the terms of which vary depending on the deployment option chosen. The multi-tenant SaaS solution is sold for a monthly subscription. As the scope of deployment grows, especially for enterprise organizations with large distributed development teams, it makes sense to switch to the hosted model. Small development shops can also enjoy a price advantage with the SaaS model; with the recent Codesion acquisition, CollabNet is now offering a TeamForge Project with 5 ALM users that, at the end of 2010, was quoted for as little as $16.50 per user per month. CollabNet ScrumWorks is available either hosted or onsite. The Lab Management module is not included in the TeamForge ALM or SCM subscriptions; it is sold separately. The average project value for an entry-level 25-user TeamForge deployment is nearly $7,500, with the services component accounting for roughly 20%. An average-size deployment costs about $60,000, one-quarter of which is the services component. A large CollabNet deployment with 2,500 or more users costs in excess of $1m per year. CollabNet releases a major version of TeamForge approximately every eight months, with upgrades, patches, and minor versions as required. In upcoming releases, CollabNet will enhance manageability, replication, and scalability of Subversion Edge servers; add features such as enterprise visibility and tracking based on customer feedback; add a business-intelligence (BI) layer for abstracting and warehousing customers historic ALM data; and make enhancements to development and collaboration tools. CollabNet Subversion Edge will move towards becoming the default Subversion distribution, and will focus on scaling large/complex environments. CollabNet plans to deepen the integration between the flagship TeamForge and the newly acquired ScrumWorks Pro products. Enhancements and new features planned for ScrumWorks Pro include an expanded web-based interface, workflow customizations, and Kanban support. The Lab Management offering will feature deeper integration with continuous integration tools (Hudson for CollabNet TeamForge).
MARKET OPPORTUNITY
Ovum has identified the following opportunities from its 2011 Trends to Watch report for ALM: With Agile now in the mainstream, new pragmatic approaches focused around mixing and matching the right methodologies will emerge and become supported by ALM tool vendors.
177
Demand for cloud deployment will prompt SaaS-based ALM offerings, not only from new entrants, but from established players as well. With the explosion in mobile application development started by Apples iPhone/iPad and Googles Android platform, smart mobile will become a major target for enterprise applications. Security testing is entering the ALM mainstream. Embedded software development for engineered products is on an exponential growth curve and represents a lucrative market for the few ALM providers that have chosen to target it. In 2010, DevOps entered the awareness phase in the ALM market, and 2011 will see automated tooling targeting the needs of DevOps and Agile release management in particular.
Agile methodology support

Although CollabNet caught the wave of global development that followed the explosion of outsourcing in the early 2000s, it initially missed the Agile wave. TeamForge eventually added Agile methodology support, but the company was not a major player in this small but rapidly growing market sub-segment prior to its acquisition of Danube, which has given CollabNet presence as a challenger to Rally. Along with cloud services, Agile is one of CollabNets prime marketing thrusts for 2011.
Cloud
CollabNet pioneered cloud-based SCCM as the key to its strategy for supporting globally distributed software development; prior to the SourceForge acquisition in 2007, CollabNets ALM solution was exclusively a hosted offering with a web interface. It has bolstered cloud support with Lab Management, for provisioning virtual server profiles to the cloud; support for Amazon EC2; and recent the acquisition of Codesion, which expands its presence with Subversions largest hosting provider.
Mobile
While CollabNets tooling can be used for developing code for mobile targets, its products do not offer specific support for mobile development frameworks or form factors.
Security testing
CollabNet has yet to develop a strategy for integrating security testing or factoring security risk management into its TeamForge planning platform.
Embedded software development

CollabNets tooling can be used for developing code for embedding into engineered products or as part of complex systems of systems. However, like most ALM vendors, CollabNet does not target this space, and lacks the complex requirements and risk management functionality that are essential for it.
DevOps
Integration with continuous-build tools such as Hudson mark the first step towards developing the collaborative functionality necessary for managing the release of software into the operational environment. The Codesion acquisition brought CollabNet the capability to publish into public or private clouds using the publisher component. For now, CollabNet has not moved further to add functionality or alliances with providers that feed the change management, provisioning, service desk, or configuration management databases (CMDB) that are used in the operational environment. Ovum expects CollabNet to start building a partner-driven DevOps product strategy in 2011.
GO TO MARKET STRATEGY
At present, roughly 90% of CollabNets sales are direct, across all geographies, industries, and company size. CollabNet uses resellers in selected regions of Europe, Asia Pacific, and South America as supplementary go-to-market channels.
178
IMPLEMENTATION
Organizations can download a free 90-day trial of CollabNet TeamForge (limited to 25 users) from the vendors website. A virtual appliance that runs on top of VMware Player, and a native Linux installer are available for the trial, which CollabNet claims can be deployed in under an hour. Alternatively, customers can sign up for a free 30-day trial of TeamForge via Codesion cloud services and have a provisioned TeamForge project running within minutes. A small-scale deployment project involves setting up the user community and configuring project templates to suit the teams requirements. This can be completed in under a week. The resource requirements include one employee from the customer and one from CollabNet. A large-scale CollabNet implementation takes about three months to complete, involving two client-side resources and one from CollabNet. The solution rollout and implementation includes setting up the community, template configurations, and integrating third-party tools. Integrations here refer only to the out-of-thebox integrations offered by CollabNet. The creation of custom integrations through CollabNets open APIs may take longer. According to the vendor, post-deployment solution administration overheads are minimal, and range from 0.1 to 0.5 full-time equivalents (FTEs) depending on the deployment footprint. The vendor offers services to help clients assess the proposed solution deployment, along with implementation and post deployment training. CollabNet offers web-based self study and instructor-led training courses, including courses on Agile development processes. The training material can be accessed from the vendors website. Customers can also opt for onsite instructor-led training if required. CollabNet offers three levels of technical support: Silver, which provides support during the business day only; Gold, which provides round-the-clock support on weekdays, five named contacts, and a maximum of 120 support incidents per year; and Platinum, which offers 24/7 support, an unlimited number of incidents, and 10 named contacts. Additional details about CollabNets support programs are available on the vendors website. CollabNet offers multiple deployment options for TeamForge. The solution can be deployed on-premise as a virtual machine that runs on VMware Player, or can be installed natively on a server running Red Hat Enterprise Linux or CentOS. CollabNet expects to offer a native Windows installer in 2011. CollabNet also provides an SaaS offering with both single-tenancy and multi-tenancy options. TeamForge is available on Windows (VMware version only), 32/64-bit Red Hat Enterprise Linux v5.4, 32/64-bit SUSE Linux Enterprise v11.3, and 32/64-bit CentOS v5.4. TeamForge ships with PostgreSQL 8.3.8, but also supports Oracle 11 and 11g databases. TeamForge offers multiple role-based clients, including a Visual Studio client, an Eclipse client, and a Windows client for non-developer roles. In addition, the solution can be accessed over the web through browser clients. Supported browsers include Mozilla Firefox and Microsoft Internet Explorer. ScrumWorks Pro Desktop Client is compatible with Windows XP/Vista/7, Mac OS X 10.5.2+ 64-bit with Java 6, and Linux KDE windows manager. The ScrumWorks Pro Web Client is compatible with Internet Explorer 7+, Mozilla Firefox 3.0.x-3.6.x, and Safari. Subversion Edge 1.3 is supported on Windows (XP, Vista, Server 2003/2008), 32/64-bit CentOS 5.x, 32/64-bit RHEL 5.x, SUSE Linux 11.x, and Solaris 10 (SPARC/x86). CollabNet Lab Management can be deployed on all of the Windows operating systems listed above, as well as on RHEL 3+, CentOS 3+, Solaris 10 (both x86 and SPARC architectures), SUSE Enterprise Server 10/11, and VMware ESX, ESXi.
DEPLOYMENT EXAMPLES
CollabNets products are deployed at over 6,000 organizations around the globe. This number also includes Subversion support customers. The ALM offering is being used by over 700 organizations globally. The largest enterprise deployment of TeamForge supports more than 40,000 users across multiple geographical locations. CollabNet also supports open source community sites with user counts in the hundreds of thousands.
179
A US federal government agency

The organization has deployed CollabNet TeamForge to streamline its software development and delivery processes. The solution is provided as a hosted platform, serving over 7,500 users across more than 700 projects.
The IT arm of a large German enterprise

The organization wanted to standardize software development tools and processes across multiple departments. CollabNet TeamForge acts as the hub connecting multiple ALM tools, both proprietary and open source. TeamForge provides a collaborative environment, connecting end users, development tools, and development process artifacts. The ALM platform currently supports over 200 users across 115 projects. By the end of 2010, the company plans to transition nearly 80% of its 1,300 projects and 2,0004,000 users onto CollabNet TeamForge.
A large German bank

CollabNet TeamForge is the cornerstone of this banks development function, providing a collaborative development environment to over 9,000 users. CollabNet TeamForge has helped the organization to enhance developer productivity and lower development costs. CollabNet Inc. 8000 Marina Boulevard Suite 600 Brisbane, CA 94005-1865 USA Tel: +1 (650) 228 2500 Fax: +1 (650) 228 2501 www.collab.net
180
Incorporating
OVUM Butler Group
HP: HP Application Lifecycle Management
WWW.OVUM.COM
TECHNOLOGY AUDIT
HP
HP Application Lifecycle Management
SUMMARY CATALYST
HP Application Lifecycle Management (ALM) 11 is a significant release from HP Software, and a substantial element in its Business Technology Optimization portfolio. While many new products have been added to the portfolio, the key release is the ALM integration platform that binds HP Quality Center (QC) and HP Performance Center together, and the addition of an Agile project management tool to provide HPs first organic solution for the development management ALM segment. Highlights among the new generation tools include: Script-less testing tools with automated script features for rapid Agile testing, covering complex Web 2.0 and rich Internet applications (RIA). Test data management releases QA and developers from reliance on database administrators (DBAs) to manage functional and non-functional test data.
KEY FINDINGS
Strengths: Lifecycle traceability of requirements through testing and defect resolution. Scalability from single projects to multiple enterprise-wide projects with thousands of users. Workflow automation and process guidance for mature ALM. Weaknesses: Currently there is no alignment of ALM metadata with HPs Information Technology Infrastructure Library (ITIL) v3 and configuration management database (CMDB) tools. Limited integration between HPs ALM, and Project and Portfolio Management (PPM) solutions. Key Facts:
i All the tools in the HP ALM portfolio have been brought up to version 11 to
align versioning.
i HP will continue to collaborate with third parties for Agile project management
tooling.
OVUM VIEW
ALM as a mature approach to enterprise-level software development is needed today more than ever, as IT projects are expanding their reach and sophistication through service-oriented architecture (SOA), the web, and RIA, and most recently a new wave of mobile apps. In addition, year-on-year, engineered products contain an exponentially higher amount of embedded software. ALM tools are essential for managing this complexity. Quality and productivity initiatives have also transformed the software development space, with Agile and Lean process and methodology adoption reaching the mainstream. The impact that this has had on ALM relates to how the tools can support the specific metaphors and styles of working used in Agile and Lean practices. Finally, cloud computing has also influenced the ALM market, with software-as-a-service (SaaS)-hosting and delivery options helping to lower the cost of ALM.
CHAPTER 8: HP HP APPLICATION LIFECYCLE MANAGEMENT
183
HP has responded to these changes in the market with a revamp of its ALM solution, improving integration, adding new organic solutions, and also making acquisitions where the fit is good. As a result, the HP ALM solution is now based on a platform with integration enhancements between point tools which include: a new common repository that assists re-use across projects; new tooling to link ALM with third-party business process management BPM) tools to improve the requirements gathering process; and a new project planning and tracking tool for cross-product tracking for state of progress and deep dive into particular tasks, for example examining if requirements changes have affected a task, and including tracking across multiple projects as part of an enterprise release. There is the new homegrown Agile project management solution, and for QA there is TruClient, a new script-less Web 2.0 performance validation tool, and a test data management tool. The application security coverage has recently been reinforced by the Fortify acquisition, which Ovum has discussed in a companion Technology Audit: HP Application Security Center (September 2010). With so many new products and enhancements to HP ALM 11, this Technology Audit can only provide a high level view of the solution. Overall, Ovum believes that HP is paying serious attention to its Business Technology Optimization vision, and that the HP ALM 11 launch is a major effort to place HP at the forefront of ALM system capability and innovation.
Recommendations
Large enterprise Supporting the needs of large-scale projects, HP ALM 11 will cover the spectrum of project types, and on top of HPs key strengths in QA, with enhanced coverage of web services, RIA, and integrated security testing the solution offers end-to-end lifecycle coverage. In addition, the solution will support the range of development processes found in a large organization. SMEs practicing Agile development HP ALM 11 marks the first release of HPs homegrown Agile project management solution. However, this is in an early release, and the competition is much tougher for the Agile ALM tools serving the small- and medium-sized enterprise (SME) market. Which ALM solution to go for will depend on business growth predictions and the level of long-term investment that is appropriate.

HP has, in the past, offered a powerful set of enterprise solutions in ALM, but with a few code-centricrelated gaps that required partnering to create the end-to-end benefits of an ALM approach to software development. The new HP ALM offering changes that, with many new products and improved platform integration across the point tools that, in Ovums opinion, place HP in the top tier of enterprise-strength ALM systems. The solution provides core ALM features (see Figure 45) and also integration points at the extreme ends of the ALM system spectrum. There are Project Portfolio Management, BPM, and SOA governance integrations with HP and third-party solutions, as well as integration points at the DevOps end. DevOps is the new term for development-operations activity that embraces Agile values to break down traditional silos and barriers, and encourages a collaborative approach to fast and reliable application deployment. HP ALM 11 is suitable for managing different project types (from Agile to traditional/waterfall), and languages (such as Microsoft .NET, Java, and Oracle), as well as SAP enterprise resource planning (ERP) development environments. The solutions management integration capabilities, enhanced by a new common platform repository, include: Planning Initial planning of a project, to ongoing project and cross-project tracking and reporting. Requirements Requirements definition and management. Development management Integration to developer tools. QA Test planning and management, test execution/delivery, and defect management. DevOps Hand-off to application delivery/deployment. The QA functionality within the HP ALM offering is also available as separate products: HP QC 11, and HP Performance Center 11.
184
The HP QC 11 QA product has been a market leader for HP since its acquisition of Mercury. It provides a complete quality management platform that scales from single to multiple enterprise-scale projects. It includes the requirements management segment of the HP ALM solution, as well as release and cycle management, test management, defects management, and reporting. The version 11 release features a new product, HP Sprinter, which connects to QC and is a manual testing tool for exploratory testing and user acceptance testing. The graphical user interface (GUI) of the application under test appears within an environment with annotation tools so that comments can be placed directly on the screen, helping to communicate defects and issues directly to developers. Defects are also easily logged without entering QC. Data can be inserted into the application under test using a data-injection GUI. The tester actions are recorded and can be replayed as test scripts. Sprinter also allows multiple environments to be tested concurrently. Also new in this release is HP Unified Functional Testing 11, which integrates the HP Functional Testing (QuickTest Professional, QTP) and HP Service Test (integration testing) tools as one unified operation, with a new console and unified reporting capability. QTP has enhanced Web 2.0 and RIA testing, covering Adobe Flex, Microsoft Silverlight, and Windows Presentation Foundation, as well as Ajax toolkits like dojo, Yahoo! UI library, and more. There is also a free Web 2.0 extensibility accelerator that provides an integrated development environment (IDE) for building QTP extensions. A key tool launch is Service Test (ST) 11, which integrates with QTP and offers a visual test designer IDE built on Microsoft .NET, as well as having data-driven testing and an extensible framework. ST has bridged developer and QA testing by unifying activities in one tool. ST can manage composite application dependencies and application components. HP Performance Center 11 enables performance testing and validation across Web 2.0/RIA to business and legacy applications and, as part of HP ALM, enables end-to-end traceability of requirements, tests, defects, and resolutions. The tool can scale from individual projects to being a foundation for an enterprise performance testing center of excellence. It will emulate hundreds or thousands of distributed concurrent users, applying workloads to virtually any environment, identifying performance bottlenecks, stress testing end-to-end transactions, and helping in identifying scalability issues. HP Performance Center can manage multiple instances of HP LoadRunner, the performance testing tool within Performance Center. Performance Center is able to track non-functional performance requirements, their status, and their performance coverage, as well as manage performance defects through to resolution. There is traceability between performance defects and which performance requirements are affected. There is also a drag-and-drop infrastructure topology designer for reflecting the environment being tested. A new addition to Performance Center is HP LoadRunner TruClient. Where writing scripts is a challenge, particularly for Web 2.0 applications, TruClient embeds a scripting engine within the browser interface, allowing data to be parameterized while scripting transactions and logic, and also automatically making scripting suggestions. It supports Ajax applications and offers an interactive and dynamic scripting environment. With its supporting scripting engine, TruClient allows for a flexible tradeoff between automated scripting and traditional programming of scripts. It allows for a faster test cycle time and better re-use of scripts. A new HP Test Data Management (TDM) tool helps accelerate test data preparation by as much as 50%. The tool pipeline starts with applying rules to the data source and extracting required data, with masks where appropriate. The data, which can be represented as XML or flat file structures, are queried by HP ALM or the other key HP tools (e.g. QTP or LoadRunner), and the data are uploaded to the target environment for use in testing, loading to database, and so on. A major benefit of such a data tool is that it releases the relevant users (QA and developers) from requesting the data extraction from DBAs, therefore speeding up the testing cycle. HP QC Agile Accelerator is HPs organically built Agile project management tool, built on the HP QC 10 platform and supporting the paradigms of Agile development (user stories, Agile process, continuous testing, and Agile iterative development). The tool follows the Scrum and Extreme programming methodologies in language (Sprints) and constructs (Sprint, release, and product backlogs). The tool has built-in Agile charts (e.g. burn up, burn down), a dashboard, task estimation, planning, actual time spent, and reports viewable in Excel. This is the first release of the product and many further enhancements are in the pipeline.
185
HP ALM 11.00 is a Unified Platform

Designed to Help Master the End-to-End Application L ifecycle
Integrations into Strategy and Planning (PPM, Gov)
Application Lifecycle Management

Project Planning and Tracking Enterprise Release Management
Integrations into Run (Deployment, APM, ITSM)
Requirements Management
Requirements Definition Requirements Management
Development Management
Developer Defect Secure Integrations Management Development
Quality Management
Functional Performance Security
ALM Foundation
Trace ability Process Standardization Reporting Extensibility
Unified for the Core Application Lifecycle, Integrated with the Complete Application Lifecycle
Figure 45: HP ALM 11 Solution
Source: HP
The ALM platform is built on client-server architecture with a Microsoft .NET client that can be installed by users without needing local machine admin privileges. There is a system admin module for configuring domains, projects, users, and access privileges. The server side can be configured to run in cluster mode, and supports high availability and load balancing across multiple application servers. The extensible workflow engine allows creation of processes to follow methodologies, and also keeps track of project milestones and QA policies. An intelligent repository underpins the platform, providing version control and auditable tracking of stored entities.
SOLUTION ANALYSIS
The requirements definition (RD)/requirements management (RM) space is a new focus for HP. Its offering in this area is available as part of the HP QC solution and, in turn, as part of HP ALM. The recognition for requirements-driven development is gaining momentum as the importance of getting requirements done right to avoid costly waste further downstream becomes clear. Organizations are also keen to avoid falling into the waterfall process trap of having too much early requirements activity. Allowing for a requirements change management process to ensure that the project is steered to meet business needs is a goal that represents the new RD/RM paradigm. HP has answered this challenge well in Ovums opinion, and HP ALM 11 represents a strengthening of HPs leadership bid in this space. Key functionality includes testing requirements, requirement types, requirement releases and cycles, three-way traceability, requirements-based quality management, and support for rich text, versioning, base-lining, and re-use. Requirements can be searched, and the team can work in the familiar HP QC environment while managing requirements. Each requirement type (functional, business, testing, nonfunctional, etc.) has a customizable rich-text template. The integration with business process modeling tools allows business analysts to conduct the RD stage in familiar tools (e.g. Microsoft Visio, including importing diagrams from Vision).
186
The most up to date enhancements include in-built collaboration with comments linked to requirements at the point of relevance, and imported models with which to view the relevant business process and use case. Story boards such as web page navigation sequences can also be captured and linked to requirements.
Quality management
Quality management is HPs key strength based on HP QC, and also its best opportunity to grow its existing customer base into its new HP ALM solution. In Ovums opinion, the new tools that HP has added in release 11 will ensure that this flagship solution remains relevant in an increasingly competitive space. The use of innovative automation in testing scripts is paramount if HP is to retain its leadership in QA, and the new tools described above address this necessary evolution. The central role of HP QC and HP Performance Center in HP ALM ensure that existing customers can work in familiar work environments while benefiting from the extended ALM capability. The new integration built around the HP ALM platform ensures that visibility and traceability of requirements, tests, defects, fixes, and deployments are available for project monitoring and effective team collaboration.
Project management
The new release is able to support Agile practices at a fundamental level, as it contains HPs first version (labeled version 11 for synchronization reasons) of its new Agile project management solution. With further enhancements in the pipeline, customers can benefit today from the product for Agile support, but need to be clear about their immediate needs. HP is covering all options by also partnering with CollabNet and Tasktop for Agile project management; this offers some flexibility, although separate licenses are required. The integration between HP ALM and HP PPM is another work item in the pipeline. Having said that, the Tasktop integration can achieve this combination by acting as a hub for HP ALM and PPM solutions.
Application security has become a key focus today in application development and security circles, driven by new acts and regulations responding to the increase in web-related crime. Web applications have increased the attack surface, offering doors in the firewall which can be manipulated at a code level. HPs security approach can help to bridge the often siloed worlds of development and security to ensure the best front to meet the challenges posed by the most difficult to detect cyber intrusions. HP Application Security Center tools support that process, offering a comprehensive security solution set. This solution is to be integrated into HP ALM in future editions.

The target audience for HP ALM 11 is the vice president of application development and the senior leadership team supporting that role. The solution is also aimed at enterprise manager service providers (MSPs). The issue is to replace outmoded legacy processes and systems for managing software development, particularly in the face of the transformational changes impacting upon IT, which range from Agile processes and practices, to cloud computing and globally distributed teams. The key challenges that organizations face in their application delivery are: Visibility of the activity of distributed teams to enable delivery managers to keep track of projects. Automation to support re-use of artifacts, and reduce the burden of manual processes in traceability of requirements, test execution results, QA checks, and hand-offs across departments such across development operations. Due to the complex nature of Web 2.0 and RIA, there is a need to ensure that QA is keeping pace with modern applications. They need to manage change effectively without losing time and without urgent and priority change requests falling foul of antiquated processes. Modern web applications are always on, and are continually being refreshed/renovated to keep track of the pace of change.
187
HP ALM is suitable for use across all vertical industries, although HP has particular strengths in federal, state and local government, telecommunications and media solutions, financial services, process and discrete manufacturing, hospitality, transportation, insurance, and retail. HPs route to market is a mix of direct, partner, and some independent software vendors (ISVs). Key implementation and distribution partners are HP Enterprise Services, Capgemini, Accenture, and MSPs like Infosys and Wipro. Key technology partnerships include SAP, Oracle, CollabNet for developer management, and iTKO for the resale of iTKO LISA Virtualize. There are three licensing options: perpetual licensing, depending on global, area, or site footprint; user licenses, with the term available on an annual basis (or for performance testing there are virtual user days; and SaaS options, covering SaaS delivery, hosted software, and hosted testing services. Project values for deployments of this solution are as follows: Entry level ($50,000100,000) 90% software license and 10% quick start services. Average ($150,000500,000) 75% software licenses and 25% services typically covering quick start, upgrade, and some methodology and organizational support. Largest ($1m+) 60% software licenses often an enterprise license agreement and 40% services such as best practices, center of excellence services, and methodology and organizational support. The product development roadmap that HP can share includes: out-of-the-box templates and packages for best practices around delivery strategies such as Agile, updated functionality for quality project management, and third-party integrations. There is also a three-year roadmap published for integrations across various HP products covering HP ALM, HP PPM, and HP Application Security Center.
IMPLEMENTATION
The time taken to implement the solution varies based on number of users: small deployments of 50 users will typically take one week, need one full-time equivalent (FTE), and require ALM product and administration expertise. A medium deployment for 250 users will be similar to small, but will need about two weeks. A large enterprise deployment for over 250 users will take upwards of two to four weeks, and will need two FTEs with additional skills in system and database integration. HP offers extensive QuickStart implementation programs to help get users productive on HP software as rapidly as possible. Migration services are also available to help move customers from on-premise to HP SaaS, and SaaS adoption services are available for new customers. Customer support is available in three levels: foundation 9/5, foundation 24/7, and premier support. The HP ALM solution is available on Windows Server, Linux Red Hat 5.4 64-bit, Solaris 10 64-bit, and HP/UX 11.31 v3 (64-bit Itanium). The solution requires Java application servers, a database (e.g. Oracle, Microsoft SQL Server), and a web server (e.g. JBoss, Apache, Microsoft IIS).
DEPLOYMENT EXAMPLES
Case 1
HP was chosen by a leading global airline that provides services to 348 destinations in 64 countries and across six continents. The objective was to support the application lifecycle and drive ALM process improvements across its website, kiosks, and mobiles without adding staff. The companys approach was to implement HP software tools (HP QC and HP Functional Testing) to enable automated testing and timelier, more useful insights into defect, security, and performance issues. Results: Testing times were reduced by 52% and full regression of the company website was reduced from five days to 1.5 hours.
188
Testers can now devote 90% of their time to looking at new code, which is a higher-value task than rechecking old code. Automation enables test engineers to focus on new functionality while ensuring that current functions work as before. Security reports provide actionable detail, and the solution suggests fixes, streamlining developer workflows. Benefits: Defects can be fixed earlier in development cycles, when they are less costly and troublesome to address. Defect information is more useful, and supports efforts to improve quality and development processes, resulting in reduced risk of applications issues. The improved ability to accommodate business fluctuations and improved employee morale help to stabilize the workforce.
Case 2
HP Performance Center was deployed by a Netherlands-based international financial services provider that offers retail banking, wholesale banking, asset management, leasing, and real estate services. The companys co-operative-based organizational structure means that it is business-critical to support a variety of regional IT environments while building trust around adoption of a more centralized enterprise model. The company leverages HP Performance Center software to ensure that its central IT-developed applications are stable and fully meet the co-operatives business needs. IT improvements: Improved ability to ensure that software meets specifications. Scripts can be re-used, saving script development time. Click-and-script capability shortens SAP test script development from as much as four hours to less than 60 minutes. Diagnostics and reporting maximize the value of QA to the rest of the IT department. ERP memory leaks can be diagnosed. Business benefits: Applications better support the banks business needs. Automated, repeatable scripts are less expensive than paying staff to perform manual tests. Scripts can automate tests that would need hundreds of employees to be performed manually. Software applications are more stable and hardware is better sized to fit business needs.
Case 3
A global finance company specializing in real estate finance, automotive finance, commercial finance, insurance, and online banking needed to maintain uptime for a critical sales tool during periods of maximum user demand. To address this need, it deployed HP Performance Center to increase testing flexibility, improve IT efficiency, and reduce costs associated with application downtime, resulting in a four-year saving of over $21m. Benefits: Improved testing flexibility and tester productivity resulted in a $3m saving. Reduced downtime resulted in a $2.4m saving. The return on investment was 240% and payback was achieved 5.6 months after deployment. There has been a 125% increase in the number of tests conducted per week. Errors have been reduced by 44%, and monthly unscheduled downtime has been reduced by 33%.
189
Hewlett-Packard Company 3000 Hanover Street Palo Alto CA 94304-1185 USA UK Tel: +1 (650) 857 1501 Fax: +1 (650) 857 5518 www.hp.com
HP Amen Corner Cain Road, Bracknell Berkshire, RG12 1HN Tel: 0870 013 0790 (+44 207 949 0300) Fax: +44 (0)1344 36 3344
190
Incorporating
OVUM Butler Group
IBM RATIONAL: IBM Rational ALM Solution
WWW.OVUM.COM
TECHNOLOGY AUDIT
IBM Rational
IBM Rational ALM Solution
SUMMARY IMPACT
IBM Rational is well on its way to making a generational transition of its application lifecycle products to the Jazz architecture. Like many application lifecycle management (ALM) vendors, IBM Rational has amassed its product portfolio through acquisitions, with the highlight being the addition of Telelogic in a deal that was closed in 2009. As a result of its numerous acquisitions, IBM Rational has the widest portfolio of products in the ALM space, including several that overlap. Thanks to the Telelogic acquisition, IBM Rational is well positioned to target the fast growing embedded software segment, which entails managing the development of systems of systems. With the Jazz products, IBM Rational is targeting the enterprise Agile space. IBM Rationals cloud product strategy leverages the token licensing scheme that Telelogic originated prior to its acquisition by IBM.
KEY FINDINGS
Strengths: IBM Rational has the broadest portfolio of any ALM vendor. The Jazz initiative puts IBM Rational in select company, as few other ALM providers have developed common integration frameworks that interlink tools, artifacts, and processes. Rich functionality for supporting embedded software development, one of the two fastest growing segments in the ALM market. Weaknesses: Existing Rational customers can leverage the Jazz technology with Open Services for Lifecycle Collaboration (OSLC) integrations for their existing projects, but if they want to take full advantage of the Jazz platform, they may face significant migration or systems integration costs. Broad array of customization choices often requires professional services. Key Facts:
i Along with introducing new Jazz-compatible tools, IBM is phasing in the Jazz
architecture for the bulk of its toolset.
i New Jazz-native offerings, including Rational Requirements Composer (RRC),

Rational Quality Manager, and Rational Team Concert (RTC), are designed for supporting Agile processes.
CHAPTER 8: IBM RATIONAL IBM RATIONAL ALM SOLUTION
193
OVUM VIEW
Rational Software was a pioneer in ALM prior to its acquisition by IBM in late 2002. It was the first software development tools player to acquire tools spanning the major phases of managing application development and bundle them into a suite (the exception was the integrated development environment (IDE), which did not come until after the IBM acquisition). However, a major shortcoming was that the tools lacked a common architecture and were linked only through point integrations. In all fairness, Rational was hardly alone; no ALM vendor at the time had such integration. The Jazz initiative, announced in 2007, provided the long-missing integration engine. This initiative consists of a tooling integration architecture supported by RESTful-based services, a set of products that are either Jazz-compliant (e.g. incorporate the Jazz architecture natively in the tool), or Jazzcompliant (those supporting the OSLC interface that expose and consume services from other tooling supporting Jazz). Additionally, IBM Rational has established a community for developing tool and process integrations using the OSLC interface. Operating on a broad range of distributed, mainframe, and IBM power platforms, the Jazz platform relies on a RESTful-based services-oriented architecture (SOA) that is built on the design assumption that most IT software development organizations are highly decentralized; have multiple, often incompatible tools; and cannot work in environments where the management of processes and artifacts is centralized. It has opened a community site, open-services.net, for encouraging the royalty-free development of third-party integrations between any ALM tools. ALM tools that implement OSLC interfaces need not be coded to adopt language or platform-specific application programming interfaces (APIs), enabling loosely coupled integrations that are more resilient to product release changes. Conceivably, using OSLC, you could integrate any set of tools where interfaces have been developed, without using the Jazz repository server itself. This places IBM in select company in the ALM sector, where formal or de facto multi-vendor integration frameworks have never taken root. To date, Microsoft and Serena are the only tools providers that have developed generalized integration frameworks for their ALM offerings; at the other extreme, MKS has taken an opposite approach, developing a single, unified tool. IBM has introduced several new-generation tools that are native to the Jazz platform, including Rational Team Concert, a collaboration hub for coordinating ALM workflow; Rational Requirements Composer, for requirements definition; and Rational Quality Management, for quality management. IBM Rational is Jazz-enabling its legacy products incrementally, writing new OSLC-compliant integrations and, in some cases, developing next-generation versions of those products that become Jazz native. Given the maturity and unique architectures of existing products such as DOORS, Requisite Pro, or ClearCase, this process will take several years, and some IBM Rational legacy products may not be migrated natively to the Jazz platform at all. The rollout of Jazz was the first major transformational event at IBM Rational; the second was the acquisition of Telelogic. The Telelogic acquisition opened a new market for the development of large, highly complex systems that Ovum expects will act as IBM Rationals major revenue growth engine for the next three to five years. This market often referred to as systems of systems because of the likelihood that embedded software is developed for systems that interact with or are dependent on other systems is one of two growth markets for ALM that Ovum has identified (the other is enterprise Agile development). Telelogic products, such as DOORS for requirements management and Rhapsody for model-driven development, are designed for enterprises that build highly complex systems (often as part of engineered products such as automobiles, commercial and military aerospace systems, appliances, or high technology electronics products), along with infrastructure management and industrial process control systems. Both of these products are currently Jazz-enabled for supporting integrations with other tools such as RTC. The growth potential of this segment is attributable to the increasing amount of intelligence that is built into products (for instance, todays automobiles contains 80100 modular, interdependent computers that control all aspects of operation), and the growing demand for sophisticated management and control of civil infrastructure. As a result, IBM Rational software tools are very much part of IBMs Smarter Products and Smarter Cities components of its Smarter Planet market positioning.
194
Recommendations
Who should deploy? Large enterprise developments, spread across multiple teams, various operating system (OS) platforms, and geographies; organizations that design engineered products with intelligent control; and organizations that design software for the management and control of civil or industrial infrastructure. Who should not deploy? Teams with modest-scale software development with little or no process governance requirements. Such organizations will find IBM Rationals tools too rich and complex.

IBM Rational has a broad array of tools covering the software development lifecycle, from requirements management to project planning and reporting; quality management (including application security) and test lab provisioning; change and configuration management; and build/release management. Along with integrations to Tivoli products, IBM Rational provides support for DevOps collaboration, which extends management to the production phase of the application lifecycle. A partial listing of IBM Rationals tools is shown in Table 3. CATEGORY Requirements management PRODUCT Rational DOORS (req. mgmt for complex systems) Rational Requirements Composer (req. definition) Rational RequisitePro (req. mgmt for enterprise apps) Quality management Rational Quality Manager (test mgmt) Rational Test Lab Manager (test provisioning) Application security Project management Rational AppScan (several editions available) Rational Team Concert (planning) Rational Insight (reporting) Rational Publishing Engine Change and configuration management Rational Team Concert (work items and software change and configuration management (SCCM)) Rational ClearQuest (issue/defect mgmt) Rational ClearCase (SCCM) Rational Change (issue/defect mgmt) Rational Synergy (SCCM) Automation and build management Rational Team Concert (build) Rational Build Forge (build and release mgmt) VERSION 9.3 2.0 7.1 2.0 2.0 8.0 3.0 1.0 1.1 3.0 7.1 7.1 5.2 7.1 3.0 7.1.2
Source: IBM Rational
Table 3: Select IBM Rational products
Note: This Technology Audit presents a tool- rather than solution-centric view of IBM Rationals portfolio. ALM vendors are continuing to sell point tools. However, Ovums view is that such approaches are counter to the goals of ALM, which are about managing the application lifecycle, not optimizing the execution of individual tasks.
195
Ovum believes that solution-centric approaches that integrate functionality across tooling bundles are essential to aligning ALM with the business. Ovums analysis of how IBM Rational delivers ALM business solutions is found in the report Application lifecycle process integration: IBM Rational and HP (December 2009, OVUM051651). That analysis is currently being updated and will be available with the upcoming edition of the ALM Decision Matrix and Technology Evaluation report to be published in early 2011. In the meantime, use the information in this Technology Audit as general background about some of the key product building blocks that make up IBM Rationals ALM solutions.
SOLUTION ANALYSIS
IBM Rational offers multiple products that were developed and/or acquired by Rational and Telelogic prior its acquisition by IBM plus newer products that were developed natively for the Jazz platform, including Rational RequisitePro, DOORS, and Requirements Composer.
RequisitePro
As one of the products that pre-IBM Rational acquired as it built up its ALM portfolio, RequisitePro is a hybrid product that uses a combination of relational databases and/or tagged Microsoft Word documents to store, manage, and apply version control to requirements. RequisitePro has traditionally been used for developing enterprise applications, especially when doing use-case-driven development.
DOORS
DOORS, an object-oriented tool developed by Telelogic, is an extremely rich tool for managing requirements for highly complex systems. Although primarily used for managing requirements for software development, many product companies have also used DOORS for managing requirements for product engineering (encompassing software, mechanical, and electrical requirements).
Rational Requirements Composer

One of the new Jazz-native tools, RRC, is used primarily for bringing stakeholders into the requirements process, defining requirements and generating use cases that can be fed into test management, task management, and full-blown requirements management tools. It allows analysts and user experience professionals to: Create storyboards to explore design alternatives with stakeholders and validate requirements. Through a partnership with third-party firm iRise, provide richer visualizations that simulate the look, feel, and navigation of actual applications. Generate process diagrams with a subset of Business Process Modeling Notation (BPMN) (RRC is not BPMN 2.0-compliant, and therefore for now is not designed to interchange diagrams with other business process management (BPM) modeling tools). Enter and track comments and annotations for team-based requirements collaboration. IBMs major challenge has been defining a target market for RRC and communicating its value proposition to an installed base that is already heavily invested in DOORS or RequisitePro. In 2010, IBM announced plans to upgrade RRC into a full-blown next generation requirements tool for RequisitePro customers, and has delivered Beta code on jazz.net that shows improvements over RequisitePro in both the modeling of relationships among requirements, and the use of requirements across development, test, and project management. To improve collaboration capabilities with a web-oriented platform, IBM Rational should consider including these requirements capabilities in its emerging BlueWorks BPM platform, which was designed for greater accessibility. Admittedly, BPM and ALM have appealed to different audiences, but leveraging commonality among tools is more a marketing than a technical challenge. Furthermore, the line between authoring processes in BPM and ALM is getting hazier.
196
Quality management
Rational Quality Manager (RQM) provides test planning with the ability to assess quality objectives against the objectives stated in the overall test plan. Traceability between requirements and test plans is initially performed manually within the DOORS or RequisitePro requirements management tools, and then synchronized with RQM (unlike HP or MKS, requirements and test artifacts are not contained in the same repository). Once the relationship between requirements and test plan is established, updates or the generation of test cases can be automated in RQM. Ultimately, software quality is not simply about eliminating defects during the development process, but also about ensuring that the software meets service level and functional expectations while in production. This is one of the pillars of Microsoft Software Assurance, a full lifecycle approach to software quality that is Ovums vision for software quality assurance. RQM does not yet address the management of test strategy with the production environment, but it gets part of the way there. By utilizing service-level agreements (SLAs) in RQM, a synthetic performance test can be run with Rational Performance Tester (RPT) that interfaces with Tivolis ITCAM to assess the degree of health for exceeding or not meeting the defined SLAs. Additionally, asset inventories can be derived by using the Tivoli Application Dependency Discovery Manager (TDDM) tool, allowing active provisioning of the complete test and production environments through Tivoli Provisioning Manager (TPM) to enable full end-to-end quality management.
Related to software quality, security and compliance is a relatively new area for ALM tool providers. IBM (along with HP) has gained a head start with the inclusion of static (e.g. source code scanning, or white box testing) and dynamic (ethical hacking, or black box testing) testing, thanks to the Watchfire and Ounce Labs acquisitions. IBM has brought the products together under several editions including: AppScan Source Provides static analysis within the development environment for development teams. AppScan Standard and Express Provides desktop solutions for dynamic code analysis aimed at individual security practitioners. AppScan Enterprise Offers a multi-user solution supporting concurrent, multi-stream dynamic security testing, and centralized vulnerability and compliance reporting. AppScan Build Integrates security testing into build management workflow, and is the only AppScan product that integrates static and dynamic security testing. AppScan Tester Integrates dynamic testing into the quality assurance process. Although IBM (along with HP) has taken the lead in weaving security testing into its ALM portfolio, gaps in the solutions remain. Specifically, IBM should add policy and risk management capabilities that analyze the importance of what is or should be security tested.
Project management
Unlike rivals such as HP, CA, or Compuware, IBM Rational separates project and portfolio management into separate products. The latest release of the planning component, present in RTC, allows organizations to create development plans and schedules using foundational project management functionality such as dependency and constraint management. These basic project scheduling principles are applied directly to the tasks and work items that developers are working on to produce a realistic schedule. Organizations can also track risks and issues with their respective response packages, manage resources, and track status by project. Ovum expects IBM Rational to be fleshing out further program management capabilities in the future in its application lifecycle planning-oriented products. Portfolio management is handled by Focal Point (a product that originated from Telelogic), which uses a unique pair-wise, decision tree approach, whereby users constantly evaluate alternative investment paths at each decision point regarding the scope and direction of a project or product initiative. The latter point is significant; while this methodology may be too rich for IT organizations that are looking for more conventional project portfolio management (PPM) tools that address resource management issues for projects or programs, it lends itself well to the engineering companies in the Telelogic base that design and make products.
197

This is another area in which IBM Rational offers multiple products that came together through the Rational and Telelogic acquisitions as well as from the recent development of a product native to the Jazz platform. Offerings encompass: Rational Team Concert A newer product developed natively for the Jazz platform, RTC provides work item management that is driven, either by its own internal SCCM engine and/or integrations with others including ClearCase, Subversion, Git, and Synergy. It can tie in with other SCCM tools using basic REST interfaces to link work items to source code commits. RTC actually provides a superset of SCCM by supporting Agile planning, work item management, build management, tracking project health, and source code management. Release 3.0, which became available in late 2010, added more formalized planning capabilities, including risk management. RTC can be implemented in either a unified or a modular manner on distributed, mainframe, and Power (i, AIX, Linux) platforms. Rational ClearCase One of the tools that was acquired and developed by pre-IBM Rational, ClearCase provides SCCM. Rational ClearQuest One of the tools that was acquired and developed by pre-IBM Rational, ClearQuest performs issue and defect tracking. Rational Synergy One of the tools that was acquired and developed by pre-IBM Telelogic, Synergy performs SCCM. Rational Change One of the tools that was acquired and developed by pre-IBM Telelogic, Change provides a planning platform for aligning technology investments and providing traceability for code changes. IBM Rational is in the midst of a generational change, as it offers a newer SCCM engine alongside multiple legacy products. The challenge for IBM Rational will be convincing its existing customer base that they should add RTC to supplement their existing ClearCase and Change/Synergy deployments. Indeed, few customers like having to pay for additional tools when they already have one that does the job, and few, if any customers are likely to carry out a full replacement. IBM is promoting RTC as a single collaborative change management and planning platform that wraps around both ClearCase and Synergy, providing a single alternative to existing tools. IBM Rational competes with numerous open source alternatives, as well as Agile development shops preference for low-cost/no-cost lightweight tooling. However, regardless of the degree to which Agile gains acceptance, most large enterprises will use a range of development methodologies, as there will always be certain projects that lend themselves to traditional methodologies, like waterfall. Those are the shops to which the combination of Jazz and existing IBM Rational tools will have the best value proposition.
Automation and build management

Build Forge is IBM Rationals offering that addresses the management of software releases. Specifically, it validates, schedules, and manages resources and dependencies for managing software builds packaging, and staging activities. It is not focused on software compilation scripting tools such as the popular ANT or Maven, although it can be used to coordinate the usage of those tools, particularly in the context of more complex or multi-platform situations. Build Forge is focused on providing advanced distributed tool automation across the release management domain and leaves the human workflow coordination to SCCM systems. There are some provisions that support DevOps collaboration. For instance, using the Rational Automation Framework for WebSphere option, which provides deployment and configuration management capabilities for a variety of IBM WebSphere products, Build Forge can integrate with external deployment tools to coordinate with provisioning systems such as from IBM Tivoli or other thirdparty vendors. These integrations must be manually created. The DevOps grassroots movement, in conjunction with IT Infrastructure Library (ITIL) initiatives from the IT operations side, could create demand for IBM to expand Build Forge integrations with IT service management automation systems.
198

Ovum has identified the following opportunities from its 2011 Trends to Watch report for ALM: With Agile now in the mainstream, new pragmatic approaches focused around mixing and matching the right methodologies will emerge and become supported by ALM tool vendors. Demand for cloud deployment will prompt software-as-a-service (SaaS)-based ALM offerings, not only from new entrants, but from established players as well. With the explosion in mobile application development started by Apples iPhone/iPad and Googles Android platform, smart mobile will become a major target for enterprise applications. Security testing is entering the ALM mainstream. Embedded software development for engineered products is on an exponential growth curve and represents a lucrative market for the few ALM providers that have chosen to target it. In 2010, DevOps entered the awareness phase in the ALM market, and 2011 will see automated tooling targeting the needs of DevOps and Agile release management in particular.

Traditionally, development teams that have adopted Agile methodologies have tended to travel light when it comes to tooling; there has long been a prejudice against rich, functional tools because of the perception that such capabilities add process complexity. IBM and other incumbent ALM vendors such as HP, Serena, and Micro Focus which have rich offerings have suffered due to these perceptions. Furthermore, as Agile methodologies started with small, often isolated teams with limited budgets, they had to compete against low-cost or no-cost open source tooling. As IT organizations gained more experience with Agile, their next logical step was to adopt planning tools from players such as Rally or VersionOne, which specialized in Agile planning, rather than working with providers like IBM, which offered broader multi-purpose tooling. For IBM Rational, the Agile opportunity lies with development organizations that have with multiple teams that mix and match methodologies; some projects will use Agile methodologies, while others may use more traditional iterative or waterfall methodologies. For instance, while IBM Rational products like DOORS or RequisitePro may be too rich for Agile projects, the newer Jazz-native offerings including RTC, RRC, and RQM were designed to support lightweight processes and could be deployed as part of a hybrid strategy. With version 3.0, IBM RTC is now designed to support Agile, iterative, or waterfall methodologies out of the box, and so can exploit the opportunity to provide blended tooling for large development organizations that use multiple methodologies. The sweet spot for IBM Rational is clearly in the systems of systems space, where there is significant opportunity and an immature market. Clearly, demand for product-focused requirements, quality, and portfolio management is growing.
Cloud
While IBM Rational previously lagged specialized providers such as CollabNet and Rally in providing access to ALM tooling in the cloud, in 2010 the vendor embraced the cloud with a vengeance. IBM Rationals initial foray was hosted AppScan security testing and Rational Policy Manager as on-demand SaaS offerings. IBM Rational has since followed up with software delivery services for the cloud, comprising RTC, RRC, RQM, Rational Build Forge, and Rational Asset Manager hosted by IBM or business partners on IBM or third-party infrastructure, such as Amazon EC2. This is part of a multi-faceted cloud strategy from IBM Software Group to offer public and private cloud services that can be designed, configured, and/or hosted by IBM, third-party partners, or the customer. For instance, partner CloudOne offers RTC as a service. Private cloud offerings leverage the IBM CloudBurst appliance, which provisions integrated service management capabilities across hardware, middleware, applications, and tools.
199
IBM Rational is applying subscription pricing based on the token models originated at Telelogic before the IBM acquisition. Tokens provide a tiered system for pricing of the use of different software tools; for instance, the number of tokens for a complex enterprise requirements management tool such as Telelogic will be greater than that for narrower entry such as AppScan Source (single user static code security scans).
Mobile
Mobile support is a work in progress. IBM Rational builds on core capabilities that are not targetspecific, such as linking with full lifecycle tools including DOORS, RTC, and RQM. At the modeling level, the Rhapsody UML and SysML modeling tool offers pre-configured API libraries for targeting Android clients and a development kit for creating libraries for other smart mobile platforms. The goal is to design software models that can generate code for multiple mobile client targets. At the development stage, the Rational Application Developer Java IDE provides accelerators (e.g. code assists, validation, emulation, etc.) that support Android and Research in Motion. With Apples recent strategy shift allowing third-party tools to compile non-native applications to iOS (e.g. iPad, iPhone), Ovum expects IBM Rational to add capabilities for deploying for that platform. However, in general, IBM Software needs to unify its mobile development and deployment product and services strategy. It has products strewn across its software brands, addressing areas such as mobileoptimized services for user interaction, model-driven voice development, analytics for mobile services, mobile enablement of business processes, and so on. On the development and deployment sides, it trails Sybase, which has had a head start with the unified development and deployment offerings under its Sybase 365 product umbrella.
Security testing
As mentioned above, IBM Rational is ahead of the curve when it comes to offering tools for static and dynamic security testing. However, these are only the first steps. IBM Rational needs to flesh out its solutions to incorporate risk management. Nonetheless, HP excluded, this is an area that most ALM rivals have yet to address.

IBM Rational is one of the few major ALM vendors that is seriously targeting this segment. Thanks to its acquisition of Telelogic, IBM Rational has taken a forward position targeting this segment, especially with DOORS, Focal Point, Rhapsody for SysML modeling, and other products. As mentioned earlier, Ovum expects this to become Rationals primary growth engine over the next three to five years, thanks to the continued emergence of products for which software is not simply enhancing, but defining the product. With the growth of the role of embedded software comes the need to adopt a unified approach (and integrated tooling) that treats software engineering as a core pillar of product engineering. For more background information on the competitive factors in the embedded software development market, see Ovums report Software development in the product lifecycle (April 2010, OVUM052147).
DevOps
Most ALM and IT infrastructure management vendors are still figuring out how to package and incorporate capabilities that integrate the development of software with the management of IT services. For IBM, this involves a partnership between the Rational and Tivoli brands; so far, the result is that there are a number of point integrations between products such as Rational Asset Manager and Tivoli CCMDB for correlating changes in software assets to ITIL configuration items; Tivoli Service Request Manager incidents and ClearQuest defects; and Rational Performance Tester and Tivoli ITCAM for converting performance test scripts into synthetic performance tests in the operational environment. In addition, Rational is working on an OSLC-based integration between Tivoli Service Request Manager and RTC. These are useful first steps; IBM needs to weave the pieces into a more unified, process-driven approach.
IBM Rational markets to virtually every vertical industry and to companies and software development organizations of all sizes. Although Rationals reach extends to smaller firms and software teams with as few as five people, the sweet spot is large enterprises that have to manage development involving highly detailed, complex requirements, a mix of methodologies, and multi-platform or mainframe operations.
200
IBM Rational is also leveraging Telelogics presence in product engineering sectors such as aerospace, automotive, electronics, and infrastructure to expand its presence in the embedded software development space. The go to market strategy includes: Direct sales to named accounts (large accounts) in the Americas, northeast Europe, southwest Europe, Japan, and Greater China. Global systems integrators such as Accenture, many of which resell IBM Rational tools to large global accounts. Regional and local business partners that resell to select vertical industries or general business markets within select geographies. Regions such as Central and Eastern Europe, and the Middle East and Africa are served exclusively through business partners. IBM Rational works with more than 90 technology partners that extend the value of the Rational Software and Systems Delivery platform with complementary products and plug-ins under the Ready for Rational program. At present, 150 solutions are certified under this program and are listed by IBM Rational on its website. Additionally, IBM Rational has organized the Ensemble community, which currently includes 3,600 partners (of which 1,200 are IBM business partners), for building a business ecosystem around the Jazz architecture. Service providers and systems integrators often extend the Jazz platform with complementary offerings, or provide training. Examples include SVA, ARS, CM Logic LTD, and Prolifics in Europe the Middle East and Africa; and RocketGang, ATSC, Ascendant, Cloud One, ICON ATG, and Oxford in the US.
Licensing
IBM Rational provides a wide variety of licensing options. In addition to traditional perpetual licensing, IBM is promoting new forms of term licensing to provide more flexible alternatives. Specifically, it is featuring a token model first developed by Telelogic which provides a unit of value that can be applied to different IBM Rational software products. In essence, this is a weighted model that would charge more tokens for a broad enterprise suite such as DOORS than for a more specialized single-purpose tool such as AppScan Express. These tokens can be mixed and matched at will, so that customers can change their tool mixes based on need. IBM Rational also provides different tiers of support options, including enterprise passport agreements for large enterprises and transaction or incident-based programs for small to midsize businesses.
RIVALS
IBMs principal rivals include: HP Software Whose Quality Center offerings lead the market in quality management. HP is building an expanding portfolio leveraging Quality Center with project portfolio management (an area where IBM Rational offers separate products), IT financial management, security management, and other areas. Serena Which bills itself as the largest independent ALM provider (it is now solely focused on ALM, although the company is owned by a holding company with extensive holdings in other technology companies), and currently rivals IBM Rational with its own ALM integration framework. MKS IBM Rationals principal rival in the embedded software development space for product engineering companies. The companys sole product, MKS Integrity, covers SCCM, requirements management, quality management, and project management from a single code base and repository. IBM Rational also competes and coexists with product lifecycle management players such as PTC and Siemens, which are increasingly overlapping in areas such as requirements management and product portfolio management. CollabNet Which also targets large global development organizations with TeamForge, an ALM offering built around its open source Subversion SCCM system.
201
Micro Focus Has historically provided competition with mainframe tools. However, with the recent acquisitions of Borland and the QA tools of Compuware, Micro Focus is expected to become a broader competitor. Additionally, in the Agile arena, IBM Rational competes with smaller players focused on the space such as Rally and VersionOne, and in specialty markets (e.g. SCCM) with open source tools.
IMPLEMENTATION SCENARIOS
In most cases, IBM Rational tools replace manual processes or homegrown tooling.
PROFESSIONAL SERVICES
Because IBM Rational offers a broad array of products addressed to different audiences within IT and the business, there is no single scenario that describes implementation. However, IBM Rational estimates that the phased implementation of larger, enterprise ALM deployments varies from three to 16 months. As a large, diversified software provider, IBM Rational offers professional services geared towards training, mentoring, and implementation. Many of these services are also available from IBM Rational business partners.
DEPLOYMENT EXAMPLES
Major aerospace and defense contractor
A global supplier of systems and services to aircraft and engine manufacturers, airlines, and defense forces around the world faced the need to rationalize software development tools (there are over 70 tools in use across different development groups) and processes in the wake of major acquisitions. It began with a strategy of deploying RTC as the standard product engineering desktop to capture tasks and monitor their progress. The company is already using DOORS for requirements management, and is looking at the OSLC integration to DOORS that IBM Rational released in September 2010.
Healthcare financial software vendor

A software vendor that offers financial management technologies for hospitals and healthcare providers faced the need to manage source code that was scattered across multiple instances of Microsoft Visual SourceSafe. The company imported its existing SourceSafe assets into the IBM RTC environment, centralizing control over these assets and improving access to distributed teams. Developers can access source code and other development artifacts through their familiar IDEs, including Microsoft Visual Studio and the Eclipse client. Because all teams have access to the same artifacts, the company has been able to improve the efficiency of base-lining and quality management processes, governance of software development, and time to benefit. IBM Corporate Headquarters IBM Corporation 1 New Orchard Road Armonk, New York 10504-1722 US Tel: +1 (914) 499 1900 www-01.ibm.com/software/rational/
202
Incorporating
OVUM Butler Group
MICRO FOCUS: Micro Focus Caliber, Silk, and StarTeam Products 2010
WWW.OVUM.COM
TECHNOLOGY AUDIT
Micro Focus
Micro Focus Caliber, Silk, and StarTeam Products 2010
SUMMARY CATALYST
2010 was an in-between year for longtime mainframe tools provider Micro Focus, which was busy working on capitalizing on two 2009 acquisitions intended to boost its presence in the quality management space. These were the acquisitions of the last of Borland and the remaining developer test tools from Compuwares portfolio. In 2010, Micro Focus issued an important interim release of its quality assurance (QA) products. However, as of this writing, Micro Focus has yet to unveil its nextgeneration solution, fully leveraging the synergies of the Borland and Compuware assets with its existing tools catalog. This report covers the former Borland CaliberRM requirements management, Silk quality management, and StarTeam products. Prior to the acquisitions, Micro Focus was already expanding its product line toward application portfolio management. With the acquisitions, Micro Focus is preparing to compete head-on with HP Quality Center.
KEY FINDINGS
Strengths: Position of strength in legacy modernization. Borland and Compuware products offer promise of filling key gaps in application lifecycle management (ALM) product line. The Silk QA products have already begun leveraging capabilities from the former Compuware tools, such as visual scripting. Weaknesses: Lack of common integration and brand focus across the full Micro Focus product portfolio. Has not yet capitalized on the Borland and Compuware QA tools acquisitions with unified next-generation product. Limited Agile project management capabilities. Key Facts:
i Acquired QA products from Borland and Compuware have complementary

strengths.
OVUM VIEW
For many years, Micro Focus was a stealth ALM vendor because it was best known for its legacy modernization offerings. Although traditionally known for COBOL development and terminal emulation tools, it began its footsteps towards broader lifecycle offerings with the iSight product lines, which analyze application portfolios primarily through code health metrics and key financial performance indicators derived from those code metrics. However, it does not provide the extensive financial modeling or portfolio analysis that is part of what a full Application Project Portfolio Management (APPM) system offers.
CHAPTER 8: MICRO FOCUS MICRO FOCUS CALIBER, SILK, AND STARTEAM PRODUCTS 2010
205
Nonetheless, the trend at Micro Focus is offering a broader, enterprise-level ALM capability. Towards that end, in mid-2009 it announced plans to buy all of what remained of Borland, along with the QA tools of Compuware. In the long run, this could place Micro Focus in a strong position to target an underserved portion of the ALM market: IT organizations with substantial legacy application portfolios. The goal for the Borland and Compuware tool acquisitions was a single dramatic leapfrog movement to buy instant presence for a key gap in Micro Focuss product portfolio: providing full lifecycle quality and requirements management. Notably, the acquisition propels Micro Focus into territory covered by only a select few ALM vendors: automated testing. Although the domain of niche vendors, at the ALM level, it places Micro Focus in a league only with HP Quality Center, the market leader, and IBM Rational. Beyond the shock value, there was a certain logic to buying both the Borland and Compuware tools, as the former Compuware QAPartner and DevPartner family of products offered several capabilities such as risk-based test management, visual scripting, and Microsoft Visual Studio support not previously offered by the Borland Silk tools family. In the second half of 2010, Micro Focus: Released a refresh of the Silk product line that incorporated many of these Compuware features Added its first cloud support with SilkPerformer Cloudburst (for load testing) Launched Caliber RDM as the successor to Borland TeamDefine for requirements definition. The downside is that Micro Focus has not yet fully capitalized on its acquisition. More than a year after the Borland part of the acquisitions closed (Compuware closed earlier), Micro Focus has yet to fully realize its unified solution. It has found unifying products to be a more complex task than first imagined. Of course, that is a road that took players like IBM Rational nearly a decade to travel. In the refresh of Silk, Micro Focus took the first steps towards converging the product lines, but some of its best opportunities lie with its existing portfolio. A good example is with iSight; it could benefit from financial modeling functionality introduced in the short-lived Borland Management Solution (BMS) offering that Borland rolled out much too late in its existence. Additional opportunities lie with synergies to its extensive legacy modernization portfolio for which there is a significant installed base, and relative lack of competition from other ALM players. Ovum awaits how Micro Focus will respond to these opportunities. This report restricts itself to reviewing key acquired Borland products that will be part of the backbone of Micro Focuss future ALM offers. It should be considered an interim analysis. Ovum expects that Micro Focus will finally unveil the roadmap and first fruits of its next-generation solution in 2011, and will revisit this analysis at the appropriate time.
Recommendations
Ovums strategic recommendations include: Clients with short-term tactical needs Consider Silk, CaliberRM, CaliberRDM, and StarTeam tools, as they will be part of the core of Micro Focuss next-generation offering. Customers seeking strategic ALM solutions Reserve judgment until Micro Focus clarifies its next-generation roadmap. Regarding the suitability of Silk, Caliber, and StarTeam products, Ovums recommendations include: Highly skilled application development teams The Silk, CaliberRM, and CaliberRDM products form a powerful combination of test management, test automation, and requirements management and definition offerings that already have pre-built integrations. The SilkTest products were among the first that were optimized for web application testing, and have deep support for cross-platform testing encompassing Microsoft .NET and Java application environments. They also provide versatile test lab management capabilities that help development teams manage their test environments. Entry-level teams Teams with shallower skillsets, such as those using junior developers for testing, can take advantage of the visual test scripting capabilities provided in the companys TestPartner product (which originated from Compuware), which are now incorporated into SilkTest.
206
Teams embracing Agile development Micro Focus SilkTest tools support Agile processes such as continuous testing. However, at the planning level, Micro Focus defers to third parties. Teams using Agile methodologies should use Micro Focus tools only if they prefer best-of-breed solutions. Name of product(s) CaliberRM CaliberRDM StarTeam Enterprise Advantage Edition SilkCentral Test Manager SilkTest SilkPerformer
Table 4: Micro Focus quality and requirements management products
Version No. 2008 2010 2009 R2 2010 2010 2010
Source: Micro Focus

Thanks to its 2009 round of acquisitions, Micro Focus has assembled many of the pieces that that will be required for an ALM solution. The following three products spotlighted in this report, will be central to its future direction: CaliberRM/CaliberRDM Providing requirements management and definition. StarTeam Enterprise Providing software configuration and change management (SCCM). Silk A family of products covering test management and automated functional and performance testing.
SOLUTION ANALYSIS
Requirements definition and requirements management
This area is covered by two products in the Caliber family: CaliberRM and CaliberRDM.
CaliberRM
Providing all the core capabilities of a requirements management solution, CaliberRM is based on a three-tier architecture with an object-oriented database back end, and supports both rich and web clients. Like rivals such as MKS, Micro Focuss CaliberRM system uses its own proprietary file format, but with the ability to import data from Microsoft Word or comma delimited (from spreadsheet) files. Along with its sister tool, CaliberRDM, it supports traceability and uses message brokering to keep distributed users in sync. Among the core capabilities supported are baselining, requirements interdependency mapping, coverage and change impact analysis, and the ability to automatically generate test cases for SilkCentral from requirements. When used with SilkCentral (the quality management tool), Caliber acts as a common repository for requirements and test cases (or they can be stored in SilkCentral).
CaliberRDM
This requirements definition tool has had a varied history as Borland (the originator) shifted focus and functionality). Initially, the tool was intended to codify unstructured content into UML use case or BPMN models; in a later iteration, it added the kind of visual simulation associated with iRise, with the current tool offering both capabilities.
207
CaliberRDM supports requirements elicitation through rich-text authoring and visual business scenario design, with provision for integrating stakeholder feedback. Accepting the same Word or comma delimited input as CaliberRM, this tool can be read by Micro Focuss Together modeling tool (also from Borland) to generate UML use case or BPMN business process models; Word or comma limited format documentation; or storyboards, swimlanes, or workflow diagrams. CaliberRDM can be implemented standalone or as the definition module that utilizes the CaliberRM back end database.

StarTeam
Micro Focuss SCCM tool uses a typical three-tier application with SQL Server or Oracle relational database backend, server component, and both thin and thick clients. In the case of the web client, a web server also forms part of the configuration. For distributed systems and teams, an event message broker and caching agents are deployed to improving response with remote users. StarTeam supports trickle updating of change events for providing near-real time collaboration for local and remote users. StarTeam supports a number of change management asset types including change requests, tasks, requirements, and threaded discussion topics. It also includes capabilities to automate build and release readiness checks. It also supports the enforcement of role-based security models and provides some project management capabilities. They include support of hierarchical task management and backlogs; estimated and actual time for change task efforts; flagging of items by system user; and integration with Microsoft Project for generation of Gantt charts and other metrics.
Quality management and test automation

SilkCentral
SilkCentral handles test management. Using web-based clients, SilkCentral supports test planning; design; execution and reporting. It provides a unified framework for managing unit, manual, automated and performance tests, plus visibility for globally distributed testing activities. The highlight is the flexible integration with CaliberRM, which generates test cases that can be stored, either inside the CaliberRM repository or within SilkCentrals; the choice is up to the development team. Changes are flagged in SilkCentral when requirements are changed in CaliberRM. In a recent product update, Micro Focus added risk-based test capabilities patterned off capabilities that originated with QADirector, one of the acquired Compuware products. Agile support for now is limited. SilkTest offers good support for continuous testing, while SilkTest Central CaliberRM can accept user stories from Rally, a leading Agile planning tool. However, at this time Micro Focus does not offer its own Agile planning or resource management capabilities necessary for broader multi-team or enterprise-wide adoption of Agile. SilkCentral supports out-of-the-box interfaces with other quality management and issue tracking tools such as HP Quality Center and IBM Rational ClearQuest; test lab management systems such as VMware Lab Manager; requirements management tools such as Rational RequisitePro; and SCCM systems such as Microsoft Visual Studio Team Foundation Server and Visual SourceSafe, Bugzilla, Atlassian JIRA, PVCS, CVS, and Subversion.
SilkTest, SilkPerfomer
SilkTest and SilkPerformer provide functional and performance testing, respectively. SilkTest automates GUI testing across a number of rich and web client platforms. It has the ability to record and play back GUI-level actions in a wide variety of enterprise applications and has multiple user interfaces optimized for different types of test automation users. For instance, its Web 2.0 synchronization capabilities have the ability to adapt along with the dynamic client. Test scripts can be developed in the products own proprietary scripting language, or in VB.NET/C# for more advanced test script scenarios. In addition, visual testing provides a scriptless approach for less skilled users. SilkPerformer provides load and stress testing capabilities that can simulate from tens to tens of thousands of system users. It uses a controller that connects to remote load generation agents.
208
Additional capabilities include a server analysis module that can be installed along with the SilkPerformer controller that collects server metrics remotely; a monitoring capability that makes diagnostics based on data collected from agents deployed on target application servers; and a cloudbased service that can utilize the Amazon EC2 cloud for performance testing on demand, which can scale to hundreds of thousands of virtual users.

From the 2011 Trends to Watch report for ALM, Ovum has identified the following opportunities: With Agile now in the mainstream, new pragmatic approaches focused around mixing and matching the right methodologies will emerge and become supported by ALM tools vendors. Demand for cloud deployment will prompt software-as-a-service (SaaS)-based ALM offerings, not only from new entrants, but established players as well. With the explosion in mobile application development started by Apples iPhone/iPad and Googles Android platform, smart mobile will become a major target for enterprise applications. Security testing is entering the ALM mainstream. Embedded software development for engineered products is on an exponential growth curve and represents a lucrative market for the few ALM providers that have chosen to target it. In 2010, DevOps entered the awareness phase in the ALM market, and 2011 will see automated tooling targeting the needs of DevOps and Agile release management in particular.

At this point, while generally enabling Agile development, these tools provide limited explicit support for Agile methodologies, and currently lack support for managing multiple Agile projects. Ovum expects this situation to change over time as Micro Focus refreshes its acquired products.
Cloud
For now, SilkPerformer is the only tool that supports cloud deployment. Specifically, SilkPerformer CloudBurst is a service that runs on Amazon EC2, where agents can be deployed in the Amazon cloud on demand. After deployment, SilkPerformer connects to Amazon instances running the SilkPerformer agent and uses them just like on-premise agents. Ovum expects that Micro Focus will extend cloud options to other parts of its toolset when it refreshes its product suite.
Mobile
Like most ALM vendors, Micro Focus does not currently offer support for specific mobile development frameworks.
Security testing
SilkCentral can interoperate with static or dynamic code-scanning tools that could scout for security vulnerabilities; however, it lacks product integrations with specific third-party security testing tools. As a result, there are no capabilities for autopopulating security issues in defect tracking, nor any capabilities for conducting risk impact analysis and prioritization.

Micro Focus has not targeted the embedded software development market.
DevOps
Micro Focus has taken the first steps towards supporting interaction between development and IT Operations. It provides APIs to StarTeam that can receive trouble ticket submissions from IT service desks, to populate defect or issue tracking; however, it has not yet productized support for leading service desks such as BMC Remedy.
209
Ironically, Micro Focus does have an OEM agreement with BMC for converting SilkPerformer test scripts into synthetic performance tests for operational testing, but it does not yet offer support for integrating its continuous deployment capabilities with change management processes on the IT operations side.
PRODUCT ROADMAP
Micro Focuss primary goal is building a major presence in the quality management portion of the ALM market. Towards that end, it is following a two-pronged strategic approach that encompasses: Converging the capabilities of Borland, Compuware, and existing Micro Focus products into a nextgeneration offering. Building integrations to open up Micro Focuss tools portfolio to third-party offerings with the goal of providing cross-tool visibility to the application lifecycle. Although Micro Focus has yet to announce its definitive third-party strategy, its actions so far have been consistent with the best-of-breed OpenALM strategy that Borland espoused during its last days. Borlands strategy centered on publishing interfaces to its tooling. This is also similar to moves being undertaken by Micro Focuss rivals such as IBM Rational, with its Jazz framework and the OSLC (Open Services for Lifecycle Collaboration) initiative. Micro Focuss trump card is its extensive installed base with mainframe development and legacy modernization tools. Its major opportunity will be the selling of requirements and quality management to a market that is modernizing mainframe applications for distributed platforms. That should not take Micro Focus off the hook for addressing two key gaps in its product line: project portfolio management, for which it already has some existing technology, and Agile project and resource planning, which are natural complements for its QA and requirements offerings.
Timelines
CaliberRDM Micro Focus issued an interim update in August 2010 that restructured the product to include selected capabilities from the CaliberRM requirements management tool, and RDMs predecessor, Borland TeamDefine. It expects to release a full next-generation version of CaliberRDM in the second half of 2011 that will add capabilities from the former Compuware OptimalTrace requirements definition tool, on a new SOA-based architecture, and targeted at multiple classes of users. StarTeam The product will continue in its current form as the backbone of Micro Focuss change management platform. It will support a more federated architecture that will connect to third-party commercial and open-source SCCM systems, echoing a strategy that many of Micro Focuss ALM rivals are already adopting. The design and market assumption is that development teams are typically highly distributed, with few if any likely to migrate to a single enterprise SCCM standard. Silk Major releases are typically issued twice yearly, with service packs released as necessary. Development is focused on supporting new application environments such as Rich Internet Application (RIA) clients, packaged applications, and mobile; continuous quality assurance, for supporting test-driven development and similar methodologies; requirements-driven testing; riskbased testing; and test lab automation.
Although Micro Focus products are not targeted at specific verticals, the company has areas of strength in the life sciences, financial services, government/federal, manufacturing, retail, hi-tech, and telecoms industries. It also has a strong presence among system integrators. It targets the Fortune 2000, and development teams that many be as small as 1020 people up to a high of 45,000; the typical team size for IT software organizations is 100300. Based in the UK, Micro Focus is global in its market presence, with offices in over 40 countries and support centers located in the Americas; Europe, the Middle East, and Africa; and Asia Pacific. Roughly half of Micro Focuss sales are direct, with the remainder split 30% and 20% between inside sales and partners, respectively.
210
Key partnerships include: VMware for SilkCentral Test Manager Dynatrace for SilkPerformer Diagnostics, and Amazon EC2 for cloud-based SilkPerformer Cloudburst testing; Oracle and Microsoft (SQL Server) as back-end database options for StarTeam.
Licensing
Micro Focus offers perpetual licensing for CaliberRM, CaliberRDM, StarTeam (Enterprise and Enterprise Advantage Edition), SilkPerformer, SilkTest, and SilkCentral Test Manager; term licenses for CaliberRM, Caliber RDB, SilkTest, SilkPerformer, SilkCentral Test Manager; and pay-per-use for SilkPerformer CloudBurst.
Rivals
Prior to its acquisition of Borland and the QA tools of Compuware, Micro Focuss primary rivals included CA, with its mainframe-oriented development and SCCM tools, and Serena for SCCM. Serena poses an especially interesting rivalry, as the players have complementary presence and gaps: Micro Focus has the test automation that Serena lacks, while Serena has the planning offerings that Micro Focus lacks. With the 2009 acquisitions that brought in quality management, requirements management, and test automation, Micro Focus is one of the few players that could compete head to head with HP Quality Center (and IBM Rational test automation offerings as well). Once Micro Focus rolls out its next-generation ALM offering, it will find itself competing in a much broader arena that also counts IBM Rational and most other ALM vendors. As Micro Focus lacks the breadth of offerings of some of its rivals, Ovum expects that it will continue to position itself as a bestof-breed provider.
Like most requirements definitions tools, CaliberRDM is a more structured replacement for generalpurpose office productivity tools such as Microsoft Office. For StarTeam, it replaces manual, folder-level approaches to storing branches of code and associated assets. For Silk, the usual scenario for functional testing is manual or homegrown scripted testing, where results are tracked on a spreadsheet; for load testing, SilkPerformer typically replaces an existing tool. Micro Focus has described typical projects; keep in mind that the time duration, number of people, and types of skills necessary, will vary by tool. At a high level, Micro Focus estimates that pilot projects will take from two weeks to two months, requiring two to five full-time equivalents (FTEs). By contrast, the estimates for 30-user department projects are a week for installation and another week for training, occupying 3 6 FTEs, while a 500-user enterprise scale project might require 2 3 weeks of installation, with training and rollout phased over 1 2 months, requiring 5 10 FTEs.
DEPLOYMENT EXAMPLES
Leading telephony systems supplier
SilkTest is used for managing performance and functional testing for a telephony systems provider that tests its wares over more than 20 different phone types. Each phone runs on a range of different platforms and there are multiple variants within each product. For example, one telephony platform has nine different product versions, all of which have to work across a variety of platforms; flexibility is a key value-add for the companys customers. SilkTest enables the phone maker to automate and track functional and performance tests to avoid performance issues for its customers in the field.
211
Audio and web conferencing services provider

This company provides audio and web conferencing services, plus ancillary tools for customer training and collaboration. It needed reliable test automation capabilities for regression, cross-platform, and localization testing, especially for its new product releases. It is using Silk4J, the automated Java functional testing tool of the Silk family. Silk4J eliminates the need for legwork such as object mapping, enabling the QA team to focus on test development.
Major consulting and outsourcing firm

This global technology services company delivers a broad portfolio of IT and business process outsourcing services to clients across every major vertical industry sector. It has faced the need to develop cost-effective change management solutions that could be rolled out across its client base. The company developed its own implementation of StarTeam and CaliberRM to manage quality assurance workflows, and identified and implemented standardized change management best practices, followed by a formal SCCM and requirements management rollout. Implementation consolidated 80 IBM CC/CQ servers to 12 fully redundant StarTeam servers, serving more than 4,500 developers and 15,000 end users. The company has estimated that standardization of change management practices, tooling, and server consolidation has generated several million dollars in annual productivity savings, while improving quality of service. Micro Focus Ltd. One Irvington Centre 700 King Farm Blvd. Suite 400 Rockville, MD 20850-5736 USA Tel: +1 (301) 838 5000 www.microfocus.com Micro Focus Ltd. The Lawn 22-30 Old Bath Road Newbury, Berkshire RG14 1QN UK Tel: +44 (0)1635 32646
212
Incorporating
OVUM Butler Group
MICROSOFT: Microsoft Visual Studio 2010 Ultimate
WWW.OVUM.COM
TECHNOLOGY AUDIT
Microsoft
Microsoft Visual Studio 2010 Ultimate
SUMMARY CATALYST
The latest flagship application lifecycle management (ALM) solution from Microsoft is Visual Studio 2010 (VS2010) Ultimate, which incorporates Visual Studio Team Foundation Server (TFS) and the complete set of features previously available in role-based editions. Under the Ultimate offering are the Premium and Professional editions, with Microsoft Developer Network subscription now a necessary part of the offering to gain maximum benefit. TFS is part of all but the start-up editions and there is a Test Professional edition for cases where users only need features specific to testers. The Ultimate edition caters to large, complex projects with modeling support. The Windows Azure cloud-computing platform is part of the Ultimate subscription.
KEY FINDINGS
Strengths: Highly integrated ALM solution for development team and managers. IntelliTrace historical debugging feature for viewing past events leading to debug point. Architecture explorer for navigating visually through code. Test and Lab manager for manual testing and test-case management for physical or virtual labs. Weaknesses: Requirements definition only available through third-party solution add-ins; requirements management also basic out-of-the-box. Application Project Portfolio Management is limited within VS2010. Key Facts:
i Microsofts approach to ALM is to support whatever process the team chooses. i VS2010 can be used for Java projects and other languages, as well as .Net.
OVUM VIEW
Microsofts ALM offering, Visual Studio 2010, has an integration platform underlying it. This platform, TFS, is available in the Ultimate, Premium, or Professional with MSDN editions and not in the start-up editions Express and Professional with MSDN Essentials. TFS is a key part of the Microsoft ALM solution as it provides the backbone for lifecycle integration, allowing team collaboration, process methodology support, build management, and integration between different tools. Microsoft has moved away from what was fashionable in ALM back in 2008 role-specific tools. The problem with that approach is that it tends to reinforce silo work habits, and especially with Agile development rising to the fore, teams today comprise developers with mixed skills who are able to wear different hats throughout the project: database designer, architect, developer, QA, etc.
CHAPTER 8: MICROSOFT MICROSOFT VISUAL STUDIO 2010 ULTIMATE
215
The new approach taking hold across the ALM market and reflected with Microsofts latest solution is to provide tools with increasing sophistication to tackle ever-larger projects. The one exception to this is the VS2010 Test Professional with MSDN edition QA is often still a separate unit, especially in the larger organizations, and its staff have particular needs, which this edition addresses (although the features are all present in Ultimate so it not an exclusive product). Test Professional provides a nonprogrammer environment rather than the usual Visual Studio one. Central to VS2010 is Visual Studio, the Integrated Development Environment (IDE), which is often the preferred IDE for .Net platform developers, and is also used by developers for non-Microsoft projects. Compared to the 2008 release, VS2010 is significantly improved and includes new features; as a result, the solution provides enhanced development process visibility to business stakeholders, better integration capabilities, and better quality management throughout the lifecycle. The present version aims to align the software development process further with business goals. Such supporting features include: better work item traceability and work item management through hierarchies; the ability to link work items and provide change-impact analysis; improved project scheduling and enterprise-wide project management (VS2010 integrates with Microsoft Project Server), as well as Excel integration for Agile project planning; and customizable reports and portfolio-wide dashboards. A new component in VS2010 is Test and Lab Management, an environment for supporting the developbuild-deploy-test cycle that can be set up in minutes through the use of virtualization, and can be integrated with VS2010 and System Center Virtual Machine Manager. Specifically used to accelerate the creation of complex environments and ensure that developers and testers are working against the same build, Test and Lab Management automates virtual machine provisioning, build deployment, and build verification.
Recommendations
Enterprises working on large-scale Microsoft .Net projects: VS2010 Ultimate meets the needs of a large-scale project, such as supporting globally distributed teams. Furthermore, large enterprises will often use a mix of technologies, for example using Java on the server side and .Net on the client side; VS2010 can provide ALM services for source files written in any programming language. However, some of the modeling tools, such as Architect Explorer, are only applicable to .Net projects. The Requirements Definition and Management features are limited in VS2010 and users are advised to seek a compatible plug-in from the partner ecosystem. SME development projects: Businesses with a tight IT budget will want to explore the benefits of cloud computing and the Microsoft Azure platform, available as part of Microsofts software + services approach. VS2010 ties in well with Microsoft Azure for Windows application development. Businesses without Microsoft .Net development: The majority of organizations have mixed IT platform environments, with .Net and Java being most prevalent. While VS2010 supports the latest ALM features, it would not be ideal for customers not developing for Windows or .Net projects.

Microsoft VS2010 is a centralized, repository-based, integrated ALM solution. It is mostly Web based, although the Test and Lab Management module depends on a rich client. It has a built-in workflow engine, integrated reporting engine, and analytics and warehousing capabilities. The server component (TFS) acts as the central collaboration hub between the tools and manages the work items and artifacts, and their interdependencies; it also provides build management capabilities, and project and project portfolio management integration. For Agile projects the solution provides pre-built burndown, burnup, cumulative flow, and velocity reports, as well as portfolio-wide dashboards and reporting. The integrated workflow engine facilitates workflow automation and visual workflow-modeling capabilities. Process and methodology support is offered through pre-built, custom templates which enable project-structure definition and facilitate policy enforcement.
216
TFS provides integration with Microsoft SharePoint Server, Microsoft BizTalk Server, and Microsoft Project Server (Microsoft Project Portfolio Server integration is on the product roadmap), and enables integrated collaborative development, and management of software development projects. The various VS2010 edition highlights are depicted in Figure 46. VS2010 Ultimate with MSDN is the edition that has every feature Microsoft is offering in ALM.
IntelliTraceTM UML Modeling

Microsoft
Test Case Management Manual Testing

Fast Forward for Manual Testing
Visual Studio 2010
Architecture Explorer Logical Class Designer Load Testing UI Test Automation Performance Profiting
Ultimate
Layer Diagram Web Testing
Test Impact Analysis Static Code Analysis Code Metrics Database Deployment Test Data Generation Multi-core Development Cloud Development Windows Development Office Development Customizable IDE
Microsoft
Visual Studio 2010
Code Coverage Database Change Mgmt Database Unit Testing Silverlight Tools
Premium
Microsoft
SharePoint Development
Visual Studio 2010

Professional
Web Development Generate from Usage New WPF Editor
Figure 46: Microsoft VS2010 product lineup and key features
Source: Microsoft
Requirements elicitation is not covered in VS2010 and Microsoft has a number of third-party plug-ins available from its partner ecosystem. Microsoft has enhanced requirements management since the 2008 edition, so requirements are represented as work items and parent-child relationships between work items have been added, allowing for support of requirements hierarchies. The approach to work items is flexible: work items can represent development and test tasks, among others. Work items can also be traced throughout the lifecycle. For advanced requirements management, Microsoft suggests partner products such as eDev inteGreat, as well as solutions from Altova and Sparx. Comprehensive guidance documentation for requirements engineering is available from Micosofts open source project Codeplex: VS2010 TFS requirements management from Rangers Solutions.
SOLUTION ANALYSIS
ALM integration platform
In addition to being an ALM solution, VS2010 offers an ALM integration platform. In the case of requirements definition the one major gap in Microsofts ALM portfolio there are third-party plug-in solutions, including those from IBM Rational, TeamSolutions, Ravenflow, and Micro Focus. The ecosystem around TFS extends to a wide range of plug-in solutions. A sample includes ALM process methodology solutions from EMC, Ivar Jacobson Consulting, and Sogeti; application migration and modernization is extended by Fujitsu and Micro Focus, among others; coding tools by Intel, JetBrains, and Micro Focus; and application security plug-ins are available from Arxan Technologies, Coverity, Micro Focus, and PreEmptive Solutions.
217
The ALM platform is based on TFS and includes a Microsoft Management Console (MMC)-based admin console, network load-balancing ability, and the ability to scale projects across databases. On the build management front, TFS has a new feature known as gated check-in that enables users to validate the code on a separate build machine before the actual check-in takes place, thereby preserving the integrity of the repository code in case of build failure at the gate by withholding the check-in. VS2010 provides a build engine based on Windows Workflow Foundation (WWF) that enables builds to be distributed over several machines. With the help of the build agent pooling feature, a cluster of build machines can process multiple builds simultaneously.
Software architecture
One area that is weak in many rival ALM systems is support for software design modeling and architecture. The collapse of the Model Driven Architecture (MDA) movement has left a gap in the modeling community, filled by a small number of vendors, including Microsoft which has remained an active player in architecture, though it has been separate from MDA. Microsoft has in recent years become supportive of Unified Modeling Language (UML) and now offers a UML 2.1.1-compliant diagramming tool for depicting activity, use case, sequence, class, and component objects. Users can model architecture layers in diagrams and carry out code dependency validation against it. An innovative tool, Architecture Explorer, maps existing application architectures, depicting class objects and allowing users to navigate the code.
IDE
Visual Studio is one of the key IDEs on the market and is long recognized for being state-of-the-art. Although primarily designed for supporting .Net development, it is being presented by Microsoft as an all-rounder; for environments with mixed technologies, this is an advantage. The IDE now supports parallel applications development through native C++ libraries and compiler, debugger support, and a performance analyzer; enabling developers to take advantage of multi-core processors and create multithreaded applications. Other IDE enhancements include better Web and C++ development features, and application development support for the new Windows 7 platform. It will also support cloud development for the Windows Azure platform; users will be able to create and debug services in Visual Studio and deploy them on Azure through the Windows Live Developer Portal. The IntelliTrace tool can save time when debugging by providing historical information. Often when discovering a bug it is necessary with traditional debuggers to restart a session in order to see the states and events leading up to the bug. With IntelliTraces historical debugger, this information is readily available.
Database development
VS2010 Ultimate offers database developers a range of tools: database schema build and deployment, database refactoring, database change management, database unit testing, database test data generation, schema comparisons, and data access. There are SQL Server 2000+ database creation wizards and a T-SQL editor specifically for Microsoft products. Newly supported are data-tier applications (DACs); these were introduced in SQL Server 2008 R2 and contain definitions for a SQL Server database with supporting objects used in client-server or multi-tier applications. Out of the box, this functionality supports Microsoft SQL Server 2005 and 2008, but it has extensibility for third parties to support other databases, for example there are plug-ins by Quest Software for Oracle databases and IBM provides plug-ins for DB2.
Test and lab management

The lab management feature in VS2010 Ultimate simplifies testing in a virtualized environment by reducing the setup and tear-down time for virtual environments. Integration with build management workflow improves application quality through automated virtual machine provisioning, build deployment, and build verification testing. Lab management enables testers to create checkpoints on failures across multiple environments, which can then be recorded as bugs with links to virtual environments that are accessible to the developer.
218
Development and testing support

Development functionality includes: code analysis check-in policy that requires code analysis to run prior to check-in; code metrics such as cyclomatic complexity, class coupling, and depth of inheritance; managed code analysis for identifying violations of .Net design guidelines; profiling tools for identifying performance bottlenecks; line-level sampling to determine where the processor is spending the most time; and report comparison using Performance Explorer. Code quality features include adding test results to a work item or tracking tests against work items, creating unit tests, and scheduling tests. Testing features include calling one Web test from another Web test, custom host adaptors for running tests in alternate test environments, generic tests for wrapping external programs and tests not part of VS2010, load modeling for load tests, load tests for Web tests or unit tests that simulate multiple users for performance and stress testing, and manual tests such as disconnecting the power supply. Microsoft Test Runner enables test case execution, and event and system log collection, and captures images and video of test execution; the integration with TFS allows users to create defects linked with related artifacts. Other features enable test planning, test scheduling, and testing-tasks allocation. Microsoft has introduced test case management functionality in TFS, which allows users to track test cases, link test cases to requirements, and perform impact analysis. The solution highlights the test cases to be executed depending on changes made to the application, and enables quality professionals to record test execution and system information such that developers can see what was being done when a tester finds a defect. The solution also provides a template named Coded UI Test to enable testing of user interface functionality.

The target market for Visual Studio spans all industries and company sizes. Nucleus Research, on behalf of Microsoft, estimated the expected return on investment (ROI) for five cases. The case study for Dell showed an estimated 225% ROI and payback over six months, close to the average of 263% over the five cases (which ranged from 89% to 512%). An important lesson learned from the ROI studies was that large enterprises have complex work practices and work flows and it is best to review these first and make modifications that simplify these procedures before deploying VS2010. The market opportunity for Microsoft is significant as VS2010 is maturing into an integrated ALM solution that can compete against offerings from the more established players. Historically, Microsoft offered spot solutions, such as the VS IDE and Visual SourceSafe for source code control, but it has since been investing in VS2010 as a major platform, and now, with this release, offers a comprehensive solution. It is used internally at Microsoft, including for rolling out new versions of VS2010 itself, which helps with product improvement. Microsoft releases a major version of the product approximately every two years; minor updates and service packs are released as appropriate.
GO-TO-MARKET STRATEGY
Microsoft sells directly and over the phone through field and telesales forces, respectively. The company also has an indirect sales model for sales through reseller and service partners. Microsofts Visual Studio Industry partner program has over 300 entities that provide solutions to extend the capabilities of the VS2010 platform; the Visual Studio Gallery lists specific extensions and companies that work to support the Visual Studio ecosystem. Examples include Klocwork Insight for analysis of source code, and Electric Commander and Electric Accelerator for automating parallel build processes. Microsoft offers volume licensing programs with agreement periods of varying length to suit different customer categories. Software Assurance is provided for an additional fee (nearly one-third of the license price per year) and includes MSDN Premium Subscription, and access to new versions of the software.
219
IMPLEMENTATION
VS2010 Team Foundation Server relies on Windows Server, SQL Server, and Windows SharePoint Services a Windows administrator can install these if they are not already in place. VS2010 can be deployed by software development teams themselves; Microsoft also provides online documentation to help clients through the setup and installation process. Deployment services are available from Microsoft as well as from ecosystem partners. VS2010 implementation, which includes software installation and resource provisioning, takes less than a day; however, process implementation and customization time may be longer. The deployment of TFS is flexible, not just for install time customization, but also in determining what functionality is available for each project. Administration is also minimal, given an existing Microsoft platform environment with readily available administrators with general skill sets. Microsoft offers free Web-based training, available on MSDN, and training partners also offer courses, ranging in length from one day to a week. There is also an active community ecosystem around VS2010, with assistance available from online forums and guidance documentation from CodePlex the Microsoft open community code repository. Technical support is available from Microsoft Product Support. VS2010 can be enhanced with additional Microsoft products: Microsoft Project, Microsoft Excel, and Team System Web Access. Any client connecting to VS2010 requires a named user Client Access License (CAL); VS2010 Ultimate already includes one CAL. The platform requirements for VS2010 are Windows Server 2003 and later, and VS2010 client tools run additionally on Windows XP and later; TFS client-side support for Macintosh and Linux OSs are available from partners. Necessary products to support TFS are Microsoft SQL Server 2005 or 2008 and Windows SharePoint Services; licenses for these are included in the VS2010 license. There is integration support through an open and documented Web service, exposing all the functionality of VS2010. In addition, for some time Microsoft has provided the Microsoft Source Code Control Interface (MSSCCI) for third-party source code control integration. Visual Studio applications can target previous .Net framework versions 2.0, 3.0, or 3.5. There is a stand-alone client for TFS called Team Explorer that can be installed and run side by side with most applications. There is also a Webbased interface for access to project information via a Web browser. Cross-platform integration is supported through the Microsoft partner ecosystem: for instance, Java developers using Eclipse IDE on Linux, Mac OS X, or UNIX can access VS2010 source control, work item tracking, documentation, and reporting features from within the Eclipse IDE with the help of Teamprise. A complete list of extensions can be viewed at the MSDN website.
DEPLOYMENT EXAMPLES
3M: To streamline its operations, 3M deployed Microsoft Visual Studio 2010 Ultimate to take advantage of features such as work item hierarchies, improved testing, and interoperation with products like Microsoft Expression Studio design software. As a result, 3M has been able to simplify its development processes and accelerate the delivery of 3M products to the marketplace. Sogeti: Professional software services firm Sogeti needed better ALM tools to ensure on-time and glitch-free implementations across its numerous global offices. Using Visual Studio 2010 Ultimate, Sogetis employees now collaborate more easily, and developers and architects spend six fewer hours a week checking to see that software adheres to the plan. Microsoft UK Thames Valley Park Reading, RG6 1WG UK Tel: +44 (0)8706 010 100 www.microsoft.com/uk Microsoft Corporation One Microsoft Way Redmond, WA 98052-6399 USA Tel: +1 (425) 882 8080 www.microsoft.com
220
Incorporating
OVUM Butler Group
MKS: MKS Integrity V2009 sp4
WWW.OVUM.COM
TECHNOLOGY AUDIT
MKS
MKS Integrity V2009 sp4
SUMMARY IMPACT
In the ALM market, MKS has long been the exception that proves the rule. Unlike most rivals, MKS delivers broad functionality in a single product, MKS Integrity, that is based on a single data model and workflow engine. MKS has also tilted against windmills in building its product organically rather than through acquisition. Coming from its roots in source code management (SCM), MKS Integrity covers software asset management, requirements management, quality management, defect tracking, and project management. MKS is one of the few ALM vendors that has successfully targeted the embedded software development space, where it coexists and in some cases competes with PLM vendors. Supporting its targeting of embedded development, MKS provides integrations with Mathworks, Sparx, and Atego to converge engineering modeling and simulation with the application lifecycle.
KEY FINDINGS
Strengths: MKS Integrity delivers the unified ALM solution that has long eluded most of its rivals. MKS Integrity has developed strong capabilities for supporting the growing market for embedded software development. Organic product development strategy reduces integration issues. Weaknesses: As an independent vendor, MKSs modest size may be a concern for large clients. MKS currently lacks critical path and resource management capabilities useful for large projects, that would round out its ALM solution. Lacks native test automation functionality. Key Facts:
i Well focused on growing markets overlooked by most ALM rivals i MKS has built its product organically, reducing integration surprises that often
occur with suites assembled through acquisitionKey findings Information.
OVUM VIEW
Compared to most of its ALM rivals, MKS has followed a unique path: it has built its ALM product organically, rather than relying on acquisitions. As a result, MKS offers a single product, Integrity, with a common repository and code base that originated with its source code management capability, and has subsequently spread to cover requirements management, software configuration and change management, project management, quality management, and release management. Customers get the full product but manage only the artifacts that they choose, with the flexibility to extend coverage over time without the need for installing additional tooling. As a result, some customers might initially implement requirements management and subsequently extend to processes such as quality management and/or SCCM or vice versa.
CHAPTER 8: MKS MKS INTEGRITY V2009 SP4
223
Because of its requirements management and lifecycle trace capabilities, MKS has found strength in a fast-growing segment of the ALM market overlooked by most of its rivals: managing software development that is embedded into complex, highly engineered products. The market is growing in direct correlation to the growth of software content in engineered products ranging from automobiles to appliances, consumer electronics, and aerospace; increasingly, product companies are looking to software to speed innovation and differentiate their offerings. The complexity of delivering functionality that can manage requirements for software that is embedded in systems of systems, plus competition from the product lifecycle management (PLM) and engineering community, has created high barriers to entry that have kept the number of ALM players small. As an independent vendor whose strategies have been to specialize on ALM and rely on organic growth, MKS picks its targets carefully; the company does not try to be all things to all people. While MKS covers the essentials of application development project strategy, it defers to third parties for execution of test automation and test lab provisioning, model-based development, and full project portfolio management (PPM). Nonetheless, given that the lions share of the companys recent growth has centered on managing project strategy for highly complex, embedded software development for engineered products, MKS should address gaps where it comes to critical path analyses that frequently used in engineering projects, and resource management.
Recommendations
Who should deploy: Large IT and product engineering organizations ranging from 250 25,000 software and/or product engineers. These organizations typically have highly complex software development projects with complex interdependencies involving multiple teams. Because relatively few enterprises are implementing or developing large new software projects from scratch, the sweet spot for such complex projects is in development of software embedded in engineered products and systems. Who should not deploy: Smaller software development organizations with modest-sized projects. While MKS Integrity can help smaller organizations rationalize software development, the greatest benefits will accrue to highly complex projects.

As noted, in the ALM market, MKS is the exception that proves the rule. While most of its rivals cover the application lifecycle with multiple products that are bundled or integrated together, MKS offers just one product: Integrity. That product uses a common repository, process management, and workflow engine to cover requirements capture and traceability; software configuration management; engineering change management; uality management, and release management. The biggest adjustment comes for users of MKSs requirement management functionality; while most requirements management systems work with Microsoft Word document files that are referenced in a database, in MKS Integrity, requirements are discrete objects that are stored directly in a database. MKS does offer a rich text editor for document formats that resemble Microsoft Word, which can ease some of the adjustment. While business analysts and other users of requirements systems may need to adjust from working with file-based artifacts, the benefits come downstream thanks to associativity that link and automatically synchronize these objects with related artifacts (e.g., test cases, builds, code fragments, etc.). Although built on a single repository, MKS Integrity can be federated to support global development teams. Although MKS Integrity has broad coverage across the application lifecycle, it also provides clients the flexibility to implement functionality in stages; customers do not have to implement the entire tool to cover the entire application lifecycle at once. As described under implementation later in this report, staged implementation is a common use for MKS customers, who typically implement Integrity alongside third party or homegrown tools covering other processes. The most common starting points are software change and configuration management, and requirements management.
224
MS Word MS Project UML Tools Test Execution Tools
Costs, Risks, % Completion, PMBOK, Productivity, Compliance, etc.
MKS Repository
MKS Integrity
Process (ALM/ITSM), Roles, Permissions, Projects, Applications, Portfolios, Compliance
MKS Source MKS Requirements

(Unix, Java, Net, iSeries, zSeries, etc.) Package (SAP, etc.)
MKS Test
MKS Deploy
Figure 47: MKS Integrity high-level product architecture
Source: MKS
Third-party technology integrations

MKSs technology integrations include: Office productivity: MKS can import from and export to Microsoft office productivity tools including Microsoft Project, Word, and Excel Application Development: MKS Integrity provides plug-ins for Microsoft Visual Studio and Eclipse IDEs Application lifecycle: Interfaces are available for IBM Rational DOORS, ClearCase, and ClearQuest; HP Quality Center; Hudson and other continuous integration technologies; iTKO Lisa; Microfocus QALoad; build scripting tools such as Ant and Maven; Perforce SCCM; and other tools. Modeling: MKS is integrated with Mathworks MATLAB and Simulink products; Sparx Enterprise Architect; and IBM Rational Rhapsody. DevOps: MKS Integrity can accept trouble tickets from BMC Remedy IT service desk (help desk) to generate defects issues. MKS Integrity uses a SQL database for its underlying repository; IBM DB2, Microsoft SQL Server, and Oracle are supported.
SOLUTION ANALYSIS
Software Change and Code Management (SCCM)
Software code management (SCM) was the cornerstone around which the company subsequently built the Integrity ALM product. Its strategy has been vindicated because, by the time that the SCM market became commoditized by open source, MKS had already migrated the sweet spot of the Integrity product far higher in the value chain. Like most ALM providers, MKS has adapted to a world of multiple, heterogeneous source code control systems by enabling its Integrity product to interface through supported and third party adapters; MKS Integrity does not require use of its native SCM tool. Highlight SCM features cover: Hierarchical project structure for organizing source code artifacts.
225
Change and configuration management capabilities first developed for SCM have been enhanced by a visual workflow engine and scalability enhancements that for MKSs largest customers support up to 15,00 users. Core change and configuration management capabilities have been extended from source code artifacts to requirements, test cases, models, and projects. Task-based development via change packages and staged review support policy-based decisions when committing code changes to the repository. Core branching capabilities, including visually-executed automated branch inheritance, branch renaming and moving, and branch merging. Graphical history views display branch lineage. Versioning down to file and folder level. MKS takes a unique approach to the change and configuration management processes that are the logical extension of SCM (and produce an SCCM product); rather than embed it as a function dedicated to the code repository, it is a change management engine that is applied across the Integrity product to cover all processes, from source code to requirements, quality, and build management.
The key to MKSs success in penetrating high tech and manufacturing engineering clients has been its requirements management and lifecycle trace capabilities. With software content increasing in highly engineered products, the issue over who owns product (not just software) requirements has moved center stage. For products that are defined by software, changes in software requirements and configurations typically occur far more frequently than those for mechanical or electrical engineering; in most product lifecycles, requirements for physical product engineering must be frozen earlier than software because of the need to solidify component specifications for upstream suppliers. Unlike some rivals such as IBM Rationals RequisitePro, MKS Integrity does not physically store requirements in Microsoft Word documents or excel spreadsheets; it uses a structured Microsoft Officelike template for storage, and has bi-directional conversion to Office for input and external output or review. A short list of highlight features includes: Interdependency mapping, including named relationships that logically link requirements, test cases, and development activities as artifacts of tracked projects. Requirements/test cases, which can be defined in the same repository, although third party tools are used for executing test plans and generating test cases. Charts and queries display requirements, design, and model test coverage (enabling model-based test case generation). Baselining, which extends not only to requirements, but other assets such as projects, which can be used as part of a change management strategy for reducing impact on all the interdependent artifacts of a complex software or product development initiative. Multiple forms of display, which includes rich text for Microsoft Office-like rendering, flat list, and hierarchical views (which are useful for complex product designs). Integrated change management, not through rudimentary matrices, but through hierarchies that reflect the structure of the overall project. Reuse capabilities, where groups of requirements can be associated and reused in parallel development scenarios or product line variants or branches, for all related downstream artifacts. MKS offers connectors to PLM (and other enterprise systems) through support of Requirements Interchange Format (RIF, also supported by IBM Rational DOORS version 9.2.0.1 or later), or through its SOA-based integration platform; with these links, changes to requirements could be updated bi-directionally with other requirements or product lifecycle tools.
226
Quality and Defect Management

While MKS Integrity does not perform test automation (leaving that to third parties), it manages test strategy by enforcing traceability between requirements, test cases, defects, and related development activities; it can also be used for scheduling automated testing conducted by third party test tools. MKS has developed a supported integration to HP Quality Center where MKS requirements can be mapped to test requirements maintained in HP Quality Center. It includes a test results editor, where QA specialists can enter results of manual tests, query for defects from a specific test case, and generate defect reporting workflows. A framework configures MKS Integrity to capture results from automated test execution tools. Querying, reporting and dashboards display historical trend reporting on the relationship of project, requirements, and test cases over time; suspect link flagging and reporting of suspect changes. Reporting can be configured to provide custom metrics such as defect density. At test case level, MKS Integrity supports the ability to track tests by requirement and provide test case branching and reuse. At defect tracking level, it provides the ability to analyze and prioritize defect resolution, pinpoint the build containing the defect, and provide defect reports to appropriate levels of IT management. And, it can automatically generate defects from trouble tickets submitted through BMCs Remedy IT service desk. The tool has several limitations including a lack of support for managing test labs (e.g., generating a test system image so it can be provisioned test machines), and lack of support for converting test cases into synthetic scripts that can be used for testing actual performance in production.
Project Management
MKS provides a general-purpose project and portfolio management capability that can handle multiple projects; MKS claims the number can go upwards into the thousands. Like most of its rivals, MKS acknowledges the presence of Microsoft Project, and provides bi-directional synchronization support. MKS Integritys project management offering supports several project management methodologies including the V-model for waterfall; agile/scrum, through special template; and an ITIL-based approach for managing change requests. It includes core project management capabilities such as the ability to track budget metrics at multiple levels; what-if modeling to show potential impacts on resources and timelines; risk management capabilities via support of manual data entry or import of hazard or FMEA documents; Gantt chart support; tree structures for representing projects with hierarchical interdependencies; and so on. The product provides reporting at portfolio, project, product, release, sprint, and task levels, covering: Quality metrics (defects, test results, coverage) Change metrics (requirements churn, change orders and authorizations) Configuration metrics (Lines of Code, Defect Density) Reuse metrics (number of shared requirements, test cases, or subprojects) Project Management (Project cost, effort, schedule, value) Process (number of iterations, time in state, days in state, process bottlenecks) Requirements (test coverage, design coverage, risk coverage, requirements churn). While MKS Integrity provides detailed project and financial management capabilities, it does not provide a comprehensive PPM solution that performs critical path or full financial portfolio analyses.
227
Release Management
MKS provides a platform for managing the transition of software projects from development to test and deployment, supporting full traceability and rollback. It can manage the approval chain linearly, yet also support multiple code staging systems where each stage has different deployment targets and policies. As with quality management, MKS build management does not support build execution per se; but it provides APIs and web services interfaces that have enabled MKS professional services to create Maven integrations, and for the MKS customer community to develop ANT integrations. MKS relies on partners including Electric Cloud for build dependency tracking and intelligent build (generating builds only for files that have changed).
PRODUCT STRATEGY PRODUCT ROADMAP

Over the next 6 12 months, MKS will focus on: Improving scalability and performance through clustering support, user/group caching, optimizing performance of long-loading fields, enhanced searching and filtering, automated hot-fix distributions, and other measures. Upgrading the user experience through features such as an updated Microsoft Office-like front end, custom actions and views, advanced sorting, and one-click file export to Microsoft Excel and MATLAB Simulink. Releasing integration technologies including web services support, enterprise service bus orchestration capabilities, plug and play connector frameworks, and BPEL-derived business process orchestration. Expanding its agile planning templates. After the Release of MKS Integrity 10, MKS plans tighter integration with modeling and simulation tools. This is especially critical for embedded software development, as complexity of design drives the need for model-driven development. Additionally, MKS plans tighter integration with Microsoft Office and broadening the capabilities that are accessible through Eclipse and Visual Studio plug-ins.
MARKET OPPORTUNITY
From our 2011 Trends to Watch report for ALM, Ovum has identified the following opportunities: With Agile now in the mainstream, new pragmatic approaches focused around mixing and matching the right methodologies will emerge and become supported by ALM tools vendors. Demand for cloud deployment will prompt SaaS-based ALM offerings, not only from new entrants, but established players as well. With the explosion in mobile application development started by Apples iPhone/iPad and Googles Android platform, smart mobile will become a major target for enterprise applications. Security testing is entering the ALM mainstream. Embedded software development for engineered products is on an exponential growth curve and represents a lucrative market for the few ALM providers that have chosen to target it. In 2010 DevOps entered the awareness phase in the ALM market, and 2011 will see automated tooling targeting the needs of DevOps and Agile release management in particular.

MKS offers an agile/scrum template and support for configuring hybrid processes for its product set. It can support multiple agile project teams across different releases or within the same release. The agile scrum template provides drill down to details such as backlog, burndown, velocity, impediments, and risk level at story and task levels. Through use of relationship rules, MKS Integrity can manage and track interdependencies and constraints across multiple projects for allocating resources.
228
Because MKS Integrity is not specialized for agile, it is best suited for organizations that have a mix of agile and traditional iterative or waterfall projects.
Cloud
MKS has not yet taken a proactive strategy for hosting its tools; some of its customers have licensed MKS Integrity for hosting either in private or third party clouds. Although some of MKSs partners are also scoping opportunities for hosting Integrity in the cloud, they have yet to encounter sufficient demand.
Mobile
Like most ALM vendors, MKS has yet to decipher whether its tooling requires any adaptation for mobile development and deployment. With its support of embedded development, MKS has already attacked one piece of the puzzle: providing lifecycle support for software development of mobile devices themselves. While MKS Integrity can be used for managing code development for the applications that run on mobile targets, its products do not offer specific support for mobile development frameworks or form factors.
Security Testing
Web security has become a paramount issue for software developers; although MKS has directed its sights towards embedded development, increasingly, many of the smart devices for which MKS Integrity is used for managing the application lifecycle are becoming Internet-enabled; this places security as a front burner item for MKS and its rivals. MKS Integrity currently integrates with Parasoft JTest, HP QuickTest Pro, and iTKO Lisa for static and dynamic testing. However, while these tools provides spot coverage of security vulnerabilities, they were not designed for web-specific issues such as buffer stack overflows, cross-site scripting, and SQL injection. MKS also needs to address risk management in the context of web application security, an area in which it is not alone.
Embedded Software Development

Although MKS originally targeted Integrity towards enterprise customers with complex development challenges to rationalize its processes, in 2009 it refocused its business to target the embedded systems sector of the ALM market. In the Ovum report, Software Development in the product lifecycle, we found that software was changing the nature of engineered products. Where embedded dev ices were once smart add-ons to engineered products, increasingly, they are defining them. Paradoxically, while this market offers strong growth opportunity, there are also high barriers to entry a factor that plays in MKSs favor. ALM providers must have strong capabilities for managing product, not just software requirements, change, and portfolios; they must also be able to manage or reference artifacts from the engineering world. MKSs main rivals are IBM Rational and, secondarily, Serena and Polarion; increasingly, PLM offerings such as PTC Windchill and Siemens TeamCenter are competitive for customers where mechanical engineering has typically led product development. For more background on the competitive factors in the embedded software development market, see the Ovum report Software Development in the Product Lifecycle; in 2011, Ovum will follow up with more detailed analysis on the unique capabilities necessary for managing requirements in the product lifecycle.
DevOps
This is an undeveloped area for most ALM vendors; the primary evidence of MKSs strategy is its technology alliance with BMC, where it has developed an interface that enables IT service desk users of Remedy to escalate a problem report to software defect tracking. There remain gaps in two key areas: Project health. Ultimately, the measure of success for a project investment is measured after the software has migrated to production. MKS needs to extend its project and portfolio capabilities to post-deployment by transforming application performance and incident data into key performance indicators for tracking the success of a software project. Delivery management. MKS Integrity extends coverage up through build management. Like most ALM vendors, it has yet to interoperate with the change management and provisioning systems used by IT operations; we expect new capabilities here from MKS in the next year.
229
MKS is targeting verticals that are associated with product development, including automotive, aerospace and defense, medical devices, and high tech electronics. Its secondary focus targets commercial and federal IT, where it is a MarketZone partner of BMC. MKS currently has 1000 active customers. The sweet spot for organizations that benefit from MKSs approach ranges from 250 25,000 engineers. Although entry levels deals are as small as $50,000, typical projects range from $500,000 up, with about 30% of that being implementation. Most of MKSs business is sold direct, through a field organization with presence in North America, Europe, and Asia/Pacific regions. MKS also uses resellers in portions of Europe, Asia/Pacific, and South America.
Licensing
MKS offers named or concurrent user options for standard perpetual licenses; it also will support custom term licenses that could be used with SaaS deployments.
RIVALS
MKSs principle rivals include: IBM Rational, which has developed presence in the embedded software development sector with its DOORS requirements management tool, Focal Point for product portfolio management, and Rhapsody for SysML modeling. PLM players such as PTC and Siemens which are competing in areas such as requirements management and change management. Serena, for presence in commercial and federal accounts, and increasingly, in aerospace and product engineering-focused accounts.
IT organizations typically do not replace one ALM system with another; MKS Integrity replaces manual processes or homegrown tooling. Most MKS customers implement in phases and many of them may only use some of the capabilities of the Integrity product. for instance, they may start by implementing source code change and configuration management or requirements management and then may or may not implement the rest of the functionality.
PROFESSIONAL SERVICES
While MKS works with numerous medium-sized regional implementation partners, it has begun growing a relationship with CSC for implementation and maintenance in global accounts.
CUSTOMER EXAMPLES
Global mobile device manufacturer
The manufacturer faced the challenge of managing defects across a global organization while dealing with requirements from 400 carriers and managing tests for hundreds of handset models and variants. The company implemented MKS Integrity as the companys engineering platform to manage development assets and workflow. They started with defect management and implemented a global solution on Integrity in 42 days to 2000 end users (software developers, testers, and product engineers). They proceeded to requirements management, which facilitated reuse across different carrier and model variant use cases. As of now, Integrity is integrated with key authoring tools for idea management, version control, and build management. With implementation of Integrity, the manufacturer reduced product cycle time and reduced the incidence of last minute software rework. Next they will implement test management for hundreds of devices and variants across the global team.
230
Aircraft manufacturer
An aircraft manufacturer with highly complex software processes required strict compliance the software engineering framework CMMI (Capability Maturity Model Integrated). Specifically, it needed to document that it consistently practiced full traceability from requirements through functional specifications and test management. By implementing MKS Integrity, it was able to contain and manage these processes within a single system, providing real-time metrics showing compliance with these processes. The manufacturer achieved CMMI level 3 within 10 months of implementing MKS Integrity, meaning that its processes were repeatable, documented, and produced tangible improvement (in this case, significant cost and cycle time reductions) over time. The company is now able to synchronize project work breakdown structures and track them using earned value added (EVA) metrics. MKS Inc. 410 Albert Street Waterloo ONT N2L 3V3 Canada Tel: +1 (519) 884 2251 www.mks.com MKS Third floor, Dukes Court Duke Street, Woking Surrey, GU21 5BH UK Tel: +44 (0)1483 733 900
231
Incorporating
OVUM Butler Group
POLARION: Polarion ALM Enterprise 2011
WWW.OVUM.COM
TECHNOLOGY AUDIT
Polarion
Polarion ALM Enterprise 2011
SUMMARY CATALYST
Polarion offers a complete application lifecycle management (ALM) suite to provide greater visibility into the software development process. It has built its portfolio organically, with some components extending open source ALM point solutions, and by providing solutions that integrate the various disjointed process segments, each of which is usually aided by a point solution. Therefore, today, Polarion can offer an ALM suite that: Closely integrates ALM modules, facilitated by a solution-wide shared repository. Drives ALM with a workflow engine that underpins the whole lifecycle on one platform.
KEY FINDINGS
Strengths: Single shared repository-based architecture, better integrated modules. Wiki-based collaboration and knowledge management. Builds on the proven capabilities of open source software version control systems. Supports enterprise-level Agile projects. Weaknesses: Lacks requirements modeling and build roll-back capabilities. Portfolio management misses out features such as critical path analysis, financial management, and what if modeling. Key Facts:
i Completely web-based interface. i Integrates with HP Test Director for testing automation.
OVUM VIEW
Software development environments normally have many point solutions implemented to manage the different phases of the software lifecycle. These include separate tools for requirements management, bug tracking, change management, and configuration and version management. In addition to tool interoperability and integration issues, this also results in the development artifacts being scattered, making it difficult to derive useful information from these systems. Most present-day ALM systems now feature a single common repository in which to store all artifacts, which will be accessible to all of the ALM modules. Polarion, which started offering ALM solutions in 2005, is a relatively new entrant in the market. However, in this short timeframe, the company has been able to gauge industry sentiment well, and as a result was able to design its core offering Polarion ALM Enterprise in line with market demand. Web-based access, collaboration, and low-cost application lifecycle management are the key focus areas of Polarion ALM.
CHAPTER 8: POLARION POLARION ALM ENTERPRISE 2011
235
The web-based nature of the solution does not impose any specific requirements on the client side, and can be accessed through any desktop computer with a web browser, enabling easy deployment and integration with existing IT infrastructures. Its single repository architecture (based on the open source tool Subversion), provides greater traceability, and better integration capabilities than multi-repository ALM solutions. In addition, by building this ALM solution around Subversion, Polarion can keep the license costs highly competitive. This also makes Polarion a natural port of call for any enterprise user of Subversion. One area of concern for Ovum is the lack of native requirements modeling capabilities. Although integration with third-party visualization tools (such as Sparx Enterprise Architect and MID Innovator) is provided, adding modeling and visualization capabilities to the Polarion solution would simplify requirements management for large-scale projects.
Recommendations
Subversion users Polarion ALM will be an excellent choice for enterprises that are already using Subversion and would like to standardize their development environment in and around Subversion. Enterprises with significant in-house software development Polarion ALM will be a good fit for organizations that have significant in-house software development and are looking to achieve better collaboration and process support. It is suitable to the needs of distributed application development teams in large and medium enterprises that need an ALM solution covering core segments of the lifecycle. The solutions high scalability supporting in excess of 1,000 users per server makes it suitable for large enterprise applications. Product lifecycle management Polarion ALM is suitable for IT and product development. In the product space, the solution has carved a niche position. For example, in the telecoms domain, it has a strong presence with customers including Vodafone, T-Systems, and Telefonica.

Polarions ALM stack comprises requirements definition and management, test management, software change and configuration management (SCCM), project management, defect and issue management, and build and release management. The vendors ALM strategy revolves around the open source SCCM tool Subversion. Polarion ALM leverages Subversion as the central repository for all software development process artifacts, and as the version control tool and change management solution. The solution architecture is shown in Figure 48. Above the repository, the Polarion ALM server provides the various tools (audit and metrics, requirements management, change management, test case management, and build management) that can be accessed directly via a web browser, and integrates with integrated development environments (Eclipse, NetBeans, and Microsoft Visual Studio), commandline tools, and third-party ALM tools. Custom applications connect through web services, HTTP/HTTPS, Java application programming interfaces (APIs), Simple Object Access Protocol (SOAP), and Extensible Markup Language (XML). Polarions requirements management module enables users to manage different types of contents, including office documents, data artifacts, and wikis. Knowledge assets can be preserved by reusing and customizing standard requirements libraries. Polarion ALM is focused on providing solution/lifecycle-wide traceability for requirements and other development process artifacts such as change requests, test cases, and code/build versions by linking them together and maintaining corresponding interdependencies. Furthermore, the solution provides strong change and configuration management capabilities, and integrated workflow and process management capabilities. All of the aforementioned capabilities result from Polarion ALM platforms single shared-repository and groundup design. In addition, pure web access and collaboration support by wikis gives added support for distributed teams. Polarion has also come up with a plug-in for web-enabled mobile devices (such as iPhone, Android phones, etc.), which enables users to access their project details, modify work items, receive project notifications and status reports, and make approvals on the go.
236
NNING PLA
ss ce o Pr
le gi
Tra cea bil ity
Compliance
Audits & Metrics
Test Case Management
Quality
Change Management
NC
bo
rat ion
L MP
M
E
Source: Polarion
Co
ll a
Task Management
y ilit b i Vis
Figure 48: Polarion ALM architecture
Polarions project management module helps users to manage task planning and assignment; estimate, report, and monitor time spent on tasks; and monitor project progress and backlog support. The solution offers in-built support for test management, although it integrates with HP Test Director for testing automation. Rather than providing native model-driven development capabilities, Polarion offers tight integration with Sparx Enterprise Architect (providing UML, SysML, and business modeling), FreeMind MindMap, and MID Innovator. The suite offers build and release management (BRM) functionality with comprehensive features; only the capability to roll-back builds is missing. Polarion ALM also provides support for Agile methodologies, with process templates available for Scrum and Extreme programming. By using such templates, Agile teams can plan iterations and reviews, and check their process maturity with appropriate quality audits. Audits let users check projects methodology compliance in realtime, find shortfalls in methodology appliance, and learn how to fix them. Polarion ALM process support also covers Capability Maturity Model Integration (CMMI) programs through live assessments based on actual project data.
SOLUTION ANALYSIS
Requirements definition and requirements management
Polarions requirements definition and management functionalities are reinforced with its Live Documents feature, which enables stakeholders such as managers, domain experts, and developers to collaborate over the requirements-engineering process. Requirements can be authored in MS Word and, upon check-in to the Subversion repository, the solution automatically generates work items for developers. Stakeholders can review and edit requirements or any other artifacts, and receive notifications of changes therein.
EN
TA TIO
Build Management
MA
INT
EN
237
The module also facilitates collaboration among teams and team members through a wiki. The wiki feature, together with the web client that provides access to every Polarion feature, enables distributed development teams to collaborate at the repository level. Artifacts in Polarion ALM are traceable, and users can see what was changed, when, and by whom. Users can also create, edit, view, link requirements to artifacts, and vote on requirements in the solutions portal. Requirements and their interdependencies can be represented graphically through the open source FreeMind mind-mapping tool. The solution facilitates requirements dependency analysis and traceability through a twodimensional traceability matrix. Polarion ALM further provides traceability across other components such as requirements and test cases, and tasks and code modification.
Project and portfolio management

Polarion ALM provides project management and planning capabilities through the Live Plan engine. The solution allows administrators to define policies globally, as well as for projects and individuals. All actionable elements such as requirements, any project-related tasks, change requests, or test cases are referred to as work items. Live Plan further enables users to create work items containing the requirements and tasks associated with a project, and enter corresponding schedules for them. Once work items have been scheduled, the solution offers Gantt chart views of the project. Polarion ALM project plans can be exported to Microsoft Project. Polarion also provides dashboards and reports for keeping track of and analyzing project progress. Two kinds of dashboards, namely Project Overview and Project Dashboard, are available. The Project Overview provides a brief summary of key project metrics, whereas a Project Dashboard contains more detailed information about work-item trends, test coverage, process score based on adherence to global policies, top five projects, and packages based on level of code reuse. Users can set the dashboard scope to repository, project group, or an individual project. The dashboard window corresponding to an individual project contains summary information about test success ratio, build frequency, test coverage, and build success ratio, among others. The baselines feature of the solution enables teams to take snapshots of the state of a project at a point in time and create a baseline. Teams can then compare baselines for project progress, and also use them to measure capability levels against process maturity models such as CMMI and Software Process Improvement and Capability dEtermination (SPICE). Polarion ALM can manage multiple projects at a time (running in thousands), and offers the basic planning and scheduling features, but there are certain areas of portfolio management which it fails to address, including critical path analysis, financial management, and scenario/what if modeling.

At the heart of Polarion ALM Enterprise is Subversion (the current version supported is Subversion 1.6). Subversion is designed on the client-server model, and is used as a solution-wide shared-knowledge repository and SCCM module. Polarion ALMs SCCM capabilities are comparable to market-leading solutions; the pure web-based access imposes no constraints on client infrastructure and the single centralized-repository architecture enables source-code sharing, change tracking, IP protection, and governance, and provides low-cost SCCM for geographically distributed development teams. Multiple Subversion repositories are also supported so that distributed SCCM is possible. Polarions roadmap is to include alternate distributed SCCM solutions. Subversions SCCM features include versioning of files, directories, and file metadata; branching/tagging support including trunk-based branching and merging; support for continuous integration and non-exclusive file locking; atomic commits; handling of binary files; and support for merge tracking. The module also provides external stakeholders with secure repository access, and Secure Shell (SSH) tunneling for additional security.
Defect and Issue Management capabilities

The Polarion ALM platforms Defect and Issue Management (DIM) features are integrated into the main workflow, making it easier for developers to submit, and managers to review, open/closed defects and progress made by the team. In the case of unit testing, the solution automatically generates defect-log entries based on test results. End users can create logs, and link defects back to related artifacts such as requirements, test cases, build, and source code versions. Polarion ALM integrates with HP Test Director (a testing automation tool) for functions such as automated test case creation from use cases, use case refactoring, scheduled and automated testing, and automated validation of multiple deployments. However, testing management features such as requirements-based testing, historical trend analysis and reporting, continuous testing, and web services testing are built into the solution.
238
The solution provides a quality assurance dashboard by measuring process performance against established best practices such as code reuse, providing stakeholders with summary information about software quality levels for single or multiple projects. Polarion ALM also provides a CMMI-compliance report which helps development teams to improve their incumbent processes and benchmark them against the CMMI model.
Build and release management

Polarions BRM capabilities include support for continuous integration (only for public stream, not for private workspaces), build validation, intelligent build (for instance build only those files that have been changed), and dependency tracking. The solution provides traceability over all phases of software development, and supports Ant and Maven 2 for managing builds. Furthermore, it provides workflow to manage features, which Ovum believes is essential for automating the build cancellation process and reinstating the previous build.
Tool interoperability and open source dependency

Polarion ALM has grown organically and presents customers with benefits such as a shared central repository accessible from any component, tight integration among the various modules, centralized workflow, closely knit functional components, and a consistent feature set. To integrate with third-party solutions, customers can use the free existing plug-ins (Java APIs or web services APIs) or contact Polarion to create one. Another alternative is to opt for Polarions Adapter concept based on an OEM integration platform, which enables integration and data exchange between the existing/legacy tools (such as IBM Rational Doors, IBM Rational ClearCase, Perforce, and HP Quality Center) and Polarion ALM. Polarions claim of offering a low-cost ALM solution is made possible by incorporating open source modules such as Subversion, and filling gaps in functionality with third-party integrations (i.e. for Model Driven Development and testing tools). In addition, the solution makes use of other open source components: Apache Maven, Apache Web Server, Eclipse, OpenSymphony, and Quartz. Polarion therefore provides a unified framework for these components, and adds proprietary components of its own, thereby offering an ALM solution that is highly leveraged on open source.

The Polarion solution is a single monolithic application with closely integrated functionality and workflow, which Ovum believes is an advantage over ALM vendors whose solutions have grown inorganically and therefore depend on multiple repositories. Built around Web 2.0, Polarions solution has a strong focus on collaboration among distributed development teams, which is exactly what the market demands. Polarion ALM Enterprise is targeted at application development teams (IT departments) within large enterprises and software development companies, and also product development projects supporting significant embedded software components. A lower-cost solution ALM Pro is available, which may better suit small to medium-sized enterprises (SMEs).
Although the target market segment for the solution is horizontal, companies from the independent software vendor (ISV), aerospace, telecoms, engineering, electronics, and financial services sectors are significant in Polarions installed base. A present, Polarion is focused on North American (currently 35% of revenues) and European (currently 55% of revenues) markets. The company has expanded its installed base in the US over the last couple of years, and is now trying to penetrate the Asian market (currently 10% of revenues). The company has multiple routes to market, including selling through industry sales channels, direct/web-based sales, and revenue sharing with partners. Polarions business partners include EA Sparx, Eclipse, Coverity, and Agitar, among others. The company also has technology partnerships with CollabNet (Subversion), Eclipse, Apache, Maven, and XWiki. The solution is sold on a perpetual license, on a named or concurrent user basis. Polarion open source solutions are provided free of charge, while the Polarion requirements management module (standalone solution) is available for $1,290 for a single license, and the Polarion ALM platform is sold for $2,998.
239
Polarion claims that the average value for installation ranges between $12,000 and $60,000. The company provides Bronze, Silver, and Gold solution maintenance packages. Bronze includes email support during normal business hours, and maintenance costs are 20% of the license fee. Silver includes email/remote diagnostics and telephone support during normal business hours, while Gold includes 24/7 email/remote diagnostics and telephone support. A major version of Polarion is released every 12 months, with multiple minor releases during the same year. With a horizontal target market and recent consolidation in existing customer geographies, including North America and Europe, Polarion is not looking to expand its solution scope, but rather to offer a low-cost ALM solution to a wider market. The company is looking to expand its customer base and penetrate newer geographies such as Asia. Polarion is also targeting the on-demand route to sell its ALM solution to small companies and development teams, a strategy that will help the company cover a broader market.
IMPLEMENTATION
Polarion ALM installation involves downloading the software from the Polarion website. The software installation process does not require any special expertise and can be handled by in-house IT teams. Customization and subsequent roll-out usually involve a few days of engagement with Polarion consultancy services for tailoring the solution for specific organizational processes, and for familiarization with best practices. The entire deployment project takes about a week to complete. Polarion claims that the resource requirement for administration, maintenance, or support post deployment is a couple of hours every week. Polarion ALM is a closely integrated solution that does not support single-module deployments. Customers can, however, implement the solution on a per-process basis: for instance, companies can implement a single process to start with, increase the solution scope to cover more processes, and in due course bring all their ALM processes under Polarion ALM as familiarity with the solution increases. The company claims that the solution is easy to use, and that end users undergo a very short learning curve. However, web-based, classroom, and on-premises training courses are provided. The server component of the solution can be deployed on Windows and Linux/Unix platforms. There is no specific client requirement, which is usually through a web browser. Supported browsers include MS Internet Explorer and Mozilla Firefox. The solution is not dependent on any third-party software, but certain open source components leveraged by the solution, such as Subversion, are bundled with the offering. Integration with legacy systems is provided through Open APIs and a web services interface. Custom application can also exchange data with Polarion ALM through XML. The Polarion ALM solution scales as Subversion does, thus the vendor cites an example of a client running a single Polarion ALM Server with 1,400 registered users. Any third-party tool that offers failover and backup to Subversion can be used for fault tolerance purposes.
DEPLOYMENT EXAMPLES
Swisslog
Swisslog, a Switzerland-based logistics application provider, needed an ALM platform that not only offered the desired flexibility to design a Swisslog-specific workflow for requirements, bugs, and change requests, but also provided bi-directional traceability from requirement to the source code and from the source code back to the relevant bug report or requirement. The platform was also required to support planning activities like roadmaps, and iterative release plans. Swisslog now has full traceability from change requests, requirements, and tasks, enabling it to achieve CMMI-level compliance. Using Polarions web-based architecture, and the central Subversion repository, it is able to manage distributed development teams in Switzerland, Asia, North America, and Europe. Swisslog has also migrated its existing data and repositories (CVS, PVCS, and Team Track), into the Polarion system using the free migration tools and Polarions services.
Lunzer+Partner
Lunzer+Partner is a Germany-based warehouse management solution provider. Its software offering, LOGSTAR, which was initially designed for IBM system I (running on i5/OS) needed to be recreated as a new Java-based modular system, requiring the reorganization of the software development process.
240
The company needed an integrated ALM platform which would support the entire software development lifecycle (from requirements management, build and test management, to software change and configuration management), and opted for Polarion ALM. All of the development projects in the company now run on Polarion ALM. Polarion LivePlan is being used for project level planning, and all documents and requirement artifacts are stored in the Subversion repository.
Engineering Ingegneria Informatica

Engineering Ingegneria Informatica offers proprietary software solutions for various verticals in Italy. The company needed an ALM platform which would be independent of the development tools used by different teams and would be open to integration with those tools. The platform also needed to promote the reusability of information assets (documents, practices, codes) by developers, and to be accessible via the Internet to support its geographically dispersed team structure. Polarion ALM has been implemented as the application management platform for all projects, to manage, track, and trace project requirements, test cases, configurations, and defects. It has also enabled the company to achieve CMMI level 3 certification. Polarion Software GmbH Lautlinger Weg 3 70567 Stuttgart Germany Tel: +49 0711 489 9969 Fax: +49 0711 489 9969 20 Email: info@polarion.com www.polarion.com Polarion Software, Inc. 406 Tideway Drive Alameda, CA, 94501 USA Tel: +1 877 572 4005 (Toll free) Fax: +1 (510) 814 9983
241
Incorporating
OVUM Butler Group
RALLY SOFTWARE: Rally Unlimited Edition
WWW.OVUM.COM
TECHNOLOGY AUDIT
Rally Software
Rally Unlimited Edition
SUMMARY CATALYST
As Agile development techniques gain more traction in the enterprise, organizations focus is shifting from Agile adoption at the team or department level to scaling Agile processes across the organization. In light of this trend, Ovum believes that the flagship release of Rallys Agile application lifecycle management (ALM) platform Rally Unlimited Edition is well timed and will help organizations to maximize benefits from scaled-up Agile practices by: Providing end-to-end support for multiple Agile processes and facilitating real-time collaboration. Offering an extensible integration framework using hub-and-spoke architecture for connecting to existing systems.
KEY FINDINGS
Strengths: Single shared repository-based lightweight architecture. Provides superior Agile project management, reporting, and analytics. Completely web-based solution with automated workflow. Extensible through pre-built as well as custom-developed plug-ins. Weaknesses: Lacks native build management capabilities. Financial capability of portfolio management lacks depth. Key Facts:
i Out-of-the-box integration with Salesforce.com. i Integrates with HP Quality Center, as well as multiple integrated development
environments (IDEs), repositories, and build automation tools.
OVUM VIEW
Rallys on-demand ALM offering has helped the company to acquire customers at a brisk pace. The vendor now has over 2,500 customers in 60 countries and, although this number also includes AgileZen customers, it still represents significant growth in Rallys installed base, which stood at 1,500 customers 18 months ago. Compared to other ALM vendors that have recently shifted focus to the on-demand route to expand their installed base, Rally advocated and adopted the software-as-a-service (SaaS) model as its primary business model from its inception. Rally Unlimited Edition is a centralizedrepository-based product with closely integrated functionality and workflow, which has distinct advantages over multi-repository-based solutions that have grown inorganically. Rallys first release was in 2004, which has worked to its advantage as the company has been able to tailor its low-cost ALM offering to reflect market sentiment around Web 2.0-based collaboration and Agile development methodologies.
CHAPTER 8: RALLY SOFTWARE RALLY UNLIMITED EDITION
245
Rally has also been able to rapidly integrate functionality from acquired solutions such as 6th Sense Analytics and AgileZen. Notably, the present release includes advanced analytics capabilities at the core platform level. The company is now looking to grow by expanding coverage to upstream lifecycle processes such as project portfolio management and by developing vertical-specific applications on top of the core platform.
Recommendations
Enterprises with a significant in-house development function Rallys solution has been proven in multiple large enterprise-wide deployments. Organizations looking to adopt Agile practices should consider Rallys offering. It not only helps to reduce the overall development costs, but also improves time to market. For large enterprises wishing to install Rally Unlimited Edition in a private cloud, Rally offers on-premise licenses. Independent software vendors ISVs will benefit from Rallys proven Agile ALM platform. Rallys solution facilitates short release cycles, and helps organizations to improve the quality of delivered software. Small to midsized development shops Rallys ALM platform should be considered by small and midsized organizations because costs are low, it is a SaaS solution with bundled support, and no clientside infrastructure is required.

Rallys ALM stack comprises project management, project portfolio management, requirements definition and management, test management, defects and issue management, release management, and product management. The vendors flagship offering, Rally Unlimited Edition, is an ALM platform that was designed from scratch exclusively to support Agile methodologies. The solution is primarily available on-demand, as a service, but the company also provides a virtual appliance version for on-premise deployments. Salient features of the solution include out-of-the-box Agile reporting, covering custom reports and advanced analytics, reports based on data contained in third-party repositories, program management for multiple teams and cross-project reporting, user interface (UI) customization through configuration, the ability to define custom fields in the repository, work-item tagging, hierarchical structure for user stories (requirements) and projects, and out-of-the-box integration with IDEs such as Visual Studio and Eclipse, among others. As Agile development techniques gain more traction in the enterprise, organizations focus is shifting from Agile adoption at the team or department level to scaling Agile processes across the organization. In light of this trend, Ovum believes that the latest release of Rallys Unlimited Edition is well timed and will help organizations to increase their Agile footprint easily. The solution provides process and methodology support in terms of content authoring, publishing, and access, and read-write access for end users for modifying content, as well as enabling flexible prioritization of tasks. Rallys built-in workflow engine provides visual workflow modeling and automation. Although the solution does not provide pre-built templates for specific Agile methodologies, its Agile process subsumes well-known methodologies such as Scrum, XP, and Dynamic Systems Development Method (DSDM). In Ovums opinion, the workflow is well suited to support any Agile process end-to-end. In April 2010, Rally acquired AgileZen, a Lean project management tool that facilitates cross-team collaboration through Kanban boards. The AgileZen acquisition further strengthens Rallys value proposition and extends its methodology coverage to Lean. Ovum strongly believes that the core problem of managing application development can never be solved just by licensing ALM tools, and Rally shares this view. Improvement in terms of software quality, time-tomarket, and business-focused delivery can only be seen if tool adoption is supplemented with the right processes. Furthermore, this requires investments in developer skills, and training of end users as well as business stakeholders to maximize benefits. To this effect, the vendor has developed its consulting arm and provides training on Agile practices to facilitate Agile adoption, as well as to help organizations scale their processes easily with demonstrable value at each step. The company provides resources and training in Agile methodologies online: for instance, it maintains a Web 2.0 online community where customers can download connectors to third-party applications, participate in forum discussions, and access free online resources and training about Agile methodologies.
246
The Rally platform (see Figure 49) is a multi-tenant, three-tiered web application. Rally is hosted on the Apache web server, which communicates with the client over Secure Sockets Layer (SSL). The platform provides a spreadsheet-like experience inside a web browser through a customizable Ajax front-end. Users can also extend the interface with custom-developed applications or choose from the Rally Apps catalog. Also contained in the platform is a web services application programming interface (API) (REST as well as SOAP) and integration toolkits that enable organizations to integrate with third-party ALM tools and enterprise applications. The core platform comprises the repository and artifact management interface, and the analytics module. The underlying application server leverages an instance of Oracle 10GR2 DBMS as the central repository. Email notifications can be configured to be sent to the clients Simple Mail Transfer Protocol (SMTP) server through the Rally Mail Transfer Agent (MTA). As mentioned earlier, Rally Unlimited is provided both on demand and as a virtual appliance. In the case of on-premise deployments, Rally Unlimited requires a VMware ESX Server instance to operate. The virtual appliance nature of the solution enables cross-platform operability and easy deployment.
Customizable User Interface Versioned Web Services API & Integration Toolkits ALM High Performance Artifact Management
Permissions Revisions Notifications App App Install from Rally Apps Catalog or Create Your Own
Analytics
OLAP Reporting 3rd Party Connectors
Rally, Connectors Dev, PPM, CRM
ORACLE DB
Figure 49: Rally platform architecture
Source: Rally Software
In June 2010, Rally Software announced a new version of Rally Unlimited Edition. The Unlimited Edition contains all features offered by the Rally Enterprise Edition, with additional functionality such as idea management, test planning and regression test management, pre-built integrations with enterprise applications, and development sandboxes for testing custom extensions to the platform. The present release enables organizations to ensure greater business participation in application development and delivery by bringing together multiple stakeholders. It offers enhanced collaboration and participation across the entire business-IT value chain through tools such as Rally Idea Manager (OEMed from BrightIdea Webstorm), Rally Product Manager, Rally Time Tracker, and Rally Quality Manger. Rally Idea Manager enables product managers to gather user feedback through a social-community-like interface; feedback can also be imported into the Rally repository as user stories. Additionally, the tool provides reporting and analytics functionality and users can track the ideas that have been accepted from the user story (requirements) stage through to delivery. Rally Time Tracker provides real-time visibility into project costs through timesheets, and roll-up reporting allows managers to track cross-project utilization based on user story level data. Rally Product Manager is the customer relationship management (CRM) integration; it integrates with Salesforce.com and allows product managers to import customer feedback into the Rally repository and use it for feature prioritization. As mentioned earlier, Rallys platform has been developed from the ground up to support Agile development practices, and Ovum is pleased to note that the vendor has added significant capabilities since the previous Technology Audit in April 2009 (Rally Software Rally Enterprise ALM Platform). Also notable is the support for large scale, globally distributed development, and advanced analytics on all data stored in the repository. In Ovums view, Rally is among the market leaders in the Agile space, and a growing force in the ALM market overall. The fast pace of customer acquisition also stands testimony to that fact.
247
However, enterprises may have to cover a few gaps through third-party tool integrations. For instance, financial management functionality under application project portfolio management lacks depth. Additionally, Rally currently does not provide tool support for bridging the development-operations divide through features such as continuous delivery/deployment. Also, there is no integration with application performance management tools.
SOLUTION ANALYSIS
Tool interoperability and third-party integrations
Ovum is impressed by Rallys single centralized repository architecture, believing this to be the simplest way of ensuring tight integration among functional modules, as well as lifecycle traceability of artifacts. Rallys ALM platform offers multiple out-of-the-box integrations, as well as allowing customers to build their own. It supports reports based on third-party repository data. In addition, Rally allows third-party applications to be embedded in its environment, and provides support for Rally screens to be embedded in other applications. Out of the box, Rally integrates with build automation tools such as Ant, CruiseControl, Hudson, and AnthillPro; Software Change and Configuration Management (SCCM) systems such as Subversion, Git, Perforce, Rational ClearQuest, and AccuRev; and defect and issue management tools such as Bugzilla and JIRA. Rally supports HP Quality Center for test automation. Other available integrations include Codesion, Microsoft Project, Confluence, Microsoft Sharepoint, and Salesforce.com. Notably, the solution also has a mobile interface: Rally acquired the ScrumAway iPhone application earlier this year, which is now available for free as Rally for the iPhone at the Apple App Store. The application enables users to view as well as update data in the Rally repository. Rallys collaboration capabilities include web-based anywhere access, integration with popular instant messaging tools to provide real-time collaboration from within the Rally environment, and annotation of source code, change requests, and test cases. The solution allows users to insert externally hosted wiki pages but does not provide native wiki capabilities.
Project and Portfolio Management

Rallys Project and Portfolio Management (PPM) module has all of the features expected, with only the financial management portion lacking some capabilities. The solution enables enterprises to manage software delivery across teams; users can define organizational structures and project relationships, and manage and allocate resources. Stakeholders can quickly identify resource constraints and view summarized key metrics such as feature completeness and dependencies across teams through dashboards, create detailed functional requirements from high-level feature descriptions, and assign them to multiple teams by leveraging the solutions shared repository. Additional features include program-level reporting of project milestones, timelines, demand management, what-if modeling, and iteration schedules. Rally lacks risk evaluation and analysis functionality in this area. Rallys rich reporting features and dashboards provide real-time visibility into project and program status and facilitate collaboration with the help of electronic project boards and story cards for tracking an artifacts lifecycle. The centralized shared-repository model enables solution-wide metrics gathering and reporting. Rallys Agile ALM platform allows users to drag-and-drop requirements from product backlogs to iterations and releases. Users can prioritize product and release backlogs; view mash-up reports on feature completeness, defect status, and test results across multiple projects; generate real-time burndown and cumulative flowcharts;, and compare project estimates against actual data for continuous improvement.

Rally Unlimited Edition allows users to define requirements with respect to production environments and apply constraints according to performance, quality, and design needs. Users can define high-level business cases, and automatically generate detailed functional and non-functional specifications. The solution supports third-party visual requirements modeling but lacks native capability. Visual architecture modeling through languages such as Unified Modeling Language (UML) is not supported either. Other capabilities include requirements versioning, annotation, and traceability.
248
Requirements management workflow is Agile-oriented: for instance, the solution lets users prioritize and centrally manage requirements in backlogs so that developers can concentrate on features that provide maximum business value. Requirements (in the form of story cards) can be dragged and dropped into various queues and users can create, estimate, track, and test requirements throughout the lifecycle and manage the corresponding interdependencies. The solutions ability to generate test cases directly from requirements and link back the test results simplifies requirements management to a great extent. Rally Unlimited Edition is capable of managing requirements for a single team, as well as coordinating, searching, tracking, and reporting on requirements across projects.
Software quality management

Rally Unlimited Edition includes basic test-tracking and management features bundled with Rally Enterprise Edition. Users can also take advantage of enhanced quality management features through an add-on module, Rally Quality Manager, which enables the linking of test cases to requirements, regression test management, and scheduling of iterative and release acceptance tests. As mentioned earlier, the solution generates test cases automatically from requirements and provides users with a test editor where they can define test types. The test results are stored in the central repository with timestamps; workflow automation features enable users to create defects directly from failed test cases. The solution allows QA professionals to design test plans and hierarchically organize them in test suites. Reporting features enable users to view the test coverage dashboard, which includes key metrics such as number of passed, failed, and pending test cases, among others. As mentioned earlier, Rallys platform offers test automation capabilities by integrating with HP Quality Center. Rallys Agile ALM platform facilitates defect submission, scheduling, updating, and reporting. Defects can be cross-linked to business requirements and test cases such that users can see which defects are stalling acceptance of key features through the release defect dashboard. Rally Support Manager (an optional service desk module) integration enables defects submitted by customers to be imported into the repository.
Build and release management

Rally offers complete traceability among user stories, defects, work-items, and builds. Functionality support in this regard is mostly through reports and dashboards. For build and release automation, Rally integrates with popular tools such as Ant, CruiseControl, Hudson, and AnthillPro. The coordinated release feature allows teams to package features and a set of defects into a release.

The strong uptake enjoyed by Agile development practices shows that the market for Rallys solution and its installed base is growing. Rally has been both a driver and a beneficiary of this trend. Moreover, the SaaS business model has taken off; more and more organizations are opting for the payas-you-go model as it offers immediate value and customers do not have to worry about deployment infrastructure or in-house support resources. Rally has done well to exploit these trends. The vendor is now looking to expand its footprint in client organizations by applying Agile and Lean techniques to areas such as application project portfolio management.
Rally Unlimited Edition is targeted at software product vendors, web solutions providers, system integrators, and enterprise IT departments. The solution is typically aimed at IT managers and CTO/CIO-level executives. Although Rally has a horizontal target market, companies from the technology (hardware, software, and services), media, advertising, communications, consulting and business services, aerospace, financial services and insurance, healthcare, hospitality, not for profit, education, public sector, retail, transportation, and engineering sectors are significant in the companys installed base.
249
Rally has taken both direct and indirect routes to market; the company has adopted a viral leadgeneration program that supports a direct sales team. Direct sales remains the primary source of revenue for Rally. The company classifies accounts by volume, based mainly upon number of potential seats. Rally has recently introduced the global coverage model to support the needs of customers with a global presence and the potential of more than 1,000 seats. Rally has also developed channel partnerships to drive revenue growth. Rallys implementation partners include CEO Softcenters, Matrix, and Pariveda, among others. Rallys business and technology partners include Microsoft, IBM Rational, Oracle, HP, AccuRev, and BrightIdea, among others. Rally offers SaaS, perpetual hosted, and perpetual on-premise licensing. Named user licenses are offered free for up to 10 users, and Rally also offers volume discounts. Average project value for a Rally Unlimited Edition deployment for 100 users is nearly $44,000, with a license to services split of 75% and 25%, respectively. For large, enterprise-wide deployments (500 users), project value is nearly $800,000, of which 25% accounts for Rally Unlimited Edition software licenses and 75% for services including Agile rollout planning and end training. AgileZen is offered on a yearly subscription, and can be deployed for a 50-user team at $1,100 per year. In the pipeline, Rally plans to enhance the capabilities of the portfolio management module, as well as develop vertical specific applications, and provide a benchmarking service that would enable organizations to measure the return on their investment in Agile methodologies.
IMPLEMENTATION
Rally Unlimited Edition on-demand does not require any local deployment; the subscription can be configured and made active in a few hours. The on-demand version is available through a shared multitenant architecture platform. Client access is provided through Microsoft Internet Explorer or the Mozilla Firefox web browser. Pricing for the on-demand version depends on the number of users. The onpremise version is delivered as a virtual appliance with all necessary software bundled with the offering. The virtual appliance can be installed and configured in a day, without vendor or third-party support. The solution has very low resource requirements post-deployment: for instance, the typical user to administrator ratio is 1,000:1. The solutions installation can be phased. In addition to the main solution, Rally provides four add-on modules: Rally Quality Manager, Rally Product Manager, Rally Support Manager, and Rally Integration Hub. Rally Unlimited does not depend on any third-party tools or other Rally products. The solution facilitates data and workflow integration through the Rally Integration Hub (add-on module). Rally provides training and consultancy services at customer premises and through public courses, as well as through telephone and web conferencing. The vendor conducts three core workshops: Agile Practices and Agile Rollout Planning (aimed at facilitating Agile adoption), and Rally Product Implementation (for product training). Rally has also created an online resource center, known as Agile University, for public training, accessible at www.Agileuniversity.com. The company provides online and offline technical support through an online knowledge center, where users can search for the resolution of specific problems and also log support cases. The Agile Commons community, accessible at www.Agilecommons.org, allows end users to participate in forum discussions, view Rallys release roadmap, offer feedback, and download connectors to third-party ALM tools. Rally also provides online training modules that include role-based and task-based demonstrations.
250
DEPLOYMENT EXAMPLE
Research in Motion
The Blackberry maker has deployed Rally ALM on-premise for around 1,700 users. The solution has been rolled out for multiple departments in the organization. As and when new business units decide to adopt Agile practices, Research in Motions (RIMs) central tools group arranges for Agile training and user provisioning. The environment also contains tools from FeaturePlan and MKS (used for higher level planning), which are tightly integrated with Rallys platform. Rally Software 4001 Discovery Drive, Suite 220 Boulder, CO 80303 USA Tel: +1 (303) 565 2800 Fax: +1 (303) 226 1179 Email: info@rallydev.com www.rallydev.com
251
Incorporating
OVUM Butler Group
SERENA: Serena ALM products
WWW.OVUM.COM
TECHNOLOGY AUDIT
Serena
Serena ALM products
SUMMARY IMPACT
Over the past year, Serena has reaffirmed its direction as a provider of tools supporting application lifecycle management (ALM) processes. It has repurposed integration technology initially developed for its old TeamTrack issue and defect tracking tool into a broader process orchestration engine that integrates the bulk of its ALM tooling. Serena has organized its tooling into three suites, covering demand management, development management, and release management, with a business management tool that is used for visually defining and orchestrating application lifecycle workflows. Serena Business Manager (SBM) is the backbone tying together its development tooling. Besides providing a process management and integration framework, it also provides common dashboards and audit trails tracking the development lifecycle. Serena Release Automation breaks new ground, not only in managing the build process, but in physically provisioning software installation via a technology partnership with Nolio.
KEY FINDINGS
Strengths: Single integration framework enables IT organizations to orchestrate software development life cycle (SDLC) processes across multiple tools and tasks. Visual designer supports higher level, process-oriented approaches to managing the application lifecycle. Established presence in mainframe Source Code Management (SCM) that has grown its base to distributed systems. Weaknesses: User experience and licensing not unified; solution relies on installation of multiple tools. Release Automation is new, version 1.0 solution that relies on a technology partner. Lacks native test management functionality. Key Facts:
i Company returning to its roots following a period where the company focused
on building a mashups-on-demand business.
i Serenas Current direction leverages orchestration technology that originated

from its older TeamTrack issue and defect tracking tool, which was also utilized for its former mashups business.
CHAPTER 8: SERENA SERENA ALM PRODUCTS
255
OVUM VIEW
Over the years, Serena has leveraged a lucrative business in mainframe source code management to develop and acquire tooling for managing key tasks across a variety of platforms. The company took a detour after being taken private by Silver Lake Partners in 2005, positioning itself as a provider of cloudbased business mashup development that was striving to model much of its business on the Salesforce.com SaaS (Software-as-a-Service) business model. Although the spotlight was directly on the business mashups business, Serena never abandoned its core business, managing software change and configuration management; requirements management; and portfolio management. Significantly, the orchestration technology that originated with its TeamTrack defect and issue tracking tools and the former Eclipse Application Lifecycle Framework (ALF) open source project is now providing the core orchestration backbone of Serenas ALM business. Serenas new orchestrated ALM strategy uses SOAP and RESTful services to support single-sign-on to a management environment that is built on a standards based, event-oriented architecture. Serenas goal is to enable IT organizations to manage the application lifecycle as a set of configurable business processes. While customers can still buy Serenas products as point tools, acquiring them as packaged solutions enables them to take advantage of the workflow automation made possible by SBMs orchestration capabilities.
Recommendations
Small enterprises Serena offers cloud-based project and portfolio management (PPM) and agile planning solutions, backed by the SBM integration framework, which are competitive with workgroup-level tools. Arguably, smaller teams that implement these solutions may not need the process-oriented integration capabilities (they are not the sweet spot of Serenas target market). Medium and large organizations Unlike most rivals, Serena offers tools that manage the lifecycle of software development across all platform targets, from mainframe to distributed; the benefits are especially acute for organizations large enough to develop software for multiple platforms. The benefits of Serenas SBM orchestration, which applies process-oriented management of the application lifecycle, are best suited for larger IT organizations that can reuse the ALM process orchestrations across legacy and distributed platforms that are designed with Serenas tools. PRODUCT ChangeMan ZMF Dimensions CM Dimensions RM Serena PPM (formerly Mariner PPM) Serena Agile Serena Business Manager Serena Prototype Composer Serena PVCS Serena Mashup Composer VERSION 7.1 2009R3 2010R1 2010R1 2010R2 2010R2 2010R1 8.4/Pro 11 2010R1 FUNCTION Mainframe source code management Multi-platform software change and configuration management Requirements management Project portfolio management Agile project management ALM process management Requirements definition for prototyping user interfaces Software change and configuration management Visual design environment for assembling mashups
Source: Serena
Table 5: Serena products (partial sampling)
256

Serena has a long list of point tools, reflecting the companys diverse heritage. Traditionally known as a provider of mainframe software change and configuration management (SCCM) tools, the company has long competed with CA, as both offer SCCM tools that are native and heavily leverage the security features of the mainframe. Unlike CA, Serena has subsequently broadened its offerings through organic development and acquisitions to target distributed platforms as well. Today, most of its mainframe deals also include distributed platforms. The product portfolio is listed in Table 5. However, as Ovum believes that the traditional point tool approach adopted by most IT organizations typically results in highly siloed software development processes, the analysis in this report will focus on how Serena is packaging its tools into broader solutions that leverage its recently developed orchestration framework (see Figures 50 and 51).
Serena Orchestrated ALM Architecture

Demand Mgmt Demand Mgmt
Work & Project Mgmt
Release Mgmt
Requirements Mgmt
Change & Config Mgmt
Quality Mgmt
Release Control
Portfolio Analysis Request Mgmt
Release Vault Release Automation
SBM Orchestration Process Intelligence Audit Trails
User Interface Layer Process Application Layer
Process Integration Layer

HP Quality Center IBM Rational Microsoft SharePoint Other rd 3 Party Mainframe Windows Unix Linux Cloud
Figure 50: Serena Business Manager orchestration
Source: Serena
Serenas ALM solutions revolve around SBM, a process orchestration-based integration capability that is described in more detail in the solution analysis section. As Serena has built this solution around multiple existing products, it relies on a federated architecture in which each tool carries a repository for its own artifacts. Serena and its partners have been responsible for developing integrations to thirdparty tools such as HP Quality Center, Subversion, Microsoft Project, IBM Rational DOORS, and BMC Remedy. Figure 51 shows how Serena is organizing its new ALM suites.
257
Demand
Portfolio
Project Requirements (Traditional)
Project (Agile)
IDT/ Work
Change & Config
Build & Deploy
Release Mgmt
Demand Management
Request Mgmt (SBM) Dimensions RM Mariner PPM Prototype Composer
Agile Project Mgmt Prototype Composer IDM & Task Manager (SBM)
Configuration Mgmt (Dimensions CM)
Release Mgmt.
Release Vault
(Dimensions CM)
Release Control (SBM)
(PVCS) Professional Developer Prototype Composer IDM (SBM)
Agile
PVCS VM Subversion Release Automation (Nolio) Subversion
(Subversion) Standard Developer Prototype Composer IDM (SBM)
Agile
Figure 51: Serena ALM solution packaging
Source: Serena
SOLUTION ANALYSIS
Integration framework
SBM is the heart of Serenas integration backbone. It uses the integration capabilities that Serena initially developed for the now-closed Eclipse Application Lifecycle Framework (ALF) open source project; having been used at a number of customer engagements, Serena has recently productized it. SBM also includes a visual, process orchestration development and management environment that was the byproduct of technology in its old TeamTrack defect tracking and issue management tool. It is used by application development organizations to design their application lifecycle processes. SBM repurposes front-end presentation technology from Serenas Mashup Composer to provide visual dragand-drop capabilities for assembling applications from components that are pre-populated by IT into a repository. Beneath the hood, the orchestration engine uses a service-oriented architecture (SOA) (SOAP and RESTful) services stack that exposes processes and enables tools to exchange status updates and, where appropriate, development artifacts. (Note: Serena Mashup Composer was part of the older Serena Business Mashups offering, which was also abbreviated as SBM while Serena was actively promoting that business. Because Serena is now using the SBM acronym to emphasize its current Serena Business Manager product direction, our use of SBM in this report will refer to the current product. At the time of writing, Serena was updating its website, de-emphasizing references to Serena Business Mashups). Serena has used its integration framework to integrate its own tools, but those of third parties; additional SOA adapters are available for HP Quality Center, Microsoft Project, Subversion, and IBM Rational DOORS; integrations are also supported for most major third party SCCM systems. At one customer engagement, the customer already used Serena Dimensions CM, ChangeMan ZMF, and Aldon for source code management on distributed systems, mainframe, and IBM System I, respectively. Within a few weeks, Serena deployed SBM to connectors to CA Clarity (PPM) along with HP Quality Center, IT Financial Management (ITFM), IT Service Desk, and Universal CMDB; excluding HP Quality Center, the connectors were custom developed.
258
Defining an ALM process orchestration involves several steps. First, an activity, such as submitting a requirement in Dimensions RM or committing a source code update in Dimensions CM, are exposed as SOAP or RESTful services; in turn, SBM is then used to orchestrate the web services that have been defined in the Serena tools using BPEL (Business Process Execution Language). Those web services provide alerts that appear in the users tool (they do not have to go out of their working environment into a third-party tool) to show their task list. In some cases, it may also automate the task, such as triggering a pop-up approval workflow that may be necessary for committing a new or modified requirement or source code fragment. Using SBMs orchestration capability, a process could be defined to specify that: Once a business analyst checks a requirement into Dimensions RM, other team members are notified so that they can peer review the requirement. Once the requirement is approved, an alert is sent to the user of HP Quality Center to generate a test case. Once the code passes its tests, build tasks are created. Status changes in the SCCM system generate alerts that update project management. Obviously, as code may be logically linked to multiple requirements and test artifacts with builds, in turn, associated with multiple code artifacts SBM can define many-to-many relationships. Once the process is designed, SBM is used to track and audit its execution. It captures planning data from Serena PPM (the former Mariner product) that tracks requests, projects, demands and priorities; project execution from Serena Agile; code changes from Dimensions CM; the release status from planning; and so on. The user interface is a web-thin client. SBM competes with offerings such as IBM Rational Team Concert, which includes an orchestration and workflow engine, plus optionally, a source code repository. It also competes with, and potentially complements, Tasktop, which offers simpler task-oriented interfaces that is triggered by check in/check out activities from an SCCM system. Serena claims that SBM is differentiated because it provides a higher-level, business interface; SBMs visual process configuration environment does provide the ease of use that is suited for business people. While Serena sometimes uses the term business process management, in practice, this is not BPM per se. SBM is not intended as a general purpose process modeler for the enterprise, and does not use notations or languages (e.g. Business Process Modeling Notation (BPMN) or XML Process Definition Language (XPDL) that might be familiar to business process architects (it uses its own proprietary notation instead). This is a shortcoming that Serena can rectify (by offering a save as translation option), as this is not a feature that would fall under a nice to have category. Serena is hardly alone in failing to offer a BPMN process modeling language interchange with application lifecycle processes. At present, neither IBM Rationals modeling tools, nor those of HP and its partner Blueprint systems, bridge the divide (the same is true for MKS and others). Serena states that it has seen little demand from its customers for this capability, an observation that Ovum has also heard from Serenas ALM rivals. Although there has been no overt demand for either translating requirements or software development workflows to BPMN or other process modeling languages, there is latent demand. Ovum knows of many organizations in regulated industries that are facing the need to document their software development processes as part of their compliance activities. They may not be demanding BPM integration (via BPMN 2.0 support) yet, but having the ability to document and manage these processes in a mainstream BPM language would be a logical step towards that goal. Because BPMN was designed specifically for interchange, both with proprietary BPM modeling languages and BPEL for execution, it could fill that gap when it comes to documenting if requirements are documented and tested according to policy or regulatory mandate.
259
The lack of customer demand or ALM vendor support for bridging to the business process world is attributable to the silos that continue to exist in most IT organizations; ALM tools have been marketed toward software development organizations, while BPM tools have been targeted at business process architects. The challenge to ALM vendors Serena included is one of market development, not technology.
Demand Management
Serenas Demand Management is comprised of request management, project management, and requirements management. Projects that are greenlighted become part of the requirements management workflow (requirements management falls under another package, Development Management). The product bundle integrates Serena PPM (formerly Mariner PPM) and Dimensions RM (requirements management) with the SBM framework. (Note: Prototype Composer, which simulates the look and feel of new applications, is a free, optional add-on; Serena no longer actively competes head-to-head with iRise, which leads this narrow market segment). Serena PPM provides financial and resource management for analyzing the project portfolio. It allocates resources across hundreds of projects by performing critical path analyses, planning and scheduling; evaluating work demand against resource capacity; providing risk management, and conducting failure mode and effect analysis (FMEA). It accepts updates from project management that may come from any of the following sources: The project management capabilities inside Serena PPM;. Serena Agile, Serenas agile project planning tool; OpenProj, Serenas SaaS-deployed project manager, which was originally part of the hosted Serena Business Mashups service; and/or Import/export files with Microsoft Project. In turn, Serena PPM uses SBM for SOA integration with Serena Dimensions CM for SCCM and release management; Serena Dimensions RM for requirements management; and third-party tools for which SBM SOA adapters are available.
This is more of an umbrella offering that bundles different combinations of Serena tools, spanning: Project management Customers have a choice of using Serena PPM to handle the project management side, or to work with Serena Agile (for agile projects), OpenProj (for project management on demand), or Microsoft Project through file import/export. Workflow management Via Serena SBM. Source code management Customers have a choice of using ChangeMan ZMF for mainframebased environments; Dimensions CM or PVCS for enterprise- and workgroup-level SCCM, respectively; or third-party source code repositories such as Subversion for workgroup level. PVCS still has a larger installed base than Dimensions CM. Serena also has interfaces with Microsoft Team Foundation Server (TFS), IBM Rational ClearCase, and even HP Quality Center (although not intended as an SCCM system, at least one of Serenas clients employs it for that). For quality management, Serena supports the management of test cases that are generated in response to requirements, but leaves execution of test plans to third parties. Like most ALM providers lacking critical mass QA functionality, Serena supports HP Quality Centers web services API. Serena can maintain test plans inside Dimensions RM or link to HP Quality Center; Dimensions RM can also trigger the generation of test cases inside HP Quality Center through HPs web service API. Regardless of where test cases are physically stored, they are visible inside Dimensions RM, which can perform traceability and gap analysis.
260
Release Management
Thanks to a new partnership with Nolio, Serena now addresses three elements of release management, encompassing: Release Control Using SBM, this manages the workflow for generating and approving builds for final release. This entails planning, approving, and then scheduling releases. Release Control can be used for documenting the process through which releases are cleared, tracking time and costs, and auditing releases on a planned and ad hoc basis to prevent policy or compliance violations (this is especially critical in regulated industries). Originally supporting ChangeMan ZMF only, the recent 2.0 release has added support for Dimensions CM. While ChangeMan ZMF has support for build scripting, Dimensions CM does not yet integrate to popular scripting tools like ANT or Maven. Release Vault This uses Dimensions CM or ChangeMan ZMF to store version-control-approved builds for final release. It also associates related artifacts, supporting traceability including lineage of the code, when it was approved, by whom, and why for maintaining an audit trail of approvals. Release Automation This is the new piece that OEMs a tool from Nolio, a three-year-old startup that was founded by veterans of Mercury Interactives application performance group. It picks up where the release-queue leaves off, automating the workflow of physically provisioning and deploying software onto server targets. It includes a workflow designer, a publisher (which stores the release workflows), and an execution module (a management capability is under development). The workflows are designed to conduct all of the checks and isolate all of the conflicts, factoring the dependencies and idiosyncrasies of complex, multi-tier applications that may be deployed to a varying mix of physical, virtual, and cloud environments. Serena Release Automation, based on the Nolio technology, addresses a long overlooked gap between the creation of software builds and the physical provisioning onto target platforms; this is a major inflection point for DevOps.
PRODUCT STRATEGY
Serena is still clearly in the midst of a transition as it in essence refocuses on its roots in the enterprise software development space. It has embraced tooling federation with its integration and processfocused backbone, but still has work to do in consolidating the installation and unifying the user experience of its lifecycle tools. It also needs to update its messaging, as on its website many of the old product names still shine prominent.
MARKET OPPORTUNITY
From our 2011 Trends to Watch report for ALM, Ovum has identified the following opportunities: With Agile now in the mainstream, new pragmatic approaches focused around mixing and matching the right methodologies will emerge and become supported by ALM tool vendors. Demand for cloud deployment will prompt SaaS-based ALM offerings, not only from new entrants, but from established players as well. With the explosion in mobile application development started by Apples iPhone/iPad and Googles Android platform, smart mobile will become a major target for enterprise applications. Security testing is entering the ALM mainstream. Embedded software development for engineered products is on an exponential growth curve and represents a lucrative market for the few ALM providers that have chosen to target it. In 2010, DevOps entered the awareness phase in the ALM market, and 2011 will see automated tooling targeting the needs of DevOps and Agile release management in particular.
261

Serena Agile is a lighter weight planning alternative with a tool specifically designed for managing agile projects. More importantly, it carries several features that are geared toward scaling-out agile beyond individual teams, including the ability to view and manage multiple projects, shift resources across projects, compare risks for competing user stories, and manage multi-project dependencies hierarchically.
Cloud
Although not the first ALM vendor to embrace the cloud, Serenas Business Mashups was heavily promoted for on demand deployment (it was also available in on-premises mode). It gave Serena experience that has translated to hosted offerings for planning and collaboration (as opposed to source code-centric) offerings. Not surprisingly, having inherited technology from Serena Business Mashups, Serena Business Manager is also available in a hosted version; so is Serena Agile and PPM. The Dimensions products, however, are not available hosted, although Dimensions RM would be a logical candidate because of the collaborative nature of requirements management.
Mobile
Like most ALM vendors, Serena has yet to decipher whether its tooling requires any adaptation for mobile development and deployment. With its support of embedded development, Serena has already attacked one piece of the puzzle: providing lifecycle support for software development of mobile devices. However, aside from generic capabilities for managing requirements and change, Serena does not currently offer specific support for mobile development frameworks or form factors.
Security testing
Serena has not yet targeted this area. Customers have developed custom integrations with static and dynamic testing security testing tools, but Serena has yet to develop capabilities for autopopulating defect tracking (except for incidents reported through BMC Remedy) or providing impact analysis in PPM or requirements management. However, there are no barriers in SBM to creating security management processes, although, for now, Serenas management offerings (e.g. PPM, requirements management, release management) do not offer explicit support.

Although not as entrenched as IBM Rational (through the acquired Telelogic products), nor as heavily exclusively focused on this area as, MKS, embedded systems is one of Serenas three strategic markets. Ovum believes that embedded systems comprises a small, but significant portion of Serenas revenue, and is an area where the company will be increasing its focus in 2011. Serena has built presence with several high-profile aerospace clients through integrating requirements management, SCCM, and automated release with product lifecycle management (API to Siemens TeamCenter is supported); model-based development (APIs to MATLAB and Simulink); and real-time operating systems (developers can check code in and out of Dimensions CM from Wind River Systems Eclipse-based workbench). It also offers vertical industry process templates such as support of DO178B, a standard for certifying safety-critical software development practices in the aerospace sector.
DevOps
Serenas DevOps support is in two areas. Like IBM and MKS, it supports the capability to autopopulate defect/issue tracking from trouble tickets generated by the IT service desk; in this case, it is through a web services API to BMC Remedy, where problems can be automatically submitted directly to Dimensions CM or through SBM. Unlike rivals, Serena is starting to venture into DevOps integration in the release process, courtesy of the relationship with Nolio. However, at this point, the solution is extremely preliminary, focusing on physical software provisioning and deployment and at that, through loosely coupled web services APIs to separate tooling.
262
The Nolio tools that Serena OEMs for Release Automation have web services-based integrations with Serena SBM (as the release management engine), and web service or call-level interfaces with CMDBs, IT service desk, SCCM, and build automation systems. Nolio has so far built links with Nagios, HP, and CA (for application performance monitoring systems); Hudson (for continuous integration); and Serena and Subversion (for SCCM). For Nolio, there is the need to flesh out management of release automation. For Serena, the opportunity and challenge is twofold. There is plenty of opportunity to develop targeted process templates for poorly understood use cases such as public or private cloud that are growing in demand, and to extend these templates back from SCCM to the requirements and PPM level. More importantly for Serena, there is the need to protect its new stake in Release Automation; the relationship with Nolio is nonexclusive, which will be perfectly sufficient until one of Serenas rivals swoops in and makes an acquisition offer. Release Automation breaks new ground for Serena and for ALM providers, and Ovum will be surprised if Serena does not acquire Nolio within the year.
Serena currently has 3450 customers under maintenance. It emphasizes direct sales channels, with local offices and sales teams in most major markets globally. Its direct sales model combines field sales, inside sales, and field-based technical sales resources. Serena partners with local distributors in emerging markets including Asia Pacific, Latin America, Eastern Europe, and the Middle East. Typical deals average 2:1 ratio of software license to services (systems integration and enablement). Deal sizes are often as follows: Entry level Around $30,000 for a solution including PVCS and SBM; however, entry level agile projects can begin at as little as $10,000. Average Around $120,000 for Dimensions-based solutions for distributed platforms; crossplatform or mainframe deals often average $350,000. Largest Global cross-platform ALM project (including Mainframe) can easily exceed $1 million over a period of two to three years for initial implementation and rollout. The ongoing (services) cost is smaller after that, as the maintenance and improvements are mostly carried out by the customer, with only a little help from Serena Service.
Licensing
Serena offers the traditional perpetual, named, or concurrent user license deals. Annual maintenance contracts (at 21% of software license costs) are optional after the first year. For mainframe platforms, term licenses based on MIPS, LPARs, and number-of-users are available. For SaaS-based deployments of SBM, Agile, and PPM, Serena offers subscription pricing.
RIVALS
Serenas primary rivals include: IBM Rational, the market leader, which has heavy presence with large enterprises and strong presence with engineering companies developing embedded software. IBM Rational and Serena offer similar, federated integration frameworks. Microsoft, primarily in companies with sole focus on Windows and using Visual Studio.NET MKS, primarily in embedded systems development CA, only as a competitor in mainframe SCCM against ChangeMan ZMF. CollabNet, whose Subversion SCCM system has strong presence for global development organizations preferring open source.
263
IMPLEMENTATION
While Serena maintains its own professional services organization, it partners with system integrators and resellers that can support the technically complex nature of ALM implementation and process mentoring.
DEPLOYMENT EXAMPLES
Media company
A New York-based cable television provider and media firm on average delivers over 100 projects per year; these projects deliver services to subscribers. Their challenges were: Tracking and prioritizing candidate projects. Ensuring that each candidate project is analyzed and evaluated in a consistent manner. Gaining real-time visibility of costs, resources, schedules, and status in a cost-effective way. The company uses a best-of-breed approach for managing the application lifecycle. It uses Subversion for source code control; Serena PVCS Pro for staging and Gold vault for managing releases; HP Quality Center for testing; and SBM for managing the deployment process. The budgeting process starts with a business requirements document (BRD). The company initially used SBM to manage the budgeting/approval process, but faced bottlenecks because project status was reported manually. It has since adopted Serena PPM to improve control over project tracking, resource management, and capacity planning. Integrating demand management became a competitive differentiator, as improved project visibility enabled the company to proactively address problems that would otherwise have delayed and increased the cost of delivering new services to subscribers.
Global financial institution

With a long history of software development much of it on the mainframe this global financial institution had long-relied on homegrown tools to support the building and deployment of software releases. The resulting pain points included: Inability to scale with the organization With plans to expand to more divisions, resulting in the need to manage development of more applications, the organization needed a more modern ALM solution. Labor intensive and costly Existing processes required a lot of manual steps within the overall deployment process, requiring significant resources during nights and weekends to get the job done. Poor visibility and control Because much of the activity was manual, it was hard to determine what exactly was deployed, and in which environments. There was almost no traceability or control. Code freezes were difficult to implement due to a lack of enforcement. The company deployed Serenas Release Management capabilities based on Dimensions CM for deploying software to the mainframe and distributed platforms. The overall configuration workload in the application and project teams was reduced by almost 80%. Serena Software Inc. 1900 Seaport Blvd. 2nd floor Redwood City, CA 94063-5587 US Tel: +1 (650) 481 3400 www.serena.com Serena Software Europe Ltd. Abbey View, Everard Close St. Albans Hertfordshire, AL1 2PS UK Tel: +44 (0)1727 812812
264
Incorporating
OVUM Butler Group
TECHEXCEL: TechExcel DevSuite v8.3
WWW.OVUM.COM
TECHNOLOGY AUDIT
TechExcel
TechExcel DevSuite v8.3
SUMMARY IMPACT
Developing complex business and technical applications on schedule and on budget is a major challenge for many organizations. TechExcel offers an integrated application lifecycle management (ALM) platform that is suitable for Agile and non-Agile projects and processes, as well as environments that combine both methodologies. The platform addresses the key issues facing software development today, including the need to: Provide a consistent view of requirements, with requirements driving ALM. Support Agile projects, adapting to changes in requirements in a controlled manner. Enhance project progress visibility across all phases of product development.
KEY FINDINGS
Strengths: Tight integration across modules with web-based interfaces. Repository for unstructured data closely integrated with all modules. Highly scalable architecture can handle many thousands of users. Good workflow capability with pre-built Agile templates. Weaknesses: Portfolio management module does not offer financial management, demand management, or risk management capabilities. Lacks the capability of scheduling and automated execution of tests. Key Facts:
i Available as a hosted offering through a third-party service provider. i Seamless integration with TechExcels IT service management and customer
relationship management solutions.
OVUM VIEW
The need to roll out high quality software products within short timeframes and, more importantly, to thrive in an increasingly competitive market means that development teams need to be more Agile, tactical, and adaptive in their approach. In addition, there has to be seamless collaboration between project management and product development, functioning in a globally distributed team structure. TechExcels solution is suitable for this scenario and is specifically designed to achieve a balance of effective management processes and repeatable application development processes. Under the DevSuite v8.3 umbrella, TechExcel provides a suite of closely integrated modules with which to manage the application development process, from gathering and managing requirements, through to planning the development process, tracking issues, and managing testing processes. The key theme of TechExcel DevSuite is tight integration across modules that support individual phases of development.
CHAPTER 8: TECHEXCEL TECHEXCEL DEVSUITE V8.3
267
There is also a focus on visibility into relevant unstructured data in the form of ideas related to features, enhancements, and overall specifications throughout the development lifecycle. Ovum likes the suites scalability, as well as features such as customers/end users being able to submit defects during beta testing. Overall, it is a suite with good support for industry standards and will be increasingly relevant to all organizations with a significant application development function. TechExcel DevSuite is suitable across all industries and offers features that align the product well with the requirements of its target market. Strictly speaking, the solution is not comparable to the ALM leaders in terms of feature depth and coverage (for example, DevSuite lacks a portfolio management module). However, at this point, TechExcel does not indicate a wish to expand its range, and is satisfied with providing the right balance between functionality that has been built organically and connectivity to third-party products for missing ALM segments. In partnership with Rackspace, the company is also pursuing a hosted strategy for its ALM suite, targeted at small developer teams, with enhanced software-as-a-service (SaaS) features planned for first half 2011. Hosted ALM is gaining traction in the market, and TechExcel is well positioned to capitalize on this trend.
Recommendations
Software vendors and in-house software development teams DevSuite is applicable for many mid-sized and large enterprises that aim to optimize their software development processes. Its highly scalable architecture can support thousands of developers and is a strength recognized by customers, particularly in the electronic gaming market, where TechExcel has a high profile. Small enterprises with relatively simple development projects can opt for the more affordable hosted version of the solution or pick point solutions from the TechExcel stack and build their ALM solution incrementally. Organizations with distributed software development teams DevSuite supports distributed teams through its web-based access, and the soon-to-be-released database replication capability will unify global teams local repositories. The focus is on providing a development platform for large distributed teams, necessitating universal and continuous visibility into overall client requirements, which are very frequently not well defined. Organizations intending to adopt or already practicing Agile methodologies TechExcel DevSuite is a suitable ALM platform for companies of any size looking to adopt Agile development methodologies, with features such as visual workflow, pre-built Scrum templates, Agile charts, and support for hybrid Agile processes.

Built around a knowledge repository, the suite supports the complete software delivery process by providing modules from requirements management through to testing management. The DevSuite solution comprises four modules: DevSpec, DevPlan, DevTrack, and DevTest. In addition, TechExcel offers Agile Studio, a re-branding of what was previously Project Studio (a combination of DevPlan and DevTrack) while DevTest Studio is an integrated offering constituting DevTrack and DevTest. The supporting components are KnowledgeWise, which acts as the knowledge repository for all modules; and the optional VersionLink, which allows developers to integrate DevTrack with common Software Change and Configuration Management (SCCM) systems and TestLink for integrating DevTest with test management solutions. See Figure 52 for a schematic illustration of the DevSuite components. KnowledgeWise a crucial component of the suite is the centralized repository that captures all unstructured as well as structured data, from pure product ideas to customer feedback, and is the underlying knowledge engine that supports the entire suite of products. KnowledgeWise also includes a wiki that facilitates collaboration between distributed teams, provides content access through a browser, and offers external users read, write, and edit features.
268
Implement Design
lid Va o ati n
Co
nc
ep
DevTest
KnowledgeWise
Implementation
Requirement
DevTrack
Knowledge
DevSpec
VersionLink
DevPlan
SC
Figure 52: TechExcel DevSuite architecture
Pla
in nn
Source: TechExcel
The new wiki book functionality allows training documents and product manuals to be published to external users. The repository access is controlled through security privileges at project level, folder level, and item level for read, write, and viewing access. Through this common set of knowledge access points, teams from executive management through to delivery management have the visibility and access points appropriate for their environment and role. Its optional Auto-Suggest feature automatically suggests relevant topics based on user-specified criteria. KnowledgeWise analyzes the keywords and field choices of an issue or email, and presents the team member with relevant matches. It also allows teams to easily build a knowledge base by adding relevant artifacts. The existing knowledge base can also be expanded by linking to other knowledge management systems, by populating KnowledgeWise with third-party knowledge packs from vendors such as KnowledgeBroker, or by integrating with RightAnswers. TechExcel DevSpec is the requirements management component that provides a framework with which to create new requirements, specifications, and features that can be linked to development and testing implementation projects.
269
DevPlan is an integrated project management and resource planning tool that helps plan resource capability and schedule events, and ensures accountability for all phases of product design and development. The module provides representations such as Gantt charts, schedule charts, and event charts for efficient project allocation. Meanwhile, the test management module, TechExcel DevTest, spans all testing phases from test planning and team management to analyzing the test results. DevTest enables users to create and manage release and test cycles, plan and assign test tasks to testing teams, execute test coverage, and submit product defects. DevTrack, the issue tracking tool and the module with the best market presence, has a good workflow component that automatically enables the routing of issues to appropriate users and escalating issues to notify appropriate engineers or issue owners via email. All of the modules are tightly integrated; for example, an issue can be submitted to DevTrack and its lifecycle can be monitored from within the DevTest tool. DevSuite supports Agile software development methodologies through its visual workflow, including a pre-built Scrum template, and a hybrid Agile process. There is also a capability maturity model integration template for risk management. Source code can be quantified according to a range of metrics: points, time, function points, or lines of code. Agile charts, such as velocity, burn-down, and burn-up, can be produced, as well as cumulative flow.
SOLUTION ANALYSIS
Requirements definition and requirements management capabilities
For requirements definition, TechExcel has a license to provide the open source tool FreeMind, which supports visual representations (such as MindMaps) that can be used to capture requirements, and which integrates with DevSpec and KnowledgeWise (facilitating collaboration through wikis and blogs). The solution features a built-in tool to convert unstructured data into a structured data format, and has good traceability features to track the changes in requirement specifications. TechExcel DevSuite 8.3 now features full Unicode support, thereby enabling use of any character set or language. TechExcel DevSpec is specifically designed to provide visibility, traceability, and validation of the product or project requirements. DevSpec contains details on specifications, architectural diagrams, and formal design documents, and allows users to view knowledge details, thus helping them to track the entire set of artifacts relevant to the selected project and understand the project flow. It also allows users to search from the existing repository, controlled by restricted access. It allows the gathering of requirements from customers and other stakeholders, and also allows the creation of a separate template for each customer, which users can view based on their privileges. Not only can requirements information be imported from MS Office plug-ins, but these can also be uploaded for each customer separately. Project managers can select developer initiatives, such as feature ideas, using a voting points system which aggregates team opinion on the most appropriate ideas. The module also has a change request function which enables users to view the estimated time of project completion and provides change alerts to team members with the appropriate privileges.
Integrated project and resource management

TechExcel DevPlan incorporates functionalities like configurable workflows, notifications, meeting requests, and process automation. DevPlan manages items such as design document and specifications, and provides event management features for team meetings and presentations. Features such as Gantt charts, resource scheduling, baseline comparisons, and reporting are provided. It allows managers and developers with scheduling features to compare the current status of the project with that of the estimated development time. The backlog support feature enables the assignment and prioritization of work. DevPlan also has access control features so that any changes made are notified to all users of the project through email (the knowledge view of the module is similar to that of DevSpec). Although the solution can manage hundreds of projects simultaneously, it does lack some important portfolio management capabilities, such as financial management, demand management, and risk management.

Development teams can use TechExcel DevTrack for identifying and resolving issues, ranging from new features ideas to minor product defects. DevTrack automatically routes issues to appropriate users, and events such as issue creation, and status-related information such as open too long, past due, or stagnant, can be automatically escalated to issue owners via email. DevTrack also contains a secure, personalized web portal, where Beta customers can submit, track, and update issues. They can also create and assign sub-issues and direct comments to specific teams.
270
The new MyWork Dashboard enables users to add issue lists, work summaries, drill-down reports, and any of the standard built-in reports such as distribution pie charts, bar graphs, and tabular reports to allow an overview of their projects. DevTrack issues can be accessed via a Uniform Resource Locator (URL) of fixed format, thereby improving the possibilities for integration with third-party systems.
Test and quality assurance management

The DevTest module enables users to manage release and test cycles, plan and assign test tasks, and schedule and automate the execution of test cases. Interestingly, users can automatically link test tasks to the product defects. Integration with DevSpec allows users to automatically generate test cases for change requests and new requirements, thereby supporting requirements-based testing. However, the module is unable to schedule and automate the execution of tests. The solutions reporting capabilities enable users to analyze testing trends, team performance, and defect-capture efficiency. The test templates facility allows tests to be reused.
TechExcel DevSuite is an n-tier client/server web service-based application wherein the data are stored in a relational database management system (Oracle, Microsoft SQL Server, or Oracle MySQL). An application server provides sequence enforcement and maintains data integrity, and clients connect to this application server and database through web service calls. The solution also requires Microsoft Internet Information Services (IIS). Central to the solution is the KnowledgeWise repository, which contains structured and unstructured data items such as specifications, test cases, feature requests, and architecture diagrams pertaining to all applications, which are shared across all users, based on their access privileges. Every module has a knowledge view that allows access to the relevant repository artifacts.
Tool interoperability: internal and external

All of the individual modules of TechExcel DevSuite (DevSpec, DevPlan, DevTest, and DevTrack) are tightly integrated with each other. For instance, DevTrack offers test management and execution integration with DevTest Studio, wherein users can submit defects directly into DevTrack from the DevTest interface. Another interesting aspect is DevSpecs seamless integration with DevTrack, which simplifies managing project implementation into a process of creating development tasks for specifications designed in DevSpec with a definite development workflow. DevSpec integrates with DevTest to standardize QA testing by creating test case templates that are linked with one to many specifications and requirements, thus ensuring that organizations achieve requirement and specificationdriven testing. Similarly, DevPlan offers defect and issue-tracking integration with DevTrack so that users can track the whole lifecycle of an issue directly from within the DevPlan interface. DevSuite also supports integrations with third-party applications with the help of separately licensable application programming interface (API) modules: VersionLink and TestLink. Using the VersionLink module, developers can integrate DevTrack with SCCM tools such as Microsoft Visual SourceSafe (VSS), Perforce (P4), IBM Rational ClearCase, Serena PVCS, and popular open source software solutions Concurrent Version System (CVS) and Subversion (SVN). Likewise, the TestLink module provides an API to integrate DevTest with popular third-party test management systems such as AutomatedQA. Currently lacking is any model-driven development (MDD), either natively or through any integration: at a minimum it would benefit the suite to integrate with third-party modeling vendors to attract a wider customer base.
PRODUCT STRATEGY
One of the key strengths of DevSuite is its ground-up design for supporting large and distributed development teams: for example, its customers include Electronic Arts Group studios with 10,000 developers across multiple sites worldwide, and Sony Worldwide studios, which rolled out the solution to 5,000 developers.
MARKET OPPORTUNITY
From a product perspective, TechExcel DevSuites focus is on capturing early-stage requirements that are not yet well defined, and capturing development artifacts and making them universally visible throughout the lifecycle of the development process. There is also close integration between all of the core modules, ensuring that critical tasks can flow seamlessly from planning to tracking, and test management to tracking, or any of these combinations.
271
The focus from a customer perspective is on software development companies, systems integrators, game developers (a strong presence), and corporate IT development teams. The original TechExcel product DevTrack for issue tracking has significantly higher market presence and deployment cases than the other DevSuite modules, and provides an entry point for the vendors customers to adopt further modules.
TechExcel targets organizations that develop mission-critical software for their business, in particular software vendors and games development companies, as well as corporate IT departments, where managing multiple applications and complexity requires that they are simplified and brought under control. Return on investment (ROI) realization is based on improvements in developer productivity and speed of released products, subject to a level of adoption and sophistication of change management processes. TechExcel sells its products through a direct sales force, as well as strategic partners in countries where the company does not have a direct presence. TechExcel has business partnerships with Foxdata in Sweden, Razorcat in Germany, and Live RMC in Spain, with an aggressive partner acquisition strategy planned from its London-based EMEA HQ. It has technology partnerships with Rackspace (for delivering a hosted version of DevSuite), Microsoft, Oracle, and AutomatedQA, among others. The company also has strategic tie-ups with software vendors especially for SCCM integration with DevTrack including Accurev, CVS, IBM Rational ClearCase, and Microsoft VSS to name a few. DevSuite is offered as an integrated suite or as individual component modules. Licenses are fixed per user, although they can be offered in a floating concurrent format for an additional fee. Licenses can be perpetual in nature, subscription based, or leased for a limited period of time. Annual maintenance and support costs comprise 20% of the license list price at the time of purchase. The maintenance package includes 24/5 telephone, web, and email support, and all supported product updates and upgrades; 24/7 support is available for 25% of the license fees. According to TechExcel, during 2009 the average project size was around $30,000, comprising $24,000 in licenses for users (spanning business analysts, product managers, project managers, developers, testers, and management) plus $6,000 in support and implementation fees, and typically five days for training. TechExcels next major release scheduled in the first quarter of 2011 will incorporate an enhanced graphical user interface, improvements in KnowledgeWise to support IdeaSync Engine, the introduction of single log-in for all modules, and DevSuite as a full SaaS-based solution. In subsequent releases, TechExcel plans to introduce active database replication for KnowledgeWise, allowing distributed teams to work against local repositories while replicating data to other sites.
IMPLEMENTATION
According to TechExcel, the preferred individual module combinations are DevSpec with DevTrack, DevTrack with DevTest Studio, and Agile Studio with DevTrack. Many customers have deployed VersionLink to link DevTrack and existing defect tracking systems onto their preferred software configuration and change management solution. Post deployment, a TechExcel designated resource helps to allocate permissions, manage user accounts, and assist in system configuration. TechExcel claims that, in most cases, administration is a part-time responsibility, requiring less than 20% of the individuals work time (although for larger installations it is common to have multiple administrators). TechExcel provides varying levels of training based on the desired usage of the product, including webbased classes, on-site services, and classroom-based courses for a fee. The company also provides complimentary, on-demand seminars to educate client organizations on key features. TechExcel has a product management team that hosts weekly live webcasts to illustrate configuration and usage settings.
272
DEPLOYMENT EXAMPLES
Electronic Arts
Electronic Arts, one of the largest gaming companies worldwide, needed a defect tracking and overall standardization solution to handle its complete development lifecycle. The company now uses the DevTrack and DevTest modules, which has helped to standardize processes in the development and quality assurance departments and increase overall productivity, as well as providing a solid foundation for the companys overall processes. This is one of the largest implementations for TechExcel, with the solution being used by more than 10,000 users worldwide.
Gael
Gael Quality, a trading division of Gael Ltd., has been using Microsoft Excel to document tests and record results, and an in-house Microsoft Access application for defect tracking. The company needed a solution which would be able to manage thousands of test cases across the various teams and integrate the results of both manual and automated tests, in addition to fully integrating results from external sources such as TestComplete. Another important aspect was to have traceability back to the initial test requirements. DevTest Studio was the preferred solution, and was implemented within a week. Gael Quality has till date defined close to 15,000 master test cases and executed more than 117,000 test cases using DevTest Studio. Most of these test cases have been executed by TestComplete and the results fed back to DevTest by its custom execution framework.
Keyfactor
Keyfactor, one of Germanys independent gaming service providers, needed a solution that could be used for projects of different sizes while supporting remote clients and developers. The company implemented DevSuite and claims that the whole system was up and running in just a couple of weeks. The company now manages the entire project development lifecycle using DevSuite and has successfully completed more than 10 projects using the solution. TechExcel Corporate Headquarters 3675 Mt. Diablo Blvd., Suite 200 Lafayette CA 94549 US Tel: +1 925 871 3900 Fax: +1 925 871 3991 Email: sales@techexcel.com www.techexcel.com TechExcel EMEA Crown House 72 Hammersmith Road London, W14 8TH UK Tel: +44 (0)20 7470 5650 Fax: +44 (0)20 7470 5651 Email: emeainfo@techexcel.com
273
Incorporating
OVUM Butler Group
THOUGHTWORKS STUDIOS (A DIVISION OF THOUGHTWORKS, INC.): ThoughtWorks Studios Agile ALM
WWW.OVUM.COM
TECHNOLOGY AUDIT
ThoughtWorks Studios (a division of ThoughtWorks, Inc.)

ThoughtWorks Studios Agile ALM
SUMMARY CATALYST
ThoughtWorks Studios current offering is an integrated suite which combines Mingle, Go (previously known as Cruise), and Twist to provide an end-to-end Agile application lifecycle management (ALM) solution. The component tools are available individually, and so the suite can be built up incrementally. ThoughtWorks Studios also offers structured training courses on Agile methodologies, and its parent company ThoughtWorks Inc. provides consulting. The products have all been built organically by the company. Mingle is an Agile Project Management solution that, in addition to project management functionality, offers Agile requirements management (RM), release planning, defect and issue tracking, and collaboration features. Go offers superior build, configuration, and release management functionality, while Twist offers test management and functional testing capabilities with the ability to execute acceptance criteria or requirements written in a natural language (English) as tests.
KEY FINDINGS
Strengths: Advanced RM functionality for Agile development and requirements-based testing. Adaptive support for multiple processes: Agile, iterative, as well as waterfall and hybrid. Thought leader in Agile, including continuous testing and more recently continuous delivery with centralized build, configuration, and release management, with single click deployment. Supports test-driven development and test scripts using Groovy. Weaknesses: Currently does not support custom workflow modeling. Twist supports only Java and web applications testing (including .NET web apps). Key Facts:
i Supported on Microsofts Windows, Linux, and Mac OS X operating systems. i Supports the Eclipse integrated development environment (IDE).
OVUM VIEW
ThoughtWorks Agile story is almost two decades in the making, as the company was an early adopter and pioneer of Agile software development and project delivery. It is recognized for its hands-on experience, which has seen it help hundreds of IT environments around the world succeed with Agile. With the release of its Agile ALM suite in 2009, ThoughtWorks Studios brought to market a comprehensive solution that provides an all-round view of the Agile software development lifecycle.
CHAPTER 8: THOUGHTWORKS STUDIOS THOUGHTWORKS STUDIOS AGILE ALM
277
Ovums vision of an ALM solution is one in which there are greater benefits from an integrated offering, such as a single workflow running across all tools. It remains to be seen whether ThoughtWorks Studios agrees with this vision, as its approach of a looser coupling is dictated by its perception of a market that prefers best-of-breed solutions. Overall, in Ovums view, ThoughtWorks solution is versatile, and will not force behavior change onto its practitioners. For a product to deliver anywhere close to its advertised benefits, it needs to co-opt the end users. Enduser adoption is usually hinged upon how flexible the product is in mirroring the way different teams work, and how well it fits into their existing environment. ThoughtWorks Agile ALM has been designed to be flexible, being suitable for different processes and team cultures, as well as facilitating collaboration. Its ease of use also helps drive user tool adoption. Within most organizations, Agile adoption changes over time: Agile at day 100 and day 300 will look different. Agile ALM facilitates this organizational transformation process by allowing users to change and optimize their processes. The product also offers project managers visibility into actual project progress. In Ovums view, these two features of flexibility and project visibility are key strengths of the solution. ThoughtWorks Studios has been acquiring customers in the highly competitive ALM market at a rapid rate, showing how traditional ALM vendors have been slow to deliver on the Agile front. Ovum believes that ThoughtWorks Studios has the potential to become a significant player in the ALM market in the years ahead.
Recommendations
Enterprises struggling with Agile adoption Enterprises that have struggled with Agile methodology adoption should consider ThoughtWorks Studios solution. Its tools can be adopted incrementally to meet business needs, and the vendor offers multiple Agile-focused workshops to help organizations embrace Agile development. Independent software vendors In Ovums view, independent software vendors (ISVs) will benefit from ThoughtWorks offering. It promotes greater collaboration in the development function, enables management of multiple development projects and environments, and provides single click automated deployment functionality. Enterprises in the embedded space, with complex requirements processes It is unlikely that these organizations will benefit from the Agile requirements definition (RD) and management capabilities offered by ThoughtWorks Mingle. As such, they may have to resort to traditional RD/RM tools.

ThoughtWorks Studios is the product division of ThoughtWorks, a global technology and software consultancy that offers custom software development and specializes in Agile projects. Agile has been the culture of the company for almost two decades now. Agile techniques, such as continuous integration (CI), were pioneered and made mainstream at ThoughtWorks. The company has also contributed significantly to the open source movement, with tools such as CruiseControl (continuous integration environment), NUnit (unit testing), and Selenium (automated web application testing). ThoughtWorks Studios was created in 2006 as a separate, standalone business unit, based on the recognition of the need for a next-generation set of commercial, lightweight tools that can support highly collaborative, often distributed, Agile teams. In addition, the three products covered in this Technology Audit namely Mingle, Go, and Twist draw heavily on ThoughtWorks experience of serving hundreds of clients. The integrated ALM suite comprising these three products provides an end-to-end Agile ALM solution, and is complementary to ThoughtWorks series of structured training courses on various Agile methodologies and best practices to facilitate Agile adoption. The products have been built organically by the vendor, and are also available as standalone offerings. With the present release, the ThoughtWorks Studios solution includes integrations across the various applications. Going forward, ThoughtWorks Studios aims to continue to build upon its vision of holistic Agile ALM.
278
Mingle, currently in version 3.3, is an Agile Project Management solution that flexibly adapts to the way in which development teams work. In addition to project management functionality such as resource and job scheduling, metrics tracking ,and portfolio-wide reporting Mingle offers release planning, defect and issue tracking, and collaboration features. The solution provides collaborative workspaces, project wiki pages, and defect and issue tracking across the lifecycle (also incorporating business/customer feedback). Mingle also makes available a macro development toolkit for creating custom reporting macros and mashups. Go offers superior build, configuration, and release management functionality. In Ovums opinion, the solution is ideally suited to solving the last mile, taking continuous integration to the next step Release Management or, as ThoughtWorks Studios calls it, Continuous Delivery. It enables centralized build management, single click deployments, integration with test management and defect/issue tracking systems, and facilitates automation of software release processes for individual projects as well as the entire project portfolio. The key aspect of Go is release modeling; users can model their release processes as is, and gain real-time visibility as a project moves from check-in, to testing, to production deployment, through the visual deployment pipeline interface. Continuous Delivery is a concept pioneered by Jez Humble, Build and Release Principal at ThoughtWorks Studios, and influenced the design of Go. Twist offers test management and functional testing capabilities. The primary advantage offered by this solution is the ability to execute acceptance criteria or requirements written in a natural language (English) as tests. This is accomplished through support for domain-specific languages (DSLs). In addition to providing efficiency gains, it allows organizations to manage larger test suites. From a functionality perspective, ThoughtWorks Studios Agile ALM has many of the essential elements necessary in an ALM system, and will integrate with third-party tools for missing elements, such as source and version control. In Ovums view, the solution is functionally superior and comprehensive compared to competing Agile ALM offerings. However, some gaps remain, and Ovum expects the vendor to address these issues in forthcoming releases. For instance, at present, the solution does not offer custom workflow modeling capabilities (the modeling capability only comprises configurable state transitions, although quite complex models can be created). Furthermore, the integration between the three products in the suite is not as strong as one would expect when all components have been built from the ground-up by the vendor. Specifically, the solution lacks a common metadata model, and each of the three products has its own workflow. A key design consideration for ThoughtWorks was to create a loosely coupled suite in which the tools can be adopted incrementally over time. There is some disparity between Ovums view of what ALM means and how ThoughtWorks treats it. While ThoughtWorks delivers a pragmatic, market-oriented ALM solution, Ovum believes that an ALM solution should deliver more than the sum of its parts. (Note: planned integrations for 2011 include Go views in Mingle, Mingle card data within Go, and event triggers across the different workflows).
SOLUTION ANALYSIS
Solution architecture and third-party integrations
ThoughtWorks Studios Agile ALM comprises Mingle, Go, and Twist. These products have been developed largely in Ruby and run on the Java Virtual Machine via JRuby. As a result, they require a Java Runtime Environment (JRE) at the client end for operation (see Figure 53 for a schematic diagram of the offering). Detailed system requirements for each of the three products can be accessed from the vendors website. The solution offers Representational State Transfer (REST) and JavaScript Object Notation (JSON)-based application programming interfaces (APIs) for integration with third-party lifecycle tools. Mingle is a client server architecture-based product, which users simply access over standards-compliant web browsers. Go 2.0 is a server/agent-based solution that deploys agents on target machines in test, user acceptance testing (UAT), and production environments. Twist is a desktop-based application. In addition to Twist and Go integrations, Mingle combines with other lifecycle tools such as Tasktop Pro, Jira, Perforce, Subversion, Git, and Mercurial, either out-of-the-box or through plug-ins (pre-built IDE integration is limited to Eclipse, and supported databases include Oracle and Postgres). Go integrates with all commonly used build and test tools that offer a command line interface. It also supports version control systems such as Subversion, Perforce, Git, and Mercurial, while IBM Rational ClearCase and the Microsoft Visual Studio Team Foundation Server are supported through third-party connectors. Twist is an Eclipse-based tool, and can therefore leverage other lifecycle tools that integrate with Eclipse, such as Bugzilla, Jira, Trac, and Mingle.
279
Test
Functional Testing
IDE, SCMs
True-to-life visibility
Build
Release
Deployable Software
Go Twist Mingle
Build Tools
Agile Release Mgmt. Agile Testing Agile Project Mgmt.

(Scrum, XP, Lean, Kanban)
Bug Tracking
Global Collaboration
PPM
Develop
Analysis = Design
ThoughtWorks Agile ALM address entire lifecycle: Development, Production, and Maintenance
Figure 53: ThoughtWorks Studios Agile ALM
Source: ThoughtWorks Studios
Process and methodology support

ThoughtWorks Studios Agile ALM supports waterfall, Agile, custom, and hybrid development methodologies, although it is principally geared towards Agile variants. The solution currently offers native support for Scrum, XP, the Dynamic Systems Development Method (DSDM), waterfall, rapid application development (RAD), Lean, test-driven development (TDD), and hybrid, iterative processes by providing templates to help organizations get started more easily. Moreover, the solution supports multiple repositories, and provides bi-directional integrations with commonly used lifecycle tools. Thus, in Ovums opinion, the solution will fit seamlessly into existing IT environments, without forcing behavior change onto end users, especially Agile practitioners.
Application project portfolio management and project management

Traceability and real-time visibility throughout the lifecycle are built into Mingle, Twist, and Go. Mingle offers shared team workspaces, through which all team members can stay up to date with project requirements, whereas Go enables development, quality assurance, and IT operations teams to come together to ensure that the software developed meets business requirements and performance criteria, right from the analysis and design stage to the production deployment stage. Users and stakeholders can trace work items, tasks, and process artifacts throughout their lifecycle and across multiple projects. This also facilitates requirements/artifacts dependency mapping, and change impact analysis. ThoughtWorks Studios Agile ALM offers visibility, not only in terms of metrics collection and reporting across multiple projects, but also in terms of giving both delivery managers and business stakeholders insight into unstructured and informal conversations among project team members. In Ovums view, these features, coupled with uncluttered, attractive Web 2.0-style user interfaces, will not only accelerate the time-to-value for all roles within the development function, but will also facilitate auditand compliance-related measures, while providing organizations with additional insight into their software development processes.
280
Mingle provides the ability to capture informal exchanges among team members through Murmurs. Murmurs provides instant messaging support, and links these conversations together, archiving them for future reference. Mingle also enables the linking of artifacts to conversations. Another notable feature is integration with Tasktop Pro, an Eclipse IDE plug-in that enables users to effectively manage their tasks within Eclipse. This integration allows Mingle users to work within the Eclipse environment, access and link Mingle cards with artifacts within other repositories, track the time spent working on a given task, link with Murmurs to capture informal conversations, and update all details back to the Mingle instance.

The solutions RD capability is contained within Mingle. Although Ovum has on occasion been critical of the below-par RD/RM functionality contained in some Agile project management solutions, we believe that ThoughtWorks Mingle offers good support for both RD and RM, which can also be applied to non-Agile environments. Mingle enables users to define requirements in the form of story cards, link them to other artifacts in the repository, and trace them throughout the lifecycle. Ovum is particularly impressed by the requirements traceability features (Mingle and Go integration). Users can trace requirements from business intent, to source code, to final project deliverables, to software deployed in UAT/production environments. Mingle wiki pages also serve as a glossary for project terms. Other features include workflow generation, requirements dependency mapping, requirements base-lining, change impact analysis, coverage analysis, requirements-based test creation, and requirements-based reporting.
Test automation and QA management

Twist, presently in version 2.0, provides test automation capabilities for manual as well as automated functional testing. Twist supports the automated generation of tests from requirements and TDD. Acceptance criteria or requirements can be mapped to test implementation and the code base through the Scenario Editor interface; using the interface, testers can validate business requirements directly against the source code. Automated tests can be scripted using Java or the Groovy programming language. Other key aspects of Twist include reusable test assets with support for refactoring, and support for Ajax- and JavaScript-based web interfaces. However, the range of technologies that can be tested with Twist is limited; at present, the solution supports Java, Java Swing, and web applications only. Using Mingle together with Twist enables teams to capture business intent directly as test specifications, so analysts, developers, and testers can always validate that their efforts are focused on the customers end goals. The Mingle customizable dashboard and configurable Card Walls allow teams to visually track defects, manage test scenarios, and link bugs with stories for effective and collaborative test management. Mingle syncs with Go to provide real-time visibility into test run history and status. This provides accurate, real-time reporting, which gives the entire team, as well as business stakeholders, visibility into the results of tests and deployment, right from the project dashboard. Go triggers Twist functional tests after check-in or during the UAT stage but before deploying to production, to ensure changes have not broken the application.
Build, configuration, and release management

The vendor has made significant feature enhancements to what was previously known as Cruise (which offered CI capabilities) to transform it into Go, a solution that enables organizations to minimize the gap between software development and delivery. Go offers build management, configuration management, and release management functionalities. Here, configuration management refers to deployment profiles for various target environments in the client organization. ThoughtWorks Studios solution does not offer native source control and software configuration management capabilities. By gaining a centralized, graphical view of all code modules beyond the check-in stage, IT departments can effectively manage the release process and avoid the pitfalls associated with manual release and deployment. This also helps to bridge the gap between software development, quality assurance, and IT operations teams by extending the benefits of continuous integration through to the release stage. Additionally, users can see the status of all builds from within Go (most builds have multiple stages). A functional test written in Twist can be incorporated into the Go release build stages and viewed from within Go and Mingle, and users can see whether these tests have passed or failed.
281

ThoughtWorks Studios targets medium to large enterprises with its Agile ALM solution. Although the nature of the solution is horizontal, the company has gained significant traction in financial services, media, and ISVs. Other major client verticals include high technology, transportation, retail, and healthcare. The vendor sells across all major industry verticals and geographies. ThoughtWorks Studios has a direct presence in seven countries and customers in 29 countries around the globe. The solution adheres to the spirit of Agile, insofar as it is easy to deploy and provides value right from the outset. ThoughtWorks Studios makes a distinction between enterprise-wide deployments and department/SME deployments, and targets these segments separately. Moreover, the ALM suite components are available for free download from the vendors website so that organizations can evaluate the offering. Ovum appreciates the IT practitioner focus of ThoughtWorks Studios marketing strategy, and believes that a growing community of the companys tools users will help drive revenue growth.
At present, ThoughtWorks Studios sells directly in North America, the UK, India, China, and Australia, and covers all other geographies through its reseller partners. ThoughtWorks Studios route to market is skewed in favor of direct sales. Sales through the indirect channel which are primarily over the Internet account for a small percentage of overall revenues. ThoughtWorks Studios has formed partnerships with systems integrators and resellers around the world. The company partners with ThoughtWorks (its parent company) for consulting services, as well as other regional system integrators and technology providers. ThoughtWorks Studios releases a new major version of each product every year, with minor releases on a quarterly basis. Among other enhancements and features, the products future release roadmap includes unstructured collaboration support for Go, and enhancements to Mingle-Go integration.
IMPLEMENTATION
Mingle, Go, and Twist implementations do not require any specialized skills from the client side. Clientside resources required for a proof of concept implementation which can be completed in a day include an IT administrator and a build engineer (for Go), although Twist is a desktop application and can be installed and configured by end users themselves. According to ThoughtWorks Studios, largescale pilot projects can be rolled out to end users within two to three days. For a larger, department-level deployment, the client-side engagement is greater. Customer-side resources required include a system administrator for Mingle, and build, test, and release automation engineers for Go and Twist. Skills required include system and network administration, in addition to programming. Mingle and Go can be implemented in two to three days, whereas Twist may require up to two weeks for roll-out, based upon the technologies that need to be tested. Deployment team members need to identify inefficient steps in their lifecycle processes to accelerate the time-to-value and realize quick returns on investment. For enterprise-wide deployments, the time required varies according to the organizations process maturity, and the extent to which the process conforms to a standard, widely adopted development methodology. Usually the time required to implement is in the range of two to four weeks; client organizations may also have to rely on ThoughtWorks Studios for implementation services. System requirements for each of the three products can be accessed from the vendors website. While Mingle requires a relational database instance which could be either Oracle or Postgres, the other two products do not depend on any third-party software.
282
DEPLOYMENT EXAMPLES
Centrica PLC, UK
ThoughtWorks Studios Agile ALM has been deployed by Centrica to enable rapid application development and deployment. ThoughtWorks solution has been deployed for nearly 1,000 users, and is believed to be aiding business expansion in continental Europe and North America.
Shaw Communications
The organization has deployed ThoughtWorks Studios Agile project management and collaboration offering Mingle 3.3 to manage software development across multiple teams. The solution has been deployed for 600 users, and provides visibility across multiple teams and supports multiple business processes.
Trader Media Group

ThoughtWorks offering enables end-to-end management of the application lifecycle. Deployed for 450 users across the organization, the solution provides Agile project management, release management, and test automation functionality. ThoughtWorks Studios Headquarters 200 E Randolph St 25th Floor Chicago, IL 60601-6501 US Tel: +1 (312) 373 1000 Fax: +1 (312) 373 1001 www.thoughtworks-studios.com ThoughtWorks Studios Ltd 9th Floor Berkshire House 168-173 High Holborn London, WC1V 7AA UK Tel: +44 (0)20 7 497 4500 Fax: +44 (0)20 7 497 4501
283
Incorporating
OVUM Butler Group
CHAPTER 9: Vendor profiles
WWW.OVUM.COM
Aldon
Aldon has been in the ALM market for several years and its offering has a strong focus on change management, collaboration, and compliance, ensuring consistent processes across all areas of IT. Aldon positions itself as an end-to-end process automation solution provider for ALM. With a usercentric approach to process management, the solution spans key segments of ALM, integrates with third-party tools where it does not offer products, and features tight integrations with its IT governance and ITSM solutions. The Aldon ALM solution comprises several core products: Aldon Lifecycle Manager, Aldon Community Manager, Aldon Deployment Manager, Aldon Build and Release Manager, and Aldon Report Manager. Aldon Lifecycle Manager provides an extensive range of SCCM facilities. It automates and enforces consistent service delivery processes across enterprise organizations. The product is capable of supporting multiple geographically distributed development teams. The product supports permissionbased assignment and enforcement services, delivers role-based interfaces (for developers, software engineers, managers), offers enterprise application inventory management, version control (ensuring production and development environment integrity), and release-management features. Aldon Community Manager provides workflow automation and incident-management facilities. It offers incident-driven change management and the automation and delivery of workflow change requests, including approvals, escalation requirements, and release notifications. The product supports business collaboration for change request processing, prototype reviews, and user acceptance. It also provides a range of metrics and reporting facilities including key performance indicators and project status reports. Aldon Deployment Manager ensures that appropriate distribution and deployment facilities are provided to fulfill the project delivery requirements of end-user organizations. The product provides a central console for managing deployments. It covers application release scheduling, packaging and delivery of application components, distribution and installation of new and updated software releases, and support for recovery, rollback, and version-control services. Aldon Build and Release Manager works with a range of SCCM systems and offers build automation and lifecycle tracking of build packages. Compliance is facilitated by end-to-end process automation, embedded process documentation, activity logging, and enforced approval management. Build and Release Management features include point-and-click package promotion, secured and authorized package deployment and redeployment, and provisions to view complete package history and traceability. Aldon Report Manager is a web-based open source tool that provides insight into application development and maintenance activities. Users can select and run reports from one of many prepackaged templates on tasks, activities, and releases, or can create customized queries. On-the-fly audit reports enable managers to monitor and control all tasks and activities associated with a particular project. The solution offers integrations with third-party SCCM tools such as Subversion, CVS, Perforce, and Microsoft Visual SourceSafe; and with IBM Rational DOORS for requirements management. Aldon offers user training during the implementation phase in the form of classroom and web-based courses. Technical and after-sales support services are provided using webinar and telephone-support services.
Company profile
Aldon, a privately owned independent software vendor was founded in 1979 as a provider of software development tools. The company now operates out of 14 offices worldwide supporting customers in more than 60 countries. Aldons corporate headquarters are in Emeryville, California. The company also has major offices in Canada, France, the UK, Australia, Malaysia, and Singapore. Aldon specializes in the development of process-centric ALM tools. It has more than 1,300 companies using its products, and these include 70 of the Fortune 100. The companys clients include AIG, Banner Health, EON Bank, Kraft Foods, Nintendo, and Raiffeisen.
CHAPTER 9: VENDOR PROFILES
287
Aldon 6001 Shellmound Street Suite 600 Emeryville CA 94608-1901 US Tel: +1 (510) 839 3535 Fax: +1 (510) 839 2894 Email: info@aldon.com www.aldon.com
Aldon UK Ibex house Baker Street Weybridge Surrey, KT13 8AH UK Tel: +44 (0)1932 355711 Fax: +44 (0)1932 355722 Email: europe@aldon.com
AccuRev
AccuRev provides tools for software configuration management, software change management, workflow and process management; as well as direct connectors for third-party lifecycle tools. AccuRev also offers a replication server that helps reduce the amount of data transfer over the network. AccuRevs portfolio includes AccuRev SCCM, AccuReplica (repository replication), AccuWork (integrated change management and basic workflow), and AccuBridge (third-party integrations). AccuRev SCCM is a stream-based change and configuration management product that can be used in large-scale software development projects, managing hundreds of projects or managing hundreds of developers in a single project. It is flexible enough to support multiple streams supporting multiple development mainlines, complex release hierarchies, and multiple repositories for managing a large number of project teams. Other features of the AccuRev offering include visual configuration management, private developer workspaces, support for regulatory compliance (Sarbanes-Oxley), and out-of-the-box support for SCCM best practices. Configuration managers can control all ongoing development work through a visual point-and-click interface. In addition, issues and defects are linked directly to configuration items and source code, enabling quick visibility for stakeholders. The product supports only atomic transactions to ensure that the repositorys integrity is not compromised. AccuRev maintains detailed change logs, audit trails, and history that enable users to reproduce previous builds at any stage. The solution also provides native security features such as password-protected access for all resources, access control lists, and stream locking to control code changes. AccuRev offers read-only replication via AccuReplica. AccuReplica maintains a complete copy of the database and all requests for file content are made against the local replica, eliminating the time lost through file transfers over the network. The AccuRev data model also economizes on the amount of stored metadata, storing only branch/label metadata in per-project tables. AccuRev offers integrated issue tracking and change management through the AccuWork module. AccuWork offers a change package feature that facilitates comparison of deployment-ready builds by issues, and supports issue-based development. Change packages are logical groupings of code files that include the fix for one or more known issues. Users can access and manage change packages and issues from within the stream browser interface through a change palette feature, and perform operations such as merging, moving, copying, or reverting changes. Ready-to-deploy builds can be managed at the change package level to provide instant visibility into patch releases and issue fixes without having to drill down to the source file level, and can also support dependency analysis. AccuRev SCM offers bi-directional integrations with third-party lifecycle tools through the AccuBridge integration packs. Each integration pack needs to be licensed separately on a per-server basis. Integration packs are available for Rally Software Agile ALM, VersionOne, Atlassian JIRA, HP Quality Center and Agile Accelerator, IBM Rational ClearQuest, Bugzilla, and MKS Integrity Manager. AccuRev also supports several commonly used IDEs such as Eclipse, MS Visual Studio, and IntelliJ IDEA. In addition, AccuBridge SDK enables organizations to develop custom integrations with third-party tools. User-provisioning and authentication systems can also be integrated with the help of AccuBridge LDAP provisioning connector.
288
AccuRev recently introduced an Agile ALM solution known as AgileCycle, based on the AccuRev SCM system. Other products in the AgileCycle suite include the Agile Project Management tool from Rally Software, and build and release management product AnthillPro from Urbancode.
Company profile
AccuRev is a privately held company headquartered in Lexington, Massachusetts. The company was founded by Damon Poole in 1998 as an SCCM consultancy, and the first version of the flagship offering, AccuRev SCM, was released in 2002. The vendor has extensive experience in software configuration management systems and development processes and methodologies. Key clients include Netezza (IBM), McAfee, Citibank, American Airlines, Kronos, Lufthansa Systems, and i2 Limited. AccuRev 10 Maguire Road, Building 1 Lexington MA 02421 USA Tel: +1 781 861 8700 Fax: +1 781 861 8704 Email: sales@accurev.com www.accurev.com AccuRev Inc 1200 Pacific Ave Ste 275 Santa Cruz CA 95060 USA
Adobe
Adobe provides Adobe AIR, a runtime environment that allows rich internet applications (RIAs) to be developed using HTML, Ajax, Adobe Flash, and Flex technologies to run outside the browser sandbox as a native desktop application. Adobe AIR overcomes many of the constraints associated with browser-based applications, and enables users to run applications developed using web technologies that function either online or offline that are able to access and consume web services, and that communicate across networks using standard web protocols and interfaces. Developers can build applications including widgets and package them with a cross-platform installer that runs on various operating systems such as Windows Vista, Windows XP, Windows 2000, Windows 2003 Server, Macintosh OS X systems (including OS X 10.4.11, 10.5.4, and 10.5.5), and Linux (including Red Hat Fedora 8, Ubuntu 7.10, and openSUSE 10.3). Although the Adobe AIR runtime environment provides offline access, developers must consider data synchronization issues as part of the application design because no generic facility exists to operate in cached mode. Adobe AIR provides capabilities similar to those of a web browser to allow web applications to be deployed on the desktop itself. Adobe AIR supports common Ajax frameworks including Ajax applications to run on AIR. Adobe AIR provides some unique features that are not available to browserbased applications, such as complete local file system access, local database access, and local encrypted storage. To address security concerns Adobe AIR provides administrators with the ability to restrict the installation of applications, only allowing the installation of applications that are signed with certificates in the OS certificate store. Adobe AIR supports digital certificates from trusted authorities such as VeriSign and Thawte. Applications can also be self-signed with security certificates from application providers to enable users to install applications from sources they trust. Adobe AIR natively supports several European and Asian languages, which means that a single AIR application can be made available in multiple languages for multiple geographies. Adobe AIR is available on mobile platforms (such as Android), allowing developers to deliver mobile applications via mobile marketplaces and app stores. Adobe AIR offers mobile device and operating system specific functionalities such as multi-touch, gesture inputs, accelerometer, geo-location, and screen orientation. Adobe AIR runtime and software development kit (SDK) are available for free user download from Adobes website. Adobes cross-platform runtime has seen good traction so far, and is increasing its desktop presence with more than 200 million installations of the runtime.
289
Company profile
Adobe offers a range of digital imaging, design, and document technology platforms for enterprises, creative design professionals, and consumers. Founded in 1982, Adobe (NASDAQ:ADBE) is one of the worlds largest software companies. Headquartered in San Jose, California, Adobe employs more than 7,500 staff across 26 offices worldwide. Adobes flagship enterprise products include Adobe Acrobat and the Adobe Live Cycle product suite, helping connect people, enterprise systems, and business processes. It is known for its graphic design, publishing, and imaging software for print, web, and video production, and the company also offers a range of server and services-based products. Year ending 30 November Revenue ($ million) Change on previous year (%) Total Net Income/(Loss) ($ million) Adobe Systems Inc 345 Park Avenue San Jose CA 95110-2704 USA Tel: +1 (408) 536 6000 Fax: +1 (408) 537 6000 www.adobe.com 2010 3,800 29 774.7 Adobe Systems UK 3 Roundwood Avenue Stockley Park Uxbridge, UB11 1AY UK Tel: +44 (0)208 606 1100 Fax: +44 (0)208 606 4004 2009 2,945.8 (17.7) 386.5 2008 3,579.8 13.4 871.8
CA Technologies
CA Technologies offers the CA Clarity PPM for project and portfolio management. The solution is available in several editions: CA Clarity of IT Governance, CA Clarity for New Product Development, CA Clarity PPM On Demand, and CA Clarity for US Federal Government. CA Clarity PPM provides portfolio analysis and assessment to improve the decision-making process, together with project, resource, and financial management, to ensure the successful delivery of IT projects. Requirements planning and tracking features are also available. The solution integrates business relationship and demand management with PPM, and provides a unified view for requesting and subscribing to IT services. In addition, bi-directional integration between project management and enterprise change management ensures that software development tasks needed to support a project are seamlessly activated, while resource costs are automatically updated at the project level. CA Clarity PPM is based on web services architecture and is composed of a set of integrated modules, content, and an underlying platform. IT Portfolio Manager introduces the concept of a service offering, a hierarchy to capture all the investments needed to deliver a service, and the capability to perform service-level portfolio analysis. IT Financial Manager includes service-based budgeting and forecasting, chargebacks, single invoices, and cost-recovery statements. Business Relationship Manager supplies portals and dashboards to improve and facilitate the ability of IT to engage with the organization in the area of service delivery. Demand Manager provides a single view of resource demand and utilization, which allows organizations to prioritize work. Project Financial Manager has extensive capabilities including billing, invoicing, and chargeback facilities, all available in multiple currencies. Project Portfolio Manager provides a structured environment to decide which projects, programs, or initiatives to fund, which to sustain and which to cancel. Tight integration with other CA Clarity PPM modules allows both bottom-up analysis and top-down planning.
290
Process Manager allows all business processes to be managed centrally, making it easier to optimize and standardize processes across the organization. It also supports phase-gate methodologies, ensuring that best practices are adhered to throughout the organization. Project Manager provides functionality for project planning, estimating, budgeting, resource allocation, and time capture. It integrates with both Open Workbench and Microsoft Project. Resource Manager helps plan and manage resource capacity and balance it against work demand. Capacity planning features highlight capacity and demand overloads by business unit, project, or roles. New Product Development is a knowledge-based tool for the capture and storage of centralized storage of innovation ideas. Open Workbench is an open-source alternative to Microsoft Project from CA, providing desktop project scheduling, off-line scheduling, Gantt and program evaluation and review technique (PERT) charts, automatic what-if resource scheduling, project interdependency tracking, and a customizable view library. Risk and Controls Manager is a global repository that maps internal policies, procedures, and SLAs, along with external regulatory requirements, to existing or new risks and controls. Earned Value Management Report Pack delivers additional earned value metrics, portlets, and reports in CA Clarity PPM. Advanced Report Management Report Pack provides a set of commonly used metrics and reports for managing resources and monitoring resource performance. PMO Accelerator has over 20 portlets, reports, and queries that enable new or existing project management offices (PMOs) to quickly adopt proven practices in project and resource management methodology. CA offers Clarity On Demand PPM, a SaaS-based application that includes many features from the PPM for IT product and leverages advanced Web 2.0 development technologies. CA Clarity also provides a cloud-based Agile on-demand product CA Portfolio Management for Agile IT, which comprises CA Clarity On Demand and CA Agile Vision, an Agile development planning tool based on the Force.com platform. It includes features such as sprint planning, burn-down velocity charts, and virtual walls, all through a web-based user interface.
Company profile
CA Technologies (NYSE:CA) positions itself as one of the worlds largest management software providers. CA Technologies provides what it calls enterprise IT management (EITM) software to manage IT environments across the enterprise. With EITM an enterprise can manage, systems, networks, security, storage, applications and databases, securely and dynamically. Founded in 1976, CA Technologies is headquartered in Islandia, New York. CA Technologies serves 99% of the Fortune 1000 companies across every major industry worldwide. Year ending 31 March Revenue ($ million) Change on previous year (%) Total Net Income/(Loss) ($ million) CA Technologies World Headquarters One CA Plaza Islandia NY 11749 USA Tel: +1 (800) 225 5224 Fax: +1 (631) 342 6800 www.ca.com 2010 4,353 3 771 2009 4,271 0 671 2008 4,277 9 479
CA Technologies EMEA Headquarters Ditton Park, Riding Court Road Datchet, Slough Berkshire, SL3 9LL UK Tel: +44 (0)1753 577733 Fax: +44 (0)1753 825464
291
Coverity
Coverity has a unified set of tools called Coverity Integrity Center (currently in its fifth edition) that cater to a range of software testing requirements across multiple platforms. The solution enables developers to scan source code for defects, prioritize defect severity, map the impact of defects to all projects and products sharing the same code, fix the defects, and report on defect remediation. The Coverity Integrity Center modules include Coverity Static Analysis (previously known as Coverity Prevent), Coverity Dynamic Analysis (previously Coverity Thread Analyzer), Coverity Architectural Analysis, and Coverity Build Analysis. Central to this solution is the Coverity Integrity Manager, which is a unified defectmanagement interface and central information source, enabling developers to view both static and dynamic analysis defects in a single window. Coverity Static Analysis offers support for C/C++, C#, .NET, and Java programming languages. It pinpoints software defects and security vulnerabilities using a variety of approaches including metadatacompilation research. The analysis engine detects a wide range of detects at compile time, by analyzing all paths in the source code. It also analyzes the program as an entity, rather than searching each line in isolation for syntactic anomalies. Coverity Dynamic Analysis offers dynamic testing capabilities for multi-threaded Java applications suitable for code running on multi-core machines. Coverity Dynamic Analysis detects and reports problems that concurrent applications raise such as race conditions and deadlocks, both existing and potential. Coverity Architectural Analysis is a software visualization tool that includes Coveritys Software DNA Map analysis system, which visually depicts the architecture, shows calls between classes, and a class dependency structure matrix, as well as highlighting the complexity in the code base. Platforms supported include C/C++ and Java. Coverity Build Analysis offers bottleneck identification and change management functionalities for software builds. It also helps in security and compliance enforcement by providing an audit trail to verify the build as well as the files that comprise the build. Coverity also offers the Coverity Software Readiness Manager, a tool for assessing risk in Java software and its readiness for release. The code base is analyzed and the tool reports on code complexity, violation of best practices, architectural integrity, interdependencies, and test coverage. It helps project managers to decide when to stop testing and release, and where to focus refactoring if software is not ready.
Company profile
Coverity is a privately held company headquartered in San Francisco, California, with sales offices in Boston, Washington DC, and Chicago. It has distribution offices in London, Paris, Tokyo, Munich, and Seoul. The technology behind Coverity was developed in the Stanford Computer Science Department where professors and students spent three years refining a system of checking for software defects at compile time, and formed Coverity to market the technology. Key clients include RIM, ARM, HP, Symbian, EMC, UBS, Ericsson, LG, and Tellabs. Coverity Global Headquarters 185 Berry St. Suite 1600 San Francisco CA 94107 USA Tel: +1 (415) 321 5200 Fax: +1 (415) 541 9521 Email: info@coverity.com www.coverity.com Coverity EMEA Headquarters Quatro House Lyon Way Camberley Surrey, GU16 7ER UK Tel: +44 (0)1276 804790 Fax: +44 (0)1276 804676
292
Digite
Digite provides an integrated Agile application lifecycle management solution, Digite Enterprise, which provides application development and maintenance, process governance, and project and portfolio management capabilities. The products are built on a J2EE web services-compliant architecture that allows integration with existing enterprise applications. The solution is also available via SaaS delivery. Digite Enterprise comprises a software development lifecycle offering for requirements management, change management, and test management (integrates with third-party configuration-management tools), a software project management tool for project planning and scoping, task scheduling, resource planning and tracking, managing risks and issues, time capture, and collaboration. It also includes a process governance tool and and a project portfolio management tool. Digite Enterprise offers a standard and configurable project breakdown structure. It offers a choice of using standard MS Project integration or Digites native scheduler, STaRT (simple task and resource tracking). Resource management is facilitated with the help of integrated timesheets that provide visibility into resource efficiency and provision for skill-based search. All project artefacts such as issues, risks, and action items are easily traceable because they are stored in a single repository. Collaboration among users is facilitated through wikis, discussion forums, and meeting rooms. Preconfigured dashboards provide extensive details about project status and health in the form of reports and charts. Users can edit requirements, or any other artefacts such as design documents, project documentation, and notes, through a Microsoft Word interface. Digite Enterprise enables users to manage the feature/user story backlog, and plan releases and iterations to schedule the user story development. Teams can define the repository of user stories, establish hierarchy between them, and rank them in order of development priority. They can also provide rough effort estimates so that the overall release/iteration can be planned. Each release can then be broken down into fixed duration sprints/iterations. Users can also create task/resource assignments where actual effort can be tracked. Tight integration with the rest of Digites ALM modules such as defect management, test management, and configuration management enables users to establish complete traceability among all software artefacts. Digite Enterprise also provides modules for tracking and managing planned releases and work packages. Each enhancement can have its own set of tasks with defined development phases (requirement, design, construction, testing and deployment) and a set of deliverables. Digites integrated solution ensures drill-down metrics and dashboards at each individual enhancement level. Digite Process Governance offers a platform for organizations to maintain consistency across all of their projects by capturing and reusing project knowledge, enabling managers to standardize and adopt the organizations best practices.
Company profile
Headquartered in Mountain View, California, Digite is a privately held company that provides ALM solutions for enterprises. The company has offices in the US and India. It has more than 250,000 users worldwide across industry verticals such as consulting and professional services, financial services, public sector, high technology, and telecommunications. Key customers include Convergys, Infosys, HCL technologies, Virtusa, Tech Mahindra, Wipro, Deloitte, and Dun & Bradstreet. Digite Inc 82 Pioneer Way Suite # 102 Mountain View CA 94041 USA Tel: +1 650 210 3900 Fax: +1 650 210 3901 www.digite.com Digite Infotech Pvt Ltd First Floor, Crimpage Corporation Plot No.57, Street No.17 MIDC, Marol, Andheri (East) Mumbai 400 093 India Tel: +91 22 6708 0100
293
edgeIPK
edgeIPK provides a runtime and development environment that helps organizations develop business applications (rich or thin client, off-line, portal based or composite applications) using a write-oncepublish-multiple-times model. Its product edgeConnect comprises a run time environment (edgeConnect RTE) and an IDE (edgeConnect IDE). edgeConnect is built on the Model View Controller (MVC) architecture that enforces the separation of the front end interface (Presentation or View), business logic (Process or Controller), and data (Integration or Model). edgeConnect uses its own XML schema behind the scenes which is never exposed to the user. EdgeConnects runtime engine (RTE) includes code efficiency measurements, multi-threading, and caching, managed security that enforces mandatory security checks for data integrity, failover managed at the application server level (RTE runs as a servlet), and scalability, which enables users to leverage multi-processing or distributed computing models. The IDE includes a version-control and releasemanagement system for multi-developer projects, and a deployment tool for creating and configuring deployment packages for specific environments. edgeConnect has a graphical user interface development environment that allows users to easily and rapidly build the presentation layer of complex screen/form-based applications. The tool is compatible with the current generation of web browsers and makes use of dynamic HTML or JavaScript where appropriate, or pure thin clients. By separating the logical process (model) from the presentation (view), it is possible to make changes in the process only once and have the effects ripple through multiple presentations. edgeConnect also provides the capability to auto-generate screens from databases, XML schemas, and web service definitions, accelerating development times. This process can be reversed so that database tables can be generated from the screen layout and its field definitions. Names of the fields can be edited later to make them user-friendly and readable.
Company profile
Founded in 2001, edgeIPK is a privately held company headquartered in Newbury, UK where operations and research and development are carried out. It also has a presence in the Netherlands and North America. edgeIPK has extensive experience in the financial services sector, with blue-chip clients including Allianz Insurance and Brit Insurance. Backed by venture capital from Favonius Ventures, a specialist European financial services tech tund, it was able to create an open presentation platform for composite, online, off-line, and portal applications. edgeIPK 3 Station Yard Station Road Hungerford Berkshire RG17 ODY UK Tel: +44 (0)1635 231231 Fax: +44 (0)1488 685160 E-mail: info@edgeipk.com www.edgeipk.com
294
Electric Cloud
Electric Cloud offers three products to support the automation of software production: ElectricCommander v3.5, ElectricAccelerator v5.0, and ElectricInsight v3.0. ElectricCommander automates and manages the software build, test, and deploy processes. It is development language and build-tool agnostic, which means that any source code can be compiled and built using a wide range of build scripts and popular tools such as Ant and Visual Studio. It allows users to define tasks through a visual interface, schedule jobs, and provide comprehensive reporting and charting capabilities. It also extracts and displays data from the defect tracker along with relevant build and test results, with managers receiving a notification when the issue has been resolved by QA. The product supports multiple teams across distributed locations through the use of a web-interface built using Ajax. Support for the Android development environment is now available via plug-ins for the Gerrit and Git development tools, including automated testing features for Android devices. Integration with Amazon Web Services platform allows ElectricCommander to manage computing resources running on the cloud platform. ElectricAccelerator provides parallel processing capabilities by managing and executing jobs across a cluster of standard servers including virtual machines. ElectricAccelerator provides deep, fine-grained parallelization, ensuring that large enterprise-level project builds can be compiled in a few hours. ElectricAccelerator also adheres to the concept of sub-builds, reducing a build to the smallest subset of changes. The sub-build concept identifies and builds only the components that are necessary, resulting in fewer broken builds and the ability to compile and test frequently without affecting the rest of the system. ElectricInsight mines the output of ElectricAccelerator to provide a graphical representation of the build for diagnostics and troubleshooting. The graphical interface helps analyze parallel tasks, rapidly identify build breakages, report on longest serial chains, make file manifests, and provides serialization analysis. Reports such as derived file analysis reports, root conflicts, and job time reports, help increase a build managers productivity. Users can see where dependency conflicts arise and take actions to fine-tune build scripts and help accelerate parallel running.
Company profile
Electric Cloud is a private company headquartered in Sunnyvale, California, with a sales and support office in Oxford, UK, and sales teams throughout the US. The company was founded in 2002 by software technologists who were seeking to address automation and poor productivity in the build-to-deploy phase of software development. Electric Cloud is backed by venture capitalists. It targets mobile device providers, semiconductor and other embedded applications manufacturers, automotive suppliers, independent software vendors, transactional websites, and enterprise IT teams. Key clients include Motorola, Qualcomm, LSI Logic, Broadcom, Brocade, Freescale, Intuit, PayPal, Ericsson, and Thales. Electric Cloud Inc 676 W Maude Avenue Sunnyvale CA 94085 USA Tel: +1 (408) 419 4300 Fax: +1 (408) 419 4399 Email: info@electric-cloud.com www.electric-cloud.com Electric Cloud Europe 7200 The Quorum Oxford Business Park North Garsington Road Oxford, OX4 2JZ, UK Tel: +44 (0)1865 487177 Email: europe.info@electric-cloud.com
295
Kovair
Kovairs ALM solution, Kovair Global Lifecycle (also known as Kovair Global Studio), is an integrated set of application lifecycle management tools for requirements management, software change management, test management, defect/issue management, project management, release management, and risk management. The solution is available for both on-premise and SaaS deployments. The product consists of three components: Kovair Lifecycle Applications, Kovair Process Automation Technology, and Omnibus Integration Bus for IT. The Kovair Lifecycle Applications module provides access to the various built-in tools through a single Ajax-based web interface, and leverages a single shared repository. The module provides a task-based process engine that enables users to add any number of functional entities and maintains the relationships between various entities. The module facilitates capturing suggestions/comments from websites or emails, providing automated workflow for the suggestion-review process, and enabling suggestions/comments to be converted to formal requirements. Other requirements-management features include capturing requirements from Microsoft Office applications, text/XML files, automated requirements review/approval process, traceability between requirements and other artefacts, release roadmap planning, and graphic reporting. Test-management features include test-case definition and execution, traceability between requirements and test cases, and integration with Quality Center, WinRunner, and native Kovair tools. The module provides some project management features such as task analysis, time management, resource analysis and management, and project progress monitoring. The portfolio management capabilities include status reporting and the ability to drill down to individual projects. The process automation capabilities of Kovair Global Lifecycle include process modeling through a visual process designer, process documentation, and workflow automation through the Omniprocess workflow engine. The module supports independent yet synchronized processes at multiple levels such as projects, modules, releases, requirements, issues, and entities. Other solution features include diagrammatic process representation, drag-and-drop process designing, parallel process support, policy-based task assignment, conditional branching, process modification without impacting running process instances, support for process level variables, and the ability to restart a process at any time. By leveraging web services, the Omnibus bi-directional integration middleware enables the integration of various third-party tools with Kovair Global Lifecycle. The module facilitates integration with SCM systems such as ClearCase, Perforce, Subversion, CVS, and StarTeam; with IT operations and service desk tools such as CA Univision, BMC Remedy, and HP Peregrine; with PPM tools such as CA Clarity, HP PPM Center, Compuware, and Planview; and with test management tool HP Quality Center. Kovair Global Lifecycle/RM is a web-based requirements capture, analysis, and management solution, enabling co-operation across geographically distributed teams. Requirements are captured using the native text editor through email, imported from Microsoft Word/Excel, or any third-party capture tool. It provides requirements traceability, requirements reuse, change management, requirements baselining, requirements ranking, release roadmap, and supports various development methodologies including Agile methodologies such as Scrum and XP. Kovair Global Lifecycle Change Management provides automated workflow capabilities. The tool provides software change management capabilities in areas such as defects, functionality change, enhancement requests, and compliance requirements. It supports on-demand as well as recurring change management. It further provides features such as change request submission by any user from any location, workflow creation based on change type, and complete traceability to requirements when used as a part of the Kovair Global Lifecycle integrated suite. Kovair Global Lifecycle Incident Management leverages the common repository to map incidents and known issues for quick resolution. The solution links related incidents and maintains traceability between related records. It enables users to view and review others work, and also provides process automation capabilities, and traceability of relationships with other processes such as problem management, configuration, and change management. It also integrates with other third-party help desk and issue management products such as BMC Remedy.
296
Kovair Global Lifecycle Release Management is a web-based tool that provides a workflow designer to define release management processes. The tool maintains traceability between release and other artefacts such as change items, test cases, and defects. Other features include graphic reporting, realtime cross-team collaboration, user and group-level security through role-based access, and role-based task allocation.
Company profile
Kovair provides ALM solutions for enterprises. The companys Kovair Global Lifecycle flagship product is aimed at enterprises looking to automate their workflow and develop and implement reliable processes for managing the application lifecycle. The company has acquired customers from industry verticals such as ISVs, IT service providers, banking and financial services, healthcare, defense, pharmaceutical, government, and the public sector. Kovair Software Inc 1533 California Circle, Suite # 210 Milpitas CA 95035, USA Tel: +1 (408) 262 0200 Fax: +1 (408) 904 4794 Email: sales@kovair.com www.kovair.com Kovair India Development Center Plot J-5, Block EP & GP, Sector V Salt Lake, Kolkata WB 700091, India Tel: +91 33 2357 2588 Fax: +91 33 2357 2588
OutSystems
OutSystems Platform, also known as Agile Platform, is a software development platform for developing, deploying, managing, and changing enterprise web applications (Microsoft .NET or Java-based). The platform adopts an approach known as short cut software development (SCSD), which differentiates itself from previous approaches (rapid application development and fourth generation languages) by being open standards-based and providing the ability to quickly develop applications that can be modified during any phase of the project. It is based on IntelliWarp technology, a context-aware patternbased technology that allows developers to deliver web applications without writing any code. The product requires no programming expertise and is therefore suitable for business analysts to learn. Many database transactions are also automated, but if necessary SQL can be manually written for advanced operations. Specialist tasks may require a skilled programmer to add custom code using the platforms extensibility features. There are several extensibility points that can be used including JavaScript, HTML, low level extensions, and advanced SQL. The OutSystems Platform supports Agile software development methodologies through its flexible approach. The platform can be used to compose, change, and operate enterprise applications that integrate with existing systems and reach users via web browsers, email, and mobile interfaces. The deployed solution is pure .NET or Java, with no ties to OutSystems, protecting investments and making them fully portable. The OutSystems Platform comprises OutSystems Service Studio, OutSystems Service Center, and OutSystems Integration Studio. OutSystems Service Studio is an integrated visual environment for application designers to compose, change, and automatically deploy enterprise applications. Service Studio is a desktop application that can be freely installed in every application developers PC. Output from Service Studio is an XML specification of the application(s), which contains all layer and building-block information. Deployment targets include the web and mobile devices. OutSystems Service Center is a centralized management console that co-ordinates all administration, monitoring, auditing, operation, and deployment of all applications and integration components. OutSystems Integration Studio is an integrated visual environment where application designers can create connectors for integrating with existing enterprise systems. Once published, the connectors can be easily used as visual building blocks using OutSystems Service Studio to compose applications.
297
Company profile
OutSystems is a privately owned company based in Lisbon, Portugal, with offices in London, UK, Madrid, Spain, Redwood City, US, and Maarssen, the Netherlands. The company was founded in March 2001 and secured funding from Dutch venture capital firm Nesbic CTe Fund Fortis bank with series B round of funding from PME Investimentos. In 2003 the company reached operational breakeven. OutSystems Rua Central Park, Ed. 6-2A 2795-242, Linda-a-Velha Portugal Tel: +351 (0) 21 4153730 Fax: +351 (0) 21 4153731 E-mail: info@outsystems.com www.outsystems.com OutSystems UK Castle Hill House Castle Hill, Windsor SL4 1PD, UK Tel: +44 (0)175 383 9499 Fax: +44 (0)175 383 1113
Oracle
Oracle JDeveloper and Oracle Application Development Framework (ADF) 11g Release 1, is a significant milestone in Oracles support for application development. JDeveloper is an advanced IDE for developing applications using Java, XML, SQL and PL/SQL, HTML, JavaScript, Business Process Execution Language (BPEL), and Portal/Oracle WebCenter development. Developers writing stored procedures in PL/SQL can also use Oracle SQL Developer, which shares a common IDE with JDeveloper. JDeveloper users can create, assemble, and reuse components to build rich, interactive applications. JDeveloper offers support for the latest versions of Java SE and Java EE features such as advanced code editors with code insight, code completion, refactoring, and live code auditing simplify the coding process. Tools such as debuggers, code auditors and profilers ensure optimum code quality, and visual editors and modelers help rapidly create Java EE components. In addition it also supports integrated web services development. Oracle ADF is a Java framework for creating Java EE-based enterprise applications and uses the ModelView-Control architecture. The framework helps integrate the various aspects of an application including data access, business services development, the web, mobile and desktop interfaces, data binding, and security. Following its acquisition of Sun, Oracle has brought the following additional products under its umbrella, including: Oracle Solaris Studio: Formerly known as Sun Studio, Oracle Solaris Studio offers a collection of tools for developers who work in C/C++ and FORTRAN environments. The toolset supports Oracle Solaris and Linux operating systems, and delivers an integrated development platform for building high-performance applications for SPARC and x86-based Oracle Sun Enterprise Servers. Oracle Solaris Studio includes compilers, source code and memory debuggers, performance analysis tools and tools for analyzing thread level data. The toolset supports OpenMP, an API for shared memory multiprocessing in C/C++ and FORTRAN, and also includes math libraries. Oracle Solaris Studio facilitates application migration to multi-core systems through compilers that aid application optimization for multi-core architectures and provide autoparallelization of single threaded applications as well as advanced multi-core tools that enable users to develop, debug and tune parallel, multi threaded programs. The IDE and performance analysis tools provide improved ABI (Application Binary Interface) compatibility support which helps reduce applications shared library dependencies. Based on Netbeans, the Oracle Solaris Studio IDE is specifically geared for C/C++ developers enabling users to improve productivity. Performance analysis features enable users to drill down from the source code level to machine executable code and correlate the two. It also provides users with an algorithm to change the hardware system counters and monitor processor level execution data. The performance analyzer tool can be used to profile single threaded as well as multi threaded applications. The comprehensive tool set increases developer efficiency, maximizes application performance and greatly simplifies multi-core development.
298
NetBeans: NetBeans is an open source IDE which was originally developed for Java. The current v6.9 of the tool supports multiple technologies, including: Ajax; C/C++; Java EE 6; JavaFX 1.3; Java ME; Java SE; JavaScript; Mobile; PHP; Profiler; Python; Groovy; Refactor; Representational State Transfer (REST); SOAP; XML; OSGi and GUI builder, among others. As evident from the range of technologies supported, including contributions from the community the tool enables developers to create desktop, enterprise, web and mobile applications in Java, C/C++, and also in dynamic programming languages such as Groovy, Python, Ruby, and PHP. NetBeans supports multiple platforms, including Windows, Linux, Mac OS X, and Solaris.
Company profile
Oracle Corporation, with its headquarters in Redwood Shores, California, in the USA, and 300 offices worldwide in 145 countries, is the worlds second largest, independent software company, and has its roots in information management. The companys product lines cover database, middleware, tools, and enterprise application products; it also offers consulting, education, and support services. Oracles technology can be found in nearly every industry worldwide and in 98% of the Fortune 100 company offices. The company has made a number of acquisitions, the most significant ones being PeopleSoft and Siebel, pitching Oracle right into the heart of the enterprise applications market, and Sun Microsystems, giving Oracle a hardware manufacturing capability as well as the guardianship of Java. Several smaller, but significant acquisitions of middleware, Business Intelligence, BPM and Service oriented Architecture (SOA) vendors, have given Oracle a very broad product set with potential appeal to a very large market. Some of Oracles customers in this space include: Alcoa, Bank of the West, Booz Allen, British Telecom, Dell, General Dynamics, Telenor, Woodside, and Xstrata Copper. Year ending 31 May Revenue ($ million) Change on previous year (%) Total Net Income/(Loss) ($ million) Oracle Corporation 500 Oracle Parkway Redwood Shores CA 94065 USA Tel: +1 (650) 506 7000 www.oracle.com 2010 26,820 15.3 6,135 2009 23,252 3.6 5,593 2008 22,430 24.6 5,521
Oracle UK Headquarters Oracle Parkway, Thames Valley Park Reading, Berkshire RG6 1RA UK Tel: +44 (0)118 924 0000
Perforce Software
Perforce Software provides the Perforce SCM System. The Perforce SCM product follows the client/server architecture, and provides multiple clients for end-user access. The solution scales to support large globally-distributed deployments. The components of the Perforce SCM system include the P4D server, a merge tool, a reporting tool, direct connectors to third-party ALM products, caching/repository replication for distributed development teams, a defect-tracking tool, and clients for end-user access through command line, visual UI, web, and the Windows platform. In addition to on-premise deployments, Perforce also offers subscription licenses. Perforce P4D server is the main module that manages access to files stored in the repository. The server is responsible for provisioning and tracking user access, the details of which are stored in a central database. The merge tool, P4Merge, enables users to merge code branches using a graphic interface.
299
Supported features include three-way merging, side-by-side file comparisons, and support for third-party merge tools. The reporting tool, P4Report, enables querying and reporting over artefacts stored in the system repository. The product lacks native reporting capabilities, and makes this up by integrating with popular third-party reporting tools such as Crystal Reports, Microsoft Access, and Microsoft Excel through Open Database Connectivity (ODBC). The product integrates with a number of third-party ALM products, including IDEs, defect tracking tools, automated build tools, design tools, and Microsoft Office desktop productivity applications. Perforce also offers integration with the Eclipse IDE, through P4Eclipse, which features two visual tools: Merge Quest which illustrates branch dependencies, and Folder Diff which helps in diagnosing the cause of bugs and measures the amount of code which needs to be resolved in pending merge scenarios. The products inbuilt defect-tracking system enables change request management operations, and for greater feature support the product integrates with leading defect-tracking systems including Atlassian JIRA among others. An innovative feature of the Perforce SCM system is its explicit support for distributed development. The product ships with a Proxy server, P4P, which provides a document and file cache, making documents and artefacts stored in a remote repository that is locally available, reducing the response times for distributed teams. The local copy of documents and resources held in the cache are periodically synchronized with the central system repository. The shelving feature enables developers to cache modified files in the Perforce Server, without checking them in as a changed version. Users can therefore view both active work in progress and files that are on hold. The product ships with a number of client access tools including the command-line client P4, which also supports scripting, and proves useful for Unix and Linux-based systems. The visual UI client, P4V, supports a number of platforms including Windows, Mac OS X, Linux, Solaris, and FreeBSD. P4V provides in-built merge features, a time-lapse tool which allows users to see a consolidated view of all changes made to an individual file, and a revision graph tool which provides a tree representation of the complete branch history for a given file, including branch points, edits, and merges. The Perforce web client, P4Web, provides access to complete Perforce SCM system functionality through browser-based interfaces.
Company profile
Perforce Software is a privately held company founded in 1995, headquartered in Alameda, California. It has international offices in Canada, the UK, and Australia. The Perforce SCM system has been deployed at 5,000 organizations across the globe in the gaming, electronics, healthcare, and financial services sectors. Key customers include AMD, Electronic Arts, National Instruments, Qualcomm, Nvidia, ATI Technologies, Deutsche Bank, BBC News Online, and Washington Mutual. Perforce Software Inc 2320 Blanding Avenue Alameda CA 94501 USA Tel: +1 (510) 864 7400 Fax: +1 (510) 864 5340 E-mail: info@perforce.com www.perforce.com Perforce Software UK Ltd West Forest Gate Wellington Road, Wokingham Berkshire, RG40 2AQ UK Tel: +44 (0)845 345 0116 Fax: +44 (0)845 345 0117 Email: uk@perforce.com
RADTAC
RADTAC helps clients select and introduce Agile practices gradually alongside existing processes so that continuous improvement does not hamper productivity levels. RADTAC leverages its experience with transformational change management, in particular with the cultural changes necessary for improvements in performance to provide business value to customers. RADTAC provides customized consulting, training, and delivery services for individual clients in several areas: Agile practices such as XP, Scrum (including fully accredited Scrum training and certified Scrum Masters), test-driven development, Lean software development, Agile Unified Process, and DSDM/Atern (including fully accredited DSDM Training and certified DSDM Practitioners).
300
Project methodologies and management approaches supported by RADTAC include Project Management Institute Project Management Body of Knowledge (PMBOK) and Projects In Controlled Environments (PRINCE2) including fully accredited PRINCE2 training and certified PRINCE2 Practitioners, portfolio and program management approaches such as the OGCs Managing Successful Programs (MSPs), and process, quality, and service management approaches Capability Maturity Model Integration (CMMI) and ITIL. The RADTAC PragmaticAgile Transformation Service has five service strands that work together for effective transition to a sustainable step change in performance. Each of these service strands can be supplied individually or in any combination as appropriate. Transition management: RADTAC helps the client manage the transition as a proper project or program with clear vision, objectives, measures, fully engaged stake holders, and cost management. Guidance and standards routemap: RADTAC helps the client select, customize, and integrate the right Agile practices with project lifecycle management processes. It provides guidance and sets up a feedback mechanism. Training and education: RADTAC works with the client to define the portfolio of training and education on Agile practices and project lifecycle disciplines, ranging from detailed practitioner courses to short targeted briefings. Support: RADTAC provides consultancy to senior management, Agile experts, technical and business project staff, and skilled interim resources such as project managers, architects, analysts, developers, and testers. Capability building: RADTAC provides coaches, consultants, trainers, and interim personnel to improve all of the related capabilities that clients need to address to make their new Agile practices sustainable.
Company profile
RADTAC is a privately held company founded in 2000. It is headquartered in London, UK, and has another office near Bristol. The company adheres to the Associate Business Model to deliver services to its clients. This enables it to cover a wide geography. It has delivered services to client projects throughout the UK and Ireland, in Europe, India, Hong Kong, Singapore, Canada, and the US. Key clients include Fidelity International, Friends Provident, Jardine Lloyd Thompson (Profund Solutions), BT, British Airways, and HM Customs & Excise. RADTAC Head Office St. Martins House 16 St Martins le Grand London EC1A 4EN, UK Tel: +44 (0)207 397 8340 Fax: +44 (0)207 307 8341 Email: enquiries@radtac.co.uk www.radtac.co.uk
Rogue Wave Software

The acquisition of TotalView Technologies in 2009 marked Rogue Waves entry into the high-end debugging and analysis tools market. TotalViews ReplayEngine 1.8 enables users to record program state in execution and deterministically replay it, simplifying troubleshooting activities. This is a new concept of reverse debugging, where the flow of an application can be moved backwards in time, as well as forward-stepped, providing a far stronger approach to debugging than purely forward-stepping. ReplayEngine functions as a plug-in product inside the TotalView debugger environment. By leveraging the capabilities of TotalView debugger, ReplayEngine can handle distributed multi-process and multithreaded programs.
301
The product records program input, such as network and file I/O, captures thread behavior by forcing single-thread execution at a time, and records context switches among threads. In replay mode, operations are similar to that of rewind in a media player. It lets users review any part of the recorded program execution, set breakpoints/watchpoints, and jump to any part of the recorded history. Users can record and replay the most recent or the most relevant parts of long-running programs, controlling the amount of memory resources used for storing recorded history. One innovative characteristic of ReplayEngine when compared with other ways of recording program execution history is that it is compatible with a wide range of real-world application architectures. The debugger supports various message passing interface (MPI) implementations such as MPIVH1, MPICH2, Open MPI, MVAPICH, Intel MPI, HP MPI, and local area multicomputer (LAM). MPI support has been expanded to include Nemesis and SSM channels for MPICH2, and shared memory configuration for OpenMPI, Intel MPI, HP MPI, and LAM MPI. For multi-process debugging TotalView allows users to control anywhere from a single process, or an arbitrary group of processes, through to an entire parallel job. For multi-threaded debugging TotalView allows independent control of each thread, control at the level of thread groups, or operations that affect the entire processes. Users can switch their view of variables and data between the processes to which the debugger is attached, to drill down to the lowest level of granularity in any thread or process. TotalView enables the debugging of multi-process and multi-threaded programs by taking control of the new remote or local process, or thread, as it is created. The product allows users to see the value of a variable in each process or thread simultaneously, and places the executing processes and threads into groups so that operations such as start, stop, step, and examine can be carried out on groups of processes and threads. Users can switch between processes and threads by selecting the process or thread name in the root window. TotalView debugger allows for editing and changing programs on-the-fly for investigation without needing to recompile, provides comprehensive memory debugging features, and with the introduction of ReplayEngine, has covered a backward traceability feature gap that existed in debugging products. The product also supports Unix, Linux, Mac OS X, and Windows (Windows XP and Windows 7) implementation platforms.
Company profile
Headquartered in Boulder, Colorado, Rogue Wave Software is a provider of development tools and embedded libraries for high-performance computing (HPC) application design, testing, and deployment. Founded in 1989, the company is backed by venture capital funding from Battery Ventures. It has its sales, distribution, and support offices in the US, the UK, France, Germany, and Japan. Targeted industries include financial services, aerospace and defense, government laboratories, commercial enterprise, telecommunications, energy, academia, and ISVs. The company expanded its presence and offerings in the HPC development solutions market with its 2009 acquisitions of Visual Numerics (a provider of embeddable mathematical and statistical algorithms and visualization software), and TotalView with its interactive analysis and debugging tools. Customers for the TotalView product range include Applied Research Associates, OpenGeoSolutions, CINECA, STFC Daresbury Laboratory, Weston Geophysical, Simulia, Ultra Electronics, and Stanford University. Rogue Wave Software 5500 Flatiron Parkway Suite 200 Boulder CO 80301 USA Tel: +1 (800) 487 3217 Fax: +1 (303) 473 9137 Email: sales@roguewave.com www.roguewave.com Rogue Wave Software UK Ltd Dukesbridge House 23 Duke Street, Reading RG1 4SA , UK Tel: +44 (0)845 054 9950 Fax: +44 (0)845 054 9951 Email: sales@roguewave.co.uk
302
Salesforce.com
Salesforce.com provides ISVs and customer organizations the ability to create custom on-demand business applications through its Force.com platform. Application developers can leverage the platform to create enterprise applications that run on the cloud. Force.com facilitates rapid custom application development through various capabilities for integration, reusability, analytics, security, user interface development, database access, and mobility. Users can create, deploy, and share their applications with other organizations, and developers can also look for potential customers through the Salesforce AppExchange. All applications developed on the Force.com platform automatically integrate with Salesforce.com. The Force.com application framework enables developers to customize existing applications or build new applications without extensive coding efforts. Developers can specify data layout on a page, create workflows, and build a data-based reporting structure. The s-control feature allows developers to create client-side functionality without having to write code. Developers can also create custom links and buttons that reference custom code for specific functionality. In addition, Force.com provides a web services-based API that gives developers access to platform functionality. The web services API enables developers to extend Force.com features to external applications, leveraging technologies and platforms including Ajax, Adobe ColdFusion, Java, Adobe Flex, Google, Mac OS X Cocoa, .NET and Visual Basic, Perl, PHP, Python, Ruby on Rails, Amazon, and FaceBook. The Visual Process Manager enables users to design and run multi-step business processes in the cloud. Force.com also provides a procedural programming language, Apex, for development on the cloud. Apex enables developers to extend their application functionality with the help of triggers that contain generic application logic. The Visualforce component is a GUI designer tool that facilitates presentation layer design. In addition, users can use Visualforce pages to create data mashups or Web 2.0-style mashups by integrating data from various Force.com objects or web services. Page components are described through a set of tags that developers can use while creating Visualforce pages. The page interface and underlying data are connected through Force.com controllers that represent data objects. Visualforce allows developers to create their own controllers for greater flexibility, and gives them access to an expanded set of data sources. Force.com contains an underlying layer of database services that enables the creation of data objects for storage, and allows developers to define formulae and perform calculations on data. Formulae enable the creation of fields inside data objects, define validation criteria, and calculate data items for reporting. Force.com provides a set of functions and a formula editor that developers can use to create formulae. In addtion, the platform provides an eclipse-based IDE for application development. Force.com also enables the creation and deployment of mobile applications that work on BlackBerry, iPhone, Windows Mobile, and other mobile devices. Salesforce AppExchange is the application exchange platform. AppExchange allows businesses to select the appropriate on-demand applications developed by Salesforce, development partners, and customers. AppExchange helps both ISVs and on-demand application users by providing a common platform to trade.
Company Profile
Salesforce.com was founded in 1999 and is a leading provider of on-demand business applications, and platform-as-a-service. The company is headquartered in San Francisco, California, with offices in Europe, North America, India, and Asia-Pacific. Salesforce SFA is the flagship CRM application of the Salesforce service suite. Customers include Corporate Express, Daiwa Securities, Expedia Corporate Travel, Dow Jones Newswires, SunTrust Bank, and Kaiser Permanente. Salesforces global business partners include IBM, Microsoft, BEA Systems (Oracle), Sun Microsystems, Tibco, PricewaterhouseCoopers, and Miller Heiman. Salesforce is publicly listed (NYSE:CRM). Below are the companys key financials.
303
Year ending 31 January Revenue ($ million) Change on previous year (%) Total Net Income/(Loss) ($ million) Salesforce.com Inc The Landmark @ One Market Suite 300, San Francisco, CA 94105 USA Tel: +1 (415) 901 7000 Fax: +1 (415) 901 7040 Email: info@salesforce.com www.salesforce.com
2010 1305.5 21.1 80.7 EMEA Headquarters Route de la Longeraie 9 1110 Morges Switzerland Tel: +41 (0) 2169 53700 Fax: +41 (0) 2169 53701
2009 1076.7 43.8 43.4
2008 748.7 50.6 18.36
Email: info@emea.salesforce.com
Sapient
Sapient ResultSpace is an ALM platform offering aimed at organizations that have adopted or are looking to adopt Agile, Lean, or iterative methods of software development. ResultSpace enables users to manage projects and project tasks. It facilitates the estimation, planning, and tracking of projects, provides requirements-management functionality and requirements traceability through the lifecycle, and includes other features such as defect management, issue and risk management, collaboration over documents and source code, version control, software asset, and knowledge management. ResultSpace is a web-based product that employs filtering and scoring algorithms to simplify user searches for information and documents. The solution allows users to plan development process iterations, allocate resources, and assign tasks. Capacity indicators help development process managers to plan for capacity utilization by providing visual indicators for under or over-utilization. ResultSpace ALM uses the open-source tool Subversion for the software change and configuration management (SCCM) and knowledge management repository. Access to code and artefacts stored in the repository is through HTTP Secure Sockets Layer (SSL). ResultSpace also leverages the WebDAV server that allows users to collaborate on documents online. For managing Agile processes, ResultSpace provides a story-management feature that provides stakeholders with a consolidated view of user stories. Each reported story contains related artefacts, tasks and work-items, tests, defects, and issues. The solution uses a common repository to enable better traceability throughout the lifecycle, and to create and maintain relationships between process artefacts. Change is managed and propagated with the help of a change log, which records the changes made to user stories and related artefacts, and provides audit trails for investigation at a later date. ResultSpace also provides an integrated help feature, structured according to key practitioner areas such as administration, planning, and execution, providing easy navigability and structured learning experiences. ResultSpace facilitates end-user collaboration through in-built wikis. The functionality enables users to store related documentation for all development process artefacts as wiki pages linked to the artefacts themselves for easy reference. Related artefacts can also be linked to one another, providing traceability through the lifecycle. The wiki feature also allows users to create custom project dashboards using pre-built charts such as burnup and burndown reports, defects, test cases, and other project metrics. ResultSpace reporting and data-mining features enable stakeholders to prioritize information delivery needs with the help of customizable business rules and workflows. Data-mining capabilities allow users to view real-time reports for metrics such as project progress and product quality. There are also customizable report templates available for reporting on iteration checkpoints and defects.
304
ResultSpace allows organizations flexible deployment options. Depending on usage needs and the size of the organization, customers may choose to deploy ResultSpace on-premise, or access it online via SaaS. The on-demand version of ResultSpace is known as ResultSpace Managed Services. Sapient provides 24x7 technical support around the globe for ResultSpace customers. For on-premise deployments, Sapient provides professional services to help enterprises with deployment and integration with existing enterprise applications.
Company profile
Sapient is a global provider of IT consultancy and services. The business is divided into Sapient Nitro (formerly Sapient Interactive and Nitro Ltd), Sapient Global Markets, and Sapient Government Services. Sapient Nitro is a marketing agency involved with brand and marketing strategies, and web design and development. Sapient Global Markets offers advisory services, analytics, technology development, and business process solutions to organizations in the financial and commodity markets. Sapient Government Services helps organizations with business and IT strategy formulation, process and system designing, program management, and custom development, as well as systems integration and outsourced services including software testing, application support, and maintenance. Sapient went public in 1991, and acquired the Derivatives Consulting Group (DCG) in August 2008. The company is headquartered in Boston, Massachusetts. It employs nearly 7,000 people, and has 23 other offices in North America, Europe, and India. Sapients clients include Global 2000 organizations. From a vertical perspective, Sapient clients come from various industry verticals including media and entertainment, retail, travel and hospitality, technology, communications, financial services, transportation, automotive, healthcare and life sciences, education, energy and utilities, and government agencies. Customers include BP, Essent Energie, Hilton International, Janus, Sony Electronics, and Verizon. Sapient revenue figures for the last three years are in the table below. Year ending 31 December Revenue ($ million) Change on previous year (%) Total Net Income/(Loss) ($ million) Sapient 131 Dartmouth Street 3rd Floor Boston MA 02116 USA Tel: +1 (617) 621 0200 Fax: +1 (617) 621 1300 Email: info@sapient.com www.sapient.com 2010 863.5 29.5 43.8 2009 666.7 29.5 88.12 2008 687.5 21.5 62.48
Sapient UK Headquarters Eden House 8 Spital Square London, E1 6DU UK Tel: +44 (0)207 786 4500 Fax: +44 (0)207 786 4600 Email: info@sapient.com
Seapine Software
Seapine Software offers integrated software development and testing tools to streamline application development and QA processes. Seapine ALM comprises of a set of modules for requirements management (TestTrack RM), development workflow and issue management (TestTrack Pro), software configuration management (Surround SCM), test case management (TestTrack TCM), and automated software testing and load testing (QA Wizard Pro) functions, providing complete traceability of artefacts and greater control over the application development process.
305
Requirement planning, workflow, traceability, review, change management, and reporting are handled by TestTrack RM. The customizable nature of the tool allows workflow configuration, renaming and custom field creation, and defining event-based triggers. End-to-end traceability of project artefacts is achieved through seamless integration with TestTrack Pro, TestTrack TCM, and software configuration management tools such as CVS and Serena PVCS. It supports the import of requirement documents in MS Word format, and a hierarchical folder-like arrangement structure provides a familiar method for organizing project information. Collaboration is facilitated through automated email notifications, RSS feeds, and email conversation tracking. TestTrack RMs list, detail, distribution, and trend reports and charts enable the tracking of requirement-related metrics and data analysis. Surround SCM is the configuration management module that tracks changes to the development artefacts and controls access to files. The data is stored in an RDBMS database, allowing customers to leverage third-party reporting tools, existing database infrastructure, and backup strategies. It provides micro-level control and reporting of the change process of individual files. In addition, it allows users to define, control, and track the individual state of files undergoing change. Change actions can be grouped together and users can also view the history to see which files were processed together. Custom metadata can be defined for every file, such as users assigned to work on a file, file description, file ownership, or due date. Triggers can be defined to automate state transitions, enforce workflow rules, send email notifications, run external applications, modify custom fields, perform data validation, and log information. Integration with IDEs such as Visual Studio and Eclipse allows developers to check code in and out of the software repository from their preferred development environment. TestTrack Pro is the issue management tool for tracking and managing defects, feature requests, change requests, development assignments, customer issues, and other work items. When an issue is reported more than once, TestTrack Pro tracks all instances as a single issue. It supports distributed teams with centralized and automated issue management. The TestTrack add-ins enable integration with popular IDEs such as Microsoft Visual Studio, while TestTrack TCM manages all the aspects of software testing process including test case creation, scheduling, execution, measuring, and reporting. Thousands of test cases can be created and run using the TestTrack TCM client, which can also be accessed over the web to run tests, without requiring client installation. Test cases can be run in groups, and test case execution time (planned and actual) can be tracked to improve resource planning. Test data and automated test scripts can be attached to each test case. Seapines QA Wizard Pro is a tool to automate the functional and regression testing of web, Windows, and Java applications as well as the load testing of web applications. Load testing helps in determining the behavior of a web application by simulating real-world scenarios and increasing the load on the web server. The product offers dual-mode scripting that supports the needs of novice and experienced testers, while novice testers create simple scripts in the GUI-based grid view, more experienced testers can use the text view to write codes in scripting languages. All the test assets are stored in a central application repository, and changes to these assets are reported to all team members from this location. The QA Wizard Pro and TestTrack TCM integration helps manage script scheduling and execution. The QA Wizard Pro and TestTrack pro combination also helps to streamline the test-fix-verify process and automatically pushes new defects into the defect tracking workflow.
Company profile
Seapine Software is a Cincinnati, Ohio-based provider of quality-centric application lifecycle management solutions. The company was established in 1995 and has a total of 100 employees across offices in the US, the UK, Australia, and Germany. The company offers products for software development issue management, advanced automated testing and load testing, requirements management, and test case management. Seapine has more than 8,500 customers globally, including ABS Bank, Amgen, Cisco, Pfizer, Siemens Energy, US Department of Defense, US Air Force, Walt Disney Motion Pictures & Television, and Wachovia.
306
Seapine Software Inc 5412 Courseview Drive Suite 200 Mason Ohio 45040 USA Tel: +1 (513) 754 1655 Fax: +1 (919) 754 1660 Email: sales@seapine.com www.seapine.com
Seapine UK Headquarters Marimed House 4 Lindley Place Kew Surrey, TW9 3PN UK Tel: +44 (0)208 948 9460 Fax: +44 (0)208 948 9462 Email: salesuk@seapine.com
Tasktop Technologies
Tasktop is the company behind the innovative task-focused interfaces for productivity gains by organizing digital work around tasks and showing only the relevant information for each task, based on how the user interacts with the information through various software applications. These interfaces reduce the need to repeatedly search and scroll to find information while minimizing the impact of interruptions because information relevant to each task can be instantly recalled. The company created Mylyn, an open source task-focused interface for software developers using the Eclipse platform. Mylyn has been adopted by hundreds of thousands of Eclipse users and has become the most widely used Eclipse project that is not IBM-sponsored (BZ Media, 2008). The user community has built dozens of connectors to enable the use of Mylyn with their project-tracking systems. The company took its open-source product into the commercial arena with Tasktop Pro, a full-featured supported product based on Mylyn, with additional productivity features. Tasktop extends the productivity benefits of Mylyn to a developers entire work day by adding task-focused support for activities such as web browsing, and working with documents, email, and calendars. A stand-alone version of Tasktop makes the benefits of the task-focused interface accessible to non-technical users. Tasktop recently announced Code2Cloud, a tie-in with SpringSource, a division of VMware. Code2Cloud builds on Mylyn, Hudson, Git, and Google Web Toolkit (GWT) to provide turnkey application-hosting. Users sign in from the SpringSource Tool Suite (STS), start the wizard, and the application, source code, tasks, and continuous integration builds are hosted. After fixing a defect in STS, a Hudson build is automatically kicked off. When successful, the build is automatically deployed at any deployment destination of choice such as VMforce. The running application is then connected back to the application lifecycle tools, and monitoring technologies automatically create defects with the code related to runtime problems captured as a Mylyn task context. Code2Cloud provides a new platform for innovation around the intersection of application lifecycle tools and cloud deployment. The solution is designed from the ground up to be extensible and integrated. On the extensibility front, it provides a set of REST APIs for Mylyn-based IDE access, and even the GWTbased web front end is entirely built on these APIs. In addition, Tasktop provides full support for the Open Services for Lifecycle Collaboration (OSLC) web services specification on which Tasktop collaborated with IBM. The entire solution is open-source, ensuring that developers extending the solution are free to innovate and not limited to the inherent limitations of API boundaries.
Company profile
Tasktop Technologies, headquartered in Vancouver, Canada, is a private company founded by Dr Mik Kersten, CEO, and Dr Gail Murphy, CTO. Tasktop has a market presence that is way in excess of the companys size, which as of early 2010 numbered fewer than 20 staff. The presence is restricted to the Eclipse community, and as a small player, the company is sticking to its niche and ignoring the temptation to expand its task filtering into a broader workflow or process management solution.
307
Apart from IBMs Rational Team Concert and Microsoft VSTS, few other tools are available that address task management for best-of-breed environments. Because few vendors view the company as competitive threat, this has cleared the way for Tasktop to gain wide third-party support. Like most vendors that have built products around open-source technology, Tasktop has a dual-tiered marketing strategy, using the open-source Mylyn project, from which the company and its product originated, as a loss leader for creating market awareness in the open-source community while positioning the commercial product with extended functionality and formal support. For Mylyn, the dividing line with its commercial product is logical, as the integrations to other commercial tools require the extended product. The result is a company whose presence has been considered sufficiently non-threatening to the rest of the Eclipse tools provider community that its product has become the de facto ALM integration layer for the Eclipse development platform. This is an accomplishment that eluded the abortive application lifecycle framework (ALF) Eclipse project led by Serena. Tasktop has implementation and distribution partnerships with many vendors in the ALM space including HP, IBM Rational, Microsoft, SpringSource (now part of VMware), Rally Software, Atlassian, CollabNet, ThoughtWorks Studios, and VersionOne. Tasktop Technologies Inc 127-970 Burrard Street Vancouver BC V6Z 2R4 Canada Tel: +1 (778) 588 6896 Email: Info@Tasktop.Com www.tasktop.com
Tomos
Tomos offers a lightweight application lifecycle management solution that is available as both a hosted and a SaaS-based offering. The product is built on a LAMP architecture comprising of Red Hat Linux Enterprise 5, Apache 2.2.8, MySQL 5.0.51a, and PHP 5.2.5. The web-based nature of the solution ensures access for any team member irrespective of location. Tomos is available for a subscription price of $49 per user per month. The trial version can also be downloaded and tried for 30 days. The Tomos offering contains individual modules for version/build management, requirements management, test authoring, test execution, defect tracking, collaboration, analytics, and reporting. The requirements management module enables users to define, gather, and manage their requirements. Key features include a tree or grid-based structure, complete traceability from requirements to test cases to defects, tracking the audit history of requirements, the ability to attach files to requirement specification, and automated email notifications to team members on changes in requirement specification. The Version and Build management tool enables the creation and testing of multiple builds, assigning version control for all software assets (requirements, test cases, defects), run-test execution reports, and determining the readiness of the build. The Test Authoring module enables users to generate test cases and tie them up with the requirements and defects. These test cases can be organized and managed based on the risk and status, and there is complete traceability of the history of a test case to ensure that the test suite is always updated. Once defined, the test cases can be executed through a single click while the appropriate metrics are rendered. Email notifications are sent in the event of a test case failure, and the uncovered defects are immediately created and submitted to the defect management module. Defects can be organized according to type, assigned a severity and priority level, or even traced back to the developer who produced them. Tomos facilitates collaboration among developers by allowing them to create and organize hierarchical folder structures for storing and sharing their artefacts and assets including release notes, functional specs, user guides, project plans, test plans, attachments; pictures, wav files, Word and Excel docs, and emails.
308
Tomos analytics features include the availability of role-based dashboards, the ability to add or remove widgets as required, and drill-down capabilities. Reports can be generated according to module, role, build, key metrics, priority, and time frame. Tomos also has a social networking interface where product teams can share ideas and provide feedback via forums, blogs, groups, and instant messaging.
Company profile
Headquartered in New York, Tomos is a provider of SaaS application lifecycle management solutions. The company was established in 2007. The product was developed by a team of senior management and technology experts in the ALM space, who recognized a need for an inexpensive web-based system to manage the development and testing processes. Clients include Wyoming Office of State Lands & Investment and House party Inc. Tomos Software LLC 360 Lexington Avenue 9th Floor New York NY 10017 USA Tel: +1 (212) 682 1315 Fax: +1 (646) 417 6104 Email: info@reachsimplicity.com www.reachsimplicity.com
VersionOne
The ALM offering from VersionOne, VersionOne Ultimate, is primarily designed for supporting Agile software development and provides the project planning and management functionality needed in Agile projects. All the key aspects of Agile practices, such as Scrum, are addressed: managing backlogs, estimating stories, prioritizing stories, capturing action to be taken during retrospectives, and tracking real-time progress using velocity and burn-down charts. The product is also a platform for open ALM, providing connectors for many leading ALM tools, and also making it easy for connectors to be built on the royalty-free open-source integration platform. The product is a rich Internet application delivered through a standard browser and can be delivered as a hosted software-as-a-service offering or deployed on-site. VersionOne Ultimate is suited to the SME market transitioning to Agile and has an easy-to-navigate layout. The open connectivity platform provides the option to continue to use existing ALM tools or select best-of-breed/spot ALM products. The vendor offers prospective customers an opportunity to evaluate the product before making a commitment. The growth of Agile adoption adds a degree of complexity and change to the development process, requiring greater discipline than other approaches. A new generation of ALM tools is therefore needed to support Agile teams as they follow their chosen methodology. VersionOne Ultimate supports Agile management with the following features: Product planning to plan and manage requirements, epics, stories, and goals across multiple projects, products and teams. Release planning to plan, forecast, and report progress on releases and teams in a simple drag-anddrop environment. Sprint planning to iteratively plan stories, defects, tasks, tests, and impediments in a single easy-touse environment. Tracking to track progress painlessly using the interactive storyboard, taskboard, testboard, and burndown charts. Review to close out iterations and capture issues and action items from retrospectives in one place. Reporting and analytics, with more than 50 pre-packaged Agile metrics and reports plus a new custom analytics platform for greater visibility.
309
Test management to plan and track acceptance and regression testing activities in the same tool as the stories and defects are managed. Collaboration for constant communication and coordination with customers and between team members. Open-source integration, with open, web services API, Java, and .NET SDKs, and free open-source integration connectors. Product roadmapping to create, collaborate, and visually communicate product strategy using VersionOnes flexible roadmapping capability.
Company profile
VersionOne is a privately owned company headquartered in Atlanta, Georgia, with offices in San Francisco, California and Vancouver, Canada. The company was founded in 2002 and the following year it released XP- and Scrum-specific project management tools, providing a hosted/SaaS option as well as on-site options. Since then, VersionOne has launched many products and initiatives in the Agile software development space. VersionOne has about 100 employees and over 1,000 corporate customers of the solution, including 30 of the Fortune 100, with more than 30,000 Agile teams in 170 countries using its products. VersionOne Inc 6220 Shiloh Road Suite 400 Alpharetta GA 30005 USA Tel: +1 (678) 268 3320 Fax: +1 (678) 513 0720 Email: info@versionone.com www.versionone.com
Zend
Zend is the company behind the PHP programming language, and as well as supporting the language, it offers products to enhance the experience of working with PHP. The OSS Zend Framework offers PHP developers an equivalent to the popular Java Struts, and the Zend Server offers performance monitoring, application management, debugging, cluster management, job queues for off-line processing, and a Java bridge for integration with Java EE. Zend Studio for Eclipse is a development environment for building rich Internet applications. The Zend product portfolio comprises Zend Core, Zend Framework, Zend Server, and Zend Studio. Zend Core: Production-quality PHP 5 for PHP professionals. The Zend-certified PHP distribution has the most up-to-date version of PHP, with tested PHP extensions, database drivers, and other enhancements, to provide a reliable PHP package. Zend Core is bundled with Zend Framework (the open-source PHP framework), Apache web server, and MySQL and IBM DB2 Express-C databases. It also supports Microsoft Internet Information Web servers, Microsoft Windows Server 2008, Informix, and Oracle databases. Zend Framework: The PHP application framework is designed to help build web applications and services using pre-built object-oriented libraries and patterns, increasing productivity and accelerating development. Features include Ajax support using JavaScript Object Notation, a search engine based on Lucene, a facility to consume and publish web services, easy access to data through data adapters to key databases, and an object-oriented PHP 5 class library. The framework is similar to the popular Java Struts framework, supporting a model-view-controller pattern.
310
Zend Server: The enterprise-level web application server for running and managing PHP applications in Linux, Windows, or IBM system i platforms. It performs core performance and management functions and provides additional enterprise functionality for multi-server environments. Features include in-built application performance monitoring, code tracing, the ability to handle large-scale applications with high responsiveness, centralized configuration management (through the Zend Server Cluster Manager) and SNMP integration, Java Bridge for integration with Java objects and services, and job queues for off-line processing. Zend Studio: A PHP IDE that leverages the popular Eclipse platform. Currently in v8, it combines Zend technology with the Eclipse PDT project to create an IDE for developing rich web applications. Zend Studio provides Zend PHP editor technology, in-built database connectivity, integrated debugging, profiling, code coverage and testing capabilities, and multi-language support and extensibility. It allows debugging of the JavaScript front-end and the PHP back-end in one combined session through a set of integrated Ajax tools. Support for VMware workstation virtual machines ensures deployment of PHP code onto a local virtual system, reducing expense and effort of using a remote physical server. In addition, the IDE integrates with Zend Core and Zend Server, and provides team support with extensible version-control capabilities. The debugger enables one-click reproduction of issues with comments and data reported by the end user for backward traceability and quick resolution.
Company profile
Zend is a privately held company headquartered in Cupertino, California, with offices in Ramat Gan, Israel, Munich, Germany, and Rosny sous Bois, France. Zend is the company associated with the PHP language. It was founded in 1999, four years after Rasmus Lerdorf first introduced the language. Zend is backed by venture capitalist firms Azure Capital, Greylock Partners, Index Ventures, Intel Capital, Platinum Neurone Ventures, SAP Ventures, and Walden Israel Capital. Including its developer products, Zend has more than 20,000 customers worldwide. Zend Technologies Inc 19200 Stevens Creek Blvd Cupertino CA 95014-2530 USA Tel: +1 (408) 253 8800 Fax: +1 (408) 253 8801 www.zend.com Zend Germany Zend Technologies GmbH Bayerstrasse 83 80335 Munich Germany Tel: +49 (0) 895 161 990 Fax: +49 (0) 895 161 9920
311
Incorporating
OVUM Butler Group
Appendix
WWW.OVUM.COM
Glossary
Ajax (formerly Asynchronous JavaScript and XML) A form of RIA technology that uses JavaScript and XML, which are ubiquitously supported on all modern web browses, making Ajax the most platform-independent RIA approach. ALF (Eclipse ALM Framework) The Eclipse ALM project, terminated in 2008. Agile development A group of software development methodologies subscribing to the Agile manifesto that are built around several core concepts: iterative and incremental development; short development and testing cycles aimed at producing workable code; rapid customer feedback; continuous integration testing; and small self-organizing teams. There are numerous variants of Agile methodologies including Scrum, eXtreme Programming (XP, which also introduced test-driven development), and Lean development. ALM (application lifecycle management) The combination of business management and software engineering made possible by tools that facilitate and integrate: project and portfolio management; requirements definition and management; software change and configuration management; architecture, design, and modeling; coding and testing; test management; build, release, and deployment management; defect and issue management; application performance management; and software asset management. ALM involves processes, methodologies, and people, facilitated by automation. API (application programming interface) The prescribed method by which an application program can make requests to an operating system or to another program. APM (application performance management) The monitoring, analysis, optimization, simulation, and troubleshooting of application performance. Usually applied in production but also applicable during development. Modern systems include customer user experience monitoring. APPM (application project and portfolio management) An application-centric extension to project portfolio management (PPM) that goes beyond the project stage to factor the costs and value of applications in production. There are several approaches to extending PPM to runtime. It can take an application-centric view that factors service desk incidents and reported problems, an IT service-oriented view that tracks the portfolio of IT services that encompass the application, or an asset management-based view that treats the completed project as a software asset. See also project portfolio management (PPM). Asset lifecycle management Also known as asset management, asset lifecycle management tracks the portfolio of fixed or IP assets based on cost, risk, control, compliance, and business performance objectives from the creation or acquisition of the asset to its retirement. See also ITAM and SAM. BI (business intelligence) The process of extracting, collating, analyzing, and distributing pertinent data from existing systems to support management decision-making. BPEL (Business Process Execution Language) An OASIS standard for assembling a set of discrete services into an end-to-end process flow using web services. BOM (bill of material) This is the list of raw materials, assemblies, and components that comprise a product. There are two main forms of BOM including the engineering BOM, which describes the product the way it is designed, and the manufacturing BOM that describes the product by the way it is built. There are other variants that address sales and service. The BOM is relevant to the application lifecycle when embedded software is designed into the electronic content of products.
CHAPTER 10: APPENDIX
315
BPM (business process management) A set of standards, languages, methods, and software applications for managing the automation of IT processes within a business, complementary to SOA. BPMN (business process modeling notation) A standard for graphically representing business processes in a workflow. BSM (business service management) Software to help organizations manage their applications and systems along business lines rather than IT lines. Business as a service Using the cloud for delivering business process outsourcing (BPO). CADCAM (computer-aided design/computer-aided manufacturing) A general term describing the use of software tools for aiding the design, engineering analysis, and manufacture of products. Change This is a generic term describing an event that results in a new status for specific artefacts. As a generic term, it is used in the context of the SDLC/application lifecycle, and for ITIL in ITSM. Change impact analysis This is the process of analyzing the effects of introducing new or modified artefacts such as IT service, software, or infrastructure, and is typically conducted when planning new releases of software or IT services. The term is frequently used in conjunction with the SDLC/application lifecycle and ITIL. See also change management. Change management The discipline for controlling the impact of change through use of standardized processes that track changes, analyze their impact, and as a result of change-impact analysis, are then administered to minimize disruption. A generic concept, this term is used in the context of the SDLC/application lifecycle, and for ITIL in ITSM. CI (configuration item) An ITIL term covering artefacts that are tracked in a CMDB. CIs are elements of IT infrastructure and their configurations that are essential for delivering IT services. There is no set prescription for what artefacts to track with Cis. Every organization will have different requirements as to what CIs should be included in a CMDB. See also CMDB, ITIL. Cloud computing A model for delivering compute resources over the Internet, cloud computing provides services using shared resources and/or software that is provided on demand. There are many different forms of cloud computing that may use public resources (where cloud resources are available to any customer on demand) or private resources (where cloud resources are dedicated to a specific enterprise). For example, Amazon EC2 is an example of a public cloud. See also EC2, IaaS, PaaS, and SaaS. CMDB (configuration management database) Defines the configuration items (IT and non-IT components) used in the delivery of corporate IT services, their attributes, and their relationships with other configuration items. It is the single source of truth for component information and is a vital enabler to many ITIL v3 processes, including change, incident, and capacity management. It can also operate as a meta-database that links to configuration item data held within various data sources throughout the organization, a configuration management system. CMM (Capability Maturity Model) The Software Engineering Institute (SEI) has produced a number of CMMs, these have now been superseded by the latest Capability Maturity Model Integration (CMMI), with sub-divisions, for example, in software. CMMI provides best-practice characterization of management processes with the aim of achieving high-quality production.
316
CMMI (Capability Maturity Model Integration) A process improvement approach that provides organizations with the essential elements for effective process improvement. It can be used to guide process improvement across a project, a division, or an entire organization. CMMI helps integrate traditionally separate organizational functions, set process improvement goals and priorities, provide guidance for quality processes, and provide a point of reference for appraising current processes. Configuration management The process that tracks all configurations, it is used in the context of the application lifecycle for performing version control of software, and in ITIL for tracking the configuration items (CIs) of an IT service. See also CMDB, SCM, and SCCM. CPU (central processing unit) One or more CPUs give a computer the ability to run software, such as an operating system. DSDM (Dynamic Systems Development Method) An Agile development methodology, originally rooted in Rapid Application Development (RAD) that is based on iterative development practices that respond to changing requirements. Dependency management The tracking of interdependencies between software and other software, and with IT infrastructure. Dependency management is critical for managing cloud-based software deployment. EA (enterprise architect) Enterprise architects work with stakeholders, from management to domain experts, to build a holistic view of the organizations strategy, processes, information, and information technology assets. EAs are responsible for linking business mission, strategy, and processes of an organization to its IT strategy. Their goal is to deliver an architecture that supports the most efficient and secure IT environment meeting a companys business needs. See also enterprise architecture. EC2 (Elastic Compute Cloud) Amazons commercial web service that allows paying customers to rent computers on which to run their own computer applications. EC2 allows scalable deployment of applications by providing a web services interface through which customers can request an arbitrary number of virtual machines (server instances) on which they can load any software of their choice. See also cloud computing. Embedded software Software that plays an integral role in the electronics with which it is supplied. Most engineered products contain electronics that provide better control, more efficient operation, or more capabilities that are all powered by software. Embedded software is also known as firmware. Enterprise architecture A comprehensive framework that defines how information and technology will support the business operations and provide benefit for the business. It illustrates the organizations core mission, each component critical to performing that mission, and how each of these components is interrelated. Also see EA (enterprise architect). ERP (enterprise resource planning) A suite of software that aims to support all the core functions of an enterprise, including areas such as inventory control, accounting, production, logistics, and human resources, in an integrated whole. Governance The process of decision-making and the process by which decisions are made. GUI (graphical user interface) A screen-based user interface that includes icons, pull-down menus, and a mouse. IaaS (infrastructure-as-a-service) Software infrastructure available as a service in the cloud. IDE (integrated development environment) An IT solution to enable IT developers and programmers to produce software.
317
IM (instant messaging) A tool used for communicating real-time text messages between two or more people, usually with presence awareness. Incident An ITIL term describing events that disrupt the standard operation or delivery of IT services. Incident management An ITIL process for restoring normal service operation as quickly as possible after isolated occurrences of service disruption. IP (intellectual property) A catch-all for works of creativity, usually associated with copyright, trademark, or patent rights. IP is also used for internet protocol, see TCP/IP. ISO (International Organization for Standardization) The worldwide federation of national standards bodies, which fosters Open Systems Interconnection (OSI) a universal reference model for communication protocols. ISV (independent software vendor) A company that develops and markets proprietary software products for running under one or more computer operating systems. ITAM (IT asset management) Tracks the portfolio of IT assets, including software and hardware, based on cost, risk, control, compliance, and business performance objectives from the creation or acquisition of the asset to its retirement. See also asset lifecycle management and SAM. ITBM (IT business management) The IT practice that is responsible for overall IT governance. See also ITFM. ITFM (IT financial management) An ITIL process focused on providing accurate and cost-effective management of IT service delivery. ITFM encompasses budgeting for planning IT expenditures, IT accounting for tracking actual expenditures, and charging for providing the mechanism for assigning costs to users of IT services. See also ITBM and ITIL. ITIL (formerly IT infrastructure library) A comprehensive non-proprietary and publicly available set of guidelines for best-practice IT services management. Each library module provides a code of practice intended to improve IT efficiencies, reduce risks, and increase the effectiveness and quality of IT services management and infrastructure. Formerly known as the IT infrastructure library, the name was officially shortened to ITIL with the release of version 3 of the ITIL framework. See also change management, CMDB, CI, configuration management, incident, incident management, ITFM, ITSM, problem, problem management, RACI, release management, and service level management. ITSM (IT service management) The discipline of managing IT services from the customers perspective of how those services contribute to the business. Although frequently confused with ITIL, the two terms are not synonymous. ITSM is a discipline, while ITIL is one of the best-known frameworks for practicing the ITSM discipline. See also ITIL. Java EE (Java Platform Enterprise Edition) Formerly known as J2EE Java 2, Enterprise Edition, Java EE defines the standard for developing multi-tier applications using Java. Java EE simplifies enterprise applications by basing them on standardized modular components, by providing a complete set of services to those components, and by handling many details of application behavior automatically without the need for complex programming. MDA (model driven architecture) The approach to application development of first building models that can be subsequently engineered into code through varying degrees of automation.
318
MDD (model driven development) A software development methodology that focuses on creating models, or abstractions that are closer to specific domain, rather than computing (or algorithmic) concepts. OMG (Object Management Group) An early industry consortium that defined the original Corba specifications. OS (operating system) An interface between hardware and user, it is responsible for the management and coordination of activities and the sharing of the limited resources of the computer. OSS (open-source software) Defined as computer software for which the source code and certain other rights normally reserved for copyright holders are provided under a software license that meets the open-source definition or that is in the public domain. PaaS (platform-as-a-service) The delivery of a computing platform as a service over the web. PDM (product data management) A business function within PLM that handles the creation, management, and publication of product data such as design specifications and metadata. See also PLM. PLM (product lifecycle management) The process of managing the entire lifecycle of a product, from conception through design, manufacture, service, and disposal. When managed properly, PLM integrates people, data, processes, and business systems to provide benefits such as faster time to market, improved product quality, and reduced prototyping costs. As a commercial software solution, PLM is offered by CADCAM and ERP suppliers. See also PLM. PMBOK (Project Management Body of Knowledge) This is a project management guide published by the Project Management Institute. The fourth edition was released on December 31, 2008. PMO (project or program management office) The PMO in a business or professional enterprise is the department or group that defines and maintains process standards generally related to project management within the organization. The PMO strives to standardize and introduce economies of repetition in the execution of projects. The PMO is the source of documentation, guidance, and metrics on the practice of project and program management and execution. PPM (project portfolio management) A set of methods and tools for analyzing and managing a set of projects as a group, helping to monitor project progress, allocate resources, optimize budgets, and rationalize project activity, particularly in aligning projects with business needs. See also application project portfolio management. Problem An ITIL term for characterizing issues that emerge from repeated incidents that have similar symptoms and causes. Problem management An ITIL term for describing the process for resolving root causes of problems that occur in the delivery of IT services. Product lead time The amount of time necessary for conceiving, designing, and manufacturing a product, and delivering it to market. Product lead time is the first half of the product lifecycle. See also PLM. QA (quality assurance) Refers either to the function, or to the organization unit dedicated to carrying out the function, of ensuring that a solution (application, product, or service) is fit for purpose, suitable for the customer, and built to an intended quality level (for example, measured by number of defects below a threshold level).
319
QoS (quality of service) Quality of service is the ability to provide different priorities to different applications, users, or data flows, or to guarantee a certain level of performance to a data flow. RACI (responsible, accountable, consulted, informed) This is standard terminology describing how roles and responsibilities in ITIL processes are defined. It is particularly useful for describing how software development and IT operations (ITO) staff collaborate. RAD (rapid application development) A software development methodology involving rapid development of prototypes and mock interfaces to enable customer/end-user feedback early in the development lifecycle. A formal iterative RAD methodology was devised by James Martin in 1991. A looser RAD practice was linked to tools such as Microsoft Visual Basic and Sybase PowerBuilder, which facilitated a RAD approach. Release management This is a generic concept for managing the introduction of new software or IT services into production, and the term is therefore used in both the SDLC/application lifecycle and in ITIL. Release management is typically conducted after change impact analysis, which provides guidance on how to reduce disruptions associated with releasing new software or IT services, and change management, which provides the goahead to plan a new release. See also change management and change impact analysis. Requirements definition The process of gathering requirements from stakeholders for new software application development projects. See also requirements management. Requirements management The process of codifying and maintaining requirements for downstream use by software development teams and business stakeholders. See also requirements definition. REST (representational state transfer) REST uses basic HTTP post, get, put, and delete calls to perform CRUD (create, retrieve, update, delete) operations against a data source that is identified by its web address, or URI (uniform resource indicator). See also RESTful services. RESTful services A simpler more popular alternative to web services that is used for data-oriented services that uses REST software design principles instead of SOAP messages. See also REST. RIA (rich Internet application) The technology behind web browsers and Internet platforms that allows applications to run across the Internet asynchronously, offering a rich, desktop-like user experience. Ajax helped push RIA into the mainstream of application development. ROI (return on investment) A term to describe how much of a return, usually profit or cost-saving, results from a completed business task in relation to the original investment made. SaaS (software-as-a-service) An application deployment model where software is hosted remotely and provided as a service over the web. Multi-tenancy (where multiple customers share the same central application) and a subscription payment model were core characteristics of the original SaaS concept, but new tenancy models, such as isolated tenancy, are now included. SAM (software asset management) SAM aims to ensure the provision of the right IT applications to the correct people, at the right time, and at the right cost. It includes all the processes and infrastructure required for the effective management and protection of software assets within an organization, through all the phases of their lifecycle. See also asset lifecycle management and ITAM. SCCM (software change and configuration management) Similar to SCM, but with specific features for managing source code changes and workspace management. SCM and SCCM are often used interchangeably.
320
SCM (software configuration management) A system for tracking, version controlling, and storing all entities related to a project. SDK (software development kit) A set of resources and APIs for developing custom extensions and integrations to an application. SEI (Software Engineering Institute) The Carnegie Mellon University institute behind CMMI (formerly CMM). Service contract Service contracts formally stipulate the terms and conditions by which a service will be deployed and made available to service consumers. Provisions encompass SLAs, security, and indemnification for non-delivery of service. See also SLA, service policy, and SOA governance. Service level management This is the process for monitoring service levels and taking actions for enforcing SLAs at runtime. A generic concept, service level management is typically used for ITIL and SOA runtime governance. Service policy Service policies stipulate the rules governing all aspects of SOA service consumption/utilization. See also SLA, service contract, and SOA governance. SI (systems integrator) An individual or company that combines various components and programs into a functioning system customized for a particular customers needs. SLA (service level agreement) An agreed level of service between the provider of that service and the customer. See also SOA governance and service contracts. SLM (software lifecycle management) Software lifecycle management is Ovums revised term for the application lifecycle management (ALM) space. It represents a philosphy and approch to managing software development as well as tools. SME (small to medium-sized enterprise) A company comprising no more than 500 employees. SOA (service oriented architecture) An architectural concept that places process components delivered as consumable services at its heart. In its modern incarnation this architecture is chiefly based on web services, providing a services platform layer that exposes business and operational services, and is typically part of enterprise architecture. SOA governance The process for determining when SOA services should be created, how they should be created, enforcing service contracts at runtime, and for determining how and when services should be reused, modified, deprecated, or retired. In practice, there are two levels of SOA governance covering design time (creation of services) and runtime (enforcement of service contracts and policies related to them). Software assurance A category that spans QA, software security, and application performance management, and provides a holistic approach to delivering working software. SPL (software product lines) A discipline for adapting software engineering practices to the needs of companies that develop families of products. SPL operates on two core principles. First, products are comprised of features, which specify the characteristics, behavior, and function of embedded software (as opposed to what specific lines of code or components comprise it). Second, products within a product line are not designed from scratch, but are configured from product features approved for that specific family. Supply chain A system of organizations, people, technology, activities, information, and resources involved in moving a product from supplier to customer.
321
Supply chain management The integration of business processes and operations required for directing and optimizing the supply chain. The goal is to fulfill customer demands through the most efficient use of resources based on cost and delivery goals. TCO (total cost of ownership) A type of calculation designed to help consumers and enterprise managers assess both direct and indirect costs and benefits related to the purchase of any IT component. TCP/IP (Transmission Control Protocol/Internet Protocol) TCP and IP were developed by a Department of Defense (DOD) research project to connect a number different networks designed by different vendors into a network of networks (the Internet). It was initially successful because it delivered a few basic services that everyone needs (file transfer, electronic mail, and remote logon) across a very large number of client and server systems. Several computers in a small department can use TCP/IP (along with other protocols) on a single LAN. The IP component provides routing from the department to the enterprise network, then to regional networks, and finally to the global Internet. TDD (test-driven development) An approach to developer testing where the developer writes a unit test, then writes the code, runs the unit test, modifies/refactors the code until the test passes, and continues this cycle with the next unit test. TQM (total quality management) A quality management practice derived/influenced by a number of sources, including the Toyota Production System. UAT (user acceptance testing) The phase in the development lifecycle when an application, or a part of an application, is delivered to the end users for testing. Users are mainly interested in testing the application to meet their usage requirements rather than seeking to break the application, which is the job of dedicated testers. UI (user interface) The front end of an application covering the human-machine interface. This will often involve a graphical UI, GUI. UML (Unified Modeling Language) UML is an object-modeling and specification language used in software engineering. Virtualization The abstracting of IT resources from physical resources. This is an extremely broad term because there are many types of virtualization strategies that cover software, operating system images, storage, and so on. VM (virtual machine) A virtual machine is a tightly isolated software container that can run its own operating systems and applications as if it were a physical computer. XML (eXtensible Markup Language) Markup language derived from SGML, defined in 1998 by the W3C as a Recommendation. Used as a meta-language to describe data, it is now finding widespread applications in areas such as application integration, content management, electronic data interchange, and wireless communications. XML is extensible because unlike HTML, the markup symbols are unlimited and self-defining. Using an XSL Style sheet, XML can be transformed for display as HTML on a web page, or to alternative formats for display on other types of client device. XP (eXtreme Programming) An Agile development methodology originated by Kent Beck (among others) and noted for featuring test-driven development and pair programming.
322
Further reading
Application Lifecycle Governance: Demolishing governance silos to reduce costs and improve alignment, Ovum, August 2010 DevOps: Connecting IT development and operations, September 2010, Ovum, OI00005-001 HTML5: Standards are only half the battle, November 2010, Ovum, OI00005-005 Iterative and Incremental Development: A Brief History, Latman and Basili, IEEE Computer, pp.47-56, 2003 Requirements Management for Development Projects: Bridging the gap between traditional and Agile Requirements Management, February 2010, Ovum Software Development in the Product Lifecycle, April 2010, Ovum, 052147 Web Application Development: four key trends to follow, May 2010, Ovum The following is a selection of reports that discuss mobile application development from the Ovum Telecoms team: Mobile application developer survey: smartphone OS and runtime support, Ovum, 052595 Mobile application developer survey: network and channel requirements, Ovum, 052270 Of iPhones and Androids: redefining the smartphone and other devices, Ovum, 050487 Telecoms in 2020: devices and platforms, Ovum, 051688 Mobile application download and revenue forecast: 2010-15, Ovum, 052544 Mobile application stores: Apple App Store, Ovum 050321 (one of a series of reports on the major mobile application stores) Rich Internet applications in mobile and consumer electronics: impact analysis, Ovum, 045232.
Methodology
This report is based on research conducted by Ovum experts in the ALM field, comprising interviews with the leading vendors, producing Technology Audits, and through their experience in covering the subject matter for a number of years Specific methodology details on the evaluation of the Ovum ALM 2011 Decision Matrix is given in chapter 7.
Authors
Michael Azoff, Principal Analyst, Ovum (IT Solutions) michael.azoff@ovum.com Tony Baer, Senior Analyst, Ovum (Enterprise Solutions) tony.baer@ovum.com Chandranshu Singh, Senior Analyst, Ovum (IT Solutions) chandranshu.singh@ovum.com
Ovum consulting
We hope that the analysis in this report will help you make informed and imaginative business decisions. If you have further requirements, Ovums consulting team may be able to help you. For more information about Ovums consulting capabilities, please contact us directly at consulting@ovum.com.
Disclaimer
All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of the publisher, Ovum (a subsidiary company of Datamonitor plc).
323
Incorporating
OVUM Butler Group
WWW.OVUM.COM
This Report reveals: Ovums Application Lifecycle Management (ALM) 2011/2012 Decision Matrix results. Feature-by-feature comparison of the latest products from the leading ALM vendors. Ovums market analysis and trends in the ALM space. The business value in software lifecycle management. The trends in Agile and Lean process adoption. How the new wave of smart mobile devices is impacting application development. The growth of embedded software and the interface between ALM and product lifecycle management. Trends to watch including DevOps, continuous delivery, and Scrum Kanban. How ALM is evolving and the rise of SaaS delivery.
Incorporating
OVUM Butler Group

Ovum Europe
119 Farringdon Road, London, EC1R 3DA, United Kingdom t: +44 (0)20 7551 9850 e: info@ovum.com
Ovum Australia
Level 5, 459 Little Collins Street, Melbourne 3000, Australia t: +61 (0)3 9601 6700 f: +61 (0)3 9670 8300 e: info@ovum.com
Ovum New York

245 Fifth Avenue, 4th Floor, New York, NY 10016, United States t: +1 212 652 5302 f: +1 212 202 4684 e: info@ovum.com
Driving business value through collaborative intelligence
OI00068-004

Ovum - Technology Evaluation and Comparison Report - Software-Lifecycle-Management-2011-2012

Cargado por

Información del documento

Descripción original:

Derechos de autor

Formatos disponibles

Compartir este documento

Compartir o incrustar documentos

Opciones para compartir

¿Le pareció útil este documento?

¿Este contenido es inapropiado?

Copyright:

Formatos disponibles

Ovum - Technology Evaluation and Comparison Report - Software-Lifecycle-Management-2011-2012

Cargado por

Copyright:

Formatos disponibles

Incorporating

Technology Evaluation and Comparison Report

OVUM Butler Group

Software Lifecycle Management 2011/2012

Enterprise IT Knowledge Centre

Published by Ovum Published June 2011 Ovum

Software Lifecycle Management 2011/2012

CONTENTS SOFTWARE LIFECYCLE MANAGEMENT 2011/2012

CONTENTS SOFTWARE LIFECYCLE MANAGEMENT 2011/2012

CONTENTS SOFTWARE LIFECYCLE MANAGEMENT 2011/2012

CONTENTS SOFTWARE LIFECYCLE MANAGEMENT 2011/2012

Technology Evaluation and Comparison Report

OVUM Butler Group

CHAPTER 1: Executive summary

1.1 Executive summary

CHAPTER 1: EXECUTIVE SUMMARY

SOFTWARE LIFECYCLE MANAGEMENT 2011/2012

CHAPTER 1: EXECUTIVE SUMMARY

SOFTWARE LIFECYCLE MANAGEMENT 2011/2012

The ALM decision matrix

CHAPTER 1: EXECUTIVE SUMMARY

Shortlist Consider Explore

SOFTWARE LIFECYCLE MANAGEMENT 2011/2012

1.2 Report objectives and structure

Chapter 3. The Ovum SLM 2011 technology evaluation model

Chapter 4. Agile and Lean process adoption for software development

Chapter 5. Enterprise mobile development: trends, platforms, and tools

Chapter 6. ALM in systems/product engineering: two case studies

Chapter 7. Technology comparison

Chapter 8. Technology audits

Chapter 9. Vendor profiles

CHAPTER 1: EXECUTIVE SUMMARY

Technology Evaluation and Comparison Report

OVUM Butler Group

CHAPTER 2: 2011 Trends to watch: application lifecycle management

CHAPTER 2: 2011 TRENDS TO WATCH: APPLICATION LIFECYCLE MANAGEMENT

2.2 Application lifecycle management matters

SOFTWARE LIFECYCLE MANAGEMENT 2011/2012

ALM systems are becoming more in tune with user needs

ALM systems must offer business value

ALM systems have traditionally been sold to developers and IT directors...

CHAPTER 2: 2011 TRENDS TO WATCH: APPLICATION LIFECYCLE MANAGEMENT

2.4 Key trend two: Software-as-a-Service ALM sees strong growth

the ALM market without offering a competitive SaaS solution.

Developers are gaining faith in SaaS

SOFTWARE LIFECYCLE MANAGEMENT 2011/2012

2.5 Key trend three: ALM for mobile development

CHAPTER 2: 2011 TRENDS TO WATCH: APPLICATION LIFECYCLE MANAGEMENT

Demand for mobile development environments is emerging

Mobile platform wars get serious and impact developers

Shades of gray emerge

SOFTWARE LIFECYCLE MANAGEMENT 2011/2012

2.6 Key trend four: application and security management

CHAPTER 2: 2011 TRENDS TO WATCH: APPLICATION LIFECYCLE MANAGEMENT

2.7 Key trend five: ALM and product lifecycle management

Agile developers and concurrent engineers talk the same language

SOFTWARE LIFECYCLE MANAGEMENT 2011/2012

2.8 Key trend six: emergence of DevOps

There is an Agile connection: continuous delivery

CHAPTER 2: 2011 TRENDS TO WATCH: APPLICATION LIFECYCLE MANAGEMENT

IT organizations should approach innovations such as Agile and cloud selectively.

Plan for mobile development

Bake security into the application lifecycle