Architecting Testing Solutions
Some products go through several weeks of testing, while some just days of testing before deployment. In most cases, software ends up being over schedule and over budget, and the first function that faces the axe is Testing. This is a dangerous ploy as this may erode your brand equity in the market.

The testing function has always faced challenges. In most companies, enough time and effort is not allocated to testing activities. Companies tend to miscalculate what a successful test effort requires. Rotation of engineers across projects, lack of adequate testing knowledge, cost, etc., often hamper high quality testing, thereby affecting quality of the deliverables. Also, it may not be possible to have a dedicated testing team within a project all the time. In such cases, more often than not, developers double up as Testers.

As a solution to these issues, a new role of Test Architect has been identified at MphasiS. The test architect will provide testing solutions to development/maintenance engagements, define test strategies and manage all test phases to make successful releases and mentor test engineers.

This has been a successful implementation, especially for small projects, where on occasions; Developers have donned the role of Tester.

Test Architect (TA): The Role
A Test Architect has a wider scope within Mphasis. A TA is expected to provide testing solutions across domains and technologies. He is responsible for:

•	Requirements study and analysis
•	Effort estimation for Testing
•	Test Strategy and planning
•	Test Management
•	Release Recommendations
•	Defect Prevention
•	Appropriate Tools identification

Projects benefit from the testing solutions provided by the Test Architect. He acts as an aide to the Project Manager by providing Testing solutions, understanding requirements in detail and understanding the client’s expectations.

Figure-1: Project Structure

The test team consists of a Test Architect, Business Analysts and Test Engineers. Business Analysts will impart domain knowledge, ensure complete coverage of functional requirements and drive business perspective.

Testing Lifecycle
MphasiS follows the QALC methodology - early life cycle testing technique that prevents defect leakage. Architects focus on all aspects of the different testing phases.
As seen in the adjacent figure, SDLC is represented in at the center. QALC blends in nicely with the SDLC right from the beginning right through to the last stage of “Production Release”.

The RAMPS (Reliability, Availability, Maintainability, Performance and Scalability) methodology is followed to test non-functional requirements. These attributes are platform agnostic and focus on non functional requirements, bringing focus on operational requirements with regard to goals and perception.

Testing effort estimation
Use case and function point techniques are used to estimate the testing effort. As per the IFPUG, the number of function points will determine the number of test cases for that project, the formula being (FP) 1.2. Total testing effort is calculated by adding the effort for each test case. Use case technique gives a good estimation for the testing effort.

Table-1: Effort comparison

Project	Method	Estimated Test Cases	Estimated Effort-PD	Actual Test Cases	Actual Effort-PD
HiMS	Function Points	761		875
STS-IPO	Use case points		166		160

Test Plan/ Strategy

•	Gathering business requirements to propose possible scenarios for testing
•	Set testing goals
•	Estimate testing effort
•	Incorporate suggestions from domain experts/business analysts
•	Identify various testing methods like unit, integration, system, regression etc. suitable for the project
•	Study user acceptance criteria for incorporation into test case design
•	Develop tools for code coverage and defect tracking
•	Assess risk in terms of hardware/software/human resources
•	Set the release criteria in terms of organization standards

Architecture Audit – Traceability Verification
Traceability matrix is maintained with respect to testing requirements and their influence on one another. Any changes to any requirement would reflect the impact through the traceability matrix.

There would be an update on Traceability matrix and the tests would be carried at integration and system integration stages with the data defined as per customization sought. New tests would be developed during the iterative intermediate builds for the missed out issues noticed at the beginning of the next build. This iterative process would continue until the final release is made to the customer.

Test Case Design Methodologies
The number of tests would be based on use profile, risk analysis, priority of requirement, equivalence class, boundary value analysis, and cause and effect analysis. Some times testing becomes much more complex because there are multiple factors that affect the software and these factors simultaneously affect the behavior. This leads to a complex suite of test cases. In such projects, orthogonal array testing is employed to obtain encouraging results.

Build Discipline and Release Management
In order to get sufficient time for testing, the TA works closely with the Project Manager for the scheduled completion of development. This brings in build discipline and the testing team has sufficient time to test the build. The number of cycles is decided for testing based on the defects, their severity and priority. The TA will co-ordinate with the development and testing teams to ensure bugs are fixed. The defects are analyzed and compared between the test iterations.

TA not only ensures that the project goals are met, before making the release, but also follows the “Testing to Help Ship on Demand” principle. Here, the TA also decides if the project can be delivered to the customer with the presence of defects, to meet the deadline. However the remaining defects will be listed and fixed subsequent to release.

Defect Prevention
Some of the metrics collected during testing are code review efficiency, testing efficiency, testing effectiveness and code coverage. Code review efficiency ranges between 60% - 70% while testing efficiency ranges between 75% - 90%. In order to improve the testing efficiency, consistently to 90% - 95%, Pareto analysis is carried out based on the taxonomy factors.

Test Automation

Selection of Tool
Business Requirements - It is important to create a list of initial requirements for a clear understanding of the capabilities and constraints of the tool
Compatibility Issues - Any testing tool we choose will need to be compatible with the operating system(s), development environment(s) and the third party software with which the application integrates
Tool Audience - The skill level of the users will play a part in determining the best tool
Budget Constraints - The test tool budget must take licensing, training and implementation costs into consideration

Implementation
After selecting the automation tool, the following is considered:

•	Training time
•	Implement one step at a time: apply the tool to a single problem rather than trying to apply it to all problems at once
•	Immediately adopt measures to evaluate performance of the tool and project, so the management is apprised of progress over time
•	Communicate clearly, often, and in as many different ways as possible with all of the people in the organization affected by this new tool
•	Finally, regularly update requirements. Traceability should be maintained to ensure that automation covers all the requirements

Test Methodology

Main approaches used are
•	Rapid application – multi-tier approach framework
•	Key word Driven approach
•	Data Driven

Advantages of the Data Driven method:
•	Platform and Test-Tool independent
•	Can be used with any Test Tool which provides a "scripting language"
•	Allows the tester to create Automated Test Scripts without having to learn the scripting language
•	Can be enhanced by adding Business Function Utilities, which provide the advantages attainable by both methods
•	Additional Test Script re-usability can be obtained by using parameterized input rather than data at key points in the Test Case

Case Study