- Ant integration
Probably most Java projects today are using Ant (or Maven) to manage their build process, including running unit tests (or functional tests).
Thus, Ant integration is one of those features a code coverage tool cannot afford not to have. However, there are subtle differences in how nicely the code coverage related targets fit into an existing build script. Yes, we will see some examples later on. Of course, most tools also provide an alternative, standalone way of running the analysis, either from the command line or via a GUI application.
- Report formats
Another obvious feature a code coverage tool must have is reports. Again, there are differences in the type and quality of supported reports. Some tools provide only textual summaries in Ant console output, others produce huge tables of names and numbers in HTML, and others produce nice pictures, others offer to render all this in PDF as well. We'll see examples of the main types of reports in the next section.
- Source code linking
Somewhat related to the previous item, source code linking is something one can't live without once having gotten the taste of it. In practice, source code linking means that, as a part of the code coverage report, the tool has generated annotated copies of the actual source code, highlighting the parts which are not covered by your tests. I wouldn't be surprised if this particular feature was the single biggest benefactor in code coverage tools reaching critical mass. Seeing the code block that is causing your coverage to stay away from the "green" is a lot more efficient than seeing that a particular method contains some code that isn't covered by the tests. All of our selected examples include source code linking in their feature set.
- Checks
It doesn't take too long after someone has introduced a rule when someone else introduced the idea to enforce that rule. That has also happened to code coverage. Some tools provide a means to pull up the red flag if code coverage drops below a given level. In the context of Ant integration, the build script might typically fail the build until the tests have been reinforced to cover the naked parts of the code.
- Historical reports
Few tools provide a way to collect a history of coverage data and produce historical reports illustrating how your project's code coverage has been fluctuating in the course of time. This is also a nice feature, although some might consider it rather irrelevant.
Monday, August 10, 2009
Code Coverage - Typical features
Thursday, August 6, 2009
Testing Estimation Process
Introduction
In my opinion, one of the most difficult and critical activities in IT is the estimation process. I believe that it occurs because when we say that one project will be accomplished in such time by at such cost, it must happen.
If it does not happen, several things may follow: from peers' comments and senior management's warnings to being fired depending on the reasons and seriousness of the failure.
Before even thinking of moving to Systems test at my organization, I always heard from the development group members that the estimations made by the Systems test group were too long and expensive. Then, when I arrived at my new seat, I tried to understand the testing estimation process.
The testing estimation process in place was quite simple. The inputs for the process, provided by the development team, were: the size of the development team and the number of working days needed for building a solution before starting systems tests.
The testing estimation process said that the number of testing engineers would be half of the number of development engineers and one third of the number of development working days.
A spreadsheet was created in order to find out the estimation and calculate the duration of tests and testing costs. They are based on the following formulas:
Testing working days = (Development working days) / 3.
Testing engineers = (Development engineers) / 2.
Testing costs = Testing working days * Testing engineers * person daily costs.
As the process was only playing with numbers, it was not necessary to register anywhere how the estimation was obtained.
To exemplify how the process worked, if one development team said that to deliver a solution for systems testing it would need 4 engineers and 66 working days then, the systems test would need 2 engineers (half) and 21 working days (one third). So, the solution would be ready for delivery to the customer after 87 (66+21) working days.
Just to be clear, in testing time, it was not included the time for developing the testcases and preparing the testing environment. Normally, it would need an extra 10 days for the testing team.
The Rules
1st Rule: Estimation shall be always based on the software requirements
All estimation should be based on what would be tested, i.e., the software requirements.
Normally, the software requirements were only established by the development team without any or just a little participation from the testing team. After the specification have been established and the project costs and duration have been estimated, the development team asks how long would take for testing the solution. The answer should be said almost right away. Then, the software requirements shall be read and understood by the testing team, too. Without the testing participation, no serious estimation can be considered.
2nd Rule: Estimation shall be based on expert judgment
Before estimating, the testing team classifies the requirements in the following categories:
" Critical: The development team has little knowledge in how to implement it;
" High: The development team has good knowledge in how to implement it but it is not an easy task;
" Normal: The development team has good knowledge in how to implement.
The experts in each requirement should say how long it would take for testing them. The categories would help the experts in estimating the effort for testing the requirements.
3rd Rule: Estimation shall be based on previous projects
All estimation should be based on previous projects. If a new project has similar requirements from a previous one, the estimation is based on that project.
4th Rule: Estimation shall be based on metrics
My organization has created an OPD, Organization Process Database, where the project metrics are recorded. We have recorded metrics from three years ago obtained from dozens of projects.
The number of requirements is the basic information for estimating a testing project. From it, my organization has metrics that guide us to estimate a testing project. The table below shows the metrics used to estimate a testing project. The team size is 01 testing engineer.
Metric Value
1.Number of testcases created for each requirement 4,53
2.Number of testcases developed by Working day 14,47
3.Number of testcases executed by Working day 10,20
4.Number of ARs for testcase 0,77
5.Number of ARs verified by Working day 24,64
For instance, if we have a project with 70 functional requirements and a testing team size of 2 engineers, we reach the following estimates:
Metric Value
Number of testcases - based on metric 1 317,10
Preparation phase - based on metric 2 11 working days
Execution phase - based on metric 3 16 working days
Number of ARs - based on metric 4 244 ARs
Regression phase - based on metric 5 6 working days
The testing duration is estimated in 22 (16+6) working days. Plus, 11 working days for preparing it.
5th Rule: Estimation shall never forget the past
I have not sent away the past. The testing team continues using the old process and the spreadsheet. After the estimation is done following the new rules, the testing team estimates again using the old process in order to compare both results.
Normally, the results from the new estimate process are cheaper and faster than the old one in about 20 to 25%. If the testing team gets a different percentage, the testing team returns to the process in order to understand if something was missed.
6th Rule: Estimation shall be recorded
All decisions should be recorded. It is very important because if requirements change for any reason, the records would help the testing team to estimate again. The testing team would not need to return for all steps and take the same decisions again. Sometimes, it is an opportunity to adjust the estimation made earlier.
7th Rule: Estimation shall be supported by tools
A new spreadsheet has been created containing metrics that help to reach the estimation quickly. The spreadsheet calculates automatically the costs and duration for each testing phase.
There is also a letter template that contains some sections such as: cost table, risks, and free notes to be filled out. This letter is sent to the customer. It also shows the different options for testing that can help the customer decides which kind of test he needs.
8th Rule: Estimation shall always be verified
Finally, All estimation should be verified. I've created another spreadsheet for recording the estimations. The estimation is compared to the previous ones recorded in a spreadsheet to see if they have similar trend. If the estimation has any deviation from the recorded ones, then a re-estimation should be made.
Tuesday, July 28, 2009
Test Matices Sample
The same holds for a true "quality metric". The most familiar one is defects per thousand lines of (uncommented) code. But this metric assumes that:
1) you count the lines of code
2) the complexity of the code isn't an issue
3) the programmers aren't playing games (like using continuance characters
so that what could have been written in one line isn't done in five lines)
4) all defects are uncovered in the code in a single pass
5) each defect discovered is all others
6) defects are uncovered in a linear manner between revisions or builds
The fact is that first you need to know what your goal is. Then you need to
discover or create a metric that will help you achieve that goal. Then you
need to implement it and be prepared to adjust it.
You can't use measurements (metrics) to find faults, at least not in software, so that's not a reasonable goal. You can use metrics to help determine if most of the defects have been discovered already. You can use them to tell you how much longer it will take to uncover a reasonable amount of defects. For either of these metrics you will need to know how previous projects of similar size and complexity (using similar languages, etc.) were done in order to get a reasonable comparison.
Posted by Walter Gorlitz
| Test Case: File Open # | Test | Test Cases/ Samples | Pass/ | No. of | Bug# | Comments |
| N/A | Setup for [Product Name] | setup | - | - | - |
|
| 1.1 | Test that file types supported by the program can be opened | 1.1 | P/F | # | # |
|
| 1.2 | Verify all the different ways to open file (mouse, keyboard and accelerated keys) | 1.2 | P/F | # | # | |
| 1.3 | Verify files can be open from the local drives as well as network | 1.3 | P/F | # | # |
NOTE: This is only a sample of a Test Matrices based on editor application. In real environments your Case's contents and layout most likely will be different, depends on your project and company requirements.
Friday, July 24, 2009
The Interview Guide for Testers
This paper aims at uncovering the essential elements that the interviewers and interviewees need to be aware of during the Software Testing interviews. In my earlier paper, I had discussed about the essential skills that a tester needs to possess and they are as follows: Understanding, Listening, Observation, Test Planning, Test Designing, Test Execution, Defect Reporting and Analysis and Test Automation.
From the interviewer perspective, this paper discusses about how to evaluate software testers in an interview. From the interviewee perspective, this paper discusses about the necessary assertive skills that a tester needs to possess for getting through the interview.
Evaluation of Understanding and Listening Skills
The first and foremost activity of Software Testing is to understand the system requirements of the software to be tested.
The key references for these system requirements in most of the projects are formal software requirements specification documents or software functional specification documents or the use case documents. In order to evaluate the tester’s skills in understanding these formal documents, a sample of this formal requirement specification document may be provided to the interviewee. The interviewer may request the interviewee to read/ understand the requirements and explain the same. The interviewee shall read/ understand the requirements and explain the same to the interviewer without any discrepancies and ambiguity.
If provided with any ambiguous requirements, the interviewee shall identify them and seek clarification from the interviewer. If provided with any missing requirements, the interviewee shall provide justification as to why he/she feels that there are few missing requirements and get them defined by the interviewer. If the interviewee finds any difficulty in understanding the requirements, he/ she may get them clarified from the interviewer. The interviewer shall welcome the assertive communication skills of the interviewee.
Software Testing cannot be performed based on “assumed requirements” and all the requirements shall be explicitly defined (except for implicit requirements, which cannot be defined). The interviewee may emphasize this fundamental software testing standpoint during the interview.
Requirement Example
Let us take the case of the classical triangle software to be tested. The triangle software requires 3 positive integer inputs, which are the lengths of the 3 sides of the triangle (say A, B and C).
The software evaluates the following the logical expression: A+B>C && B+C>A && C+A>B. If this logical expression evaluates to True, the software displays the status as green in the system console indicating that the inputs are valid lengths of the sides of the triangle and vice versa.
The interviewer may provide the interviewee with a documenttesting standpoint during the interview. If the interviewee does not understand any of the sections in the requirement document or if the requirements are ambiguous, he/she shall get them clarified by the interviewer. The interviewer shall welcome those clarifications and shall not discourage the interviewee with respect to the clarifications sought. When the interviewee was asked to explain the software functionality, he/she shall explain it without any discrepancies. explaining the above mentioned requirement with few modifications to suit his/her need. To evaluate the understanding/ listening skills of the interviewee, the logical expression may not be mentioned in the document. When the interviewee was asked to read and understand the requirement, he/she shall raise this as an issue of missing requirement to the interviewer. The interviewee shall not assume anything about how the software validates the inputs and explain a new set of expressions such as C^2=A^2+B^2 (which applies only for right angle triangle) to the interviewer. “Defining Requirements” is always not a goal of Software Testing. The interviewee may emphasize this fundamental software
Test Planning and Designing Skills
Test Planning is an activity that focuses on establishing the path way for all the Software Testing activities.
The Software Testers shall keep abreast of his/her knowledge about the latest industrial trends of executing the software testing projects. The interviewer may request the interviewee to explain the industrial hot topics/ news and have a small talk about the same. The interviewer may not concentrate on asking straight forward questions such as “What is Regression Testing?”, “What is Defect Severity?”, “What is Functional Testing?”, “What is UAT?”, etc. These questions may be good for a university question paper but not during the recruitment interviews and that too exclusively. The interviews shall focus on the application side of it. The small talk/ discussion shall focus more on the “Why?”, “Where?” and “How?” of these terminologies along with “What?”. The interviewee shall be able to correlate the testing methodologies and techniques and devise the testing approach for the software under test and explain the same to the interviewer. For the triangle sample, the interviewee shall be able to devise the testing strategy and explain the same to the interviewer. The interviewer may request the interviewee to explain about the testing metrics that he/she need to capture in order to report the status and progress of the testing activities to the project management team.
The interviewer shall request the interviewee to explain about the test plan/ approach/ designing techniques for his/her current project. This is supposed to be illegal as these results in the dissemination of confidential information of the interviewee’s current organization to the public.
To evaluate the test designing skills the interviewer may provide a sample requirement like the triangle sample and request the interviewee to design few tests for the testing the requirement and explain the techniques used for designing the tests. For the triangle sample, the interviewee shall use equivalence class partitioning, decision table technique and the multiple condition coverage technique for designing the tests. The interviewee shall be capable of explaining all these techniques and the tests to the interviewer. The interviewer may request the interviewee to write down few test cases for the sample requirements that he/she provided. The interviewee shall write the test cases clearly – the test steps shall have enough details up to the level of key presses and mouse clicks and the expected results shall be non-ambiguous.
Test Execution and Defect Reporting Skills
To evaluate the test execution and defect reporting skills, the interviewer may request the interviewee to explain what he/she think to be the ideal process to be followed for test execution and defect reporting. As mentioned previously, the interviewer shall not request the interviewee to explain what he/she is currently following in his/her current project.
The interviewer may initiate a small talk/ debate regarding the defect life cycle, defect attributes, defect management tools and the test reporting tools. The interviewer may request the interviewee to write down a sample defect providing the defect details. The interviewee shall write the defect report clearly with detailed steps to reproduce, expected and actual results, non-ambiguously.
Test Automation SkillsTest Automation is a wonderful phenomenon by which the testing
To evaluate the automation tool knowledge, the interviewer may initiate a small talk or discussion based on the respective automation tool. As mentioned previously, the questions may not focus on the automation activities in the interviewees’ current project.
cost is drastically reduced. On the other side, if there is no proper planning for Automation Script Creation & Maintenance, there is a risk that the Automation Suite may get outdated and may not be usable. This renders ROI=0. The testers shall be aware of this business risk and identify the automation candidates accordingly. The interviewers may provide a set of test cases that includes automation candidates (simple & complex) and non-automation candidates, to the interviewees for identifying the automation candidates. The interviewees may ask for clarifications, if any, and they shall be able to identify the automation candidates successfully. The interviewer may request the interviewee to explain the various Automation metrics that he/she need to capture in order to report the Automation ROI and other figures of interest to the project management team.
Conclusion
The interviewer shall use several sample requirements and concentrate on evaluating the interviewees’ application of the software testing principles and techniques.
This paper is purely a guideline and it talks about the basic things to focus on. The interviewers are requested to amend this guideline with their own approaches for conducting the interviews.
The interviewees shall reply answers aptly and boldly – bold enough to challenge the interviewer in case of any wrong questions put forth by the interviewer where he/she may expect you to put the question back to him in this case.
Wednesday, July 15, 2009
UML
Introduction
The Unified Modeling Language (UML) is a standard language for specifying, visualizing, constructing, and documenting the artifacts of software systems, as well as for business modeling and other non-software systems. The UML represents a collection of best engineering practices that have proven successful in the modeling of large and complex systems.1 The UML is a very important part of developing object oriented software
Goals of UML
The primary goals in the design of the UML were:- Provide users with a ready-to-use, expressive visual modeling language so they can develop and exchange meaningful models.
- Provide extensibility and specialization mechanisms to extend the core concepts.
Be independent of particular programming languages and development processes.
Provide a formal basis for understanding the modeling language.
Encourage the growth of the OO tools market.
Support higher-level development concepts such as collaborations, frameworks, patterns and components.
Integrate best practices.
Why Use UML?
As the strategic value of software increases for many companies, the industry looks for techniques to automate the production of software and to improve quality and reduce cost and time-to-market. These techniques include component technology, visual programming, patterns and frameworks. Businesses also seek techniques to manage the complexity of systems as they increase in scope and scale.
In particular, they recognize the need to solve recurring architectural problems, such as physical distribution, concurrency, replication, security, load balancing and fault tolerance. Additionally, the development for the World Wide Web, while making some things simpler, has exacerbated these architectural problems. The Unified Modeling Language (UML) was designed to respond to these needs.
History of UML
Identifiable object-oriented modeling languages began to appear between mid-1970 and the late 1980s as various methodologists experimented with different approaches to object-oriented analysis and design.
The number of identified modeling languages increased from less than 10 to more than 50 during the period between 1989-1994. Many users of OO methods had trouble finding complete satisfaction in any one modeling language, fueling the "method wars." By the mid-1990s, new iterations of these methods began to appear and these methods began to incorporate each other’s techniques, and a few clearly prominent methods emerged.
The development of UML began in late 1994 when Grady Booch and Jim Rumbaugh of Rational Software Corporation began their work on unifying the Booch and OMT (Object Modeling Technique) methods. In the Fall of 1995, Ivar Jacobson and his Objectory company joined Rational and this unification effort, merging in the OOSE (Object-Oriented Software Engineering) method.
As the primary authors of the Booch, OMT, and OOSE methods, Grady Booch, Jim Rumbaugh, and Ivar Jacobson were motivated to create a unified modeling language for three reasons. First, these methods were already evolving toward each other independently. It made sense to continue that evolution together rather than apart, eliminating the potential for any unnecessary and gratuitous differences that would further confuse users. Second, by unifying the semantics and notation, they could bring some stability to the object-oriented marketplace, allowing projects to settle on one mature modeling language and letting tool builders focus on delivering more useful features. Third, they expected that their collaboration would yield improvements in all three earlier methods, helping them to capture lessons learned and to address problems that none of their methods previously handled well.
The efforts of Booch, Rumbaugh, and Jacobson resulted in the release of the UML 0.9 and 0.91 documents in June and October of 1996. During 1996, the UML authors invited and received feedback from the general community. They incorporated this feedback, but it was clear that additional focused attention was still required.
While Rational was bringing UML together, efforts were being made on achieving the broader goal of an industry standard modeling language. In early 1995, Ivar Jacobson (then Chief Technology Officer of Objectory) and Richard Soley (then Chief Technology Officer of OMG) decided to push harder to achieve standardization in the methods marketplace. In June 1995, an OMG-hosted meeting of all major methodologists (or their representatives) resulted in the first worldwide agreement to seek methodology standards, under the aegis of the OMG process.
During 1996, it became clear that several organizations saw UML as strategic to their business. A Request for Proposal (RFP) issued by the Object Management Group (OMG) provided the catalyst for these organizations to join forces around producing a joint RFP response. Rational established the UML Partners consortium with several organizations willing to dedicate resources to work toward a strong UML 1.0 definition. Those contributing most to the UML 1.0 definition included: Digital Equipment Corp., HP, i-Logix, IntelliCorp, IBM, ICON Computing, MCI Systemhouse, Microsoft, Oracle, Rational Software, TI, and Unisys. This collaboration produced UML 1.0, a modeling language that was well defined, expressive, powerful, and generally applicable. This was submitted to the OMG in January 1997 as an initial RFP response.
In January 1997 IBM, ObjecTime, Platinum Technology, Ptech, Taskon, Reich Technologies and Softeam also submitted separate RFP responses to the OMG. These companies joined the UML partners to contribute their ideas, and together the partners produced the revised UML 1.1 response. The focus of the UML 1.1 release was to improve the clarity of the UML 1.0 semantics and to incorporate contributions from the new partners. It was submitted to the OMG for their consideration and adopted in the fall of 1997.
Writing a use case
Degree of detailAlistair Cockburn, in Writing Effective Use Cases, identified three levels of detail in writing use cases:
- A brief use case consists of a few sentences summarizing the use case. It can be easily inserted in a spreadsheet cell, and allows the other columns in the spreadsheet to record priority, technical complexity, release number, and so on.
- A casual use case consists of a few paragraphs of text, summarizing the use case.
- A fully dressed use case is a formal document based on a detailed template with fields for various sections; and it is the most common understanding of the meaning of a use case. Fully dressed use cases are discussed in detail in the next section on use case templates.
Some software development processes do not require anything more than a simple use case to define requirements. However, some other development processes require detailed use cases to define requirements. The larger and more complex the project, the more likely that it will be necessary to use detailed use cases.
The level of detail in a use case often differs according to the progress of the project. The initial use cases may be brief, but as the development process unfolds the use cases become ever more detailed. This reflects the different requirements of the use case. Initially they need only be brief, because they are used to summarize the business requirement
Rational Unified Process invites developers to write a brief use case description in the use case diagram, with a casual description as comments and a detailed description of the flow of events in a textual analysis. All those can usually be input into the use case tool (e.g., a UML Tool, SysML Tool), or can be written separately in a text editor
Use case Templates
Degree of detailThere is no standard template for documenting detailed use cases. There are a number of competing schemes, and individuals are encouraged to use templates that work for them or the project they are on. Standardization within each project is more important than the detail of a specific template. There is, however, considerable agreement about the core sections; beneath differing terminologies and orderings there is an underlying similarity between most use cases.
- Use case name
A use case name provides a unique identifier for the use case. It should be written in verb-noun format (e.g., Borrow Books, Withdraw Cash), should describe an achievable goal (e.g., Register User is better than Registering User) and should be sufficient for the end user to understand what the use case is about.
Goal-driven use case analysis will name use cases according to the actor's goals, thus ensuring use cases are strongly user centric. Two to three words is the optimum. If more than four words are proposed for a name, there is usually a shorter and more specific name that could be used. - Version
Often a version section is needed to inform the reader of the stage a use case has reached. The initial use case developed for business analysis and scoping may well be very different from the evolved version of that use case when the software is being developed. Older versions of the use case may still be current documents, because they may be valuable to different user groups. - Goal
Without a goal a use case is useless. There is no need for a use case when there is no need for any actor to achieve a goal. A goal briefly describes what the user intends to achieve with this use case. - Summary
A summary section is used to capture the essence of a use case before the main body is complete. It provides a quick overview, which is intended to save the reader from having to read the full contents of a use case to understand what the use case is about. Ideally, a summary is just a few sentences or a paragraph in length and includes the goal and principal actor. - Actors
An actor is someone or something outside the system that either acts on the system – a primary actor – or is acted on by the system – a secondary actor. An actor may be a person, a device, another system or sub-system, or time. Actors represent the different roles that something outside has in its relationship with the system whose functional requirements are being specified. An individual in the real world can be represented by several actors if they have several different roles and goals in regards to a system. - Preconditions
A preconditions section defines all the conditions that must be true (i.e., describes the state of the system) for the trigger (see below) to meaningfully cause the initiation of the use case. That is, if the system is not in the state described in the preconditions, the behavior of the use case is indeterminate.
Note that the preconditions are not the same thing as the "trigger" (see below): the mere fact that the preconditions are met does NOT initiate the use case.
However, it is theoretically possible both that a use case should be initiated whenever condition X is met and that condition X is the only aspect of the system that defines whether the use case can meaningfully start. If this is really true, then condition X is both the precondition and the trigger, and would appear in both sections. But this is rare, and the analyst should check carefully that they have not overlooked some preconditions which are part of the trigger. If the analyst has erred, the module based on this use case will be triggered when the system is in a state the developer has not planned for, and the module may fail or behave unpredictably.
- Triggers
A 'triggers' section describes the event that causes the use case to be initiated. This event can be external, temporal or internal. If the trigger is not a simple true "event" (e.g., the customer presses a button), but instead "when a set of conditions are met", there will need to be a triggering process that continually (or periodically) runs to test whether the "trigger conditions" are met: the "triggering event" is a signal from the trigger process that the conditions are now met.
There is varying practice over how to describe what to do when the trigger occurs but the preconditions are not met. - One way is to handle the "error" within the use case (as an exception). Strictly, this is illogical, because the "preconditions" are now not true preconditions at all (because the behaviour of the use case is determined even when the preconditions are not met).
- Another way is to put all the preconditions in the trigger (so that the use case does not run if the preconditions are not met) and create a different use case to handle the problem. Note that if this is the local standard, then the use case template theoretically does not need a preconditions section!
- Basic course of events
At a minimum, each use case should convey a primary scenario, or typical course of events, also called "basic flow" or "happy flow". The main basic course of events is often conveyed as a set of usually numbered steps. For example: - The system prompts the user to log on.
- The user enters his name and password.
- The system verifies the logon information..
- The system logs user on to system.
- Alternative paths
Use cases may contain secondary paths or alternative scenarios, which are variations on the main theme. Each tested rule may lead to an alternate path and when there are many rules the permutation of paths increases rapidly, which can lead to very complex documents. Sometimes it is better to use conditional logic or activity diagrams to describe use case with many rules and conditions.
Exceptions, or what happens when things go wrong at the system level, may also be described, not using the alternative paths section but in a section of their own. Alternative paths make use of the numbering of the basic course of events to show at which point they differ from the basic scenario, and, if appropriate, where they rejoin. The intention is to avoid repeating information unnecessarily.
An example of an alternative path would be: - The system recognizes cookie on user's machine.
- Go to step 4 (Main path)
- The system does not recognize user's logon information
- Go to step 1 (Main path)
- Postconditions
The post-conditions section describes what the change in state of the system will be after the use case completes. Post-conditions are guaranteed to be true when the use case ends. - Business rules
Business rules are written (or unwritten) rules or policies that determine how an organization conducts its business with regard to a use case. Business rules are a special kind of requirement. Business rules may be specific to a use case or apply across all the use cases, or across the entire business. Use cases should clearly reference BRs that are applicable and where they are implemented.
Business Rules should be encoded in-line with the Use Case logic and execution may lead to different post conditions. E.g. Rule2. that a cash withdraw will lead to an update of the account and a transaction log leads to a post condition on successful withdrawal - but only if Rule1 which says there must be sufficient funds tests as true. - Notes
Experience has shown that however well-designed a use case template is, the analyst will have some important information that does not fit under a specific heading. Therefore all good templates include a section (eg "Notes to Developers") that allows less-structured information to be recorded. - Author and date
This section should list when a version of the use case was created and who documented it. It should also list and date any versions of the use case from an earlier stage in the development which are still current documents. The author is traditionally listed at the bottom, because it is not considered to be essential information; use cases are intended to be collaborative endeavors and they should be jointly owned.
An example of an exception path would be:
According to Anthony J H Simons and Ian Graham (who openly admits he got it wrong - using 2000 use cases at Swiss Bank), alternate paths were not originally part of use cases. Instead, each use case represented a single user's interaction with the system. In other words, each use case represented one possible path through the system. Multiple use cases would be needed before designs based on them could be made. In this sense, use cases are for exploration, not documentation.
An Activity diagram can give an overview of the basic path and alternatives path.
Benefits of use cases
Use cases are a mature model to capture user (person or system) proffered interaction requirements and begin to establish some of the functional requirements before construction of a new system begins. Proponents prefer them to large, monolithic documents which they believe cannot be simultaneously complete and meaningful, and regard them as an excellent technique for capturing the functional requirements of a system. Proponents cite these advantages:
- Well written Use cases have proven to be easily understandable by business users, and thus to act as a bridge between them and software developers.
- Uniquely identified use cases can be traced back to business requirements or stakeholder needs.
- Use case partitioning can be used to organise and structure the requirements model, permitting common behaviour to be factored out.
- >
- Use cases can serve as a basis for the estimating, scheduling, and validating efforts.
- Use cases are reusable within a project - Citation needed A use case (like anything) can evolve at each iteration, from a method of capturing requirements, to development guidelines for programmers, to a test case and finally into user documentation.
- Use cases make it easy to take a staged delivery approach to projects; they can be relatively easily added and removed from a software project as priorities change.
- Use cases that represent interactions between a user and a system (others will represent interactions between systems) make it possible for user interface designers to become involved in the development process before, or in parallel with, software developers (although use cases are also said to discourage inappropriate premature design).
- Use cases and use case diagrams recorded using UML can be maintained with widely available CASE tools, and thus be fully integrated with other analysis and design deliverables created using a CASE tool. The result is a complete requirements, design, and implementation repository.Citation needed
- Test cases (system, user acceptance and functional) can be directly derived from use cases
- Use cases are critical for the effective execution of Performance Engineering.
Limitations of use cases
Use cases have limitations:
Use case flows are not well suited to easily capturing non-interaction based requirements of a system (such as algorithm or mathematical requirements) or non-functional requirements (such as platform, performance, timing, or safety-critical aspects). These are better specified declaratively elsewhere.
- Use cases templates do not automatically ensure clarity. Clarity depends on the skill of the writer(s).
There is a learning curve involved in interpreting use cases correctly, for both end users and developers. As there are no fully standard definitions of use cases, each group must gradually evolve its own interpretation. Some of the relations, such as extends, are ambiguous in interpretation and can be difficult for stakeholders to understand.
Proponents of Extreme Programming often consider use cases too needlessly document-centric, preferring to use the simpler approach of a user story.
Use case developers often find it difficult to determine the level of user interface (UI) dependency to incorporate in a use case. While use case theory suggests that UI not be reflected in use cases, it can be awkward to abstract out this aspect of design, as it makes the use cases difficult to visualize.
Use cases can be over-emphasized. In Object Oriented Software Construction (2nd edition), Bertrand Meyer discusses issues such as driving the system design too literally from use cases and using use cases to the exclusion of other potentially valuable requirements analysis techniques.
Use cases have received some interest as a starting point for test design. Some use case literature, however, states that use case pre and postconditions should apply to all scenarios of a use case (i.e., to all possible paths through a use case) which is limiting from a test design standpoint. If the postconditions of a use case are so general as to be valid for all possible use case scenarios, they are likely not to be useful as a basis for specifying expected behavior in test design. For example, the outputs and final state of a failed attempt to withdraw cash from an ATM are not the same as a successful withdrawal: if the postconditions reflect this, they too will differ; if the postconditions don’t reflect this, then they can’t be used to specify the expected behavior of tests. An alternate perspective on use case pre & postconditions more suitable for test design based on model-based specification
Use Case Based Testing
Overview:
IBM Research has developed Use Case Based Testing (UCBT), which is a technique for generating test cases and recommended configurations for system level testing. In our approach, testers build a test model based on the standard UML notions of use cases, actors, and the relationships between these elements. The use cases are enhanced with additional information, including the inputs from actors, the outputs to the actors, and how the use case affects the state of the system.
Newly developed algorithms use this model to generate a test suite which provides a specified level of coverage of each use case. We also generate workload configurations that combine the test cases according to requirements specified in the model. The generation algorithm performs minimization to reduce the number of test cases required to cover the system to the specified level. The workload configurations are based on desired percentages associated with each actor that the tester provides. These features form a powerful basis for model-based test case generation.
What testing phase does UCBT address?
UCBT addresses phases where the tester is interested in exploring behavior that flows through multiple use cases, which are typically the late function, system, and solution phases of the test life cycle. In late function level test, the tester has tested the use cases individually, and is now interested in looking at combinations of the use cases. System test addresses the situation when all required functionality for the system is present, and the tester seeks to ensure the proper functioning of the system as a whole. An important component of system test is also ensuring that the system can handle customer-like scenarios and workloads. Solution test addresses the situation in which several complete systems are combined to provide complex functionality through some process that involves the systems. These processes can be captured, modeled, and tested using UCBT.
What does the tester do in UCBT?
To do UCBT, the tester needs to identify four things:
- the use cases of interest,
- the actors involved in using the system,
- the input, output, and system effects for the use cases,
- the flows of interest between the use cases.
UA use case is a semantically meaningful function that provides some value from the user's point of view. For example, saving a file in a word processing system would be represented by the Save File
UCBT produces two types of test cases. The first type are abstract test cases, which are not executable. These are suitable for incorporation in a test plan and are provided in structured English. They show the ordering of use cases to be performed and the inputs and expected results that each use case should have during the test. The second type of test cases are in a format known as ATS, which can be used to create executable test cases using another tool known as TCBeans. TCBeans was developed by IBM Research in Haifa. The test suite produced by UCBT is minimized in size by performing a two pass optimization based upon the input parameter interactions that the tester specifies, as well as the flows of interest. This ensures that the number of test cases is reasonable, so execution and evaluation can be done with a feasible amount of effort.
