• Scenario 1
• Scenario 2
• Scenario 3    
  
• Scenario 4
• Scenario 5
• Scenario 6

Scenario 4: Comprehensive test suite, many faults to fix

Your Situation

You have developed your test suite ahead of time, and the first test candidate falls a little short of perfection (to say the least). How do you avoid being overwhelmed by error reports next time you run your test suite?

Solution

SVaT can hook up to your Issue Tracker / Bug Reporting system to switch off certain test commands until they are reported as fixed - then switch them back in automatically.

The steps you take, using SVaT, are outlined here:
Expand all, Collapse All

1. Preparation: Install SVaT and the AUT (more...less...)
   Install SVaT (the System Verification and Test suite) and the AUT (the application under test).
   
   Time: at least 15 minutes
   
   Install SVaT (the System Verification and Test suite) and the AUT (the application under test), i.e.
   1. Install SVaT
      Run the provided SVaT installer and make a few decisions about where to install SVaT.
      
      You need at least the Personal Edition of SVaT for this.
      
      If you have not already installed Java you must install it before you can install SVaT. Java is included with SVaT, or you can use an existing installation if appropriate.
      
      Time: 5 minutes
      
      
   2. Install SVaT Plug-in
      
      SVaT needs a dedicated plug-in to link to your Issue Tracker / Bug reporting system, which you can get separately from Advanced QA Solutions Ltd. Install it by following its installation instructions.
      
      Time: 5 minutes
      
      
   3. Install the Application Under Test (AUT)
      Next step is to install the application under test following whatever instructions your R&D group gave you.
      
      Time: at least 5 minutes
      
      
2. Load Design: Load the current SVaT design into SVaT
   
   Load your current design into SVaT. You may also wish to load the last set of results to make it easier to find the commands that have failed (since these results are overlaid on the Design).
   
   
3. Faults Linking: Link failed test commands to faults (more...less...)
   
   Load the faults from your Issue Tracker / Bug Reporting system into SVaT's Verification Designer through the plug-in, and assign each relevant fault to the relevant command(s).
   
   Time: from 11 minutes
   
   
   Load the faults from your Issue Tracker / Bug Reporting system into SVaT's Verification Designer through the plug-in, and assign each relevant fault to the relevant command(s):

   1. Load the faults into SVaT
      Load the faults into SVaT's Verification Designer using the plug-in.
      
      Time: 1 minute
      
      
   2. Assign a faults to test commands
      Find the commands that should be ignored until the fault has been fixed (and is marked as such in the Issue Tracker / Bug Reporting system). Link each such command to the relevant fault.
      
      Time: from 10 minutes
      
      
4. Generation and Execution: Generate test scripts, run them against the AUT (more...less...)
   
   Then generate the executable test scripts from your design again, and execute them with the Test Performer.
   
   Time: from 6 minutes
   
   
   Then generate the executable test scripts from your design again, and execute them with the Test Performer:

   1. Generate test scripts
      When your test scripts have been updated, generate a new set of test scripts for the appropriate test execution tool. The test script generation only requires you to specify a few parameters about the properties of the generated test scripts, e.g. which test automation tool, full or reduced set, etc.
      
      Time: 1-10 minutes (depending on number of test scripts generated)
      
      
   2. Run the generated test scripts
      Once the test scripts have been generated, they can be executed using the automated test tool they were generated for, in this case SVaT's Test Performer.
      
      Time: from 5 minutes
      
      
5. Fault detection: Examination of Results (more...less...)
   
   Once the test scripts have been executed, you can load the results back into SVaT and examine them in direct relation to the original Design. The original design is overlaid with the test results, and errors highlighted there. This can save a great deal of time because it draws attention directly to the failing steps and can eliminate the need to investigate duplicate errors.
   
   Time: 5 minutes to 1 hour (depending on number of errors)
   
   
   Once the test scripts have been executed, you can load the results back into SVaT and examine them in direct relation to the original Design. The original design is overlaid with the test results, and errors highlighted there. This can save a great deal of time because it draws attention directly to the failing steps and can eliminate the need to investigate duplicate errors.
   
   However, you will also remark that the linked (and not fixed) commands do not show any results - sparing you the time to investigate them again. The way SVaT's Verification Designer shows the results can often save a great deal of time because it can eliminate the need to investigate duplicate errors.
   
   Time: 5 minutes to 1 hour (depending on number of errors)
   
   
6. Outcomes
   
   The benefits of this approach includes:

   • You avoid investigating already known faults yet do not have to delete anything from your test scripts.
   • You do not have to keep track of the modifications to the test scripts due to known errors - SVaT does that for you.
   • The commands linked to faults are automatically included in the test scripts when a fault has been marked as fixed or closed and the test scripts are re-generated.