• Scenario 1
• Scenario 2
• Scenario 3    
  
• Scenario 4
• Scenario 5
• Scenario 6

Scenario 5: Finding edge faults

Your Situation

Your testing regime is now well-regarded, thanks to your use of SVaT, but customers have reported faults using certain data inputs. However, the same functionality works fine with the data you have tested the application against. How do you make sure you can test with the reported data, without having to substantially rework all the relevant test scripts?

Solution

With the SVaT Professional Edition you can extend your functional test workflows with advanced data-driven capabilities.

The steps you take, using SVaT, are outlined here:
Expand all, Collapse All

1. Preparation: Install SVaT and the AUT (more...less...)
   Install SVaT (the System Verification and Test suite) and the AUT (the application under test).
   
   Time: at least 10 minutes
   
   Install SVaT (the System Verification and Test suite) and the AUT (the application under test), i.e.
   1. Install SVaT
      Run the provided SVaT installer and make a few decisions about where to install SVaT.
      
      You need at least the Professional Edition of SVaT for this.
      
      If you have not already installed Java you must install it before you can install SVaT. Java is included with SVaT, or you can use an existing installation if appropriate.
      
      Time: 5 minutes
      
      
   2. Install the Application Under Test (AUT)
      Next step is to install the application under test following whatever instructions your R&D group gave you.
      
      Time: at least 5 minutes
      
      
2. Load Design: Load your design into the Verification Designer
   
   Next load your design into SVaT's Verification Designer.
   
   Time: 2 minutes
   
   
3. Create Data: Create a dataset and link to commands (more...less...)
   
   Create a Dataset (basically a table with your input and output values) in SVaT's Verification Designer. Enter your data into the table. Link the relevant command parameters in your design to the table.
   
   Time: 10-15 minutes
   
   
   Create a Dataset (basically a table with your input and output values) in SVaT's Verification Designer. Enter your data into the table. Link the relevant command parameters in your design to the table.
   
   By linking a parameter of a command to a Dataset you automatically tell the Verification Designer that the script is data-driven. The only difference you will see is the values you'll see in each command parameter. Everything else remains identical to non data-driven work-flows.
   
   Time: 10-15 minutes
   
   
4. Generation and Execution: Generate test scripts, run them against the AUT (more...less...)
   
   Once the dataset is created and the relevant parameter values have been linked to it, you can generate a set of executable test scripts directly from your design. Such test scripts can then be executed directly by the Test Performer or another test automation tool.
   
   Time: from 6 minutes
   
   
   Once the dataset is created and the relevant parameter values have been linked to it, you can generate a set of executable test scripts directly from your design. Such test scripts can then be executed directly by the Test Performer or another test automation tool.

   1. Generate test scripts
      When the dataset is created and the relevant parameter values have been linked to it, you can generate a set of test scripts that can be executed by the Test Performer or another test automation tool. The test script generation only requires you to specify a few parameters of the generated test scripts, e.g. for which test automation tool, full or reduced set, etc.
      
      Each data parameter now sets of a complete new set of test vectors, and the number of tests expands greatly compared with the non-data driven case.
      
      Time: 1-10 minutes (depending on number of test scripts generated)
      
      
   2. Run the generated test scripts
      Once the test scripts have been generated, they can be executed using the automated test tool they were generated for, in this case SVaT's Test Performer.
      
      These tests may now take many times longer than before because you have so many more tests to run.
      
      Time: from 5 minutes
      
      
5. Fault detection: Examination of Results (more...less...)
   
   Once the test scripts have been executed, you can load the results back into SVaT and examine them in direct relation to the original Design.
   
   Time: 5 minutes to 1 hour (depending on number of errors)
   
   
   Once the test scripts have been executed, you can load the results back into SVaT and examine them in direct relation to the original Design.
   
   You will see the results directly represented represented in relation to your original test design. The original design is overlaid with the test results, and errors highlighted there This can save a great deal of time because it draws attention directly to the failing steps and can eliminate the need to investigate duplicate errors.
   
   Although the number of test vectors was hugely increased by the addition of data, the test representation remains as compact as before, saving a great deal of time in identifying areas at fault.
   
   Time: 5 minutes to 1 hour (depending on number of errors)
   
   
6. Outcomes
   
   The benefits of this approach includes:

   • Minimal difference between a "normal" and a data-driven test script.
   • The data and the work-flows of the test scripts are kept separate.
   • Even though there is a massive increase in the number of tests, the data displayed to you is kept very simple and straight forward.
   • You have an automated trace of the tests you have made and executed, and can use this as documentation in your test reports and as an audit trail later, if needed!
   • If you should be unavailable at a critical time, then it will be much easier for somebody else to build upon your test scripts because the syntax used in SVaT is so intuitive.