Fault Seeding Experiment to Evaluate Effectiveness of System Tests
Heamets, Juhan (2020)
2020
All rights reserved. This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:amk-202003183629
https://urn.fi/URN:NBN:fi:amk-202003183629
Tiivistelmä
In this thesis the effectiveness of ABB’s system tests was evaluated using fault seeding experiment. Purpose for this thesis was to show how effective ABB’s system tests are for finding type of defects for that they are designed for. Work was carried out at the drive software testing team where systems tests are used to evaluate drives behavior. One of the goals of ABB’s software testing is to minimize the possibilities of defects reaching the customer, so that end user could receive bug free product.
Thesis introduced testing levels and approaches used in the drive software testing, as well as how effectiveness of testing could be measured in the different testing levels. Work also introduced different types of faults used in the mutation testing.
In the thesis, faults were injected to the source code manually by altering the drive source code using different fault seeding methods. System tests were run with modified drive software and results analyzed to find out does the test cases find the defects in the drive source code as expected.
Results showed that in most cases ABB’s system tests were capable of detecting defects as expected. Work brought up locations and types of defects not detected by the system testing and reasons for them. Thesis also gave an overview of systematic mutation testing process.
Thesis introduced testing levels and approaches used in the drive software testing, as well as how effectiveness of testing could be measured in the different testing levels. Work also introduced different types of faults used in the mutation testing.
In the thesis, faults were injected to the source code manually by altering the drive source code using different fault seeding methods. System tests were run with modified drive software and results analyzed to find out does the test cases find the defects in the drive source code as expected.
Results showed that in most cases ABB’s system tests were capable of detecting defects as expected. Work brought up locations and types of defects not detected by the system testing and reasons for them. Thesis also gave an overview of systematic mutation testing process.