Unit and Regression Tests of Scientific Software: A Study on SWMM

Description

Testing helps assure software quality by executing programs and uncovering bugs. Scientific software developers often find it challenging to carry out systematic and automated testing due to reasons such as inherent model uncertainties and complex floating-point computations. Extending the recent work on analyzing the unit tests written by the developers of the Storm Water Management Model (SWMM), we report in this paper the investigation of both unit and regression tests of SWMM. The results show that the 1,458 SWMM tests have a 54.0% code coverage and an 82.4% User’s Manual coverage. Meanwhile, an examination of eight regression tests from a test set shows a 79.5% code coverage and a near 100% User’s Manual coverage. We also observe a “getter-setter-getter” testing pattern from the SWMM unit tests and suggest a diversified way of designing or adopting regression tests. This dataset is associated with the following publication: Peng, Z., X. Lin, M. Simon, and N. Niu. Unit and Regression Tests of Scientific Software: A Study on SWMM. Journal of Computational Science. Elsevier B.V., Amsterdam, NETHERLANDS, 53: 101347, (2021).

Resources

Name Format Description Link
53 regression%20test%20data.xlsx https://pasteur.epa.gov/uploads/10.23719/1519338/regression%20test%20data.xlsx
53 Mappings_of_SWMM_unit_tests.xlsx https://pasteur.epa.gov/uploads/10.23719/1519338/Mappings_of_SWMM_unit_tests.xlsx
53 test%20study%20on%20swmm%28data%20table%29.xlsx https://pasteur.epa.gov/uploads/10.23719/1519338/test%20study%20on%20swmm%28data%20table%29.xlsx

Tags

  • test-coverage
  • users-manual
  • unit-testing
  • storm-water-management-model-swmm
  • scientific-software
  • regression-testing

Topics

Categories