Software Failure and Reliability Assessment Tool (SFRAT)


Description

The key to the success of all software is its reliability. The Software Failure and Reliability Assessment Tool (SFRAT) is an open source application to estimate and predict the reliability of a software system during test and operation. It allows users to answer the following questions about a software system during test:

  1. Is the software ready to release (has it achieved a specified reliability goal)?
  2. How much more time and test effort will be required to achieve a specified goal?
  3. What will be the consequences to the system’s operational reliability if not enough testing resources are available?
SFRAT runs under the R statistical programming framework and can be used on computers running Windows, Mac OS X, or Linux

Resources

WARNING: Web instance is for demonstration only. Please do not upload sensitive data to the site
Web instance
Example failure data sets
SFRAT Github repository
User's Guide
Contributor's Guide

Automated Report generation script
Script instruction manual

Contributions

  1. Dr. Shinji Inoue from Kansai University, Kansai, Osaka, Japan translated the GUI to Japanese version. Available at:
    SFRAT - Japanese Version
  2. Dr. Hiroyuki Okamura from Hiroshima University, Hiroshima, Japan contributed 11 additional models to the tool. Available at:
    SFRAT_Okamura
  3. Aerospace Corporation contributed three models. Available at:
    SFRAT_Aerospace
  4. Barry Von Tobel from MITRE developed a script to automatically transport failure data collected from JIRA tool to SFRAT compatible data format. Available at:
    JIRA export data to SFRAT data format
NOTE: If you are interested in helping to create a translation of the tool interface and manual in your language, please contact Dr. Fiondella

Software Defect Estimation Tool (SweET)

Description

The Software Defect Estimation Tool (SweET) is an open source application to track error identification and removal efforts during the software development lifecycle. SwEET is a free and open source version of the SoftWare Error Estimation Program (SWEEP) and SweET uses Weibull software reliability growth model utilizing Expectation Conditional Maximization algorithm to ensure stability and performance of the model fitting process. SweET simplifies four models of SWEEP into three modes:

  1. Mode A: Time-based model: Estimates and tracks errors during system test and integration cycles.
  2. Mode B: Phase-based and planning aid model: Predict and track defects for multiple phases and can provide defect information before running any code, whereas the planning aid model generates an error discovery profile based on the phase based historical data to help a software prohect achieve its objectives.
  3. Mode C: Defect injection model: Allows the user to understand the probable defect injection profile and resulting efficiency and effectiveness of the verification process.
SweET runs under the Python 3.x programming framework and can be used on computers running Windows, Mac OS X, or Linux

Resources

Example data sets
SweET Github repository
User's Guide

Publications

Invited Talks

  1. ECM Algorithms & An Open Source Tool, Tottori University, Tottori, Japan, July 4, 2017.
  2. Software Reliability Engineering: Algorithms and Tools, Florida State University, Tallahassee, FL, February 27, 2017.
  3. Software Reliability Research - Assessments & Tools, Office of the Secretary of Defense, Developmental Test and Evaluation Functional Integrated Product Team Meeting, Washington, DC, Jan 11, 2017.
  4. Software Reliability Research - Assessments & Tools, Office of the Deputy Assistant Secretary of Defense for Developmental Test & Evaluation, Alexandria, VA, Oct 28, 2016.
  5. Software Reliability: Tools and Algorithms, IEEE Boston Reliability Society, Lexington, MA, Sep 14 2016.
  6. Software Reliability Engineering: Algorithms and Tools, Naval Postgraduate School, Monterey, CA, May 27 2016.
  7. Software Reliability: Tools and Models, Workshop on Rigorous Test and Evaluation for Defense, Aerospace, and National Security, Arlington, VA, April 12, 2016.
  8. Algorithms and Tools for Software Reliability Engineering, University of Maryland, Dec 2, 2015.
  9. An Open Source Software Reliability Tool and Model Fitting Algorithm, Wright State University, Oct 7, 2015.
  10. Software Reliability Engineering: Practice and Theory, UMass- Naval Undersea Warfare Center (NUWC) Lecture Series, Nov 20, 2014.

Presentations/Tutorials

  1. V. Nagaraju, T. Wandji, and L. Fiondella, Free and Open Source Tools to Assess Software Reliability and Security, Presented at the National Defense Industrial Association (NDIA) Annual Systems Engineering Conference, Springfield, VA, October, 2017.
  2. V. Nagaraju, T. Wandji, and L. Fiondella, Software Reliability: Modeling and Tools, Tutorial presented at the 85th Military Operations Research Symposium (MORS 2017), West Point, NY, June 2017.
  3. V. Nagaraju, T. Wandji, and L. Fiondella, Software Reliability Modeling, Presented at the Science of Test Workshop, Springfield, VA, April, 2017.
  4. V. Nagaraju, T. Wandji, and L. Fiondella, Software Failure and Reliability Assessment Tool (SFRAT): An Open Source Application for the Practitioner and Research, Presented at the Conference on Applied Statistics in Defense, Washington, DC, October, 2016.
  5. V. Nagaraju, L. Fiondella, T. Wandji, Introduction to Mathematical Software Reliability Models, In Proc. 63nd Annual Reliability and Maintainability Symposium (RAMS 2017), Orlando, FL, Jan 2017.
  6. L. Fiondella, A. Nikora, and T. Wandji, Software Reliability and Security: Challenges and Crosscutting Themes, Fast abstract In Proc. International Symposium on Software Reliability Engineering, Ottawa, Canada, Nov, 2016.
  7. V. Nagaraju, T. Wandji, and L. Fiondella, An Open Source Tool to support Software Reliability Assessment, Presented to WG 17 Logistics, Reliability and Maintainability and WG 24 Test and Evaluation (T&E) at the 84rd Military Operations Research Symposium (MORS 2016), Quantico, VA, June 2016. Nominated for the 85th Barchi Prize.
  8. V. Nagaraju and L. Fiondella, Introduction to Mathematical Software Reliability Models, In Proc. 62nd Annual Reliability and Maintainability Symposium (RAMS 2016), Tucson, AZ, Jan 2016.
  9. V. Nagaraju and L. Fiondella, An Adaptive EM Algorithm for NHPP Software Reliability Models, UMassD Sigma Xi Research Exhibition, Apr 2015.
  10. L. Fiondella and T. Wandji, An Open Source Application Architecture for Software Reliability Models, In Proc. International Applied Reliability Symposium (ARS), Tucson, AZ, Jun 2015.

Acknowledgements

This material is based upon work supported by the National Science Foundation under Grant Number s (1526128 and 1749635), the Naval Air Systems Command (NAVAIR) through the Systems Engineering Research Center, under Research Task 139, and by NAVAIR under contract N00421-16-T-0373.

Work at JPL was supported under NASA Prime Contract NNN12AA01C, under Task Plan No. 81-19648.