[Feature-dev] Test quality metrics and recovery plan

[Mike Wong]

Hi all,

Just to confirm, there is an unresolved bug in our code base and Mauricio and I will address it.

I've written some software to perform our regression test against our submissions to the FEATURE Subversion server, and the results are good and bad. As we increase coverage in our tests there are more failures. This is good because we know that our tests are pushing the envelope of the utility of FEATURE and our testing process is improving. This is bad because somewhere along the line we destroyed previously existing functionality in FEATURE and didn't know about it (because the test for the functionality didn't exist).

Now, of course we can always roll back to r418 and restore all the old functionality and try to re-implement our changes. However, my plan is to move forward, find the error that caused the loss of functionality, patch that error, and apply the patch to the latest revision.

Good news is that r454 passes all tests, so we know that r454 has equivalent functionality to r418 (for all functions that we test).

Bottom line: As a team we need to make a commitment to software testing, and we need to share the QA duties. As we continue to eliminate unused code and understand the interactions between classes, we need to finalize the API for each class and then write tests (i.e. unit tests) to test each method.

Best regards,

- m.

 __________________________________
Mike Wong, Staff Research Associate
Center for Computing for Life Sciences (CCLS)
San Francisco State University
1600 Holloway Avenue, Hensill Hall 301, SF, CA 94132
(415) 405-2119
mikewong at sfsu.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://simtk.org/pipermail/feature-dev/attachments/20090624/67b04d86/attachment.html