[Feature-dev] Re: Test error
Mike Wong
mikewong at sfsu.edu
Thu Jul 2 10:28:37 PDT 2009
Hi Gurgen,
Right. What you saw was an earlier form of the regression test suite where all the tests passed. Then I extended the test coverage and corrected the test data.
Notice now that the trypsin-ser-og-positive and negative have 2 parts and that part 1/2 passes, but part 2/2 fails; part 2 has new test data.
With the extended test coverage, the system began to fail. At this point I tasked Mauricio and myself to find the bug and eliminate it.
The patch has not yet been applied and committed to Subversion; I expect that to happen sometime today, hopefully.
Once the patch is put in place and works, then I'll also commit the retroactive test system.
Best regards,
- m.
__________________________________
Mike Wong, Staff Research Associate
Center for Computing for Life Sciences (CCLS)
San Francisco State University
1600 Holloway Avenue, Hensill Hall 301, SF, CA 94132
(415) 405-2119
mikewong at sfsu.edu
________________________________
From: Gurgen Tumanian <tumanian at gmail.com>
To: Mike Wong <mikewong at sfsu.edu>
Cc: Mauricio Ardila <willy_se_free at hotmail.com>; FEATURE development team <feature-dev at simtk.org>
Sent: Wednesday, July 1, 2009 10:44:29 PM
Subject: Re: [Feature-dev] Re: Test error
Mike,
There is something that I have noticed, and I have to share, to clarify, if that's a result of your fixes or it is just that testing framework is not stable.
I was running the tests as in rev 536. The model was faulty indeed, with zeros all the way, but the tests did fail, in contrary to what we saw when running tests on your PC.
here is the result of make check
regression/copy-tools.................ok
regression/unpack-tools...............ok
regression/copy-feature...............ok
regression/unpack-feature.............ok
regression/configure-feature..........ok
regression/build-feature..............ok
regression/trypsin-ser-og-positive....ok 1/2
# Failed test at regression/trypsin-ser-og-positive.t line 6.
# Looks like you failed 1 test of 2.
regression/trypsin-ser-og-positive....dubious
Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 2
Failed 1/2 tests, 50.00% okay
regression/trypsin-ser-og-negative....ok 1/2
# Failed test at regression/trypsin-ser-og-negative.t line 6.
# Looks like you failed 1 test of 2.
regression/trypsin-ser-og-negative....dubious
Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 2
Failed 1/2 tests, 50.00% okay
regression/trypsin-ser-og-model.......ok 1/2
# Failed test at regression/trypsin-ser-og-model.t line 6.
# Looks like you failed 1 test of 2.
regression/trypsin-ser-og-model.......dubious
Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 2
Failed 1/2 tests, 50.00% okay
regression/build-1bqy-pointfile.......ok
regression/featurize-1bqy.............ok
regression/score-1bqy.................ok
regression/build-1ufo-pointfile.......ok
regression/featurize-1ufo.............ok
regression/score-1ufo.................ok
Failed Test Stat Wstat Total Fail List of Failed
-------------------------------------------------------------------------------
regression/trypsin-ser-og-model.t 1 256 2 1 2
regression/trypsin-ser-og-negative.t 1 256 2 1 2
regression/trypsin-ser-og-positive.t 1 256 2 1 2
Failed 3/15 test scripts. 3/18 subtests failed.
Files=15, Tests=18, 48 wallclock secs (35.84 cusr + 6.72 csys = 42.56 CPU)
Failed 3/15 test programs. 3/18 subtests failed.
make[1]: *** [test] Error 255
make[1]: Leaving directory `/home/tumanian/dev/feature/trunk/tests'
make: *** [test] Error 2
I can send the test.log if needed.
Regards
Gurgen
On Wed, Jul 1, 2009 at 3:56 PM, Mike Wong <mikewong at sfsu.edu> wrote:
Hi Mauricio,
I found one of the problems. It's a combination of Proteins.h and Atoms.h.
The class Proteins is a subclass of Atoms. Atoms has an Objects instance called 'atoms'. Protein also has an Objects instance called 'atoms'. Therefore the instances of atoms are overloaded, and local scope takes precedence.
This is precisely the kind of problem that we can prevent with both a coding convention and using strong type-checking.
Best regards,
- m.
__________________________________
Mike Wong, Staff Research Associate
Center for Computing for Life Sciences (CCLS)
San Francisco State University
1600 Holloway Avenue, Hensill Hall 301, SF, CA 94132
(415) 405-2119
mikewong at sfsu.edu
________________________________
From: Mauricio Ardila <willy_se_free at hotmail.com>
To: mike <mikewong at sfsu.edu>
Sent: Wednesday, July 1, 2009 11:43:40 AM
Subject: RE: Test error
hi Mike
yesterday i found out that the change the schedule of my flight to 5:30 so i will not be able to make the meeting of today.
additionally the only thing i found out was that in neighborhood.h
in lines 24 to 30 a new Point3DList list is created and the it is populated with the coordinates of each atom.
however there was a change in which instead of using
retList->push_back(&protein->GetAtom(i)->coord);
now
Atom *atom;
for (int i = 0; i < numAtoms; i++) {
atom = (*protein)[ i ];
retList->push_back( &(atom->coord) );
the logic is the same, but the [] operator does seems to create a problem.
so, in conclusion that was all that i found that can cause a problem.
i was not able to fully complete pass the test but i think it is a start.
let me know if you need anything
Mauricio
________________________________
Date: Wed, 24 Jun 2009 21:08:53 -0700
From: mikewong at sfsu.edu
Subject: Re: Test error
To: willy_se_free at hotmail.com
Hi Mauricio,
Thursday at 3pm sounds good. See you at CCLS. We may wander over to the library and look at some software testing books.
Thanks,
- m.
__________________________________
Mike Wong, Staff Research Associate
Center for Computing for Life Sciences (CCLS)
San Francisco State University
1600 Holloway Avenue, Hensill Hall 301, SF, CA 94132
(415) 405-2119
mikewong at sfsu.edu
________________________________
From: Mauricio Ardila <willy_se_free at hotmail.com>
To: mike <mikewong at sfsu.edu>
Sent: Wednesday, June 24, 2009 4:26:05 PM
Subject: RE: Test error
do you wanna meet tomorrow like at 3?
Mauricio
________________________________
Date: Tue, 23 Jun 2009 11:21:15 -0700
From: mikewong at sfsu.edu
Subject: Re: Test error
To: willy_se_free at hotmail.com
Hi Mauricio,
Yes, when's a good time for you this week? I'm free Wed-Fri 1-4pm.
Best regards,
- m.
__________________________________
Mike Wong, Staff Research Associate
Center for Computing for Life Sciences (CCLS)
San Francisco State University
1600 Holloway Avenue, Hensill Hall 301, SF, CA 94132
(415) 405-2119
mikewong at sfsu.edu
________________________________
From: Mauricio Ardila <willy_se_free at hotmail.com>
To: mike <mikewong at sfsu.edu>
Sent: Thursday, June 18, 2009 3:10:16 PM
Subject: RE: Test error
Hi mike
do you wanna meet soon to talk about this new task?
Mauricio
________________________________
Date: Thu, 18 Jun 2009 12:51:31 -0700
From: mikewong at sfsu.edu
Subject: Test error
To: feature-dev at simtk.org
CC: tumanyan at sfsu.edu; mardila at sfsu.edu
Hi all,
I've recently discovered that a hole in our test coverage has been glossing over a particularly bad problem; we may be generating faulty models.
Therefore, I'm tasking myself and Mauricio to become Grade-A Test Engineers. Anyone who's interested in this unglamorous, but unbelievably important, process, please contact me. This is especially important for those who are thinking of a career in industry.
Software Quality Assurance texts spend huge amounts of time explaining how to design tests so that they don't fail. There's very little discussion on tests that have poor coverage, or simply faulty tests, and what to do when you find yourself in those situations. This is where practice diverges from theory, and one of the areas where experience matters most. There is significant hands-on learning that will happen here, something that can potentially help you nail an interview and land a job or a performance review and score a promotion.
Developers can continue to develop, but sometime in the near future we will need a temporary code freeze to implement the bug patches.
Best regards,
- m.
__________________________________
Mike Wong, Staff Research Associate
Center for Computing for Life Sciences (CCLS)
San Francisco State University
1600 Holloway Avenue, Hensill Hall 301, SF, CA 94132
(415) 405-2119
mikewong at sfsu.edu
________________________________
Stay up to date on your PC, the Web, and your mobile phone with Windows Live
________________________________
Windows Live Hotmail now works up to 70% faster. Sign up today.
________________________________
Windows Live Hotmail now works up to 70% faster. Sign up today.
_______________________________________________
Feature-dev mailing list
Feature-dev at simtk.org
https://simtk.org/mailman/listinfo/feature-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://simtk.org/pipermail/feature-dev/attachments/20090702/2ccf3d8c/attachment-0001.html
More information about the Feature-dev
mailing list