[Feature-dev] Re: FEATURE Test Failures

[Trevor Blackstone]

Updated my working copy, ran 'make clean' in tests/ directory. Still 
getting failed smoke-tests so I don't think the testout files were 
dirty.  Here's the steps to reproduce the errors I see (starting from a 
clean environment):

$ svn checkout --username <name> -r <revision> 
https://simtk.org/svn/feature/trunk
$ cd trunk
$ aclocal
$ automake
$ autoconf
$ ./configure
$ make test
Here I get failed tests/smoke/trypsin-ser-og-positive.t and 
tests/smoke/trypsin-ser-og-negative.t, so I break before the files are 
cleaned up
Running a diff or awk shows the .ff files are not identical on certain 
vector entries. E.g. for entry 34:
$ awk < test/testout/trypsin_ser_og.pos.ff '{ print $35 }'
shows all zeroes, while
$ awk < test/hits/trypsin_ser_og.pos.ff '{ print $35 }'
gives different integers.

Worst case scenario is that featurize has not been producing output 
according to the r418 standard for some time (since r550 or possibly 
earlier). I'd just like a second opinion, or maybe someone can post 
their current r588 featurize output (e.g. 
test/testout/trypsin_ser_og.pos.ff) for comparison.

Trevor

Mike Wong wrote:
> If the feature vectors are mostly '0', then the problem could be a 
> dirty test environment.
>
> Go to the 'tests/' directory and 'make clean' and re-run the tests.
>
> I've changed the test Makefile so all tests clean up prior to running. 
> This behavior may not be the friendliest for retroactive testing, but 
> we'll see...
>
> Blast, does that mean that I need to add tests that test the tests to 
> complete our QA process?
>
> Cheers,
>
> - m.
>  
> __________________________________
> Mike Wong, Staff Research Associate
> Center for Computing for Life Sciences (CCLS)
> San Francisco State University
> 1600 Holloway Avenue, Hensill Hall 301, SF, CA 94132
> (415) 405-2119
> mikewong at sfsu.edu
>
>
> ------------------------------------------------------------------------
> *From:* Trevor Blackstone <tblackstone at gmail.com>
> *To:* Mike Wong <mikewong at sfsu.edu>
> *Cc:* Gurgen Tumanyan <tumanian at gmail.com>; Marc Sosnick 
> <marcsosnick at mac.com>
> *Sent:* Tue, November 17, 2009 11:03:10 AM
> *Subject:* FEATURE Test Failures
>
> Mike, I just finished a clean Fedora install and checked out the 
> sources, but running 'make test' is giving me errors in the feature 
> vectors.  I thought it was just a precision error because I didn't 
> replace the tests/hits folder with my own version, but looking at a 
> diff of the *.ff files shows entries in each vector that are 
> consistently 0 in the current build, but are whole numbers in the r418 
> hits.  E.g., when I generate tests/testout/trypsin_ser_og.pos.ff, 
> vector entry 34 is always 0, but the same entries in 
> tests/hits/trypsin_ser_og.pos.ff contain 7, 0, 21, 8, 4....  Several 
> other entries have differences as well. I've checked out several 
> versions back to r550 and so far the results are the same - each 
> revision differs from the hits with entries all set to 0.
>
> I'm hoping this is a simple error on my part for not configuring the 
> test environment correctly, but could someone take a look at some of 
> the older revisions to confirm if a problem exists, or if I should 
> even be concerned with this?
>
> Trevor
>