[Vp-integration-subgroup] [Vp-reproduce-subgroup] White paper submitted
Jacob Barhak
jacob.barhak at gmail.com
Thu Apr 22 18:05:00 PDT 2021
Ok John,
Your interpretation is interesting. and discusses another problem of non
unity in standardizations - yet there is competition in that world as well.
Yet to stay on point, can you please name one proper formal standard that
will nullify this sentence?
Note that a programming language spec like python or SBML are not standards
like HTML that is governed by W3C. Katherine will talk about standards and
Standards Development Organizations next week - so please tune in.
However, you know of standards that nullify that sentence, please name
those and send us links - we should all know about those in this group.
This is why we are here. This discussion is important outside the white
paper context.
Jacob
On Thu, Apr 22, 2021 at 7:28 PM John Rice <john.rice at noboxes.org> wrote:
> Agree with the statement Jim quoted being incorrect. But in common terms
> it’s intent applied.
> More correct would be “we have too many “standards” and keep creating
> more”.
>
> Or we don’t have A standard.
>
> For argument:
>
> But the pgm’ing language of individual models does not have to be nor
> should it be only one. I don’t care what language you program in. I only
> want you models data and then only if I understand all the details of every
> assumption used, and the ones you consciously left out, and why; the
> pedigree for all the data that fills the variables that apply to the
> assumption and the data used to arrive at the real world data that you used
> to validate you model, and the evidence of reproducibility and, and, and.
> I don’t care what coding language you use. The code is not the model, its
> just the mechanism for putting the model in a computational machine. If
> the model is good, you will get the same output from whatever language you
> use. If not either the language or the coder has a problem or the MODEL is
> so poorly documented that no two coders would get the same answer if the
> coded it in the same language.
>
> Typed with two thumbs on my iPhone. (757) 318-0671
>
> “Upon this gifted age, in its dark hour,
> Rains from the sky a meteoric shower
> Of facts . . . they lie unquestioned, uncombined.
> Wisdom enough to leech us of our ill
> Is daily spun; but there exists no loom
> To weave it into fabric.”
>
> –Edna St. Vincent Millay,
>
>
>
> On Apr 22, 2021, at 17:28, Jacob Barhak <jacob.barhak at gmail.com> wrote:
>
>
> Thanks Jim,
>
> Your contributions are valuable. I especially liked the wet lab example.
>
> I am not sure about FAIR - please explain what they do - there are so many
> types of repositories today that I lose track of the differences between
> those - if you can explain it will help.
>
> Specifically about standards. The sentence you marked is correct - we do
> need standards - python is a nice specification and perhaps can be
> considered a community standard by some, yet it is not a formal standard -
> although it is very nicely structured these days after the BDFL concept is
> gone and the PEP system for changes has been working for a long time.
>
> You mentioned ISO, yet there are multiple SDO's so unless ISO is special
> for some reason, there may be room to expand it to many others. For example
> Katherine Morse will be speaking about SISO in the webinar next week and
> she is also an IEEE member. We also reference a work presented in CDISC in
> the white paper - might as well mention CDISC at the same time. By the way,
> products of both SISO and CDISC are free for use - you may want to check
> those.
>
> I will try to see if I can bring someone from CDISC after SISO - no
> promises - yet since you opened the topic, I will try to do something for
> this subgroup.
>
> I am still not sure how we will merge your contributions. Those were noted
> and I will do my best to find a method to integrate your contributions. For
> now, let us please discuss those and how to best integrate them - yet
> integration will happen only after we see the review.
>
> I see you added the affiliation in your signature, so it will make things
> easier to add your contributions.
>
> And Jim, the big time sink was handling the reference in multiple systems
> - Cureus has its own mechanism to enter references which has its own logic
> and forms and every journal is different, so with 97 references, things
> will take time - this is why I ask that new contributions will be limited
> with references - if each person adds one reference - it takes each one a
> minute - it takes hours for the person entering all those into the journal
> system - those entries are through web forms by hand, so if I wronged
> someone in a past life, there is a real chance for payback here. Hopefully
> future contributors will choose to be nice.
>
> Jacob
>
>
>
>
>
>
>
>
>
> On Thu, Apr 22, 2021 at 11:55 AM Sluka, James Patrick <jsluka at indiana.edu>
> wrote:
>
>> Congratulations Jakob and everyone in the two subgroups!
>>
>>
>>
>> I think one take home lesson from Jacob’s comments is to not use Google
>> Docs to create a shared manuscript. 😊
>>
>>
>>
>> I realize it is too late, but I only saw the working draft recently. Here
>> are a few comments that I suspect the reviewers may also comment on.
>>
>>
>>
>> Comments on “Model Integration in Computational Biology: The Role of
>> Reproducibility, Credibility and Utility”
>>
>> 22 April 2021, J. Sluka jsluka at iu.edu
>>
>>
>> https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit
>>
>>
>>
>>
>> There is just as much a “reproducibility crisis” in wet lab and clinical
>> research as there is in computational research (PMIDs: 19174838, 32661593).
>> The problem is not unique to computational work. A perfect computational
>> model that is reproducible, well documented, shared, and findable (e.g.,
>> FAIR) is still wrong if the underlying biological data is wrong. Even when
>> a wet-lab experiment produces reproducible data (e.g., results in cell
>> culture), that data is not necessarily applicable to the situation in the
>> whole organism.
>>
>>
>>
>> FAIR (https://fairsharing.org/) should be mentioned as an organization
>> that is attempting to address issues of sharing and reproducibility.
>>
>>
>>
>> Journals are a big contributor to the reproducibility crisis (in both
>> computational and wet lab). Too many journals publish articles without
>> including executable code.
>>
>>
>>
>> Challenges in locating models via search are largely due to lack of use
>> of proper and unique names for biological, mathematical, and computational
>> concepts. Models should be annotated to standard bio-ontologies and all
>> basic text formats (including PDF, DOC, python, C++, …) are indexable by
>> web search engines like Google. We have shown that annotated python code,
>> in a web accessible place, is locatable with exquisitely high fidelity if
>> the python includes bio-ontologies IDs for the biological, mathematical,
>> and computational aspects. Try a google searching with “chebi:46195 dbbr”.
>> Returns exactly one document, a python program modeling acetaminophen
>> (chebi:46195).
>>
>>
>>
>> Standards in data are just as important as standards in modeling.
>>
>>
>>
>> *“*
>> *We lack standards for all of the key elements that need to be
>> represented: the objects, the processes (behaviors and interactions) they
>> participate in, the initial and boundary conditions and the dynamics and
>> events that govern their evolution.” *The sentence above is not true.
>> Many of the needed standards do exist. There are standards and controlled
>> vocabularies to describe most biological concepts; genes, proteins, cell,
>> organs, species, molecules, processes (mitosis, chemotaxis, …) etc. There
>> are also extensive standards for describing experimental systems. What is
>> lacking is (1) the domain of describing the mathematical and computational
>> instantiation of a model and (2) using existing standards and nomenclature
>> to describe the biological concepts within a model. Computational models
>> developed in standard programming languages (R, Python, …) are extremely
>> difficult to annotate and generally are not. They *could be annotated
>> but the tools don’t exist*. (Imagine a capability like Doxygen (
>> https://www.doxygen.nl/) that includes tags not just for creating
>> documentation of the code but also specific tags to identify biological
>> concepts. Existing filters pull out documentation for the code, a
>> ‘biological’ annotation filter could pull out the description of the
>> biological model independent of how it is represented in the code.)
>>
>>
>>
>> Should mention international standards organizations such as ISO. ISO is
>> actively developing standards in many *biomedical data* areas. The
>> internet works because of the W3C/ISO standards. In addition, ISO is
>> developing standards for modeling. Unfortunately, ISO standards are not
>> free.
>>
>>
>>
>> Finally, virtually all biological science is concerned with “models”.
>> Computational models are not fundamentally different and suffer many of the
>> same intrinsic flaws as in vitro, in vivo and clinical research. Wet lab
>> experiments are not easily annotated and generally are not. So again, the
>> computational area is not unique in significant issues in “*Role of
>> Reproducibility, Credibility and Utility.”*
>>
>>
>>
>> James P. Sluka, PhD
>> Biocomplexity Institute
>>
>> Intelligent Systems Engineering
>>
>> School of Informatics, Computing and Engineering
>>
>> Indiana University
>>
>> Bloomington, IN USA
>>
>> Office: 812-855-2441
>>
>> Cell: 317-331-7465
>>
>> JSluka at Indiana.edu
>>
>>
>>
>>
>>
>>
>>
>> *From:* Vp-integration-subgroup <
>> vp-integration-subgroup-bounces at lists.simtk.org> *On Behalf Of *Jacob
>> Barhak
>> *Sent:* Thursday, April 22, 2021 5:50 AM
>> *To:* James Glazier <jaglazier at gmail.com>
>> *Cc:* vp-reproduce-subgroup at lists.simtk.org;
>> vp-integration-subgroup at lists.simtk.org; Jonathan Karr <
>> jonrkarr at gmail.com>; John Gennari <gennari at uw.edu>; Winston Garira <
>> Winston.Garira at univen.ac.za>
>> *Subject:* [Vp-integration-subgroup] White paper submitted
>>
>>
>>
>> Greetings White paper contributors,
>>
>>
>>
>> Since no objection was raised the paper was just submitted to Cureus as
>> previously elected.
>>
>>
>>
>> You will find an updated version after changes necessary for submission
>> were implemented in this link:
>>
>>
>> https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit?usp=sharing
>>
>>
>>
>>
>>
>> The changes were mostly minor typos and reference management - a few
>> references needed correction and one was deleted since it no longer showed
>> any relevant information.
>>
>>
>>
>> to give you all perspective - only formatting and handling of references
>> towards publication in the necessary format took 2 work days dedicated for
>> this alone.
>>
>>
>>
>> So all those who want to add anything to the paper to be considered post
>> review, I urge you to:
>>
>> 1. submit any changes now - do not wait - once the review is returned no
>> changes or additions will be considered and my experience is that Cureus
>> provides review rapidly, so time is limited.
>>
>> 2. If you add references, please provide a DOI / link to help process
>> those. And please avoid adding many references - processing those takes a
>> long time.
>>
>> 3. Do not send changes to me alone using reply - please use REPLY ALL so
>> everyone will see the discussion - we want to be as transparent as possible
>>
>>
>>
>>
>>
>> I hope for a quick review process.
>>
>>
>>
>> Jacob
>>
>>
>>
>>
>>
>> On Fri, Apr 16, 2021 at 6:08 PM Jacob Barhak <jacob.barhak at gmail.com>
>> wrote:
>>
>> Greetings white paper contributors
>>
>>
>>
>> The voting period on the venue has ended and the selected venue we will
>> send the white paper to is Cureus.
>>
>>
>>
>> John Rice and myself voted for that venue and will cover the publication
>> fees.
>>
>>
>>
>> In case of any issue with this venue we will move to iScience that James
>> Gazier voted for.
>>
>>
>>
>> The paper will need formatting to fit the venue - it handles references
>> in a specific way. However, I intend to mostly cut and paste the text in
>> this version - without change:
>>
>>
>> https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit?usp=sharing
>>
>>
>>
>> The process we went through ensures we reach a level of consensus and
>> that the manuscript can be legally submitted on behalf of you all. If
>> anyone has any strong objections to stop the submission process, please
>> stop now. Otherwise I will start the submission process next week.
>>
>>
>>
>> Please remember this is a large team with many people so there will be
>> compromises. Moreover, the paper will undergo review and we will have to
>> make changes.
>>
>>
>>
>> During the writing process I presented some deadlines and denied
>> contributions that happened after the deadline was over. I know Tomas
>> Halikar wanted to contribute and now Jim Saluka wants to contribute. I also
>> know John Gennari schedule prevented him from properly reviewing the paper
>> and I know that Sheriff asked for several modifications. All this can be
>> corrected when the paper gets reviewed and we get the review - we will then
>> open the paper for modifications and potentially other contributions.
>> However, the time then will be limited, so to conserve time, I suggest that
>> those interested in changes or additions, continue the discussion in
>> parallel to the formal submission process.
>>
>>
>>
>> Cureus review process is typically quick compared to many venues I
>> encountered in the past. Once the paper comes back from review we will have
>> limited time to respond, so to conserve time I will consider only text that
>> we submitted and contributions.modifications made until that time. So if
>> you have any important additions, please create your own copy of the
>> document and publicly share the link with these mailing lists. Similar to
>> what Sheriff did.
>>
>>
>>
>> So please do not stop discussion on the paper - As Jonathan Karr
>> suggested, we may have different versions suitable for different venues so
>> your contributions are valuable - however, for the sake of getting our work
>> published I ask that we do things in a timely manner.
>>
>>
>>
>> Again, if you have things to communicate about the paper - do not wait
>> until review is back - it will be too late then - instead comment now so
>> your contributions can be considered when post review modifications start.
>>
>>
>>
>> I hope we have a fast review and can make this work amplified by Cureus
>> soon.
>>
>>
>>
>> Jacob
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Apr 15, 2021 at 7:13 PM Jacob Barhak <jacob.barhak at gmail.com>
>> wrote:
>>
>> Greetings Contributors,
>>
>>
>>
>> Since there are only a few hours before the voting period ends, I will
>> vote to break a tie between CureuUs and iScience.
>>
>>
>>
>> I was about to abstain, yet it seems the tie needs to be broken
>> distinctively.
>>
>>
>>
>> I will vote for Cureus. My reasons are:
>>
>> - I am familiar with this venue submission process
>>
>> - the venue has opened up to allow more references - thus dismissing my
>> original remark on it
>>
>> - I will be the person handling submission and it will save me time to
>> submit somewhere familiar.
>>
>> - The review process in this venue is relatively fast.
>>
>> - It has an option for rushed pubmed central publication
>>
>> - the open publication costs are reasonable compared to many other venues
>>
>>
>>
>> Unless 3 other people will vote for another venue in the next few hours,
>> it seems this is the venue that the paper will be submitted to for review.
>> However, things may change.
>>
>>
>>
>> John and I will split the publication costs since we both voted on this
>> venue. If anyone else wants to split costs, feel free to publicly vote for
>> this venue - otherwise it will be John and myself.
>>
>>
>>
>> Please note that there are a few more hours to vote - a bit less than 6
>> hours from the time of this email - if you support another conclusion for a
>> venue, you are welcome to vote.
>>
>>
>>
>> Jacob
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Apr 14, 2021 at 11:03 AM Jacob Barhak <jacob.barhak at gmail.com>
>> wrote:
>>
>> Greetings white paper contributors,
>>
>>
>>
>> Please vote on a venue for the paper. There are less than two days to
>> vote to influence the submission venue.
>>
>>
>>
>> We had 2 votes by now
>>
>> Cureus by John Rice
>>
>> iScience by James Glazier
>>
>>
>>
>> John Rice was first, yet did not reply to all as requested and sent me
>> the email - I forwarded his vote to the list first. James was the first
>> that replied to all, so technically he is the first valid vote.
>>
>>
>>
>> So currently there is a tie that should be broken by first vote which is
>> open to interpretation. Since time is running out, I suggest people choose
>> to make the choice distinctive, otherwise I will vote to break the tie and
>> was hoping to avoid voting since I had a lot of influence already and
>> wanted to yield control.
>>
>>
>>
>> So please, if you have a preference on venue, please vote for one venue
>> by 1am CDT April 16th.
>>
>>
>>
>>
>>
>> I hope we have a conclusive decision.
>>
>>
>>
>> Jacob
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Apr 12, 2021, 17:17 James Glazier <jaglazier at gmail.com> wrote:
>>
>> iScience
>>
>> James A. Glazier
>>
>>
>>
>> On Mon, Apr 12, 2021 at 12:40 PM Jacob Barhak <jacob.barhak at gmail.com>
>> wrote:
>>
>> Greetings White paper Contributors,
>>
>>
>>
>> This is your chance to decide on the venue. If you name is on this list,
>> please take a few minutes anv vote for the venue:
>>
>> Jonathan Karr
>> Rahuman Sheriff
>> James Osborne
>> Gilberto Gonzalez Parra
>> Eric Forgoston
>> Ruth Bowness
>> Yaling Liu
>> Robin Thompson
>> Winston Garira
>> Marcella Torres
>> Hana M. Dobrovolny
>> Tingting Tang
>> William Waites
>> James Glazier
>> James R. Faeder
>>
>>
>>
>> Currently one vote was cast and unless there will be more votes, the
>> venue voted for will be chosen. So if you have a strong preference, this is
>> your chance to influence the publication venue.
>>
>>
>>
>> You will find eligible venues below as well as additional details.
>>
>>
>>
>> I look forward to your votes.
>>
>>
>>
>> Jacob
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Apr 8, 2021 at 7:04 PM John Rice <john.rice at noboxes.org> wrote:
>>
>> No objection to any but will VOTE CUREUS.
>>
>>
>>
>> My understanding it was created to provide a peer reviewed open source
>> indexed journal that could accommodate new forms of papers and new relevant
>> topical areas. Good timing for them lifting the limit on references, so
>> assume this paper could be submitted in current form subject only to
>> reviewers’ response.
>>
>>
>>
>> Don’t know that it has a model credibility related topic section yet.
>>
>>
>>
>> John
>>
>>
>>
>>
>>
>> Typed with two thumbs on my iPhone. (757) 318-0671
>>
>>
>>
>> “Upon this gifted age, in its dark hour,
>>
>> Rains from the sky a meteoric shower
>>
>> Of facts . . . they lie unquestioned, uncombined.
>>
>> Wisdom enough to leech us of our ill
>>
>> Is daily spun; but there exists no loom
>>
>> To weave it into fabric.”
>>
>>
>>
>> –Edna St. Vincent Millay,
>>
>>
>>
>>
>>
>>
>> On Apr 8, 2021, at 01:23, Jacob Barhak <jacob.barhak at gmail.com> wrote:
>>
>>
>>
>> Greeting White paper contributors,
>>
>>
>>
>> The old Thread that contained discussions towards creations and approval
>> of the white paper have become too long, so I started a new maling thread.
>>
>>
>>
>> You can find the old discussion thread here:
>>
>>
>> https://lists.simtk.org/pipermail/vp-reproduce-subgroup/2021-April/000052.html
>>
>>
>>
>> To summarize, we have reached a point where 17 authors approved the
>> following version for submission, pending some minor changes like
>> spelling and grammar correction.
>>
>>
>>
>> To avoid any confusion - here is the paper version we approved is here:
>>
>>
>> https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit?usp=sharing
>>
>>
>>
>> At this point we need to select a venue to submit the paper to. Here is a
>> short list I collected after incorporating all the suggestions and removed
>> all venues that had any objections. To your convenience I added additional
>> notes form personal knowledge - I did not look at issues such as
>> publication fees for open access - different venues may have different
>> rules and may require some additional investment, so please look at the
>> venue you are choosing and learn the limitations/benefits before you vote:
>>
>>
>>
>> Here is the short list:
>>
>> 1. Cureus - will require cutting out some references due to limitation
>> 2. Nature - if you vote for this venue please specify flavour such as
>> Nature Scientific Reports
>> 3. Science
>> 4. Briefings in Bioinformatics
>> 5. Trends in Biotechnology - requires distilling the paper
>> 6. Journal of The Royal Society Interface
>> 7. Annual Review of Public Health
>> 8. BMJ
>> 9. Annual Review of Biomedical Engineering
>> 10. F1000research - if you vote for this this venue please
>> specify Gateway / Collection
>> 11. iScience
>> 12. bulletin of mathematical biology
>> 13. Bioinformatics.
>>
>>
>>
>> I ask that each contributor who has a preference among those Journals
>> reply to all to this message and pick one venue. Please pick only one
>> considering all aspects of the venue. You are welcome to include your
>> reasoning, yet vote towards only one venue. You are welcome to change your
>> mind - yet only your last vote will count.
>>
>>
>>
>> In case of a tie in the number of votes, the venue that got the first
>> counted vote will be chosen.
>>
>>
>>
>> After we select, I will put the work to format the submission towards
>> that venue and include all necessary submission matters. In case of fees,
>> those who voted for the venue will be responsible for covering publication
>> fees.
>>
>>
>>
>> At this point I assume no one has any objections to any venues and we are
>> all ok with submitting this version - so we are just prioritizing according
>> to the majority of wishes while keeping the process transparent and giving
>> some incentive to early bird vote.
>>
>>
>>
>> If any of my assumptions are not correct, please correct me now!
>>
>>
>>
>> This is the best way I think we can create consensus in such a large
>> group - and consensus is legally necessary for publication.
>>
>>
>>
>> I ask that we limit the voting time to approximately one week. So votes
>> should be cast by 1am April 16th CDT.
>>
>>
>>
>> I will send another reminder during this week, yet I assume one week is
>> sufficient to make a simple choice of prefered venue and those not voting
>> elect to abstain from choosing and prefer the majority choice.
>>
>>
>>
>> I look forward to your votes.
>>
>>
>>
>> Jacob
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Vp-reproduce-subgroup mailing list
>> Vp-reproduce-subgroup at lists.simtk.org
>> https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
>>
>> _______________________________________________
>> Vp-reproduce-subgroup mailing list
>> Vp-reproduce-subgroup at lists.simtk.org
>> https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
>>
>>
>>
>>
>> --
>>
>> James A. Glazier
>>
>> Indiana University
>>
>> _______________________________________________
> Vp-reproduce-subgroup mailing list
> Vp-reproduce-subgroup at lists.simtk.org
> https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.simtk.org/pipermail/vp-integration-subgroup/attachments/20210422/cbb3e4a3/attachment-0001.html>
More information about the Vp-integration-subgroup
mailing list