[Vp-integration-subgroup] [Vp-reproduce-subgroup] White paper submitted
John Rice
john.rice at noboxes.org
Thu Apr 22 17:28:38 PDT 2021
Agree with the statement Jim quoted being incorrect. But in common terms it’s intent applied.
More correct would be “we have too many “standards” and keep creating more”.
Or we don’t have A standard.
For argument:
But the pgm’ing language of individual models does not have to be nor should it be only one. I don’t care what language you program in. I only want you models data and then only if I understand all the details of every assumption used, and the ones you consciously left out, and why; the pedigree for all the data that fills the variables that apply to the assumption and the data used to arrive at the real world data that you used to validate you model, and the evidence of reproducibility and, and, and. I don’t care what coding language you use. The code is not the model, its just the mechanism for putting the model in a computational machine. If the model is good, you will get the same output from whatever language you use. If not either the language or the coder has a problem or the MODEL is so poorly documented that no two coders would get the same answer if the coded it in the same language.
Typed with two thumbs on my iPhone. (757) 318-0671
“Upon this gifted age, in its dark hour,
Rains from the sky a meteoric shower
Of facts . . . they lie unquestioned, uncombined.
Wisdom enough to leech us of our ill
Is daily spun; but there exists no loom
To weave it into fabric.”
–Edna St. Vincent Millay,
On Apr 22, 2021, at 17:28, Jacob Barhak <jacob.barhak at gmail.com> wrote:
Thanks Jim,
Your contributions are valuable. I especially liked the wet lab example.
I am not sure about FAIR - please explain what they do - there are so many types of repositories today that I lose track of the differences between those - if you can explain it will help.
Specifically about standards. The sentence you marked is correct - we do need standards - python is a nice specification and perhaps can be considered a community standard by some, yet it is not a formal standard - although it is very nicely structured these days after the BDFL concept is gone and the PEP system for changes has been working for a long time.
You mentioned ISO, yet there are multiple SDO's so unless ISO is special for some reason, there may be room to expand it to many others. For example Katherine Morse will be speaking about SISO in the webinar next week and she is also an IEEE member. We also reference a work presented in CDISC in the white paper - might as well mention CDISC at the same time. By the way, products of both SISO and CDISC are free for use - you may want to check those.
I will try to see if I can bring someone from CDISC after SISO - no promises - yet since you opened the topic, I will try to do something for this subgroup.
I am still not sure how we will merge your contributions. Those were noted and I will do my best to find a method to integrate your contributions. For now, let us please discuss those and how to best integrate them - yet integration will happen only after we see the review.
I see you added the affiliation in your signature, so it will make things easier to add your contributions.
And Jim, the big time sink was handling the reference in multiple systems - Cureus has its own mechanism to enter references which has its own logic and forms and every journal is different, so with 97 references, things will take time - this is why I ask that new contributions will be limited with references - if each person adds one reference - it takes each one a minute - it takes hours for the person entering all those into the journal system - those entries are through web forms by hand, so if I wronged someone in a past life, there is a real chance for payback here. Hopefully future contributors will choose to be nice.
Jacob
On Thu, Apr 22, 2021 at 11:55 AM Sluka, James Patrick <jsluka at indiana.edu> wrote:
> Congratulations Jakob and everyone in the two subgroups!
>
>
>
> I think one take home lesson from Jacob’s comments is to not use Google Docs to create a shared manuscript. 😊
>
>
>
> I realize it is too late, but I only saw the working draft recently. Here are a few comments that I suspect the reviewers may also comment on.
>
>
>
> Comments on “Model Integration in Computational Biology: The Role of Reproducibility, Credibility and Utility”
>
> 22 April 2021, J. Sluka jsluka at iu.edu
>
> https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit
>
>
>
> There is just as much a “reproducibility crisis” in wet lab and clinical research as there is in computational research (PMIDs: 19174838, 32661593). The problem is not unique to computational work. A perfect computational model that is reproducible, well documented, shared, and findable (e.g., FAIR) is still wrong if the underlying biological data is wrong. Even when a wet-lab experiment produces reproducible data (e.g., results in cell culture), that data is not necessarily applicable to the situation in the whole organism.
>
>
>
> FAIR (https://fairsharing.org/) should be mentioned as an organization that is attempting to address issues of sharing and reproducibility.
>
>
>
> Journals are a big contributor to the reproducibility crisis (in both computational and wet lab). Too many journals publish articles without including executable code.
>
>
>
> Challenges in locating models via search are largely due to lack of use of proper and unique names for biological, mathematical, and computational concepts. Models should be annotated to standard bio-ontologies and all basic text formats (including PDF, DOC, python, C++, …) are indexable by web search engines like Google. We have shown that annotated python code, in a web accessible place, is locatable with exquisitely high fidelity if the python includes bio-ontologies IDs for the biological, mathematical, and computational aspects. Try a google searching with “chebi:46195 dbbr”. Returns exactly one document, a python program modeling acetaminophen (chebi:46195).
>
>
>
> Standards in data are just as important as standards in modeling.
>
>
> “We lack standards for all of the key elements that need to be represented: the objects, the processes (behaviors and interactions) they participate in, the initial and boundary conditions and the dynamics and events that govern their evolution.”
> The sentence above is not true. Many of the needed standards do exist. There are standards and controlled vocabularies to describe most biological concepts; genes, proteins, cell, organs, species, molecules, processes (mitosis, chemotaxis, …) etc. There are also extensive standards for describing experimental systems. What is lacking is (1) the domain of describing the mathematical and computational instantiation of a model and (2) using existing standards and nomenclature to describe the biological concepts within a model. Computational models developed in standard programming languages (R, Python, …) are extremely difficult to annotate and generally are not. They could be annotated but the tools don’t exist. (Imagine a capability like Doxygen (https://www.doxygen.nl/) that includes tags not just for creating documentation of the code but also specific tags to identify biological concepts. Existing filters pull out documentation for the code, a ‘biological’ annotation filter could pull out the description of the biological model independent of how it is represented in the code.)
>
> Should mention international standards organizations such as ISO. ISO is actively developing standards in many biomedical data areas. The internet works because of the W3C/ISO standards. In addition, ISO is developing standards for modeling. Unfortunately, ISO standards are not free.
>
>
>
> Finally, virtually all biological science is concerned with “models”. Computational models are not fundamentally different and suffer many of the same intrinsic flaws as in vitro, in vivo and clinical research. Wet lab experiments are not easily annotated and generally are not. So again, the computational area is not unique in significant issues in “Role of Reproducibility, Credibility and Utility.”
>
>
>
> James P. Sluka, PhD
> Biocomplexity Institute
>
> Intelligent Systems Engineering
>
> School of Informatics, Computing and Engineering
>
> Indiana University
>
> Bloomington, IN USA
>
> Office: 812-855-2441
>
> Cell: 317-331-7465
>
> JSluka at Indiana.edu
>
>
>
>
>
>
>
> From: Vp-integration-subgroup <vp-integration-subgroup-bounces at lists.simtk.org> On Behalf Of Jacob Barhak
> Sent: Thursday, April 22, 2021 5:50 AM
> To: James Glazier <jaglazier at gmail.com>
> Cc: vp-reproduce-subgroup at lists.simtk.org; vp-integration-subgroup at lists.simtk.org; Jonathan Karr <jonrkarr at gmail.com>; John Gennari <gennari at uw.edu>; Winston Garira <Winston.Garira at univen.ac.za>
> Subject: [Vp-integration-subgroup] White paper submitted
>
>
>
> Greetings White paper contributors,
>
>
>
> Since no objection was raised the paper was just submitted to Cureus as previously elected.
>
>
>
> You will find an updated version after changes necessary for submission were implemented in this link:
>
> https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit?usp=sharing
>
>
>
>
>
> The changes were mostly minor typos and reference management - a few references needed correction and one was deleted since it no longer showed any relevant information.
>
>
>
> to give you all perspective - only formatting and handling of references towards publication in the necessary format took 2 work days dedicated for this alone.
>
>
>
> So all those who want to add anything to the paper to be considered post review, I urge you to:
>
> 1. submit any changes now - do not wait - once the review is returned no changes or additions will be considered and my experience is that Cureus provides review rapidly, so time is limited.
>
> 2. If you add references, please provide a DOI / link to help process those. And please avoid adding many references - processing those takes a long time.
>
> 3. Do not send changes to me alone using reply - please use REPLY ALL so everyone will see the discussion - we want to be as transparent as possible
>
>
>
>
>
> I hope for a quick review process.
>
>
>
> Jacob
>
>
>
>
>
> On Fri, Apr 16, 2021 at 6:08 PM Jacob Barhak <jacob.barhak at gmail.com> wrote:
>
> Greetings white paper contributors
>
>
>
> The voting period on the venue has ended and the selected venue we will send the white paper to is Cureus.
>
>
>
> John Rice and myself voted for that venue and will cover the publication fees.
>
>
>
> In case of any issue with this venue we will move to iScience that James Gazier voted for.
>
>
>
> The paper will need formatting to fit the venue - it handles references in a specific way. However, I intend to mostly cut and paste the text in this version - without change:
>
> https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit?usp=sharing
>
>
>
> The process we went through ensures we reach a level of consensus and that the manuscript can be legally submitted on behalf of you all. If anyone has any strong objections to stop the submission process, please stop now. Otherwise I will start the submission process next week.
>
>
>
> Please remember this is a large team with many people so there will be compromises. Moreover, the paper will undergo review and we will have to make changes.
>
>
>
> During the writing process I presented some deadlines and denied contributions that happened after the deadline was over. I know Tomas Halikar wanted to contribute and now Jim Saluka wants to contribute. I also know John Gennari schedule prevented him from properly reviewing the paper and I know that Sheriff asked for several modifications. All this can be corrected when the paper gets reviewed and we get the review - we will then open the paper for modifications and potentially other contributions. However, the time then will be limited, so to conserve time, I suggest that those interested in changes or additions, continue the discussion in parallel to the formal submission process.
>
>
>
> Cureus review process is typically quick compared to many venues I encountered in the past. Once the paper comes back from review we will have limited time to respond, so to conserve time I will consider only text that we submitted and contributions.modifications made until that time. So if you have any important additions, please create your own copy of the document and publicly share the link with these mailing lists. Similar to what Sheriff did.
>
>
>
> So please do not stop discussion on the paper - As Jonathan Karr suggested, we may have different versions suitable for different venues so your contributions are valuable - however, for the sake of getting our work published I ask that we do things in a timely manner.
>
>
>
> Again, if you have things to communicate about the paper - do not wait until review is back - it will be too late then - instead comment now so your contributions can be considered when post review modifications start.
>
>
>
> I hope we have a fast review and can make this work amplified by Cureus soon.
>
>
>
> Jacob
>
>
>
>
>
>
>
> On Thu, Apr 15, 2021 at 7:13 PM Jacob Barhak <jacob.barhak at gmail.com> wrote:
>
> Greetings Contributors,
>
>
>
> Since there are only a few hours before the voting period ends, I will vote to break a tie between CureuUs and iScience.
>
>
>
> I was about to abstain, yet it seems the tie needs to be broken distinctively.
>
>
>
> I will vote for Cureus. My reasons are:
>
> - I am familiar with this venue submission process
>
> - the venue has opened up to allow more references - thus dismissing my original remark on it
>
> - I will be the person handling submission and it will save me time to submit somewhere familiar.
>
> - The review process in this venue is relatively fast.
>
> - It has an option for rushed pubmed central publication
>
> - the open publication costs are reasonable compared to many other venues
>
>
>
> Unless 3 other people will vote for another venue in the next few hours, it seems this is the venue that the paper will be submitted to for review. However, things may change.
>
>
>
> John and I will split the publication costs since we both voted on this venue. If anyone else wants to split costs, feel free to publicly vote for this venue - otherwise it will be John and myself.
>
>
>
> Please note that there are a few more hours to vote - a bit less than 6 hours from the time of this email - if you support another conclusion for a venue, you are welcome to vote.
>
>
>
> Jacob
>
>
>
>
>
>
>
>
>
> On Wed, Apr 14, 2021 at 11:03 AM Jacob Barhak <jacob.barhak at gmail.com> wrote:
>
> Greetings white paper contributors,
>
>
>
> Please vote on a venue for the paper. There are less than two days to vote to influence the submission venue.
>
>
>
> We had 2 votes by now
>
> Cureus by John Rice
>
> iScience by James Glazier
>
>
>
> John Rice was first, yet did not reply to all as requested and sent me the email - I forwarded his vote to the list first. James was the first that replied to all, so technically he is the first valid vote.
>
>
>
> So currently there is a tie that should be broken by first vote which is open to interpretation. Since time is running out, I suggest people choose to make the choice distinctive, otherwise I will vote to break the tie and was hoping to avoid voting since I had a lot of influence already and wanted to yield control.
>
>
>
> So please, if you have a preference on venue, please vote for one venue by 1am CDT April 16th.
>
>
>
>
>
> I hope we have a conclusive decision.
>
>
>
> Jacob
>
>
>
>
>
>
>
>
>
> On Mon, Apr 12, 2021, 17:17 James Glazier <jaglazier at gmail.com> wrote:
>
> iScience
>
> James A. Glazier
>
>
>
> On Mon, Apr 12, 2021 at 12:40 PM Jacob Barhak <jacob.barhak at gmail.com> wrote:
>
> Greetings White paper Contributors,
>
>
>
> This is your chance to decide on the venue. If you name is on this list, please take a few minutes anv vote for the venue:
>
> Jonathan Karr
> Rahuman Sheriff
> James Osborne
> Gilberto Gonzalez Parra
> Eric Forgoston
> Ruth Bowness
> Yaling Liu
> Robin Thompson
> Winston Garira
> Marcella Torres
> Hana M. Dobrovolny
> Tingting Tang
> William Waites
> James Glazier
> James R. Faeder
>
>
>
> Currently one vote was cast and unless there will be more votes, the venue voted for will be chosen. So if you have a strong preference, this is your chance to influence the publication venue.
>
>
>
> You will find eligible venues below as well as additional details.
>
>
>
> I look forward to your votes.
>
>
>
> Jacob
>
>
>
>
>
>
>
> On Thu, Apr 8, 2021 at 7:04 PM John Rice <john.rice at noboxes.org> wrote:
>
> No objection to any but will VOTE CUREUS.
>
>
>
> My understanding it was created to provide a peer reviewed open source indexed journal that could accommodate new forms of papers and new relevant topical areas. Good timing for them lifting the limit on references, so assume this paper could be submitted in current form subject only to reviewers’ response.
>
>
>
> Don’t know that it has a model credibility related topic section yet.
>
>
>
> John
>
>
>
>
> Typed with two thumbs on my iPhone. (757) 318-0671
>
>
>
> “Upon this gifted age, in its dark hour,
>
> Rains from the sky a meteoric shower
>
> Of facts . . . they lie unquestioned, uncombined.
>
> Wisdom enough to leech us of our ill
>
> Is daily spun; but there exists no loom
>
> To weave it into fabric.”
>
>
>
> –Edna St. Vincent Millay,
>
>
>
>
>
>
> On Apr 8, 2021, at 01:23, Jacob Barhak <jacob.barhak at gmail.com> wrote:
>
>
> Greeting White paper contributors,
>
>
>
> The old Thread that contained discussions towards creations and approval of the white paper have become too long, so I started a new maling thread.
>
>
>
> You can find the old discussion thread here:
>
> https://lists.simtk.org/pipermail/vp-reproduce-subgroup/2021-April/000052.html
>
>
>
> To summarize, we have reached a point where 17 authors approved the following version for submission, pending some minor changes like spelling and grammar correction.
>
>
>
> To avoid any confusion - here is the paper version we approved is here:
>
> https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit?usp=sharing
>
>
>
> At this point we need to select a venue to submit the paper to. Here is a short list I collected after incorporating all the suggestions and removed all venues that had any objections. To your convenience I added additional notes form personal knowledge - I did not look at issues such as publication fees for open access - different venues may have different rules and may require some additional investment, so please look at the venue you are choosing and learn the limitations/benefits before you vote:
>
>
>
> Here is the short list:
>
> Cureus - will require cutting out some references due to limitation
> Nature - if you vote for this venue please specify flavour such as Nature Scientific Reports
> Science
> Briefings in Bioinformatics
> Trends in Biotechnology - requires distilling the paper
> Journal of The Royal Society Interface
> Annual Review of Public Health
> BMJ
> Annual Review of Biomedical Engineering
> F1000research - if you vote for this this venue please specify Gateway / Collection
> iScience
> bulletin of mathematical biology
> Bioinformatics.
>
>
> I ask that each contributor who has a preference among those Journals reply to all to this message and pick one venue. Please pick only one considering all aspects of the venue. You are welcome to include your reasoning, yet vote towards only one venue. You are welcome to change your mind - yet only your last vote will count.
>
>
>
> In case of a tie in the number of votes, the venue that got the first counted vote will be chosen.
>
>
>
> After we select, I will put the work to format the submission towards that venue and include all necessary submission matters. In case of fees, those who voted for the venue will be responsible for covering publication fees.
>
>
>
> At this point I assume no one has any objections to any venues and we are all ok with submitting this version - so we are just prioritizing according to the majority of wishes while keeping the process transparent and giving some incentive to early bird vote.
>
>
>
> If any of my assumptions are not correct, please correct me now!
>
>
>
> This is the best way I think we can create consensus in such a large group - and consensus is legally necessary for publication.
>
>
>
> I ask that we limit the voting time to approximately one week. So votes should be cast by 1am April 16th CDT.
>
>
>
> I will send another reminder during this week, yet I assume one week is sufficient to make a simple choice of prefered venue and those not voting elect to abstain from choosing and prefer the majority choice.
>
>
>
> I look forward to your votes.
>
>
>
> Jacob
>
>
>
>
>
>
>
> _______________________________________________
> Vp-reproduce-subgroup mailing list
> Vp-reproduce-subgroup at lists.simtk.org
> https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
>
> _______________________________________________
> Vp-reproduce-subgroup mailing list
> Vp-reproduce-subgroup at lists.simtk.org
> https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
>
>
>
>
>
> --
>
> James A. Glazier
>
> Indiana University
>
_______________________________________________
Vp-reproduce-subgroup mailing list
Vp-reproduce-subgroup at lists.simtk.org
https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.simtk.org/pipermail/vp-integration-subgroup/attachments/20210422/e08e9ed6/attachment-0001.html>
More information about the Vp-integration-subgroup
mailing list