[Vp-integration-subgroup] [Vp-reproduce-subgroup] White paper revision

Fri May 14 15:38:00 PDT 2021

I think this is a great step forward. I think many more readers would be
interested in this direction. I applaud John for getting this started!

One quick thought, I personally felt that credibility/validation was a key
aspect of this paper. To me, credibility/validation goes beyond
reproducibility (which is often roughly equated to replicability). To me
fewer papers have covered this higher bar. Could you integrate this theme
more strongly? The paper could still be organized around 12 (or whatever
number shakes) or challenges.

Also, to motivate the focus on credibility, it could be helpful to cite an
instance where the pandemic models were wrong or where lack of credibility
(trust) inhibited use of a model. For the former, one example that comes to
mind is the widely cited IHME model which was substantially off in the
Spring. At one point I recall the model was predicting ~70,000 total deaths
from the pandemic when 65,000 deaths had already occurred. It seemed to me
that the predicted death total would be surpassed in just a week or two
from the point that the prediction was made. This seems like a case where
more accurate models and better validation might have been useful.

Jonathan

On Fri, May 14, 2021 at 5:48 PM John Gennari <gennari at uw.edu> wrote:

>
> Potential co-authors:
>
> As promised, attached find the skeleton of a complete re-writing and
> revision of the white paper. This new version is tentatively titled "A
> dozen challenges to biosimulation model reproducibility and integration",
> reflecting a new focus to the paper.
>
> I have also placed this document on a shared drive, where all of you have
> the ability to comment (not edit):
> https://docs.google.com/document/d/1VvyP3YZQdQYjj8DFKOpQ4pn_0pdDGgiT/edit
> (I hope I've done the settings correctly! Let me know if not.)
>
> It's important to note that this new draft is just a skeleton, and not a
> complete paper yet. I have provided a new introduction based on Fig 1 and
> Table 1 as I suggested in my earlier email. I have not yet adapted or
> modified any of the 12 subsections that correspond to the 12 challenges.
> This is work that could be distributed among co-authors, of course.
>
> I also have not yet created any good organization of these 12 challenges,
> although I put in one idea (in yellow, at the end). Others would be very
> welcome.
>
> Although I think this provides a much stronger organization to the paper,
> I'm aiming to nonetheless be faithful to the central ideas of the earlier
> draft, and thus I view it as a revision, rather than a completely new
> paper.
>
> My main goal for disseminating the attached is to get feedback -- is this
> a reasonable direction to head in for the revision? Are there critical
> elements or ideas that I'm missing from the introduction? As you can see
> below, Eric has kindly set up a "whenisgood" poll for a potential meeting
> on Monday or Tuesday (5/17 or 5/18). Eric, please let us know if we have a
> quorum for any particular time. (The earlier you can let us know, the
> better!)
>
> Of course, if you aren't available to chat on Mon/Tues, email comments and
> discussion are welcome!
>
> -John Gennari
>
>
> On 5/12/2021 8:04 PM, Eric Forgoston wrote:
>
> Here is a link to a meeting poll for next Monday/Tuesday, 17/18 May.
> Please fill out by Friday, 14 May, so we can set a day/time to discuss the
> revision.
>
> http://whenisgood.net/kdrzmkj <https://whenisgood.net/kdrzmkj>
>
> Cheers,
>
> Eric
> ---------------------
> Dr. Eric Forgoston
> Professor of Applied Mathematics
> Chair, Department of Applied Mathematics and Statistics
> Montclair State University
> Montclair, NJ  07043 USA
> +1 973 655-7242
> https://eric-forgoston.github.io/
>
>
> On Wed, May 12, 2021 at 2:09 PM John Gennari <gennari at uw.edu> wrote:
>
>>
>> This is definitely a good idea. The challenge might be to schedule a time
>> when a sufficient quorum of us are available....
>>
>> My plan over the rest of this week is to draft up a significant
>> re-working of the paper, and then distribute that to all. This won't be a
>> complete revision, but rather some example of the kinds of things to say,
>> and some outlines of the proposed re-organization. I should have this done
>> by Friday, or this weekend at the latest. So then a Zoom call next week
>> (Monday, May 17?) might make sense to talk thru the ideas that I send out
>> by this Friday.
>>
>> -John G.
>>
>>
>> On 5/11/2021 7:37 PM, Eric Forgoston wrote:
>>
>> Dear all,
>>
>> It is probably worth having a zoom chat to discuss the revision so that
>> we are all working in concert.  In particular, John Gennari sent an email
>> last week on May 3 with some thoughts that are well-worth discussing.
>>
>> Is there interest in doing this?
>>
>> Cheers,
>>
>> Eric
>> ---------------------
>> Dr. Eric Forgoston
>> Professor of Applied Mathematics
>> Chair, Department of Applied Mathematics and Statistics
>> Montclair State University
>> Montclair, NJ  07043 USA
>> +1 973 655-7242
>> https://eric-forgoston.github.io/
>>
>>
>> On Tue, May 11, 2021 at 10:26 AM Jacob Barhak <jacob.barhak at gmail.com>
>> wrote:
>>
>>> Greetings white paper contributors,
>>>
>>> The voting period is over and the vote did not change, so it is now time
>>> to act.
>>>
>>> The paper will be revised as requested by 10 voters and the target
>>> journal will be Bioinformatics - James Glazier and Yaling Liu will share
>>> publication costs.
>>>
>>> All those who asked to revise are requested to make the revisions and
>>> discuss those.
>>>
>>> I ask to reach a revised version by May 25 so that we can finish an
>>> approval round by June 1st.
>>>
>>> So far only Sheriff provided suggestions for revisions prior to
>>> submission so I suggest we start discussing those:
>>> https://docs.google.com/document/d/1Ag4ipuybjtthxgV0YjXqYP7AwwNSYcWh/edit
>>>
>>> However, since it is out of format and out of date, I suggest that all
>>> those who want to make changes login and request to access the main
>>> document and introduce changes there:
>>>
>>> https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit?usp=sharing
>>>
>>> I do ask that this will not be delayed any further and those who asked
>>> for changes give this task priority so it can be completed in time.
>>>
>>> I look forward to your contributions.
>>>
>>>             Jacob
>>>
>>>
>>> On Sun, May 9, 2021 at 4:25 PM Jacob Barhak <jacob.barhak at gmail.com>
>>> wrote:
>>>
>>>> Greetings white paper contributors,
>>>>
>>>> Allow me to remind you that the Voting and vote changes are still
>>>> possible until 11-May 1am CDT
>>>>
>>>> The current votes I saw are:
>>>>
>>>> 10 votes for revision
>>>> Jonathan Karr
>>>> Eric Forgoston
>>>> William Waites
>>>> James Glazier  - RS Interface or Bioinformatics
>>>> Rahuman Sheriff
>>>> Yaling Liu - Bioinformatics
>>>> John Rice
>>>> Winston Garira
>>>> Gilberto Gonzalez-Parra
>>>> James Osborne
>>>>
>>>> It seems 2 people supported Bioinformatics - James Glazier and Yaling
>>>> Liu  so if this vote is accepted, they will split publication costs and the
>>>> terget venue for revisions will be Bioinformatics.
>>>>
>>>> There is currently 1 minority vote:
>>>> Jacob Barhak for direct submission to Nature -  Scientific Reports
>>>>
>>>> If revision is selected, time will be limited, so I suggest those who
>>>> voted to start working on revisions.
>>>>
>>>> Hopefully this reminder will move us forward.
>>>>
>>>>             Jacob
>>>>
>>>>
>>>>
>>>> On Sat, May 1, 2021 at 12:18 PM Jacob Barhak <jacob.barhak at gmail.com>
>>>> wrote:
>>>>
>>>>> Greetings white paper contributors,
>>>>>
>>>>> It is time to vote again for the target venue. Here are the options
>>>>> again.
>>>>>
>>>>>    1. Cureus - resubmission after addressing editor comments
>>>>>    2. Nature - if you vote for this venue please specify flavour such
>>>>>    as Nature Scientific Reports
>>>>>    3. Science
>>>>>    4. Briefings in Bioinformatics
>>>>>    5. Trends in Biotechnology - requires distilling the paper
>>>>>    6. Journal of The Royal Society Interface
>>>>>    7. Annual Review of Public Health
>>>>>    8. BMJ
>>>>>    9. Annual Review of Biomedical Engineering
>>>>>    10. F1000research - if you vote for this this venue please
>>>>>    specify Gateway / Collection
>>>>>    11. bulletin of mathematical biology
>>>>>    12. Bioinformatics.
>>>>>    13. Do not submit now - instead open for revisions for 2 weeks and
>>>>>    then submit. If you choose this option also vote for the target venue after
>>>>>    revisions so that we will not have to delay further.
>>>>>
>>>>>
>>>>> I will ask that contributors pick one journal from that list - I ask
>>>>> that you REPLY ALL so votes will be transparent and time of vote will be
>>>>> registered since first to vote will break ties.
>>>>>
>>>>> The voting period will be until Tuesday 11-May 1am CDT
>>>>>
>>>>> Again, if a journal costs for open publication, whoever voted, will
>>>>> split publication costs. If anyone on this list is funded for this, please
>>>>> vote.
>>>>>
>>>>> I urge contributors to vote - just so that we will have a preference
>>>>> order to follow in case of rejection.
>>>>>
>>>>> Looking forward to your votes.
>>>>>
>>>>>             Jacob
>>>>>
>>>> _______________________________________________
>>> Vp-reproduce-subgroup mailing list
>>> Vp-reproduce-subgroup at lists.simtk.org
>>> https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.simtk.org/pipermail/vp-integration-subgroup/attachments/20210514/706c6c72/attachment-0001.html>