<div dir="ltr">I think this is a great step forward. I think many more readers would be interested in this direction. I applaud John for getting this started!<div><br></div><div>One quick thought, I personally felt that credibility/validation was a key aspect of this paper. To me, credibility/validation goes beyond reproducibility (which is often roughly equated to replicability). To me fewer papers have covered this higher bar. Could you integrate this theme more strongly? The paper could still be organized around 12 (or whatever number shakes) or challenges.</div><div> </div><div>Also, to motivate the focus on credibility, it could be helpful to cite an instance where the pandemic models were wrong or where lack of credibility (trust) inhibited use of a model. For the former, one example that comes to mind is the widely cited IHME model which was substantially off in the Spring. At one point I recall the model was predicting ~70,000 total deaths from the pandemic when 65,000 deaths had already occurred. It seemed to me that the predicted death total would be surpassed in just a week or two from the point that the prediction was made. This seems like a case where more accurate models and better validation might have been useful. </div><div><br></div><div>Jonathan</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, May 14, 2021 at 5:48 PM John Gennari <<a href="mailto:gennari@uw.edu" target="_blank">gennari@uw.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p><br>
</p>
<p>Potential co-authors: <br>
</p>
<p>As promised, attached find the skeleton of a complete re-writing
and revision of the white paper. This new version is tentatively
titled "A dozen challenges to biosimulation model reproducibility
and integration", reflecting a new focus to the paper. </p>
<p>I have also placed this document on a shared drive, where all of
you have the ability to comment (not edit):
<a href="https://docs.google.com/document/d/1VvyP3YZQdQYjj8DFKOpQ4pn_0pdDGgiT/edit" target="_blank">https://docs.google.com/document/d/1VvyP3YZQdQYjj8DFKOpQ4pn_0pdDGgiT/edit</a>
(I hope I've done the settings correctly! Let me know if not.) <br>
</p>
<p>It's important to note that this new draft is just a skeleton,
and not a complete paper yet. I have provided a new introduction
based on Fig 1 and Table 1 as I suggested in my earlier email. I
have not yet adapted or modified any of the 12 subsections that
correspond to the 12 challenges. This is work that could be
distributed among co-authors, of course. <br>
</p>
<p>I also have not yet created any good organization of these 12
challenges, although I put in one idea (in yellow, at the end).
Others would be very welcome. <br>
</p>
<p>Although I think this provides a much stronger organization to
the paper, I'm aiming to nonetheless be faithful to the central
ideas of the earlier draft, and thus I view it as a revision,
rather than a completely new paper. <br>
</p>
<p>My main goal for disseminating the attached is to get feedback --
is this a reasonable direction to head in for the revision? Are
there critical elements or ideas that I'm missing from the
introduction? As you can see below, Eric has kindly set up a
"whenisgood" poll for a potential meeting on Monday or Tuesday
(5/17 or 5/18). Eric, please let us know if we have a quorum for
any particular time. (The earlier you can let us know, the
better!) <br>
</p>
<p>Of course, if you aren't available to chat on Mon/Tues, email
comments and discussion are welcome! <br>
</p>
<p>-John Gennari</p>
<p><br>
</p>
<div>On 5/12/2021 8:04 PM, Eric Forgoston
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">Here is a link to a meeting poll for next
Monday/Tuesday, 17/18 May. Please fill out by Friday, 14 May,
so we can set a day/time to discuss the revision.
<div><br>
</div>
<div><a href="https://whenisgood.net/kdrzmkj" style="color:blue;font-family:sans-serif;text-align:center" target="_blank">http://whenisgood.net/kdrzmkj</a><br>
<div><br>
</div>
<div>Cheers,</div>
<div><br>
</div>
<div>Eric<br clear="all">
<div>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div><span>---------------------<br>
Dr. Eric Forgoston<br>
Professor of Applied
Mathematics<br>
Chair, Department of
</span><span><span><span>Applied
Mathematics and
Statistics</span></span><br>
Montclair State
University<br>
Montclair, NJ 07043
USA<br>
+1 973 655-7242 <br>
<span><a href="https://eric-forgoston.github.io/" target="_blank">https://eric-forgoston.github.io/</a></span></span><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, May 12, 2021 at 2:09
PM John Gennari <<a href="mailto:gennari@uw.edu" target="_blank">gennari@uw.edu</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p><br>
</p>
<p>This is definitely a good idea. The challenge might be to
schedule a time when a sufficient quorum of us are
available....</p>
<p>My plan over the rest of this week is to draft up a
significant re-working of the paper, and then distribute
that to all. This won't be a complete revision, but rather
some example of the kinds of things to say, and some
outlines of the proposed re-organization. I should have
this done by Friday, or this weekend at the latest. So
then a Zoom call next week (Monday, May 17?) might make
sense to talk thru the ideas that I send out by this
Friday. <br>
</p>
<p>-John G.</p>
<p><br>
</p>
<div>On 5/11/2021 7:37 PM, Eric Forgoston wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Dear all,</div>
<div><br>
</div>
It is probably worth having a zoom chat to discuss the
revision so that we are all working in concert. In
particular, John Gennari sent an email last week on May
3 with some thoughts that are well-worth discussing.
<div><br>
</div>
<div>Is there interest in doing this?</div>
<div><br>
</div>
<div>Cheers,</div>
<div><br>
</div>
<div>Eric<br>
<div>
<div>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div><span>---------------------<br>
Dr. Eric
Forgoston<br>
Professor of
Applied
Mathematics<br>
Chair,
Department of
</span><span><span><span>Applied
Mathematics
and Statistics</span></span><br>
Montclair
State
University<br>
Montclair, NJ
07043 USA<br>
+1 973
655-7242 <br>
<span><a href="https://eric-forgoston.github.io/" target="_blank">https://eric-forgoston.github.io/</a></span></span><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, May 11, 2021
at 10:26 AM Jacob Barhak <<a href="mailto:jacob.barhak@gmail.com" target="_blank">jacob.barhak@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Greetings white paper contributors,<br>
<div><br>
</div>
<div>The voting period is over and the vote did not
change, so it is now time to act.</div>
<div><br>
</div>
<div>The paper will be revised as requested by 10
voters and the target journal will be
Bioinformatics - James Glazier and Yaling Liu will
share publication costs.</div>
<div><br>
</div>
<div>All those who asked to revise are requested to
make the revisions and discuss those.</div>
<div><br>
</div>
<div>I ask to reach a revised version by May 25 so
that we can finish an approval round by June 1st.</div>
<div><br>
</div>
<div>So far only Sheriff provided suggestions for
revisions prior to submission so I suggest we
start discussing those:</div>
<div><a href="https://docs.google.com/document/d/1Ag4ipuybjtthxgV0YjXqYP7AwwNSYcWh/edit" target="_blank">https://docs.google.com/document/d/1Ag4ipuybjtthxgV0YjXqYP7AwwNSYcWh/edit</a><br>
</div>
<div><br>
</div>
<div>However, since it is out of format and out of
date, I suggest that all those who want to
make changes login and request to access the main
document and introduce changes there:</div>
<div><a href="https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit?usp=sharing" target="_blank">https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit?usp=sharing</a><br>
</div>
<div><br>
</div>
<div>I do ask that this will not be delayed any
further and those who asked for changes give this
task priority so it can be completed in time.</div>
<div><br>
</div>
<div>I look forward to your contributions. </div>
<div><br>
</div>
<div> Jacob</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Sun, May 9,
2021 at 4:25 PM Jacob Barhak <<a href="mailto:jacob.barhak@gmail.com" target="_blank">jacob.barhak@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Greetings white paper contributors,
<div><br>
</div>
<div>Allow me to remind you that the Voting and
vote changes are still possible until 11-May
1am CDT </div>
<div><br>
</div>
<div>The current votes I saw are:</div>
<div><br>
10 votes for revision<br>
Jonathan Karr<br>
Eric Forgoston<br>
William Waites<br>
James Glazier - RS Interface or
Bioinformatics<br>
Rahuman Sheriff<br>
Yaling Liu - Bioinformatics<br>
John Rice<br>
Winston Garira<br>
Gilberto Gonzalez-Parra<br>
James Osborne</div>
<div><br>
It seems 2 people supported Bioinformatics -
James Glazier and Yaling Liu so if this vote
is accepted, they will split publication costs
and the terget venue for revisions will be
Bioinformatics. <br>
<br>
There is currently 1 minority vote:<br>
Jacob Barhak for direct submission to Nature -
Scientific Reports </div>
<div><br>
</div>
<div>If revision is selected, time will be
limited, so I suggest those who voted to start
working on revisions. </div>
<div><br>
</div>
<div>Hopefully this reminder will move us
forward.</div>
<div><br>
</div>
<div> Jacob</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Sat, May 1,
2021 at 12:18 PM Jacob Barhak <<a href="mailto:jacob.barhak@gmail.com" target="_blank">jacob.barhak@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Greetings white paper
contributors,
<div><br>
</div>
<div>It is time to vote again for the
target venue. Here are the options again. </div>
<div>
<div>
<ol>
<li style="margin-left:15px">Cureus -
resubmission after addressing editor
comments</li>
<li style="margin-left:15px">Nature -
if you <span>vote</span> for this
venue please specify flavour such as
Nature Scientific Reports</li>
<li style="margin-left:15px">Science</li>
<li style="margin-left:15px">Briefings
in Bioinformatics</li>
<li style="margin-left:15px">Trends in
Biotechnology - requires distilling
the paper</li>
<li style="margin-left:15px">Journal
of The Royal Society Interface<br>
</li>
<li style="margin-left:15px">Annual
Review of Public Health<br>
</li>
<li style="margin-left:15px">BMJ</li>
<li style="margin-left:15px">Annual
Review of Biomedical Engineering<br>
</li>
<li style="margin-left:15px">F1000research
- if you <span>vote</span> for this
this venue please specify Gateway /
Collection </li>
<li style="margin-left:15px">bulletin
of mathematical biology<br>
</li>
<li style="margin-left:15px">Bioinformatics.<br>
</li>
<li style="margin-left:15px">Do not
submit now - instead open for
revisions for 2 weeks and then
submit. If you choose this
option also vote for the target
venue after revisions so that we
will not have to delay further. </li>
</ol>
<div><br>
</div>
</div>
<div>I will ask that contributors pick one
journal from that list - I ask that you
REPLY ALL so votes will be
transparent and time of vote will be
registered since first to vote will
break ties.</div>
<div><br>
</div>
<div>The voting period will be until
Tuesday 11-May 1am CDT </div>
</div>
<div><br>
</div>
<div>Again, if a journal costs for open
publication, whoever voted, will split
publication costs. If anyone on this list
is funded for this, please vote. </div>
<div><br>
</div>
<div>I urge contributors to vote - just so
that we will have a preference order to
follow in case of rejection.</div>
<div><br>
</div>
<div>Looking forward to your votes. </div>
<div><br>
</div>
<div> Jacob</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
_______________________________________________<br>
Vp-reproduce-subgroup mailing list<br>
<a href="mailto:Vp-reproduce-subgroup@lists.simtk.org" target="_blank">Vp-reproduce-subgroup@lists.simtk.org</a><br>
<a href="https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup" rel="noreferrer" target="_blank">https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup</a><br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote></div>