<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">John-<o:p></o:p></p>
<p class="MsoNormal">Is that the correct link? The link is to the National Academies (not the NSF) and the work described stared about 4 years ago and ended more than a year ago.<o:p></o:p></p>
<p class="MsoNormal">Jim S<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Vp-integration-subgroup <vp-integration-subgroup-bounces@lists.simtk.org>
<b>On Behalf Of </b>John Rice<br>
<b>Sent:</b> Thursday, May 20, 2021 12:53 PM<br>
<b>To:</b> Rahuman Sheriff <sheriff@ebi.ac.uk><br>
<b>Cc:</b> vp-reproduce-subgroup@lists.simtk.org; vp-integration-subgroup@lists.simtk.org; Jonathan Karr <jonrkarr@gmail.com>; John Gennari <gennari@uw.edu>; Winston Garira <Winston.Garira@univen.ac.za><br>
<b>Subject:</b> [Vp-integration-subgroup] With link this time.<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Check This from NSF Today<o:p></o:p></p>
<div>
<div>
<div>
<p class="MsoNormal">I’ve only glances at it. <o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><a href="https://www.nationalacademies.org/our-work/reproducibility-and-replicability-in-science?utm_source=NASEM+News+and+Publications&utm_campaign=9578a7dc42-Eblast_Decoding_Science_General_2021_05_20&utm_medium=email&utm_term=0_96101de015-9578a7dc42-110837174&goal=0_96101de015-9578a7dc42-110837174&mc_cid=9578a7dc42&mc_eid=b42e7aaa4d">https://www.nationalacademies.org/our-work/reproducibility-and-replicability-in-science?utm_source=NASEM+News+and+Publications&utm_campaign=9578a7dc42-Eblast_Decoding_Science_General_2021_05_20&utm_medium=email&utm_term=0_96101de015-9578a7dc42-110837174&goal=0_96101de015-9578a7dc42-110837174&mc_cid=9578a7dc42&mc_eid=b42e7aaa4d</a><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
On May 18, 2021, at 08:32, Rahuman Sheriff <<a href="mailto:sheriff@ebi.ac.uk">sheriff@ebi.ac.uk</a>> wrote:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Dear John G,<o:p></o:p></p>
<div>
<p class="MsoNormal">Many thanks for the update.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I like the new ordering :)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">@Jacob, as you mentioned you have 6 versions, could you please add the link to those versions in John G document, so they are all inked.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I give consent to to rephrase my contribution or even remove part or all of my contribution to the white paper and present the ideas in other sections if required to make the paper coherent and flow well.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The white paper has a great collections of ideas, I hope we can get it into a good shape soon for submission and benefit the scientific community.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Best regards<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Sheriff<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On 18 May 2021, at 13:28, James Osborne <<a href="mailto:jmosborne@unimelb.edu.au">jmosborne@unimelb.edu.au</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Sorry I wasn't on the call Yesterday (it was 1 am for me so not really achievable). Looking at the emails looks like it was useful. <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Happy to help how I can, in terms of areas as someone on the multicellular side of life i'm probably most use on 3 but happy to support others.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">James <o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Tue, May 18, 2021 at 3:12 PM John Gennari <<a href="mailto:gennari@uw.edu" target="_blank">gennari@uw.edu</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="100%" style="width:100.0%;float:left">
<tbody>
<tr>
<td style="padding:0in 0in 0in 0in">
<p class="MsoNormal"><b><span style="color:red">External email: </span></b><span style="color:red">Please exercise caution<o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><o:p> </o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">All: About 9 of us had a lively Zoom meeting today to chat about the manuscript. By the end, it was a productive meeting, and I'm hoping that this email will capture some key outputs
from the meeting. I apologize if I said some things that were a bit "inflammatory". Obviously 2 years would be much too long to get this paper out-the door.
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I saw two outcomes. First, we had some nice ideas and discussion about re-ordering (initiated by Tomas Helikar). In the below, I'm going to propose one possible ordering, but this
is certainly a work-in-progress. The reason that I think ordering is important is that it will give us a much better ability to write a strong concluding section, where we talk about themes and the larger arc of our ideas.
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Second, we agreed that we should nominate "point persons" who would be in charge of at least the initial cut of each of the subsections. As Jacob pointed out, this information should
be easy to get from older email and history of the development of the paper. During the zoom meeting, we associated some co-authors with some sections, but our coverage wasn't perfect (see challenge #12). Hopefully people will "stand up" and admit that some
section of text is theirs. <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">So in the below, I include the original title of the section, a few words about the content of that section, and then a name (or several names) of co-authors who will be the "point
person" to make sure that the appropriate content is included. Obviously, all co-authors can and should chime in on any part of the text, but the point person should make sure that the key ideas are included.
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The basic ordering idea for the dozen challenges was to follow the life-cycle of model development, execution, sharing and integration, and eventually implementation. So...
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">*********************************************<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>(1) "Data and measurement definitions</b>". Before you can build a model, you must have data. So data availability and measurement standards is the place to start.
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>People:
</b>Hana D, Jacob B<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>(2) "The variety of modeling languages</b>" This is about the choice of modeling languages, such as using SBML, CellML, or Matlab. As I said on the phone call, this is sort of
about "syntax"--how do you write down your model? <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>People:</b> John G, Jon K, Rahuman S.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>(3) "The variety of modeling paradigms and scales"
</b>Separately from modeling syntax, we must acknowledge modeling paradigms with very different semantics. Some clear examples are PDEs versus ODEs versus rule-based systems (and obviously one can combine these). Certainly semantics might impact syntax (the
prior challenge), in that certain modeling language might be appropriate only for some paradigms.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">People: James G, Eric F (?)
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>(4) "Units standardization</b>" A common reason that models are not reproducible are errors in units, or misunderstanding about units, or simply a lack of information about units.
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">People: Jacob B, Hana D<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>(5) "A lack of annotations in models</b>". Once researchers publish models, they must annotate the model so that others can understand it. Quality annotation is essential for
both search and reproducibility. <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">People: John G.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>(6) "Models are hard to locate"</b> If your goal is to reproduce, understand and possibly reuse or integrate some other model, one must first find that model. This requires annotation
(prior section) and repositories (Physiome Model Repository, BioModels) and search platforms (ModeleXchange).
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">People: Jon K, John G.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>(7) "Common platforms to execute models"
</b>A model is pretty worthless as a static object. For folk to understand and reproduce models they must be executable. Alas, there is no single or consistent way of executing a model -- and of course, this interacts direction with section #2 and #3, above:
Execution platforms are usually only for one modeling paradigm, and often for one modeling language. The BioSimulators work goes here.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">People: Jon K.
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>(8) "Credibility and validity of models</b>" Once a model is published, how do folk know it is right? Model validation is a big topic and challenge. Credibility follows (in part)
from validation, but also requires transparency and reproducibility, etc. <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">People: John Rice, Jon K, Jacob B<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>(9) "Environments to adapt and integrate models</b>" As I see it, one of the end-targets for this manuscript is to better enable model integration, to build better models. There
are many challenges with the task of integrating two (or more) models. (One that has recently been discussed is that even if model A and model B are valid and correct, there is no guarantee that the combined model A+B is correct. I liked what William Waites
and Katherine Morse posted on this subject.) This section is where SBML-comp and SemGen environments can be mentioned.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">People: John G.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">(<b>10) "Challenges for stochastic models"
</b>Special challenges specific to stochaistic modeling. An obvious point to mention is repeatability -- stochastic models don't necessarily give the same results with the same inputs.
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">People: James G., Eric F<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>(11) "Licensing barriers"
</b>Issues around "open source" and CC0 licensing. <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">People: Jacob B<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>(12) "Barriers to model implementations and applications"</b> (I might suggest this be re-phrased for better clarity). What this section should discuss are challenges is getting
a community to actually use models for "real-world" applications or decision making. This is more of a cultural/societal challenge, and thus seems like a nice big-picture way to end.
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>People: ??
</b>I don't have any names here...<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">*********************************************<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">We didn't really talk much about it in the Zoom meeting, but there have been ideas tossed around about a "baker's dozen", i.e., adding a 13th challenge. We could also potentially
merge some of the above. <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The "point persons" listed above is obviously a subset of co-authors. That's fine and appropriate. Just for transparency, I follow what I think is pretty standard policy for authorship
issues, and nicely summarized by the<span style="font-family:"Helvetica",sans-serif"> International Committee of Medical Journal Editors (ICMJE); see 2019 updated document at
</span><a href="http://www.icmje.org/icmje-recommendations.pdf" target="_blank">http://www.icmje.org/icmje-recommendations.pdf</a> (Or see, below my signature, a summary of the key points of this document).
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Finally, I've made the document editable by all at
<a href="https://docs.google.com/document/d/1VvyP3YZQdQYjj8DFKOpQ4pn_0pdDGgiT/edit?ts=60a294c2" target="_blank">
https://docs.google.com/document/d/1VvyP3YZQdQYjj8DFKOpQ4pn_0pdDGgiT/edit?ts=60a294c2</a>
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt">-John G.<br>
==========================================================================<br>
Associate Professor & Graduate Program Director <a href="mailto:gennari@uw.edu" target="_blank">
<gennari@uw.edu></a><br>
Dep't of Biomedical Informatics and telephone:206-616-6641<br>
Medical Education, box 358047 <br>
University of Washington<br>
Seattle, WA 98109-4714 <a href="http://faculty.washington.edu/gennari/" target="_blank">
http://faculty.washington.edu/gennari/</a><br>
==========================================================================<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.25in">
<span style="font-size:10.0pt;font-family:"Helvetica",sans-serif">The ICMJE recommends that authorship be based on the following 4 criteria:
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.75in">
<span style="font-size:10.0pt;font-family:"Helvetica",sans-serif">1. Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.75in">
<span style="font-size:10.0pt;font-family:"Helvetica",sans-serif">2. Drafting the work or revising it critically for important intellectual content; AND
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.75in">
<span style="font-size:10.0pt;font-family:"Helvetica",sans-serif">3. Final approval of the version to be published; AND
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.75in">
<span style="font-size:10.0pt;font-family:"Helvetica",sans-serif">4. Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.75in">
<o:p> </o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal">_______________________________________________<br>
Vp-integration-subgroup mailing list<br>
<a href="mailto:Vp-integration-subgroup@lists.simtk.org">Vp-integration-subgroup@lists.simtk.org</a><br>
<a href="https://lists.simtk.org/mailman/listinfo/vp-integration-subgroup">https://lists.simtk.org/mailman/listinfo/vp-integration-subgroup</a><o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br>
_______________________________________________<br>
Vp-reproduce-subgroup mailing list<br>
<a href="mailto:Vp-reproduce-subgroup@lists.simtk.org">Vp-reproduce-subgroup@lists.simtk.org</a><br>
<a href="https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup">https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup</a><o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>