<div dir="ltr">Dear John and all others,<div><br></div><div>thanks for the very nice summary of the constructive meeting. </div><div><br></div><div>I would like to contribute to (8) "Credibility and validity of models" (as per some comments and addition on regulatory guidance, especially V&V40), given the chapter will be open for new contributions on top of the existing ones. </div><div><br></div><div>All the best </div><div>Alexander </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 18 May 2021 at 07:12, John Gennari <<a href="mailto:gennari@uw.edu">gennari@uw.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p><br>
</p>
<p>All: About 9 of us had a lively Zoom meeting today to chat about
the manuscript. By the end, it was a productive meeting, and I'm
hoping that this email will capture some key outputs from the
meeting. I apologize if I said some things that were a bit
"inflammatory". Obviously 2 years would be much too long to get
this paper out-the door. <br>
</p>
<p>I saw two outcomes. First, we had some nice ideas and discussion
about re-ordering (initiated by Tomas Helikar). In the below, I'm
going to propose one possible ordering, but this is certainly a
work-in-progress. The reason that I think ordering is important is
that it will give us a much better ability to write a strong
concluding section, where we talk about themes and the larger arc
of our ideas. <br>
</p>
<p>Second, we agreed that we should nominate "point persons" who
would be in charge of at least the initial cut of each of the
subsections. As Jacob pointed out, this information should be easy
to get from older email and history of the development of the
paper. During the zoom meeting, we associated some co-authors with
some sections, but our coverage wasn't perfect (see challenge
#12). Hopefully people will "stand up" and admit that some section
of text is theirs. <br>
</p>
<p>So in the below, I include the original title of the section, a
few words about the content of that section, and then a name (or
several names) of co-authors who will be the "point person" to
make sure that the appropriate content is included. Obviously, all
co-authors can and should chime in on any part of the text, but
the point person should make sure that the key ideas are included.
<br>
</p>
<p>The basic ordering idea for the dozen challenges was to follow
the life-cycle of model development, execution, sharing and
integration, and eventually implementation. So... <br>
</p>
<p>*********************************************<br>
</p>
<p><b>(1) "</b><b>Data</b><b> and measurement definitions</b>".
Before you can build a model, you must have data. So data
availability and measurement standards is the place to start. <br>
</p>
<p><b>People: </b>Hana D, Jacob B<br>
</p>
<p><b>(2) "</b><b>The variety of modeling languages</b>" This is
about the choice of modeling languages, such as using SBML,
CellML, or Matlab. As I said on the phone call, this is sort of
about "syntax"--how do you write down your model? <br>
</p>
<p><b>People:</b> John G, Jon K, Rahuman S.</p>
<p><b>(3) "</b><b>The variety of modeling paradigms and scales"</b><b>
</b>Separately from modeling syntax, we must acknowledge modeling
paradigms with very different semantics. Some clear examples are
PDEs versus ODEs versus rule-based systems (and obviously one can
combine these). Certainly semantics might impact syntax (the prior
challenge), in that certain modeling language might be appropriate
only for some paradigms.</p>
<p>People: James G, Eric F (?) <br>
</p>
<p><b>(4) "</b><b>Units standardization</b>" A common reason that
models are not reproducible are errors in units, or
misunderstanding about units, or simply a lack of information
about units. <br>
</p>
<p>People: Jacob B, Hana D</p>
<p><b>(5) "</b><b>A lack of annotations in models</b>". Once
researchers publish models, they must annotate the model so that
others can understand it. Quality annotation is essential for both
search and reproducibility. <br>
</p>
<p>People: John G.</p>
<p><b>(6) "</b><b>Models are hard to locate"</b> If your goal is to
reproduce, understand and possibly reuse or integrate some other
model, one must first find that model. This requires annotation
(prior section) and repositories (Physiome Model Repository,
BioModels) and search platforms (ModeleXchange). <br>
</p>
<p>People: Jon K, John G.</p>
<p><b>(7) "</b><b>Common platforms to execute models" </b>A model
is pretty worthless as a static object. For folk to understand and
reproduce models they must be executable. Alas, there is no single
or consistent way of executing a model -- and of course, this
interacts direction with section #2 and #3, above: Execution
platforms are usually only for one modeling paradigm, and often
for one modeling language. The BioSimulators work goes here.</p>
<p>People: Jon K. <br>
</p>
<p><b>(8) "</b><b>Credibility </b><b>and validity of models</b>"
Once a model is published, how do folk know it is right? Model
validation is a big topic and challenge. Credibility follows (in
part) from validation, but also requires transparency and
reproducibility, etc. <br>
</p>
<p>People: John Rice, Jon K, Jacob B</p>
<p><b>(9) "</b><b>Environments to adapt and integrate models</b>" As
I see it, one of the end-targets for this manuscript is to better
enable model integration, to build better models. There are many
challenges with the task of integrating two (or more) models. (One
that has recently been discussed is that even if model A and model
B are valid and correct, there is no guarantee that the combined
model A+B is correct. I liked what William Waites and Katherine
Morse posted on this subject.) This section is where SBML-comp and
SemGen environments can be mentioned.</p>
<p>People: John G.</p>
<p>(<b>10) "Challenges for stochastic models" </b>Special
challenges specific to stochaistic modeling. An obvious point to
mention is repeatability -- stochastic models don't necessarily
give the same results with the same inputs. <br>
</p>
<p>People: James G., Eric F<br>
</p>
<p><b>(11) "Licensing barriers" </b>Issues around "open source" and
CC0 licensing. <br>
<b></b></p>
<p>People: Jacob B<br>
</p>
<p><b>(12) "Barriers to model implementations and applications"</b>
(I might suggest this be re-phrased for better clarity). What this
section should discuss are challenges is getting a community to
actually use models for "real-world" applications or decision
making. This is more of a cultural/societal challenge, and thus
seems like a nice big-picture way to end. <br>
</p>
<p><b>People: ?? </b>I don't have any names here...<br>
</p>
<p>*********************************************</p>
<p>We didn't really talk much about it in the Zoom meeting, but
there have been ideas tossed around about a "baker's dozen", i.e.,
adding a 13th challenge. We could also potentially merge some of
the above. <br>
</p>
<p class="MsoNormal">The "point persons" listed above is obviously a
subset of co-authors. That's fine and appropriate. Just for
transparency, I follow what I think is pretty standard policy for
authorship issues, and nicely summarized by the<span style="font-family:Helvetica,sans-serif">
International
Committee of Medical Journal Editors (ICMJE); see 2019 updated
document at </span><a href="http://www.icmje.org/icmje-recommendations.pdf" target="_blank">http://www.icmje.org/icmje-recommendations.pdf</a>
(Or see, below my signature, a summary of the key points of this
document). <br>
</p>
<p class="MsoNormal">Finally, I've made the document editable by all
at
<a href="https://docs.google.com/document/d/1VvyP3YZQdQYjj8DFKOpQ4pn_0pdDGgiT/edit?ts=60a294c2" target="_blank">https://docs.google.com/document/d/1VvyP3YZQdQYjj8DFKOpQ4pn_0pdDGgiT/edit?ts=60a294c2</a>
<br>
<span style="font-family:Helvetica,sans-serif"></span></p>
<p></p>
<p>-John G.<br>
==========================================================================<br>
Associate Professor & Graduate Program Director
<a href="mailto:gennari@uw.edu" target="_blank"><gennari@uw.edu></a><br>
Dep't of Biomedical Informatics and
telephone:206-616-6641<br>
Medical Education, box 358047 <br>
University of Washington<br>
Seattle, WA 98109-4714
<a href="http://faculty.washington.edu/gennari/" target="_blank">http://faculty.washington.edu/gennari/</a><br>
==========================================================================<br>
<br>
</p>
<p>
</p>
<p class="MsoNormal" style="margin-left:0.25in"><span style="font-size:10pt;line-height:107%;font-family:Helvetica,sans-serif">The
ICMJE recommends that
authorship be based on the following 4 criteria: </span></p>
<p style="margin-left:0.75in"><span style="font-size:10pt;line-height:107%;font-family:Helvetica,sans-serif"><span>1.<span> </span></span></span><span style="font-size:10pt;line-height:107%;font-family:Helvetica,sans-serif">Substantial
contributions to the conception or design of the work; or the
acquisition,
analysis, or interpretation of data for the work; AND</span></p>
<p style="margin-left:0.75in"><span style="font-size:10pt;line-height:107%;font-family:Helvetica,sans-serif"><span>2.<span> </span></span></span><span style="font-size:10pt;line-height:107%;font-family:Helvetica,sans-serif">Drafting
the work or revising it critically for important intellectual
content; AND </span></p>
<p style="margin-left:0.75in"><span style="font-size:10pt;line-height:107%;font-family:Helvetica,sans-serif"><span>3.<span> </span></span></span><span style="font-size:10pt;line-height:107%;font-family:Helvetica,sans-serif">Final
approval of the version to be published; AND </span></p>
<p style="margin-left:0.75in"><span style="font-size:10pt;line-height:107%;font-family:Helvetica,sans-serif"><span>4.<span> </span></span></span><span style="font-size:10pt;line-height:107%;font-family:Helvetica,sans-serif">Agreement
to be accountable for all aspects of the work in ensuring that
questions
related to the accuracy or integrity of any part of the work are
appropriately
investigated and resolved.</span></p>
<p style="margin-left:0.75in"><span style="font-size:10pt;line-height:107%;font-family:Helvetica,sans-serif"><br>
</span></p>
<p></p>
<p><br>
</p>
</div>
_______________________________________________<br>
Vp-integration-subgroup mailing list<br>
<a href="mailto:Vp-integration-subgroup@lists.simtk.org" target="_blank">Vp-integration-subgroup@lists.simtk.org</a><br>
<a href="https://lists.simtk.org/mailman/listinfo/vp-integration-subgroup" rel="noreferrer" target="_blank">https://lists.simtk.org/mailman/listinfo/vp-integration-subgroup</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div style="font-family:sans-serif;font-size:13px;white-space:pre-wrap;padding:2px 0px 2px 13px;color:rgb(54,57,79)"><div style="font-stretch:normal;line-height:normal;white-space:normal;padding:2px 0px;color:rgb(35,35,73)">Alexander Kulesza</div><div style="font-stretch:normal;font-size:11px;line-height:normal;white-space:normal;padding:2px 0px;color:rgb(52,70,115)">Team leader</div><div style="font-stretch:normal;font-size:11px;line-height:normal;white-space:normal;padding:2px 0px;color:rgb(82,135,189)">Modeling & simulation / Biomodeling</div><div style="font-stretch:normal;font-size:11px;line-height:normal;white-space:normal;padding:2px 0px;color:rgb(128,134,150)"><a href="mailto:alexander.kulesza@novadiscovery.com" style="color:rgb(128,134,150)" target="_blank">alexander.kulesza@novadiscovery.com</a></div><div style="font-stretch:normal;font-size:11px;line-height:normal;white-space:normal;padding:2px 0px;color:rgb(128,134,150)">+33 7 82 92 44 62</div><div style="white-space:normal;width:100px;margin:15px 0px;font-size:28px;color:rgb(35,35,73)"><div style="font-family:Montserrat,arial;font-weight:600;letter-spacing:-1px">nova</div><div style="font-size:9px">DISCOVERY</div></div><div style="color:rgb(51,51,51);font-stretch:normal;font-size:11px;line-height:normal;white-space:normal;padding:2px 0px;margin:0px"><a href="http://www.novadiscovery.com/" title="Visit our website" style="color:rgb(128,134,150)" target="_blank">www.novadiscovery.com</a></div><div style="font-stretch:normal;font-size:11px;line-height:normal;white-space:normal;padding:2px 0px;color:rgb(128,134,150)">1 Place Verrazzano, 69009 Lyon<br>+33 9 72 53 13 01</div></div></div></div>
<br>
<i style="color:rgb(187,187,187);font-family:sans-serif;font-size:10px;text-align:justify">This message is intended only for the personal and confidential use of the designated recipient(s) named above. If you are not the intended recipient of this message you are hereby notified that any review, dissemination, distribution or copying of this message is strictly prohibited. Email transmission cannot be guaranteed to be secure or error-free. Therefore, we do not represent that this information is complete or accurate and it should not be relied upon as such. All information is subject to change without notice.</i>