[Vp-reproduce-subgroup] [Vp-integration-subgroup] Paper review and revision check list
Eric Forgoston
eric.forgoston at montclair.edu
Fri Jan 14 09:17:28 PST 2022
Approve.
---------------------
Eric Forgoston
Professor of Applied Mathematics
Chair, Department of Applied Mathematics and Statistics
Montclair State University
Montclair, NJ 07043 USA
+1 973 655-7242
https://eric-forgoston.github.io/
On Fri, Jan 14, 2022 at 12:12 PM John Rice <john.rice at noboxes.org> wrote:
> I also approve.
>
> On Jan 14, 2022, at 12:05, James Glazier <jaglazier at gmail.com> wrote:
>
>
> APPROVE
>
> On Fri, Jan 14, 2022 at 11:02 AM Yaling Liu <yal310 at lehigh.edu> wrote:
>
>> I also APPROVE
>>
>> Thanks for all the efforts!
>>
>> Yaling
>>
>> On Fri, Jan 14, 2022 at 1:38 AM Jacob Barhak <jacob.barhak at gmail.com>
>> wrote:
>>
>>> Thanks Marcella,
>>>
>>> In that case, it seems we have no reviewer objection and hopefully the
>>> editor will be positive as well . To speed up the process we need
>>> the following::
>>> 1. Sheriff - please send a high resolution image for Figure 1
>>> 2. Alex - Please send a high resolution image for Figure 2
>>> 3. Everyone please review the manuscript and Reply to all in this email
>>> with one word - either APPROVE, DISAPPROVE, ABSTAIN
>>>
>>> Where:
>>>
>>> APPROVE means you are ok with the version we reached - please note that
>>> we do have some new elements there and your approval means you understand
>>> what it means - please read the paper well and understand the nuances of
>>> approval.
>>>
>>> DISAPPROVE- means that you no longer agree with the version we reached
>>> and we need to discuss the paper more and withdraw it - if you have some
>>> serious objection, it is the last chance to get the paper back to the
>>> drawing board
>>>
>>> ABSTAIN - You have no objection for the paper being published, yet you
>>> want your name withdrawn
>>>
>>> Also, if anyone else has any objections for this paper being published,
>>> please raise them now so we can discuss them.
>>>
>>> I call again to John Gennarri to join us as an author - he did
>>> substantial work and many of his contributions are in the final version -
>>> including the order in figure 2. I can trace back many of his contributions
>>> and I think he should get credit for it - hopefully my last call will be
>>> accepted and he will rejoin.
>>>
>>> William - I did not forget your second affiliation - I just could not
>>> change it in the system - I will write to the editor about that when we get
>>> the final acceptance notice.
>>>
>>> I look forward to seeing your final decisions - hopefully it will be
>>> fast this time.
>>>
>>> Jacob
>>>
>>>
>>> On Thu, Jan 13, 2022 at 8:50 PM Torres, Marcella <mtorres at richmond.edu>
>>> wrote:
>>>
>>>> Sorry, I was not clear enough: no response is required for Reviewer 2,
>>>> as they have endorsed the manuscript. A response to Reviewer 1 is overdue,
>>>> however; see the email below.
>>>>
>>>> Frontiers in Systems Biology Editorial Office has sent you a message.
>>>> Please click 'Reply' to send a direct response
>>>>
>>>> This is to remind you that your response to the reviewer 1 is overdue
>>>> in the interactive review forum of your manuscript 822606 under
>>>> consideration for publication in Frontiers in Systems Biology, section
>>>> Multiscale Mechanistic Modeling.
>>>> To access the review forum, click:
>>>>
>>>> http://www.frontiersin.org/Review/EnterReviewForum.aspx?activationno=5f3876ba-0de4-47c1-8dff-1c2bcb5c6b11&retab=1
>>>> <http://links.email.frontiersin.org/ls/click?upn=99YMbkkQ8WNJiHLectmHSn5XnTmYe57aPeiziTpYNtyfClFbQqFlwPq2nUMZBzQ7srQvkLkPbc-2F-2BZbZ-2BczlOsMnoBdURYE7vBE6wrLrCJR4-2BCXMD-2FyylaGS-2FaxCSwEXxntm7RnNdDdmDL5k3W2WQo0ie3jVFLbWE6g0wuBJBzfQ-3DOLrv_B-2Fy9jbSVGoT8qhSwbw0AFtm6DlDsGMQJjIadnFC13-2FJyATqJzBD73VYfuDTc-2F-2BJaS4kVj6eazmIcCMUijXaF7cwmpIPfXo62oyUmieGoN2C1Y5Or-2Fqw1mDFet-2FbgSMXcJvyRgL8VJvJK-2Bw-2BULW-2FawzD0QPdwLV1hkLM4-2F9kG-2BJqcbrOIV4DgfEtFEs9z2qt9wTumRZRHdyMGfQyetT9cia1k0sDolZ7ukB5Ik6E4DTmj3yHv4voIEhFHR5lhm-2Bbs>
>>>>
>>>> Should you require a deadline extension for your resubmission, please
>>>> do let us know by selecting "Request Extension" in the review forum, if you
>>>> have not done so already. We encourage you to submit your revised
>>>> manuscript with tracked changes to facilitate the review.
>>>>
>>>> We look forward to receiving your revised manuscript. For any issues,
>>>> please immediately contact the Associate Editor as well as the Editorial
>>>> Office.
>>>>
>>>> Best regards,
>>>>
>>>> Your Frontiers in Systems Biology team
>>>>
>>>> Frontiers in Systems Biology Editorial Office
>>>> www.frontiersin.org
>>>> Avenue du Tribunal Federal 34
>>>> Lausanne, Switzerland | T 41(0)21 510 17 93
>>>>
>>>> For technical issues, please contact our IT Helpdesk -
>>>> support at frontiersin.org
>>>>
>>>>
>>>>
>>>>
>>>> Marcella Torres, Ph.D.
>>>> Director of Mathematical Studies
>>>> University of Richmond
>>>> Jepson Hall Room 212
>>>> 221 Richmond Way
>>>> Richmond, VA 23173
>>>> (804) 289-8081
>>>> Pronouns: she/her
>>>>
>>>> ------------------------------
>>>> *From:* Jacob Barhak <jacob.barhak at gmail.com>
>>>> *Sent:* Thursday, January 13, 2022 8:14 PM
>>>> *To:* Torres, Marcella <mtorres at richmond.edu>
>>>> *Cc:* Rahuman Sheriff <sheriff at ebi.ac.uk>;
>>>> vp-reproduce-subgroup at lists.simtk.org <
>>>> vp-reproduce-subgroup at lists.simtk.org>;
>>>> vp-integration-subgroup at lists.simtk.org <
>>>> vp-integration-subgroup at lists.simtk.org>; John Gennari <gennari at uw.edu>;
>>>> Faeder, James R <faeder at pitt.edu>
>>>> *Subject:* Re: [Vp-reproduce-subgroup] [Vp-integration-subgroup] Paper
>>>> review and revision check list
>>>>
>>>> *External Email:* Use caution in opening links, attachments, and
>>>> buying gift cards.
>>>>
>>>> Thanks Marcella, Thanks Gilberto,
>>>>
>>>> Since the second reviewer did not respond, I think we have more time -
>>>> we did respond in time however.
>>>>
>>>> The figures were provided by Alex and Sheriff - they seem pretty good
>>>> resolution to me, yet if we want increased resolution, they will have to
>>>> generate those again from the same software that generated thm originally -
>>>> Basically the images have to be at least 300 dpi and export it as pdf and
>>>> extract it as tiff or eps - not jpg.
>>>>
>>>> Section 2.2.2 in the instruction provides information about the image
>>>> quality:
>>>> https://www.frontiersin.org/about/author-guidelines
>>>>
>>>> Hopefully we can have those ready.
>>>>
>>>> However, I do ask that all authors be ready to approve the paper
>>>> assuming the other reviewer will be positive - we worked on thai for a year
>>>> and there were many discussions - before this paper gets published I want
>>>> all authors to be good with the final outcome. We will do this formally
>>>> like we did before - yet I want everyone to be ready to avoid any delays.
>>>> Hopefully it will take us less than 3 weeks this time.
>>>>
>>>> Jacob
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jan 13, 2022 at 1:05 PM Torres, Marcella <mtorres at richmond.edu>
>>>> wrote:
>>>>
>>>> Hi again all,
>>>>
>>>> On the 7th I shared comments from Reviewer 1 that we are now late
>>>> responding to. Is it possible to provide high resolution figures, or no
>>>> (see below)?
>>>>
>>>> Reviewer 2 has endorsed the manuscript.
>>>>
>>>>
>>>> Reviewer 1 comment:
>>>> *Reviewer 1* | 07 Jan 2022 | 14:36
>>>> #3
>>>>
>>>> I would like to thank the authors for having fully addressed my
>>>> comments. I'm satisfied about the quality the manuscript has been reached
>>>> in this latest version. Finally, I would only suggest the authors to
>>>> provide an high quality resolution of both new figures. Thank you.
>>>>
>>>>
>>>> Marcella Torres, Ph.D.
>>>> Director of Mathematical Studies
>>>> University of Richmond
>>>> Jepson Hall Room 212
>>>> 221 Richmond Way
>>>> Richmond, VA 23173
>>>> (804) 289-8081
>>>> Pronouns: she/her
>>>>
>>>> ------------------------------
>>>> *From:* Vp-reproduce-subgroup <
>>>> vp-reproduce-subgroup-bounces at lists.simtk.org> on behalf of Rahuman
>>>> Sheriff <sheriff at ebi.ac.uk>
>>>> *Sent:* Monday, January 10, 2022 3:09 PM
>>>> *To:* Jacob Barhak <jacob.barhak at gmail.com>
>>>> *Cc:* vp-reproduce-subgroup at lists.simtk.org <
>>>> vp-reproduce-subgroup at lists.simtk.org>;
>>>> vp-integration-subgroup at lists.simtk.org <
>>>> vp-integration-subgroup at lists.simtk.org>; John Gennari <gennari at uw.edu>;
>>>> Faeder, James R <faeder at pitt.edu>
>>>> *Subject:* Re: [Vp-reproduce-subgroup] [Vp-integration-subgroup] Paper
>>>> review and revision check list
>>>>
>>>> *External Email:* Use caution in opening links, attachments, and
>>>> buying gift cards.
>>>>
>>>> Thanks Jakob for submitting the revision and others for the support.
>>>> Thanks William for editing my response.
>>>>
>>>> Best
>>>> Sheriff
>>>>
>>>>
>>>> On 7 Jan 2022, at 04:52, Jacob Barhak <jacob.barhak at gmail.com> wrote:
>>>>
>>>> Thanks Hana, Thanks William, and thanks to Gilberto, Alex, and Sheriff,
>>>>
>>>> With your help and some effort, the revised paper was resubmitted in
>>>> time - just now.
>>>>
>>>> William, please keep on reminding me about your second affiliation - I
>>>> kept a comment as a reminder to do this - the submission system did
>>>> not allow making any changes in Author information - not sure why - if this
>>>> gets accepted we should ask the editor to add your affiliation manually.
>>>>
>>>> I urge all other authors to look at the changes we made and our
>>>> response. We will have to approve the manuscript with everyone before it
>>>> gets published - hopefully we will convince the reviewers this review round.
>>>>
>>>> Jacob
>>>>
>>>> On Thu, Jan 6, 2022 at 3:51 PM Dobrovolny, Hana <h.dobrovolny at tcu.edu>
>>>> wrote:
>>>>
>>>> I've read through the paper and fixed a few grammar mistakes.
>>>>
>>>> Hana
>>>>
>>>>
>>>> *******************************************************
>>>> Dr. Hana Dobrovolny
>>>> Associate Professor of Biophysics
>>>> Texas Christian University
>>>> TCU Box 298840
>>>> Fort Worth, TX 76129
>>>>
>>>> phone: (817) 257-6379 fax: (817) 257-7742
>>>> email: h.dobrovolny at tcu.edu
>>>> *******************************************************
>>>>
>>>>
>>>> ------------------------------
>>>> *From:* Vp-integration-subgroup <
>>>> vp-integration-subgroup-bounces at lists.simtk.org> on behalf of Jacob
>>>> Barhak <jacob.barhak at gmail.com>
>>>> *Sent:* January 5, 2022 11:48 PM
>>>> *To:* William Waites; James Glazier
>>>> *Cc:* vp-reproduce-subgroup at lists.simtk.org;
>>>> vp-integration-subgroup at lists.simtk.org; John Gennari; Faeder, James R
>>>> *Subject:* Re: [Vp-integration-subgroup] [Vp-reproduce-subgroup] Paper
>>>> review and revision check list
>>>>
>>>>
>>>> *[EXTERNAL EMAIL WARNING]* DO NOT CLICK LINKS or open attachments
>>>> unless you recognize the sender and know the content is safe.
>>>> Greetings paper contributors,
>>>>
>>>> Gilberto has addressed the final point and added text in the paper and
>>>> review, thus concluding our response.
>>>>
>>>> I went over the paper from start to finish again and cleaned it up.
>>>>
>>>> There are two last comments in the paper addressed to William Waites
>>>> and James Glazier - if you can quickly check those, it would help.
>>>>
>>>> If someone else wants to read the paper once more, please do this today
>>>> - otherwise I will convert the paper and our response to reviewers and
>>>> submit it to the Journal portal.
>>>>
>>>> If anyone has any objections, now is the time to raise them so we can
>>>> withdraw rather than submit.
>>>>
>>>> Here is the link to the revised paper:
>>>>
>>>> https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit?usp=sharing
>>>> <https://urldefense.com/v3/__https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit?usp=sharing__;%21%21K6Z8K8YTIA%21Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0hvK3OmJ8$>
>>>>
>>>> Here is the link to response for reviewers:
>>>>
>>>> https://docs.google.com/document/d/1bO1CDbMMlyyweYayP9rGohPIfgvIhfe14sFchtJMJ-s/edit?usp=sharing
>>>> <https://urldefense.com/v3/__https://docs.google.com/document/d/1bO1CDbMMlyyweYayP9rGohPIfgvIhfe14sFchtJMJ-s/edit?usp=sharing__;%21%21K6Z8K8YTIA%21Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0haRkeB0o$>
>>>>
>>>>
>>>> Hopefully there will be no objections and can submit this tomorrow.
>>>>
>>>> Jacob
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Jan 4, 2022 at 11:30 PM Jacob Barhak <jacob.barhak at gmail.com>
>>>> wrote:
>>>>
>>>> Hi Alex, Hi William, Hi Gilberto, and all other contributors.
>>>>
>>>> Alex, William and Gilberto are mentioned since we are the most active
>>>> contributors.
>>>>
>>>> We have 2 days to finish edits.
>>>>
>>>> William - I attempted to add your suggestions to the paper
>>>> regarding issue 11. You are free to revise further.
>>>>
>>>> Gilberto - I merged some of your texts, yet it seems you are the only
>>>> one I saw interested in resolving issue 12 where the reviewer asks to
>>>> remove the "Model Application and Implementation Barriers" section. Please
>>>> suggest your corrections as soon as possible. If you cannot do this by
>>>> tomorrow I will do my best to add a response based on your idea. Yet I
>>>> really hope you can do this before me.
>>>>
>>>> Please finalize any edits you have - and if anyone has any objection to
>>>> the text, please raise it within one day since after tomorrow I will start
>>>> closing down all issues.
>>>>
>>>> I would ask for a volunteer that will go over the paper once more
>>>> after its finalized and check for typos and grammar - we do not want the
>>>> paper to return again due to this issue. Time is tight, so please volunteer
>>>> fast.
>>>>
>>>> Hopefully we can get it done in time.
>>>>
>>>> Jacob
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Jan 4, 2022 at 7:25 AM Alexander Kulesza <
>>>> alexander.kulesza at novadiscovery.com> wrote:
>>>>
>>>> Dear Jacob, dear all,
>>>> thanks for endorsing my figure. I'll gladly add this. Please find
>>>> enclosed the .png.
>>>>
>>>> What you say makes sense and I agree that exaggerating changes is not
>>>> speeding up approval. Please still consider it as a tool to perform "major"
>>>> change (ordering of sections is a major change of a paper in my opinion)
>>>> without changing too much the sections as such. But I agree that this could
>>>> be tried out should the reviewers insist.
>>>>
>>>> I therefore reverted the table change (disconnecting ist from Figure 1)
>>>> and suggest the following Figure caption:
>>>>
>>>> "Figure 2: Sketch of difficulties persistent that impede
>>>> reproducibility, credibility, utility and integration of models, especially
>>>> in computational biology. An assignment of these difficulties to the
>>>> four different concepts interpreting them as hurdles is attempted. We would
>>>> like to point out that this sequential assessment model indicated in the
>>>> graph is only one of the possibilities a modeler would assess the
>>>> suitability of available models. In the rest of the paper we therefore
>>>> address each difficulty separately. "
>>>>
>>>> With that sentence we can discuss the difficulties in any order.
>>>>
>>>> Best
>>>> Alexander
>>>>
>>>> On Tue, 4 Jan 2022 at 13:24, Jacob Barhak <jacob.barhak at gmail.com>
>>>> wrote:
>>>>
>>>> Yes Alex,
>>>>
>>>> Your image is much better than mine, please use it instead in the paper
>>>> and send a link to the image itself in an acceptable image format so I can
>>>> upload it to the submission system later on.
>>>>
>>>> As for changing the table and order, I am not sure. Although it may be
>>>> possible. We need to think about it. Yet we really are crunched in time, so
>>>> I am leaning to say no.
>>>>
>>>> First, because some of the difficulties span more than one category,
>>>> for example unit standardization affects integration as well as
>>>> reproducibility, only it appears first at reproducibility, therefore it was
>>>> placed there in the figure.
>>>>
>>>> Second, if you look at the text there are some transition sentences
>>>> among sections, that connect them, so swapping those around may just mess
>>>> things. I am not sure it if will also mess up acronym definitions on the
>>>> first appearance as well, so shuffling order is not something's I would do
>>>> unless necessary. And it was not requested by reviewers anyway.
>>>>
>>>> I am also trying to minimize changes and focus only on what was
>>>> requested rather than change a lot more. Otherwise we may never finish
>>>> changing things since each one of the 18 of us will have a one last item
>>>> they MUST have corrected. I personally did not add some recent publications
>>>> I had, although I added one publication James asked to add in the previous
>>>> approval round and corrected one that Sheriff mentioned. Yet I really want
>>>> to avoid opening the paper to significant changes.
>>>>
>>>> If you start opening corrections beyond what was requested, this will
>>>> drag on forever since we do have a large list of contributors and each one
>>>> of us will generate a new idea. So please focus on what was asked unless
>>>> absolutely necessary.
>>>>
>>>> So I suggest not changing the table, and changing only the figure and
>>>> keeping order intact. I hope I am projecting the sense of urgency - we had
>>>> the paper lying around for a couple of weeks during the holiday without
>>>> much progress, I ask not to escalate things just because a deadline is
>>>> looming.
>>>>
>>>> If you have extra time on your hands in the next couple of days, I
>>>> suggest you focus on addressing the last two points in the response that we
>>>> have not resolved yet. I know Gilberto wanted to fix one of them, yet you
>>>> are welcome to do it. Those are things requested by the review
>>>>
>>>> Hopefully my arguments make sense.
>>>>
>>>> Jacob
>>>>
>>>> On Tue, Jan 4, 2022, 06:45 Alexander Kulesza <
>>>> alexander.kulesza at novadiscovery.com> wrote:
>>>>
>>>> Dear all,
>>>>
>>>> I have thought about the additional Figure requested by the reviewer.
>>>> Jacob your Figure nicely shows the assignment of the difficulties we detail
>>>> in our central table with the 4 big concepts.
>>>>
>>>> I feel however that the text-heavy figure is a lot o redundancy with
>>>> the rest of the elements (existing figure & table). Therefore I wanted to
>>>> throw another suggestion forward (inspired by systematic reviews narrowing
>>>> down eligible studies/data).
>>>>
>>>> @all please notice that the assignment I would do is a bit different to
>>>> what is currently done in the paper. The order of sections would have to be
>>>> altered. Not sure if that is acceptable.
>>>> <image.png>
>>>> The table with Difficulties (hurdles) and solutions would then be
>>>>
>>>> Difficulty
>>>> Potential Solutions
>>>>
>>>> Reproducibility
>>>> Models are written in different languages
>>>> Common transport specifications such as SBML or CellML, and proper
>>>> documentation and annotation
>>>> Models are hard to locate
>>>> Archive web sites such as: BioModels, SimTK, IMAGWiki, and the future
>>>> modeleXchange
>>>> Lack of common platforms for executing models and simulations
>>>> Platforms such as BioSimulators, and runBioSimulations
>>>> Unit standardization
>>>> Standardization efforts, and machine learning solutions such as
>>>> ClinicalUnitMapping.com
>>>> Credibility
>>>> Models have built-in barriers to Evaluating model credibility
>>>> Better modeling practices, documentation, and tests.
>>>> Data availability and measurement definitions
>>>> Models that merge human interpretation, and newer measurement devices
>>>>
>>>> Missing annotations in models
>>>> Adoption of policies such as those COMBINE suggests
>>>> Utility
>>>> Models are not consistently licensed to allow for reuse
>>>> Abandoning some old school open source licenses and promoting licenses
>>>> that release to public domain
>>>> Different scales and modeling paradigms
>>>> Standardization effort and centralization tools
>>>> Stochastic modeling difficulties
>>>> Development of tools that guarantee repeatability and standards to
>>>> address stochastic simulations
>>>> Integration
>>>> Model application and implementation barriers
>>>> Education of modelers, users, and the public
>>>> Modeling requires adaptation towards integration
>>>> Tools for composing models such as SBML-Comp, and SemGen
>>>>
>>>> Following this assignment.
>>>>
>>>> Please let me know what you think. I have added this suggestion in
>>>> track changes mode to the revision draft.
>>>>
>>>> Best
>>>> Alexander
>>>>
>>>>
>>>> On Mon, 3 Jan 2022 at 07:27, Jacob Barhak <jacob.barhak at gmail.com>
>>>> wrote:
>>>>
>>>> Thanks Alex,
>>>>
>>>> You did great by adding the new material that the reviewers actually
>>>> asked for. This is great - I found myself adding the new FDA document to
>>>> find out that you already added it - I just voice read it ona long drive
>>>> and figured it fits well - it seems you were faster - great.
>>>>
>>>> I merged your edits and I am ok with adding your response to the
>>>> reviewers - I only had an issue with one example paragraph that you deleted
>>>> - I left it with a comment - please check if it is possible to keep this
>>>> example somewhere in the text.
>>>>
>>>> I will assemble a letter that includes all those responses point by
>>>> point and will try to use your text almost verbatim if possible. You really
>>>> bring important knowledge with you.
>>>>
>>>> It seems most points have been answered to some level - even a figure
>>>> was created - although I wish I was a better artist.
>>>>
>>>> I will try to assemble the response letter for everyone within the next
>>>> few days. And unless there will be objections I will try to pass it by
>>>> reviewers with a new version. If the new version is accepted, we will do a
>>>> final approval round before publication.
>>>>
>>>> Many thanks to all those who contributed.
>>>>
>>>> Jacob
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Sun, Jan 2, 2022 at 7:27 AM Alexander Kulesza <
>>>> alexander.kulesza at novadiscovery.com> wrote:
>>>>
>>>> Dear all,
>>>>
>>>> Happy new year to everyone. I hope that you are all well and safe and
>>>> have spent a nice holiday season.
>>>>
>>>> I have tried to address
>>>>
>>>> *Reviewer 2 *
>>>>
>>>>
>>>> *Moreover, when the authors mention ASME V&V40 procedure, it could be
>>>> useful to spend few words about the model credibility pipeline drawn for
>>>> predictive models in biomedicine, such as the one suggested in "Credibility
>>>> of in Silico Trial Technologies-A Theoretical Framing", Viceconti, M.,
>>>> Juarez, M.A., Curreli, C., ...Russo, G., Pappalardo, F., IEEE Journal of
>>>> Biomedical and Health Informatics, 2020, 24(1), pp. 4–13, 8884189. For this
>>>> specific aspect, the authors should talk about "specific question of
>>>> interest" and not "specific purpose" in the paragraph called "Credibility
>>>> of Models" to follow in such a way a standardised language. *
>>>>
>>>> I have worked on the followng document in "track changes" mode:
>>>>
>>>> https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit
>>>> <https://urldefense.com/v3/__https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit__;%21%21K6Z8K8YTIA%21Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0hrtFQ-XI$>
>>>> Actually, I ended up in changing rather drastically the section. I
>>>> think that all ideas and all text is conserved, but I would very much like
>>>> to encourage you to read, comment and challenge this revision. Please tell
>>>> me if that goes to far. No problem to revert to an earlier version if
>>>> necessary.
>>>>
>>>> I suggest the following formulation in the rebuttal letter:
>>>>
>>>> When trying to better describe the "credibility pipeline" (CoU
>>>> definition, verification, validation, uncertainty quantification, e.g. as
>>>> described in Viceconti et al. 2019) or others, we noticed that the section
>>>> about credibility could be restructured in order to put the reader (not
>>>> necessarily familiar with regulatory assessment of models) in the position
>>>> to follow the argumentation.
>>>>
>>>> We now roughly follow the following flow of writing:
>>>>
>>>> - Every model needs a purpose and credibility is tied to that purpose
>>>> as well as repeatability and reproducibility
>>>> - Credibility is essential for models that have impact on regulated
>>>> areas or people's lives which is why regulatory authorities issue guidance
>>>> - The probably most advanced guidance ASME V&V40 suggests a risk-based
>>>> approach and pipeline to establish credibility of a model. It is
>>>> overarching and widely applicable
>>>> - Regulators and modelers work together (for example in frame of the
>>>> model informed drug development program MIDD) in order to better
>>>> understand, better apply and better uptake different kinds of models in
>>>> regulatory applications
>>>> - More work is needed to harmonize, stay up to date and to be more
>>>> inclusive
>>>>
>>>> We hope that with the rather extensive change of the wording and
>>>> additional passages as well as citations (see below) we could address the
>>>> concern of reviewer 2 (and the general remarks of reviewer 1).
>>>>
>>>> " In 2018 the American Society of Mechanical Engineers (ASME) issued an
>>>> important guidance ASME V&V 40 (ASME, 2018) of how to assess credibility of
>>>> computational models of medical devices through verification and validation
>>>> (V&V). The guideline is centered around the definition of the context of
>>>> use (CoU) of the model which is formulated based on the questions of
>>>> interest the model will answer. The CoU is then analyzed in terms of the
>>>> “model risk” - being the influence the model exerts on a decision and the
>>>> potential consequences these decisions might incur. Commensurate with this
>>>> model risk the modeler suggests establish the credibility goals, perform
>>>> verification validation and uncertainty quantification actions and then
>>>> assess the outcome of this exercise in order to allow judging the
>>>> acceptability of the model CoU. Key to this guidance is its overarching
>>>> nature which also allows adoption in other (e.g. drug development) fields
>>>> irrespective of the model type (Kuemmel 2020, Viceconti 2019)."
>>>>
>>>> We also feel that it is essential to underline the cross-discipline
>>>> viewpoint of the ASME V&V40 which is further elaborated by the cited paper
>>>> by formalizing the verification, validation and uncertainty quantification
>>>> VVUQ pipeline across model types.
>>>>
>>>> We therefore have added the following statement
>>>> " In the paper by (Viceconti 2019) the verification, validation and
>>>> uncertainty quantification (VVUQ) pipeline is streamlined to different
>>>> types of models. It is, perhaps, the closest to score credibility across
>>>> model types from mechanistic physics driven models to machine learning
>>>> models. However, it is still short of including very recent developments
>>>> such as ensemble models, although it touches upon the topic."
>>>>
>>>> We additionally strengthened our argumentation by citing additional
>>>> literature
>>>>
>>>> Bai, JPF, Earp, JC, Florian, J, et al. Quantitative systems
>>>> pharmacology: Landscape analysis of regulatory submissions to the US Food
>>>> and Drug Administration. CPT Pharmacometrics Syst Pharmacol. 2021; 10:
>>>> 1479– 1484. doi.org:10.1002/psp4.12709
>>>>
>>>> FDA (2021b). Assessing the Credibility of Computational Modeling and
>>>> Simulation in Medical Device Submissions. FDA
>>>> Available at:
>>>> https://www.fda.gov/regulatory-information/search-fda-guidance-documents/assessing-credibility-computational-modeling-and-simulation-medical-device-submissions
>>>> <https://urldefense.com/v3/__https://www.fda.gov/regulatory-information/search-fda-guidance-documents/assessing-credibility-computational-modeling-and-simulation-medical-device-submissions__;%21%21K6Z8K8YTIA%21Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0harP7IHc$>
>>>> [Accessed January 2, 2022]
>>>>
>>>> Galluppi, G.R., Brar, S., Caro, L., Chen, Y., Frey, N., Grimm, H.P.,
>>>> Rudd, D.J., Li, C.-C., Magee, M., Mukherjee, A., Nagao, L., Purohit, V.S.,
>>>> Roy, A., Salem, A.H., Sinha, V., Suleiman, A.A., Taskar, K.S., Upreti,
>>>> V.V., Weber, B. and Cook, J. (2021), Industrial Perspective on the Benefits
>>>> Realized From the FDA’s Model-Informed Drug Development Paired Meeting
>>>> Pilot Program. Clin Pharmacol Ther, 110: 1172-1175.
>>>> doi:10.1002/cpt.2265
>>>>
>>>> Kuemmel, C., Yang, Y., Zhang, X., Florian, J., Zhu, H., Tegenge, M.,
>>>> Huang, S.-M., Wang, Y., Morrison, T. and Zineh, I. (2020), Consideration
>>>> of a Credibility Assessment Framework in Model-Informed Drug Development:
>>>> Potential Application to Physiologically-Based Pharmacokinetic Modeling and
>>>> Simulation. CPT Pharmacometrics Syst. Pharmacol., 9: 21-28. doi.org
>>>> :10.1002/psp4.12479
>>>>
>>>> Musuamba, FT, Skottheim Rusten, I, Lesage, R, et al. Scientific and
>>>> regulatory evaluation of mechanistic in silico drug and disease models in
>>>> drug development: Building model credibility. CPT Pharmacometrics Syst
>>>> Pharmacol. 2021; 10: 804– 825. doi:10.1002/psp4.12669
>>>>
>>>> Viceconti M, Emili L, Afshari P, et al. Possible Contexts of Use for In
>>>> Silico Trials Methodologies: A Consensus-Based Review. IEEE J Biomed
>>>> Health Inform. 2021;25(10):3977-3982. doi:10.1109/JBHI.2021.3090469
>>>>
>>>>
>>>> All the best
>>>> Alexander
>>>>
>>>> On Sun, 19 Dec 2021 at 03:10, Jacob Barhak <jacob.barhak at gmail.com>
>>>> wrote:
>>>>
>>>>
>>>> Greetings to the paper contributors,
>>>>
>>>> You may have seen my other message where I posted the reviews to our
>>>> paper. I am sending those again below to start a new thread discussing
>>>> possible revisions.
>>>>
>>>> Here is a checklist for revisions required:
>>>>
>>>>
>>>> 1. We should discuss the references that reviewer 1 raised and add
>>>> them as reference
>>>> 2. Reviewer 1 asked for more discussion around V&V40 - I
>>>> believe Alex added the text there originally - Alex do you think you can
>>>> address the request of the reviewer?
>>>> 3. Reviewer 1 asked to focus on COVID-19 at the title - I am not
>>>> sure we wish to limit our scope since many of our ideas are applicable far
>>>> beyond COVID-19 - I leave it up for discussion in the group on how to
>>>> address this request by the reviewer
>>>> 4. Reviewer 1 asked for additional graphics - I am unsure how to
>>>> address this beyond ur current diagram - ideas will help
>>>> 5. We need to check the "Contribution to the field" section for
>>>> grammar and spelling - if someone can contribute more elegant text - now is
>>>> the time - I looked over it now and found no issues - yet I may have missed
>>>> something - this section may have been missed since it was added last and
>>>> perhaps vetted less than other sections. Also Reviewer 2 asked for grammar
>>>> corrections, so it is worthwhile proof reading the paper as a whole.
>>>> 6. Reviewer 2 asks for major rewrite to emphasize why the problems
>>>> are there - I think we should explain that the core of the paper is the
>>>> table and perhaps emphasize it in the paper beyond what the current text -
>>>> we also should reply to the reviewer and explain that the paper is composed
>>>> of contributions from a large group and we made an effort to include every
>>>> voice in the choir - each of those voices is important and needs to be
>>>> preserved - hopefully it will convince the reviewer that change we will add
>>>> will be sufficient
>>>> 7. Reviewer 2 asks to revise the introduction - I believe some
>>>> changes are possible - yet the introduction includes contributions from
>>>> many authors - at least 10 - and I fear losing something important someone
>>>> contributed - if someone has an idea on how to address this reviewer
>>>> without a painful transformation, please reply to this message.
>>>> 8. Reviewer 2 wants additional discussion around the
>>>> "reproducibility crisis" section and asks a valid question about
>>>> expectations we should address - "“Computational biomedical modeling… was
>>>> expected to be less affected by the reproducibility crisis.” Is this true,
>>>> and why would it be so? " - My answer to the reviewer is that unlike
>>>> biological processes that have random nature and experiments
>>>> would not repeat if repeated, while computer software should be
>>>> deterministic and it should be repeatable if designed well. Unfortunately
>>>> we are not experiencing this promise - yet I believe the reviewer wants
>>>> more discussion beyond this section so I am happy to discuss this.
>>>> 9. Reviewer 2- writes: "Models are Hard to Locate: Are the authors
>>>> suggesting that entire simulation workflows, from model construction to
>>>> analysis, should be publicly available? At what point does one consider
>>>> intellectual property? Do the authors advocate for such extensive
>>>> publishing for all models, or only ones that are intended to be widely
>>>> re-used? - those are important points we need to discuss - we need to
>>>> better explain the difficulties we are having when creating models and the
>>>> reviewer is absolutely correct about expanding the discussion to IP - I
>>>> suggest we create a thread for this discussion and reference it - I
>>>> suggest it we merged with the licensing issue that I will address later -
>>>> there is a strong connection there - the reviewer was very observant and
>>>> sees the bigger problem. However,we need to eventually distill our
>>>> discussion to recommendations that will inline.
>>>> 10. Reviewer 2 asks that we fix the unit standardization section -
>>>> I believe Hana and myself were the largest contributors there - Hana - I
>>>> will start a discussion on that topic in a separate email where we can
>>>> publicly discuss how to fix this - others will be welcome to contribute.
>>>> 11. Reviewer 2 asks that we better handle the section on Data
>>>> availability and measurement definitions: in think we need to emphasize
>>>> solutions and separate it from issues that may not be solvable. Ideas are
>>>> welcome.
>>>> 12. Reviewer 2 asks good questions with regards to licensing
>>>> following our text- I personally have good answers to the reviewer and
>>>> William and I had some discussion on the topic in this list I suggest we
>>>> expand this discussion in a separate thread - hopefully William and perhaps
>>>> others will join the discussion. This discussion should also address the IP
>>>> issues raised by the reviewer for the "models are hard to locate section."
>>>> 13. Reviewer 2 asks to handle the Model Application and
>>>> Implementation Barriers section. We should decide what to do there, the
>>>> section may need expansion since the ideas there are solid, yet the section
>>>> is short so perhaps enhancing it makes better sense. I am open to
>>>> suggestions.
>>>>
>>>>
>>>> Those are the items I located and my suggestions. It seems we need
>>>> attention from Alex, Hana, myself and william. However, anyone on the list
>>>> is welcome to participate and suggest changes.
>>>>
>>>> I will start the discussion threads on specific topics. Hopefully we
>>>> can get it done quickly.
>>>>
>>>> Jacob
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ########### Original Reviews #############
>>>>
>>>> There are 2 reviews - both require major changes. I am copying the
>>>> relevant text below. If more appear, I will let you know, yet I only got
>>>> this message today although the reviews are dated a few days ago.
>>>>
>>>> Reviewer 1:
>>>> Recommendation for the Editor: Substantial revision is required
>>>>
>>>> Please list your revision requests for the authors and provide your
>>>> detailed comments, including highlighting limitations and strengths of the
>>>> review. If you have additional comments based on Q2 and Q3 you can add them
>>>> as well.
>>>>
>>>> Karr and co-authors made an interesting and exaustive point about the
>>>> reproducibility crisis that leads to inability to reuse and integrate
>>>> models, especially about COVID-19 disease.
>>>> Within the manuscript some typos and missing information are present.
>>>> I'd suggest the authors to revise the entire manuscript especially in
>>>> terms of the state of the art, revising and updating the most relevant
>>>> examples of computational models dealing with COVID-19 and in general about
>>>> some semi-standardised proposals about the pipeline to follow for the
>>>> verification and validation of model credibility. In particular, the
>>>> authors failed to mention and cite some major results on in silico modeling
>>>> about COVID-19 up to now. See for example:
>>>> a."In silico trial to test COVID-19 candidate vaccines: a case study
>>>> with UISS platform", Russo, G., Pennisi, M., Fichera, E., ...Viceconti, M.,
>>>> Pappalardo, F., BMC Bioinformatics, 2020, 21, 527.
>>>> b. Russo G, Di Salvatore V, Sgroi G, Parasiliti Palumbo GA, Reche PA,
>>>> Pappalardo F. "A multi-step and multi-scale bioinformatic protocol to
>>>> investigate potential SARS-CoV-2 vaccine targets" [published online ahead
>>>> of print, 2021 Oct 5]. Brief Bioinform. 2021;bbab403.
>>>> doi:10.1093/bib/bbab403.
>>>> Moreover, when the authors mention ASME V&V40 procedure, it could be
>>>> useful to spend few words about the model credibility pipeline drawn for
>>>> predictive models in biomedicine, such as the one suggested in "Credibility
>>>> of in Silico Trial Technologies-A Theoretical Framing", Viceconti, M.,
>>>> Juarez, M.A., Curreli, C., ...Russo, G., Pappalardo, F., IEEE Journal of
>>>> Biomedical and Health Informatics, 2020, 24(1), pp. 4–13, 8884189.
>>>> For this specific aspect, the authors should talk about "specific
>>>> question of interest" and not "specific purpose" in the paragraph called
>>>> "Credibility of Models" to follow in such a way a standardised language.
>>>> Moreover, the authors should refer to COVID-19 also in the title, with
>>>> a specific mention about the fact that the main topic of model integration
>>>> in computational biology will be discussed inside the COVID-19 context.
>>>> Furthermore, for the model key concepts such as
>>>> Reusability-Extensibility-Extractability-Portability the authors should
>>>> described and outlined through a graphical sketch or visual representation
>>>> that summarises these key point.
>>>> The authors should also fix some grammar and writing typos present in
>>>> the "Contribution to the field" section.
>>>>
>>>> a. Is the quality of the figures and tables satisfactory?
>>>> - No
>>>>
>>>> b. Does the reference list cover the relevant literature adequately and
>>>> in an unbiased manner?
>>>> - No
>>>>
>>>> c. Does this manuscript refer only to published data? (unpublished or
>>>> original data is not allowed for this article type)
>>>> - Yes
>>>>
>>>> d. Does the review include a balanced, comprehensive, and critical view
>>>> of the research area?
>>>> - Yes
>>>>
>>>>
>>>> Reviewer 2
>>>>
>>>> Please list your revision requests for the authors and provide your
>>>> detailed comments, including highlighting limitations and strengths of the
>>>> review. If you have additional comments based on Q2 and Q3 you can add them
>>>> as well.
>>>>
>>>>
>>>> The article often reads as a stream-of-consciousness account of the
>>>> discussions that took place but lacks a clear thesis or recommendations. It
>>>> is not clear what, if anything, the authors are advocating for. It is not
>>>> always clear why the issues being discussed are problematic, or that they
>>>> can be reasonably addressed. Some of the issues raised are indeed important
>>>> and should be discussed, but the paper lacks focus and does not tell a
>>>> cohesive story. I believe this manuscript requires a major re-write to be
>>>> suitable for publication. The authors should consider narrowing the scope
>>>> of the discussion and focusing on a cohesive set of recommendations or open
>>>> questions. More specifically, I make a few suggestions below:
>>>>
>>>> Major comments
>>>> 1. The introduction is long and repeats itself (e.g., “much less is
>>>> known about how viral infections spread throughout the body…” is repeated
>>>> verbatim). It is not clear from the introduction what the main goal of the
>>>> paper is or why the “reproducibility crisis” is truly a crisis. Why is the
>>>> discussion of composition and black/white box models relevant to the
>>>> introduction? Further, this section is subtitled “the promise of modeling”,
>>>> which does not seem to match the content.
>>>> 2. The Reproducibility Crisis: “Computational biomedical modeling… was
>>>> expected to be less affected by the reproducibility crisis.” Is this true,
>>>> and why would it be so? One would think that more complex models would
>>>> suffer more from a lack of reproducibility. It may be helpful to define
>>>> what exactly the “reproducibility crisis” refers to.
>>>> 3. Models are Hard to Locate: Are the authors suggesting that entire
>>>> simulation workflows, from model construction to analysis, should be
>>>> publicly available? At what point does one consider intellectual property?
>>>> Do the authors advocate for such extensive publishing for all models, or
>>>> only ones that are intended to be widely re-used?
>>>> 4. Unit standardization: The conversion from PFU or TCID50 to
>>>> individual virions is likely to differ across viruses – are the authors
>>>> focused on COVID here? Are the authors advocating for a standard conversion
>>>> factor? It is not clear what the purpose of this discussion is. As the
>>>> authors mention, different scales require different units. Even at a single
>>>> scale, different models may require different units for numerical reasons.
>>>> It is not clear what the authors are advocating for here.
>>>> 5. Data availability and measurement definitions: This section seems to
>>>> outline limitations of available data, but again makes no recommendations
>>>> or proposed solution to any of the issues raised. Is this the intention?
>>>> Most of the issues raised here reflect limitations of experimental science
>>>> or data privacy, which likely cannot be meaningfully addressed by the
>>>> modeling community.
>>>> 6. Models are Not Consistently Licensed…: Are the authors implying here
>>>> that all modeling work should be published with no rights reserved? Is it
>>>> reasonable to expect modelers to make their work freely usable by others
>>>> for profit? Is it reasonable for institutions to allow this? How much does
>>>> this really contribute to reproducibility and utility?
>>>> 7. Model Application and Implementation Barriers: This section seems
>>>> unnecessary and out of place.
>>>> 8. There are grammar and punctuation errors scattered throughout;
>>>> please edit carefully.
>>>>
>>>>
>>>> a. Is the quality of the figures and tables satisfactory?
>>>> - Not Applicable
>>>>
>>>> b. Does the reference list cover the relevant literature adequately and
>>>> in an unbiased manner?
>>>> - Yes
>>>>
>>>> c. Does this manuscript refer only to published data? (unpublished or
>>>> original data is not allowed for this article type)
>>>> - Yes
>>>>
>>>> d. Does the review include a balanced, comprehensive, and critical view
>>>> of the research area?
>>>> No answer given.
>>>>
>>>>
>>>> ========
>>>> _______________________________________________
>>>> Vp-reproduce-subgroup mailing list
>>>> Vp-reproduce-subgroup at lists.simtk.org
>>>> https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
>>>> <https://urldefense.com/v3/__https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup__;%21%21K6Z8K8YTIA%21Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0hhiovw3M$>
>>>>
>>>>
>>>>
>>>> --
>>>> Alexander Kulesza
>>>> Team leader
>>>> Modeling & simulation / Biomodeling
>>>> alexander.kulesza at novadiscovery.com
>>>> +33 7 82 92 44 62
>>>> nova
>>>> DISCOVERY
>>>> www.novadiscovery.com
>>>> <https://urldefense.com/v3/__http://www.novadiscovery.com/__;%21%21K6Z8K8YTIA%21Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0hwI_2O8g$>
>>>> 1 Place Verrazzano, 69009 Lyon
>>>> +33 9 72 53 13 01
>>>>
>>>> *This message is intended only for the personal and confidential use of
>>>> the designated recipient(s) named above. If you are not the intended
>>>> recipient of this message you are hereby notified that any review,
>>>> dissemination, distribution or copying of this message is strictly
>>>> prohibited. Email transmission cannot be guaranteed to be secure or
>>>> error-free. Therefore, we do not represent that this information is
>>>> complete or accurate and it should not be relied upon as such. All
>>>> information is subject to change without notice.*
>>>>
>>>>
>>>>
>>>> --
>>>> Alexander Kulesza
>>>> Team leader
>>>> Modeling & simulation / Biomodeling
>>>> alexander.kulesza at novadiscovery.com
>>>> +33 7 82 92 44 62
>>>> nova
>>>> DISCOVERY
>>>> www.novadiscovery.com
>>>> <https://urldefense.com/v3/__http://www.novadiscovery.com/__;%21%21K6Z8K8YTIA%21Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0hwI_2O8g$>
>>>> 1 Place Verrazzano, 69009 Lyon
>>>> +33 9 72 53 13 01
>>>>
>>>> *This message is intended only for the personal and confidential use of
>>>> the designated recipient(s) named above. If you are not the intended
>>>> recipient of this message you are hereby notified that any review,
>>>> dissemination, distribution or copying of this message is strictly
>>>> prohibited. Email transmission cannot be guaranteed to be secure or
>>>> error-free. Therefore, we do not represent that this information is
>>>> complete or accurate and it should not be relied upon as such. All
>>>> information is subject to change without notice.*
>>>>
>>>>
>>>>
>>>> --
>>>> Alexander Kulesza
>>>> Team leader
>>>> Modeling & simulation / Biomodeling
>>>> alexander.kulesza at novadiscovery.com
>>>> +33 7 82 92 44 62
>>>> nova
>>>> DISCOVERY
>>>> www.novadiscovery.com
>>>> <https://urldefense.com/v3/__http://www.novadiscovery.com/__;%21%21K6Z8K8YTIA%21Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0hwI_2O8g$>
>>>> 1 Place Verrazzano, 69009 Lyon
>>>> +33 9 72 53 13 01
>>>>
>>>> *This message is intended only for the personal and confidential use of
>>>> the designated recipient(s) named above. If you are not the intended
>>>> recipient of this message you are hereby notified that any review,
>>>> dissemination, distribution or copying of this message is strictly
>>>> prohibited. Email transmission cannot be guaranteed to be secure or
>>>> error-free. Therefore, we do not represent that this information is
>>>> complete or accurate and it should not be relied upon as such. All
>>>> information is subject to change without notice.*
>>>>
>>>> _______________________________________________
>>>> Vp-integration-subgroup mailing list
>>>> Vp-integration-subgroup at lists.simtk.org
>>>> https://lists.simtk.org/mailman/listinfo/vp-integration-subgroup
>>>>
>>>>
>>>> _______________________________________________
>>> Vp-integration-subgroup mailing list
>>> Vp-integration-subgroup at lists.simtk.org
>>> https://lists.simtk.org/mailman/listinfo/vp-integration-subgroup
>>>
>>
>>
>> --
>> -- ------------------
>> Yaling Liu, Professor
>> Department of Bioengineering, Director of Graduate Studies
>> Department of Mechanical Engineering and Mechanics
>> Lehigh University
>> www.lehigh.edu/~yal310 <http://www.lehigh.edu/%7Eyal310>
>> _______________________________________________
>> Vp-reproduce-subgroup mailing list
>> Vp-reproduce-subgroup at lists.simtk.org
>> https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
>>
>
>
> --
> James A. Glazier, PhD
> Professor of Intelligent Systems Engineering, Adjunct Professor of Physics
> Director Biocomplexity Institute
> Indiana University, Bloomington
> (812) 391-2159
> _______________________________________________
> Vp-reproduce-subgroup mailing list
> Vp-reproduce-subgroup at lists.simtk.org
> https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
> _______________________________________________
> Vp-reproduce-subgroup mailing list
> Vp-reproduce-subgroup at lists.simtk.org
> https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.simtk.org/pipermail/vp-reproduce-subgroup/attachments/20220114/880beeda/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 28718 bytes
Desc: not available
URL: <https://lists.simtk.org/pipermail/vp-reproduce-subgroup/attachments/20220114/880beeda/attachment-0001.png>
More information about the Vp-reproduce-subgroup
mailing list