[Vp-integration-subgroup] [Vp-reproduce-subgroup] Paper review and revision check list

[Gilberto Gonzalez-Parra]

The status has changed to
17 Jan 2022
Review of Review Editor 1 finalized.
13 Jan 2022
Editorial Office reminded you to respond to reviewer 1 and/or resubmit your
manuscript in the discussion forum.
10 Jan 2022
Review of Review Editor 2 finalized.
So both reviewers endorsed it.   BTW, approve.

***************************************************************************
Gilberto C. Gonzalez-Parra, Ph.D in Applied Mathematics.
Faculty of the Mathematics Department
New Mexico Tech, NM, USA.
****************************************************************************


On Mon, Jan 17, 2022 at 4:07 PM Faeder, James R <faeder at pitt.edu> wrote:

> Approve.
>
>
>
> Thanks Jacob and everyone else for all of your efforts on the revisions.
>
>
>
> Best regards,
>
>
>
> Jim
>
>
>
> *From: *Rahuman Sheriff <sheriff at ebi.ac.uk>
> *Date: *Monday, January 17, 2022 at 5:02 PM
> *To: *Jacob Barhak <jacob.barhak at gmail.com>
> *Cc: *vp-reproduce-subgroup at lists.simtk.org <
> vp-reproduce-subgroup at lists.simtk.org>,
> vp-integration-subgroup at lists.simtk.org <
> vp-integration-subgroup at lists.simtk.org>, John Gennari <gennari at uw.edu>,
> Faeder, James R <faeder at pitt.edu>, Alexander Kulesza <
> alexander.kulesza at novadiscovery.com>
> *Subject: *Re: [Vp-reproduce-subgroup] [Vp-integration-subgroup] Paper
> review and revision check list
>
> Dear Jacob,
>
> Please find the Figure 1 attached.
>
>
>
> It is not clear to me whether the manuscript has been accepted. If the
> reviewer1’s response is overdue, we need to wait. If the review1 doesn’t
> respond at all, the editor might assign the manuscript to another reviewer.
>
>
>
> Irrespective of that I approve.
>
>
>
> On a different note, if I remember correctly, Tomas Helikar also
> contributed to the ordering of the table which is used in the figure.
> probably he mad other contributions too in John Gennari’s version.
>
>
>
> Best regards
>
> Sheriff
>
>
>
>
>
>
>
> On 14 Jan 2022, at 13:19, Alexander Kulesza <
> alexander.kulesza at novadiscovery.com> wrote:
>
>
>
> Dear Jacob, dear all,
>
>
>
> APPROVE
>
>
>
> Attached, Figure 2 in HR in different formats.
>
>
>
> Best
>
> Alexander
>
>
>
> On Fri, 14 Jan 2022 at 07:38, Jacob Barhak <jacob.barhak at gmail.com> wrote:
>
> Thanks Marcella,
>
>
>
> In that case, it seems we have no reviewer objection and hopefully the
> editor will be positive as well . To speed up the process we need
> the following::
>
> 1. Sheriff - please send a high resolution image for Figure 1
>
> 2. Alex - Please send a high resolution image for Figure 2
>
> 3. Everyone  please review the manuscript and Reply to all in this email
> with one word - either APPROVE, DISAPPROVE, ABSTAIN
>
>
>
> Where:
>
>
>
> APPROVE means you are ok with the version we reached - please note that we
> do have some new elements there and your approval means you understand what
> it means - please read the paper well and understand the nuances of
> approval.
>
>
>
> DISAPPROVE- means that you no longer agree with the version we reached and
> we need to discuss the paper more and withdraw it - if you have some
> serious objection, it is the last chance to get the paper back to the
> drawing board
>
>
>
> ABSTAIN - You have no objection for the paper being published, yet you
> want your name withdrawn
>
>
>
> Also, if anyone else has any objections for this paper being published,
> please raise them now so we can discuss them.
>
>
>
> I call again to John Gennarri to join us as an author - he did
> substantial work and many of his contributions are in the final version -
> including the order in figure 2. I can trace back many of his contributions
> and I think he should get credit for it - hopefully my last call will be
> accepted and he will rejoin.
>
>
>
> William - I did not forget your second affiliation - I just could not
> change it in the system - I will write to the editor about that when we get
> the final acceptance notice.
>
>
>
> I look forward to seeing your final decisions - hopefully it will be fast
> this time.
>
>
>
>                Jacob
>
>
>
>
>
> On Thu, Jan 13, 2022 at 8:50 PM Torres, Marcella <mtorres at richmond.edu>
> wrote:
>
> Sorry, I was not clear enough: no response is required for Reviewer 2, as
> they have endorsed the manuscript. A response to Reviewer 1 is overdue,
> however; see the email below.
>
>
>
> Frontiers in Systems Biology Editorial Office has sent you a message.
> Please click 'Reply' to send a direct response
>
> This is to remind you that your response to the reviewer 1 is overdue in
> the interactive review forum of your manuscript 822606 under consideration
> for publication in Frontiers in Systems Biology, section Multiscale
> Mechanistic Modeling.
> To access the review forum, click:
>
> http://www.frontiersin.org/Review/EnterReviewForum.aspx?activationno=5f3876ba-0de4-47c1-8dff-1c2bcb5c6b11&retab=1
> <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Flinks.email.frontiersin.org%2Fls%2Fclick%3Fupn%3D99YMbkkQ8WNJiHLectmHSn5XnTmYe57aPeiziTpYNtyfClFbQqFlwPq2nUMZBzQ7srQvkLkPbc-2F-2BZbZ-2BczlOsMnoBdURYE7vBE6wrLrCJR4-2BCXMD-2FyylaGS-2FaxCSwEXxntm7RnNdDdmDL5k3W2WQo0ie3jVFLbWE6g0wuBJBzfQ-3DOLrv_B-2Fy9jbSVGoT8qhSwbw0AFtm6DlDsGMQJjIadnFC13-2FJyATqJzBD73VYfuDTc-2F-2BJaS4kVj6eazmIcCMUijXaF7cwmpIPfXo62oyUmieGoN2C1Y5Or-2Fqw1mDFet-2FbgSMXcJvyRgL8VJvJK-2Bw-2BULW-2FawzD0QPdwLV1hkLM4-2F9kG-2BJqcbrOIV4DgfEtFEs9z2qt9wTumRZRHdyMGfQyetT9cia1k0sDolZ7ukB5Ik6E4DTmj3yHv4voIEhFHR5lhm-2Bbs&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394336406%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=7dVmtKt2hDq7kUvtb4aVCPr6wVbvscW%2FkpNJfRyCbxg%3D&reserved=0>
>
> Should you require a deadline extension for your resubmission, please do
> let us know by selecting "Request Extension" in the review forum, if you
> have not done so already. We encourage you to submit your revised
> manuscript with tracked changes to facilitate the review.
>
> We look forward to receiving your revised manuscript. For any issues,
> please immediately contact the Associate Editor as well as the Editorial
> Office.
>
> Best regards,
>
> Your Frontiers in Systems Biology team
>
> Frontiers in Systems Biology Editorial Office
> www.frontiersin.org
> <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.frontiersin.org%2F&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=krhxuVZzVOJqUd4AEugg36RzVEWhu%2BC%2BAExEjAfXqRU%3D&reserved=0>
> Avenue du Tribunal Federal 34
> Lausanne, Switzerland | T 41(0)21 510 17 93
>
> For technical issues, please contact our IT Helpdesk -
> support at frontiersin.org
>
>
>
>
>
> Marcella Torres, Ph.D.
>
> Director of Mathematical Studies
>
> University of Richmond
>
> Jepson Hall Room 212
>
> 221 Richmond Way
>
> Richmond, VA 23173
>
> (804) 289-8081
>
> Pronouns:  she/her
>
>
> ------------------------------
>
> *From:* Jacob Barhak <jacob.barhak at gmail.com>
> *Sent:* Thursday, January 13, 2022 8:14 PM
> *To:* Torres, Marcella <mtorres at richmond.edu>
> *Cc:* Rahuman Sheriff <sheriff at ebi.ac.uk>;
> vp-reproduce-subgroup at lists.simtk.org <
> vp-reproduce-subgroup at lists.simtk.org>;
> vp-integration-subgroup at lists.simtk.org <
> vp-integration-subgroup at lists.simtk.org>; John Gennari <gennari at uw.edu>;
> Faeder, James R <faeder at pitt.edu>
> *Subject:* Re: [Vp-reproduce-subgroup] [Vp-integration-subgroup] Paper
> review and revision check list
>
>
>
> *External Email:* Use caution in opening links, attachments, and buying
> gift cards.
>
>
>
> Thanks Marcella, Thanks Gilberto,
>
>
>
> Since the second reviewer did not respond, I think we have more time - we
> did respond in time however.
>
>
>
> The figures were provided by Alex and Sheriff - they seem pretty good
> resolution to me, yet if we want increased resolution, they will have to
> generate those again from the same software that generated thm originally -
> Basically the images have to be at least 300 dpi and export it as pdf and
> extract it as tiff or eps - not jpg.
>
>
>
> Section 2.2.2 in the instruction provides information about the image
> quality:
>
> https://www.frontiersin.org/about/author-guidelines
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.frontiersin.org%2Fabout%2Fauthor-guidelines&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=iprHKAd21gvnL7ENomziZX1FPiJ%2BiOdZYsbnaGfeMLk%3D&reserved=0>
>
>
>
> Hopefully we can have those ready.
>
>
>
> However, I do ask that all authors be ready to approve the paper assuming
> the other reviewer will be positive - we worked on thai for a year and
> there were many discussions - before this paper gets published I want all
> authors to be good with the final outcome. We will do this formally like we
> did before - yet I want everyone to be ready to avoid any delays. Hopefully
> it will take us less than 3 weeks this time.
>
>
>
>                 Jacob
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Jan 13, 2022 at 1:05 PM Torres, Marcella <mtorres at richmond.edu>
> wrote:
>
> Hi again all,
>
>
>
> On the 7th I shared comments from Reviewer 1 that we are now late
> responding to. Is it possible to provide high resolution figures, or no
> (see below)?
>
>
>
> Reviewer 2 has endorsed the manuscript.
>
>
>
> <image.png>
>
> Reviewer 1 comment:
>
>  *Reviewer 1* | 07 Jan 2022 | 14:36
>
> *#3*
>
> I would like to thank the authors for having fully addressed my comments.
> I'm satisfied about the quality the manuscript has been reached in this
> latest version. Finally, I would only suggest the authors to provide an
> high quality resolution of both new figures. Thank you.
>
>
>
>
>
> Marcella Torres, Ph.D.
>
> Director of Mathematical Studies
>
> University of Richmond
>
> Jepson Hall Room 212
>
> 221 Richmond Way
>
> Richmond, VA 23173
>
> (804) 289-8081
>
> Pronouns:  she/her
>
>
> ------------------------------
>
> *From:* Vp-reproduce-subgroup <
> vp-reproduce-subgroup-bounces at lists.simtk.org> on behalf of Rahuman
> Sheriff <sheriff at ebi.ac.uk>
> *Sent:* Monday, January 10, 2022 3:09 PM
> *To:* Jacob Barhak <jacob.barhak at gmail.com>
> *Cc:* vp-reproduce-subgroup at lists.simtk.org <
> vp-reproduce-subgroup at lists.simtk.org>;
> vp-integration-subgroup at lists.simtk.org <
> vp-integration-subgroup at lists.simtk.org>; John Gennari <gennari at uw.edu>;
> Faeder, James R <faeder at pitt.edu>
> *Subject:* Re: [Vp-reproduce-subgroup] [Vp-integration-subgroup] Paper
> review and revision check list
>
>
>
> *External Email:* Use caution in opening links, attachments, and buying
> gift cards.
>
>
>
> Thanks Jakob for submitting the revision and others for the support.
> Thanks William for editing my response.
>
>
>
> Best
>
> Sheriff
>
>
>
>
>
> On 7 Jan 2022, at 04:52, Jacob Barhak <jacob.barhak at gmail.com> wrote:
>
>
>
> Thanks Hana, Thanks William, and thanks to Gilberto, Alex, and Sheriff,
>
>
>
> With your help and some effort, the revised paper was resubmitted in time
> - just now.
>
>
>
> William, please keep on reminding me about your second affiliation - I
> kept a comment as a reminder to do this - the submission system did
> not allow making any changes in Author information - not sure why - if this
> gets accepted we should ask the editor to add your affiliation manually.
>
>
>
> I urge all other authors to look at the changes we made and our response.
> We will have to approve the manuscript with everyone before it gets
> published - hopefully we will convince the reviewers this review round.
>
>
>
>                     Jacob
>
>
>
> On Thu, Jan 6, 2022 at 3:51 PM Dobrovolny, Hana <h.dobrovolny at tcu.edu>
> wrote:
>
> I've read through the paper and fixed a few grammar mistakes.
>
> Hana
>
>
>
> *******************************************************
> Dr. Hana Dobrovolny
> Associate Professor of Biophysics
> Texas Christian University
> TCU Box 298840
> Fort Worth, TX 76129
>
> phone: (817) 257-6379 fax: (817) 257-7742
> email: h.dobrovolny at tcu.edu
> *******************************************************
>
>
> ------------------------------
>
> *From:* Vp-integration-subgroup <
> vp-integration-subgroup-bounces at lists.simtk.org> on behalf of Jacob
> Barhak <jacob.barhak at gmail.com>
> *Sent:* January 5, 2022 11:48 PM
> *To:* William Waites; James Glazier
> *Cc:* vp-reproduce-subgroup at lists.simtk.org;
> vp-integration-subgroup at lists.simtk.org; John Gennari; Faeder, James R
> *Subject:* Re: [Vp-integration-subgroup] [Vp-reproduce-subgroup] Paper
> review and revision check list
>
>
>
> *[EXTERNAL EMAIL WARNING]* DO NOT CLICK LINKS or open attachments unless
> you recognize the sender and know the content is safe.
>
> Greetings paper contributors,
>
>
>
> Gilberto has addressed the final point and added text in the paper and
> review, thus concluding our response.
>
>
>
> I went over the paper from start to finish again and cleaned it up.
>
>
>
> There are two last comments in the paper addressed to William Waites and
> James Glazier - if you can quickly check those, it would help.
>
>
>
> If someone else wants to read the paper once more, please do this today -
> otherwise I will convert the paper and our response to reviewers and submit
> it to the Journal portal.
>
>
>
> If anyone has any objections, now is the time to raise them so we can
> withdraw rather than submit.
>
>
>
> Here is the link to the revised paper:
>
>
> https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit?usp=sharing
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM%2Fedit%3Fusp%3Dsharing__%3B!!K6Z8K8YTIA!Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0hvK3OmJ8%24&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=S1YsLAtsQlJqc20P7ofS05W5Y9Pyv0SfL%2BTE0I2YuRo%3D&reserved=0>
>
>
>
> Here is the link to response for reviewers:
>
>
> https://docs.google.com/document/d/1bO1CDbMMlyyweYayP9rGohPIfgvIhfe14sFchtJMJ-s/edit?usp=sharing
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1bO1CDbMMlyyweYayP9rGohPIfgvIhfe14sFchtJMJ-s%2Fedit%3Fusp%3Dsharing__%3B!!K6Z8K8YTIA!Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0haRkeB0o%24&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BTKMupNEDY49hRuK6XNsObExXYK%2BXBzZPW%2F9n3g4SpU%3D&reserved=0>
>
>
>
>
>
> Hopefully there will be no objections and can submit this tomorrow.
>
>
>
>               Jacob
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Tue, Jan 4, 2022 at 11:30 PM Jacob Barhak <jacob.barhak at gmail.com>
> wrote:
>
> Hi Alex, Hi William, Hi Gilberto, and all other contributors.
>
>
>
> Alex, William and Gilberto are mentioned since we are the most active
> contributors.
>
>
>
> We have 2 days to finish edits.
>
>
>
> William - I attempted to add your suggestions to the paper regarding issue
> 11. You are free to revise further.
>
>
>
> Gilberto - I merged some of your texts, yet it seems you are the only one
> I saw interested in resolving issue 12 where the reviewer asks to remove
> the "Model Application and Implementation Barriers" section. Please suggest
> your corrections as soon as possible. If you cannot do this by tomorrow I
> will do my best to add a response based on your idea. Yet I really hope you
> can do this before me.
>
>
>
> Please finalize any edits you have - and if anyone has any objection to
> the text, please raise it within one day since after tomorrow I will start
> closing down all issues.
>
>
>
> I would ask for a  volunteer that will go over the paper once more after
> its finalized and check for typos and grammar - we do not want the paper to
> return again due to this issue. Time is tight, so please volunteer fast.
>
>
>
> Hopefully we can get it done in time.
>
>
>
>                 Jacob
>
>
>
>
>
>
>
>
>
>
>
> On Tue, Jan 4, 2022 at 7:25 AM Alexander Kulesza <
> alexander.kulesza at novadiscovery.com> wrote:
>
> Dear Jacob, dear all,
>
> thanks for endorsing my figure. I'll gladly add this. Please find enclosed
> the .png.
>
>
>
> What you say makes sense and I agree that exaggerating changes is not
> speeding up approval. Please still consider it as a tool to perform "major"
> change (ordering of sections is a major change of a paper in my opinion)
> without changing too much the sections as such. But I agree that this could
> be tried out should the reviewers insist.
>
>
>
> I therefore reverted the table change (disconnecting ist from Figure 1)
> and suggest the following Figure caption:
>
>
>
> "Figure 2: Sketch of difficulties persistent that impede reproducibility,
> credibility, utility and integration of models, especially in computational
> biology. An assignment of these difficulties to the four different concepts
> interpreting them as hurdles is attempted. We would like to point out that
> this sequential assessment model indicated in the graph is only one of the
> possibilities a modeler would assess the suitability of available models.
> In the rest of the paper we therefore address each difficulty separately. "
>
>
>
> With that sentence we can discuss the difficulties in any order.
>
>
>
> Best
>
> Alexander
>
>
>
> On Tue, 4 Jan 2022 at 13:24, Jacob Barhak <jacob.barhak at gmail.com> wrote:
>
> Yes Alex,
>
>
>
> Your image is much better than mine, please use it instead in the paper
> and send a link to the image itself in an acceptable image format so I can
> upload it to the submission system later on.
>
>
>
> As for changing the table and order, I am not sure. Although it may be
> possible. We need to think about it. Yet we really are crunched in time, so
> I am leaning to say no.
>
>
>
> First, because some of the difficulties span more than one category, for
> example unit standardization affects integration as well as
> reproducibility, only it appears first at reproducibility, therefore it was
> placed there in the figure.
>
>
>
> Second, if you look at the text there are some transition sentences among
> sections, that connect them, so swapping those around may just mess things.
> I am not sure it if will also mess up acronym definitions on the first
> appearance as well, so shuffling order is not something's I would do unless
> necessary. And it was not requested by reviewers anyway.
>
>
>
> I am also trying to minimize changes and focus only on what was requested
> rather than change a lot more. Otherwise we may never finish changing
> things since each one of the 18 of us will have a one last item they MUST
> have corrected. I personally did not add some recent publications I had,
> although I added one publication James asked to add in the previous
> approval round and corrected one that Sheriff mentioned. Yet I really want
> to avoid opening the paper to significant changes.
>
>
>
>  If you start opening corrections beyond what was requested, this will
> drag on forever since we do have a large list of contributors and each one
> of us will generate a new idea. So please focus on what was asked unless
> absolutely necessary.
>
>
>
> So I suggest not changing the table, and changing only the figure and
> keeping order intact. I hope I am projecting the sense of urgency - we had
> the paper lying around for a couple of weeks during the holiday without
> much progress, I ask not to escalate things just because a deadline is
> looming.
>
>
>
> If you have extra time on your hands in the next couple of days, I suggest
> you focus on addressing the last two points in the response that we have
> not resolved yet. I know Gilberto wanted to fix one of them, yet you are
> welcome to do it. Those are things requested by the review
>
>
>
> Hopefully my arguments make sense.
>
>
>
>          Jacob
>
>
>
> On Tue, Jan 4, 2022, 06:45 Alexander Kulesza <
> alexander.kulesza at novadiscovery.com> wrote:
>
> Dear all,
>
>
>
> I have thought about the additional Figure requested by the reviewer.
> Jacob your Figure nicely shows the assignment of the difficulties we detail
> in our central table with the 4 big concepts.
>
>
>
> I feel however that the text-heavy figure is a lot o redundancy with the
> rest of the elements (existing figure & table). Therefore I wanted to throw
> another suggestion forward (inspired by systematic reviews narrowing down
> eligible studies/data).
>
>
>
> @all please notice that the assignment I would do is a bit different to
> what is currently done in the paper. The order of sections would have to be
> altered. Not sure if that is acceptable.
>
> <image.png>
>
> The table with Difficulties (hurdles) and solutions would then be
>
> *Difficulty*
>
> *Potential Solutions*
>
>
>
> Reproducibility
>
> Models are written in different languages
>
> Common transport specifications such as SBML or CellML, and proper
> documentation and annotation
>
> Models are hard to locate
>
> Archive web sites such as: BioModels, SimTK, IMAGWiki, and the future
> modeleXchange
>
> Lack of common platforms for executing models and simulations
>
> Platforms such as BioSimulators, and runBioSimulations
>
> Unit standardization
>
> Standardization efforts, and machine learning solutions such as
> ClinicalUnitMapping.com
> <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fclinicalunitmapping.com%2F&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RP3nfesOyElFQcdz%2B1b4ri3IiJVgWtlKYE3mEDI6Qqw%3D&reserved=0>
>
>
> Credibility
>
> Models have built-in barriers to Evaluating model credibility
>
> Better modeling practices, documentation, and tests.
>
> Data availability and measurement definitions
>
> Models that merge human interpretation, and newer measurement devices
>
> Missing annotations in models
>
> Adoption of policies such as those COMBINE suggests
>
> Utility
>
> Models are not consistently licensed to allow for reuse
>
> Abandoning some old school open source licenses and promoting licenses
> that release to public domain
>
> Different scales and modeling paradigms
>
> Standardization effort and centralization tools
>
> Stochastic modeling difficulties
>
> Development of tools that guarantee repeatability and standards to address
> stochastic simulations
>
> Integration
>
> Model application and implementation barriers
>
> Education of modelers, users, and the public
>
> Modeling requires adaptation towards integration
>
> Tools for composing models such as SBML-Comp, and SemGen
>
>
>
> Following this assignment.
>
>
>
> Please let me know what you think. I have added this suggestion in track
> changes mode to the revision draft.
>
>
>
> Best
>
> Alexander
>
>
>
>
>
> On Mon, 3 Jan 2022 at 07:27, Jacob Barhak <jacob.barhak at gmail.com> wrote:
>
> Thanks Alex,
>
>
>
> You did great by adding the new material that the reviewers actually asked
> for. This is great - I found myself adding the new FDA document to find out
> that you already added it - I just voice read it ona  long drive and
> figured it fits well - it seems you were faster - great.
>
>
>
> I merged your edits and I am ok with adding your response to the reviewers
> - I only had an issue with one example paragraph that you deleted - I left
> it with a comment - please check if it is possible to keep this example
> somewhere in the text.
>
>
>
> I will assemble a letter that includes all those responses point by point
> and will try to use your text almost verbatim if possible. You really bring
> important knowledge with you.
>
>
>
> It seems most points have been answered to some level - even a figure was
> created - although I wish I was a better artist.
>
>
>
> I will try to assemble the response letter for everyone within the next
> few days. And unless there will be objections I will try to pass it by
> reviewers with a new version. If the new version is accepted, we will do a
> final approval round before publication.
>
>
>
> Many thanks to all those who contributed.
>
>
>
>               Jacob
>
>
>
>
>
>
>
>
>
>
>
> On Sun, Jan 2, 2022 at 7:27 AM Alexander Kulesza <
> alexander.kulesza at novadiscovery.com> wrote:
>
> Dear all,
>
>
>
> Happy new year to everyone. I hope that you are all well and safe and have
> spent a nice holiday season.
>
>
>
> I have tried to address
>
>
>
> *Reviewer 2 *
>
>
> *Moreover, when the authors mention ASME V&V40 procedure, it could be
> useful to spend few words about the model credibility pipeline drawn for
> predictive models in biomedicine, such as the one suggested in "Credibility
> of in Silico Trial Technologies-A Theoretical Framing", Viceconti, M.,
> Juarez, M.A., Curreli, C., ...Russo, G., Pappalardo, F., IEEE Journal of
> Biomedical and Health Informatics, 2020, 24(1), pp. 4–13, 8884189. For this
> specific aspect, the authors should talk about "specific question of
> interest" and not "specific purpose" in the paragraph called "Credibility
> of Models" to follow in such a way a standardised language.*
>
>
>
> I have worked on the followng document in "track changes" mode:
>
> https://docs.google.com/document/d/1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM/edit
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1IMEgmdNkx-EsnOjGuegpenSIMmKIkK00Lc8Gred3QxM%2Fedit__%3B!!K6Z8K8YTIA!Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0hrtFQ-XI%24&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6KsnhIzKT9PPkrSZfRlQhd0V2SaAM8rfQZfDweQeQdA%3D&reserved=0>
>
> Actually, I ended up in changing rather drastically the section. I think
> that all ideas and all text is conserved, but I would very much like to
> encourage you to read, comment and challenge this revision. Please tell me
> if that goes to far. No problem to revert to an earlier version if
> necessary.
>
>
>
> I suggest the following formulation in the rebuttal letter:
>
>
>
> When trying to better describe the "credibility pipeline" (CoU definition,
> verification, validation, uncertainty quantification, e.g. as described in
> Viceconti et al. 2019) or others, we noticed that the section about
> credibility could be restructured in order to put the reader (not
> necessarily familiar with regulatory assessment of models) in the position
> to follow the argumentation.
>
>
>
> We now roughly follow the following flow of writing:
>
>
>
>  - Every model needs a purpose and credibility is tied to that purpose as
> well as repeatability and reproducibility
>
> -  Credibility is essential for models that have impact on regulated areas
> or people's lives which is why regulatory authorities issue guidance
>
> -  The probably most advanced guidance ASME V&V40 suggests a risk-based
> approach and pipeline to establish credibility of a model. It is
> overarching and widely applicable
>
> - Regulators and modelers work together (for example in frame of the model
> informed drug development program MIDD) in order to better understand,
> better apply and better uptake different kinds of models in regulatory
> applications
>
> - More work is needed to harmonize, stay up to date and to be more
> inclusive
>
>
>
> We hope that with the rather extensive change of the wording and
> additional passages as well as citations (see below) we could address the
> concern of reviewer 2 (and the general remarks of reviewer 1).
>
>
>
> " In 2018 the American Society of Mechanical Engineers (ASME) issued an
> important guidance ASME V&V 40 (ASME, 2018) of how to assess credibility of
> computational models of medical devices through verification and validation
> (V&V). The guideline is centered around the definition of the context of
> use (CoU) of the model which is formulated based on the questions of
> interest the model will answer. The CoU is then analyzed in terms of the
> “model risk” - being the influence the model exerts on a decision and the
> potential consequences these decisions might incur. Commensurate with this
> model risk the modeler suggests establish the credibility goals, perform
> verification validation and uncertainty quantification actions and then
> assess the outcome of this exercise in order to allow judging the
> acceptability of the model CoU. Key to this guidance is its overarching
> nature which also allows adoption in other (e.g. drug development) fields
> irrespective of the model type (Kuemmel  2020, Viceconti 2019)."
>
>
>
> We also feel that it is essential to underline the cross-discipline
> viewpoint of the ASME V&V40 which is further elaborated by the cited paper
> by formalizing the verification, validation and uncertainty quantification
> VVUQ pipeline across model types.
>
>
>
> We therefore have added the following statement
>
> " In the paper by (Viceconti 2019) the verification, validation and
> uncertainty quantification (VVUQ) pipeline is streamlined to different
> types of models. It is, perhaps, the closest to score credibility across
> model types from mechanistic physics driven models to machine learning
> models. However, it is still short of including very recent developments
> such as ensemble models, although it touches upon the topic."
>
>
>
> We additionally strengthened our argumentation by citing additional
> literature
>
>
>
> Bai, JPF, Earp, JC, Florian, J, et al. *Quantitative systems
> pharmacology: Landscape analysis of regulatory submissions to the US Food
> and Drug Administration. *CPT Pharmacometrics Syst Pharmacol. 2021; 10:
> 1479– 1484. doi.org
> <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoi.org%2F&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zuowetFS1kefr%2B59xCv2XXXXFJyS4Cpv3MPKP17oog0%3D&reserved=0>
> :10.1002/psp4.12709
>
>
>
> FDA (2021b). Assessing the Credibility of Computational Modeling and
> Simulation in Medical Device Submissions. *FDA*
> Available at:
> https://www.fda.gov/regulatory-information/search-fda-guidance-documents/assessing-credibility-computational-modeling-and-simulation-medical-device-submissions
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fwww.fda.gov%2Fregulatory-information%2Fsearch-fda-guidance-documents%2Fassessing-credibility-computational-modeling-and-simulation-medical-device-submissions__%3B!!K6Z8K8YTIA!Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0harP7IHc%24&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=wYBocQKcF02Ml5ptm9CFm1%2Fs4QyqqCeD490LyIFu%2BYc%3D&reserved=0>
> [Accessed January 2, 2022]
>
>
>
> Galluppi, G.R., Brar, S., Caro, L., Chen, Y., Frey, N., Grimm, H.P., Rudd,
> D.J., Li, C.-C., Magee, M., Mukherjee, A., Nagao, L., Purohit, V.S., Roy,
> A., Salem, A.H., Sinha, *V., Suleiman, A.A., Taskar, K.S., Upreti, V.V.,
> Weber, B. and Cook, J. (2021), Industrial Perspective on the Benefits
> Realized From the FDA’s Model-Informed Drug Development Paired Meeting
> Pilot Program*. Clin Pharmacol Ther, 110: 1172-1175. doi:10.1002/cpt.2265
>
>
>
> Kuemmel, C., Yang, Y., Zhang, X., Florian, J., Zhu, H., Tegenge, M.,
> Huang, S.-M., Wang, Y., Morrison, T. and Zineh, I. (2020), *Consideration
> of a Credibility Assessment Framework in Model-Informed Drug Development:
> Potential Application to Physiologically-Based Pharmacokinetic Modeling and
> Simulation.* CPT Pharmacometrics Syst. Pharmacol., 9: 21-28. doi.org
> <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoi.org%2F&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zuowetFS1kefr%2B59xCv2XXXXFJyS4Cpv3MPKP17oog0%3D&reserved=0>
> :10.1002/psp4.12479
>
>
>
> Musuamba, FT, Skottheim Rusten, I, Lesage, R, et al. Scientific and
> regulatory evaluation of mechanistic in silico drug and disease models in
> drug development: Building model credibility. CPT Pharmacometrics Syst
> Pharmacol. 2021; 10: 804– 825. doi:10.1002/psp4.12669
>
>
>
> Viceconti M, Emili L, Afshari P, et al. Possible Contexts of Use for In
> Silico Trials Methodologies: A Consensus-Based Review. *IEEE J Biomed
> Health Inform*. 2021;25(10):3977-3982. doi:10.1109/JBHI.2021.3090469
>
>
>
>
>
> All the best
>
> Alexander
>
>
>
> On Sun, 19 Dec 2021 at 03:10, Jacob Barhak <jacob.barhak at gmail.com> wrote:
>
>
>
> Greetings to the paper contributors,
>
>
>
> You may have seen my other message where I posted the reviews to our
> paper.  I am sending those again below to start a new thread discussing
> possible revisions.
>
>
>
> Here is a checklist for revisions required:
>
>
>
>    1.  We should discuss the references that reviewer 1 raised and add
>    them as reference
>    2. Reviewer 1 asked for more discussion around V&V40 - I believe Alex
>    added the text there originally - Alex do you think you can address the
>    request of the reviewer?
>    3. Reviewer 1 asked to focus on COVID-19 at the title - I am not sure
>    we wish to limit our scope since many of our ideas are applicable far
>    beyond COVID-19 - I leave it up for discussion in the group on how to
>    address this request by the reviewer
>    4. Reviewer 1 asked for additional graphics - I am unsure how to
>    address this beyond ur current diagram - ideas will help
>    5. We need to check the "Contribution to the field" section for
>    grammar and spelling - if someone can contribute more elegant text - now is
>    the time - I looked over it now and found no issues - yet I may have missed
>    something - this section may have been missed since it was added last and
>    perhaps vetted less than other sections. Also Reviewer 2 asked for grammar
>    corrections, so it is worthwhile proof reading the paper as a whole.
>    6. Reviewer 2 asks for major rewrite to emphasize why the problems are
>    there - I think we should explain that the core of the paper is the table
>    and perhaps emphasize it in the paper beyond what the current text - we
>    also should reply to the reviewer and explain that the paper is composed of
>    contributions from a large group and we made an effort to include every
>    voice in the choir - each of those voices is important and needs to be
>    preserved - hopefully it will convince the reviewer that change we will add
>    will be sufficient
>    7. Reviewer 2 asks to revise the introduction  - I believe some
>    changes are possible - yet the introduction includes contributions from
>    many authors - at least 10 - and I fear losing something important someone
>    contributed - if someone has an idea on how to address this reviewer
>    without a painful transformation, please reply to this message.
>    8.  Reviewer 2 wants additional discussion around the "reproducibility
>    crisis" section  and asks a valid question about expectations we should
>    address - "“Computational biomedical modeling… was expected to be less
>    affected by the reproducibility crisis.” Is this true, and why would it be
>    so? "  - My answer to the reviewer is that unlike biological processes that
>    have random nature and experiments would not repeat if repeated, while
>    computer software should be deterministic and it should be repeatable if
>    designed well.  Unfortunately we are not experiencing this promise - yet I
>    believe the reviewer wants more discussion beyond this section so I am
>    happy to discuss this.
>    9. Reviewer 2- writes:  "Models are Hard to Locate: Are the authors
>    suggesting that entire simulation workflows, from model construction to
>    analysis, should be publicly available? At what point does one consider
>    intellectual property? Do the authors advocate for such extensive
>    publishing for all models, or only ones that are intended to be widely
>    re-used? - those are important points we need to discuss - we need to
>    better explain the difficulties we are having when creating models and the
>    reviewer is absolutely correct about expanding the discussion to IP - I
>    suggest we create a thread for this discussion and reference it - I
>    suggest it we merged with the licensing  issue that I will address later -
>    there is a strong connection there - the reviewer was very observant and
>    sees the bigger problem. However,we need to eventually distill our
>    discussion to recommendations that will inline.
>    10. Reviewer 2 asks that we fix the unit standardization section - I
>    believe Hana and myself were the largest contributors there - Hana - I will
>    start a discussion on that topic in a separate email where we can publicly
>    discuss how to fix this - others will be welcome to contribute.
>    11. Reviewer 2 asks that we better handle the section on Data
>    availability and measurement definitions: in think we need to emphasize
>    solutions and separate it from issues that may not be solvable. Ideas are
>    welcome.
>    12. Reviewer 2 asks good questions with regards to licensing following
>    our text- I personally have good answers to the reviewer and William and I
>    had some discussion on the topic in this list I suggest we expand this
>    discussion in a separate thread - hopefully William and perhaps others will
>    join the discussion. This discussion should also address the IP issues
>    raised by the reviewer for the "models are hard to locate section."
>    13. Reviewer 2 asks to handle the Model Application and Implementation
>    Barriers section. We should decide what to do there, the section may need
>    expansion since the ideas there are solid, yet the section is short so
>    perhaps enhancing it makes better sense. I am open to suggestions.
>
>
>
> Those are the items I located and my suggestions. It seems we need
> attention from Alex, Hana, myself and william. However, anyone on the list
> is welcome to participate and suggest changes.
>
>
>
> I will start the discussion threads on specific topics. Hopefully we can
> get it done quickly.
>
>
>
>          Jacob
>
>
>
>
>
>
>
>
>
>
>
>
>
> -
>
>
>
>
>
>
>
>
>
>
>
> ########### Original Reviews #############
>
>
>
> There are 2 reviews - both require major changes. I am copying the
> relevant text below.  If more appear, I will let you know, yet I only got
> this message today although the reviews are dated a few days ago.
>
>
>
> Reviewer 1:
>
> Recommendation for the Editor: Substantial revision is required
>
>
>
> Please list your revision requests for the authors and provide your
> detailed comments, including highlighting limitations and strengths of the
> review. If you have additional comments based on Q2 and Q3 you can add them
> as well.
>
>
>
> Karr and co-authors made an interesting and exaustive point about the
> reproducibility crisis that leads to inability to reuse and integrate
> models, especially about COVID-19 disease.
> Within the manuscript some typos and missing information are present.
> I'd suggest the authors to revise the entire manuscript especially in
> terms of the state of the art, revising and updating the most relevant
> examples of computational models dealing with COVID-19 and in general about
> some semi-standardised proposals about the pipeline to follow for the
> verification and validation of model credibility. In particular, the
> authors failed to mention and cite some major results on in silico modeling
> about COVID-19 up to now. See for example:
> a."In silico trial to test COVID-19 candidate vaccines: a case study with
> UISS platform", Russo, G., Pennisi, M., Fichera, E., ...Viceconti, M.,
> Pappalardo, F., BMC Bioinformatics, 2020, 21, 527.
> b. Russo G, Di Salvatore V, Sgroi G, Parasiliti Palumbo GA, Reche PA,
> Pappalardo F. "A multi-step and multi-scale bioinformatic protocol to
> investigate potential SARS-CoV-2 vaccine targets" [published online ahead
> of print, 2021 Oct 5]. Brief Bioinform. 2021;bbab403.
> doi:10.1093/bib/bbab403.
> Moreover, when the authors mention ASME V&V40 procedure, it could be
> useful to spend few words about the model credibility pipeline drawn for
> predictive models in biomedicine, such as the one suggested in "Credibility
> of in Silico Trial Technologies-A Theoretical Framing", Viceconti, M.,
> Juarez, M.A., Curreli, C., ...Russo, G., Pappalardo, F., IEEE Journal of
> Biomedical and Health Informatics, 2020, 24(1), pp. 4–13, 8884189.
> For this specific aspect, the authors should talk about "specific question
> of interest" and not "specific purpose" in the paragraph called
> "Credibility of Models" to follow in such a way a standardised language.
> Moreover, the authors should refer to COVID-19 also in the title, with a
> specific mention about the fact that the main topic of model integration in
> computational biology will be discussed inside the COVID-19 context.
> Furthermore, for the model key concepts such as
> Reusability-Extensibility-Extractability-Portability the authors should
> described and outlined through a graphical sketch or visual representation
> that summarises these key point.
> The authors should also fix some grammar and writing typos present in the
> "Contribution to the field" section.
>
>
>
> a. Is the quality of the figures and tables satisfactory?
> - No
>
> b. Does the reference list cover the relevant literature adequately and in
> an unbiased manner?
> - No
>
> c. Does this manuscript refer only to published data? (unpublished or
> original data is not allowed for this article type)
> - Yes
>
> d. Does the review include a balanced, comprehensive, and critical view of
> the research area?
> - Yes
>
>
>
>
>
> Reviewer 2
>
>
>
> Please list your revision requests for the authors and provide your
> detailed comments, including highlighting limitations and strengths of the
> review. If you have additional comments based on Q2 and Q3 you can add them
> as well.
>
>
>
>
>
> The article often reads as a stream-of-consciousness account of the
> discussions that took place but lacks a clear thesis or recommendations. It
> is not clear what, if anything, the authors are advocating for. It is not
> always clear why the issues being discussed are problematic, or that they
> can be reasonably addressed. Some of the issues raised are indeed important
> and should be discussed, but the paper lacks focus and does not tell a
> cohesive story. I believe this manuscript requires a major re-write to be
> suitable for publication. The authors should consider narrowing the scope
> of the discussion and focusing on a cohesive set of recommendations or open
> questions. More specifically, I make a few suggestions below:
>
> Major comments
> 1. The introduction is long and repeats itself (e.g., “much less is known
> about how viral infections spread throughout the body…” is repeated
> verbatim). It is not clear from the introduction what the main goal of the
> paper is or why the “reproducibility crisis” is truly a crisis. Why is the
> discussion of composition and black/white box models relevant to the
> introduction? Further, this section is subtitled “the promise of modeling”,
> which does not seem to match the content.
> 2. The Reproducibility Crisis: “Computational biomedical modeling… was
> expected to be less affected by the reproducibility crisis.” Is this true,
> and why would it be so? One would think that more complex models would
> suffer more from a lack of reproducibility. It may be helpful to define
> what exactly the “reproducibility crisis” refers to.
> 3. Models are Hard to Locate: Are the authors suggesting that entire
> simulation workflows, from model construction to analysis, should be
> publicly available? At what point does one consider intellectual property?
> Do the authors advocate for such extensive publishing for all models, or
> only ones that are intended to be widely re-used?
> 4. Unit standardization: The conversion from PFU or TCID50 to individual
> virions is likely to differ across viruses – are the authors focused on
> COVID here? Are the authors advocating for a standard conversion factor? It
> is not clear what the purpose of this discussion is. As the authors
> mention, different scales require different units. Even at a single scale,
> different models may require different units for numerical reasons. It is
> not clear what the authors are advocating for here.
> 5. Data availability and measurement definitions: This section seems to
> outline limitations of available data, but again makes no recommendations
> or proposed solution to any of the issues raised. Is this the intention?
> Most of the issues raised here reflect limitations of experimental science
> or data privacy, which likely cannot be meaningfully addressed by the
> modeling community.
> 6. Models are Not Consistently Licensed…: Are the authors implying here
> that all modeling work should be published with no rights reserved? Is it
> reasonable to expect modelers to make their work freely usable by others
> for profit? Is it reasonable for institutions to allow this? How much does
> this really contribute to reproducibility and utility?
> 7. Model Application and Implementation Barriers: This section seems
> unnecessary and out of place.
> 8. There are grammar and punctuation errors scattered throughout; please
> edit carefully.
>
>
>
>
>
> a. Is the quality of the figures and tables satisfactory?
> - Not Applicable
>
> b. Does the reference list cover the relevant literature adequately and in
> an unbiased manner?
> - Yes
>
> c. Does this manuscript refer only to published data? (unpublished or
> original data is not allowed for this article type)
> - Yes
>
> d. Does the review include a balanced, comprehensive, and critical view of
> the research area?
> No answer given.
>
>
>
>
>
> ========
>
> _______________________________________________
> Vp-reproduce-subgroup mailing list
> Vp-reproduce-subgroup at lists.simtk.org
> https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Flists.simtk.org%2Fmailman%2Flistinfo%2Fvp-reproduce-subgroup__%3B!!K6Z8K8YTIA!Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0hhiovw3M%24&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ft20m5jvIWQiK3FsYU9aVPypT%2FE1jvz9aXw%2FLa1TIDQ%3D&reserved=0>
>
>
>
>
> --
>
> Alexander Kulesza
>
> Team leader
>
> Modeling & simulation / Biomodeling
>
> alexander.kulesza at novadiscovery.com
>
> +33 7 82 92 44 62
>
> *nova*
>
> DISCOVERY
>
> www.novadiscovery.com
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2F%2Fwww.novadiscovery.com%2F__%3B!!K6Z8K8YTIA!Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0hwI_2O8g%24&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8lm%2BN5l%2F2FMAHXhNvHwf1PoE%2BgoRl40Rx%2Fmb7KVq7UM%3D&reserved=0>
>
> 1 Place Verrazzano, 69009 Lyon
> +33 9 72 53 13 01
>
>
> *This message is intended only for the personal and confidential use of
> the designated recipient(s) named above. If you are not the intended
> recipient of this message you are hereby notified that any review,
> dissemination, distribution or copying of this message is strictly
> prohibited. Email transmission cannot be guaranteed to be secure or
> error-free. Therefore, we do not represent that this information is
> complete or accurate and it should not be relied upon as such. All
> information is subject to change without notice.*
>
>
>
>
> --
>
> Alexander Kulesza
>
> Team leader
>
> Modeling & simulation / Biomodeling
>
> alexander.kulesza at novadiscovery.com
>
> +33 7 82 92 44 62
>
> *nova*
>
> DISCOVERY
>
> www.novadiscovery.com
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2F%2Fwww.novadiscovery.com%2F__%3B!!K6Z8K8YTIA!Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0hwI_2O8g%24&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8lm%2BN5l%2F2FMAHXhNvHwf1PoE%2BgoRl40Rx%2Fmb7KVq7UM%3D&reserved=0>
>
> 1 Place Verrazzano, 69009 Lyon
> +33 9 72 53 13 01
>
>
> *This message is intended only for the personal and confidential use of
> the designated recipient(s) named above. If you are not the intended
> recipient of this message you are hereby notified that any review,
> dissemination, distribution or copying of this message is strictly
> prohibited. Email transmission cannot be guaranteed to be secure or
> error-free. Therefore, we do not represent that this information is
> complete or accurate and it should not be relied upon as such. All
> information is subject to change without notice.*
>
>
>
>
> --
>
> Alexander Kulesza
>
> Team leader
>
> Modeling & simulation / Biomodeling
>
> alexander.kulesza at novadiscovery.com
>
> +33 7 82 92 44 62
>
> *nova*
>
> DISCOVERY
>
> www.novadiscovery.com
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__http%3A%2F%2Fwww.novadiscovery.com%2F__%3B!!K6Z8K8YTIA!Qmy11SwUfgLnvEtA_Yy8DDA-OCSuTnTFPcnArqqv3zx410koRSJz-doddC0hwI_2O8g%24&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8lm%2BN5l%2F2FMAHXhNvHwf1PoE%2BgoRl40Rx%2Fmb7KVq7UM%3D&reserved=0>
>
> 1 Place Verrazzano, 69009 Lyon
> +33 9 72 53 13 01
>
>
> *This message is intended only for the personal and confidential use of
> the designated recipient(s) named above. If you are not the intended
> recipient of this message you are hereby notified that any review,
> dissemination, distribution or copying of this message is strictly
> prohibited. Email transmission cannot be guaranteed to be secure or
> error-free. Therefore, we do not represent that this information is
> complete or accurate and it should not be relied upon as such. All
> information is subject to change without notice.*
>
> _______________________________________________
> Vp-integration-subgroup mailing list
> Vp-integration-subgroup at lists.simtk.org
> https://lists.simtk.org/mailman/listinfo/vp-integration-subgroup
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.simtk.org%2Fmailman%2Flistinfo%2Fvp-integration-subgroup&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=C9gkV%2FQlxmxyiMpBEeqUzaYr%2B%2Benn3lSlNTewl35%2FKQ%3D&reserved=0>
>
>
>
> _______________________________________________
> Vp-reproduce-subgroup mailing list
> Vp-reproduce-subgroup at lists.simtk.org
> https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.simtk.org%2Fmailman%2Flistinfo%2Fvp-reproduce-subgroup&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Xz5dFj5Al0WeSq1UKSe4O7SjAe%2BUFTdtQFrBhq9%2Bzcg%3D&reserved=0>
>
>
>
>
> --
>
> Alexander Kulesza
>
> Team leader
>
> Modeling & simulation / Biomodeling
>
> alexander.kulesza at novadiscovery.com
>
> +33 7 82 92 44 62
>
> *nova*
>
> DISCOVERY
>
> www.novadiscovery.com
> <https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.novadiscovery.com%2F&data=04%7C01%7Cfaeder%40pitt.edu%7C0569624c1f584e9ae62c08d9da04bd05%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637780537394492600%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=LlOTa%2Fgo%2BooxaqsnvZX%2BNvoQBJYJC9%2FiVZJDmEPMBz4%3D&reserved=0>
>
> 1 Place Verrazzano, 69009 Lyon
> +33 9 72 53 13 01
>
>
> *This message is intended only for the personal and confidential use of
> the designated recipient(s) named above. If you are not the intended
> recipient of this message you are hereby notified that any review,
> dissemination, distribution or copying of this message is strictly
> prohibited. Email transmission cannot be guaranteed to be secure or
> error-free. Therefore, we do not represent that this information is
> complete or accurate and it should not be relied upon as such. All
> information is subject to change without notice.*
> <ModelIntegrationInComputationalBiology-Hurdles.png><ModelIntegrationInComputationalBiology-Hurdles.svg><ModelIntegrationInComputationalBiology-Hurdles.eps>_______________________________________________
> Vp-reproduce-subgroup mailing list
> Vp-reproduce-subgroup at lists.simtk.org
> https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
>
>
> _______________________________________________
> Vp-reproduce-subgroup mailing list
> Vp-reproduce-subgroup at lists.simtk.org
> https://lists.simtk.org/mailman/listinfo/vp-reproduce-subgroup
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.simtk.org/pipermail/vp-integration-subgroup/attachments/20220117/54f41aef/attachment-0001.html>