[Vp-integration-subgroup] About using a multi-scale mortality model in the ensemble
Jacob Barhak
jacob.barhak at gmail.com
Thu Dec 31 01:27:40 PST 2020
Ok Fil,
Your corrected model code is now on Github at:
https://github.com/Jacob-Barhak/COVID19Models/tree/main/COVID19_Mortality_Castiglione
I chose MIT license for now, just to make sure you are comfortable with
this initially. I would have liked to put it under CC0 license so that it
can be more easily reused - however, I cannot really do it without you
understanding the consequences - after all it means you giving away your IP
- think about it as a gift to the world - it will make any type of reuse
much easier for various purposes. In fact I am not the only one
recommending this license, yet I cannot make the decision unless the
creators approve.
Here is a link to the license language:
https://creativecommons.org/choose/zero/
If after you read the language you approve the change, I will change the
license to CC0.
Yet for now I am more interested in how to plug your model into the
ensemble.
The Reference Model was recently using a certain approach to
calculate mortality - it had the probability of mortality extracted as a
function of age from 3 models without deciding on time and the time of
mortality extracted from merging 2 models. The model would know ahead of
time when the individual might die according to the mortality time model
and then test a random number against this threshold mortality probability
at that time during simulation for infected individuals. I did not have a
density function I can build upon like the one you provided. Basically what
I had is a spike function for each individual so using your function will
make the ensemble much richer since it replaces a combination of multiple
model components, yet it requires a different approach to modeling
mortality and recovery since I need to merge the spike functions with your
smoother distribution somehow. I will need to think of the best way to do
this and it may take a few days.
Since I need now to also remodel recovery, it bring us back to the issue
you raised of not having a defined state and instead having a continuous
function with a cutoff required.
Also, recall that in an ensemble there are multiple other models that
influence the result - the system will pick a mixture of models that work
together best to explain the phenomenon observed. So it is more about the
contribution of each model and sometimes it is hard to predict which models
will work well together. Yet I think that having more models will help us
better understand things, so your model will for sure contribute to
understanding once I figure out how to merge it.
Hopefully I can get it done soon. Until then thank you for your fast
responses.
Jacob
On Wed, Dec 30, 2020 at 2:33 AM Filippo Castiglione <
filippo.castiglione at gmail.com> wrote:
> Hi Jacob, it looks all good to me but the definition of time (which I
> forgot to tell you before, sorry ;) that in the simulator (therefore in the
> data I sent you and therefore in the mortality model) is 8 hours: it just
> means that you have to divide time by 3 to get days. So the period is 30
> days, not 90 days. Apart from that, it all looks great. Thanks for the
> copyright. Feel free to put it on you git with the license you see fit.
>
> I am curious to see how it works in your ensamble model.
>
> Best wishes
> Fil
>
> > On 30 Dec 2020, at 08:47, Jacob Barhak <jacob.barhak at gmail.com> wrote:
> >
> > Thanks Fil,
> >
> > You helped a lot. However, to make sure we are on the same page, I
> decided to implement the function using python and show you the plot - just
> to make sure I implemented your instructions correctly - I rather show you
> my understanding so you can tell me if this looks reasonable to you.
> >
> > Attached is the link to an archive that contains the code and the
> output.
> >
> https://drive.google.com/file/d/12AU3xsjUk8xVBbb7UmO4hE-ldGj_QJ08/view?usp=sharing
> >
> > Once you open the archive you should see the interactive plot inside as
> an html file - you should be able to open it with a modern browser like
> chrome, edge, or firefox
> >
> > As for the code I assigned copyright to you and added some
> documentation. If you have someone in your team familiar with python , you
> just need to install holoviews and panel to create the plots. I did not put
> the code in Github - I leave this to you - I am just interested in using
> the function in my ensemble and need verification that this is correct. Yet
> if you want, I will be happy to upload it to my github account if we decide
> on a proper license like MIT / BSD or even preferred CC0 to release this to
> the public without copyright restrictions.
> >
> > For now, please just verify this is correct - I noticed peak mortality
> increases by age from 50 days to 61 days - is this correct or did I
> misunderstand something?
> >
> > Hopefully this is correct and I can continue to plug in the model into
> the ensemble.
> >
> > Jacob
> >
> >
> >
> >
> > On Sun, Dec 27, 2020 at 11:13 PM Filippo Castiglione <
> filippo.castiglione at gmail.com> wrote:
> > Jacob,
> >
> > I forgot to say that after you have calculated the function F’, which is
> the probability-density-function of the event “death”, you have to multiply
> to an age-dependent coefficient which can be found in fig.3 panel A of my
> paper divided by 100 (because there it is expressed in percent).
> >
> > In other words for instance the probability to die at day x with respect
> to the age-class 40-49 is
> >
> > 1.25/100 * F’(x,a,b,c) with
> >
> > a=95.351153808322
> > b=0.173066025988826
> > c=51.92498287875
> >
> > Sorry for the lack of precision.
> > I hope it is clearer now.
> > Fil
> >
> >
> >
> > > On 27 Dec 2020, at 15:51, Filippo Castiglione <
> filippo.castiglione at gmail.com> wrote:
> > >
> > > Hi Jacob, let me reply below
> > >
> > >> I was thinking of a function that defines the probability of
> mortality as a function of time since infection, with the good chance of
> surviving. This function should look like figure 2 in your paper where
> higher age individuals have higher chance of dying, yet after a certain
> point in time those who survived no longer can die, indicating recovery.
> > >>
> > >> It seems that the function you suggested is a little different. If I
> understand you correctly, you already stratified the group that will die
> and you are trying to show the probability of an individual dying in a
> certain day from infection knowing they will die.
> > >
> > > Ok (in my last reply I was incorrect), then the function you are
> looking for is the derivative of the one I sent you (normalised dividing by
> 100), namely,
> > >
> > > F(x,a,b,c) = S(x,a,b,c) / 100
> > >
> > > and its derivative is
> > >
> > > F’(x,a,b,c) = (a * b exp(-b * (-c + x))) / (100 (1 + exp(-b *
> (-c + x)))^2)
> > >
> > > Now the values of the parameters a,b,c are those I sent you last time
> IF
> > > we consider the age-UNstratified population.
> > >
> > > Otherwise, if we go by age (age ranges as in the paper), then the
> parameters are:
> > >
> > > age range 0-39
> > > a=96.56299632152
> > > b=0.185777403544838
> > > c=49.8164210315686
> > >
> > > age range 40-49
> > > a=95.351153808322
> > > b=0.173066025988826
> > > c=51.92498287875
> > >
> > > age range 50-59
> > > a=94.2976479245251
> > > b=0.165018635066338
> > > c=54.1494851021362
> > >
> > > age range 60-69
> > > a=91.3990353489634
> > > b=0.158299883031816
> > > c=55.5935133562513
> > >
> > > age range 70-79
> > > a=88.9048923236574
> > > b=0.152924609509616
> > > c=57.7447222979953
> > >
> > > age range 80+
> > > a=85.6674967019556
> > > b=0.14040123117131
> > > c=61.383806850923
> > >
> > >> Please verify that I am correct. Since I need to know how to use your
> function.
> > >
> > > Let me know if F’ (F-prime) is the function you were looking for. It
> should be it.
> > >
> > >> You have two more interesting elements in the paper that can start
> interesting discussions:
> > >>
> > >> 1. You approach the definition of a comorbidity by a level of damage
> to tissue.
> > >
> > > Yes, given the elements in my simulation, that is a possible choice
> for the definition (not really of the comorbidity) of the “severity” of the
> disease. It is a tricky point to link some computer output to a “clinical
> endpoint”, that is, something used by clinician to assess the disease
> status. But a very important point.
> > >
> > > All the best
> > > Fil
> > >
> > >> 2. You end the paper by discussing what needs to be done to make
> models used in medical settings.
> > >> Those topics deserve discussion in separate email threads. Yet for
> now I wish to make sure I understood your model correctly so I can
> implement it properly.
> > >>
> > >> Happy holidays
> > >>
> > >>
> > >> Jacob
> > >>
> > >>
> > >> On Tue, Dec 22, 2020, 13:38 Filippo Castiglione <
> filippo.castiglione at gmail.com> wrote:
> > >> Dear Jacob, the preprint is out. You can have a look at it at
> > >>
> > >> https://biorxiv.org/cgi/content/short/2020.12.20.423670v1
> > >>
> > >> However you might not need to read it to figure out your probability
> function.
> > >> In fact, the curve I sent you last time is
> > >>
> > >> f(t) = 1-P(t)
> > >>
> > >> where P(t) is the probability of dying at time t. To be precise you
> have to take
> > >>
> > >> P(t) = 1- f(t)/100
> > >>
> > >> because f(t) is expressed in percent.
> > >>
> > >> it is of course a rough approximation with rough assumptions (e.g.,
> the choice of the cutoff theta I was mentioning in my last mail) but might
> be a starting point (!?)
> > >>
> > >> Hope it helps.
> > >> All the best
> > >> Fil
> > >>
> > >>
> > >>> On 22 Dec 2020, at 10:42, Jacob Barhak <jacob.barhak at gmail.com>
> wrote:
> > >>>
> > >>> Thanks Filippo,
> > >>>
> > >>> Your example is nice since you include a functional form that can be
> reproduced fairly easily by plugging in the coefficients. However, it seems
> I will have to wait until you upload the paper to biorxiv so I can better
> understand what you are showing and if I can reuse it. Allow me to explain
> my thoughts.
> > >>>
> > >>> Your plot indicates that after 80 days only about 10% of the
> population have a viral load larger than theta. Also you indicate that your
> arbitrary death definition is viral load theta above 1000 in day 30.
> > >>>
> > >>> What I am looking for is a little bit different, What I want is the
> probability of death as a function on time. For example, I assume the
> probability at the day of infection will be zero since the virus did not
> spread in the body and the probability will then gradually rise as days
> pass until a peak and then drop again and will be close to zero after
> several weeks when subjects recover and can no longer die from the disease.
> > >>>
> > >>> The question is if it is possible to derive such a function from
> your data? - and if yes, exactly how?
> > >>>
> > >>> When integrating models, many time the issue is that the models
> presented in the publication need some manipulation, Many times this
> manipulation is trivial, yet many other times it requires some
> transformation and deep comprehension of the details and sometimes the
> model is unusable for some purposes because it calculates something
> different. This is what I am trying to figure out in this case and I need
> more information.
> > >>>
> > >>> Unless you can help extract the function I am looking for before
> publication, I will have to wait for the preprint and will be happy to
> continue this public conversation on this thread.
> > >>>
> > >>> Jacob
> > >>>
> > >>> On Mon, Dec 21, 2020 at 4:46 AM Filippo Castiglione <
> filippo.castiglione at gmail.com> wrote:
> > >>> Dear Jacob, please see attachment as a reply to your mail.
> > >>>
> > >>> I could parametrise the “model” further with respect to patient’s
> age but I though the reply in attachment might be already ok for your
> purpose. You tell me.
> > >>>
> > >>> As for the permission to use it for commercial purposes I can
> declare no restrictions (so far ;).
> > >>>
> > >>> Best wishes
> > >>> Fil
> > >>>
> > >>>
> > >>>
> > >>>> On 20 Dec 2020, at 23:53, Jacob Barhak <jacob.barhak at gmail.com>
> wrote:
> > >>>>
> > >>>> Well Filippo,
> > >>>>
> > >>>> Since the discussion is relevant to the comorbidities subgroup and
> to the integration subgroup, you will find both mailing lists CCd. I
> suggest we try to keep this discussion public and merge it with the working
> group activities. I believe others will find interest in this discussion as
> it may be relevant to their activities. I am copying the entire email
> conversation below.
> > >>>>
> > >>>> To your question about if I am suggesting to use your whole model
> to simulate mortality?
> > >>>>
> > >>>> The answer is: almost.
> > >>>>
> > >>>> Your model has important utility towards calculating mortality.
> However, I only need your output at this point. What I am asking for is the
> output of your model as a function. Something like:
> > >>>>
> > >>>> probability_death = f (days since infection)
> > >>>>
> > >>>> And at this initial point f does not have to be in functional form.
> It is enough for simulation purposes to have an x,y table where x is from 0
> to 60 in days for each day and the associated mortality probability.
> > >>>>
> > >>>> And it is ok if you give me several assumptions with different
> cutoff values in your model.
> > >>>>
> > >>>> The ensemble model will balance those assumptions with other models
> and assumptions. If the assumptions do not merge well with other
> assumptions/models, the ensemble model should assign them less influence.
> > >>>>
> > >>>> So if you can use the model you showed at the webinar to derive
> different mortality functions and send those to me as a table - it will be
> great.
> > >>>>
> > >>>> However, please also make sure you give me permission to use those
> for commercial purposes - I do intend to monetize my technology and
> therefore avoid using information that has restrictions. I did declare a
> conflict of interest and I am making it public. However, I hope we can
> still collaborate.
> > >>>>
> > >>>> Jacob
> > >>>>
> > >>>> --
> > >>>> Jacob Barhak Ph.D.
> > >>>> Sole Proprietor, Software Developer, and Computational Disease
> Modeler
> > >>>> https://sites.google.com/view/jacob-barhak/home
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Sun, Dec 20, 2020 at 2:40 AM Filippo Castiglione <
> filippo.castiglione at gmail.com> wrote:
> > >>>> Dear Jacob, sure this conversation is open.
> > >>>>
> > >>>> Coming back to the mortality model question are you suggesting to
> use my whole model to simulate mortality? If yes, then It is possible at
> the condition that we define when the simulation should stop for “death”.
> > >>>>
> > >>>>
> > >>>> Best
> > >>>>
> > >>>> --- Filippo (@iPhone)
> > >>>>
> > >>>>> On 19 Dec 2020, at 21:48, Jacob Barhak <jacob.barhak at gmail.com>
> wrote:
> > >>>>>
> > >>>>>
> > >>>>> Thanks Filippo,
> > >>>>>
> > >>>>> Yes, you are correct, the mortality models are not accurate. All
> models are assumptions - none of them are really accurate. The Reference
> Model was built to figure out how well they perform and provide a reference
> modelers can compare their models to.
> > >>>>>
> > >>>>> Your suggestions make sense and in fact I already have several
> mortality models plugged into my model. Yet those are simplified and not
> multi scale models.
> > >>>>>
> > >>>>> The value in your approach is that it makes this multi scale - if
> you would plug your model in the ensemble and the ensemble will prefer it
> to other models - this will give the multi scale model validation of sorts.
> > >>>>>
> > >>>>> And with your permission, I would like to add this conversation to
> the co-morbidities mailing list that I plan to create or to the integration
> mailing list in the integration working group portal:
> > >>>>> https://lists.simtk.org/pipermail/vp-integration-subgroup/
> > >>>>>
> > >>>>> Hopefully you are ok with making this conversation public and part
> of the working group discussions.
> > >>>>>
> > >>>>> Jacob
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Sat, Dec 19, 2020 at 2:48 AM Filippo Castiglione <
> f.castiglione at iac.cnr.it> wrote:
> > >>>>> Hi Jacob.
> > >>>>>
> > >>>>> The only thing I feel to suggest is to use survival curves of
> covid patients. I don’t know if you can recall them somewhere from clinical
> available data (e.g.,
> https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-020-09721-2).
>
> > >>>>> You might be interested in age-related of such curves
> (Kaplan-Meier curves plot 1-P(death)).
> > >>>>>
> > >>>>> I might calculated them starting from my simulations but they
> would be inaccurate because based on an artificial definition of “death”
> that, of course, I don’t have. Let me know if you want to try it anyway but
> I suggest to look for them around.
> > >>>>>
> > >>>>> Another very simple stylised model of mortality would be to use a
> reverse sigmoid function to represent the upside-down sigmoid shape (long
> right tail) of a generic Kaplan-Meier plot centred around the "typical
> value” time of death. For instance such t could be in the range 7-15 days,
> for instance, 1-0.2/(1+exp(7-x)) where 1-0.2 is the asymptotic value and 7
> is the typical death t.
> > >>>>>
> > >>>>> Let me know if I expressed my thoughts clearly.
> > >>>>>
> > >>>>> All the best
> > >>>>> Fil
> > >>>>>
> > >>>>>> On 18 Dec 2020, at 18:38, Jacob Barhak <jacob.barhak at gmail.com>
> wrote:
> > >>>>>>
> > >>>>>> Hi Filippo,
> > >>>>>>
> > >>>>>> Following your talk at the webinar yesterday, you may recall that
> I was interested in a mortality model based on time.
> > >>>>>>
> > >>>>>> I wanted to communicate with you about it to see what is possible.
> > >>>>>>
> > >>>>>> I am looking for models to plug into my ensemble model. You can
> see quickly how it looks in this video - I am pointing at the point where I
> show some ensemble results:
> > >>>>>> https://youtu.be/aTB8-XEZheU?t=1000
> > >>>>>>
> > >>>>>> I am actively looking for several types of models to integrate
> into my model. One of the models can be mortality models. Currently I use
> two different models for mortality - one of overall risk and one for death.
> I do not have a probability of death per day from infection and looking at
> your plots I figured you may have something - even at a level of an
> assumption.
> > >>>>>>
> > >>>>>> Do you think it is possible to get such models from you and if
> so, under what terms?
> > >>>>>>
> > >>>>>> Hopefully you can help.
> > >>>>>>
> > >>>>>> Jacob
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Mon, Dec 7, 2020 at 7:57 AM Jacob Barhak <
> jacob.barhak at gmail.com> wrote:
> > >>>>>> Ok Filippo,
> > >>>>>>
> > >>>>>> You are in a similar situation to many others.
> > >>>>>>
> > >>>>>> Your situation is very common and well understood.
> > >>>>>>
> > >>>>>> Jacob
> > >>>>>>
> > >>>>>> On Mon, Dec 7, 2020, 02:37 Filippo Castiglione <
> f.castiglione at iac.cnr.it> wrote:
> > >>>>>> Dear Jacob, sorry for the late response.
> > >>>>>>
> > >>>>>> I recently realised I had subscribed to too many groups to be
> practically able to follow, so opted out from many of them included the
> comorbidity. I asked Bruce to update the groups’ lists.
> > >>>>>>
> > >>>>>> All the best
> > >>>>>> Fil
> > >>>>>>
> > >>>>>>> On 6 Dec 2020, at 06:23, Jacob Barhak <jacob.barhak at gmail.com>
> wrote:
> > >>>>>>>
> > >>>>>>> Hi Filippo,
> > >>>>>>>
> > >>>>>>> You were listed as interested in comorbidities within the viral
> pandemic working group. I am going over each member personally to see why
> they signed up.
> > >>>>>>>
> > >>>>>>> I looked at your web site:
> > >>>>>>>
> > >>>>>>> https://www.iac.cnr.it/~filippo/about-me.html
> > >>>>>>>
> > >>>>>>> Your interest in co-morbidities is visible directly from what
> you state in your web site:
> > >>>>>>> "particular interest in the immune system and related
> pathologies"
> > >>>>>>>
> > >>>>>>> I wonder why you did not respond to my email to step up and
> co-lead this sub-working group. After all you have many publications and
> experience in leadership.
> > >>>>>>>
> > >>>>>>> Hopefully my personal email will get a response.
> > >>>>>>>
> > >>>>>>> Jacob
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> Jacob Barhak Ph.D.
> > >>>>>>> https://sites.google.com/view/jacob-barhak/home
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> > >>
> > >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.simtk.org/pipermail/vp-integration-subgroup/attachments/20201231/65b55b11/attachment-0001.html>
More information about the Vp-integration-subgroup
mailing list