[Vp-integration-subgroup] About using a multi-scale mortality model in the ensemble
Jacob Barhak
jacob.barhak at gmail.com
Thu Dec 31 17:47:21 PST 2020
And Fil,
One more observation,
Since this function we discussed is a density function, the probability of
mortality on a certain day should be 3 times larger as well. I corrected
the Github repository to reflect this.
https://github.com/Jacob-Barhak/COVID19Models/tree/main/COVID19_Mortality_Castiglione
Please verify this is correct and the numbers the plots are showing make
sense.
Meanwhile I wish you a happy new year.
Jacob
On Thu, Dec 31, 2020 at 3:27 AM Jacob Barhak <jacob.barhak at gmail.com> wrote:
> Ok Fil,
>
> Your corrected model code is now on Github at:
>
> https://github.com/Jacob-Barhak/COVID19Models/tree/main/COVID19_Mortality_Castiglione
>
> I chose MIT license for now, just to make sure you are comfortable with
> this initially. I would have liked to put it under CC0 license so that it
> can be more easily reused - however, I cannot really do it without you
> understanding the consequences - after all it means you giving away your IP
> - think about it as a gift to the world - it will make any type of reuse
> much easier for various purposes. In fact I am not the only one
> recommending this license, yet I cannot make the decision unless the
> creators approve.
>
> Here is a link to the license language:
> https://creativecommons.org/choose/zero/
>
> If after you read the language you approve the change, I will change the
> license to CC0.
>
> Yet for now I am more interested in how to plug your model into the
> ensemble.
>
> The Reference Model was recently using a certain approach to
> calculate mortality - it had the probability of mortality extracted as a
> function of age from 3 models without deciding on time and the time of
> mortality extracted from merging 2 models. The model would know ahead of
> time when the individual might die according to the mortality time model
> and then test a random number against this threshold mortality probability
> at that time during simulation for infected individuals. I did not have a
> density function I can build upon like the one you provided. Basically what
> I had is a spike function for each individual so using your function will
> make the ensemble much richer since it replaces a combination of multiple
> model components, yet it requires a different approach to modeling
> mortality and recovery since I need to merge the spike functions with your
> smoother distribution somehow. I will need to think of the best way to do
> this and it may take a few days.
>
> Since I need now to also remodel recovery, it bring us back to the issue
> you raised of not having a defined state and instead having a continuous
> function with a cutoff required.
>
> Also, recall that in an ensemble there are multiple other models that
> influence the result - the system will pick a mixture of models that work
> together best to explain the phenomenon observed. So it is more about the
> contribution of each model and sometimes it is hard to predict which models
> will work well together. Yet I think that having more models will help us
> better understand things, so your model will for sure contribute to
> understanding once I figure out how to merge it.
>
> Hopefully I can get it done soon. Until then thank you for your fast
> responses.
>
> Jacob
>
>
>
>
> On Wed, Dec 30, 2020 at 2:33 AM Filippo Castiglione <
> filippo.castiglione at gmail.com> wrote:
>
>> Hi Jacob, it looks all good to me but the definition of time (which I
>> forgot to tell you before, sorry ;) that in the simulator (therefore in the
>> data I sent you and therefore in the mortality model) is 8 hours: it just
>> means that you have to divide time by 3 to get days. So the period is 30
>> days, not 90 days. Apart from that, it all looks great. Thanks for the
>> copyright. Feel free to put it on you git with the license you see fit.
>>
>> I am curious to see how it works in your ensamble model.
>>
>> Best wishes
>> Fil
>>
>> > On 30 Dec 2020, at 08:47, Jacob Barhak <jacob.barhak at gmail.com> wrote:
>> >
>> > Thanks Fil,
>> >
>> > You helped a lot. However, to make sure we are on the same page, I
>> decided to implement the function using python and show you the plot - just
>> to make sure I implemented your instructions correctly - I rather show you
>> my understanding so you can tell me if this looks reasonable to you.
>> >
>> > Attached is the link to an archive that contains the code and the
>> output.
>> >
>> https://drive.google.com/file/d/12AU3xsjUk8xVBbb7UmO4hE-ldGj_QJ08/view?usp=sharing
>> >
>> > Once you open the archive you should see the interactive plot inside as
>> an html file - you should be able to open it with a modern browser like
>> chrome, edge, or firefox
>> >
>> > As for the code I assigned copyright to you and added some
>> documentation. If you have someone in your team familiar with python , you
>> just need to install holoviews and panel to create the plots. I did not put
>> the code in Github - I leave this to you - I am just interested in using
>> the function in my ensemble and need verification that this is correct. Yet
>> if you want, I will be happy to upload it to my github account if we decide
>> on a proper license like MIT / BSD or even preferred CC0 to release this to
>> the public without copyright restrictions.
>> >
>> > For now, please just verify this is correct - I noticed peak mortality
>> increases by age from 50 days to 61 days - is this correct or did I
>> misunderstand something?
>> >
>> > Hopefully this is correct and I can continue to plug in the model into
>> the ensemble.
>> >
>> > Jacob
>> >
>> >
>> >
>> >
>> > On Sun, Dec 27, 2020 at 11:13 PM Filippo Castiglione <
>> filippo.castiglione at gmail.com> wrote:
>> > Jacob,
>> >
>> > I forgot to say that after you have calculated the function F’, which
>> is the probability-density-function of the event “death”, you have to
>> multiply to an age-dependent coefficient which can be found in fig.3 panel
>> A of my paper divided by 100 (because there it is expressed in percent).
>> >
>> > In other words for instance the probability to die at day x with
>> respect to the age-class 40-49 is
>> >
>> > 1.25/100 * F’(x,a,b,c) with
>> >
>> > a=95.351153808322
>> > b=0.173066025988826
>> > c=51.92498287875
>> >
>> > Sorry for the lack of precision.
>> > I hope it is clearer now.
>> > Fil
>> >
>> >
>> >
>> > > On 27 Dec 2020, at 15:51, Filippo Castiglione <
>> filippo.castiglione at gmail.com> wrote:
>> > >
>> > > Hi Jacob, let me reply below
>> > >
>> > >> I was thinking of a function that defines the probability of
>> mortality as a function of time since infection, with the good chance of
>> surviving. This function should look like figure 2 in your paper where
>> higher age individuals have higher chance of dying, yet after a certain
>> point in time those who survived no longer can die, indicating recovery.
>> > >>
>> > >> It seems that the function you suggested is a little different. If I
>> understand you correctly, you already stratified the group that will die
>> and you are trying to show the probability of an individual dying in a
>> certain day from infection knowing they will die.
>> > >
>> > > Ok (in my last reply I was incorrect), then the function you are
>> looking for is the derivative of the one I sent you (normalised dividing by
>> 100), namely,
>> > >
>> > > F(x,a,b,c) = S(x,a,b,c) / 100
>> > >
>> > > and its derivative is
>> > >
>> > > F’(x,a,b,c) = (a * b exp(-b * (-c + x))) / (100 (1 + exp(-b *
>> (-c + x)))^2)
>> > >
>> > > Now the values of the parameters a,b,c are those I sent you last time
>> IF
>> > > we consider the age-UNstratified population.
>> > >
>> > > Otherwise, if we go by age (age ranges as in the paper), then the
>> parameters are:
>> > >
>> > > age range 0-39
>> > > a=96.56299632152
>> > > b=0.185777403544838
>> > > c=49.8164210315686
>> > >
>> > > age range 40-49
>> > > a=95.351153808322
>> > > b=0.173066025988826
>> > > c=51.92498287875
>> > >
>> > > age range 50-59
>> > > a=94.2976479245251
>> > > b=0.165018635066338
>> > > c=54.1494851021362
>> > >
>> > > age range 60-69
>> > > a=91.3990353489634
>> > > b=0.158299883031816
>> > > c=55.5935133562513
>> > >
>> > > age range 70-79
>> > > a=88.9048923236574
>> > > b=0.152924609509616
>> > > c=57.7447222979953
>> > >
>> > > age range 80+
>> > > a=85.6674967019556
>> > > b=0.14040123117131
>> > > c=61.383806850923
>> > >
>> > >> Please verify that I am correct. Since I need to know how to use
>> your function.
>> > >
>> > > Let me know if F’ (F-prime) is the function you were looking for. It
>> should be it.
>> > >
>> > >> You have two more interesting elements in the paper that can start
>> interesting discussions:
>> > >>
>> > >> 1. You approach the definition of a comorbidity by a level of damage
>> to tissue.
>> > >
>> > > Yes, given the elements in my simulation, that is a possible choice
>> for the definition (not really of the comorbidity) of the “severity” of the
>> disease. It is a tricky point to link some computer output to a “clinical
>> endpoint”, that is, something used by clinician to assess the disease
>> status. But a very important point.
>> > >
>> > > All the best
>> > > Fil
>> > >
>> > >> 2. You end the paper by discussing what needs to be done to make
>> models used in medical settings.
>> > >> Those topics deserve discussion in separate email threads. Yet for
>> now I wish to make sure I understood your model correctly so I can
>> implement it properly.
>> > >>
>> > >> Happy holidays
>> > >>
>> > >>
>> > >> Jacob
>> > >>
>> > >>
>> > >> On Tue, Dec 22, 2020, 13:38 Filippo Castiglione <
>> filippo.castiglione at gmail.com> wrote:
>> > >> Dear Jacob, the preprint is out. You can have a look at it at
>> > >>
>> > >> https://biorxiv.org/cgi/content/short/2020.12.20.423670v1
>> > >>
>> > >> However you might not need to read it to figure out your probability
>> function.
>> > >> In fact, the curve I sent you last time is
>> > >>
>> > >> f(t) = 1-P(t)
>> > >>
>> > >> where P(t) is the probability of dying at time t. To be precise you
>> have to take
>> > >>
>> > >> P(t) = 1- f(t)/100
>> > >>
>> > >> because f(t) is expressed in percent.
>> > >>
>> > >> it is of course a rough approximation with rough assumptions (e.g.,
>> the choice of the cutoff theta I was mentioning in my last mail) but might
>> be a starting point (!?)
>> > >>
>> > >> Hope it helps.
>> > >> All the best
>> > >> Fil
>> > >>
>> > >>
>> > >>> On 22 Dec 2020, at 10:42, Jacob Barhak <jacob.barhak at gmail.com>
>> wrote:
>> > >>>
>> > >>> Thanks Filippo,
>> > >>>
>> > >>> Your example is nice since you include a functional form that can
>> be reproduced fairly easily by plugging in the coefficients. However, it
>> seems I will have to wait until you upload the paper to biorxiv so I can
>> better understand what you are showing and if I can reuse it. Allow me to
>> explain my thoughts.
>> > >>>
>> > >>> Your plot indicates that after 80 days only about 10% of the
>> population have a viral load larger than theta. Also you indicate that your
>> arbitrary death definition is viral load theta above 1000 in day 30.
>> > >>>
>> > >>> What I am looking for is a little bit different, What I want is the
>> probability of death as a function on time. For example, I assume the
>> probability at the day of infection will be zero since the virus did not
>> spread in the body and the probability will then gradually rise as days
>> pass until a peak and then drop again and will be close to zero after
>> several weeks when subjects recover and can no longer die from the disease.
>> > >>>
>> > >>> The question is if it is possible to derive such a function from
>> your data? - and if yes, exactly how?
>> > >>>
>> > >>> When integrating models, many time the issue is that the models
>> presented in the publication need some manipulation, Many times this
>> manipulation is trivial, yet many other times it requires some
>> transformation and deep comprehension of the details and sometimes the
>> model is unusable for some purposes because it calculates something
>> different. This is what I am trying to figure out in this case and I need
>> more information.
>> > >>>
>> > >>> Unless you can help extract the function I am looking for before
>> publication, I will have to wait for the preprint and will be happy to
>> continue this public conversation on this thread.
>> > >>>
>> > >>> Jacob
>> > >>>
>> > >>> On Mon, Dec 21, 2020 at 4:46 AM Filippo Castiglione <
>> filippo.castiglione at gmail.com> wrote:
>> > >>> Dear Jacob, please see attachment as a reply to your mail.
>> > >>>
>> > >>> I could parametrise the “model” further with respect to patient’s
>> age but I though the reply in attachment might be already ok for your
>> purpose. You tell me.
>> > >>>
>> > >>> As for the permission to use it for commercial purposes I can
>> declare no restrictions (so far ;).
>> > >>>
>> > >>> Best wishes
>> > >>> Fil
>> > >>>
>> > >>>
>> > >>>
>> > >>>> On 20 Dec 2020, at 23:53, Jacob Barhak <jacob.barhak at gmail.com>
>> wrote:
>> > >>>>
>> > >>>> Well Filippo,
>> > >>>>
>> > >>>> Since the discussion is relevant to the comorbidities subgroup and
>> to the integration subgroup, you will find both mailing lists CCd. I
>> suggest we try to keep this discussion public and merge it with the working
>> group activities. I believe others will find interest in this discussion as
>> it may be relevant to their activities. I am copying the entire email
>> conversation below.
>> > >>>>
>> > >>>> To your question about if I am suggesting to use your whole model
>> to simulate mortality?
>> > >>>>
>> > >>>> The answer is: almost.
>> > >>>>
>> > >>>> Your model has important utility towards calculating mortality.
>> However, I only need your output at this point. What I am asking for is the
>> output of your model as a function. Something like:
>> > >>>>
>> > >>>> probability_death = f (days since infection)
>> > >>>>
>> > >>>> And at this initial point f does not have to be in functional
>> form. It is enough for simulation purposes to have an x,y table where x is
>> from 0 to 60 in days for each day and the associated mortality probability.
>> > >>>>
>> > >>>> And it is ok if you give me several assumptions with different
>> cutoff values in your model.
>> > >>>>
>> > >>>> The ensemble model will balance those assumptions with other
>> models and assumptions. If the assumptions do not merge well with other
>> assumptions/models, the ensemble model should assign them less influence.
>> > >>>>
>> > >>>> So if you can use the model you showed at the webinar to derive
>> different mortality functions and send those to me as a table - it will be
>> great.
>> > >>>>
>> > >>>> However, please also make sure you give me permission to use those
>> for commercial purposes - I do intend to monetize my technology and
>> therefore avoid using information that has restrictions. I did declare a
>> conflict of interest and I am making it public. However, I hope we can
>> still collaborate.
>> > >>>>
>> > >>>> Jacob
>> > >>>>
>> > >>>> --
>> > >>>> Jacob Barhak Ph.D.
>> > >>>> Sole Proprietor, Software Developer, and Computational Disease
>> Modeler
>> > >>>> https://sites.google.com/view/jacob-barhak/home
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> On Sun, Dec 20, 2020 at 2:40 AM Filippo Castiglione <
>> filippo.castiglione at gmail.com> wrote:
>> > >>>> Dear Jacob, sure this conversation is open.
>> > >>>>
>> > >>>> Coming back to the mortality model question are you suggesting to
>> use my whole model to simulate mortality? If yes, then It is possible at
>> the condition that we define when the simulation should stop for “death”.
>> > >>>>
>> > >>>>
>> > >>>> Best
>> > >>>>
>> > >>>> --- Filippo (@iPhone)
>> > >>>>
>> > >>>>> On 19 Dec 2020, at 21:48, Jacob Barhak <jacob.barhak at gmail.com>
>> wrote:
>> > >>>>>
>> > >>>>>
>> > >>>>> Thanks Filippo,
>> > >>>>>
>> > >>>>> Yes, you are correct, the mortality models are not accurate. All
>> models are assumptions - none of them are really accurate. The Reference
>> Model was built to figure out how well they perform and provide a reference
>> modelers can compare their models to.
>> > >>>>>
>> > >>>>> Your suggestions make sense and in fact I already have several
>> mortality models plugged into my model. Yet those are simplified and not
>> multi scale models.
>> > >>>>>
>> > >>>>> The value in your approach is that it makes this multi scale - if
>> you would plug your model in the ensemble and the ensemble will prefer it
>> to other models - this will give the multi scale model validation of sorts.
>> > >>>>>
>> > >>>>> And with your permission, I would like to add this conversation
>> to the co-morbidities mailing list that I plan to create or to the
>> integration mailing list in the integration working group portal:
>> > >>>>> https://lists.simtk.org/pipermail/vp-integration-subgroup/
>> > >>>>>
>> > >>>>> Hopefully you are ok with making this conversation public and
>> part of the working group discussions.
>> > >>>>>
>> > >>>>> Jacob
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>> On Sat, Dec 19, 2020 at 2:48 AM Filippo Castiglione <
>> f.castiglione at iac.cnr.it> wrote:
>> > >>>>> Hi Jacob.
>> > >>>>>
>> > >>>>> The only thing I feel to suggest is to use survival curves of
>> covid patients. I don’t know if you can recall them somewhere from clinical
>> available data (e.g.,
>> https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-020-09721-2).
>>
>> > >>>>> You might be interested in age-related of such curves
>> (Kaplan-Meier curves plot 1-P(death)).
>> > >>>>>
>> > >>>>> I might calculated them starting from my simulations but they
>> would be inaccurate because based on an artificial definition of “death”
>> that, of course, I don’t have. Let me know if you want to try it anyway but
>> I suggest to look for them around.
>> > >>>>>
>> > >>>>> Another very simple stylised model of mortality would be to use a
>> reverse sigmoid function to represent the upside-down sigmoid shape (long
>> right tail) of a generic Kaplan-Meier plot centred around the "typical
>> value” time of death. For instance such t could be in the range 7-15 days,
>> for instance, 1-0.2/(1+exp(7-x)) where 1-0.2 is the asymptotic value and 7
>> is the typical death t.
>> > >>>>>
>> > >>>>> Let me know if I expressed my thoughts clearly.
>> > >>>>>
>> > >>>>> All the best
>> > >>>>> Fil
>> > >>>>>
>> > >>>>>> On 18 Dec 2020, at 18:38, Jacob Barhak <jacob.barhak at gmail.com>
>> wrote:
>> > >>>>>>
>> > >>>>>> Hi Filippo,
>> > >>>>>>
>> > >>>>>> Following your talk at the webinar yesterday, you may recall
>> that I was interested in a mortality model based on time.
>> > >>>>>>
>> > >>>>>> I wanted to communicate with you about it to see what is
>> possible.
>> > >>>>>>
>> > >>>>>> I am looking for models to plug into my ensemble model. You can
>> see quickly how it looks in this video - I am pointing at the point where I
>> show some ensemble results:
>> > >>>>>> https://youtu.be/aTB8-XEZheU?t=1000
>> > >>>>>>
>> > >>>>>> I am actively looking for several types of models to integrate
>> into my model. One of the models can be mortality models. Currently I use
>> two different models for mortality - one of overall risk and one for death.
>> I do not have a probability of death per day from infection and looking at
>> your plots I figured you may have something - even at a level of an
>> assumption.
>> > >>>>>>
>> > >>>>>> Do you think it is possible to get such models from you and if
>> so, under what terms?
>> > >>>>>>
>> > >>>>>> Hopefully you can help.
>> > >>>>>>
>> > >>>>>> Jacob
>> > >>>>>>
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> On Mon, Dec 7, 2020 at 7:57 AM Jacob Barhak <
>> jacob.barhak at gmail.com> wrote:
>> > >>>>>> Ok Filippo,
>> > >>>>>>
>> > >>>>>> You are in a similar situation to many others.
>> > >>>>>>
>> > >>>>>> Your situation is very common and well understood.
>> > >>>>>>
>> > >>>>>> Jacob
>> > >>>>>>
>> > >>>>>> On Mon, Dec 7, 2020, 02:37 Filippo Castiglione <
>> f.castiglione at iac.cnr.it> wrote:
>> > >>>>>> Dear Jacob, sorry for the late response.
>> > >>>>>>
>> > >>>>>> I recently realised I had subscribed to too many groups to be
>> practically able to follow, so opted out from many of them included the
>> comorbidity. I asked Bruce to update the groups’ lists.
>> > >>>>>>
>> > >>>>>> All the best
>> > >>>>>> Fil
>> > >>>>>>
>> > >>>>>>> On 6 Dec 2020, at 06:23, Jacob Barhak <jacob.barhak at gmail.com>
>> wrote:
>> > >>>>>>>
>> > >>>>>>> Hi Filippo,
>> > >>>>>>>
>> > >>>>>>> You were listed as interested in comorbidities within the viral
>> pandemic working group. I am going over each member personally to see why
>> they signed up.
>> > >>>>>>>
>> > >>>>>>> I looked at your web site:
>> > >>>>>>>
>> > >>>>>>> https://www.iac.cnr.it/~filippo/about-me.html
>> > >>>>>>>
>> > >>>>>>> Your interest in co-morbidities is visible directly from what
>> you state in your web site:
>> > >>>>>>> "particular interest in the immune system and related
>> pathologies"
>> > >>>>>>>
>> > >>>>>>> I wonder why you did not respond to my email to step up and
>> co-lead this sub-working group. After all you have many publications and
>> experience in leadership.
>> > >>>>>>>
>> > >>>>>>> Hopefully my personal email will get a response.
>> > >>>>>>>
>> > >>>>>>> Jacob
>> > >>>>>>>
>> > >>>>>>> --
>> > >>>>>>> Jacob Barhak Ph.D.
>> > >>>>>>> https://sites.google.com/view/jacob-barhak/home
>> > >>>>>>>
>> > >>>>>>
>> > >>>>>
>> > >>>
>> > >>
>> > >
>> >
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.simtk.org/pipermail/vp-integration-subgroup/attachments/20201231/64cbef53/attachment-0001.html>
More information about the Vp-integration-subgroup
mailing list