[Population Modeling] Discussing the Population Modeling panel in SummerSim
Stefan Scholz
stefan.scholz at uni-bielefeld.de
Fri Mar 4 02:12:55 PST 2016
Dear all,
could we agree on a mixture of the topics, like "data sources, parameter
estimation and calibration"? My topic would be on "Social (media)
network data in models in the absence of survey data: An example of the
German MSM-population". It would take some creativity to fit that into
"model calibration" ;-)
I hope this comes not to late!
Thanks,
Stefan
Am 16.02.2016 um 17:44 schrieb Dammann, Olaf:
>
> All:
>
> I like “model calibration” quite a bit.
>
> My paper would be on “Model calibration: Four levels of calibration –
> A critical appraisal”
>
> Thanks,
>
> Olaf
>
> *From:*Jacob Barhak [mailto:jacob.barhak at gmail.com]
> *Sent:* Tuesday, February 16, 2016 5:36 AM
> *To:* Jeljer Hoekstra
> *Cc:* Stefan Scholz; popmodwkgrpimag-news at simtk.org; Mélanie Prague;
> Dammann, Olaf
> *Subject:* Re: [Population Modeling] Discussing the Population
> Modeling panel in SummerSim
>
> Hi Melanie, Hi Jeljer, Hi Stefan, Hi Olaf,
>
> Since the 14-March deadline for paper submission is coming closer, and
> since we were discussing a panel, I would appreciate it if you can
> decide on a common prefix for the panel.
>
> So far we had a discussion revolving around estimation, validation,
> calibration. Please choose a common denominator title to fit all your
> work that will form a base for the panel and submit it to the list.
>
> So far, here are the titles you suggested as I extracted them from
> your posts:
>
> Melanie: 'simulation vs. estimation’
>
> Olaf: "data sources, constraints, validation issues"
>
> Stefan & Jeljer: "model calibration" -
>
> The last topic of "Model Calibration" seems to be a common denominator
> so far, yet still possible for debate - after all you had some nice
> discussions and may have a better idea.
>
> Yet if the last topic prefix is comfortable to you all, then I ask
> that you will submit your paper title to the mailing list to set the
> expectations from the panel and leave you all sufficient time to write
> the short paper.
>
> I look forward to see your paper titles.
>
> Jacob
>
> On Mon, Feb 8, 2016 at 9:29 AM, Jeljer Hoekstra
> <jeljer.hoekstra at rivm.nl <mailto:jeljer.hoekstra at rivm.nl>> wrote:
>
> Dear Jacob, Stefan and others,
>
> I agree with Jacob that estimation and calibration generate parameter
> values based on observed data. I never explicitly thought about in
> those terms but that is what it is.
> I have always used the terms in combination; calibrate a model and
> estimate a parameter. I don't consider the one an automated search and
> the other a manual human search per se . Usually if you estimate a
> parameter the focus is on one parameter and a lot of statistical
> theory and software exist to do that automatically. Whereas if you
> calibrate a model the focus is often on all or a group of parameters
> in the model, which is perhaps somewhat messier and needs more human
> interference. I am not sure though.
>
> I am interessted in the topic Stefan raised. What if you find
> parameters in the literature or you have estimated them yourself with
> some dataset and then you test and adjust those variables (calibrate?)
> so your model replicates some other dataset better. How much change
> in those parameters do you accept, keeping in mind that the parameter
> may be interpreted slightly different in the calibrated model.
> Furthermore, if you calibrate your model you need some goodness of fit
> criterium. I wonder if people have experience with weighing output
> variables including a mixture of categorical variables (e.g.
> dead/alive, smoking) and continous variables( e.g. BMI, cholesterol
> levels).
>
> Validation is a related subject. I consider a model validated if it
> can mimic, to some degree a dataset that was not used to
> calibrate/estimate it. Obviously also here you will need some goodness
> of fit criterium to see if the model is validated or not. In my
> experience we do not often have the luxury of a complete extra dataset
> for validation. So we end up in the discussion Stefan mentioned, if
> the model is calibrated how far do we accept parameters to be
> different from those estimated elsewhere.
>
> @stefan, thanks for poining out GAMLSS we will have a look.
>
> best wishes
> Jeljer
>
>
>
> From: Jacob Barhak <jacob.barhak at gmail.com
> <mailto:jacob.barhak at gmail.com>>
> To: Stefan Scholz <stefan.scholz at uni-bielefeld.de
> <mailto:stefan.scholz at uni-bielefeld.de>>,
> Cc: "popmodwkgrpimag-news at simtk.org
> <mailto:popmodwkgrpimag-news at simtk.org>"
> <popmodwkgrpimag-news at simtk.org
> <mailto:popmodwkgrpimag-news at simtk.org>>, Mélanie Prague
> <melanie.prague at isped.u-bordeaux2.fr
> <mailto:melanie.prague at isped.u-bordeaux2.fr>>, Jeljer Hoekstra
> <jeljer.hoekstra at rivm.nl <mailto:jeljer.hoekstra at rivm.nl>>, "Dammann,
> Olaf" <Olaf.Dammann at tufts.edu <mailto:Olaf.Dammann at tufts.edu>>
> Date: 05/02/2016 00:50
> Subject: Re: [Population Modeling] Discussing the Population Modeling
> panel in SummerSim
>
> ------------------------------------------------------------------------
>
>
>
>
> So Stefan,
>
> Since this is part of the discussion on your panel topic, you will
> have to choose the topic and I will try not to intervene much. Yet do
> allow me to add a note regarding calibration and estimation.
>
> Both calibration and estimation generate model parameters as outputs
> from observed known data.
>
> Will you agree with me that the term calibration would be more
> appropriate to human manipulated parameters while estimation is
> perhaps more general term that includes automated machine algorithm
> methods?
>
> I have seen the term estimation used for parameter estimation using
> Delphi style human voting, so I think "estimation" would include
> "calibration" as a sub category.
>
> Even though there is always some sort of human input to the modeling
> process, it seems things are becoming more automated these days. What
> term would you use for heavily machine dependent estimation algorithms
> as opposed to human tightly controlled calibration? Is there a term
> for those methods anyone in the list prefers using?
>
> Hopefully this will contribute to the panel discussion.
>
> Jacob
>
>
> On Thu, Feb 4, 2016 at 4:28 AM, Stefan Scholz
> <stefan.scholz at uni-bielefeld.de
> <mailto:stefan.scholz at uni-bielefeld.de>> wrote:
> Dear all,
>
> I just want to add the issue of "model calibration" which seems to me
> as being related to estimation and validation. Just to give a short
> example what I mean by calibration: We use some information of a data
> set to estimate our model parameters, run the model based on those
> parameters and see that the results are not externally valid. We the
> iteratively change some of the input parameters until the model
> results are externally valid. I would find it very interesting to
> discuss if calibration should be performed and if so, how far from the
> originally estimated parameters you would accept your calibrated
> values to be. [I hope I am not stating the obvious or missed some
> guidance on this topic already available ;-) ]
>
> @ Jelster: If you use R, maybe the GAMLSS-package developed by Prof.
> Mikis Stasinopoulos is helpful to you. As far as I understand it, the
> package was developed from the need of parameter estimation in
> agent-based modeling. You can estimate all parameters of large list of
> probability distributions. So let's say you want to estimate the
> probability of getting diabetes conditional on age, sex, education,
> etc. You can estimate a general linear model using a beta-distribution
> and include the resulting coefficients to include them in your model.
> So, every person in your model can calculate the parameters of the
> beta-distribution based on their age, sex, education, etc. and you can
> draw random numbers from that distribution to determine whether a
> person gets diabetes or not.
>
> Best,
> Stefan
>
>
> Am 03.02.2016 um 01:45 schrieb Jacob Barhak:
>
> Hi Melanie, Hi Jeljer, Hi Olaf, Hi Stefan,
>
> It seems the panel is forming nicely. I will try to summarize what we
> had so far and help figure the rest.
>
> 1. The topic seems to revolve around "estimation and validation in
> population modeling" with some variations. If you are all ok with this
> general topic, I suggest we stick with it as a base.
>
> 2. It seems there is agreement on separate papers with the same title
> prefix. So please allocate time on writing a short 3 page paper. Since
> panelists are not closely affiliated, each will review the papers of
> another panelist which will contribute to panel cohesion since the
> panelists will influence each others final paper. Note that the review
> process is public and non-blind.
>
> 3. Presentations followed by a period of questions to all panelists
> seems to be the choice. I assume there will be 20-30 minutes per
> panelist, yet we will have to set timing once we know number of
> presentations.
>
> Olaf asked about other presenters. Yes, there will be other
> presentations by non panelists. In fact any one of you can choose to
> detach from the panel and submit a paper on their own. I will send a
> CFP to the list following this message.
>
> The difference for panelists would be:
> 1. Panelists will have some discount that SCS promised - I have no
> exact details yet. This makes sense since they will have more
> involvement.
> 2. Panelists will gain extra exposure which you are already getting
> with these communications.
> 3. If time is available, panelists will get more time for discussion
> beyond other presenters. I will communicate with organizers to see
> what is possible beyond that. Yet for now, assume the panel is part
> of the BMPM track.
>
> Note that SummerSim is a Multi-conference, so having a panel may
> attract more people. From the past, you should expect about 10-20 in
> the room for the presentation if last years are indicative. I suspect
> a panel can attract more.
>
> So for panelists still interested, please:
>
> 1. Confirm that you are ok with the topic and format by sending an
> email to this list. Or continue discussing the topic until consensus
> is reached. And you can split into two panels with separate topics, or
> announce you are interested in a paper outside the panel.
>
> 2. Start writing a short 3 page paper to submit to the SCS web site.
> Recall that the title prefix should be the same for all papers if you
> are in the panel.
>
> 3. Allocate time to review a paper or two by another panelist. This
> review will be public.
>
> Hopefully this explains the next steps and I hope more panelists would
> express interest in the topic forming.
>
> Jacob
>
>
>
> Dear All,
> I agree with Stefan re 1 and 2.
>
> 1. I like the idea of talking about input-output stuff - data sources,
> constraints, validation issues.
>
> 2. Fully agree with Stefan.
>
> 3. If we have slides, this should be flash talks, not longish formal
> presentations. I am still unclear whether we have presentations from
> conference participants who are NOT panelists?
>
> My 2 cents
> Olaf
>
>
> --
> Olaf Dammann, MD
> Professor of Public Health & Community Medicine
> Tufts University School of Medicine
> Boston, MA 02111
>
>
> On Jan 21, 2016, at 4:00 AM, Stefan Scholz
> <stefan.scholz at uni-bielefeld.de
> <mailto:stefan.scholz at uni-bielefeld.de>> wrote:
>
> Hi all,
>
> thanks for all your efforts! Here are my thoughts on the three points:
>
> 1. Topic: I am not quite sure whether the topics really do mean the
> same thing. I would understand Melanies suggestion as "what are the
> differences in the results/*outputs* from estimation vs. simulation"
> whereas I would understand the topic "estimation in population
> modeling" more as estimation of model *input *parameters. (Please tell
> me, if I got that wrong!) I think both are interesting topics and
> maybe we could bring them both together under the general topic of
> validity (external and internal). (i.e. how do we estimate model input
> to get externally valid results and how do we assess the latter)
>
> 2. I would vote for separate papers under the same topic, sharing the
> same prefix.
>
> 3. I would go for option B). Option A) is fine as well, but we should
> make clear if there are some contrary opinions on this topic. If
> panelists agree on almost every topic, this might get boring. Also, I
> would see the number of people in the audience a critical factor for a
> panel. If we are a small group, sharing the methods used for
> estimation and discussing it in the group might be more beneficial to
> all attendees.
>
> Best,
> Stefan
>
> Am 19.01.2016 um 19:42 schrieb Jacob Barhak:
> Hi Melanie, Hi Olaf, Hi Stefan, Hi Carl,
>
> You all expressed interest in appearing in a population modeling panel
> in SummerSim.
>
> Melanie also suggested a topic:
> "differences and extrapolation concerns around 'simulation vs.
> estimation’ in bio-medical area"
>
> At this point, I wish interested parties to discuss the following:
>
> 1. The topic - feel free to suggest alternative topics/titles and we
> can see how having the panel will contribute to the topic. Note that
> if we end up with different topics, it is also ok since others may
> join to support the topic you suggested. Hopefully there will be
> synergy, yet complementary topics or even different opinions are
> possible. This discussion itself is valuable.
>
> 2. Writing Format. The conference includes a paper. Part of the
> discussion should be how do you prefer to be published. Do you want a
> joint paper? Or would you like each to submit a short paper with
> similar topics? This would probably be tied to the topic you suggest.
> Yet note that whatever paper format chosen, it will undergo public
> non-blind review.
>
> 3. Presentation format: How would you like the talk to be?
> Possibilities include: A) Totally informal discussion where panelists
> converge amongst themselves, possibly with moderation and questions
> from he audience. B) Presentations with a projector of each panelist
> and then a period of questions. C) A combination of both, for example
> very short introductions with a projector and then a discussion.
> Assume half an hour per panelist, yet this may change.
>
> As a default starting point for discussion, allow me to suggest the
> following:
>
> 1. Topic Estimation in population modeling - its generalization for
> what Melanie suggested - feel free to reshape it any way comfortable
> to you.
>
> 2. Writing format: Very short separate papers using the topic as a
> title prefix. to have a common prefix fro all panelists. Here is an
> example: Estimation in population Modeling - application in Disease
> Models.
>
> 3. Presentation format: Short digital introductions of about 15
> minutes each - with only a few slides and a discussion that will start
> with expanding prepared topics encountered during discussions and
> review and then answering questions from the audience.
>
> This default can be changed during discussion.
>
> Please feel free to join this discussion if you are interested in
> appearing in a population modeling panel in SummerSim - even if you
> are not personally addressed. This post is initially directed to those
> who expressed interest on this list, yet we can certainly expand the
> scope to include more panelists, and I know of interest by others at
> this point.
>
> I look forward to your opinions.
>
> Jacob
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> PopModWkGrpIMAG-news mailing list
> PopModWkGrpIMAG-news at simtk.org <mailto:PopModWkGrpIMAG-news at simtk.org>
> https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
>
>
> --
> Stefan Scholz
>
> University of Bielefeld
> Faculty of Public Health
> Department of Health Economics and Health Management
> P.O. Box 10 01 31
> D-33501 Bielefeld, Germany
>
> Phone: +49 0521 | 106-2648
> Mail: stefan.scholz at uni-bielefeld.de
> <mailto:stefan.scholz at uni-bielefeld.de>
> _______________________________________________
> PopModWkGrpIMAG-news mailing list_
> _PopModWkGrpIMAG-news at simtk.org <mailto:PopModWkGrpIMAG-news at simtk.org>_
> _https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
>
> _______________________________________________
> PopModWkGrpIMAG-news mailing list_
> _PopModWkGrpIMAG-news at simtk.org <mailto:PopModWkGrpIMAG-news at simtk.org>_
> _https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
>
>
> --
> Stefan Scholz
>
> University of Bielefeld
> Faculty of Public Health
> Department of Health Economics and Health Management
> P.O. Box 10 01 31
> D-33501 Bielefeld, Germany
>
> Phone: +49 0521 | 106-2648
> Mail: stefan.scholz at uni-bielefeld.de
> <mailto:stefan.scholz at uni-bielefeld.de>
>
>
>
> Proclaimer RIVM http://www.rivm.nl/Proclaimer
> <http://www.rivm.nl/Proclaimer>
>
--
Stefan Scholz
University of Bielefeld
Faculty of Public Health
Department of Health Economics and Health Management
P.O. Box 10 01 31
D-33501 Bielefeld, Germany
Phone: +49 0521 | 106-2648
Mail: stefan.scholz at uni-bielefeld.de
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://simtk.org/pipermail/popmodwkgrpimag-news/attachments/20160304/a26248c0/attachment-0001.html>
More information about the PopModWkGrpIMAG-news
mailing list