[Population Modeling] Discussing the Population Modeling panel in SummerSim
Jacob Barhak
jacob.barhak at gmail.com
Fri Mar 4 06:36:42 PST 2016
So Stefan,
The panel arrangement is up to you. It would have been nice to have a clear
common denominator in the title, yet not necessary. The common denominator
can be population modeling.
This changes the original plan a bit, yet it is still ok if you are all
comfortable with it and want to present as part of the panel.
I would still ask for panelists to review each others work to be familiar
with details and make a more cohesive panel.
It will be nice to bring out commonalities and differences between works.
Jacob
On Mar 4, 2016 4:13 AM, "Stefan Scholz" <stefan.scholz at uni-bielefeld.de>
wrote:
> Dear all,
>
> could we agree on a mixture of the topics, like "data sources, parameter
> estimation and calibration"? My topic would be on "Social (media) network
> data in models in the absence of survey data: An example of the German
> MSM-population". It would take some creativity to fit that into "model
> calibration" ;-)
>
> I hope this comes not to late!
>
> Thanks,
> Stefan
>
> Am 16.02.2016 um 17:44 schrieb Dammann, Olaf:
>
> All:
>
> I like “model calibration” quite a bit.
>
> My paper would be on “Model calibration: Four levels of calibration – A
> critical appraisal”
>
> Thanks,
>
> Olaf
>
>
>
>
>
>
>
>
>
> *From:* Jacob Barhak [mailto:jacob.barhak at gmail.com
> <jacob.barhak at gmail.com>]
> *Sent:* Tuesday, February 16, 2016 5:36 AM
> *To:* Jeljer Hoekstra
> *Cc:* Stefan Scholz; popmodwkgrpimag-news at simtk.org; Mélanie Prague;
> Dammann, Olaf
> *Subject:* Re: [Population Modeling] Discussing the Population Modeling
> panel in SummerSim
>
>
>
> Hi Melanie, Hi Jeljer, Hi Stefan, Hi Olaf,
>
>
>
> Since the 14-March deadline for paper submission is coming closer, and
> since we were discussing a panel, I would appreciate it if you can decide
> on a common prefix for the panel.
>
>
>
> So far we had a discussion revolving around estimation, validation,
> calibration. Please choose a common denominator title to fit all your work
> that will form a base for the panel and submit it to the list.
>
>
>
> So far, here are the titles you suggested as I extracted them from your
> posts:
>
> Melanie: 'simulation vs. estimation’
>
> Olaf: "data sources, constraints, validation issues"
>
> Stefan & Jeljer: "model calibration" -
>
>
>
> The last topic of "Model Calibration" seems to be a common denominator so
> far, yet still possible for debate - after all you had some nice
> discussions and may have a better idea.
>
>
>
> Yet if the last topic prefix is comfortable to you all, then I ask
> that you will submit your paper title to the mailing list to set the
> expectations from the panel and leave you all sufficient time to write the
> short paper.
>
>
>
> I look forward to see your paper titles.
>
>
>
> Jacob
>
>
>
>
>
>
>
> On Mon, Feb 8, 2016 at 9:29 AM, Jeljer Hoekstra <jeljer.hoekstra at rivm.nl>
> wrote:
>
> Dear Jacob, Stefan and others,
>
> I agree with Jacob that estimation and calibration generate parameter
> values based on observed data. I never explicitly thought about in those
> terms but that is what it is.
> I have always used the terms in combination; calibrate a model and
> estimate a parameter. I don't consider the one an automated search and the
> other a manual human search per se . Usually if you estimate a parameter
> the focus is on one parameter and a lot of statistical theory and software
> exist to do that automatically. Whereas if you calibrate a model the focus
> is often on all or a group of parameters in the model, which is perhaps
> somewhat messier and needs more human interference. I am not sure though.
>
> I am interessted in the topic Stefan raised. What if you find parameters
> in the literature or you have estimated them yourself with some dataset and
> then you test and adjust those variables (calibrate?) so your model
> replicates some other dataset better. How much change in those parameters
> do you accept, keeping in mind that the parameter may be interpreted
> slightly different in the calibrated model.
> Furthermore, if you calibrate your model you need some goodness of fit
> criterium. I wonder if people have experience with weighing output
> variables including a mixture of categorical variables (e.g. dead/alive,
> smoking) and continous variables( e.g. BMI, cholesterol levels).
>
> Validation is a related subject. I consider a model validated if it can
> mimic, to some degree a dataset that was not used to calibrate/estimate it.
> Obviously also here you will need some goodness of fit criterium to see if
> the model is validated or not. In my experience we do not often have the
> luxury of a complete extra dataset for validation. So we end up in the
> discussion Stefan mentioned, if the model is calibrated how far do we
> accept parameters to be different from those estimated elsewhere.
>
> @stefan, thanks for poining out GAMLSS we will have a look.
>
> best wishes
> Jeljer
>
>
>
> From: Jacob Barhak <jacob.barhak at gmail.com>
> To: Stefan Scholz <stefan.scholz at uni-bielefeld.de>,
> Cc: "popmodwkgrpimag-news at simtk.org" <
> popmodwkgrpimag-news at simtk.org>, Mélanie Prague <
> melanie.prague at isped.u-bordeaux2.fr>, Jeljer Hoekstra <
> jeljer.hoekstra at rivm.nl>, "Dammann, Olaf" <Olaf.Dammann at tufts.edu>
> Date: 05/02/2016 00:50
> Subject: Re: [Population Modeling] Discussing the Population
> Modeling panel in SummerSim
> ------------------------------
>
>
>
>
> So Stefan,
>
> Since this is part of the discussion on your panel topic, you will have to
> choose the topic and I will try not to intervene much. Yet do allow me to
> add a note regarding calibration and estimation.
>
> Both calibration and estimation generate model parameters as outputs from
> observed known data.
>
> Will you agree with me that the term calibration would be more appropriate
> to human manipulated parameters while estimation is perhaps more general
> term that includes automated machine algorithm methods?
>
> I have seen the term estimation used for parameter estimation using Delphi
> style human voting, so I think "estimation" would include "calibration" as
> a sub category.
>
> Even though there is always some sort of human input to the modeling
> process, it seems things are becoming more automated these days. What term
> would you use for heavily machine dependent estimation algorithms as
> opposed to human tightly controlled calibration? Is there a term for those
> methods anyone in the list prefers using?
>
> Hopefully this will contribute to the panel discussion.
>
> Jacob
>
>
> On Thu, Feb 4, 2016 at 4:28 AM, Stefan Scholz <
> stefan.scholz at uni-bielefeld.de> wrote:
> Dear all,
>
> I just want to add the issue of "model calibration" which seems to me as
> being related to estimation and validation. Just to give a short example
> what I mean by calibration: We use some information of a data set to
> estimate our model parameters, run the model based on those parameters and
> see that the results are not externally valid. We the iteratively change
> some of the input parameters until the model results are externally valid.
> I would find it very interesting to discuss if calibration should be
> performed and if so, how far from the originally estimated parameters you
> would accept your calibrated values to be. [I hope I am not stating the
> obvious or missed some guidance on this topic already available ;-) ]
>
> @ Jelster: If you use R, maybe the GAMLSS-package developed by Prof. Mikis
> Stasinopoulos is helpful to you. As far as I understand it, the package was
> developed from the need of parameter estimation in agent-based modeling.
> You can estimate all parameters of large list of probability distributions.
> So let's say you want to estimate the probability of getting diabetes
> conditional on age, sex, education, etc. You can estimate a general linear
> model using a beta-distribution and include the resulting coefficients to
> include them in your model. So, every person in your model can calculate
> the parameters of the beta-distribution based on their age, sex, education,
> etc. and you can draw random numbers from that distribution to determine
> whether a person gets diabetes or not.
>
> Best,
> Stefan
>
>
> Am 03.02.2016 um 01:45 schrieb Jacob Barhak:
>
> Hi Melanie, Hi Jeljer, Hi Olaf, Hi Stefan,
>
> It seems the panel is forming nicely. I will try to summarize what we had
> so far and help figure the rest.
>
> 1. The topic seems to revolve around "estimation and validation in
> population modeling" with some variations. If you are all ok with this
> general topic, I suggest we stick with it as a base.
>
> 2. It seems there is agreement on separate papers with the same title
> prefix. So please allocate time on writing a short 3 page paper. Since
> panelists are not closely affiliated, each will review the papers of
> another panelist which will contribute to panel cohesion since the
> panelists will influence each others final paper. Note that the review
> process is public and non-blind.
>
> 3. Presentations followed by a period of questions to all panelists seems
> to be the choice. I assume there will be 20-30 minutes per panelist, yet we
> will have to set timing once we know number of presentations.
>
> Olaf asked about other presenters. Yes, there will be other presentations
> by non panelists. In fact any one of you can choose to detach from the
> panel and submit a paper on their own. I will send a CFP to the list
> following this message.
>
> The difference for panelists would be:
> 1. Panelists will have some discount that SCS promised - I have no exact
> details yet. This makes sense since they will have more involvement.
> 2. Panelists will gain extra exposure which you are already getting with
> these communications.
> 3. If time is available, panelists will get more time for discussion
> beyond other presenters. I will communicate with organizers to see what is
> possible beyond that. Yet for now, assume the panel is part of the BMPM
> track.
>
> Note that SummerSim is a Multi-conference, so having a panel may attract
> more people. From the past, you should expect about 10-20 in the room for
> the presentation if last years are indicative. I suspect a panel can
> attract more.
>
> So for panelists still interested, please:
>
> 1. Confirm that you are ok with the topic and format by sending an email
> to this list. Or continue discussing the topic until consensus is reached.
> And you can split into two panels with separate topics, or announce you are
> interested in a paper outside the panel.
>
> 2. Start writing a short 3 page paper to submit to the SCS web site.
> Recall that the title prefix should be the same for all papers if you are
> in the panel.
>
> 3. Allocate time to review a paper or two by another panelist. This review
> will be public.
>
> Hopefully this explains the next steps and I hope more panelists would
> express interest in the topic forming.
>
> Jacob
>
>
>
> Dear All,
> I agree with Stefan re 1 and 2.
>
> 1. I like the idea of talking about input-output stuff - data sources,
> constraints, validation issues.
>
> 2. Fully agree with Stefan.
>
> 3. If we have slides, this should be flash talks, not longish formal
> presentations. I am still unclear whether we have presentations from
> conference participants who are NOT panelists?
>
> My 2 cents
> Olaf
>
>
> --
> Olaf Dammann, MD
> Professor of Public Health & Community Medicine
> Tufts University School of Medicine
> Boston, MA 02111
>
>
> On Jan 21, 2016, at 4:00 AM, Stefan Scholz <stefan.scholz at uni-bielefeld.de>
> wrote:
>
> Hi all,
>
> thanks for all your efforts! Here are my thoughts on the three points:
>
> 1. Topic: I am not quite sure whether the topics really do mean the same
> thing. I would understand Melanies suggestion as "what are the differences
> in the results/*outputs* from estimation vs. simulation" whereas I would
> understand the topic "estimation in population modeling" more as estimation
> of model *input *parameters. (Please tell me, if I got that wrong!) I
> think both are interesting topics and maybe we could bring them both
> together under the general topic of validity (external and internal). (i.e.
> how do we estimate model input to get externally valid results and how do
> we assess the latter)
>
> 2. I would vote for separate papers under the same topic, sharing the same
> prefix.
>
> 3. I would go for option B). Option A) is fine as well, but we should make
> clear if there are some contrary opinions on this topic. If panelists agree
> on almost every topic, this might get boring. Also, I would see the number
> of people in the audience a critical factor for a panel. If we are a small
> group, sharing the methods used for estimation and discussing it in the
> group might be more beneficial to all attendees.
>
> Best,
> Stefan
>
> Am 19.01.2016 um 19:42 schrieb Jacob Barhak:
> Hi Melanie, Hi Olaf, Hi Stefan, Hi Carl,
>
> You all expressed interest in appearing in a population modeling panel in
> SummerSim.
>
> Melanie also suggested a topic:
> "differences and extrapolation concerns around 'simulation vs. estimation’
> in bio-medical area"
>
> At this point, I wish interested parties to discuss the following:
>
> 1. The topic - feel free to suggest alternative topics/titles and we can
> see how having the panel will contribute to the topic. Note that if we end
> up with different topics, it is also ok since others may join to support
> the topic you suggested. Hopefully there will be synergy, yet complementary
> topics or even different opinions are possible. This discussion itself is
> valuable.
>
> 2. Writing Format. The conference includes a paper. Part of the discussion
> should be how do you prefer to be published. Do you want a joint paper? Or
> would you like each to submit a short paper with similar topics? This would
> probably be tied to the topic you suggest. Yet note that whatever paper
> format chosen, it will undergo public non-blind review.
>
> 3. Presentation format: How would you like the talk to be? Possibilities
> include: A) Totally informal discussion where panelists converge amongst
> themselves, possibly with moderation and questions from he audience. B)
> Presentations with a projector of each panelist and then a period of
> questions. C) A combination of both, for example very short introductions
> with a projector and then a discussion. Assume half an hour per panelist,
> yet this may change.
>
> As a default starting point for discussion, allow me to suggest the
> following:
>
> 1. Topic Estimation in population modeling - its generalization for what
> Melanie suggested - feel free to reshape it any way comfortable to you.
>
> 2. Writing format: Very short separate papers using the topic as a title
> prefix. to have a common prefix fro all panelists. Here is an
> example: Estimation in population Modeling - application in Disease Models.
>
> 3. Presentation format: Short digital introductions of about 15 minutes
> each - with only a few slides and a discussion that will start with
> expanding prepared topics encountered during discussions and review and
> then answering questions from the audience.
>
> This default can be changed during discussion.
>
> Please feel free to join this discussion if you are interested in
> appearing in a population modeling panel in SummerSim - even if you are not
> personally addressed. This post is initially directed to those who
> expressed interest on this list, yet we can certainly expand the scope to
> include more panelists, and I know of interest by others at this point.
>
> I look forward to your opinions.
>
> Jacob
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> PopModWkGrpIMAG-news mailing list
> PopModWkGrpIMAG-news at simtk.org
> https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
>
>
> --
> Stefan Scholz
>
> University of Bielefeld
> Faculty of Public Health
> Department of Health Economics and Health Management
> P.O. Box 10 01 31
> D-33501 Bielefeld, Germany
>
> Phone: +49 0521 | 106-2648
> Mail: stefan.scholz at uni-bielefeld.de
> _______________________________________________
> PopModWkGrpIMAG-news mailing list
> PopModWkGrpIMAG-news at simtk.org
> https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
>
> _______________________________________________
> PopModWkGrpIMAG-news mailing list
> PopModWkGrpIMAG-news at simtk.org
> https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
>
>
> --
> Stefan Scholz
>
> University of Bielefeld
> Faculty of Public Health
> Department of Health Economics and Health Management
> P.O. Box 10 01 31
> D-33501 Bielefeld, Germany
>
> Phone: +49 0521 | 106-2648
> Mail: stefan.scholz at uni-bielefeld.de
>
>
>
> Proclaimer RIVM http://www.rivm.nl/Proclaimer
> <http://www.rivm.nl/Proclaimer>
>
>
>
>
> --
> Stefan Scholz
>
> University of Bielefeld
> Faculty of Public Health
> Department of Health Economics and Health Management
> P.O. Box 10 01 31
> D-33501 Bielefeld, Germany
>
> Phone: +49 0521 | 106-2648
> Mail: stefan.scholz at uni-bielefeld.de
>
>
> _______________________________________________
> PopModWkGrpIMAG-news mailing list
> PopModWkGrpIMAG-news at simtk.org
> https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://simtk.org/pipermail/popmodwkgrpimag-news/attachments/20160304/357b85b9/attachment-0001.html>
More information about the PopModWkGrpIMAG-news
mailing list