[Population Modeling] Discussing the Population Modeling panel in SummerSim
Jeljer Hoekstra
jeljer.hoekstra at rivm.nl
Fri Mar 4 02:42:09 PST 2016
Dear all,
I like model calibration, but think we could expand.
My paper would be on "Model calibration: An example of 25 years of cohort
data and a lifestyle and disease model." It is work in progress, so I am
not sure about the results I will be able to present yet.
What is the next step, do we send an abstract? If yes what is the
deadline? and do we send it to the mailing list?
best wishes
Jeljer
From: Stefan Scholz <stefan.scholz at uni-bielefeld.de>
To:
Cc: "popmodwkgrpimag-news at simtk.org" <popmodwkgrpimag-news at simtk.org>
Date: 04/03/2016 11:13
Subject: Re: [Population Modeling] Discussing the Population
Modeling panel in SummerSim
Sent by: popmodwkgrpimag-news-bounces at simtk.org
Dear all,
could we agree on a mixture of the topics, like "data sources, parameter
estimation and calibration"? My topic would be on "Social (media) network
data in models in the absence of survey data: An example of the German
MSM-population". It would take some creativity to fit that into "model
calibration" ;-)
I hope this comes not to late!
Thanks,
Stefan
Am 16.02.2016 um 17:44 schrieb Dammann, Olaf:
All:
I like “model calibration” quite a bit.
My paper would be on “Model calibration: Four levels of calibration – A
critical appraisal”
Thanks,
Olaf
From: Jacob Barhak [mailto:jacob.barhak at gmail.com]
Sent: Tuesday, February 16, 2016 5:36 AM
To: Jeljer Hoekstra
Cc: Stefan Scholz; popmodwkgrpimag-news at simtk.org; Mélanie Prague;
Dammann, Olaf
Subject: Re: [Population Modeling] Discussing the Population Modeling
panel in SummerSim
Hi Melanie, Hi Jeljer, Hi Stefan, Hi Olaf,
Since the 14-March deadline for paper submission is coming closer, and
since we were discussing a panel, I would appreciate it if you can decide
on a common prefix for the panel.
So far we had a discussion revolving around estimation, validation,
calibration. Please choose a common denominator title to fit all your work
that will form a base for the panel and submit it to the list.
So far, here are the titles you suggested as I extracted them from your
posts:
Melanie: 'simulation vs. estimation’
Olaf: "data sources, constraints, validation issues"
Stefan & Jeljer: "model calibration" -
The last topic of "Model Calibration" seems to be a common denominator so
far, yet still possible for debate - after all you had some nice
discussions and may have a better idea.
Yet if the last topic prefix is comfortable to you all, then I ask
that you will submit your paper title to the mailing list to set the
expectations from the panel and leave you all sufficient time to write the
short paper.
I look forward to see your paper titles.
Jacob
On Mon, Feb 8, 2016 at 9:29 AM, Jeljer Hoekstra <jeljer.hoekstra at rivm.nl>
wrote:
Dear Jacob, Stefan and others,
I agree with Jacob that estimation and calibration generate parameter
values based on observed data. I never explicitly thought about in those
terms but that is what it is.
I have always used the terms in combination; calibrate a model and
estimate a parameter. I don't consider the one an automated search and the
other a manual human search per se . Usually if you estimate a parameter
the focus is on one parameter and a lot of statistical theory and software
exist to do that automatically. Whereas if you calibrate a model the focus
is often on all or a group of parameters in the model, which is perhaps
somewhat messier and needs more human interference. I am not sure though.
I am interessted in the topic Stefan raised. What if you find parameters
in the literature or you have estimated them yourself with some dataset
and then you test and adjust those variables (calibrate?) so your model
replicates some other dataset better. How much change in those parameters
do you accept, keeping in mind that the parameter may be interpreted
slightly different in the calibrated model.
Furthermore, if you calibrate your model you need some goodness of fit
criterium. I wonder if people have experience with weighing output
variables including a mixture of categorical variables (e.g. dead/alive,
smoking) and continous variables( e.g. BMI, cholesterol levels).
Validation is a related subject. I consider a model validated if it can
mimic, to some degree a dataset that was not used to calibrate/estimate
it. Obviously also here you will need some goodness of fit criterium to
see if the model is validated or not. In my experience we do not often
have the luxury of a complete extra dataset for validation. So we end up
in the discussion Stefan mentioned, if the model is calibrated how far do
we accept parameters to be different from those estimated elsewhere.
@stefan, thanks for poining out GAMLSS we will have a look.
best wishes
Jeljer
From: Jacob Barhak <jacob.barhak at gmail.com>
To: Stefan Scholz <stefan.scholz at uni-bielefeld.de>,
Cc: "popmodwkgrpimag-news at simtk.org" <
popmodwkgrpimag-news at simtk.org>, Mélanie Prague <
melanie.prague at isped.u-bordeaux2.fr>, Jeljer Hoekstra <
jeljer.hoekstra at rivm.nl>, "Dammann, Olaf" <Olaf.Dammann at tufts.edu>
Date: 05/02/2016 00:50
Subject: Re: [Population Modeling] Discussing the Population
Modeling panel in SummerSim
So Stefan,
Since this is part of the discussion on your panel topic, you will have to
choose the topic and I will try not to intervene much. Yet do allow me to
add a note regarding calibration and estimation.
Both calibration and estimation generate model parameters as outputs from
observed known data.
Will you agree with me that the term calibration would be more appropriate
to human manipulated parameters while estimation is perhaps more general
term that includes automated machine algorithm methods?
I have seen the term estimation used for parameter estimation using Delphi
style human voting, so I think "estimation" would include "calibration" as
a sub category.
Even though there is always some sort of human input to the modeling
process, it seems things are becoming more automated these days. What term
would you use for heavily machine dependent estimation algorithms as
opposed to human tightly controlled calibration? Is there a term for those
methods anyone in the list prefers using?
Hopefully this will contribute to the panel discussion.
Jacob
On Thu, Feb 4, 2016 at 4:28 AM, Stefan Scholz <
stefan.scholz at uni-bielefeld.de> wrote:
Dear all,
I just want to add the issue of "model calibration" which seems to me as
being related to estimation and validation. Just to give a short example
what I mean by calibration: We use some information of a data set to
estimate our model parameters, run the model based on those parameters and
see that the results are not externally valid. We the iteratively change
some of the input parameters until the model results are externally valid.
I would find it very interesting to discuss if calibration should be
performed and if so, how far from the originally estimated parameters you
would accept your calibrated values to be. [I hope I am not stating the
obvious or missed some guidance on this topic already available ;-) ]
@ Jelster: If you use R, maybe the GAMLSS-package developed by Prof. Mikis
Stasinopoulos is helpful to you. As far as I understand it, the package
was developed from the need of parameter estimation in agent-based
modeling. You can estimate all parameters of large list of probability
distributions. So let's say you want to estimate the probability of
getting diabetes conditional on age, sex, education, etc. You can estimate
a general linear model using a beta-distribution and include the resulting
coefficients to include them in your model. So, every person in your model
can calculate the parameters of the beta-distribution based on their age,
sex, education, etc. and you can draw random numbers from that
distribution to determine whether a person gets diabetes or not.
Best,
Stefan
Am 03.02.2016 um 01:45 schrieb Jacob Barhak:
Hi Melanie, Hi Jeljer, Hi Olaf, Hi Stefan,
It seems the panel is forming nicely. I will try to summarize what we had
so far and help figure the rest.
1. The topic seems to revolve around "estimation and validation in
population modeling" with some variations. If you are all ok with this
general topic, I suggest we stick with it as a base.
2. It seems there is agreement on separate papers with the same title
prefix. So please allocate time on writing a short 3 page paper. Since
panelists are not closely affiliated, each will review the papers of
another panelist which will contribute to panel cohesion since the
panelists will influence each others final paper. Note that the review
process is public and non-blind.
3. Presentations followed by a period of questions to all panelists seems
to be the choice. I assume there will be 20-30 minutes per panelist, yet
we will have to set timing once we know number of presentations.
Olaf asked about other presenters. Yes, there will be other presentations
by non panelists. In fact any one of you can choose to detach from the
panel and submit a paper on their own. I will send a CFP to the list
following this message.
The difference for panelists would be:
1. Panelists will have some discount that SCS promised - I have no exact
details yet. This makes sense since they will have more involvement.
2. Panelists will gain extra exposure which you are already getting with
these communications.
3. If time is available, panelists will get more time for discussion
beyond other presenters. I will communicate with organizers to see what is
possible beyond that. Yet for now, assume the panel is part of the BMPM
track.
Note that SummerSim is a Multi-conference, so having a panel may attract
more people. From the past, you should expect about 10-20 in the room for
the presentation if last years are indicative. I suspect a panel can
attract more.
So for panelists still interested, please:
1. Confirm that you are ok with the topic and format by sending an email
to this list. Or continue discussing the topic until consensus is reached.
And you can split into two panels with separate topics, or announce you
are interested in a paper outside the panel.
2. Start writing a short 3 page paper to submit to the SCS web site.
Recall that the title prefix should be the same for all papers if you are
in the panel.
3. Allocate time to review a paper or two by another panelist. This review
will be public.
Hopefully this explains the next steps and I hope more panelists would
express interest in the topic forming.
Jacob
Dear All,
I agree with Stefan re 1 and 2.
1. I like the idea of talking about input-output stuff - data sources,
constraints, validation issues.
2. Fully agree with Stefan.
3. If we have slides, this should be flash talks, not longish formal
presentations. I am still unclear whether we have presentations from
conference participants who are NOT panelists?
My 2 cents
Olaf
--
Olaf Dammann, MD
Professor of Public Health & Community Medicine
Tufts University School of Medicine
Boston, MA 02111
On Jan 21, 2016, at 4:00 AM, Stefan Scholz <stefan.scholz at uni-bielefeld.de
> wrote:
Hi all,
thanks for all your efforts! Here are my thoughts on the three points:
1. Topic: I am not quite sure whether the topics really do mean the same
thing. I would understand Melanies suggestion as "what are the differences
in the results/outputs from estimation vs. simulation" whereas I would
understand the topic "estimation in population modeling" more as
estimation of model input parameters. (Please tell me, if I got that
wrong!) I think both are interesting topics and maybe we could bring them
both together under the general topic of validity (external and internal).
(i.e. how do we estimate model input to get externally valid results and
how do we assess the latter)
2. I would vote for separate papers under the same topic, sharing the same
prefix.
3. I would go for option B). Option A) is fine as well, but we should make
clear if there are some contrary opinions on this topic. If panelists
agree on almost every topic, this might get boring. Also, I would see the
number of people in the audience a critical factor for a panel. If we are
a small group, sharing the methods used for estimation and discussing it
in the group might be more beneficial to all attendees.
Best,
Stefan
Am 19.01.2016 um 19:42 schrieb Jacob Barhak:
Hi Melanie, Hi Olaf, Hi Stefan, Hi Carl,
You all expressed interest in appearing in a population modeling panel in
SummerSim.
Melanie also suggested a topic:
"differences and extrapolation concerns around 'simulation vs. estimation’
in bio-medical area"
At this point, I wish interested parties to discuss the following:
1. The topic - feel free to suggest alternative topics/titles and we can
see how having the panel will contribute to the topic. Note that if we end
up with different topics, it is also ok since others may join to support
the topic you suggested. Hopefully there will be synergy, yet
complementary topics or even different opinions are possible. This
discussion itself is valuable.
2. Writing Format. The conference includes a paper. Part of the discussion
should be how do you prefer to be published. Do you want a joint paper? Or
would you like each to submit a short paper with similar topics? This
would probably be tied to the topic you suggest. Yet note that whatever
paper format chosen, it will undergo public non-blind review.
3. Presentation format: How would you like the talk to be? Possibilities
include: A) Totally informal discussion where panelists converge amongst
themselves, possibly with moderation and questions from he audience. B)
Presentations with a projector of each panelist and then a period of
questions. C) A combination of both, for example very short introductions
with a projector and then a discussion. Assume half an hour per panelist,
yet this may change.
As a default starting point for discussion, allow me to suggest the
following:
1. Topic Estimation in population modeling - its generalization for what
Melanie suggested - feel free to reshape it any way comfortable to you.
2. Writing format: Very short separate papers using the topic as a title
prefix. to have a common prefix fro all panelists. Here is an
example: Estimation in population Modeling - application in Disease
Models.
3. Presentation format: Short digital introductions of about 15 minutes
each - with only a few slides and a discussion that will start with
expanding prepared topics encountered during discussions and review and
then answering questions from the audience.
This default can be changed during discussion.
Please feel free to join this discussion if you are interested in
appearing in a population modeling panel in SummerSim - even if you are
not personally addressed. This post is initially directed to those who
expressed interest on this list, yet we can certainly expand the scope to
include more panelists, and I know of interest by others at this point.
I look forward to your opinions.
Jacob
_______________________________________________
PopModWkGrpIMAG-news mailing list
PopModWkGrpIMAG-news at simtk.org
https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
--
Stefan Scholz
University of Bielefeld
Faculty of Public Health
Department of Health Economics and Health Management
P.O. Box 10 01 31
D-33501 Bielefeld, Germany
Phone: +49 0521 | 106-2648
Mail: stefan.scholz at uni-bielefeld.de
_______________________________________________
PopModWkGrpIMAG-news mailing list
PopModWkGrpIMAG-news at simtk.org
https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
_______________________________________________
PopModWkGrpIMAG-news mailing list
PopModWkGrpIMAG-news at simtk.org
https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
--
Stefan Scholz
University of Bielefeld
Faculty of Public Health
Department of Health Economics and Health Management
P.O. Box 10 01 31
D-33501 Bielefeld, Germany
Phone: +49 0521 | 106-2648
Mail: stefan.scholz at uni-bielefeld.de
Proclaimer RIVM http://www.rivm.nl/Proclaimer
--
Stefan Scholz
University of Bielefeld
Faculty of Public Health
Department of Health Economics and Health Management
P.O. Box 10 01 31
D-33501 Bielefeld, Germany
Phone: +49 0521 | 106-2648
Mail: stefan.scholz at uni-bielefeld.de
_______________________________________________
PopModWkGrpIMAG-news mailing list
PopModWkGrpIMAG-news at simtk.org
https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
Proclaimer RIVM http://www.rivm.nl/Proclaimer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://simtk.org/pipermail/popmodwkgrpimag-news/attachments/20160304/a68ffcac/attachment-0001.html>
More information about the PopModWkGrpIMAG-news
mailing list