[Population Modeling] Discussing the Population Modeling panel in SummerSim
Dammann, Olaf
Olaf.Dammann at tufts.edu
Fri Mar 4 09:20:27 PST 2016
Dear All,
I just received some unpleasant health-related news and will not be able to participate.
I hope we will find a pinch-hitter among the list members.
All best,
Olaf
From: popmodwkgrpimag-news-bounces at simtk.org [mailto:popmodwkgrpimag-news-bounces at simtk.org] On Behalf Of Jacob Barhak
Sent: Friday, March 04, 2016 9:37 AM
To: Stefan Scholz
Cc: popmodwkgrpimag-news at simtk.org
Subject: Re: [Population Modeling] Discussing the Population Modeling panel in SummerSim
So Stefan,
The panel arrangement is up to you. It would have been nice to have a clear common denominator in the title, yet not necessary. The common denominator can be population modeling.
This changes the original plan a bit, yet it is still ok if you are all comfortable with it and want to present as part of the panel.
I would still ask for panelists to review each others work to be familiar with details and make a more cohesive panel.
It will be nice to bring out commonalities and differences between works.
Jacob
On Mar 4, 2016 4:13 AM, "Stefan Scholz" <stefan.scholz at uni-bielefeld.de<mailto:stefan.scholz at uni-bielefeld.de>> wrote:
Dear all,
could we agree on a mixture of the topics, like "data sources, parameter estimation and calibration"? My topic would be on "Social (media) network data in models in the absence of survey data: An example of the German MSM-population". It would take some creativity to fit that into "model calibration" ;-)
I hope this comes not to late!
Thanks,
Stefan
Am 16.02.2016 um 17:44 schrieb Dammann, Olaf:
All:
I like “model calibration” quite a bit.
My paper would be on “Model calibration: Four levels of calibration – A critical appraisal”
Thanks,
Olaf
From: Jacob Barhak [mailto:jacob.barhak at gmail.com]
Sent: Tuesday, February 16, 2016 5:36 AM
To: Jeljer Hoekstra
Cc: Stefan Scholz; popmodwkgrpimag-news at simtk.org<mailto:popmodwkgrpimag-news at simtk.org>; Mélanie Prague; Dammann, Olaf
Subject: Re: [Population Modeling] Discussing the Population Modeling panel in SummerSim
Hi Melanie, Hi Jeljer, Hi Stefan, Hi Olaf,
Since the 14-March deadline for paper submission is coming closer, and since we were discussing a panel, I would appreciate it if you can decide on a common prefix for the panel.
So far we had a discussion revolving around estimation, validation, calibration. Please choose a common denominator title to fit all your work that will form a base for the panel and submit it to the list.
So far, here are the titles you suggested as I extracted them from your posts:
Melanie: 'simulation vs. estimation’
Olaf: "data sources, constraints, validation issues"
Stefan & Jeljer: "model calibration" -
The last topic of "Model Calibration" seems to be a common denominator so far, yet still possible for debate - after all you had some nice discussions and may have a better idea.
Yet if the last topic prefix is comfortable to you all, then I ask that you will submit your paper title to the mailing list to set the expectations from the panel and leave you all sufficient time to write the short paper.
I look forward to see your paper titles.
Jacob
On Mon, Feb 8, 2016 at 9:29 AM, Jeljer Hoekstra <jeljer.hoekstra at rivm.nl<mailto:jeljer.hoekstra at rivm.nl>> wrote:
Dear Jacob, Stefan and others,
I agree with Jacob that estimation and calibration generate parameter values based on observed data. I never explicitly thought about in those terms but that is what it is.
I have always used the terms in combination; calibrate a model and estimate a parameter. I don't consider the one an automated search and the other a manual human search per se . Usually if you estimate a parameter the focus is on one parameter and a lot of statistical theory and software exist to do that automatically. Whereas if you calibrate a model the focus is often on all or a group of parameters in the model, which is perhaps somewhat messier and needs more human interference. I am not sure though.
I am interessted in the topic Stefan raised. What if you find parameters in the literature or you have estimated them yourself with some dataset and then you test and adjust those variables (calibrate?) so your model replicates some other dataset better. How much change in those parameters do you accept, keeping in mind that the parameter may be interpreted slightly different in the calibrated model.
Furthermore, if you calibrate your model you need some goodness of fit criterium. I wonder if people have experience with weighing output variables including a mixture of categorical variables (e.g. dead/alive, smoking) and continous variables( e.g. BMI, cholesterol levels).
Validation is a related subject. I consider a model validated if it can mimic, to some degree a dataset that was not used to calibrate/estimate it. Obviously also here you will need some goodness of fit criterium to see if the model is validated or not. In my experience we do not often have the luxury of a complete extra dataset for validation. So we end up in the discussion Stefan mentioned, if the model is calibrated how far do we accept parameters to be different from those estimated elsewhere.
@stefan, thanks for poining out GAMLSS we will have a look.
best wishes
Jeljer
From: Jacob Barhak <jacob.barhak at gmail.com<mailto:jacob.barhak at gmail.com>>
To: Stefan Scholz <stefan.scholz at uni-bielefeld.de<mailto:stefan.scholz at uni-bielefeld.de>>,
Cc: "popmodwkgrpimag-news at simtk.org<mailto:popmodwkgrpimag-news at simtk.org>" <popmodwkgrpimag-news at simtk.org<mailto:popmodwkgrpimag-news at simtk.org>>, Mélanie Prague <melanie.prague at isped.u-bordeaux2.fr<mailto:melanie.prague at isped.u-bordeaux2.fr>>, Jeljer Hoekstra <jeljer.hoekstra at rivm.nl<mailto:jeljer.hoekstra at rivm.nl>>, "Dammann, Olaf" <Olaf.Dammann at tufts.edu<mailto:Olaf.Dammann at tufts.edu>>
Date: 05/02/2016 00:50
Subject: Re: [Population Modeling] Discussing the Population Modeling panel in SummerSim
________________________________
So Stefan,
Since this is part of the discussion on your panel topic, you will have to choose the topic and I will try not to intervene much. Yet do allow me to add a note regarding calibration and estimation.
Both calibration and estimation generate model parameters as outputs from observed known data.
Will you agree with me that the term calibration would be more appropriate to human manipulated parameters while estimation is perhaps more general term that includes automated machine algorithm methods?
I have seen the term estimation used for parameter estimation using Delphi style human voting, so I think "estimation" would include "calibration" as a sub category.
Even though there is always some sort of human input to the modeling process, it seems things are becoming more automated these days. What term would you use for heavily machine dependent estimation algorithms as opposed to human tightly controlled calibration? Is there a term for those methods anyone in the list prefers using?
Hopefully this will contribute to the panel discussion.
Jacob
On Thu, Feb 4, 2016 at 4:28 AM, Stefan Scholz <stefan.scholz at uni-bielefeld.de<mailto:stefan.scholz at uni-bielefeld.de>> wrote:
Dear all,
I just want to add the issue of "model calibration" which seems to me as being related to estimation and validation. Just to give a short example what I mean by calibration: We use some information of a data set to estimate our model parameters, run the model based on those parameters and see that the results are not externally valid. We the iteratively change some of the input parameters until the model results are externally valid. I would find it very interesting to discuss if calibration should be performed and if so, how far from the originally estimated parameters you would accept your calibrated values to be. [I hope I am not stating the obvious or missed some guidance on this topic already available ;-) ]
@ Jelster: If you use R, maybe the GAMLSS-package developed by Prof. Mikis Stasinopoulos is helpful to you. As far as I understand it, the package was developed from the need of parameter estimation in agent-based modeling. You can estimate all parameters of large list of probability distributions. So let's say you want to estimate the probability of getting diabetes conditional on age, sex, education, etc. You can estimate a general linear model using a beta-distribution and include the resulting coefficients to include them in your model. So, every person in your model can calculate the parameters of the beta-distribution based on their age, sex, education, etc. and you can draw random numbers from that distribution to determine whether a person gets diabetes or not.
Best,
Stefan
Am 03.02.2016 um 01:45 schrieb Jacob Barhak:
Hi Melanie, Hi Jeljer, Hi Olaf, Hi Stefan,
It seems the panel is forming nicely. I will try to summarize what we had so far and help figure the rest.
1. The topic seems to revolve around "estimation and validation in population modeling" with some variations. If you are all ok with this general topic, I suggest we stick with it as a base.
2. It seems there is agreement on separate papers with the same title prefix. So please allocate time on writing a short 3 page paper. Since panelists are not closely affiliated, each will review the papers of another panelist which will contribute to panel cohesion since the panelists will influence each others final paper. Note that the review process is public and non-blind.
3. Presentations followed by a period of questions to all panelists seems to be the choice. I assume there will be 20-30 minutes per panelist, yet we will have to set timing once we know number of presentations.
Olaf asked about other presenters. Yes, there will be other presentations by non panelists. In fact any one of you can choose to detach from the panel and submit a paper on their own. I will send a CFP to the list following this message.
The difference for panelists would be:
1. Panelists will have some discount that SCS promised - I have no exact details yet. This makes sense since they will have more involvement.
2. Panelists will gain extra exposure which you are already getting with these communications.
3. If time is available, panelists will get more time for discussion beyond other presenters. I will communicate with organizers to see what is possible beyond that. Yet for now, assume the panel is part of the BMPM track.
Note that SummerSim is a Multi-conference, so having a panel may attract more people. From the past, you should expect about 10-20 in the room for the presentation if last years are indicative. I suspect a panel can attract more.
So for panelists still interested, please:
1. Confirm that you are ok with the topic and format by sending an email to this list. Or continue discussing the topic until consensus is reached. And you can split into two panels with separate topics, or announce you are interested in a paper outside the panel.
2. Start writing a short 3 page paper to submit to the SCS web site. Recall that the title prefix should be the same for all papers if you are in the panel.
3. Allocate time to review a paper or two by another panelist. This review will be public.
Hopefully this explains the next steps and I hope more panelists would express interest in the topic forming.
Jacob
Dear All,
I agree with Stefan re 1 and 2.
1. I like the idea of talking about input-output stuff - data sources, constraints, validation issues.
2. Fully agree with Stefan.
3. If we have slides, this should be flash talks, not longish formal presentations. I am still unclear whether we have presentations from conference participants who are NOT panelists?
My 2 cents
Olaf
--
Olaf Dammann, MD
Professor of Public Health & Community Medicine
Tufts University School of Medicine
Boston, MA 02111
On Jan 21, 2016, at 4:00 AM, Stefan Scholz <stefan.scholz at uni-bielefeld.de<mailto:stefan.scholz at uni-bielefeld.de>> wrote:
Hi all,
thanks for all your efforts! Here are my thoughts on the three points:
1. Topic: I am not quite sure whether the topics really do mean the same thing. I would understand Melanies suggestion as "what are the differences in the results/outputs from estimation vs. simulation" whereas I would understand the topic "estimation in population modeling" more as estimation of model input parameters. (Please tell me, if I got that wrong!) I think both are interesting topics and maybe we could bring them both together under the general topic of validity (external and internal). (i.e. how do we estimate model input to get externally valid results and how do we assess the latter)
2. I would vote for separate papers under the same topic, sharing the same prefix.
3. I would go for option B). Option A) is fine as well, but we should make clear if there are some contrary opinions on this topic. If panelists agree on almost every topic, this might get boring. Also, I would see the number of people in the audience a critical factor for a panel. If we are a small group, sharing the methods used for estimation and discussing it in the group might be more beneficial to all attendees.
Best,
Stefan
Am 19.01.2016 um 19:42 schrieb Jacob Barhak:
Hi Melanie, Hi Olaf, Hi Stefan, Hi Carl,
You all expressed interest in appearing in a population modeling panel in SummerSim.
Melanie also suggested a topic:
"differences and extrapolation concerns around 'simulation vs. estimation’ in bio-medical area"
At this point, I wish interested parties to discuss the following:
1. The topic - feel free to suggest alternative topics/titles and we can see how having the panel will contribute to the topic. Note that if we end up with different topics, it is also ok since others may join to support the topic you suggested. Hopefully there will be synergy, yet complementary topics or even different opinions are possible. This discussion itself is valuable.
2. Writing Format. The conference includes a paper. Part of the discussion should be how do you prefer to be published. Do you want a joint paper? Or would you like each to submit a short paper with similar topics? This would probably be tied to the topic you suggest. Yet note that whatever paper format chosen, it will undergo public non-blind review.
3. Presentation format: How would you like the talk to be? Possibilities include: A) Totally informal discussion where panelists converge amongst themselves, possibly with moderation and questions from he audience. B) Presentations with a projector of each panelist and then a period of questions. C) A combination of both, for example very short introductions with a projector and then a discussion. Assume half an hour per panelist, yet this may change.
As a default starting point for discussion, allow me to suggest the following:
1. Topic Estimation in population modeling - its generalization for what Melanie suggested - feel free to reshape it any way comfortable to you.
2. Writing format: Very short separate papers using the topic as a title prefix. to have a common prefix fro all panelists. Here is an example: Estimation in population Modeling - application in Disease Models.
3. Presentation format: Short digital introductions of about 15 minutes each - with only a few slides and a discussion that will start with expanding prepared topics encountered during discussions and review and then answering questions from the audience.
This default can be changed during discussion.
Please feel free to join this discussion if you are interested in appearing in a population modeling panel in SummerSim - even if you are not personally addressed. This post is initially directed to those who expressed interest on this list, yet we can certainly expand the scope to include more panelists, and I know of interest by others at this point.
I look forward to your opinions.
Jacob
_______________________________________________
PopModWkGrpIMAG-news mailing list
PopModWkGrpIMAG-news at simtk.org<mailto:PopModWkGrpIMAG-news at simtk.org>
https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
--
Stefan Scholz
University of Bielefeld
Faculty of Public Health
Department of Health Economics and Health Management
P.O. Box 10 01 31
D-33501 Bielefeld, Germany
Phone: +49 0521 | 106-2648
Mail: stefan.scholz at uni-bielefeld.de<mailto:stefan.scholz at uni-bielefeld.de>
_______________________________________________
PopModWkGrpIMAG-news mailing list
PopModWkGrpIMAG-news at simtk.org<mailto:PopModWkGrpIMAG-news at simtk.org>
https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
_______________________________________________
PopModWkGrpIMAG-news mailing list
PopModWkGrpIMAG-news at simtk.org<mailto:PopModWkGrpIMAG-news at simtk.org>
https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
--
Stefan Scholz
University of Bielefeld
Faculty of Public Health
Department of Health Economics and Health Management
P.O. Box 10 01 31
D-33501 Bielefeld, Germany
Phone: +49 0521 | 106-2648
Mail: stefan.scholz at uni-bielefeld.de<mailto:stefan.scholz at uni-bielefeld.de>
Proclaimer RIVM http://www.rivm.nl/Proclaimer<http://www.rivm.nl/Proclaimer>
--
Stefan Scholz
University of Bielefeld
Faculty of Public Health
Department of Health Economics and Health Management
P.O. Box 10 01 31
D-33501 Bielefeld, Germany
Phone: +49 0521 | 106-2648
Mail: stefan.scholz at uni-bielefeld.de<mailto:stefan.scholz at uni-bielefeld.de>
_______________________________________________
PopModWkGrpIMAG-news mailing list
PopModWkGrpIMAG-news at simtk.org<mailto:PopModWkGrpIMAG-news at simtk.org>
https://simtk.org/mailman/listinfo/popmodwkgrpimag-news
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://simtk.org/pipermail/popmodwkgrpimag-news/attachments/20160304/93420d9a/attachment-0001.html>
More information about the PopModWkGrpIMAG-news
mailing list