The validation assessment tool AdViSHE is a tool for structured reporting on all relevant aspects of validation (conceptual model, input data, implemented software program, and model outcomes) but does not indicate any particular methodology. In a recent Value in Health article, Isaac Corro Ramos and colleagues provide further details on one of these aspects: the validation of health economic (HE) model outcomes against empirical data. In particular, they present a new Bayesian method to determine how well HE model outcomes compare to empirical data. Validity is based on a pre-established accuracy interval where the model outcomes should fall. The method uses the outcomes of a probabilistic sensitivity analysis and results in a posterior distribution around the probability that HE model outcomes can be regarded as valid.
The applicability of their method was shown in a case study. A published diabetes model (MICADO) was used to validate the outcome “number of patients who are on dialysis or with end-stage renal disease”. Results indicate that a high probability of a valid outcome is associated with relatively wide accuracy intervals. In particular, 25% deviation from the observed outcome implied approximately 60% expected validity.
The paper concludes that current practice in HE model validation can be improved by using an alternative method based on assessing whether the model outcomes fit to empirical data at a predefined level of accuracy. This method has the advantage of assessing both model bias and parameter uncertainty and resulting in a quantitative measure of the degree of validity that penalises models predicting the mean of an outcome correctly but with overly wide credible intervals.
Isaac Corro Ramos, George A.K. van Voorn, Pepijn Vemer, Talitha L. Feenstra and Maiwenn J. Al. “A New Statistical Method to Determine the Degree of Validity of Health Economic Model Outcomes against Empirical Data”. Value in Health. May 2017. DOI: http://dx.doi.org/10.1016/j.jval.2017.04.016
advishe.wordpress.com 