Introduction
About
The IVI-RA Model Interface runs the Innovation and Value Initiative's open-source rheumatoid arthritis individual patient simulation model, which is part of the Open Source Value Project. The model estimates the value of disease-modifying anti-rheumatic drugs (DMARDs), which include conventional DMARDs (cDMARDs), biologic DMARDs (bDMARDs), and JAK inhibitors, for patients with moderate to severe rheumatoid arthritis. Full model documentation and source code is available here. By using this interface, you acknowledge and accept all terms and conditions.
Quick start
- Simulate: Run the simulation with default settings by clicking the Run simulation tab and clicking the Simulate button in the leftmost box.
- View model results: Examine results in the View model results tab. Expected clinicical and economic outcomes, cost-effectiveness analyses, and multi-criteria decision analyses are provided.
Custom analysis
- Setup model:
- Population: Select population characteristics.
- Treatment sequences: Choose up to five treatment sequences.
- Model structure: Select a structure for modeling the progression of rheumatoid arthritis, the impact of morbidity on on quality of life, and time to treatment discontinuation.
- Parameter values: Based on the model structure, choose values for input parameters.
- Simulation settings: Choose simulation setting such as the discount rate and time horizon.
- Run simulation: Run the underlying simulation model. Choose the number of patients to simulate and the number of times to sample the parameters for the probabilistic sensitivity analysis (PSA).
- View inputs used in simulation: View your selected treatment sequences, a summary of the characteristics of the chosen population, and a summary of the distribution of each parameter included in the model.
- View model results:
- Modify output settings: Modify settings that can be changed without having to rerun the simulation including the choice of comparator and perspective (societal or health care sector).
- Expected outcomes: View survival curves and progression of costs, disease, and quality of life over time.
- Cost-effectiveness analysis: Examine standard cost-effectiveness results relating to the value of the treatment sequence and the uncertainty associated with that value including incremental cost-effectiveness ratios (ICERs) and summaries of the PSA.
- Multi-criteria decision analysis: Assess value by using multi-criteria decision analysis to weight different criteria.
Type of cohort
Treatment history
Proportion male
Weight in kilograms
Patient age
Number of previous DMARDs
Disease activity and functional status at baseline
Treatment sequences
Select up to five treatment sequences below. In each box, select the name of the treatments(s) that you would like to model. Each treatment you select will be added to the sequence, in order, from left to right. Two default treatment sequences have been preselected.
Initial treatment phase (first 6 months)
Time to treatment discontinuation
HAQ progression in the absence of tDMARDs
Utility algorithm
Treatment effects at 6 months
Statistical model for ACR response for tDMARD naive patients
A Bayesian random effects model for ordinal data was estimated. The model assumes that the four mutually exclusive ACR response categories follow a multinomial distribution. The probability of each category was modeled using a probit link function. Coefficients from the model are below.
Estimate
Standard error
Probability of ACR response for tDMARD experienced patients relative to tDMARD naive patients
Statistical model for change in HAQ at 6 months for tDMARD naive patients
A Bayesian random effects model for continuous data was estimated. The 'Mean for cDMARDs' estimate is the mean change in HAQ at 6 months for patients using cDMARDs. All other coefficients can be interpreted as the additional change in HAQ associated with a given treatment.
Estimate
Standard error
Coefficient for tDMARD experienced patients relative to tDMARD naive patients
Statistical model for change in DAS28 at 6 months for tDMARD naive patients
A Bayesian random effects model for continuous data was estimated. The 'Mean for cDMARDs' estimate is the mean change in DAS28 at 6 months for patients using cDMARDs. All other coefficients can be interpreted as the additional change in DAS28 associated with a given treatment.
Estimate
Standard error
Coefficient for tDMARD experienced patients relative to tDMARD naive patients
Treatment response mappings
Relationship between ACR and EULAR response
HAQ change by EULAR response category
Estimate
Standard error
HAQ change by ACR response category
Estimate
Standard error
Treatment -> ACR -> HAQ
Treatment -> ACR -> EULAR -> HAQ
Longterm HAQ progression
Rebound post treatment
Difference between overall and age-specific annual HAQ progression rate
Mean
Standard error
Constant annual HAQ progression by treatment
Estimate
Standard error
Mortality
Log odds - impact of baseline HAQ on probability of mortality
Log hazard ratio - impact of change in HAQ from baseline on mortality rate
Estimate
Standard error
Time to treatment discontinuation
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Estimate
Standard error
Adverse events
Log of serious infection rate by treatment
Estimate
Standard error
Formal health care sector costs
Treatment cost
Hospital days per year by HAQ score range
Estimate
Standard error
Cost per hospital day by HAQ score range
Estimate
Standard error
Serious infection cost
Annualized cost of managing RA
Estimate
Standard error
Utility
Serious infection utility loss
Productivity loss
Run simulation
Number of patients to simulate
Number of times to sample parameters for the PSA
Discount rate
Time horizon
Population
Treatment sequences
Model structure
Sampled parameter values
Selected treatment sequences
Mean clinical and economic outcomes
Survival curves
Mean HAQ over time
Mean cumulative QALYs
Mean cumulative costs
Time to treatment discontinuation
Proportion switching during first 6 months
Proportion remaining on treatment conditional on not switching during the first 6 months
Selected treatment sequences
Modify settings
Cost-effectiveness plane
Incremental cost-effectiveness relative to comparator
Cost-effectiveness acceptability frontier (CEAF)
Probability of being most cost effective
Cost-effectiveness acceptability curve (CEAC)
Expected incremental net monetary benefit
Distribution of incremental net monetary benefits
Expected value of perfect information
Selected treatment sequences
MCDA criteria
Linear partial value function
Weighting
Overall value
Probability of ranking
Selected treatment sequences
What is value to the healthy?
Cost-effectiveness analyses typically estimate the value of a treatment to sick patients. Lakdawalla et al. (2017) provide a mathematical framework for analyzing value from the perspective of healthy individuals, who face the risk of disease in the future.
Treatments for rheumatoid arthritis provide value to healthy individuals because they may develop rheumatoid arthritis in the future. We refer to this value as conventional value.
Risk-averse individuals may derive additional value from treatments for rheumatoid arthritis because these treatments reduce physical risk . And while innovation certainly increases financial risk , the increase in financial risk can be mitigated by healthcare insurance. We refer to the value of treatments to risk-averse individuals as insurance value without health insurance when they do not have healthcare insurance and insurance value with health insurance when they do have health insurance.
Modify settings
Value to the healthy
Glossary of key terms
Arm: A sequence of rheumatoid arthritis treatments used by a patient over their lifetime
bDMARDs Biologic DMARDs
bDMARD experienced: A patient that has previously failed at least one bDMARD
bDMARD naive: A patient that has previously failed cDMARDs but has not yet used a bDMARD
CDAI: The clinical disease activity index
cDMARDs: Conventional DMARDs
Comparator: The treatment arm used as a reference in the analysis
CEAC: Cost-effectiveness acceptability curve. The probability that each arm provides greater net monetary benefits than the comparator for a given willingness to pay
CEAF: Cost-effectiveness acceptability frontier. The probability that the optimal treatment (i.e., the treatment with the highest expected net monetary benefit) is cost-effective for a given willingness to pay
Cost-effectiveness plane: A scatter plot of the distribution of incremental costs and incremental QALYs from the PSA
DAS28: A disability activity score based on an assessment of 28 joints
Discount factor: Annual factor by which future costs and health gains should be discounted
DMARDs: Disease-modifying anti-rheumatic drugs
EULAR response: A response criteria that classifies patients as non-responders, moderate responders, or good responders to treatment based on the change and level of their DAS28 score
Expected incremental net-monetary benefit: The incremental net-monetary benefit averaged across all randomly sampled parameter sets in the PSA
Expected value of perfect information: The maximum amount that a decision maker would be willing to pay to obtain perfect information and reduce all uncertainty
HAQ: The health assessment questionnaire disability index. A patient reported outcome ranging from 0-3 measuring the degree of difficulty patients have with daily activities
ICER: Incremental cost-effectiveness ratio. Calculated by dividing incremental costs by incremental QALYs
Incremental costs: Differences in costs between a treatment arm and the comparator
Incremental net-monetary benefit: The difference in net monetary benefits between a treatment arm and the comparator arm
Incremental QALYs: Difference in QALYs between a treatment arm and the comparator
LCGM: Latent class growth model
Net-monetary benefit: The monetized value of a QALY (i.e., willingness to pay times the number of QALYs) minus costs
Pairwise comparison: Difference in outcomes between a given treatment arm and the comparator
Probability of being most cost effective: The number of samples from the PSA that each arm provides the greatest net monetary benefits among all arms for a given willingness to pay
PSA: Probabilistic sensitivity analysis. Propagates uncertainty in the input parameters throughout the model to quantify uncertainty in expected model outcomes. Generates a distribution of mean model outcomes by randomly sampling each parameter in the model from a probability distribution and calculating mean model outcomes for each randomly sampled parameter set
QALYs: Quality-adjusted life-years
SDAI: The simplified disease activity index
Treatment arm: Arms in the model that are compared to the comparator
WAC: Wholesale acquisition cost
Willingness to pay: Amount that a decision maker is willing to pay for an additional QALY
Frequently asked questions
What type of simulation is used?
The IVI-RA model is an individual patient simulation (IPS) that simulates patients one at a time.
How long are the model cycles?
Model cycles are 6 months long, which is consistent with other models and with clinical trials.
How many iterations do I need to run?
To get familiar with the interface and to explore the implications of different modeling assumptions, we suggest that you start by simulating 100 patients and sampling each parameter 100 times for the PSA. Simulations using these settings run in a few seconds and generate stable point estimates; however, this “quick” simulation may overestimate uncertainty because the model still has some stochastic uncertainty (i.e., uncertainty due to random variation among the simulated individuals). For users interested in completely eliminating the effect of stochastic uncertainty, IVI recommends increasing the number of simulated patients, but warns that this can significantly increase run time.
How were the default settings selected?
The current default model structure is based on NICE's 2016 technology assessment. The default model structure may change after we receive public comments and the public comments are reviewed by an expert panel. Default parameter values are based on studies that, in our view, allow us to generate the most accurate estimates possible for a given model structure.
How are bDMARD naive and a bDMARD experienced patients treated within the model?
The simulation can be run for either bDMARDs naive or bDMARDs experienced patients. When the model is run for bDMARDs naive patients, network meta-analysis (NMA) results for bDMARDs naive patients are used for first line therapies and NMA results for patients with inadequate response to biologic treatments are used for all subsequent lines. In contrast, when the model is run for bDMARDs experienced patients, NMA results for patients with inadequate response to biologic treatment are used for all treatment lines.
What is the difference between a homogeneous and a heterogeneous cohort?
If the model is run for a homogeneous cohort, then the model simulates a cohort of male and female patients identical in all respects other than their weight and gender. If a heterogeneous cohort is selected, then characteristics vary across patients according to the settings chosen by the user.
Why does the simulated mean HAQ score become increasingly noisy over time?
The model simulates the mean HAQ score for every individual until death, which means that the mean HAQ score is an average across patients that have survived to a given model cycle. Since the number of living simulated patients becomes smaller over time, there are fewer patients to calculate a mean HAQ score for, so it becomes increasingly noisy.
What programming language is the model written in and where can I find the source code?
The model is written in R and available with the iviRA package; however, all computationally intensive code is written in C++ so that the model can be run in a reasonable amount of time. Documentation describing how to use the R package as well as links to the underlying source code on GitHub can be found here.
Where can I find the model documentation?
A detailed description of the model is available on the R package website here.
About this app
This web application runs version 2 of IVI-RA model, part of the Innovation and Value Initiative's Open Source Value Project. The underlying model is available as the iviRA R package. Model documentation is available on the package website here and source code for the package can be found on GitHub here.
The app is developed and maintained by Devin Incerti, Ming Xu, and Jeroen Jansen.
Terms and conditions
PLEASE READ THESE TERMS AND CONDITIONS CAREFULLY
These terms and conditions describe the terms and conditions applicable to your use of the Innovation and Value Initiative (“IVI”) website and Model Interface.
YOUR USE OF THIS WEBSITE AND MODEL INTERFACE IS CONDITIONED UPON YOUR ACCEPTANCE OF THE TERMS AND CONDITIONS. BY USING THE MODEL INTERFACE OR ACCESSING ANY PART OF THIS WEBSITE, YOU AGREE TO BE BOUND BY THE TERMS AND CONDITIONS. IF YOU DO NOT AGREE TO THESE TERMS AND CONDITIONS, YOU ARE NOT AUTHORIZED TO ACCESS OR USE THIS WEBSITE OR THE MODEL INTERFACE.
THIS WEBSITE DOES NOT PROVIDE MEDICAL ADVICE
This website and the Model Interface are designed to provide information on value to a wide range of healthcare stakeholders, but not to be used as a clinical decision tool. Every effort has been made to provide the most current and correct information possible. Nonetheless, inadvertent technical inaccuracies or other errors may occur. IVI cannot be held responsible or liable for errors, inaccuracies, omissions or any perceived harm to users of this website or the Model Interface. Users of this information are encouraged to confirm the information contained herein with other sources. Any information is not intended to replace medical advice offered by healthcare providers. Use of this website or the Model Interface does not create a physician-patient relationship.
NO WARRANTIES
ALL CONTENT ON THIS WEBSITE IS PROVIDED ON AN “AS IS” BASIS WITHOUT WARRANTY OF ANY KIND EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, ACCURACY AND NON-INFRINGEMENT. IVI MAKES NO WARRANTY AS TO THE ACCURACY, COMPLETENESS OR RELIABILITY OF ANY CONTENT AVAILABLE THROUGH THIS WEBSITE OR THE MODEL INTERFACE. YOU ARE RESPONSIBLE FOR VERIFYING ANY INFORMATION BEFORE RELYING ON IT. USE OF THE WEBSITE AND THE CONTENT AVAILABLE ON THE WEBSITE IS AT YOUR SOLE RISK.
MODIFICATION
These terms and conditions may be modified at any time. Such modifications shall be effectively immediately. Your continued use of the IVI website or Model Interface shall constitute acknowledgement and acceptance of any such modifications.
LIMITATION OF LIABILITY
IVI shall not be liable for any direct, indirect, consequential, special, exemplary, punitive, or any other monetary or other damages, fees, fines, penalties, or liabilities arising out of or related to this website, the Model Interface and/or information provided herein.
INDEMNIFICATION
To the furthest extent permitted by law, you shall indemnify, defend, and hold harmless IVI and its members from any and all liabilities, claims, and causes of action (including all costs, expenses and attorney’s fees on account thereof) arising out of your use of the Model Interface, including but not limited to claims that IVI or its members were solely negligent or responsible for any harm, loss, or other liability relating to your use of the Model Interface.
GOVERNING LAW
The law of the State of New York shall govern these terms and conditions, without reference to its choice of law rules.
SEVERABILITY
If any provision of these terms and conditions is held invalid or unenforceable by any court of competent jurisdiction, the other provisions of these terms and conditions shall remain in full force and effect.