Introduction

About

The IVI-RA Model Interface runs the Innovation and Value Initiative's open-source rheumatoid arthritis individual patient simulation model, which is part of the Open Source Value Project. The model estimates the value of disease-modifying anti-rheumatic drugs (DMARDs), which include conventional DMARDs (cDMARDs), biologic DMARDs (bDMARDs), and JAK inhibitors, for patients with moderate to severe rheumatoid arthritis. Full model documentation and source code is available here. By using this interface, you acknowledge and accept all terms and conditions.

Quick start

  • Simulate: Run the simulation with default settings by clicking the Run simulation tab and clicking the Simulate button in the leftmost box.
  • View model results: Examine results in the View model results tab. Expected clinicical and economic outcomes, cost-effectiveness analyses, and multi-criteria decision analyses are provided.

Custom analysis

  1. Setup model:
    1. Population: Select population characteristics.
    2. Treatment sequences: Choose up to five treatment sequences.
    3. Model structure: Select a structure for modeling the progression of rheumatoid arthritis, the impact of morbidity on on quality of life, and time to treatment discontinuation.
    4. Parameter values: Based on the model structure, choose values for input parameters.
    5. Simulation settings: Choose simulation setting such as the discount rate and time horizon.
  2. Run simulation: Run the underlying simulation model. Choose the number of patients to simulate and the number of times to sample the parameters for the probabilistic sensitivity analysis (PSA).
  3. View inputs used in simulation: View your selected treatment sequences, a summary of the characteristics of the chosen population, and a summary of the distribution of each parameter included in the model.
  4. View model results:
    1. Modify output settings: Modify settings that can be changed without having to rerun the simulation including the choice of comparator and perspective (societal or health care sector).
    2. Expected outcomes: View survival curves and progression of costs, disease, and quality of life over time.
    3. Cost-effectiveness analysis: Examine standard cost-effectiveness results relating to the value of the treatment sequence and the uncertainty associated with that value including incremental cost-effectiveness ratios (ICERs) and summaries of the PSA.
    4. Multi-criteria decision analysis: Assess value by using multi-criteria decision analysis to weight different criteria.

Type of cohort

Treatment history

Proportion male

Weight in kilograms

Patient age

Number of previous DMARDs

Disease activity and functional status at baseline

Treatment sequences

Select up to five treatment sequences below. In each box, select the name of the treatments(s) that you would like to model. Each treatment you select will be added to the sequence, in order, from left to right. Two default treatment sequences have been preselected.


Initial treatment phase (first 6 months)

Time to treatment discontinuation

HAQ progression in the absence of tDMARDs

Utility algorithm

Treatment effects at 6 months

Statistical model for ACR response for tDMARD naive patients

A Bayesian random effects model for ordinal data was estimated. The model assumes that the four mutually exclusive ACR response categories follow a multinomial distribution. The probability of each category was modeled using a probit link function. Coefficients from the model are below.

Estimate

Standard error

Inputs are not required when modeling change in HAQ during the first 6 months using the pathway Treatment -> HAQ.

Probability of ACR response for tDMARD experienced patients relative to tDMARD naive patients

Specify the lower and upper bounds of the probability of ACR 20/50/70 for patients patients who have already failed at least one tDMARD as a proportion of the probability of ACR20/50/70 for patients using their first tDMARD. ACR20, ACR50, and ACR70 are defined as at least 20%, 50%, and 70% improvements respectively. This proportion is assumed to be uniformly distributed.

Inputs are not required when modeling change in HAQ during the first 6 months using the pathway Treatment -> HAQ.

Statistical model for change in HAQ at 6 months for tDMARD naive patients

A Bayesian random effects model for continuous data was estimated. The 'Mean for cDMARDs' estimate is the mean change in HAQ at 6 months for patients using cDMARDs. All other coefficients can be interpreted as the additional change in HAQ associated with a given treatment.

Estimate

Standard error

Inputs are only required when modeling change in HAQ during the first 6 months using the pathway Treatment -> HAQ.

Coefficient for tDMARD experienced patients relative to tDMARD naive patients

Specify the lower and upper bounds of the coefficient for patients who have already failed at least one tDMARD as a proportion of the coefficient for patients using their first tDMARD. This proportion is assumed to be uniformly distributed.

Inputs are only required when modeling change in HAQ during the first 6 months using the pathway Treatment -> HAQ.

Statistical model for change in DAS28 at 6 months for tDMARD naive patients

A Bayesian random effects model for continuous data was estimated. The 'Mean for cDMARDs' estimate is the mean change in DAS28 at 6 months for patients using cDMARDs. All other coefficients can be interpreted as the additional change in DAS28 associated with a given treatment.

Estimate

Standard error

Inputs are only required when modeling treatment switching during the first 6 months using the pathway Treatment -> ∆DAS28 -> DAS28 -> Switch

Coefficient for tDMARD experienced patients relative to tDMARD naive patients

Specify the lower and upper bounds of of the coefficient for patients who have already failed at least one tDMARD as a proportion of the coefficient for patients using their first tDMARD. This proportion is assumed to be uniformly distributed.

Inputs are only required when modeling treatment switching during the first 6 months using the pathway Treatment -> ∆DAS28 -> DAS28 -> Switch

Treatment response mappings

Relationship between ACR and EULAR response

The contingency table represents the frequency distribution of ACR and EULAR response. Each cell is the number of patients with a response in given ACR and EULAR category. Increasing the number of patients in a given cell increases the strength of the relationship between a given ACR category and EULAR cetegory. The numbers below can be edited by clicking in a cell.

HAQ change by EULAR response category

Estimate

Standard error

HAQ change by ACR response category

Estimate

Standard error

Inputs are only required when modeling the relationship between treatment and HAQ through one of the ACR response pathways:
Treatment -> ACR -> HAQ
Treatment -> ACR -> EULAR -> HAQ

Longterm HAQ progression

Rebound post treatment

Specify the lower and upper bounds of the rebound factor, which is defined as the HAQ increase at treatment discontinuation as a fraction of the initial HAQ improvement. The rebound factor is assumed to be uniformly distributed.

Difference between overall and age-specific annual HAQ progression rate

Mean

Standard error

Constant annual HAQ progression by treatment

If the LCGM model structure is chosen, HAQ progression for patients on cDMARDs is modeled using the LCGM; otherwise, the HAQ for patients on cDMARDs is assumed to change at a constant linear rate. In contrast, HAQ progression for patients on tDMARDs is always assumed to change at a constant linear rate. Below, choose the linear rate for the tDMARDs and, if the LCGM model structure was not chosen, for cDMARDs as well.

Estimate

Standard error

Mortality

Log odds - impact of baseline HAQ on probability of mortality

Log hazard ratio - impact of change in HAQ from baseline on mortality rate

Estimate

Standard error

Time to treatment discontinuation

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Estimate

Standard error

Time to treatment discontinuation is based on serious infection rates in the selected model structure.

Adverse events

Log of serious infection rate by treatment

Estimate

Standard error

Formal health care sector costs

Treatment cost

Manually change treatment cost inputs in the table below. Initial dosage refers to dosing during the first 6 months of treatment while annual dosage refers to dosage (per year) therafter (i.e., during the maintenance phase of the model). The amount by which price are decreased after accounting for discounts and rebates are assumed to range from "Discount (lower)" to "Discount (upper)".

Hospital days per year by HAQ score range

Estimate

Standard error

Cost per hospital day by HAQ score range

Estimate

Standard error

Serious infection cost

Annualized cost of managing RA

Estimate

Standard error

Utility

Utility during each model cycle is simulated as a function of the HAQ sore and patient characteristics using the utility mapping algorithm speciified during the selection of the model structure. In addition, each serious infection is associated with a one month loss of utility. Specifiy the loss of this utility in the box to the right.

Serious infection utility loss

Productivity loss

Specify the earnings loss ($) associated with a 1-unit increase in HAQ

Run simulation

Number of patients to simulate

Number of times to sample parameters for the PSA

Discount rate

Time horizon

Population

Treatment sequences

Model structure

Sampled parameter values

Selected treatment sequences

Mean clinical and economic outcomes

Note: 95% credible intervals are in parentheses.

Survival curves

Mean HAQ over time

Mean cumulative QALYs

Mean cumulative costs

Time to treatment discontinuation


Proportion switching during first 6 months
Proportion remaining on treatment conditional on not switching during the first 6 months

Selected treatment sequences

Modify settings

Cost-effectiveness plane


Incremental cost-effectiveness relative to comparator

Note: 95% credible intervals are in parentheses.

Cost-effectiveness acceptability frontier (CEAF)

Probability of being most cost effective

Cost-effectiveness acceptability curve (CEAC)


Expected incremental net monetary benefit


Distribution of incremental net monetary benefits


Expected value of perfect information

Selected treatment sequences

MCDA criteria

Linear partial value function

Outcomes for each criteria on the original scale are mapped to outcomes on a common scale ranging from 0 to 100 using a linear partial value function. In other words, the relationship between scores on the original scale and the common scale is linear. If low performance on the original scale is better, then the relationship between performance on the original scale and the common scale follows a negative straight line; converseley, if high performance on the original scale is better, then the relationship between performance on the original scale and the common scale follows a posive straight line.

Weighting

Overall value

Probability of ranking

Selected treatment sequences

What is value to the healthy?

Cost-effectiveness analyses typically estimate the value of a treatment to sick patients. Lakdawalla et al. (2017) provide a mathematical framework for analyzing value from the perspective of healthy individuals, who face the risk of disease in the future.

Treatments for rheumatoid arthritis provide value to healthy individuals because they may develop rheumatoid arthritis in the future. We refer to this value as conventional value.

Risk-averse individuals may derive additional value from treatments for rheumatoid arthritis because these treatments reduce physical risk . And while innovation certainly increases financial risk , the increase in financial risk can be mitigated by healthcare insurance. We refer to the value of treatments to risk-averse individuals as insurance value without health insurance when they do not have healthcare insurance and insurance value with health insurance when they do have health insurance.

Modify settings

Value to the healthy

Glossary of key terms

Arm: A sequence of rheumatoid arthritis treatments used by a patient over their lifetime

bDMARDs Biologic DMARDs

bDMARD experienced: A patient that has previously failed at least one bDMARD

bDMARD naive: A patient that has previously failed cDMARDs but has not yet used a bDMARD

CDAI: The clinical disease activity index

cDMARDs: Conventional DMARDs

Comparator: The treatment arm used as a reference in the analysis

CEAC: Cost-effectiveness acceptability curve. The probability that each arm provides greater net monetary benefits than the comparator for a given willingness to pay

CEAF: Cost-effectiveness acceptability frontier. The probability that the optimal treatment (i.e., the treatment with the highest expected net monetary benefit) is cost-effective for a given willingness to pay

Cost-effectiveness plane: A scatter plot of the distribution of incremental costs and incremental QALYs from the PSA

DAS28: A disability activity score based on an assessment of 28 joints

Discount factor: Annual factor by which future costs and health gains should be discounted

DMARDs: Disease-modifying anti-rheumatic drugs

EULAR response: A response criteria that classifies patients as non-responders, moderate responders, or good responders to treatment based on the change and level of their DAS28 score

Expected incremental net-monetary benefit: The incremental net-monetary benefit averaged across all randomly sampled parameter sets in the PSA

Expected value of perfect information: The maximum amount that a decision maker would be willing to pay to obtain perfect information and reduce all uncertainty

HAQ: The health assessment questionnaire disability index. A patient reported outcome ranging from 0-3 measuring the degree of difficulty patients have with daily activities

ICER: Incremental cost-effectiveness ratio. Calculated by dividing incremental costs by incremental QALYs

Incremental costs: Differences in costs between a treatment arm and the comparator

Incremental net-monetary benefit: The difference in net monetary benefits between a treatment arm and the comparator arm

Incremental QALYs: Difference in QALYs between a treatment arm and the comparator

LCGM: Latent class growth model

Net-monetary benefit: The monetized value of a QALY (i.e., willingness to pay times the number of QALYs) minus costs

Pairwise comparison: Difference in outcomes between a given treatment arm and the comparator

Probability of being most cost effective: The number of samples from the PSA that each arm provides the greatest net monetary benefits among all arms for a given willingness to pay

PSA: Probabilistic sensitivity analysis. Propagates uncertainty in the input parameters throughout the model to quantify uncertainty in expected model outcomes. Generates a distribution of mean model outcomes by randomly sampling each parameter in the model from a probability distribution and calculating mean model outcomes for each randomly sampled parameter set

QALYs: Quality-adjusted life-years

SDAI: The simplified disease activity index

Treatment arm: Arms in the model that are compared to the comparator

WAC: Wholesale acquisition cost

Willingness to pay: Amount that a decision maker is willing to pay for an additional QALY

Frequently asked questions

What type of simulation is used?

The IVI-RA model is an individual patient simulation (IPS) that simulates patients one at a time.

How long are the model cycles?

Model cycles are 6 months long, which is consistent with other models and with clinical trials.

How many iterations do I need to run?

To get familiar with the interface and to explore the implications of different modeling assumptions, we suggest that you start by simulating 100 patients and sampling each parameter 100 times for the PSA. Simulations using these settings run in a few seconds and generate stable point estimates; however, this “quick” simulation may overestimate uncertainty because the model still has some stochastic uncertainty (i.e., uncertainty due to random variation among the simulated individuals). For users interested in completely eliminating the effect of stochastic uncertainty, IVI recommends increasing the number of simulated patients, but warns that this can significantly increase run time.

How were the default settings selected?

The current default model structure is based on NICE's 2016 technology assessment. The default model structure may change after we receive public comments and the public comments are reviewed by an expert panel. Default parameter values are based on studies that, in our view, allow us to generate the most accurate estimates possible for a given model structure.

How are bDMARD naive and a bDMARD experienced patients treated within the model?

The simulation can be run for either bDMARDs naive or bDMARDs experienced patients. When the model is run for bDMARDs naive patients, network meta-analysis (NMA) results for bDMARDs naive patients are used for first line therapies and NMA results for patients with inadequate response to biologic treatments are used for all subsequent lines. In contrast, when the model is run for bDMARDs experienced patients, NMA results for patients with inadequate response to biologic treatment are used for all treatment lines.

What is the difference between a homogeneous and a heterogeneous cohort?

If the model is run for a homogeneous cohort, then the model simulates a cohort of male and female patients identical in all respects other than their weight and gender. If a heterogeneous cohort is selected, then characteristics vary across patients according to the settings chosen by the user.

Why does the simulated mean HAQ score become increasingly noisy over time?

The model simulates the mean HAQ score for every individual until death, which means that the mean HAQ score is an average across patients that have survived to a given model cycle. Since the number of living simulated patients becomes smaller over time, there are fewer patients to calculate a mean HAQ score for, so it becomes increasingly noisy.

What programming language is the model written in and where can I find the source code?

The model is written in R and available with the iviRA package; however, all computationally intensive code is written in C++ so that the model can be run in a reasonable amount of time. Documentation describing how to use the R package as well as links to the underlying source code on GitHub can be found here.

Where can I find the model documentation?

A detailed description of the model is available on the R package website here.

About this app

This web application runs version 2 of IVI-RA model, part of the Innovation and Value Initiative's Open Source Value Project. The underlying model is available as the iviRA R package. Model documentation is available on the package website here and source code for the package can be found on GitHub here.

The app is developed and maintained by Devin Incerti, Ming Xu, and Jeroen Jansen.

Terms and conditions

PLEASE READ THESE TERMS AND CONDITIONS CAREFULLY

These terms and conditions describe the terms and conditions applicable to your use of the Innovation and Value Initiative (“IVI”) website and Model Interface.

YOUR USE OF THIS WEBSITE AND MODEL INTERFACE IS CONDITIONED UPON YOUR ACCEPTANCE OF THE TERMS AND CONDITIONS. BY USING THE MODEL INTERFACE OR ACCESSING ANY PART OF THIS WEBSITE, YOU AGREE TO BE BOUND BY THE TERMS AND CONDITIONS. IF YOU DO NOT AGREE TO THESE TERMS AND CONDITIONS, YOU ARE NOT AUTHORIZED TO ACCESS OR USE THIS WEBSITE OR THE MODEL INTERFACE.

THIS WEBSITE DOES NOT PROVIDE MEDICAL ADVICE

This website and the Model Interface are designed to provide information on value to a wide range of healthcare stakeholders, but not to be used as a clinical decision tool. Every effort has been made to provide the most current and correct information possible. Nonetheless, inadvertent technical inaccuracies or other errors may occur. IVI cannot be held responsible or liable for errors, inaccuracies, omissions or any perceived harm to users of this website or the Model Interface. Users of this information are encouraged to confirm the information contained herein with other sources. Any information is not intended to replace medical advice offered by healthcare providers. Use of this website or the Model Interface does not create a physician-patient relationship.

NO WARRANTIES

ALL CONTENT ON THIS WEBSITE IS PROVIDED ON AN “AS IS” BASIS WITHOUT WARRANTY OF ANY KIND EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, ACCURACY AND NON-INFRINGEMENT. IVI MAKES NO WARRANTY AS TO THE ACCURACY, COMPLETENESS OR RELIABILITY OF ANY CONTENT AVAILABLE THROUGH THIS WEBSITE OR THE MODEL INTERFACE. YOU ARE RESPONSIBLE FOR VERIFYING ANY INFORMATION BEFORE RELYING ON IT. USE OF THE WEBSITE AND THE CONTENT AVAILABLE ON THE WEBSITE IS AT YOUR SOLE RISK.

MODIFICATION

These terms and conditions may be modified at any time. Such modifications shall be effectively immediately. Your continued use of the IVI website or Model Interface shall constitute acknowledgement and acceptance of any such modifications.

LIMITATION OF LIABILITY

IVI shall not be liable for any direct, indirect, consequential, special, exemplary, punitive, or any other monetary or other damages, fees, fines, penalties, or liabilities arising out of or related to this website, the Model Interface and/or information provided herein.

INDEMNIFICATION

To the furthest extent permitted by law, you shall indemnify, defend, and hold harmless IVI and its members from any and all liabilities, claims, and causes of action (including all costs, expenses and attorney’s fees on account thereof) arising out of your use of the Model Interface, including but not limited to claims that IVI or its members were solely negligent or responsible for any harm, loss, or other liability relating to your use of the Model Interface.

GOVERNING LAW

The law of the State of New York shall govern these terms and conditions, without reference to its choice of law rules.

SEVERABILITY

If any provision of these terms and conditions is held invalid or unenforceable by any court of competent jurisdiction, the other provisions of these terms and conditions shall remain in full force and effect.