### Introduction

### About

The IVI-RA Model Interface runs the Innovation and Value Initiative's open-source rheumatoid arthritis individual patient simulation model, which is part of the Open Source Value Project. The model estimates the value of disease-modifying anti-rheumatic drugs (DMARDs), which include conventional DMARDs (cDMARDs), biologic DMARDs (bDMARDs), and JAK inhibitors, for patients with moderate to severe rheumatoid arthritis. Full model documentation and source code is available here. By using this interface, you acknowledge and accept all terms and conditions.

### Quick start

**Simulate**: Run the simulation with default settings by clicking the Run simulation tab and clicking the*Simulate*button in the leftmost box.**View model results**: Examine results in the View model results tab. Expected clinicical and economic outcomes, cost-effectiveness analyses, and multi-criteria decision analyses are provided.

### Custom analysis

**Setup model**:**Population**: Select population characteristics.**Treatment sequences**: Choose up to five treatment sequences.**Model structure**: Select a structure for modeling the progression of rheumatoid arthritis, the impact of morbidity on on quality of life, and time to treatment discontinuation.**Parameter values**: Based on the model structure, choose values for input parameters.**Simulation settings**: Choose simulation setting such as the discount rate and time horizon.

**Run simulation**: Run the underlying simulation model. Choose the number of patients to simulate and the number of times to sample the parameters for the probabilistic sensitivity analysis (PSA).**View inputs used in simulation**: View your selected treatment sequences, a summary of the characteristics of the chosen population, and a summary of the distribution of each parameter included in the model.**View model results**:**Modify output settings**: Modify settings that can be changed without having to rerun the simulation including the choice of comparator and perspective (societal or health care sector).**Expected outcomes**: View survival curves and progression of costs, disease, and quality of life over time.**Cost-effectiveness analysis**: Examine standard cost-effectiveness results relating to the value of the treatment sequence and the uncertainty associated with that value including incremental cost-effectiveness ratios (**ICERs**) and summaries of the PSA.**Multi-criteria decision analysis**: Assess value by using multi-criteria decision analysis to weight different criteria.

### Type of cohort

### Treatment history

### Proportion male

### Weight in kilograms

### Patient age

### Number of previous DMARDs

### Disease activity and functional status at baseline

### Treatment sequences

Select up to five treatment sequences below. In each box, select the name of the treatments(s) that you would like to model. Each treatment you select will be added to the sequence, in order, from left to right. Two default treatment sequences have been preselected.

### Initial treatment phase (first 6 months)

### Time to treatment discontinuation

### HAQ progression in the absence of tDMARDs

### Utility algorithm

### Treatment effects at 6 months

### Statistical model for ACR response for tDMARD naive patients

A Bayesian random effects model for ordinal data was estimated. The model assumes that the four mutually exclusive ACR response categories follow a multinomial distribution. The probability of each category was modeled using a probit link function. Coefficients from the model are below.

*Estimate*

*Standard error*

### Probability of ACR response for tDMARD experienced patients relative to tDMARD naive patients

### Statistical model for change in HAQ at 6 months for tDMARD naive patients

A Bayesian random effects model for continuous data was estimated. The 'Mean for cDMARDs' estimate is the mean change in HAQ at 6 months for patients using cDMARDs. All other coefficients can be interpreted as the additional change in HAQ associated with a given treatment.

*Estimate*

*Standard error*

### Coefficient for tDMARD experienced patients relative to tDMARD naive patients

### Statistical model for change in DAS28 at 6 months for tDMARD naive patients

A Bayesian random effects model for continuous data was estimated. The 'Mean for cDMARDs' estimate is the mean change in DAS28 at 6 months for patients using cDMARDs. All other coefficients can be interpreted as the additional change in DAS28 associated with a given treatment.

*Estimate*

*Standard error*

### Coefficient for tDMARD experienced patients relative to tDMARD naive patients

### Treatment response mappings

### Relationship between ACR and EULAR response

### HAQ change by EULAR response category

*Estimate*

*Standard error*

### HAQ change by ACR response category

*Estimate*

*Standard error*

Treatment -> ACR -> HAQ

Treatment -> ACR -> EULAR -> HAQ

### Longterm HAQ progression

### Rebound post treatment

### Difference between overall and age-specific annual HAQ progression rate

*Mean*

*Standard error*

### Constant annual HAQ progression by treatment

*Estimate*

*Standard error*

### Mortality

### Log odds - impact of baseline HAQ on probability of mortality

### Log hazard ratio - impact of change in HAQ from baseline on mortality rate

*Estimate*

*Standard error*

### Time to treatment discontinuation

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

*Estimate*

*Standard error*

### Adverse events

### Log of serious infection rate by treatment

*Estimate*

*Standard error*

### Formal health care sector costs

### Treatment cost

### Hospital days per year by HAQ score range

*Estimate*

*Standard error*

### Cost per hospital day by HAQ score range

*Estimate*

*Standard error*

### Serious infection cost

### Annualized cost of managing RA

*Estimate*

*Standard error*

### Utility

### Serious infection utility loss

### Productivity loss

### Run simulation

### Number of patients to simulate

### Number of times to sample parameters for the PSA

### Discount rate

### Time horizon

### Population

### Treatment sequences

### Model structure

### Sampled parameter values

### Selected treatment sequences

### Mean clinical and economic outcomes

### Survival curves

### Mean HAQ over time

### Mean cumulative QALYs

### Mean cumulative costs

### Time to treatment discontinuation

##### Proportion switching during first 6 months

##### Proportion remaining on treatment conditional on not switching during the first 6 months

### Selected treatment sequences

### Modify settings

### Cost-effectiveness plane

### Incremental cost-effectiveness relative to comparator

### Cost-effectiveness acceptability frontier (CEAF)

### Probability of being most cost effective

### Cost-effectiveness acceptability curve (CEAC)

### Expected incremental net monetary benefit

### Distribution of incremental net monetary benefits

### Expected value of perfect information

### Selected treatment sequences

### MCDA criteria

### Linear partial value function

### Weighting

### Overall value

### Probability of ranking

### Selected treatment sequences

### What is value to the healthy?

Cost-effectiveness analyses typically estimate the value of a treatment to sick patients. Lakdawalla et al. (2017) provide a mathematical framework for analyzing value from the perspective of healthy individuals, who face the risk of disease in the future.

Treatments for rheumatoid arthritis provide value to healthy
individuals because they may develop rheumatoid arthritis in the future. We
refer to this value as **conventional value**.

Risk-averse individuals may derive additional value from
treatments for rheumatoid arthritis because these treatments reduce *
physical risk *. And while innovation certainly increases
* financial risk *, the increase in financial risk can be mitigated by
healthcare insurance. We refer to the value of treatments to risk-averse individuals
as ** insurance value without health insurance ** when they do not have
healthcare insurance and ** insurance value with health insurance** when
they do have health insurance.

### Modify settings

### Value to the healthy

### Glossary of key terms

**Arm**: A sequence of rheumatoid arthritis treatments used by a patient over their lifetime

**bDMARDs** Biologic DMARDs

**bDMARD experienced**: A patient that has previously failed at least one bDMARD

**bDMARD naive**: A patient that has previously failed cDMARDs but has not yet used a bDMARD

**CDAI**: The clinical disease activity index

**cDMARDs**: Conventional DMARDs

**Comparator**: The treatment arm used as a reference in the analysis

**CEAC**: Cost-effectiveness acceptability curve. The probability that each arm provides greater net monetary benefits than the comparator for a given willingness to pay

**CEAF**: Cost-effectiveness acceptability frontier. The probability that the optimal treatment (i.e., the treatment with the highest expected net monetary benefit) is cost-effective for a given willingness to pay

**Cost-effectiveness plane**: A scatter plot of the distribution of incremental costs and incremental QALYs from the PSA

**DAS28**: A disability activity score based on an assessment of 28 joints

**Discount factor**: Annual factor by which future costs and health gains should be discounted

**DMARDs**: Disease-modifying anti-rheumatic drugs

**EULAR response**: A response criteria that classifies patients as non-responders, moderate responders, or good responders to treatment based on the change and level of their DAS28 score

**Expected incremental net-monetary benefit**: The incremental net-monetary benefit averaged across all randomly sampled parameter sets in the PSA

**Expected value of perfect information**: The maximum amount that a decision maker would be willing to pay to obtain perfect information and reduce all uncertainty

**HAQ**: The health assessment questionnaire disability index. A patient reported outcome ranging from 0-3 measuring the degree of difficulty patients have with daily activities

**ICER**: Incremental cost-effectiveness ratio. Calculated by dividing incremental costs by incremental QALYs

**Incremental costs**: Differences in costs between a treatment arm and the comparator

**Incremental net-monetary benefit**: The difference in net monetary benefits between a treatment arm and the comparator arm

**Incremental QALYs**: Difference in QALYs between a treatment arm and the comparator

**LCGM**: Latent class growth model

**Net-monetary benefit**: The monetized value of a QALY (i.e., willingness to pay times the number of QALYs) minus costs

**Pairwise comparison**: Difference in outcomes between a given treatment arm and the comparator

**Probability of being most cost effective**: The number of samples from the PSA that each arm provides the greatest net monetary benefits among all arms for a given willingness to pay

**PSA**: Probabilistic sensitivity analysis. Propagates uncertainty in the input parameters throughout the model to quantify uncertainty in expected model outcomes. Generates a distribution of mean model outcomes by randomly sampling each parameter in the model from a probability distribution and calculating mean model outcomes for each randomly sampled parameter set

**QALYs**: Quality-adjusted life-years

**SDAI**: The simplified disease activity index

**Treatment arm**: Arms in the model that are compared to the comparator

**WAC**: Wholesale acquisition cost

**Willingness to pay**: Amount that a decision maker is willing to pay for an additional QALY

### Frequently asked questions

### What type of simulation is used?

The IVI-RA model is an individual patient simulation (IPS) that simulates patients one at a time.

### How long are the model cycles?

Model cycles are 6 months long, which is consistent with other models and with clinical trials.

### How many iterations do I need to run?

To get familiar with the interface and to explore the implications of different modeling assumptions, we suggest that you start by simulating 100 patients and sampling each parameter 100 times for the PSA. Simulations using these settings run in a few seconds and generate stable point estimates; however, this “quick” simulation may overestimate uncertainty because the model still has some stochastic uncertainty (i.e., uncertainty due to random variation among the simulated individuals). For users interested in completely eliminating the effect of stochastic uncertainty, IVI recommends increasing the number of simulated patients, but warns that this can significantly increase run time.

### How were the default settings selected?

The current default model structure is based on NICE's 2016 technology assessment. The default model structure may change after we receive public comments and the public comments are reviewed by an expert panel. Default parameter values are based on studies that, in our view, allow us to generate the most accurate estimates possible for a given model structure.

### How are bDMARD naive and a bDMARD experienced patients treated within the model?

The simulation can be run for either bDMARDs naive or bDMARDs experienced patients. When the model is run for bDMARDs naive patients, network meta-analysis (NMA) results for bDMARDs naive patients are used for first line therapies and NMA results for patients with inadequate response to biologic treatments are used for all subsequent lines. In contrast, when the model is run for bDMARDs experienced patients, NMA results for patients with inadequate response to biologic treatment are used for all treatment lines.

### What is the difference between a homogeneous and a heterogeneous cohort?

If the model is run for a homogeneous cohort, then the model simulates a cohort of male and female patients identical in all respects other than their weight and gender. If a heterogeneous cohort is selected, then characteristics vary across patients according to the settings chosen by the user.

### Why does the simulated mean HAQ score become increasingly noisy over time?

The model simulates the mean HAQ score for every individual until death, which means that the mean HAQ score is an average across patients that have survived to a given model cycle. Since the number of living simulated patients becomes smaller over time, there are fewer patients to calculate a mean HAQ score for, so it becomes increasingly noisy.

### What programming language is the model written in and where can I find the source code?

The model is written in R and available with the *iviRA* package; however, all computationally intensive code is written in C++ so that the model can be run in a reasonable amount of time. Documentation describing how to use the R package as well as links to the underlying source code on GitHub can be found here.

### Where can I find the model documentation?

A detailed description of the model is available on the R package website here.

### About this app

This web application runs version 2 of IVI-RA model, part of the Innovation and Value Initiative's Open Source Value Project. The underlying model is available as the iviRA R package. Model documentation is available on the package website here and source code for the package can be found on GitHub here.

The app is developed and maintained by Devin Incerti, Ming Xu, and Jeroen Jansen.

### Terms and conditions

### PLEASE READ THESE TERMS AND CONDITIONS CAREFULLY

These terms and conditions describe the terms and conditions applicable to your use of the Innovation and Value Initiative (“IVI”) website and Model Interface.

YOUR USE OF THIS WEBSITE AND MODEL INTERFACE IS CONDITIONED UPON YOUR ACCEPTANCE OF THE TERMS AND CONDITIONS. BY USING THE MODEL INTERFACE OR ACCESSING ANY PART OF THIS WEBSITE, YOU AGREE TO BE BOUND BY THE TERMS AND CONDITIONS. IF YOU DO NOT AGREE TO THESE TERMS AND CONDITIONS, YOU ARE NOT AUTHORIZED TO ACCESS OR USE THIS WEBSITE OR THE MODEL INTERFACE.

### THIS WEBSITE DOES NOT PROVIDE MEDICAL ADVICE

This website and the Model Interface are designed to provide information on value to a wide range of healthcare stakeholders, but not to be used as a clinical decision tool. Every effort has been made to provide the most current and correct information possible. Nonetheless, inadvertent technical inaccuracies or other errors may occur. IVI cannot be held responsible or liable for errors, inaccuracies, omissions or any perceived harm to users of this website or the Model Interface. Users of this information are encouraged to confirm the information contained herein with other sources. Any information is not intended to replace medical advice offered by healthcare providers. Use of this website or the Model Interface does not create a physician-patient relationship.

### NO WARRANTIES

ALL CONTENT ON THIS WEBSITE IS PROVIDED ON AN “AS IS” BASIS WITHOUT WARRANTY OF ANY KIND EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, ACCURACY AND NON-INFRINGEMENT. IVI MAKES NO WARRANTY AS TO THE ACCURACY, COMPLETENESS OR RELIABILITY OF ANY CONTENT AVAILABLE THROUGH THIS WEBSITE OR THE MODEL INTERFACE. YOU ARE RESPONSIBLE FOR VERIFYING ANY INFORMATION BEFORE RELYING ON IT. USE OF THE WEBSITE AND THE CONTENT AVAILABLE ON THE WEBSITE IS AT YOUR SOLE RISK.

### MODIFICATION

These terms and conditions may be modified at any time. Such modifications shall be effectively immediately. Your continued use of the IVI website or Model Interface shall constitute acknowledgement and acceptance of any such modifications.

### LIMITATION OF LIABILITY

IVI shall not be liable for any direct, indirect, consequential, special, exemplary, punitive, or any other monetary or other damages, fees, fines, penalties, or liabilities arising out of or related to this website, the Model Interface and/or information provided herein.

### INDEMNIFICATION

To the furthest extent permitted by law, you shall indemnify, defend, and hold harmless IVI and its members from any and all liabilities, claims, and causes of action (including all costs, expenses and attorney’s fees on account thereof) arising out of your use of the Model Interface, including but not limited to claims that IVI or its members were solely negligent or responsible for any harm, loss, or other liability relating to your use of the Model Interface.

### GOVERNING LAW

The law of the State of New York shall govern these terms and conditions, without reference to its choice of law rules.

### SEVERABILITY

If any provision of these terms and conditions is held invalid or unenforceable by any court of competent jurisdiction, the other provisions of these terms and conditions shall remain in full force and effect.