Opening Up ICER's Black Box

The Institute for Clinical Effectiveness Review (ICER) has been making a name for itself as it undertakes value assessment of new medicines. The Institute has been working for over ten years, and not everyone has been supportive of their work. One of the criticisms has been a lack of transparency in ICERs approach; something that ICER has been well aware of. Now ICER is opening up the black box.

At the end of March, ICER released the details of a pilot program that will allow manufacturers to scrutinize executable versions of draft cost-effectiveness models. Now companies will be able to take a deeper look into the calculations that can result in headlines proclaiming poor – or good – value for money for their products. Companies will be able to look at the structure of models, the estimates, assumptions and the calculations. The pilot covers reviews of therapies for migraine prevention and elagolix for endometriosis-associated pain.

ICER says it’s doing this to better equip companies to provide valuable feedback during the public consultation phase of ICER’s work on a new drug. Companies will have to pay a fee though, to get access. This is to cover the costs of producing sharable executable files and supporting documentation. Companies will also need to sign up to agreements covering what is to be kept confidential or sign up to a licensing agreement. Companies won’t be able to change the model though; it’s strictly to help them contribute comments to ICER, not to use the model for any other use.

ICER has discussed the ambition of sharing more in the past, and the pilot has been able to overcome issues such as maintaining academic intellectual property, at least with the specific academics and institutions involved in the pilot.

The pilot is starting with providing access for companies, but ICER has signalled that in the future others – like patient groups – could see the same level of detail. Quite what they’ll do with it is unclear, as not everyone will have the expertise to be able to critique. That might open bigger questions about meaningful engagement with multiple stakeholder groups.

Just as is now common for ICER, their announcement has prompted responses. The Innovation and Value Initiative Program (IVI) – brainchild of the consulting firm, the Precision Medicine Group - has pointed out that models should be available to all, and further that such models should be customizable. They also raise the concern that confidentiality or licensing can limit the flow of information. Worth noting that IVI is running an open-source value project: essentially open source models, starting with rheumatoid arthritis and covering a number of products currently used in its treatment.

It’s not just ICER and its models where there is debate about transparency for the black boxes of cost effectiveness analysis for drugs. There is a wider debate too; experience with ICER could be relevant to how other agencies and academics approach the issue, so expect to hear more in the future.