As chair of the Scientific Advisory Committee (SAG) of the Multiple Sclerosis Risk Sharing Scheme I strongly reject the suggestion that our advice is based on any considerations other than scientific rigour.¹

Members of DH and of the 4 companies attend meetings with observer status only. Final decisions relating to the scheme, e.g. on whether to apply price adjustments at any point, are taken between the parties to the scheme informed by advice from my committee.

I have no doubt that the scientific advice for which the SAG was responsible was correct – it would have been unsafe to draw any conclusions about the effectiveness of the MS drugs from the result of the two year follow up. The reasons for this conclusion are well set out in the BMJ paper describing these two year results.²

They relate in large part to the construction and interpretation of the counter-factual data-set (the data-set with which the RSS data were compared). It turned out that the scheme did not have access to the raw counterfactual data but only to data derived from the raw data.

Moreover it proved impossible to work out precisely how the derived data had been calculated from the raw data. The upshot is that we did not know how exactly to mimic the derivation process so that the two data sets could be compared on a level playing field.

For this and other reasons, I wrote to the Department of Health before the data analysis was undertaken to say that I did not think a valid comparison could be made at the two year point and to say that we were searching for an alternative counterfactual data set more suited to this particular purpose.

McCabe and colleagues ³ imply that biases can not explain the results later obtained but they are wrong, since the sensitivity analyses show that the conclusions can be reversed according to the assumptions made (only a proportion of which were discussed by McCabe). Do these authors really think that an uncertain analysis based on observational data at two years should trump the RCT results at the same time period?

The SAG is charged only with providing scientific advice on the risk sharing scheme. However, I would like to respond to some of the policy related comments in the critical articles as citizen, albeit with experience of the scheme.

I discern two broad policy relevant criticisms:

1. The price of the drugs should have been adjusted (downwards) notwithstanding the uncertain parameter estimates mentioned above.

2. The Risk Sharing Scheme was a bad idea.

Regarding the price adjustment (or lack thereof) critics argue that the rules determining price adjustment had been set in advance and hence the price should have been adjusted on the basis of the observed data quite irrespective of the massive sensitivity of the estimate of effectiveness to un-testable assumptions in the analysis plan. Since the methodology for the scheme had not been worked out when it was agreed in 2002,

I understood that one of the functions of the SAG was to advise the scheme on the validity of interim results. The results were silent on the actual effectiveness of the drugs which is what the parties to the agreement were betting on. At least in retrospect this was an unsound bet, since neither side understood the risks to which tax-payers and share-holders were exposed. Nullifying a bet is only a bad idea if it can lead to ‘moral hazard’. That can only arise if the intention is to repeat wagers of this type.

This takes me to the second point – the suitability of price adjustment schemes. I have some sympathy with Raftery’s argument that such schemes are a poor policy instrument and, if used at all, I do not think that they should be used for short term decision making.

My reluctance to accept observational data to measure effectiveness has been reinforced by the fiendish methodological difficulties encountered by this risk sharing scheme.

However, the story is not yet fully told – we have read only the first of five exciting instalments. Follow up in the clinical trials on which a licence was granted averages about two years. Cost-effectiveness estimates are based on a model in which the two year data are extrapolated over ten years. This model is more sensitive to long term outcomes than to outcomes at two years. Observations over 10 years of follow up will be a substantial improvement on extrapolation of data from trials that followed patients up for only two years. Moreover the risk sharing scheme plans to base further analysis on an alternative counter- factual data set that the SAG believes will overcome some of the difficulties described in the BMJ article, since we will now have access to raw data.

The study will provide important prognostic data along with information on the relationship between short and long term effectiveness. Such data are available for the common cancers but not for multiple sclerosis.⁴ In summary, I advocate long term observational studies when trials have been truncated before the collection of the most important long term data and I agree with Raftery that price adjustment mechanisms are a poor policy instrument for short term price adjustment.

¹. Raftery J Multiple sclerosis risk sharing scheme: a costly failure. BMJ 2010;340:C1672

². Boggild M, Palace J, Barton P, Ben-Shlomo Y, Bregenzer T, Dobson C, Gray R. Multiple sclerosis risk sharing scheme: two year results of clinical cohort study with historical comparator. BMJ. 2009;339:b4677.

³. McCabe et al. Continuing the multiple sclerosis risk sharing scheme is unjustified. BMJ 2010;340:C1786

⁴. Bowater RJ, Bridge LJ and Lilford RJ. The relationship between progression- free and post-progression survival in treating four types of metastatic cancer. Cancer Letters 262(1): 48-53; 2008.

Competing interests: I am chair of the Scientific Advisory Committee to the risk sharing scheme

Richard J Lilford,
Prof of Clinical Epidemiology
University of Birmingham

Source: British Medical Journal © 2010 BMJ Publishing Group Ltd. (08/06/10)

The most expensive publicly funded drug trial in history is condemned today as a "fiasco" which has wasted hundreds of millions of NHS cash and raised fresh concerns about the influence of the pharmaceutical industry.

The scheme involved four drugs for multiple sclerosis launched in the 1990s, Biogen Inc's Avonex, Bayer's Betaseron, and Merck KGaA's Rebif, as well as Copaxone from Israel's Teva Pharmaceuticals, which were hailed as the first treatment to delay progression of the disabling neurological condition that affects 100,000 people in the UK.

It was set up in 2002 after the National Institute for Clinical Excellence (Nice) unexpectedly ruled that the drugs were not cost effective and should not be used on the NHS. To head off opposition from patient groups and the pharmaceutical industry, the Department of Health established the largest NHS "patient access scheme", to provide patients with the drugs, costing an average £8,000 a year, on the understanding that if they turned out to be less effective than expected, the drug companies would reduce the price.

The first report on the outcome was due after two years but was not published until last December, seven years later. It showed that the drugs failed to delay the onset of disability in patients – defined as walking with a stick or using a wheelchair – and may even have hastened it. On that basis, the drug companies would have had to pay the NHS to use them to make them cost effective.

Despite this finding, the price was not reduced and the scientific advisory group monitoring the scheme advised that "further follow up and analyses" were required. It said that disability may yet improve, the disease may have become more aggressive and the measure of disability used may have underestimated benefit. There were 5,583 patients in the scheme at a cost to the NHS of around £50m a year, amounting to £350m over seven years to 2009. The Multiple Sclerosis Society said twice as many patients were using the drugs outside the trial, implying a total NHS cost of £700m for a treatment that does not work.

In a series of articles in today's British Medical Journal, experts criticise the scheme. James Raftery, professor of health technology assessment at the University of Southampton and an adviser to Nice, said the scientific advisory group included representatives from the four drug companies, two MS groups, and the neurologists treating patients, all of whom had lobbied for the continued use of the drugs on the NHS.

"The independence of this group is questionable," he said. "Monitoring and evaluation of outcomes must be independent. Transparency is essential, involving annual reports, access to data, and rights to publish. Any of these might have helped avoid the current fiasco."

"The scheme was a success for the drug companies, who sold at close to full price to the NHS. For the NHS, however, the scheme can be judged only a costly failure."

Professor Christopher McCabe, head of health economics at the University of Leeds, writing with colleagues in the BMJ, said: "None of the reasons for delaying the price review withstand critical assessment." Professor McCabe told The Independent: "We should be asking questions about paying for these drugs. In terms of disability avoidance, the evidence is not there."

And in a second commentary, George Ebers, Professor of Clinical Neurology at the University of Oxford, believes that the outcome measures used in the scheme were flawed. He also says that the scheme’s findings raise questions about industrial-academic relationships and their governance. “The scheme may have been well intentioned, but perhaps the public interest would be served by an independent inquiry,” he writes.

Alastair Compston, professor of neurology at the University of Cambridge, defended the scheme. He said that despite a disappointing outcome, the scheme had "advanced the situation for people with multiple sclerosis" by improving understanding and care of the disease. Neil Scolding, professor of neurosciences at the University of Bristol, said the proportion of British patients treated with drugs (10-15 per cent) was tiny compared to France and Germany (40-50 per cent). He said the scheme had also led to the appointment of 250 multiple sclerosis nurses.

"[Though] expensive and flawed, if it turns out to have been no better than a clever wooden horse, then the army of MS healthcare specialists it delivered may make it more than worthwhile," he wrote. The MS Society claimed success for the scheme up to 2007 but after publication of the results last December, withdrew its support.

MS: why the drugs don't work

Multiple sclerosis is a chronic disease. It may take 40 years to run its course. In developing drugs to slow its progression, doctors have used brain scans to show lesions which the drugs appeared to prevent, and gave quicker results. Some experts thought the lesions were the disease but little effort was made to check. But preventing lesion formation does not prevent disability caused by the condition. The drugs deal with the lesions, not the disease.

Multiple sclerosis risk sharing scheme: a costly failure - full BMJ article.

Source: The Independent Copyright 2010 Independent Print Limited  & Reuters © Thomson Reuters 2010 (04/06/10)

Since 2002, the scheme has enabled the NHS to prescribe the drugs while their cost-effectiveness was evaluated.

But the charity argues the study is flawed, and fears the scheme is hindering access to other therapies. "The scheme has been so badly run, so badly administered and has so many problems with the methodology and the data collection that's there's nothing meaningful coming out of it."

The government insists the scheme has significantly improved the overall care and support available to MS patients.

The Government has spent around 350m pounds on drugs for people with Multiple Sclerosis despite the drug watchdog NICE ruling they weren’t cost effective. Eight years after a compromise agreement was reached with the pharmaceutical industry to make disease modifying drugs available, the UK's largest MS charity is calling for the risk-sharing scheme that made it possible to be scrapped.

The Donal MacIntyre programme investigated the scheme’s progress and heard serious concerns about the clinical study designed to test how well the drugs work.

You can listen to programme by clicking the link below:

Donal MacIntyre programme 21/02/10

Objective To generate evidence on the longer term cost effectiveness of disease modifying treatments in patients with relapsing-remitting multiple sclerosis.
Design Prospective cohort study with historical comparator.

Setting Specialist multiple sclerosis clinics in 70 centres in the United Kingdom.

Participants Patients with relapsing-remitting multiple sclerosis who started treatment from May 2002 to April 2005 under the UK risk sharing scheme.

Interventions Treatment with interferon beta or glatiramer acetate in accordance with guidelines of the UK Association of British Neurologists.

Main outcome measures Observed utility weighted progression in disability at two years’ follow-up assessed on the expanded disability status scale (EDSS) compared with that expected by applying the progression rates in a comparator dataset, modified for patients receiving treatment by multiplying by the hazard ratio derived separately for each disease modifying treatment from the randomised trials.

Results In the primary per protocol analysis, progression in disability was worse than that predicted and worse than that in the untreated comparator dataset ("deviation score" of 113%; excess in mean disability status scale 0.28). In sensitivity analyses, however, the deviation score varied from –72% (using raw baseline disability status scale scores, rather than applying a "no improvement" algorithm) to 156% (imputing missing data for year two from progression rates for year one).

Conclusions It is too early to reach any conclusion about the cost effectiveness of disease modifying treatments from this first interim analysis. Important methodological issues, including the need for additional comparator datasets, the potential bias from missing data, and the impact of the "no improvement" rule, will need to be addressed and long term follow-up of all patients is essential to secure meaningful results. Future analyses of the cohort are likely to be more informative, not least because they will be less sensitive to short term fluctuations in disability.

Mike Boggild, consultant neurologist¹, Jackie Palace, consultant neurologist², Pelham Barton, senior lecturer in mathematical modelling³, Yoav Ben-Shlomo, professor of clinical epidemiology⁴, Thomas Bregenzer, director, biostatistics⁵, Charles Dobson, senior projects officer⁶, Richard Gray, director⁷

¹ The Walton Centre, Liverpool L9 7LJ, ² John Radcliffe Hospital, Oxford OX3 9DU, ³ Health Economics Unit, University of Birmingham, Birmingham B15 2TT, ⁴ Department of Social Medicine, Canynge Hall, Bristol BS8 2PS, ⁵ Biostatistics Unit, Parexel International, 14050 Berlin, Germany, ⁶ Department of Health, Quarry House, Leeds LS2 7UE, ⁷ University of Birmingham Clinical Trials Unit, Birmingham B15 2TT

Source: BMJ © 2009 BMJ Publishing Group Ltd.  (02/12/09)

The risk-sharing scheme between the government and pharmaceutical companies to supply Beta-interferon to patients is going more slowly than was hoped and may not be completed during the 18 month period.

The findings, made by the MS Trust and the MS Society, have been presented to the Department of Health. Although progress is being made in many respects, fundamental issues surrounding funding and resources remain.

The research was carried out among MS Lead Officials in Strategic Health Authorities and Health Authorities/Health Boards in England, Scotland and Wales. The aim was to gain an accurate picture of health bodies' progress in implementing the risk-sharing scheme and to explore some of pharmaceutical companies to the issues surrounding take-up.

On the positive side, the scheme appears to have been widely accepted and is seen as compulsory. As a result, it is being implemented across England, Scotland and Wales and all participants were hopeful that they would have the scheme fully in place by the end of 2003.

However, inadequate funding is still a key issue. Some areas state that sufficient funds have yet to be allocated, although the start of the new financial year should help resolve this problem.

A lack of suitably trained staff is also hampering progress. This is leading to lengthy waiting times for patients to be assessed and treated.

An estimated 1,600 Scots with multiple sclerosis are set to benefit from a deal to make Beta-interferon available to all those who may be helped by the drug. MS patients north of the border have welcomed the agreement between the government and drug manufacturers.

Not all Scottish health boards have been able to afford the drug, which can slow down the progress of the disease in some patients.

But under the risk-sharing scheme announced on Monday, the National Health Service will pay for beta interferon and other similar drugs where they prove effective.

"I congratulate everyone who has played a part, including those who worked for the recent record-breaking petition to the Scottish Parliament" said  Mark Hazelwood, MS Society for Scotland However, the industry will have to foot a proportion of the bill where they are not successful.

Neurologists will now begin the process of assessing patients.

An estimated 10,500 people in Scotland have from MS, a disease of the nervous system for which there is no cure. It is thought that some 1,600 Scottish MS patients will be deemed suitable to receive the drugs over the next 18 months.

The Multiple Sclerosis Society for Scotland said the move was a major victory for people with MS.

Source: BBC News Copywrie BBC MMII (04/02/02)

The National Institute of Clinical Excellence (NICE) have made their final decision, that Betainterferon and Copaxone (glatiramer acetate) are not to be made available on the NHS to people with MS who are not already getting them. NICE argued that neither drug had been proven to bring enough benefit to enough people.

However, extended clinical trials to study the effectiveness of these drugs in treating MS have been proposed by the government.

Under the scheme, up to 10,000 patients who meet the clinical criteria for therapy would be prescribed the treatment and monitored over a number of years. The Department of Health wants to establish just how many MS patients would benefit from using these drugs.

Its' own advisory body has warned that both treatments are not currently cost-effective, and should only be made widely available if a new way to improve value for money is found.

The scheme will go ahead if the Department of Health can negotiate a deal to share the financial risk between the government and the drug companies. Chief executive of the MS Society, Peter Cardy, welcomed the proposed trials saying: "We know this drug does not work for everybody, and that its benefits are limited. But if this trial takes place, then we should at the end of it know much more about who exactly the drug works for, and what its benefits are."

Source:  BBC News Copywrite BBC MMII (04/02/02)

Following the Department of Health's announcement today (Monday, 4
February 2002), a statement was issued jointly by the Association of British Neurologists, the Multiple Sclerosis Society, the MS Trust and the UK MS Specialist Nurses Association:

"We warmly welcome the Department of Health's scheme. While NICE has been evaluating the new disease modifying treatments for MS over more than two years, we have advocated that these drugs should be available to patients in the UK.

We intend now to work closely with the Department to help ensure the scheme is effectively implemented. This will be a major undertaking.

There will be heavy demand on health authorities and primary care trusts to fund the treatments. That must not compromise services to other NHS patients, including those with MS for whom the drugs are not suitable.

The new scheme will mean that as many as 20 to 30,000 MS patients may need to be assessed to identify around 10,000 expected to be eligible under the ABN guidelines.

To avoid causing distressing delays to patients, this will require major expansion of neurological services. Additional sessions will be needed
for neurologists, MS specialist nurses, medical secretaries and other support staff. We understand the funding to do this has been agreed nationally and will be implemented locally to allow neurologists to assess all potentially eligible patients, treat those who are eligible, and then follow them up over the next 10 years to assess their progress.

We know that the Department intends to ensure that the required additional therapies and infrastructure are provided through health authorities and primary care trusts without compromising existing
services. Neurological services are already very stretched, with outpatient waiting times more than six months in many areas and double that in some.

Our organisations will co-operate fully in the scheme and expect to play an active role in monitoring and feeding back information through regular meetings with the Department."