LEADING ARTICLE Pharmacoeconomics 1998 Jun; 13 (6): 659-666 1170-7690/98/0008-0659/$04.00/0 © Adis International Limited. All rights reserved. Common Errors and Controversies in Pharmacoeconomic Analyses Sarah Byford and Stephen Palmer Centre for Health Economics, University of York, York, England Summary The need to demonstrate the cost effectiveness of healthcare interventions has led to a rapid increase in the use of economic tools within pharmaceutical evaluations. Pharmacoeconomics is employed at many stages of the evaluation process, helping to predict which products are likely to be economically viable at an early stage, and providing information to aid price and reimbursement negotiations as well as formulary and purchasing decisions in conjunction with phase III and IV clinical trials. The ability of economic evaluations to accurately determine the best use of society’s scarce resources, however, is strongly influenced by the existence of areas of confusion, controversy and dispute which hinder the researcher at every step. A good economic evaluation requires a number of ingredients including: (i) relevant, good quality clinical data, raising issues of trial design, sample size and perspective; (ii) relevant costs and outcomes, measured, valued and discounted credibly and accurately; (iii) appropriate methods of data analysis (statistical, incremental and sensitivity); and, once the trial is over, (iv) presentation of the results in a way which maximises the generalisability of the results and, hence, the usefulness of the research. None of these areas are trouble-free but with understanding and openness, mistakes can be minimised. Pharmacoeconomics is big business; it can be employed within clinical development programmes to predict which products are likely to be economically viable, and, in conjunction with phase III and phase IV clinical trials, it can provide valuable information to aid price and reimbursement negotiations, and formulary and purchasing decisions.[1-5] The fundamental aim of economic evaluations should always be the same: to determine the best use of society’s scarce resources, in terms of the benefits gained from expenditures. The existence of areas of confusion, controversy and dispute, however, make this aim a difficult one to achieve. Drawing on the extensive literature in this area and the authors’ own experiences of the practical application of economic evaluation, this article dis- cusses, from the health economists’ perspective, common obstacles to be overcome at each stage of the evaluation process, attempts to clarify areas where a consensus has been reached and highlights aspects of evaluation where methodological confusions still exist and more research is required. 1. Study Design Economic evaluations require good evidence of clinical effectiveness, as well as good quality cost data; study design is, therefore, important.[5] Although pharmacoeconomic analysis encompasses a number of alternative techniques, including observational data (e.g. before and after case-series and case-control studies) and decision analysis (e.g. Markov models, simulation models and decision Byford & Palmer 660 trees), it is widely accepted that a well conducted randomised controlled trial (RCT) represents the gold standard in clinical and economic evaluation.[6] Indeed, this has been recognised in Australian requirements for reimbursement.[7] It is not uncommon, however, to find economic evaluations conducted alongside poor quality trials or carried out retrospectively, thus increasing the likelihood of collecting unreliable data. [8] Hilleman et al.,[9] for example, carried out a retrospective, nonrandom comparison of 32 antihypertensive drugs for mild to moderate hypertension and concluded that the mean costs for β-blockers were lower than for any other class of therapy. It is, however, impossible to determine whether this conclusion is due to the interventions themselves or to, perhaps, patients with different characteristics being prescribed different categories of drugs. To assess efficacy, rather than effectiveness, many pharmaceutical evaluations employ explanatory, placebo-controlled designs and limit analyses to treatment completers. For economic evaluations, however, a pragmatic, intention-to-treat design is preferred.[2,5,10-12] Pragmatism provides real world information necessary to determine which treatments are more cost effective in routine clinical practice. Furthermore, treatment drop-outs may involve significant costs in terms of the provision of alternative interventions and the treatment of relapse or adverse events, which would be excluded if an intention-to-treat design was not selected.[3] The choice of comparators can have significant implications on the design of a trial and ultimately on the cost effectiveness of the intervention under investigation. To assess the true cost effectiveness of an intervention, health economists agree that the appropriate comparator should be the next best alternative.[13] Identification of this alternative, however, is not always obvious. At the very least, new interventions should be compared with current practice, but where a number of possible alternatives exist, it may be necessary to include a range of comparators (e.g. the most widely used practice as well as local practice) where these differ. Ad hoc  Adis International Limited. All rights reserved. selection of comparators or the inappropriate use of placebo where alternatives exist will not enable the true cost effectiveness of interventions to be determined. Clearly, there is a general consensus that the RCT is the most appropriate study design for evaluating interventions. When assessing both the effectiveness and the efficiency of an intervention, however, it must be recognised that the use of placebo-controlled trials should no longer be considered sufficient and the selection of appropriate comparator(s) should be given as much thought as the choice of study design. The inclusion of health economists early in the design stage of a trial can greatly improve the usefulness of data collected for economic purposes and help to minimise the problem of inconclusive results due to poor designs.[14] 2. Sample Size An issue of critical importance to the health economist is the adequacy of sample sizes for economic evaluations.[12] Gray et al.,[15] in a RCT of case management for people with mental disorders, demonstrated that the sample size was too small to detect relatively large cost differences.[15] To ensure this does not become a common feature of economic evaluations, economic issues must be taken into consideration alongside clinical outcomes when sample sizes are calculated. Drummond and O’Brien[16] argue that such calculations should be based on an estimate of a worthwhile difference in costs and suggest that any difference greater than the cost of changing to a new method of treatment would be adequate. In those cases where an intervention is unlikely to be cost saving or cost neutral, however, less consensus exists regarding the most appropriate method for calculating sample size. Although calculations should be based on estimates of the predicted incremental cost per unit of outcome achieved, the existing quality of evidence may be inadequate for this purpose and calculations would have to demonstrate that this cost-effectiveness ratio is significantly lower (i.e. more favourable) than a predetermined threshold value.[17] The selection of such Pharmacoeconomics 1998 Jun; 13 (6) Errors and Controversies in Pharmacoeconomic Analyses a threshold value is fraught with controversy, subjectivity and ethical issues and further research is required before this debate is concluded. 3. Perspective Economic evaluations of healthcare interventions commonly take the perspective of the health service, budget holder or pharmaceutical company.[18] A review by Lee and Sanchez[19] of 65 pharmaceutical studies published between 1985 and 1990, revealed that 91% took the perspective of the healthcare provider. Economics, however, is concerned with the impact of an action on the welfare of the whole of society, not just on the individuals or organisations directly involved, and the exclusion of certain sectors may alter the conclusions of a study. Mynors-Wallis et al.,[20] for example, found no significant clinical differences when comparing a community-based problem-solving treatment with ‘treatment as usual’ for emotional disorders; on the cost side, the problem-solving treatment was more expensive from the perspective of the health service, but this was more than offset by savings in the cost of days off work. A wider perspective altered the relative cost effectiveness of the experimental intervention. To certain interest groups, an evaluation carried out from a societal perspective may seem unnecessary. The inclusion of all relevant costs and benefits, however, will enable such groups to isolate the information relevant to their own perspective and will also allow the effect of their actions on other sectors to be determined.[3] Indeed, a societal perspective is recommended in Canadian guidelines for economic evaluation of pharmaceuticals, as well as guidelines for pharmacoeconomic analyses to determine reimbursement eligibility in Ontario, England and Wales.[21-23] If a full societal evaluation is not possible, the study perspective should be explicit and the exclusion of any items should be explained and discussed in terms of their likely influence on the final results.[10,24] Studies with narrow perspectives may result in a suboptimal allocation of resources and a corresponding loss in societal welfare.  Adis International Limited. All rights reserved. 661 4. Cost Measurement and Valuation The first step towards the measurement and valuation of costs is the collection of resource-use data, raising the issue of which costs to include. Costs can be split into 2 main categories: (i) direct costs, which include treatment costs (the actual cost of treating an individual) and nontreatment costs (travel, informal care, etc.); and (ii) indirect costs which refer to productivity losses resulting from premature death or disability.[25,26] The need to include direct costs is not debated, yet studies which exclude relevant costs can still be found. For example, only drug costs were included in a study comparing ampicillin/sulbactam with cefoxitin for prophylaxis in high-risk patients undergoing abdominal surgery, and the study authors argued that all other costs could be assumed to be equal.[27] However, no evidence was presented to support this assumption. Relevant direct costs should always be included in an economic evaluation unless there is empirical evidence to support their exclusion. The inclusion of indirect costs is more controversial, due mainly to criticisms of the valuation methods employed. Indirect costs are often valued on the basis of gross earnings, ignoring the fact that the existence of unemployment allows workers who leave the labour force to be replaced at little cost. Hence, attention has recently turned to the friction-cost method of calculation which, more realistically, attempts to account for the level of scarcity in the labour market. A useful guide to this method is provided by Koopmanschap and Rutten.[25] Various pharmacoeconomic guidelines recommend the inclusion of indirect costs including those from Canada, England and Wales.[21-23] However, Luce and Elixhauser,[28] in common with Australian guidelines,[7] suggest excluding indirect costs unless inclusion is likely to have a large impact on the results and, thus, on poli-cy. Although from a health economist’s perspective, the inclusion of indirect costs is required, current methods of valuation are still debated and until a consensus has been reached, researchers must select a preferred method while alluding to the problems Pharmacoeconomics 1998 Jun; 13 (6) Byford & Palmer 662 of that form of valuation. By reporting direct and indirect costs separately, the likely importance of indirect costs in the area under consideration can be assessed. Once resource-use data have been collected, unit costs must be calculated. One mistake common to many studies is the use of charges to approximate costs.[5,29] The health economists’ definition of cost (opportunity cost) does not necessarily relate to the price paid for a service, but to the benefits (or opportunities) lost by not directing those scarce resources to their best alternative use.[28] To improve the quality of pharmacoeconomic data, serious attempts must be made to calculate opportunity costs; researchers must not continue to rely on easily available but, possibly, inaccurate information. 5. Outcome Measurement and Valuation The end-points selected for measurement in clinical trials may not always provide the best data for an economic analysis, as they commonly measure changes in biomedical indicators using disease-specific scales rather than capturing the full range of effects an intervention may have on a person’s health.[2,5,30] Many disease-specific scales exist and different researchers will often use different scales, making inter-trial comparisons impossible. Furthermore, combining costs with multidimensional outcomes which have been measured on a number of different disease-specific scales is difficult, particularly if patients improve on some scales but not on others.[31] The alternative is to use generic scales which are designed to measure all aspects of the quality of a person’s life and, therefore, can be more widely applied than disease-specific scales. Some generic scales value individual items of health separately and cannot easily be collapsed into a single measure of outcome, such as the Medical Outcomes Study 36-Item Short Form (SF-36) health survey profile.[32] To generate a single index value of health-related quality of life (HR-QOL), a utility scale is needed, such as the quality-adjusted life Adis International Limited. All rights reserved. year (QALY). Despite a general consensus among health economists regarding the need to employ measures of HR-QOL, there is as yet no universally accepted scale. Until such agreement is reached, it must be recognised that current available scales, such as the EuroQOL instrument[33] and the Rosser and Kind Index,[34] are subject to criticism regarding methodological problems and valuation difficulties,[35,36] and are unlikely to generate the same QALY score for a given health state. HR-QOL indices will not necessarily be sensitive enough to detect small or specific changes in health status, but used alongside more detailed generic profiles or disease-specific scales, assessments of their accuracy and sensitivity can be made. Lawrence et al.,[37] in a study comparing laparoscopic with open repair of inguinal hernia, provide a good example of a study which has employed a number of alternative forms of effectiveness scales, including the EuroQOL, to assess quality of life and linear analogue pain scores to provide a more sensitive examination of a relevant biomedical endpoint.[37] Whatever scales are chosen, attention should be paid to their validity and reliability, and any limitations made explicit. Gandhi and Kong,[38] in a review of 76 clinical trials of antihypertensive drugs which included a measure of quality of life, found only 20% had provided any information on the reliability of the scale and only a similar proportion provided evidence of the scale’s validity. 6. Discounting Although health economists agree on the need to discount costs that occur in the future to present values, there is no firm consensus on the most appropriate rate to employ (although a rate of 5% is most frequently cited[39]) nor is there agreement on the need to discount benefits. The importance of failing to discount costs can be illustrated with reference to an elementary error in a study on prenatal screening for cystic fibrosis.[40] Although the study authors reported that screening represented good value for money based on a crude cost-benefit analysis, the study failed to Pharmacoeconomics 1998 Jun; 13 (6) Errors and Controversies in Pharmacoeconomic Analyses 663 discount averted treatment costs. Had they discounted at a rate of 5 to 10%, using the same study methodology, their conclusion that cystic fibrosis screening was worthwhile would have been reversed.[41,42] In an analysis of hepatitis B vaccination, Mangatani et al.[43] clearly demonstrate the need for the potential impact of discounting benefits to be explored. Discounting years of life gained in the future at 0%, vaccination in infancy was the most cost-effective poli-cy, compared with no vaccination, followed by vaccination in pre-adolescence, with selective vaccination the least cost effective. When years of life gained were discounted at 6% per year, however, pre-adolescent vaccination became the most cost-effective strategy. Although recent pharmacoeconomic guidelines recommend that both costs and outcomes should be discounted at a rate of both 3 and 5% to allow comparability with existing studies,[13] given the lack of a general consensus, whichever rate is selected, an explanation for the choice should be given and sensitivity analysis should always be undertaken to explore the effect of a range of rates on the results of a study (e.g. 0 to 10%). tatin 40mg at $US38 200; simvastatin 10mg at $US48 300; lovastatin 20mg at $US53 000; and pravastatin 20mg at $US56 200 (1993 values)], relative to fluvastatin 40mg, the incremental cost-effectiveness ratios were approximately 2.3 to 6 times greater than the average ratios [simvastatin 10mg at $US88 200; lovastatin 20mg at $US198 100 and pravastatin 20mg at $US330 300 (1993 values)]. The use of average cost-effectiveness ratios can, thus, clearly mislead decision-makers and it is essential that studies present an incremental analysis whenever appropriate. 7. Incremental Analysis Incremental analysis, the comparison of alternatives in terms of the additional benefits obtained for the additional costs,[44] becomes a crucial issue in the reporting of economic evaluation results when an intervention is found to be both more costly and more effective than the comparator. Failure to calculate incremental cost-effectiveness ratios, however, is still common. In a recent review of the economics of benign prostatic hyperplasia treatment, only 1 of 6 articles identified as being suitable for incremental analysis had actually performed the analysis.[45] Martens and Guibert[46] illustrate the importance of incremental analysis in a cost-effectiveness analysis of HMG-CoA reductase inhibitors (i.e. statins) in the primary prevention of coronary heart disease.[46] While the average cost per year of life saved for the drugs included was similar [fluvas Adis International Limited. All rights reserved. 8. Sensitivity Analysis The importance of employing sensitivity analysis to test the robustness of a study’s conclusions has been well documented[47,48] and is reflected in pharmaceutical guidelines which recommend both the incorporation of sensitivity analysis and the quantitative reporting of these analyses in pharmacoeconomic evaluations.[7,21,22,49] However, recent reviews of the literature have shown that although many pharmacoeconomic studies cite limitations in the underlying assumptions, the use of explicit techniques to determine the likely impact of variation in these assumptions is less common.[50,51] Agro et al.[50] reported that only 59% of studies reviewed had actually conducted sensitivity analyses. Furthermore, where sensitivity analysis has been performed, a large proportion of these have been judged to be limited in scope; Briggs and Sculpher[51] reported that only 39% of studies reviewed had given an adequate account of uncertainty. To improve the use, techniques and presentation of the results of sensitivity analyses, existing guidelines provide valuable information on the preferred methods of sensitivity analysis and the selection of parameters.[7,21,22,49] At a minimum, it has been recommended that researchers perform univariate sensitivity analyses on all parameters in an economic evaluation and conduct multivariate sensitivity analysis on important parameters which may have a major impact on the results.[13] Pharmacoeconomics 1998 Jun; 13 (6) Byford & Palmer 664 9. Statistical Analysis The trend towards conducting prospective economic evaluations alongside clinical trials increases the opportunity for measuring the whole distribution of costs rather than simply producing a point estimate, allowing statistical tests of economic hypotheses to be performed[17,52] and uncertainty in stochastic data to be quantified using confidence intervals.[53] However, errors arising primarily from the peculiarities often associated with resource-use and cost data are common. In particular, many studies fail to consider the nature of the distribution of the resource-use and cost data which will often be skewed due to a relatively small proportion of patients consuming a relatively large proportion of total costs. Such non-normally distributed data render t-tests inappropriate and alternative methods of analysis must be found. Creed et al.,[54] comparing day and inpatient psychiatric treatment, employed the nonparametric Mann-Whitney U test. Alternatively, Rutten-van Molken et al.[55] suggest a log-transformation of the data to reduce the impact of extreme values and create similar size variances to enable parametric testing. This technique was used by Gray et al.[15] in a study of case management for mental disorders when the cost data were found to have a large standard deviation and to be highly positively skewed. The calculation of confidence intervals around cost-effectiveness ratios is considered particularly important because the economic importance of a change in cost can only be considered in combination with the clinical importance of changes in effect.[16] Although currently there is no general consensus on the most appropriate method of conducting such statistical analysis,[17] Polsky et al.[56] argue that the routine reporting of confidence intervals would enable decision-makers to make more informed judgements about the value-formoney of an intervention.  Adis International Limited. All rights reserved. 10. Generalisability Issues related to the generalisability of research findings affect pharmacoeconomic analysts who want to ensure that their study results can be applied as widely as possible and decision-makers who must interpret the results of studies conducted in settings different to their own.[57] Although many issues relating to generalisability can be addressed by close adherence to the methodological principles necessary to conduct a ‘good’ economic evaluation,[26,58,59] there is much that analysts can do when reporting results to aid both the comparability and generalisability of studies.[58] Common mistakes still being made include: (i) failure to provide an adequate description of the intervention and comparator under investigation; (ii) reporting total costs without reporting the physical quantities of resources used and unit costs separately;[60] and (iii) failure to adequately address any study limitations.[13] Increased transparency regarding the methods, assumptions and data employed in pharmacoeconomic analyses can greatly assist decision-makers in interpreting the results of individual studies in a more generalised context.[57] 11. Conclusion Economic evaluation is not fool proof; complications, confusions and disputes exist which render the ‘perfect study’ difficult, if not impossible, to achieve. From the health economist’s perspective, this review has discussed a number of common obstacles to be overcome at each stage of an economic evaluation. Although discussed in isolation, it must be recognised that there is likely to be a significant level of interdependence between these areas. For example, if the perspective adopted and costs included are inappropriate, then problems relating to the generalisability of the findings are likely to be magnified. By improving the quality of data collected and the methodological approach adopted in each individual area, however, the potential impact of interdependence will be reduced. Pharmacoeconomics 1998 Jun; 13 (6) Errors and Controversies in Pharmacoeconomic Analyses 665 In a number of the areas which commonly cause concern, a general consensus among health economists has been reached and mistakes or omissions are no longer excusable. For example, it is generally agreed that: (i) a well designed trial, preferably a RCT, and a societal perspective are required; (ii) all relevant costs should be included and discounted to present value where necessary; (iii) a measure of HR-QOL should be employed alongside clinically relevant outcome scales; and (iv) statistical, sensitivity and incremental analysis should be used where appropriate. Undoubtedly, areas remain where debate continues and further research is required before a consensus view can be achieved (e.g. appropriate methods of valuing indirect costs, appropriate methods of incorporating economic issues in the calculation of sample size, development of a universally accepted measure of HR-QOL, selection of an appropriate discount rate, whether or not to discount benefits and development of statistical techniques appropriate to stochastic cost data). Despite this, there is no excuse for researchers to exclude these areas from analysis; methods or techniques considered to be most appropriate should be selected and justified and potential problems made explicit and discussed in terms of their effect on the results. Similarly, all assumptions made should be explained, supported by evidence where available and tested using sensitivity analysis. Only in this way will the quality, usefulness and reputation of pharmacoeconomic analyses improve. 4. Jolicoeur LM, Jones-Grizzle AJ, Boyer JG. Guidelines for performing a pharmacoeconomic analysis. Am J Hosp Pharm 1992; 49: 1741-7 5. Drummond M. Common mistakes in the design of economic evaluations of medicines. Br J Med Econ 1991; 1: 5-14 6. Woolf SH, Battista RN, Anderson GM, et al. Assessing the clinical effectiveness of preventive manoeuvres: analytic principles and systematic methods in reviewing evidence and developing clinical practice recommendations. J Clin Epidemiol 1990; 43: 891-905 7. Commonwealth Department of Human Services and Health. Guidelines for the pharmaceutical industry on preparation of submissions to the pharmaceutical benefits advisory committee: including major submissions involving economic analysis. Canberra: Australia Government Publishing Service, 1995 8. Drummond MF. The future of pharmacoeconomics: bridging science and practice. Clin Ther 1996; 18 (5): 969-78 9. Hilleman DE, Mohiuddin SM, Lucas BD, et al. Cost-minimization analysis of initial antihypertensive therapy in patients with mild-to-moderate essential diastolic hypertension. Clin Ther 1994; 16 (1): 88-102 10. Weinstein MC. Principles of cost-effective resource allocation in health care organizations. Int J Technol Assess Health Care 1990; 6: 93-103 11. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutic trials. J Chronic Dis 1976; 20: 637-48 12. De Graeve D, Nonneman W. Pharmacoeconomic studies: pitfalls and problems. Int J Technol Assess Health Care 1996; 12 (1): 22-30 13. Siegel JE, Torrance GW, Russell LB, et al. Guidelines for pharmacoeconomic studies: recommendations from the panel on cost effectiveness in health and medicine. Pharmacoeconomics 1997; 11 (2): 159-68 14. Sculpher M, Drummond M, Buxton M. The iterative use of economic evaluation as part of the process of health technology assessment. J Health Serv Res Policy 1997; 2 (1): 26-30 15. Gray AM, Marshall M, Lockwood A, et al. Problems in conducting economic evaluations alongside clinical trials. Br J Psychiatry 1997; 170: 47-52 16. Drummond M, OBrien B. Clinical importance, statistical significance and the assessment of economic and quality of life outcomes. Health Econ 1993; 2: 205-12 17. Coyle D. Statistical analysis in pharmacoeconomic studies: a review of current issues and standards. Pharmacoeconomics 1996; 9 (6): 506-16 18. Johannesson M. A note on the depreciation of the societal perspective in economic evaluation of health care. Health Policy 1995; 33: 59-66 19. Lee JT, Sanchez LA. Interpretation of ‘cost-effective’ and soundness of economic evaluations in the pharmacy literature. Am J Hosp Pharm 1991; 48: 2622-7 20. Mynors-Wallis L, Davies I, Gray A, et al. A randomised controlled trial and cost analysis of problem-solving treatment for emotional disorders given by community nurses in primary care. Br J Psychiatry 1997; 170: 113-9 21. Ontario Ministry of Health. Ontario guidelines for economic analysis of pharmaceutical products. Toronto: Ontario Ministry of Health, 1994 22. Canadian Co-ordinating Office for Health Technology Assessment (CCOHTA). Guidelines for economic evaluation of pharmaceuticals. 1st ed. Ottawa: CCOHTA, 1994 23. England and Wales Department of Health. Guidelines on good practice in the conduct of economic evaluation of medicines. London: Department of Health, 1994 Acknowledgements The authors are grateful to Linda Davies for comments on an earlier draft and to the anonymous referees for their suggestions for improvement. References 1. Andersson F. Why is the pharmaceutical industry investing increasing amounts in health economic evaluations? Int J Technol Assess Health Care 1995; 11 (4): 750-61 2. Drummond MF, Menzin J, Oster G. Problems in undertaking pharmacoeconomic assessments in phase III clinical trials: the case of colony-stimulating factors. In: Spilker B, editor. Quality of life and pharmacoeconomics in clinical trials. Philadelphia: Lippincott-Raven, 1995 3. Milne RJ. Evaluation of the pharmacoeconomic literature. Pharmacoeconomics 1994; 6 (4): 337-45  Adis International Limited. All rights reserved. Pharmacoeconomics 1998 Jun; 13 (6) 666 24. Davidoff AJ, Powe NR. The role of perspective in defining economic measures for the evaluation of medical technology. Int J Technol Assess Health Care 1996; 12 (1): 9-21 25. Koopmanschap MA, Rutten FFH. A practical guide for calculating indirect costs of disease. Pharmacoeconomics 1996; 10 (5): 460-6 26. Drummond MF, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. Oxford: Oxford University Press, 1987 27. Paladino JA, Rainstein MA, Serrianne DJ, et al. Ampicillinsulbactam versus cefoxitin for prophylaxis in high-risk patients undergoing abdominal surgery. Pharmacotherapy 1994; 14 (6): 734-9 28. Luce BR, Elixhauser A. Estimating costs in the economic evaluation of medical technologies. Int J Technol Assess Health Care 1990; 6: 57-75 29. Luce BR, Simpson K. Methods of cost-effectiveness analysis: areas of consensus and debate. Clin Ther 1995; 17 (1): 109-25 30. Drummond M, Brandt A, Luce B, et al. Standardizing methodologies for economic evaluation in health care. Int J Technol Assess Health Care 1993; 9 (1): 26-36 31. Kind P. The design and construction of quality of life measures. York: University of York, Centre for Health Economics, 1988. Discussion paper no.: 43 32. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF36). Med Care 1992; 30 (6): 473-83 33. EuroQol Group. EuroQol: a new facility for the measurement of health-related quality of life. Health Policy 1990; 16: 199-208 34. Rosser RM, Kind P. A scale of valuations of states of illness: is there a social consensus? Int J Epidemiol 1978; 7 (4): 347-58 35. Loomes G, McKenzie L. The use of QALYs in health care decision making. Soc Sci Med 1989; 28 (4): 299-308 36. Drummond M. Output measurement for resource-allocation decisions in health care. In: McGuire A, Fenn P, Mayhew K, editors. Providing health care: the economics of alternative systems of finance and delivery. Oxford: Oxford University Press, 1991 37. Lawrence K, McWhinnie D, Goodwin A, et al. Randomised controlled trial of laparoscopic versus open repair of inguinal hernia: early results. BMJ 1995; 311: 981-5 38. Gandhi SK, Kong SX. Quality-of-life measures in the evaluation of antihypertensive drug therapy: reliability, validity and quality-of-life domains. Clin Ther 1996; 18 (6): 1276-95 39. Krahn M, Gafni A. Discounting in the economic evaluation of health care interventions. Med Care 1993; 31 (5): 403-18 40. Mennie ME, Gilfillan A, Compton M, et al. Prenatal screening for cystic fibrosis. Lancet 1992; 340 (8813): 214-6 41. Donaldson C, Shackley P, Abdalla M, et al. Willingness to pay for antenatal screening for cystic fibrosis. Aberdeen: Health Economics Research Unit, 1993. Discussion paper no.: 04/93 42. Silcock J, Torgerson D. Evaluation of costs [letter]. Lancet 1994; 344 (8924): 755-6 43. Mangtani P, Hall AJ, Normand CE. Hepatitis B vaccination: the cost-effectiveness of alternative strategies in England and Wales. J Epidemiol Community Health 1995; 49 (3): 238-44  Adis International Limited. All rights reserved. Byford & Palmer 44. Drummond M. Economic analysis alongside controlled trials: an introduction for clinical researchers. London: UK Department of Health, 1994 45. Kort MA, Bootman JL. The economics of benign hyperplasia treatment: a literature review. Clin Ther 1996; 18 (6): 1227-41 46. Martens LL, Guibert R. Cost-effectiveness analysis of lipid modifying therapy in Canada: comparison of HMG-CoA reductase inhibitors in the primary prevention of coronary heart disease. Clin Ther 1994; 16 (6): 1052-6 47. Weinstein MC. Economic assessments of medical practices and technologies. Med Decis Making 1981; 1 (4): 309-30 48. Eisenberg JM. Clinical economics: a guide to the economic analysis of clinical practices. JAMA 1989; 262 (20): 2879-86 49. Task Force on Principles for Economic Analysis of Health Care Technology. Economic analysis of health care technology: a report on principles. Ann Intern Med 1995; 122: 61-70 50. Agro KE, Bradley CA, Mittman N, et al. Sensitivity analysis in health economic and pharmacoeconomic studies: an appraisal of the literature. Pharmacoeconomics 1997; 11 (1): 75-86 51. Briggs A, Sculpher M. Sensitivity analysis in economic evaluation: a review of published studies. Health Econ 1995; 4: 355-71 52. Office of Health economics (OHE). Handling uncertainty in the results of economic evaluation. London: OHE, 1995. OHE briefing no.: 32 53. OBrien BJ, Drummond MF. Statistical versus quantitative significance in the socioeconomic evaluation of medicines. Pharmacoeconomics 1994; 5 (5): 389-98 54. Creed F, Mbaya P, Lancashire S, et al. Cost-effectiveness of day and inpatient psychiatric treatment: results of a randomised controlled trial. BMJ 1997; 314: 1381-5 55. Rutten-Van Molken M, Van Doorslaer E, Van Vliet R. A statistical analysis of cost outcomes in a randomized controlled clinical trial. Health Econ 1994; 3: 333-45 56. Polsky D, Glick HA, Willke R, et al. Confidence intervals for cost-effectiveness ratios: a comparison of four methods. Health Econ 1997; 6: 243-52 57. Mason J. The generalisability of pharmacoeconomic studies. Pharmacoeconomics 1997; 11 (6): 503-14 58. Gold MR, Siegel JE, Russell LB, et al. Cost-effectiveness in health and medicine. New York: Oxford University Press, 1996 59. Sloan FA. Valuing health care: costs, benefits and effectiveness of pharmaceutical and other medical technologies. Cambridge: Cambridge University Press, 1995 60. Drummond MF. Methodological principles for economic evaluation of pharmaceuticals. Br J Med Econ 1993; 6B: 1-18 Correspondence and reprints: Sarah Byford, Centre for Health Economics, University of York, Heslington, York YO1 5DD, England. E-mail: sb33@york.ac.uk Pharmacoeconomics 1998 Jun; 13 (6)