Methodologic challenges to evaluating the health economic impact of nutritional interventions

Enhancing the diet by increasing or decreasing consumption of selected foods or nutrients is a well-accepted method of improving population health in certain settings. However, the economic impact of most dietary interventions is largely unknown. Studies in the past decade have started to estimate the effects on healthcare costs, often through economic models, but the evaluation of nutritional interventions is challenging and faces uncertainties unique to the domain, compared with more straightforward analyses of medical or surgical interventions. This paper discusses three main areas of uncertainty: measurement of consumption, assessing effectiveness, and considerations about the economic inputs to a model. Insight into these sources of uncertainty can help readers assess the quality of existing studies, and provide guidance for their future research.


Introduction
Enriching foods with vitamins or minerals that are either lost in processing or that could fill a public health gap, known as fortification, has been shown to be an effective, and perhaps costeffective, strategy to reduce the burden of certain nutrition-related diseases in the general population.Success stories include folic acid fortification of grains as a means of reducing neural tube defects, iodine fortification of salt as a method for eliminating iodine-deficiency disorders, and the addition of vitamin D to milk as a strategy for preventing rickets.Similarly, dietary modifications to enhance or reduce intake of specific foods or nutrients may be a useful tool against certain chronic diseases.The World Health Organization has recognized public health primary prevention efforts as essential to reducing the burden of chronic disease worldwide (World Health Organization 2003).Recent examples include discouraging intake of high-sodium foods and of trans-unsaturated fatty acids.
With few exceptions, health economic evaluations of these and other nutrition interventions have only begun to appear in the scientific literature in the past decade, with several reviews making a strong case for this type of analysis, but also recognizing inconsistency in methods used (Lenoir-Wijnkoop, Dapoigny et al. 2011, Nuijten and Lenoir-Wijnkoop 2011, Gyles, Lenoir-Wijnkoop et al. 2012, Koponen, Sandell et al. 2012, Lenoir-Wijnkoop, Jones et al. 2013).Cost-effectiveness analysis, which is frequently used in healthcare analyses, measures the benefit in terms of nonmonetary metrics, sometimes referred to as "natural" units, such as disability-adjusted life years (DALYs), quality-adjusted life years (QALYs), or events avoided (e.g., cardiovascular events avoided, hospitalizations avoided).Cost-effectiveness methods are well-documented in the pharmaceutical and medical device industries (Ramsey, Willke et al. 2005, Berger, Mamdani et al. 2009, Johnson, Crown et al. 2009, Briggs, Weinstein et al. 2012, Ramaekers, Joore et al. 2013, Ramsey, Willke et al. 2015).Although there are some methodological differences across fields and variations across countries based on different standards and healthcare coverage, costeffectiveness analyses are fairly straightforward to apply in evaluating prescription medication or medical procedures.Estimating costs, exposure and outcomes is not difficult in a randomized controlled trial, for example, as researchers know how much drug has been consumed, how many patients benefit from treatment, how many experience adverse events and the costs of treatments received for those events.Considering population-wide nutrient intake or dietary improvement raises more uncertainty and challenges.That said, there have been several health economic evaluations of nutritional interventions, including but not limited to folic acid fortification (Grosse, Waitzman et al. 2005, Bentley, Weinstein et al. 2009), reductions in sodium intake (Martikainen, Soini et al. 2011, Collins, Mason et al. 2014, Meier, Senftleben et al. 2015, Nghiem, Blakely et al. 2015, Wilson, Nghiem et al. 2016), reductions in fat intake (Dalziel andSegal 2007, Martikainen, Soini et al. 2011), and increased intake of fiber (Schmier, Miller et al. 2014, Abdullah, Marinangeli et al. 2017), of dairy products (Scrafford, Schmier et al. 2017), and of healthy dietary patterns (Scrafford, Barraj et al. 2019).
This paper introduces readers to some of the sources of uncertainty in this field, providing guidance for study design and the evaluation and interpretation of published research.Three major topics are discussed: measuring food and nutrient consumption, applying epidemiologic principles in attributing benefits, and assessing economic impact.Each of these areas includes sources of uncertainty unique to population-wide nutritional interventions, as summarized in Table 1.The goal of this article is to improve the quality and value of nutritional economics analyses in the general population, particularly in high-income regions; parenteral or medical nutrition and protein-energy malnutrition are beyond the scope of this discussion.

Consumption
It is almost always necessary to make assumptions about population-level dietary consumption for nutritional economic analyses, unlike clinical trials for pharmaceutical products, in which actual intake is known (see Table 1).Relevant consumption information can include amount, preparation methods, and concomitant consumption, as each can affect the nutrient content and bioavailability of nutrients.Large-scale prospective cohort studies, such as the Women's Health Initiative, the Nurses' Health Study, and the Health Professionals Follow-Up Study, are uncommon and typically lack broad generalizability; smaller studies on institutionalized populations, such as nursing home residents or prisoners, can provide substantial detail (Gesch, Hammond et al. 2002, Simmons, Zhuo et al. 2010, Simmons, Keeler et al. 2015), but findings also cannot necessarily be projected to general populations.(Whereas drug data are applied to other patients with the same disease, nutrition/food data cannot as readily be applied to a general population.)Other sources must be consulted to estimate food intake, and readers must consider the implications of the accuracy of their sources and assessment methods, and completeness of these data.In controlled trials, whether of drug or nutritional interventions, the amount and frequency of consumption are generally known with precision, as are the characteristics of consumers and non-consumers.In assessments of public health nutrition interventions, by contrast, even though the effect of a given intervention on a specific individual may be known, assessment of the effect at the population level relies on assumptions about consumption patterns and population characteristics.In other words, drugs are limited in who shares their benefits and who has access to them (often only by prescription) so there is little 'wasting' of the resource and it less often goes to people who will not benefit.When we measure the effect of a drug, we know that only patients who have a given condition will benefit.Public health nutrition interventions cannot be so well targeted.When we send a message that coffee is good (or bad), we do not know the genetic/medical background of the recipients of that message.We know exactly who has access to a drug and who is using that intervention but we know much less about the individuals who are changing their dietary habits.In this section, we articulate advantages and disadvantages of existing data sources and assumptions for estimating intake patterns.

Data sources for estimating food and nutrient intake
The National Health and Nutrition Examination Survey (NHANES) is the most comprehensive and broadly generalizable resource for cross-sectional food intake information in the general population of the United States.It provides detailed intake information that can be analyzed by age, gender, and other sociodemographic characteristics, making it a powerful tool for understanding consumption patterns.However, the NHANES survey and interview methods have several notable limitations.First, the data are self-reported and lack validation based on comparison with other assessment methods.Although self-reported food intake is not necessarily inaccurate (Freedman, Commins et al. 2015), memory of past diet is susceptible to error and can be biased by current health status and other influences.Another problem, at least for food frequency data, is the lack of information on portion size (Joachim 1998), a key characteristic of food and nutrient intake.Portion size, which has been scrutinized for decades as computer-and photograph-aided tools have been developed and tested, can be challenging to assess accurately and uniformly, as researchers try to convert or match data from questionnaires and similar surveys to clinical studies or meta-analyses that provide intake information by weight or volume.Another limitation that reduces generalizability is that NHANES excludes the approximately 3% of Americans who are institutionalized.
There are other limitations to the use food questionnaires and diaries in general for assessing dietary intake.For instance, they are susceptible to bias due to selective participation or completeness, they may be unable to distinguish modest differences in intake and subtle associations with health outcomes, they do not differentiate among diverse food items within a broad category, and they often do not capture important details about food preparation and dietary behaviors (Kristal, Peters et al. 2005, Kristal andPotter 2006).
There may be a role for retail data in estimating population-wide food consumption patterns.However, individual food items may not be consistently identifiable, may not be available in all markets or databases, and may vary widely in nutrition and energy content.Retail databases also have other limitations, in that purchase does not indicate consumption, nor do they provide guidance on who is consuming foods within each household unit.Typically, these databases also exclude certain retail channels, and their costs can be prohibitive.The Economic Research Service publishes a table with data sources and their advantages and disadvantages (Economic Research Service 2017).

Assumptions about dietary changes
Changing food consumption is rarely a zero-sum game.While a choice between a conventional and a fortified or alternative product could theoretically yield almost identical nutrient and energy content, typically there are trade-offs to consider.Understanding the impact of these changes requires longitudinal and detailed follow-up for health and economic outcomes.For example, if a person considers switching to a low-calorie beverage as permission to eat a caloriedense food, then the beneficial impact of the initial change could be overstated and it would be incorrect to assume that there was no subsequent loss in health or increase in healthcare resource needs; studies that use this approach are appropriately cautious in their pronouncements (Cawley, Meyerhoefer et al. 2017).Unless there are data to examine such trade-offs, reports should be transparent if they assume that all else remains unchanged so that readers can consider whether the assumption is valid, or if further analyses are needed.

Applying Epidemiologic Principles in Attributing Benefits
Large, randomized controlled trials (RCTs) are designed to optimize the detection of differences in treatment effects between the intervention and comparator groups, as randomization is intended to balance the influence of other factors, particularly selection bias, that may confound the effect of the intervention on the health outcome of interest.RCTs are required for evaluating pharmaceutical or device interventions for safety and efficacy, but have also been conducted in nutrition research, commonly for nutrients, but even for foods (Ncube, Greiner et al. 2001, Persson, Ahmed et al. 2001, Monarrez-Espino, Lopez-Alarcon et al. 2011).In contrast, observational studies have many more methodological challenges and potential sources of bias with which to contend.Observational studies are already common in public health nutrition but in the pharmaceutical space there is considerable discussion about "real-world evidence," that is, observational data gathered outside of clinical trials and how such information can contribute to understanding post-approval product use.The population of interest remains limited, as access to the intervention is limited, particularly for prescription drugs.Community-based or real-world analyses face more challenges.Systematic and often incompletely assessed differences in the distribution of relevant characteristics in the general population complicate understanding and assigning benefits of potential nutritional interventions.We find attribution of benefits, the timing for assigning of benefits and costs, and the choice of endpoints areas of potential uncertainty.

Epidemiology
Changes in behavioral or environmental risk factors for one chronic disease often affect the risk of other health outcomes.This pluripotency of certain risk factors can make it challenging to identify the population attributable fraction of all relevant health endpoints, that is, the proportional amount that a particular risk factor contributes to the presence or reduction of causally related diseases in the population.Consider weight loss, for which there is evidence for health benefits across multiple conditions such as heart disease, stroke, diabetes, and certain types of cancer.A high body mass index is known to be associated with greater healthcare costs (DiBonaventura, Lay et al. 2015), but this association is likely due to higher rates of comorbid disease rather than overweight or obesity itself.Looking at each condition that may be affected by a dietary intervention individually and summing benefits across all conditions would likely overestimate economic benefits; conversely, focusing on healthcare costs for only one disease at a time and ignoring synergistic effects would underestimate such benefits.Therefore, studies that present potential healthcare benefits of interventions should not sum across all relevant conditions without adjusting outcomes for the overlap.In clinical trials, patients are randomized to control for the presence of comorbid conditions, but analyses of the general population present more challenges in understanding the prevalence of existing comorbidities and knowing whether to attribute a change to an intervention or to other underlying differences between comparison groups (see Table 1).In some cases it may be possible to reduce overlap; for example, a study could identify the proportion of cardiovascular disease (CVD) cost that is attributable to diabetes and count it only once, rather than attributing the cost to CVD and diabetes or evaluating the burden associated with malnutrition across multiple conditions (Goates, Du et al. 2016).However, clear attribution to each disease is not always possible.
Additionally, assuming that benefits are attributable entirely to nutritional interventions likely overestimates their impact.Patients seeking to improve cardiovascular health, for example, may undertake a multi-pronged approach to lifestyle changes, consisting of not only changing their diets, but also quitting smoking, reducing alcohol intake, and/or exercising regularly.Data on concomitant lifestyle changes often are not available in food consumption databases, but they must be considered in interpreting findings.Attributing benefits to dietary changes alone ignores the potential benefits of other contemporaneous changes and may overestimate the benefit of the diet.
In nutritional epidemiology, estimating health impacts of individual foods, nutrients, and other dietary compounds is challenging.Foods and their components are not consumed in isolation, and observational studies of a community-wide public health campaign may be unable to disentangle independent associations of correlated exposures with specific health outcomes.This difficulty can also result in overestimation of the economic impact of interventions that affect a single food item, nutrient, or non-nutritive dietary component.
Finally, another challenge is related to how to apportion benefit in the presence of certain susceptible sub-groups for which risks or benefits may differ from those in the general population.For example, the United States Department of Agriculture (USDA) has eliminated its previous recommendation to limit dietary cholesterol intake (Kanter, Kris-Etherton et al. 2012).However, for certain sub-populations, such as diabetics or those with certain genetic risk factors, limitation of dietary cholesterol may still be beneficial.
Studies are underway to begin to understand the interaction between genetic factors and the effectiveness of dietary interventions (Portal, Markoski et al. 2016, Rizzi, Conti et al. 2016); knowing more about the genetic profile of the population and how certain genetic variants interact with foods and their components could increase the accuracy of projections about the effectiveness of dietary interventions.Data on genetic or other factors that may affect the apportionment of benefit or risk, so far, are rarely available at a sufficient level of detail and degree of scientific certainty to inform population-level nutrition economic models.

Timing and attribution
Beneficial health effects associated with changes in diet and nutrition may take time to accrue.For many health conditions, available data are cross-sectional (i.e., providing information on the prevalence at a given moment in time), but not longitudinal (i.e., providing information on changes over time), thereby precluding analyses of long-term changes in frequency or severity.Thus, modeling of long-term benefits and costs can be especially challenging.If food costs are included in a model, they may need to be assigned for multiple years until a benefit is seen.The time until a benefit is realized may depend on various personal and clinical characteristics, which also may vary over time.While information to incorporate the lag time until benefits appear may not be available, at a minimum reports should acknowledge that changes are not instantaneous.

Intermediate versus resource use endpoints
The choice of endpoint is essential to evaluating the economic impact of nutritional interventions.For example, programs that provide nutrition education may increase participants' knowledge and behavior in the short term, and changes in food and nutrient intake might lower cholesterol levels in the medium term.These intermediate endpoints, though, cannot be quantified economically; that is, lower cholesterol on its own is not associated with cost savings.That is, just having a drop in cholesterol does not mean that any fewer healthcare resources are used.It is the linking to cardiovascular events (that have costs) that is important for an economic model.A difference must be shown in an endpoint that can be linked to resource use, such as fewer admissions for diabetic ketoacidosis or a reduction in hospitalizations for myocardial infarction, to which a cost can be assigned.Short-term metrics, such as the cost per additional fruit served to middle school students in an obesity-fighting nutritional intervention (Ladapo, Bogart et al. 2016), cannot be compared with longer-term endpoints measured in other studies, nor do they articulate the overall health-oriented goal of the program.

Economic model inputs
Despite extensive recommendations in the health economic literature about assumptions that are used as input parameters to models and the structure and design of economic models (Briggs, Weinstein et al. 2012, Husereau, Drummond et al. 2013), there are unique questions and issues relevant to applying healthcare cost findings to the general population that have not been addressed.Health economic analyses should be specific about the cost perspective that they take, that is, whether they include only costs borne by insurance (often described as the thirdparty payer cost); only selected medical costs, such as outpatient or hospital costs; or societal costs, which can include non-medical and indirect costs associated with changes in employment, government transfers, and caregiver assistance.Much of the information on healthcare costs in the United States is presented in terms of direct medical costs, yet multiple other sources of costs are relevant, especially for nutritional interventions.Analyses in Europe tend to take a broader societal perspective, as is more typical of health economic analyses (Lenoir-Wijnkoop, Dapoigny et al. 2011).Each of the types of costs below (medical, food, program, and societal) should ideally be incorporated into a comprehensive analysis of cost-effectiveness of a nutritional intervention, yet each is associated with challenges that might limit the inclusion of accurate information into economic models.

Medical costs
While there are estimates available for the medical costs associated with chronic diseases, the application of these costs to the general population must be cautious.Studies occasionally, but not routinely, present direct medical costs by age, sex, or geography.Having these subgroupspecific costs available to apply proportionately would be ideal, since consumption patterns, prevalence of disease, and the impact of a nutritional intervention differ across populations and therefore they should be accounted for in evaluating population-level impacts.Disease costs are not binary and should take into account whether the condition(s) of interest will be delayed, less severe, or averted entirely.For some conditions, costs are available by stage, severity, or control level; however, for many other conditions, costs are not available at such a detailed level.
Estimating medical care resource use and costs is usually straightforward for interventional or observational studies, as they are often tracked as part of these studies, while estimating costs at the population level may be more difficult.

Cost of food
Existing models that examine changes in food intake tend to mention but not estimate the cost of the food itself.Food and supplement costs would likely be borne only by the consumer (i.e., patient), and thus one could argue that an evaluation of direct medical costs to society could reasonably exclude these out-of-pocket costs.Few studies have considered how to improve diet without changing costs to consumers (Scarborough, Kaur et al. 2016, Maillot, Vieux et al. 2017, Reidlinger, Sanders et al. 2017), yet this may be a relevant issue, particularly to people with low incomes or using food assistance programs.
Evaluating the societal benefit and cost of a nutritional intervention might be more difficult if manufacturers promote products heavily.Short-term promotional pricing could shift consumption and may affect both targeted and non-targeted consumer populations.For example, individuals who already have low CVD risk may switch to "heart-healthy" foods because they are heavily promoted, even though these individuals may not improve their CVD health from this dietary change.Such promotions might affect dividends and income to manufacturersanother indirect, non-medical societal economic impact.While studies in the health economic literature rarely consider the potential change in food costs or impacts on this macro level, such costs should be considered whenever possible.

Program costs
Public health nutrition interventions have costs even if they are of little interest to commercial insurers, who may ultimately benefit financially from changes in consumption.To some extent, the public health infrastructure is a fixed cost.However, it is a limited resource and could be spent in other activities.Mass-media campaigns have been shown to be capable of modifying health behaviors (Wakefield, Loken et al. 2010) and dietary habits (Afshin, Abioye et al. 2018).
Other than studies evaluating the cost-effectiveness of folic acid fortification (Romano, Waitzman et al. 1995, Tice, Ross et al. 2001, Grosse, Waitzman et al. 2005 (Xu, Alexander et al. 2015).Similarly, a series of studies have explored cost-effectiveness of government-sponsored sunscreen promotion programs in Australia (Gordon, Scuffham et al. 2009, Hirst, Gordon et al. 2012, Doran, Ling et al. 2016).Similar approaches, specifically including the costs of the program in the overall calculation of costs and effectiveness, could be applied to nutritional public health messaging and programs and serve as examples for future nutritional intervention analyses.

Societal costs
Including indirect costs such as lost productivity, which has been calculated for many disease states and can often be incorporated into health economic analyses, opens the discussion to other types of societal costs.For example, more municipalities are levying taxes on certain foods, beverages, and consumer products in an effort to decrease consumption.If the revenue from the taxes is directed to programs that cover expenses related to the conditions of interest, some amount of continued consumption might be beneficial for society, if not for individual consumers.Taxes that successfully drive down consumption, as some sugar-sweetened beverage taxes appear to do (Duffey, Gordon-Larsen et al. 2010, Merritt, Taylor et al. 2010, Colchero, Molina et al. 2017, Lal, Mantilla-Herrera et al. 2017), would generate less revenue in subsequent years if consumption continued to decrease.Substitutions could have a negative health and economic impact, such as the prediction of increased consumption of selected alcohol drinks subsequent to implementing a tax on sugar-sweetened beverages (Quirmbach, Cornelsen et al. 2018).Cost-of-illness studies in the US often include lost productivity, but other societal costs such as paid and unpaid caregiving, taxes, and government transfers are more complex to estimate and, therefore, rarely included.A probable reason for not including these costs is the lack of availability rather than an explicit decision to exclude them.For instance, Fayet-Moore and colleagues made several assumptions and had to use data that were collected more than a decade earlier to populate their model of productivity cost savings associated with cardiovascular disease in Australia (Fayet-Moore, George et al. 2018).

Conclusions
Due to the sources of uncertainty discussed here, analyses of the cost-effectiveness of large-scale dietary changes are in many ways more complex than traditional pharmaceutical intervention cost analyses.This complexity is perhaps even more of a concern for nutrition studies that are focused on the general population, rather than populations that suffer from malnutrition or require nutrition support therapy, where many of the existing best practices for health economic analyses can be easily transferred.Despite these challenges, a complete assessment of the impact of nutritional interventions on public health should consider potential health economic outcomes.
This summary is broad in scope and is intended to raise questions and heighten awareness rather than to provide universally relevant answers.We have highlighted areas that are likely to introduce substantial uncertainty in models, specifically, consumption, disease burden, and the types of costs included.Uncertainty in any one of these model inputs may be explored with sensitivity analysis, but the full range of uncertainty, given all of the inputs and assumptions, is often substantial.With careful planning and analysis, health economic evaluations can provide meaningful contributions to the accumulation of evidence supporting nutritional interventions.

Table 1 .
Comparison of Sources of Uncertainty in Nutritional vs. Pharmaceutical Interventions (Brown, Kotz et al. 2014al.2009), there are few published evaluations of the public health costs of dietary interventions in the United States.Informative examples can be gleaned from non-dietary public health interventions designed for general populations.For example, there are several analyses of the costs and cost-effectiveness of smoking cessation programs, including the UK's "Stoptober" program(Brown, Kotz et al. 2014) and the Centers for Disease Control and Prevention's Tips from Former Smokers campaign