Measuring and stimulating progress on implementing widely recommended food environment policies: the New Zealand case study

Abstract

Background

Monitoring the degree of implementation of widely recommended food environment policies by national governments is an important part of stimulating progress towards better population nutritional health.

Methods

The Healthy Food Environment Policy Index (Food-EPI) was applied for the second time in New Zealand in 2017 (initially applied in 2014) to measure progress on implementation of widely recommended food environment policies. A national panel of 71 independent (n = 48) and government (n = 23) public health experts rated the extent of implementation of 47 policy and infrastructure support good practice indicators by the Government against international best practice, using an extensive evidence document verified by government officials. Experts proposed and prioritised concrete actions needed to address the critical implementation gaps identified.

Results

Inter-rater reliability was good (Gwet’s AC2 > 0.8). Approximately half (47%) of the indicators were rated as having ‘low’ or ‘very little, if any’ implementation compared to international benchmarks, a decrease since 2014 (60%). A lower proportion of infrastructure support (29%) compared to policy (70%) indicators were rated as having ‘low’ or ‘very little, if any’ implementation. The experts recommended 53 actions, prioritising nine for immediate implementation; three of those prioritised actions were the same as in 2014. The vast majority of experts agreed that the Food-EPI is likely to contribute to beneficial policy change and increased their knowledge about food environments and policies.

Conclusion

The Food-EPI has the potential to increase accountability of governments to implement widely recommended food environment policies and reduce the burden of obesity and diet-related diseases.

Background

The prevalence of overweight and obesity is increasing worldwide [1], and has doubled for both children and adults in 73 countries since 1980 [2]. Excess body weight accounted for 4 million deaths and 120 million disability-adjusted life-years worldwide in 2015 [2]. It has been established that unhealthy food environments are a major driver of unhealthy population diets and obesity [3, 4].

Effective government policies and actions are essential to increase the healthiness of food environments and to reduce these high levels of obesity, non-communicable diseases (NCDs), and their related inequalities. It is critical that governments implement widely recommended preventive policies and actions to match the magnitude of the burden that unhealthy diets are creating [5]. Monitoring the degree of implementation of those widely recommended policies and actions is an important part of ensuring progress towards better population nutritional health [6].

The International Network for Food and Obesity/Non-communicable diseases Research, Monitoring and Action Support (INFORMAS) [7] developed a tool and process, The Healthy Food Environment Policy Index (Food-EPI) [8], to assess the extent of implementation of recommended food environment policies by national Governments compared to international best practice. The Food-EPI comprises a ‘policy’ component with seven domains on specific aspects of food environments and an ‘infrastructure support’ component with six domains to strengthen obesity and NCD prevention systems (Additional file 1). Good practice indicators contained in these domains encompass policies and infrastructure support necessary to improve the healthiness of food environments and to help prevent obesity and diet-related NCDs. The Food-EPI indicators are consistent with, and supportive of, the list of proposed policy options for Member States included in WHO’s Global Action Plan for the Prevention and Control of NCDs (2013–2020) [9], the WHO’s high level Commission report on ending childhood obesity [10] and the World Cancer Research Fund International NOURISHING Food Policy Framework for Healthy Diets [11, 12].

The Food-EPI tool and process have been through several phases of development, including a review of literature and policy documents, subsequent revision by a group of international experts from low-, middle- and high-income countries [8], and pilot testing in New Zealand in 2013 [13]. The refined tool was then used in the baseline assessment of New Zealand’s policies and infrastructure support in relation to international best practice in 2014 [14, 15] and in a range of other countries globally, such as Thailand (in 2015) [16], the United Kingdom (in 2016) [17], Australia (in 2017) [18] and others (not yet published).

This study applied the Food-EPI tool and process in New Zealand for the second time ahead of upcoming elections and compared progress on policy implementation since 2014, when the first Food-EPI was conducted. The New Zealand Expert Panel rated the extent of implementation of policies on food environments and infrastructure support systems by the New Zealand government between 2014 and 2017 compared to international best practice. They proposed and prioritised actions needed to address critical implementation gaps identified. In addition, they evaluated the value, importance and potential impact of the Food-EPI tool and process.

Methods

The study was approved by the University of Auckland Human Participants Ethics Committee (reference number 018605). A mixed methods design was used to obtain the ratings of the level of implementation of widely recommended good practice policies and infrastructure support and to identify and prioritise concrete actions to fill implementation gaps (Additional file 1).

Expert panel

In 2014, an expert panel was formed by invitations being sent to a wide range of public health experts (academics, researchers and practitioners, and representatives of non-governmental organisations, including medical associations, professional bodies and service providers). Where possible, these experts were invited to participate again in the Food-EPI 2017 or new, similar experts were invited. Unlike in 2014, in addition to independent public health experts, government experts (e.g. experts from different Ministries, Health Promotion Agency and District Health Boards) were also invited to participate in the Food-EPI 2017 ratings and workshops. In 2014, their role was restricted to verifying the evidence document and participating in the workshops as observers. The experts signed an informed consent form and declared their conflicts of interest (the latter for non-government experts only).

Evidence compilation and verification

A 100-page evidence document [19] on the current degree of implementation of all 47 good practice policy and infrastructure support indicators across 13 policy and infrastructure support domains was compiled from policy documents and budgets retrieved from websites, direct communication with organisations and through Official Information Act requests. The evidence was comprehensively documented and returned to government officials to verify its completeness and accuracy. Summaries of evidence of implementation, international best practice benchmarks and progress since 2014 were compiled for each indicator [19].

International best practice exemplars (benchmarks)

Benchmarks were extracted for each of the good practice indicators from the World Cancer Research Funding NOURISHING framework [20] and obtained from international food policy experts. Benchmark policies include the 10% soda and 8% junk food taxes recently implemented in Mexico, comprehensive restrictions on unhealthy food marketing to children in Chile, sodium targets in a range of food product categories specified by law in Argentina and South Africa, and the nutrient profiling system to prevent unhealthy food products carrying health claims in Australia and New Zealand.

Rating implementation progress

An online rating tool using RedCap was developed and experts completed the ratings individually before the action workshops. Experts were sent a paper version of the full evidence document and the evidence summaries [19] were presented to them online prior to them rating each of the good practice indicators.

A total of 47 indicators comprising 23 policy indicators and 24 infrastructure support indicators were rated against international best practice using Likert scales (1–5) (Additional file 2), with a rating of 1 indicating between 0 and 20% implementation compared to international best practice and a rating of 5 indicating between 80% and 100% implementation compared to best practice.

Action and prioritisation workshops

After the online ratings, four workshops were organised across the country (Auckland, Wellington, Christchurch and Dunedin) to evaluate the implementation gaps as identified from the ratings and to propose and prioritise concrete actions for implementation by the New Zealand Government.

Experts participating in the workshops were presented with the distribution of the rating scores for each good practice indicator. They discussed the need for any action in relation to the indicator and, if a need was considered, identified actions to improve food environments and population nutrition as well as to reduce obesity and diet-related NCDs in New Zealand.

After compiling the full list of proposed actions, in the workshops, the expert panel members were asked to individually prioritise the importance and achievability of the actions using an online Qualtrics tool. Importance took into account the relative need, impact, effects on equity, and any other positive and negative effects of the action. Achievability took into account the relative feasibility, acceptability, affordability and efficiency of the action. More details on those criteria can be found in Additional file 1. Participants were asked to consider ‘acceptability to government’ as pertaining to New Zealand governments in general, not the particular government of the day. Each proposed policy action was ranked from higher to lower importance and achievability. The same process was then applied to prioritise the proposed infrastructure support actions.

Evaluation questionnaire

Before leaving the workshops, experts were asked to fill out a questionnaire to evaluate the value, importance and potential impact of the Food-EPI tool and process.

Data analysis

The mean rating for each indicator was used to determine an overall percentage level of implementation. These ratings were then categorised into the following levels of implementation based on the cut-points: high, > 75%; medium, 51–75%; low, 26–50%; and very little, if any, ≤ 25%. A bar graph was created to compare the level of implementation of the 47 indicators between 2014 and 2017. The Gwet AC2 inter-rater reliability coefficient and its variance were determined using AgreeStat software (Agreestat 2013.1, Advanced Analytics, Gaithersburg, United States of America). For estimation of the variance, the sample of subjects to rate was set at 100% since all indicators of the Food-EPI were included for rating, while the sample of raters was set at 50% (as per the response rate of experts invited), and the finite population correction was applied.

Actions with the highest rank received the maximum score while actions ranked at the bottom received a score of 1. For each action, the scores were summed per workshop and expressed as a percentage out of 100 (normalisation because the number of experts in each workshop was different) and for each action the average score across workshops was calculated for both importance and achievability. Graphs were created to plot importance of actions against achievability. Actions in the top third for importance where selected as top priorities for implementation by the New Zealand Government.

Results

Seventy-one New Zealand-based independent (n = 48) and government (n = 23) public health experts scored the degree of implementation of food environment policies and infrastructure support in New Zealand against international best practice. Twenty-eight of those experts also participated in the Food-EPI 2014. Approximately 77.5% of experts were New Zealand European, 9.9% European, 8.5% Māori, 2.8% Pacific and 4.2% Asian. Government experts who participated in the ratings were mainly local experts working in District Health Boards or public health units. In total, 45 experts participated in the action and prioritisation workshops, and 25 experts returned an evaluation questionnaire.

Ratings and progress

The inter-rater reliability (Gwet’s AC2 > 0.8) for the 2017 Food-EPI assessment indicated good agreement between experts, and there was no difference between independent and government experts. There was no difference in level of implementation ratings for any of the Food-EPI indicators between independent and government experts (data not shown). The scorecard in Fig. 1 therefore presents the results including all 71 expert panel members.

Fig. 1

Level of implementation of food environment policies and infrastructure support by the New Zealand Government in 2017 compared to international best practice (* 2014 ratings)

Approximately half (47%) of all the good practice indicators were rated as having ‘low’ or ‘very little, if any’ implementation compared with international benchmarks, a decrease since 2014 when 60% were rated as having ‘low’ or ‘very little, if any’ implementation (Fig. 1). This was not spread evenly across infrastructure support and policy indicators, with one-third (29% in 2017 and 48% in 2014) of the infrastructure indicators and two-thirds (70% in 2017 and 74% in 2014) of the policy indicators rated as having ‘low’ or ‘very little, if any’ implementation in New Zealand.

Major implementation gaps (‘very little, if any’ or ‘low’ implementation) were identified for food environment policies, especially for healthy food policies in schools, fiscal policies to support healthy food choices, implementing restrictions on unhealthy food marketing to children, supporting communities to limit the density of unhealthy food outlets in their communities (for example, around schools), supporting the food retail and service industry to reduce unhealthy food practices, and ensuring that trade and investment agreements do not negatively affect population nutrition and health (Fig. 1).

New Zealand rated well against international best practice for several infrastructure support indicators. These included having policies and procedures in place for ensuring transparency in the development of food policies, the public having access to nutrition information and key documents, and regular monitoring of body mass index, the prevalence of NCD risk factors and occurrence rates for the main diet-related NCDs and monitoring progress towards reducing health-related inequalities.

New Zealand was rated at the level of best practice for some policies such as the provision of ingredient lists and nutrient declarations on packaged foods and regulating health claims on packaged foods.

For 11 indicators there was progress noted compared to 2014 (Fig. 1). Although not rated at the level of international best practice, experts recognised progress since 2014 for implementation of the Health Star Ratings on food packages [21], initiating systems-based approaches with communities (Healthy Families [22], Healthy Auckland Together [23] and other regional platforms), developing and implementing the National Healthy Food and Drink Policy [24] in the public sector (especially in District Health Boards), and improving platforms for interaction between Government and other sectors and across Government. Experts recognised some progress for restricting unhealthy food marketing to children (related to the Government stimulating a review of the industry self-regulatory codes [25, 26]) and the development and implementation of a childhood obesity plan [27], but the extent of implementation for those indicators compared to international best practice was still rated as ‘low’.

Actions and priorities

Across the four workshops, a total of 53 common actions were proposed for 46 of the 47 good practice indicators (Additional file 2). Of the 53 actions proposed, eight infrastructure support actions and eight policy actions were ranked by the expert panel in the top third for importance (Figs. 2 and 3). Since two priority policy actions and two priority infrastructure support actions were related to the same Food-EPI indicator, the more achievable options were retained as top recommendations (i.e. voluntary instead of mandatory food composition targets and improving the childhood obesity plan rather than creating a new national nutrition plan). The top seven food policy and top seven infrastructure support actions were further condensed into nine key recommendations for the New Zealand Government (Fig. 4). Three of those nine priorities were the same as in 2014 (sugary drinks tax, healthy school food policies, restriction of junk food marketing to children). Three recommendations were new (strengthening child obesity plan, implement the new Eating and Activity Guidelines, and organise a children’s nutrition survey) and three were based on 2014 recommendations but updated (setting targets for childhood obesity and intake of nutrients of concern, increase funding, strengthen Health Star Rating System).

Fig. 2

Prioritised recommended actions (top priorities in green) for the New Zealand Government: policy actions targeting food environments (labels explained in Additional file 2)

Fig. 3

Prioritised recommended actions (top priorities in green) for the New Zealand Government: infrastructure support actions (labels explained in Additional file 2)

Fig. 4

Top priorities for improving the healthiness of New Zealand food environments as identified by the expert panel

Evaluation

Almost all experts agreed that participating in the Food-EPI process increased their knowledge about food environments and policies, that the Food-EPI is likely to contribute to beneficial policy change, and that it is important to repeat the Food-EPI every 3 years to monitor progress of implementing recommended food environment policies compared to international best practice (Fig. 5).

Fig. 5

Expert’s assessment of value, importance and potential impact of the Food-EPI

Discussion

A wide range of experts participated in the New Zealand Food-EPI 2017 and changes in the process compared to 2014 allowed government experts to be more closely engaged than in 2014. Government experts partaking in the rating and prioritisation processes were mainly local experts. National government experts (e.g. Ministry of Health (MOH)) kept their original engagement (e.g. verifying evidence document, attending workshops as observers) as a self-assessment was considered too sensitive, especially in election year.

Overall, the Food-EPI scorecard improved compared to 2014 for about one in five indicators. The scorecard shows some key areas of strength where the New Zealand Government is doing well (e.g. nutrition information panels, regulations on health claims, monitoring systems for NCDs and risk factors, and high levels of transparency and access to government information). In addition, experts recognised progress since 2014 in some areas (e.g. implementation of Health Star Ratings, systems-based approaches with communities, development of the Healthy Food and Drink Policy for the public sector and improving interactions with stakeholders). However, compared to international best practice, large implementation gaps remain, especially for the policy component of the Food-EPI.

In terms of infrastructure support, the experts noted a large gap in leadership to reduce obesity and improve public health nutrition in New Zealand. Although the Government launched a plan to tackle childhood obesity in October 2015 [27], which was recognised as an area of progress since 2014, there is a lack of actions to improve the healthiness of children’s food environments in the plan. The level of funding to address the burden of diet-related diseases in New Zealand was also rated as ‘low’. Another recognised gap in leadership is the absence of targets to reduce childhood obesity rates and inequalities and achieve WHO recommendations for average population sugar, salt and saturated fat intakes.

The experts made nine top-priority recommendations, of which one-third were the same as in 2014, one-third was new, and one-third were updated from those in 2014. A wide range of government agencies will need to be involved in implementing these recommendations. The main responsible agents for the implementation of the nine top priorities would be the Ministry of Health, the Ministry for Primary Industries, Food Standards Australia New Zealand, the Health Promotion Agency, the Ministry of Education and the Treasury. The proportion of all 53 recommendations to be implemented under agency of the following government departments would be 72% (Ministry of Health), 42% (Minister of Health), 36% (Ministry for Primary Industries), 21% (Food Standards Australia New Zealand), 8% (Ministry of Education), 8% (Treasury), 4% (Health Promotion Agency), 4% (District Health Boards), 4% (Ministry of Trade and Foreign Affairs), 4% (State Services Commission), 4% (Ministry of Business Innovation Employment), 2% (Minister of Education), and 2% (Minister of Finance). Approximately 15% of recommendations would need to involve action by Cabinet.

In the recent Australian Food-EPI, experts recognised the same areas of strength as in New Zealand, but also evaluated Australia as being at the level of international best practice for leaving Goods and Services Tax off fruit and vegetables and implementing evidence-based food-based dietary guidelines. Another area where Australia is doing better than New Zealand is school food policies, with several of the states having implemented mandatory nutrition standards in schools. The implementation of the Health Star Ratings was rated at medium level of implementation in Australia, similar as in New Zealand [28]. The Thai Food-EPI showed that none of the policy indicators were rated at the level of international best practice and that ratings by government experts were generally higher than those by independent experts [16].

The strengths of the study include the wide range of independent and government experts involved in the process, the use of comprehensive evidence on the extent of implementation of food policies to support the ratings (validated by government officials), and the construction of a scorecard to follow progress over time and in comparison to other countries. Challenges include the comparison to international best practice when some of those exemplars are still perceived as too far below the ideal and the burden on participants.

The Food-EPI provides a useful set of indicators focusing on where government actions are needed most and the process involves a wide range of stakeholders. Experts evaluated the tool and process as valuable and with potential to stimulate government action in New Zealand. The Food-EPI is currently being implemented by over 10 countries globally, including large countries like the United Kingdom [17] and Australia [28], and wider uptake will allow benchmarking of food environment policy implementation globally. This will be useful for the Decade of Action on Nutrition [29], which stimulates governments to make SMART (Specific, Measurable, Achievable, Relevant, Time-Bound) commitments on nutrition. It is anticipated that benchmarking the extent of implementation of government policies will increase accountability of governments for their actions on food environments [6].

Conclusion

In conclusion, there are some areas where New Zealand is at the level of best practice and there are some areas where there is progress compared to 2014. However, about half of the indicators on the Food-EPI scorecard show major implementation gaps still to be addressed to improve the healthiness of food environments in New Zealand. The Food-EPI has the potential to increase accountability of governments to implement widely recommended food environment policies and reduce the burden of obesity and diet-related diseases.