Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Improving Survey Response Rates from Parents in School-Based Research Using a Multi-Level Approach

  • Elizabeth J. Schilpzand ,

    elizabeth.schilpzand@mcri.edu.au

    Affiliation Murdoch Childrens Research Institute, Parkville, VIC Australia

  • Emma Sciberras,

    Affiliations Murdoch Childrens Research Institute, Parkville, VIC Australia, The Royal Children’s Hospital, Parkville, VIC Australia, The University of Melbourne, Parkville, VIC Australia

  • Daryl Efron,

    Affiliations Murdoch Childrens Research Institute, Parkville, VIC Australia, The Royal Children’s Hospital, Parkville, VIC Australia, The University of Melbourne, Parkville, VIC Australia

  • Vicki Anderson,

    Affiliations Murdoch Childrens Research Institute, Parkville, VIC Australia, The Royal Children’s Hospital, Parkville, VIC Australia, The University of Melbourne, Parkville, VIC Australia

  • Jan M. Nicholson

    Affiliations Murdoch Childrens Research Institute, Parkville, VIC Australia, Judith Lumley Centre, La Trobe University, Melbourne, VIC Australia, The Parenting Research Centre, Melbourne, VIC Australia

Abstract

Background

While schools can provide a comprehensive sampling frame for community-based studies of children and their families, recruitment is challenging. Multi-level approaches which engage multiple school stakeholders have been recommended but few studies have documented their effects. This paper compares the impact of a standard versus enhanced engagement approach on multiple indicators of recruitment: parent response rates, response times, reminders required and sample characteristics.

Methods

Parents and teachers were distributed a brief screening questionnaire as a first step for recruitment to a longitudinal study, with two cohorts recruited in consecutive years (cohort 1 2011, cohort 2 2012). For cohort 2, additional engagement strategies included the use of pre-notification postcards, improved study materials, and recruitment progress graphs provided to school staff. Chi-square and t-tests were used to examine cohort differences.

Results

Compared to cohort 1, a higher proportion of cohort 2 parents responded to the survey (76% versus 69%; p < 0.001), consented to participate (71% versus 56%; p < 0.001), agreed to teacher participation (90% versus 82%; p < 0.001) and agreed to follow-up contact (91% versus 80%; p < 0.001). Fewer cohort 2 parents required reminders (52% versus 63%; p < 0.001), and cohort 2 parents responded more promptly than cohort 1 parents (mean difference: 19.4 days, 95% CI: 18.0 to 20.9, p < 0.001).

Conclusion

These results illustrate the value of investing in a relatively simple multi-level strategy to maximise parent response rates, and potentially reduce recruitment time and costs.

Introduction

Approximately 98.8% of Australian children aged 6–15 years are enrolled in school, [1] making the school setting appropriate for ascertaining population-representative samples of children and their parents. Numerous studies have demonstrated that school-based recruitment is challenging [25] and it can be difficult to achieve the high participation rates crucial for ensuring the generalizability of research findings. Suboptimal recruitment can extend recruitment periods, presenting budgetary and logistic challenges. Numerous recommendations have been made for increasing sample response rates generally [6, 7]. However, in the school setting, few studies have documented strategies that improve response rates or have compared the effectiveness of alternate recruitment approaches. The current study addresses this gap by comparing school-based recruitment approaches in a large sample of elementary school aged children.

In order to reach parents (or guardians) through schools, researchers need to engage and maintain the goodwill and participation of a variety of stakeholders including, but not restricted to, government education agencies, district representatives, school principals, teaching and administrative staff, children and interested community members [4, 5]. While the extent to which the school community engages with and supports the research is likely to influence parents’ decisions around participation, no studies have formally examined how the intensity of school engagement influences parent response rates. The complexities inherent in multi-level recruitment present a challenge for school-based health researchers to identify and implement optimal recruitment methods.

A number of studies have described methods to improve participant response rates in school-based health studies, [4, 8] however, the majority have used non-experimental designs, provided little detail about their response rate strategies and failed to compare the effectiveness of different approaches to improving response rates. Wolfenden et al. (2009) reviewed the literature examining the effectiveness of strategies for obtaining active parental consent through schools. Their review suggested five key strategies for enhancing parent participation rates: 1) promoting the research to school principals, teachers, parents and students [2, 9]; 2) dissemination of study information using methods allowing direct rather than mediated communication (i.e., face-to- face contact with parents was more effective than receiving information via students) [10]; 3) offering incentives to teachers, peers and individual participants [11, 12]; 4) providing three follow-up reminder contacts to parents who have not made a decision regarding participation [11]; and 5) ensuring that a dedicated member of the research team co-ordinates and closely monitors the recruitment process (i.e., a dedicated study coordinator is more effective than leaving responsibility to teachers) [11]. This review suggests that in addition to targeting parents through direct communication and targeted follow-ups, the successful engagement of other members of the school community may play a role in promoting parent participation in research.

We examined the impact of an enhanced engagement approach on parent recruitment as part of the Children’s Attention Project. At baseline the study involved collecting brief parent and teacher data on all Grade 1 children (the second year of formal schooling in Australia), with recruitment occurring over two consecutive years (cohorts 1 and 2 respectively). Our goal was to identify how the engagement strategies employed for cohort 1 could be improved for cohort 2, and to evaluate impact on a range of indicators of parent response and recruitment rates. Specifically, we sought to compare the impact of the enhanced research condition (cohort 2) relative to the standard condition (cohort 1) on parent response rates, response times, reminders required, consent rates (for teacher participation and for later follow-up) and sample characteristics. Our enhanced methodology aimed to provide more frequent, direct, personalised contact with parents, combined with greater involvement of multiple stakeholders, including the Education Department, principals, teaching staff, administrative staff, and parents/primary caregivers. We hypothesised that compared to the standard condition, the enhanced method would result in higher parent response and consent rates, faster response times, fewer reminders and the recruitment of a more representative sample. Teacher responses rates were not directly targeted in the enhanced approach as these were very high for cohort 1 (98.7%). Nonetheless, we compared teacher response rates, response times, and reminders required for the two cohorts to assess whether these were maintained or adversely affected by the enhanced parent engagement approach.

Methods

Study design

This study, approved by the Human Research Ethics Committees of the Royal Children’s Hospital (#31056) and the Victorian Department of Education and Early Childhood Development (#2011_001095), was conducted as part of the Children’s Attention Project (CAP). Parents (or guardians) were asked to nominate their consent on the returned survey and provide consent for their child’s teacher to also complete a screening survey. This method of consent was approved by the ethics committee. CAP is a community-based longitudinal study of children with ADHD and non-ADHD controls that seeks to examine the developmental course of ADHD and the factors influencing outcomes. As a first step in identifying the sample for the longitudinal study, we sought to screen all children in Grade 1 at participating state government elementary schools using parent- and teacher-reported symptoms of ADHD. All children meeting pre-determined criteria on the combination of parent and teacher screeners (ADHD group), plus a matched sample of children who did not meet criteria on both screeners (non-ADHD group), were invited to participate in the next stage of the research (formal confirmatory diagnostic assessment of ADHD status, followed by baseline data collection from parents, children and teachers). The study’s aims, key research questions and design have been detailed previously [13].

Elementary schools were recruited to the study via the state authority responsible for government schools (Victorian Government Department of Education and Early Child Development). Schools in two regions (Eastern and Western) of metropolitan Melbourne, Australia, were selected to ensure coverage of diverse socioeconomic communities [14]. Forty-one schools participated in both 2011 (cohort 1 recruitment) and 2012 (cohort 2 recruitment) with an additional 2 schools who had heard about the study asking to participate in 2012.

Data collection

The study sought to collect screening data from the parents and classroom teachers of a representative sample of Grade 1 students in 2011 (cohort 1) and 2012 (cohort 2), with a goal of achieving parent and teacher response rates for over 70% of all students. Data collection took place in the third and fourth terms of the academic year for cohort 1 (June—October) and in the second and third terms of the academic year for cohort 2 (April—August). Schools provided the researchers with a list of first names, gender and postcodes for all Grade 1 students. Parents were distributed information packs inviting them to participate by completing a screening survey and providing consent for their child’s teacher to also complete a screening survey. Parent and teacher screeners each contained the 10-item Conners 3 ADHD Index [15]. Parent screeners also asked whether the child had a previous diagnosis of with ADHD and collected brief demographic details (e.g., child date of birth and gender; language spoken at home) and contact details for subsequent follow-up. Neighborhood socioeconomic disadvantage was measured by the Socio-Economic Indexes for Areas Disadvantage Index (SEIFA) [16] for the child’s postcode of residence (mean (SD) = 1000 (100); higher scores reflect less disadvantage). On the basis of combined parent and teacher screening data, a sub-sample of parents was then approached to participate in a detailed baseline assessment and the longitudinal study (involving parent, teacher and child participation). The following sections and Table 1 describe the processes used to maximise parent and teacher participation rates in screening and consent to the longitudinal study.

thumbnail
Table 1. Multi-level approach to improving response rates.

https://doi.org/10.1371/journal.pone.0126950.t001

School recruitment strategies

Strategies employed to recruit schools were consistent across both waves of recruitment. As shown in Table 1, at the school level, strategies were targeted at engagement of Education Department representatives, school principals and classroom teachers. These included researcher presentations at regional meetings of school principals, followed by individual meetings with interested principals and separately with their Grade 1 teachers to discuss project details and address any concerns raised. Schools were accepted into the study if teachers indicated support and principals signed a research agreement supporting the school’s participation. Teacher relief funding was provided to schools to compensate for teacher participation.

Standard condition parent and teacher recruitment strategies

Strategies to engage parents and teachers were structured to draw attention to the study prior to data collection (Table 1 Pre-distribution strategies), and to encourage participation during the data collection period (Table 1 Parent strategies; Teacher strategies). Parent screening surveys were sent home with children through the school and returned in sealed envelopes via classroom collection boxes or reply paid mail. Reminder letters and replacement surveys were sent to parents who did not opt-out and did not return the survey at two and four weeks after the initial distribution.

Upon receipt of parental consent, teachers were asked to complete the teacher screening survey. To streamline workloads for teachers, their screeners were sent in batches, so that teachers completed these for the majority of children in their class at one time. Reminders and replacement surveys were sent after two weeks.

Enhanced condition parent and teacher recruitment strategies

After completion of cohort 1 recruitment and data collection, feedback was sought from school staff on strategies that could be used to enhance recruitment for cohort 2. As shown in Table 1 the enhanced approach was targeted at multiple stakeholders: Education Department, principals, teaching staff, administrative staff, and parents/primary caregivers. To ensure parents were aware of study commencement and to minimise adverse reactions arising from receiving study information without prior warning, a pre-notification postcard was mailed directly to parents from the school prior to survey distribution. Parent materials were also modified to be more personalised and attractive (e.g., hand signed letters, study logo). To maintain study visibility and encourage schools to actively seek ways to promote the return of parent and teacher surveys (whether completed or returned blank with a decline to participate), school staff were provided with graphs documenting the study’s progress in their school throughout the data collection period. These were sent to principals, teaching and administrative staff, and provided a comparison of their response rate to other schools within their region, and a profile of response rates by classroom at their school.

Statistical analyses

Chi-square and t-tests were used to examine differences in response rates, response times and reminder rates between cohort 1 and cohort 2 and sample characteristics between responders and non-responders. Analyses were conducted using Stata 12.1 (Stata Corp, College Station, TX, USA).

Results

Sample characteristics

For cohort 1, responses were received from the parents of 1891 children (69.2%), with screening data provided by 1545 parents (56.5%) and the teachers of 1502 children (98.7% of those for whom consent was provided). Complete parent and teacher screening data were available for 55% of the potential cohort 1 population. For cohort 2, responses were received from the parents of 2561 children (76.1%), with screening data provided by 2401 parents (71.3%) and the teachers of 2249 children (99.3% of those for whom consent was provided). Complete parent and teacher screening data were available for 66.7% of the potential cohort 2 population.

Compared to cohort 1, children in cohort 2 were somewhat younger as they were recruited earlier in the year to the previous cohort. Cohort 2 participants were from more socially disadvantaged areas compared to cohort 1 participants (mean 1013.0 vs 1016.3)—see Table 2.

thumbnail
Table 2. Response rate and sample characteristics for cohort 1 and 2.

https://doi.org/10.1371/journal.pone.0126950.t002

Response rates differences across cohort 1 and cohort 2

As shown in Table 2, compared to parents in cohort 1, parents in cohort 2 were more likely to respond to the screening survey (76% versus 69%; p < 0.001), consent and complete the screening survey (71% versus 57%; p < 0.001), agree to teacher participation (90% versus 82%; p < 0.001) and consent to further follow-up (91% versus 80%; p < 0.001). Fewer cohort 2 parent participants received a reminder to return their surveys (52% versus 63%; p < 0.001), and on average they took less time to respond to the survey (mean difference: 19.4 days, 95% CI: 18.0 to 20.9, p < 0.001). While the proportion of children with complete parent and teacher screening data was higher for cohort 2 than cohort 1 (67% versus 55%; p<.001), there was no difference between cohorts in the number of participants recruited into the longitudinal study (65% vs 60%; p = 0.20). We received more teacher surveys for children in cohort 2, compared to those in cohort 1 (99.3% vs 98.7%, p = 0.04, however these were returned at a slower rate (mean difference: -3.9 days, 95% CI: -5.8 to -2.0, p <0.001) and required a reminder (28% vs 10%, p < 0.001).

Difference between participants and non-participants across the cohorts

For both cohorts, participants (parents who completed the screening survey) did not differ from non-participants with respect to child age (see Table 3). Parents who participated were more likely to come from socially advantaged areas. Compared to non-participants, participants in cohort 2 were more likely to be parents of female children.

thumbnail
Table 3. Sample characteristics of participants and non-participants in cohort 1 and cohort 2.

https://doi.org/10.1371/journal.pone.0126950.t003

Discussion

We examined the effectiveness of an enhanced research condition targeted at multiple stakeholders (Education Department staff, principals, teaching staff, administrative staff and parents) compared to a standard research condition in improving parent response rates. As expected, when compared to the standard condition (cohort 1), parents in the enhanced condition (cohort 2) were more likely to respond to surveys, consent to participate, agree to teacher participation, require less reminders, and respond more promptly to the survey. Our results support the value of investing in a multi-level strategy to maximise response rates in school-based research. The higher and more rapidly achieved parent response rate for cohort 2 resulted in savings in time and resources and significant cost savings to the project.

The effectiveness of the enhanced approach to recruitment highlights the impact that relatively simple strategies can have on parents’ participation in research. The main parent-focussed strategies in the enhanced research condition were pre-notification postcards alerting parents to the forthcoming survey and improved study materials (e.g., inclusion of study logo, hand signed letters). The main strategies targeting school staff were progress graphs documenting the school’s response rates. Together these additional strategies resulted in a 9% increase in the numbers of parents responding to the survey, a 15% increase in parent consent and an 8% increase in parents’ providing consent for teacher participation.

Although teacher response rates were not directly targeted in the enhanced approach as these were very high for cohort 1 (98.7%), we also found an increase in teachers’ response rate in the enhanced condition (99.3%), but teacher surveys were returned at a slower rate and more often required a reminder. The high parent response rate achieved in the enhanced condition resulted in individual teachers being asked to complete surveys for more children. The slower return rate and greater number of reminders required of teachers suggests this increased the burden on teachers, but did not prevent them from participating. Overall, while the enhanced approach fell short of our target of achieving complete parent and teacher screening data on 70% of Grade 1 students, it was substantially improved over the rate achieved in the standard condition (67% versus 55%).

It was also hypothesised that the enhanced research strategy would lead to the recruitment of a more representative sample of the population and the results were mixed. Participants in both cohorts did not differ from non-participants with respect to child age. In the enhanced research condition, there was increased participation of parents of female children, with cohort 2 participants more closely reflecting the gender balance in the community. This suggests that the enhanced research condition led to improvements in gender representativeness in the sample. The finding that a more socially advantaged sample participated in both cohorts suggests that regardless of the strategies used to improve response rates, some bias remained. It is possible that there are particular barriers to recruiting socially disadvantaged families to participate in school-based research, such as school non-attendance. This represents an important design challenge in recruiting a sample that is truly representative of the community and requires further study.

The results of this study demonstrate support for the strategies previously identified as effective in enhancing response rates. In line with Wolfenden et al. (2009), we found that promoting the research to school principals, teachers, parents was effective. Furthermore, follow-up reminder contacts to parents who had not made a decision regarding participation was crucial in boosting response rates, in line with other studies noting the importance of follow-up reminders [11]. The findings of this study emphasise the importance of having a multi-level approach, targeting multiple stakeholders (Education Department staff, principals, teaching staff, administrative staff and parents), in order to impact parent response rates in schools [2].

A limitation of our comparison between the enhanced and standard conditions was the use of the same group of schools. We cannot exclude the possibility that familiarity with study procedures and the longer established relationship with the study team may have also contributed to improved response rates. The simultaneous introduction of a range of additional strategies for cohort 2 precluded the possibility of identifying the relative effectiveness of specific strategies on parent response rates. However, the value of such comparisons may be limited. Single strategies may vary in their effectiveness across different social, economic or national contexts, and researchers typically employ as many strategies as they can afford within their practical, time and budgetary constraints. The multi-level approach used in this study requires high levels of engagement between researchers and school staff, making them most suitable for application in study designs where researchers have the resources to engage with multiple stakeholders. The paper-based survey methods used in this study are increasingly being replaced by electronic surveys, where the response rate challenges are likely to differ. Nevertheless, many of the strategies outlined are suitable across research contexts. A key message from this study is that the use of strategies that engage all levels of the school community in the business of sample recruitment/data collection, informed through consultation with the target participants, can produce significant improvements in parent response rates, with financial and time savings, and without any obvious adverse effects on the characteristics of the recruited sample and their intentions for long term participation.

Conclusions

In conclusion, our findings indicate that implementing relatively simple enhancements that seek to promote awareness of and engagement in the study across all levels of the school community can result in significant improvements in parent participation, leading to more cost-effective recruitment and a more gender representative sample. This paper provides a detailed description of the multi-level strategies used to improve parent response rates in a large community-based study to act as guide for other researchers conducting school-based recruitment.

Acknowledgments

We would like to acknowledge all research assistants, students and interns who contributed to data collection for this study. We would also like to thank the many families, teachers, and schools for their participation in this study. The study has been funded by the National Health and Medical Research Council of Australia (project grant number: 1008522) and internal funding from the Murdoch Childrens Research Institute. It has been granted approval by the Human Research Ethics Committees of the Royal Children’s Hospital (#31056) and the Victorian Department of Education and Early Childhood Development (#2011_001095).

Author Contributions

Conceived and designed the experiments: ES JN DE VA. Performed the experiments: EJS. Analyzed the data: ES EJS. Wrote the paper: EJS ES JN DE VA.

References

  1. 1. Australian Bureau of Statistics. Aus Stats 3201.0 (2010) Population by Age and Sex, Australian States and Territories, Jun 2010. Canberra, Australia: Australia Bureau of Statistics.
  2. 2. Harrington KF, Binkley D, Reynolds KD, Duvall RC, Copeland JR, Franklin F, et al. (1997) Recruitment issues in school-based research: lessons learned from the High 5 Alabama Project. J Sch Health 67: 415–421. pmid:9503347
  3. 3. Midford R, McBride N, Farrington F (2000) Conducting research in schools: Lessons learnt from experience. Health Promotion Journal of Australia: Official Journal of Australian Association of Health Promotion Professionals 10: 63–68.
  4. 4. Petosa R, Goodman RM (1991) Recruitment and retention of schools participating in school health research. J Sch Health 61: 426–429. pmid:1812320
  5. 5. Vukotich CJ Jr, Cousins J, Stebbins S (2014) Building sustainable research engagements: Lessons learned from research with schools. J Res Pract, 10:1–18.
  6. 6. Claudio L, Stingone JA (2008) Improving sampling and response rates in children’s health research through participatory methods. J Sch Health 78: 445–451. pmid:18651932
  7. 7. Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, et al. (2007) Methods to increase response rates to postal questionnaires. The Cochrane database of systematic reviews: Mr000008.
  8. 8. Wolfenden L, Kypri K, Freund M, Hodder R (2009) Obtaining active parental consent for school-based research: a guide for researchers. Aust N Z J Public Health health 33: 270–275.
  9. 9. Cline A, Schafer-Kalkhoff T, Strickland E, Hamann T (2005) Recruitment strategies for the Princeton (Ohio) city school district epidemiological study. J Sch Health 75: 189–191. pmid:15989089
  10. 10. Stein BD, Jaycox LH, Langley A, Kataoka SH, Wilkins WS, Wong M. (2007) Active parental consent for a school-based community violence screening: Comparing distribution methods. J Sch Health 77: 116–120. pmid:17302853
  11. 11. Fletcher AC, Hunter AG (2003) Strategies for obtaining parental consent to participate in research. Fam Relat 52: 216–221.
  12. 12. McMorris BJ, Clements J, Evans-Whipp T, Gangnes D, Bond L, Toumbourou JW, et al. (2004) A Comparison of methods to obtain active parental consent for an international student survey. Eval Rev 28: 64–83. pmid:14750292
  13. 13. Sciberras E, Efron D, Schilpzand EJ, Anderson V, Jongeling B, Hazell P, et al. (2013) The Children's Attention Project: a community-based longitudinal study of children with ADHD and non-ADHD controls. BMC psychiatry 13: 18. pmid:23305499
  14. 14. Barnes G (2011) Report on the generation of the 2010 Index of Community Socio-Educational Advantage (ICSEA). Sydney, NSW: Australian Curriculum, Assessment & Reporting Authority.
  15. 15. Conners CK (2008) Conners (3rd Edition). Toronto: Multi-Health Systems.
  16. 16. Australian Bureau of Statistics. Aus Stats 6523.0 (2005) Household income and Income Distribution, Australia 2003–04 (Appendix 3). Canberra, Australia: Australian Bureau of Statistics.