|
Health Communication and W. Douglas Evans, PhD 1 1 The George Washington University School of Public Health and Health Services Please address all correspondence to: Suggested Citation: Evans WD; Davis KC; Zhang Y. Health Communication and Marketing Research with New Media: Case Study of the Parents Speak Up National Campaign Evaluation. Cases in Public Health Communication & Marketing. 2008; 2:140-158. Available from: www.casesjournal.org/volume2. AbstractNew media present opportunities to evaluate health communication and marketing (HCM) media campaigns. Using efficacy testing methodologies, new media can be an important tool to enhance the evidence base on health messaging. In this paper, we illustrate the current use of and opportunities for research using new media by describing the online efficacy evaluation of the Parents Speak Up National Campaign (PSUNC). PSUNC is designed to increase parent-child communication about sex as a proximal behavioral outcome leading to delayed onset of sexual activity. The campaign primarily uses public service announcements (PSAs) and a Website to deliver messages. The use of PSAs precluded a field-based evaluation design since PSAs would not generate measurable exposure to the campaign. Thus, an online evaluation strategy using the Knowledge Networks panel was adopted to overcome this challenge and test for efficacy. The evaluation was a randomized controlled trial of the effects of campaign exposure on parent-child communication outcomes. Parents were randomized into control, exposure, and booster exposure conditions. Exposure study participants viewed and heard campaign messages online or by mailed DVDs and completed online surveys. Preliminary findings suggest the campaign increased parent-child communication about sex and affected other intended outcomes. The evaluation illustrates how online social marketing efficacy studies can overcome many traditional evaluation challenges. More widespread use of efficacy studies with new media would enhance the evidence base in social marketing. IntroductionNew media, including all forms of electronic communication, present opportunities to evaluate health communication and marketing (HCM) media campaigns. Using efficacy testing methodology, new media can be an important tool to enhance the evidence base on health messages. The Internet, in particular, provides tremendous opportunities to expose study participants to messages, assign them to experimental conditions, and test message effects. However, there are relatively few examples of using new media this way. One recent exception is an ongoing evaluation of Center for Disease Control and Prevention's (CDC) Take Charge. Take the Test campaign for promoting HIV testing. 1 There is a need for a greater understanding in the HCM community about methods for exposing audiences to messages using new media and evaluating outcomes. The purpose of this paper is to provide an overview of the use of new media in HCM research and evaluation, review a major case study of a new media HCM study, and recommend a research agenda to expand and develop the field. New Media are increasingly being used both as a tool to deliver HCM messages and as a means of evaluating campaigns. HCM can use new media in two general ways: 1) to deliver messages directly through new media channels; and 2) to expand the reach of and interaction from TV public service announcements (PSAs) (e.g., a TV PSA that directs the audience to a Website for detailed information and to interact). HCM campaigns that drive traffic to Websites have been common in the past decade. For example, the American Legacy Foundation truth campaign used a "facts and information" advertising strategy to articulate the tobacco industry's manipulative marketing practices aimed at youth and to counteract misinformation. Teens and young adults were directed to the truth website for more information about industry practices and the truth brand. This also served the campaign's purpose of engaging youth by encouraging them to interact with the truth brand as part of a social mobilization strategy.2 Delivery of HCM through new media is still small by comparison, but is growing rapidly.3 There are tremendous opportunities to deliver messages such as nutrition, tobacco control, or condom use promotion through channels like social networking Websites, pop-up ads, blogs, e-mail, and text messaging. New media offer opportunities for social marketers to inexpensively utilize innovative commercial strategies such as stealth and viral marketing.4 The data collection opportunities provided by Websites and other new media also enable social marketers to develop more sophisticated audience segmentation strategies to identify refined health behavior predictors and message strategies. Marketing research using new media allows social marketers to develop tailored messaging, such as tobacco prevention messages for an adolescent who uses a particular social networking Website and answers questions indicating he or she is open to experimenting with cigarettes. HCM campaigns have typically been evaluated through field-based effectiveness evaluations conducted during or after implementation without the use of efficacy studies. An efficacy study is designed to evaluate the effects of an intervention under optimum conditions whereas an effectiveness study evaluates the effects of an intervention under real world conditions.5 But without first determining the efficacy of the campaign messages, results of effectiveness studies can be difficult to interpret.5 For example, if a campaign produces negative or null results, an effectiveness evaluation may not be able to distinguish whether the reason the campaign failed was because the campaign was implemented in an ineffective way or because the campaign messages were not efficacious in the first place. Evans and colleagues (forthcoming) argue that more rigorous efficacy studies are needed before HCM campaigns are implemented.5 New media offer substantial opportunities to conduct such research. The literature on health communication and marketing suggests opportunities to build the knowledge base through more targeted and effective use of efficacy study designs. Opportunities for efficacy research are greater now with the advent of new media marketing research tools such as Web-based panels. Two major examples that often host academic and public sector oriented studies are Knowledge Networks (www.knowledgenetworks. com), which offers a probabilitybased sample and Harris Interactive (www. harrisinteractive.com), which offers access to a large sample of individuals who have "opted in" to their panel. There are also numerous panels, such as e-Rewards (www. erewards.com) and Mysurvey (www.mysurvey. com), that to date have been used primarily for commercial market research purposes. Most available panels make efforts to maintain samples that closely resemble the U.S. population and many are developing specialized panels, such as health care provider panels or Latino panels. A key feature of conducting an efficacy study online is the ability to expose participants to multimedia interventions via their personal computers. Each of these panels provides opportunities to show multimedia to respondents, and to select a sample based on specific target audience characteristics. These features facilitate multi-factorial designs (e.g., exposure x time) to test the efficacy of messages and specific advertising executions. At the same time, there are tradeoffs in choosing an online efficacy approach, such as lower external validity (e.g.,, multimedia not experienced in a natural setting) but significant design advantages. Evans and colleagues (forthcoming) summarized the strengths and weaknesses of efficacy and effectiveness design options in health communication research.6 In the remainder of this paper, we illustrate the current use and opportunities for future research using new media by describing a major recent example of online efficacy testing. We conclude with recommendations for future development of online HCM research. Case Study: Online Evaluation of the Parents Speak Up National CampaignBackground Based on an extensive review of the literature, a recent report summarized what is known about the risk and protective factors that affect adolescent pregnancy and sexual activity.11 This review identified proximal and distal factors influencing sexual activity among adolescents (Table 1). In addition, numerous studies have reported that parent-child communication about reproductive health issues is associated with delayed sexual initiation and reduced sexual activity among adolescents.12-14 ![]() Many strategies have been employed with mixed levels of success to target risk and protective factors associated with unintended adolescent pregnancy, but HCM has only recently emerged as an approach to address this issue, and the efficacy and effectiveness of health messages in this area have not been rigorously studied. The U.S. Department of Health and Human Services developed the Parents Speak Up National Campaign (PSUNC) to promote parent-child communication about sexual activity, as a means to delay sexual debut among adolescents. PSUNC is a national multi-media campaign that has as its major theme the message that parents should talk "early and often" to their pre-adolescent and adolescent children aged 10-14 years about delaying the onset of sexual activity. The purpose of PSUNC is to increase parent-child communication as a proximal behavioral outcome leading to delayed onset of sexual activity. The strategy of aiming messages promoting parent-child communication at this target audience has been used on a smaller scale in school- and community-based interventions, but never on a national scale in a mass communication campaign. The campaign was publicly released on June 21, 2007. PSUNC applies many of the transferable principles of marketing. In particular, the campaign uses a theory-based behavior change model that hypothesizes increased parent-child communication will result from positive message reactions to campaign advertising. Socio-demographic, family and community-level social influences will moderate these relationships. PSUNC also develops a credible and likeable "argument" for delaying initiation of sexual activity by communicating personal (social, educational, career-related) advantages of abstinence. 15 The campaign also utilizes risk communication strategies, such as promoting self-efficacy and using fear appeals, to communicate the health risks of early sexual debut and benefits of waiting along with the importance of parent-child communication. 16,17 The primary communication channel for the campaign is paid and unpaid PSAs designed for a general audience of parents, and targeted versions of the general PSAs for African American, Hispanic, and Native American audiences. The unpaid PSA campaign launched in Summer 2007, with a spot called "Talk to me" that featured pre-adolescent children encouraging their parents to talk to them about sex and waiting to become sexually active. In the Winter and Spring of 2008, the campaign continued with a second spot called "Muffinhead," which sought to allay parents fears about talking to their child, and aired as a paid PSA on cable channels nationwide. The campaign also utilizes outdoor advertising (billboards), bus media, posters, Web banners, and media kits. In addition to the public awareness campaign, PSUNC includes a second component, the 4Parents.gov Web site and associated parent and adolescent online and print guides. The 4parents.gov Web site is intended to provide information (as opposed to motivational messages) for parents about how to talk with their pre-teen or teen about waiting to have sex, accuracy of social norms among teens, perceived barriers among parents to talking with their child, broader related topics such as parent-child relationship quality and setting goals for the future, establishing rules and expectations about dating and sex, teaching refusal skills, characteristics of healthy relationships, consequences of teen sexual activity, and some special topics (e.g., what to do if the teen has already had sex, homosexuality, pregnancy, contraception, and legal issues).18 New media tools to evaluate PSUNC However, some concerns have been raised about online panels. First, since panels rely on re-interviewing panelists, systematic panel attrition can produce a panel that is unrepresentative of the target population. Second, interviewing and re-interviewing panelists may change the opinions/behaviors of the panelists, creating unrepresentative panelists (e.g., they become professional survey respondents). To investigate the prevalence and impact of these possible biases, Clinton (2001) examined the extent (and effect) of panel attrition in Knowledge Networks' panel and conducted an experiment designed to isolate the possible opinion/behavior changes introduced by panel participation.19 He found no significant attrition bias effects, nor panel bias effects. However, panels such as Knowledge Networks evolve continuously as panel composition changes. Thus more research on potential bias is needed. PSUNC efficacy study using Knowledge Networks As a result, an efficacy study design was selected to evaluate the PSUNC and its messages under controlled conditions. This represented the best possible opportunity to maximize the quality of the design and answer key evaluation questions, such as the effects of the campaign on parents' knowledge, attitudes and beliefs about parent-child communication about sex, and frequency of such communication. Knowledge Networks was chosen for the PSUNC evaluation because it offered a probability-based sample of the U.S. population and had sufficient sample of the study target population. Also, PSUNC is a Federally- funded project and the evaluation data collection required Office of Management and Budget (OMB) approval. Knowledge Networks studies have previously received OMB approval based on the panel methodology, and to the investigators' knowledge other panels had not. Finally, Knowledge Networks panellists all have Internet connectivity through a computer or Web TV, which enables a television to access the Internet, and newer technologies based on the Web TV concept. Once recruited, study participants can readily complete surveys, view and hear online video and audio PSAs, and read print ads. Each of these considerations - feasibility, practicality, and convenience of reaching the target population - should be considered in designing similar research. Evaluation Design and Methods The resulting PSUNC evaluation study is a randomized controlled trial (RCT) of parents of children aged 10 to 14, the campaign's target audience. Mothers and fathers drawn from the Knowledge Networks panel were randomly assigned to control, treatment (exposure to a core set of PSUNC messages), and treatment plus booster (core plus additional PSUNC messages) conditions and surveyed at baseline (prior to message exposure) and at 4 follow up time points: 4 weeks, 6 months, 12 months, and 18 months after baseline. Baseline data were collected from August to October 2007. The first follow up was conducted from September to November 2007 (with surveys staggered to reflect approximately 4 weeks between baseline and follow up), and this paper reports only on those findings. Subse148 quent follow ups are scheduled for 2008 and 2009. It should be noted that while the PSUNC evaluation uses data on audience campaign exposure based on media market exposure (Gross Rating Points, or GRPs), the current analysis did not control for treatment or control respondent natural exposure to the PSAs. While control respondent exposure to the campaign is very likely minimal given the campaign's low immediate reach, we did not control for potential exposure in the present study. The PSUNC baseline survey was a 64-item instrument designed to be self-administered online by study participants. Respondents viewed each question individually on-screen, and responded by clicking on a radio-style button. After completing each question they clicked "next" to advance to the following screen. The follow-up surveys were a 60-item instrument (4 demographic questions were not repeated) that was formatted and administered in exactly the same manner as the baseline. Immediately after completing the baseline survey, exposure condition respondents viewed the PSUNC ads and completed a few additional questions to confirm that they attended to them on screen. At the 4-week follow up, these same respondents began the survey session by viewing the same ads again, and then completed the survey. The control respondents only completed the survey at both baseline and follow up. The study purpose is to determine the efficacy of PSUNC messages in increasing timing and frequency, and improving content of parent-child communication over time. Thus we collected data on potential mediators of the relationship between messages and behaviour, including parent self-efficacy, outcome efficacy, attitudes and beliefs, as well as moderating influences such as sociodemographics, family dynamics, media and community-level influences. Figure 1 (next page) summarizes the RCT design and measures. ![]() Message exposure protocol During the time period between the baseline and 4-week follow up, participants received standard reminder e-mails about the next survey. Four weeks after their initial survey date, respondents were prompted to take the survey again. Treatment subjects first viewed the ads again, either online or using the DVD. All participants then completed the follow up survey. Treatment participants then answered the same questions to confirm their ad exposure and recall. At the present time, the evaluation is ongoing. At the planned 6-month follow up point, control participants will continue to complete surveys. As described in Figure 1 and the narrative above, treatment fathers will continue to receive the PSUNC stimulus as before. However, treatment mothers will be divided into booster and regular treatment groups. The regular treatment participants will continue to be exposed to the PSUNC ads as at the 4-week follow up, and will complete their surveys as before. The booster participants will view the earlier ads plus newly created PSUNC ads (the second series including the "Muffinhead" spot described earlier). They will view the ads using the same technology (video streaming or DVD) as in previous exposures. The planned 12-month and 18-month follow up surveys and exposure protocols will follow the same steps as the 6-month follow up. Multimedia Stimulus Assignment by Race/Ethnicity Each participant assigned to an exposure condition views/listens to a package of media that includes: 2 print advertisements, one 60-second radio ad, and one 60-second TV ad. The ads are targeted by race/ethnicity. Print and radio ads target general audiences, African Americans, American Indians, and Hispanics. PSUNC television ads target general audiences, African Americans, and Hispanics. Table 2 below outlines specific assignments of print, radio, and television ads by race/ethnicity among study participants assigned to exposure conditions. ![]() Analysis ResultsIn total, 1,754 parents completed both the baseline and the four-week follow-up survey. Figure 2 summarizes parent socio-demographic characteristics. Overall, respondents were primarily between the ages of 35-55 and white, and had some College education or a bachelor's degree. The sample sizes of African-American (5.9%) and Hispanic (3.4%) parents were lower than expected. However, the overall sample sizes for these sub-groups with repeated measures over multiple follow up surveys provides sufficient statistical power to make statistical comparisons by race/ethnicity over the full study period. ![]() To capture the effect of message exposure on response options reflecting campaign outcome objectives (e.g., parents talking often to their children about sex), we created dichotomous variables for each outcome to indicate whether the parent's response changed in the desired direction between baseline and follow-up. The dichotomous variables had a value of one when the parent indicated positive desirable change and zero otherwise. Parents who already provided the extreme desired response (e.g. strongly agreeing with a statement promoted by the campaign) to an outcome measure at baseline were excluded from the analysis for that specific measure since there was no opportunity for change at the follow up survey. Thus, our analysis for each outcome measure includes only participants who had room for change at baseline. Table 3 (next page) presents the average weighted indicators for desired baseline to follow-up change for select attitudinal and behavioral outcomes by exposure to treatment, gender, and race. Overall, treatment parents were more likely than control to indicate improvements in self-efficacy about talking to their children about sexual activity ("How sure are you that you can always explain to your child..." p < .05). Additionally, Hispanic parents were less likely to improve in their self-efficacy to talk to their child about sexual activity compared to non-Hispanic white parents (p < .05). Treatment parents were more likely to have spoken to their children often about being sexually active (p < .01) and to have recommended to their child that they wait to have sex (p < .05). Treatment parents were also more likely to have visited the 4parents.gov website (p < .01), which provides additional reproductive health and communication resources. ![]() DiscussionThe evolving new media environment offers opportunities not only for innovative HCM strategies, messages, and delivery channels, but also for research and evaluation methods. As usual, the commercial sector is well out in front of public health in developing and applying these methods, as indicated by the expansion of stealth, viral, and other online marketing techniques and the proliferation of new media marketing research organizations. Public health can benefit greatly from these developments. Early results from the PSUNC efficacy evaluation suggest that exposure to the campaign had a short-term effect on important parent-child beliefs and behaviors among parents. One clear advantage of online HCM evaluation research such as this is the speed and rigor with which these findings could be obtained. However, the preliminary results discussed here should be treated with caution, as they stem from bi-variate analyses that do not control for potential confounds such as socio-demographic characteristics or social environmental influences. The evaluation is in part based on the hypothesis that these variables will moderate campaign effects. Future papers will report on analyses using additional follow up data and multivariate models. Also, future analyses will control for potential exposure to the campaign in real life among respondents, which is not included in the current analysis. There are several limitations to the results presented here, which are presented to illustrate how the PSUNC campaign was evaluated and provide an initial look at measures and outcomes. First, the sample of non-white respondents is low. This reflects a larger challenge with existing online panels that are used to study specific sub-populations (e.g., parents of children age 10-14). The challenge is that while online panels may have total samples that approximate the U.S. population distribution of major socio-demographic characteristics such as age, income, gender, or race/ethnicity, studies specifically targeted at particular sub-groups may or may not yield enough panelists to reflect the overall distribution of a particular characteristic. In the case of the PSUNC evaluation, we were not able to obtain a sample of parents with children of the desired age range across all racial/ ethnic subgroups proportionate to the U.S. population. Second, this analysis does not control for natural exposure to the campaign. While initially very low due to use of unpaid media, campaign exposure among exposure and control respondents will increase over time and represents an important potential confound. Multivariate analyses will use GRP data to control for campaign exposure outside the study environment. Cases such as the PSUNC efficacy evaluation provide useful lessons for future HCM research with new media. First, most campaign evaluation has historically been based on effectiveness designs and has been quasi-experimental or observational in nature.20 PSUNC offers a prime example of how campaign evaluations can apply efficacy testing methods in a randomized experimental design. The existence of online panels with large numbers of respondents creates opportunities for such studies, and thus the means to rigorously evaluate doseresponse effects of social marketing advertisements. 21 The PSUNC efficacy evaluation was done both to maximize experimental control, and also to address challenges raised by the campaign implementation using unpaid PSAs with low reach. This design provides valuable feedback to the campaign in terms of isolating message effects on the target audience, examining differences in message effects on mothers compared to fathers, and dose-response effects of higher versus lower campaign exposure that is not typically available in effectiveness evaluations.6 Ideally, as efficacy methods become more widely used, such evaluations would be conducted prior to campaign launch and results used for planning purposes to supplement qualitative formative research, which is aimed primarily at message design. Second, online studies offer opportunities to expose participants to multimedia and carefully control how participants interact with media materials. In particular, it enables investigators to test specific message features such as sensation value,22 stylistic features of ads,23 and compare ads using different marketing or message strategies.4 Previously, this has been a major weakness of the health communication and marketing research literature: We often know whether a campaign works or not, but not (or very little) about how or why it works. We need to understand more about which specific messages and features of messages are efficacious in order to develop successful campaigns and use scarce resources effectively.6 Given these opportunities, more HCM efficacy research using new media has significant potential to advance the field. Previously, there has been some efficacy testing studies done in tobacco control, but primarily through in-person exposure to advertisements in a laboratory-type setting. 24 These prior studies have been useful in that they used experimental designs, but are extremely limited by small sample sizes, the artificial laboratory setting, and low external validity. The flexibility in experimental design and exposure to messages afforded by online testing as well as the large sample sizes of web panels raises numerous potential areas of research for future studies. First, similar research in HCM can be expanded to include HIV/AIDS, nutrition, physical activity, injury prevention, and other risk behaviors and conditions. Second, there are opportunities to address substantive research questions that have received little attention in the past, including:
While HCM efficacy research has a long way to go, the existence of new media opens the door to answering these kinds of questions. The challenge is to expand on early cases such as PSUNC and build a new media HCM research agenda. AcknowledgmentsThis study was funded by the Department of Health and Human Services, Office on Population Affairs. References
Author InformationDoug Evans, PhD, is a professor at The George Washington University's School of Public Health and Health Services and director of the MPH program in Public Health Communication and Marketing. Kevin C. Davis, MA, is a research economist at RTI International. Yun Zhang is a research analyst at RTI International. |