Evaluating E-Consultations as Catalysts for the Youth HIV/AIDS MovementSantosh Vijaykumar, MA, MCom, Saint Louis University School of Public Health (SLU-SPH); Joya Banerjee, BA, Global Youth Coalition on HIV/AIDS (GYCA); Elizabeth A. Baker, PhD, MPH, Saint Louis University School of Public Health (SLU-SPH);AbstractAs the number of young people affected by HIV/AIDS assumes massive proportions, social movements to tackle the pandemic need increased organization and innovation. The Global Youth Coalition on HIV/AIDS (GYCA) was conceived at the15th International AIDS Conference (IAC) in Bangkok, 2002, to synergize the efforts of thousands of youth organizations and networks spread around the world, by bringing them under a common umbrella. The GYCA is a decentralized organization that uses Information and Communication Technologies (ICTs) to manage programs implemented by regional and national coalition members. In this case study, we describe and critically evaluate two e-consultations that the GYCA conducted with its members: 1) a participatory, shared decision-making approach to inform the GYCA's initial strategic plan; and, 2) the development of program materials and the strategic advocacy plan for the 2006 IAC in Toronto. Our analyses identify critical strengths of e-consultations in meeting the GYCA's aims, and important areas for improvement in future replications of the e-consultation model. Finally, we discuss study implications for other public health issues, and the role of ICTs and e-consultations in technology-driven health communication programs. The Problem:Every day, 6,000 young people are infected with human immunodeficiency virus (HIV), and over half of the five million people infected each year are under 25 years old.1 Although world leaders committed that by 2005 ninety percent of young people would know how to protect themselves from infection, currently, in the hardest hit countries, less than fifty percent of youth can correctly identify modes of HIV transmission.1 Despite the impact that HIV/AIDS has on youth, most programs addressing HIV/AIDS are designed by adults without any input from young people. Young leaders at all levels-local, national, regional, and international-are often marginalized and excluded from the development of important interventions. However, at the turn of the 21st century, young leaders began taking action in their communities to prevent the spread of HIV/AIDS and to address the consequences of the disease. 
Although these youth initiatives were having a positive impact on an ad-hoc basis, it was clear that the efforts lacked synergy, thereby diminishing the overall impact. Youth activists decided that there was an increasing need for a coalition that would bring these assorted initiatives under one umbrella, streamline their endeavors, and empower them with the necessary technical resources to create a noticeable, large-scale impact. Proposed Solution:It was during this time that youth leaders congregated at the 15th International AIDS Conference (IAC), held in 2004, in Bangkok, Thailand. A series of meetings and teleconferences during and after this conference brought representatives of various worldwide youth organizations to a common understanding. They agreed that it was only by organizing the existing scatter of international organizations and networks into a collective force, that they would best be able to push young people's vulnerability to HIV infection as a priority on decision-makers' agendas.. To achieve this goal, youth delegates generated support from UNAIDS (the Joint United Nations Programme on HIV/AIDS) and the United Nations Population Fund (UNFPA) to develop the Global Youth Coalition on HIV/AIDS (GYCA). Today, the GYCA is a youth-led international nonprofit alliance that empowers young leaders worldwide with the knowledge, skills and resources to scale up HIV/AIDS interventions among their peers. The GYCA implements its programs through a decentralized network, with a New York office acting as a catalyst. When the GYCA was first created, a major challenge emerged: how could the GYCA manage an alliance of this nature? The global HIV/AIDS pandemic straddles a wide variety of issues - economic, scientific, political, social, and psychological - whose importance greatly vary by region.2 he fundamental test for this alliance thus lay in its ability to include globally diverse concerns, and address each of them with equal emphasis and effect. This was even more challenging given the need for the alliance to develop a unified vision (which still represented the needs of its worldwide membership) to guide its activities and achieve its goals. The leaders of the coalition (known also as key taskforce members) decided that the quickest and most cost-effective approach to bringing the alliance's globally scattered stakeholders on a single platform was through Information and Communication Technologies (ICT) tools. The tools in this case were online tools, available to the GYCA through its partnership with Toronto-based Taking IT Global Foundation. What are Information and Communication Technologies (ICTs)?Though a standardized definition of ICTs is yet to emerge, the Information for Development Program (a partnership of international development agencies coordinated by the Global ICT Department of the World Bank) defines them as being, "tools that facilitate communication and the processing and transmission of information and the sharing of knowledge by electronic means."3 These electronic means include radio, television, fixed (landline) and mobile phones, the Internet, and digital audio and text recording, and Internet telephony, among others. ICT applications have been extensively and effectively utilized in a wide range of settings cutting across health issues. For example, ICT tools (mainly computers and the Internet) were used to build and strengthen Mozambique's information management in the health sector.4 
In Uganda, the Satellife project used handheld devices to disseminate health information, conduct surveillance, and monitor and evaluate health programs.5 Mobile phones have been used to address asthma, diabetes, cardiovascular disease and HIV/AIDS in Croatia, Finland, Spain and South Africa, respectively.6-10 In India, telemedicine technology - a combination of personal computers, customized medical software, ECG/X-ray machines and scanners - is bridging the divide between urban and rural healthcare.11 The Internet has diffused at a faster rate than any other technical innovation in modern times.12 Health information is rapidly disseminated through e-health mechanisms (e.g. health websites), in both the developed and developing worlds.13,14 Most notably, the Internet has been used to share and manage knowledge among health workers through e-learning forums and discussion boards.15 Youth, being early adopters of new technologies, have especially maximized the potential of the Internet in using various online tools (e-forums, discussion boards, weblogs, listservs and e-consultations) to initiate social action that cuts across international boundaries.16 What are E-consultations?E-consultations facilitate online deliberation and consensus building among key stakeholders by eliciting answers to critical questions. The practice of e-consultations is an offshoot of e-governance - a movement that has been gradually adopted by western democracies since the early 1990s. E-governance promotes increased engagement of citizens in the policy and decision-making process.17 For example, the European Union hosts an ongoing e-consultation portal (http://ec.europa.eu/yourvoice/index_en.htm) that invites citizens to share their opinions with European policy makers on diverse issues. In 2005, the Canadian government invited non-governmental organizations, academics and academic institutions, student groups, and think tanks to participate in an e-consultation to inform the country's foreign policy.18 However, research that explores the efficacy and impact of e-consultations is scant and is in the earliest stages of development. A literature search for the term 'e-consultation' in the following databases offered meager results: Pubmed (five results), Proquest-ABI / Inform (11 results), and Communication and Mass Media Complete (24), of which only seven articles evaluated e-consultations as a technological tool. Despite the limited work evaluating the efficacy of this type of approach, e-consultations are being widely adopted by a number of international development agencies, including the United Nations and the World Health Organization.19,20 This case study describes and critically evaluates two e-consultations conducted by the Global Youth Coalition on HIV/AIDS (GYCA). The first (conducted in 2004) was instrumental in charting the coalition's vision and organizational strategy. The second (conducted in 2006) took place on the eve of the 16th International AIDS Conference in Toronto, Canada. This e-consultation witnessed participation from the global membership of the GYCA and helped to shape an advocacy agenda for targeting policy makers and adult allies. E-consultation 1 - Youth HIV/AIDS Leaders: Building an empowered allianceThe GYCA leaders were first exposed to e-consultations at the Bangkok International AIDS Conference. After an online discussion group and coalition website were created on the Taking It Global portal, the leaders (key taskforce members) brainstormed on the most critical issues facing youth HIV/AIDS, and subsequently launched the e-consultation. The primary objective for the e-consultation was to perform a needs assessment that would eventually guide the development of the coalition. 
The needs assessment would attempt to extract from participants their needs, goals, and aspirations in the fight against HIV/AIDS. The results of this e-consultation would guide the GYCA's organizational framework, principle values, logistical structure, and future projects, as well as strategies for recruiting and involving members (including youth residing in resource-constrained areas). In line with these objectives, the main expectation was to garner maximum participation and extract region-specific insights from geographically diverse stakeholders. The e-consultation was conducted on the Taking It Global online platform over four weeks. Key taskforce members sent invitations to participants using existing networks (such as coalitions like UNFPA's youth focal points in geographic divisions, UNICEF's Voices of Youth, and participants of the Oxfam International Youth Partnerships program) and numerous e-mail listservs. Each week, participants responded to a set of questions centered on a different theme. Key taskforce members moderated the discussions, and also picked the themes for each week based on their experiences and knowledge of the youth HIV/AIDS situation. Participants were encouraged to respond to statements in their native language, and each week the content of the previous week's discussion was summarized in English, Spanish and French to maximize member engagement.21 The exact structure of weekly themes and issues is outlined in Table 1. Table 1: Overarching discussion themes and specific issues in the first e-consultation
AThe 2001 United Nations General Assembly Special Session Declaration of Commitment on HIV/AIDS, considered to be non-binding international law. Results:365 participants representing 48 countries contributed a range of perspectives to the questions posed. The main findings of this e-consultation were:
Impact on organizational priorities: The essence of the discussions, as described above, became a guidepost for the core GYCA task force in charting the GYCA's organizational strategy and shaping the course of the GYCA's programmatic activities. The e-consultation helped coalition to identify the four priority areas that now guide its activities:
Impact on organizational structure: The e-consultation also helped to design the organizational structure of the GYCA. Today, the coalition is based out of its North Secretariat in the United States, but is driven by its 12 Task Force Members, 12 Regional Focal Points covering all continents, and multiple National Focal Points in each of the 12 regions. The e-consultation also stimulated the creation of a South Secretariat, who acts as an equal partner to the North Secretariat, thereby enhancing contributions from youth in the Global South. Dissemination of e-consultation findings: The final e-consultation report, prepared by the moderators and translated into English, Spanish and French, was shared widely with participants, and with hundreds of HIV/AIDS and reproductive health organizations via listservs and email. Participants were asked to share the report with youth who did not have access to the consultation process. E-consultation 2 -Toronto YouthForce AdvocacyWhen the GYCA was first created in December, 2004 it had a membership of nearly 350 youth leaders from 48 countries. By 2005, its membership had grown to include 2,100 youth leaders from nearly 100 countries. It was at this point that the GYCA began preparations for the 2006 International AIDS Conference, to be held in Toronto, Canada. The conference was an international gathering of 24,000 people, including activists, scientists, policymakers, and people living with HIV/AIDS. This event was strategically important to the GYCA because it presented its youth leaders with the opportunity to advocate their agenda directly alongside key adult allies from various constituencies. The coalition's leaders recognized that advocacy efforts would have to be organized and well-planned, so they decided to conduct an e-consultation. The e-consultation's main objective was to arrive at a consensus for the advocacy agenda (messages and strategy) that would be presented at the conference. The GYCA also utilized this opportunity to address other issues, such as the possibility of increasing the number of conference scholarships, which would increase the youth presence. The GYCA conducted the e-consultation as a partner of the Toronto YouthForce (TYF), an alliance of over 100 international youth non-governmental organizations and networks. First initiated at the 2002 Barcelona International AIDS Conference, YouthForce prioritized youth participation and activities during subsequent Conferences. GYCA and YouthForce members were invited to participate in the e-consultation, which was conducted on the Taking It Global e-forum. YouthForce members assisted in designing the e-consultation and its questions, facilitating the consultations each week, writing weekly summaries, compiling the results into a final report to be shared with partners and funders, and translating the results into English, Spanish and French.22 The e-consultation was conducted in April, 2006, and was similar in design to the first GYCA e-consultation. Table 2 illustrates an outline of the design of the second e-consultation (See Table 2). Table 2: E-consultation design for the International AIDS Conference, Toronto, Canada
Results: 218 participants representing 36 countries participated in the e-consultation. The main findings from this e-consultation were:
The GYCA members communicate their agenda at IAC, 2006
The above findings led to the following impact:
Critical Evaluation of GYCA's E-consultations:The realm of e-consultations has witnessed little scholarly attention - only a handful of studies have been used to measure this tool's effectiveness. There is, however, an evaluative framework proposed by Macintosh & Whyte,23 that suggests a way to critically analyze e-consultations, based on a set of predetermined parameters: 1. Was the e-consultation process conducted as planned? 2. Were the consultation objectives and what was expected of the citizens made clear? 3. Did the consultation reach the targeted audience? 4. Was the information provided appropriate? 5. Were the contributions informed and appropriate? 6. Was feedback provided both during and after the consultation? 7. Was there an impact on policy content?Macintosh and Whyte posit that it is important to understand and evaluate the sociology of Internet behavior characterized by a number of factors like: depiction of personal identities in virtual space, communication dynamics of questioning and response, access to technology and program duration. They also argue for an inter-disciplinary approach to evaluating e-consultations which includes political, technological, and social perspectives. While a political evaluation mainly analyzes whether the e-consultation conformed to best practice standards, a technological evaluation focuses on the system design of the e-consultation. Social evaluations examine the influence that stakeholders' participation has had on policy-making.24 We will use the Macintosh & Whyte (2002) framework to assess the two e-consultations. 1. Was the e-consultation process conducted as planned? The first e-consultation was comparatively sporadic and unplanned compared to the second, which the organizers approached with a preconceived strategy. The second e-consultation was systematically pre-planned, in terms of who were the targeted participants and moderators, and what the GYCA hoped to achieve. The first e-consultation was not preceded by similar planning and strategizing. In the first instance, the idea of conducting an e-consultation as a needs assessment was developed after the key taskforce members participated in a similar exercise during the Bangkok Conference. Despite the lack of a concrete blueprint, the key taskforce members envisaged a specific process: invite participants through email, brief them about the e-consultation objectives, elicit responses on one theme per week, orchestrate a consensus, and disseminate findings to respondents. This process was used to guide the first e-consultation and this process was adapted for use in the second e-consultation. Both e-consultations had various benefits for the coalition, in terms of organizational strategy building, knowledge sharing, and capacity building. However, it is important to mention that neither e-consultation used a standard organizational system/framework, which might have helped in streamlining the process and engineering them more efficiently. 2. Were the consultation objectives and what was expected of the citizens made clear? The two moderators introduced the purpose and objective of each e-consultation before its commencement in the form of a 'letter of invitation'. This letter described in detail the need for a global coalition, its proposed aims, and possible priority areas. Despite this, the language barrier (there were differences in the levels of English proficiency among participants) meant that sometimes responses were unclear or confusing, and that respondents had difficulty understanding or answering questions. 3. Did the consultation reach the targeted audience? Both e-consultations were preceded by a promotional effort. Task Force members sent invitations to hundreds of youth HIV/AIDS leaders and organizations worldwide using existing online networks and listservs. Those who received the invitation were requested to forward it to those in peer and professional circles, in order to achieve as high a response rate as possible. In the second e-consultation, all coalition members and Toronto YouthForce conference attendees were invited. The participation numbers for the two e-consultations make for an interesting contrast. The first e-consultation had a higher number of respondents (365 compared to 218 in the second e-consultation). The second e-consultation might have had fewer participants because invitations were sent only to those youth who were attending the International AIDS Conference. Youth might also have ignored the invitations for the second e-consultation. The GYCA members who unsubscribe from the listserv cite a large number of emails as the main reason they unsubscribe. However, the participation rate for the second e-consultation falls within GYCA members' normal response rate to surveys (ten percent of total membership, as cited by the GYCA staff). It must also be noted that certain audiences were not able to participate in the e-consultations (i.e., those populations who are illiterate, have no access to the internet, and/or do not speak English). Although the weekly summaries were translated into Spanish and French, it took some time for the translations to be completed and posted online. 4. Was the information provided appropriate? The invitation letter that was sent before the commencement of the e-consultations contained basic facts about the intensity of the youth HIV/AIDS issue. This letter did not provide details about specific issues, nor did it define technical jargon for the potential audience. This can partially be explained by the fact that the letter was sent to listservs for youth HIV/AIDS workers and organizations, and organizers assumed that the listserv members would be familiar with key jargon and important issues. 5. Were the contributions informed and appropriate? Outlining the purpose, objective and expectations of the e-consultation resulted in a majority of participants' responses being crafted comprehensively and in great detail. The participants' responses to questions revealed that youth HIV/AIDS cannot be viewed as a health issue alone, but must be addressed in the larger context of root causes such as poverty, unemployment, illiteracy, and gender inequality. Occasionally, respondents provided responses to questions that had been asked in a previous week, or they appeared to misunderstand the questions. The participants' responses varied in nature from being opinionated to being informative. Some respondents posed their own questions and several others made recommendations. 6. Was feedback provided both during and after the consultation? Moderators addressed specific technical problems during the course of the e-consultation and shared the weekly summaries with participants. In addition to posting summaries, moderators highlighted individual responses that discussed important, cross-cutting issues, or that suggested innovative approaches that the coalition hoped to implement. If participants posted information that was incorrect, the moderators would post the correct statistics and citations to avoid confusion. 7. Was there an impact on policy content? Both of the e-consultations successfully met their objectives. Specifically, participants' responses during the first e-consultation had a direct impact on the coalition's organizational policy:
Similarly, participants in the second e-consultation informed the youth agenda and contributed to setting the advocacy strategy by:
Lessons Learned & Recommendations for PractitionersOur evaluation results yielded information that may be useful to others interested in developing and using e-consultations. First and foremost, organizations aiming to conduct an e-consultation will benefit from developing a clear strategy (containing consultative goals, thematic areas to be discussed, a timeline, and an idea of the range of participants they wish to attract). Furthermore, it is useful for this strategy to be communicated with participants. When communicating this strategy, the organizers might find it important to be aware of how much expertise and experience the potential participants have in the e-consultation topic As with other health education initiatives, the e-consultation organizers will probably find it useful to determine the evaluative framework to be used prior to implementation. This will ensure that the organizers collect the appropriate data throughout the course of the e-consultation. It is clear that e-consultations will greatly benefit from a robust evaluative mechanism. However, we argue that despite its comprehensiveness, the Macintosh & Whyte framework may limit some comparisons across e-consultations. The assessment of the utility and findings from e-consultations would benefit from a broad range of data collection and analytical techniques, including qualitative and quantitative assessments. Future efforts at refining these methods will enhance our ability to determine the effectiveness of e-consultations. E-consultations with global participation face a major challenge in terms of being inclusive of key stakeholders in resource-constrained areas (with no or little access to technology). This is especially important since people in these areas may be at high risk for many health problems. To ensure that the voices of these stakeholders are represented, the organizers will want to develop ideas for including them in the e-consultation. A possible strategy to ensure inclusiveness is to increase the duration of the e-consultation. This will give representatives with access more time to confer with those stakeholders who have less access. Those with access may also require technical assistance to learn how to summarize others' opinions. Based on this case study, we also urge others in global health settings to design the e-consultation in more than one language, depending upon the demographics of the participants. By using English (or any single language) alone, organizers risk neglecting those audiences with low or no proficiency levels. When choosing potential participants (the target audiences), organizers will need to decide between smaller, more specific audiences, and a large, all-inclusive audience. Larger audiences will take larger promotional efforts but they also allow the organizers to potentially tap into ideas and insights that might otherwise have been left out. Lastly, organizers will want to consider what information they will want to collect from respondents (such as demographics). This study did not include information on the specific demographic distribution of respondents, thereby making it difficult to identify what region-specific issues came to the fore. It also makes it difficult to make any inferences about the audiences reached by these e-consultations. In the future, organizers of e-consultations will want to consider if they wish to and how to gather this information. Conclusions:From this case study we learn that ICT tools can be effectively used to govern and mobilize social change to improve public health problems, especially those that are affected by multi-dimensional issues like gender, economics, race, and education. It is also important to understand from this study that in order to maximize impact, ICT tools must include the populations being affected in critical decisions. In doing so, it is important to bear in mind the special challenges represented by resource-constrained areas (such as limited internet access and language barriers) that might impinge on effective participation. In a realm such as HIV/AIDS, where civic engagement is so heavily emphasized, the notion of 'participation' is germane to an e-consultation, especially because the underlying principle of the whole process is inclusiveness. Thus, it is imperative to consider that while ICTs promise greater access to information and greater connectivity, their proliferation creates new social and professional imbalances that threaten to further exclude some populations.25 Acknowledgements: We would like to thank Mila Gorokhovich from the Global Youth Coalition for HIV/AIDS (GYCA) for sharing documents and reports related to this study. We also express our appreciation to the hundreds of e-consultation participants whose contributions formed the very core of our analysis. References:
|
 |  |  |