Strengthening Independent Media in Bosnia and Herzegovina: Performance Evaluation of USAID Strengthening Independent Media (SIM) Activity

This performance evaluation examined the results of the 5-year media assistance programme Strengthening Independent Media (SIM) in Bosnia-Herzegovina (BiH) and the effects of the programme on BiH's overall media landscape. Launched in October 2010, SIM in BiH was financed by the United States Agency for International Development (USAID) and implemented by Internews, with partners Annenberg School of Communications at the University of Pennsylvania and the Media Center Sarajevo (MCS). SIM's purpose was to create a professional media sector capable of monitoring and defending itself, producing quality outputs that contribute to national cohesion and inter-ethnic trust, and putting the needs of its audience before its political and financial benefactors. SIM was originally designed as a comprehensive media assistance programme to support traditional and online media outlets, various journalists' associations in BiH, the Communications Regulatory Agency (CRA), the state regulator for broadcast media, and the Press Council (PC), the BiH self-regulating body for print media. Two years into the project's implementation, Internews and its partners recalibrated its media development strategy for BiH and streamlined its tasks into the following components: (i) support the quality and growth of online media outlets, technologies, and sources; (ii) build the quality of investigative reporting and resources; and (iii) develop local capacity expertise and practice in media policy, media law, and media literacy.

SIM's monitoring and evaluation (M&E) system employed the International Research and Exchanges Board's (IREX) Media Sustainability Index (MSI) as the primary indicator to measure the project's overall success, along with other outcome, output, and input indicators. Given the wide scope of tasks undertaken by SIM, the MEASURE-BiH evaluation team used a combination of quantitative and qualitative research methods. Between September 9 and October 25 2015, the evaluation team met with 64 representatives of the implementing partners, donor, international organisations, such as the European Union (EU) and the Organization for Security and Cooperation in Europe (OSCE), media regulatory and self-regulatory bodies, and 24 for-profit, non-profit, and individual grantees, and 2 non-beneficiary online media outlets. The Evaluation Methods and Limitations Section of the report explains details of the evaluation methodology.

General conclusions related to SIM design and programme adjustments:

• SIM's programme description was broad and overly ambitious, given the resources available. In addition, the political context within which SIM operated was worse than expected. Consequently, SIM's contribution to the high-level targeted results for the overall media sector was limited. However, in the more narrow sense, SIM's assistance increased the scope and quality of reporting of SIM beneficiaries (in particular, for human rights reporting), improved the general level of expression of pluralism of thoughts and ideas, and improved technical innovations used by beneficiaries, including introducing and improving online presence and data journalism.
• The SIM programme description recognised the political challenge to the CRA's independence and sustainability, and planned for diplomatic action in concert with the programmatic activities. However, the political environment at the start of implementation was more challenging than USAID anticipated.
• The original programme description encouraged dispersion of funds, which did not improve the quality of reporting, especially investigative reporting (IR). The decision to better target SIM support in the second phase resulted in better quality of reporting of a narrower group of beneficiaries and more societal influence, such as political debates about and criminal investigations into the corruption affairs about which Zurnal reported, or increased support to victims of domestic violence inspired by Studio N from Livno.
• USAID did not design business development support for media managers and owners as a comprehensive task or priority in the project design stage, but kept it within the realm of online media reporting skills and technical advancements for journalists and students, in which SIM assistance was provided to beneficiaries. Some of the stakeholders perceived that the sustainability of the SIM-supported contents and topics, or even of media outlets themselves, is insecure without SIM support.

General conclusions related to SIM's local partnership with MCS:

• Most beneficiaries appreciated MCS's training and workshops on innovations and standards in the online media sphere, because they offered practical skills aligned with the journalists' needs and trends of the changing media environment. Trainees reported increased use of innovations, such as website integration with social media, or the use of infographics and data journalism. MCS also built an active network of media professionals and trainers.
• MCS established "vivid cooperation" with most of the journalism departments at BiH universities and with relevant news portals in BiH. The internship programme in online media newsrooms for journalism students was successful in enhancing their skills in new technologies and multimedia production. Despite positive results, this programme's continuation beyond SIM is uncertain, mostly due to financial constraints.
• SIM's partnership with the MCS helped to institute the MC Online platform, "a visually attractive, dynamic, and relevant educational resource serving the needs of the growing online journalistic community and the wider community of citizens."
• Despite technological problems, Infobiro, created by MCS, developed into "a useful online tool for journalists and researchers", although it is still limited to press and does not archive all relevant publications from BiH.
• MCS's mentoring process with IR grantees was only partly successful, as most grantees were not inclined to use the assistance and guidance from prominent experts tied to MCS, while mentees' lack of interest was cited as the reason in the SIM reports and interviews, despite SIM mandated mentoring for grantees at a certain stage.

With regard to effectiveness in achieving results, the evaluators found that:

• SIM grants provided an opportunity for media and journalists to discuss and express a diversity of thoughts and ideas; the grants also influenced the scope and quality of reporting, especially on human rights.
• SIM's support to print and broadcast media to improve their online presences was timely and successful in the short term.
• The online media outlets that received SIM's continuous support progressed with respect to the quality of their reporting and their technical innovations, and managed to secure loyal audiences. SIM strengthened the position of online media outside of Sarajevo, as well as of thematic or niche portals of civil society organisations (CSOs).
• A notable part of SIM human rights reporting grants to media led to journalists' increased coverage of human rights topics online through forms such as data journalism.
• SIM's support to media to foster content development and new audience engagement tools resulted in the creation of a quality relationship between supported online media and their audiences. According to the online survey, audiences believe that online media are less prone to self-censorship than traditional media but equally politically influenced.
• For outlets that received SIM's support continuously, SIM's focused support, internal mentoring, and introduction of new, multimedia forms of reporting (e.g., videos online) increased the quality of reporting. Investigative reports supported through SIM raised awareness among the public and stakeholders, and put certain corruption affairs on the public agenda.
• Although portals that published SIM investigative stories did not reach as many online visitors as did other news portals, these stories were influential in steering the public interest with respect to issues such as corruption affairs reported by Zurnal. However, the investigative approach, including the lack of investigative tools and inadequate research techniques, requires further advancement, as well as broadcasting or publishing channels for such stories.
• SIM contributed to strengthening the local experts' knowledge of media policies, media law, and media literacy, although the programme did not sufficiently harness the capacities of regulatory and self-regulatory bodies and state institutions in these areas.
• Policy advocacy culture and practice in BiH did not improve significantly as a result of the programme, judging by the outreachof policy papers produced by the media policy fellows and the engagement of relevant institutions in policy advocacy. The media policy fellowship programme was successful in increasing the fellows' individual capacities to conduct policy research, but their policy papers had no significant effect on government policies to regulate the media and internet spheres.
• Media law activities were among the most successful and effective within SIM, as they engaged students and created a pool of young experts ready to pursue media law as a professional orientation. SIM published and widely distributed the Media Law Primer, which most of the interviewees found useful.
• Most stakeholders indicated that SIM sparked interest in media literacy in journalism and communications studies and beyond these programmes. Media literacy courses were introduced into the curriculum of at least 4 faculties. BiH now has at least 2 new resources on media literacy in local language, including 2 books published within SIM. The project equipped a pool of media literacy trainers with skills to spread knowledge on media literacy, and some of them continued to publish academic work on media literacy.
• The media watchdog platform Analiziraj.ba was considered an important tool for media professionalism and media literacy, though limited time meant that SIM did not develop it entirely.
• Recipients deemed grants to the PC, Association BH Journalists (BHJ), and the Private Electronic Media Association (PEM) successful and sustainable.

With regard to the efficieny/effectiveness of programme and management processes, in short, the evaluators found that SIM managed the small grants mechanism efficiently, but new contents and new approaches that the supported media outlets introduced as a result of SIM assistance are not necessarily sustainable. SIM did not include continuous expertise related to information and communication technology (ICT). Participants and partners found that capacity building events and particularly the media law and literacy clinics were effective. Yet the media fellowship programme lacked outreach and broader geographic coverage (12 out of 15 policy fellows were from Sarajevo region at the time). Only 9 out of the 15 fellows completed their policy papers, and only 6 papers were satisfactory enough to be published online (not counting SIM staff). SIM and policy fellows did not sufficiently promote policy papers or advocate the policy solutions the papers recommended. Most interviewees assessed SIM staff as supportive and knowledgeable, saying they responded timely to requests and inquiries and established good cooperation with most media outlets and universities.

Most of the grantees felt that SIM responded to their needs to build relations with their traditional and online audiences and to dedicate time to stories they otherwise would not. However, a portion of stakeholders and beneficiaries thought that the SIM approach would be tailored to the needs and capacities of traditional media or that SIM would focus on the media outlets USAID supported in earlier interventions. Universities saw SIM as a source of novel approaches in teaching media law and a new concept of media literacy. Fellows saw policy development as a necessary skill generally missing in BiH's journalism, communications, or law studies. Stakeholders had varying understandings of the concept of independent media.

Based on the findings, the evaluators make several recommendations:

• USAID should consider providing additional assistance in the media sector to address remaining needs and also to build on SIM's initial influence with respect to the following: (i) improve media outlets' relations with online audiences; (ii) cultivate the public's interest in important issues by supporting much-needed high-quality investigative journalism; (iii) to improve quality reporting for outlets supported continuously through SIM (including through mentoring and new multi-media reporting forms); and (iv) to educate and promote interest in media literacy and media law.
• Future interventions in media sector should use a more focused approach to set expected results, in line with the financial resources available and with the realities of the political environment. Future interventions also should be more focused at the beneficiary level.
• Most interviewees affected by its work saw the CRA's independence and functionality as extremely important. Thus, USAID should consider options for working with the CRA and other relevant public institutions.
• Future media interventions should work more extensively in the area of freedom of the press or media, given the country's and the region's political dynamics. Interviewees cited police raids in and cyber-attacks on newsrooms, threats and attacks against journalists, application of law in defamation cases, and political parties' influence in the Public Broadcasting System (PBS), and other sources mentioned online targeting of female journalists and war propaganda as threatening freedom of the media and their sustainability.
• Future policy and programmatic interventions in media development should include some of the following relevant areas: content development, legal support for journalists, transparency of ownership (in particular for online media), the online media registry, regulation of state aid to media outlets and regulation of advertising (in particular, through cable TV-packaged channels, as well as the lack of verifiable data on viewership in the electronic media market), and support to rigorous quality analysis of reporting, especially online.
• Future interventions should ensure that the criteria for technical and financial assistance clearly reflect the activity's objectives (e.g., in relation to transparency of ownership, regulation of hate speech, gender-sensitive language, etc.) and that both selection criteria and selection decisions are clearly communicated to all applicants.
• Future media assistance should be coherently structured in a logframe or other planning tool and should include clear political assumptions and high-quality indicators to measure performance, as well as a detailed analysis of political risks. Indicators to measure the reach or impact of journalistic pieces that beneficiaries produce or websites they support should be based on tools that measure the specific population reached relative to the country's population, instead of number of visits or unique visitors.
• Financial independence and sustainability are the key preconditions of unbiased media reporting. Grants should be complemented by assistance providing comprehensive high-quality capacity building of for-profit and non-profit media, including owners, editors, and journalists, with respect to business models, new technologies, interactive contents, reporting trends, and overall management and financial skills.
• Part of the training and mentoring within future interventions should focus on "old school" professional media standards, such as accuracy, fairness, transparency, and public interest, in particular in IR.
• Future assistance should provide support to initiatives that combine academic and civil society expertise on topics such as human rights, anti-corruption, or disaster relief with professional reporting of media outlets that have significant access to online and offline audiences.
• Future policy fellowships should be conceptualised in a more strategic way to ensure that fellows are committed to the project and that they produce higher-quality policy papers and recommendations that are more relevant to society and contribute to the work of the main supporting institutions and associations.
• Future interventions should incorporate sustainability mechanisms into the design where possible.
• Media literacy efforts could in the future be extended beyond universities and especially beyond faculties of journalism and communications, since other universities and departments expressed interest and need for this area of knowledge to expand.