/http%3A%2F%2Ffbcdn-profile-a.akamaihd.net%2Fhprofile-ak-xaf1%2Fv%2Ft1.0-1%2F10366051_1435171516735945_2961300913844171161_n.jpg%3Foh%3Df9f9a3dbdd26aad3c858c65d40b00b84%26oe%3D565E71E4%26__gda__%3D1452975867_f2874bd897142a9506849306de8c0776)
Paromita Pain Ph.D. Student
MA Specialized Journalism, Annenberg School for Communication and Journalism, University of Southern California (August 2011-August 2012)
Revisiting
CGNet Swara and
its
Impact in Rural India
1st Author
A
ffiliation
E
-
2nd Author
A
ffiliation
E
-
3rd Author
A
ffiliation
E
-
ABSTRACT
CGNet Swara is a voice
-
based platform for citizen journalism,
launched in rural India in 2010
.
Since then, CGNet Swara has
logged over
575
,000 phone calls, over
6
,
9
00 published stories, and
287
reports of specific problems that were solved via the system
.
In
this paper, we characterize the ongoing impact of CGNet Swara
using a mixed
-
methods approach that includes
70
interviews with
contributors, listeners, moderators
, journalists, officials, and other
actors
.
Our analysis also draws on the content of published posts,
two
focus group
s
, and a 9
-
day field immersion
.
Our results
highlight personal narratives of the transformative benefits CGNet
Swara has brought to rural
communities
.
While the resolution of
grievances is the most visible impact, we also uncover a diverse
portfolio of other impacts connected to contributing and listening
to the
platform
, as well as opportunities to further enhance impact
.
Our work contribut
es to the dialogue surrounding the impact of
ICTD projects, especially those
that span
multiple years.
Categories and Subject Descriptors
K.4
.0
[Computers and Society]
:
General
Keywords
Interactive voice response; IVR; CGNet Swara; impact; India
1.
INTRODUCTI
ON
As several projects in ICTD are graduating from research pilots to
scalable interventions
[7,13,19,31,32]
, it is increasingly importa
nt
to understand the opportunities and challenges
in
achieving
meaningful impact at scale
.
While the definition of “impact” in
ICTD is a matter of debate
[15,16,20,21]
, researchers typically
share a deep motivation for bringing benefits to low
-
income
stakeholders, and
there are various quantitative
[10,14]
and
qualitative
[9,10,16,21,26]
tools for evaluating such impacts.
This
paper advances our
understanding
of impact in ICTD by examining
the evolution of a project from a research prototype to a real
-
world
system encompassing tens of thousands of users.
The focus of our inquiry is
CGNet
Swara: a voice portal for citizen
journalism in rural India
.
Using low
-
en
d mobile phone
s
, users can
call
CGNet
Swara to report stories of local interest and to listen to
stories that others have recorded
.
Submitted stories are reviewed by
a team of moderators, and approved stories are made available for
listening over the web a
s well as the phone
. A follow
-
up team
appeals to government officials to act on any
problems reported,
resulting in concret
e changes in rural communities.
The story of
CGNet
Swara starts with its launch in 201
0
.
Some of
the paper authors were involved from the beginning as creators and
champions of the system, while others came later for observation,
analysis and critique
.
After
1.5 years
in the pilot stage, the emergent
behaviors of
CGNet
Swara users were char
acterized and published
(in ICTD 2012
[24]
)
.
Now,
3.5 years
after this initial inquiry,
CGNet
Swara has grown to encompass a total of
63
,
200
callers
who
have recorded over
6
,
9
00
stories and have called over
575
,000
tim
es to listen
.
Moreover, the platform has been credit
ed
with
considerable
impact, including
287
cases
(and counting)
where
users have
narrated
specific cases where
problems were solved as
a result of using
CGNet
Swara.
The contribution of this paper is a detailed characterization of
CGNet
Swara
’
s impact,
as evidenced by the personal narratives of
contributors, listeners, moderators, journalists, government
officials, and others
who have come in contact with the system
.
When we use the word “impact”, we are broadly referring to
any
change in the ecosystem that
was
reported to arise as a result of
using CGNet Swara
. We
characterize such
impact
via a
mixed
-
methods analysis
of
70 sem
i
-
structured interviews as well as field
observations, focus groups, and analysis of posts
1
.
Our analysis
provides
strong evidence that
CGNet
Swara has
resulted in
transformative
benefits
for
many of its users. While the
most visible impact is the resolution of grievances,
we
also
uncover
a broad portfolio of
other
impact
s.
Some kinds of impact are
connected to reporting on the system
–
for example, the benefit
associated with being heard, havi
ng an audience for artistic
expression, and promoting personal development
.
Other
s are
associated with listening
–
for example, allowing discovery of
previously
unvoiced issues, building awareness of local news, and
insp
i
ring confidence and agency that pos
itive changes are within
reach
.
We also discuss cases where impact is lacking or could be
improved, for example, by improving transparency of moderation
or by improving follow
-
up on compl
aints. We hope that our
account of CGNet Swara
can
inspire others to
pursue long
-
term
interventions that offer both depth and diversity of impact.
2.
RELATED WORK
2.1
Impact in ICTD
The question of impact, and how to measure it, has been the subject
of ongoing discussion in ICTD (e.g.
,
[9,10,11,15,16,20,21]
). One
of the reasons for this is that impact is complex to understand and
measure, going beyond simple statistics of uptake
and usage. Heeks
and Molla
[20]
describe three sub
-
elements to impact assessment
(outputs, outcomes, and development impacts), each of which is
more complicated to measure than the previous. Looking at their
compendium of different impact studies it quickly becomes
apparent that impact
cannot be measured on a single scale
–
and
1
I
t is important to note that we do not aim to document the exact
chain of causal events that led
to impact. For example, with
respect to grievance redressal, the impact ultimately depends on
government actors who are influenced by several different
forces, most of which we cannot directly observe.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full
cit
ation on the first page. Copyrights for components of this work owned by others
than ACM must be honored. Abstracting with credit is permitted. To copy otherwise,
or republish, to post on servers or to redistribute to lists, requires prior specific
permiss
ion and/or a fee. Request permissions from Permissions@acm.org.
ICTD '15
, May 15
-
18, 2015, Singapore, Singapore
Copyright is held by the owner/author(s). Publication rights licensed to ACM.
ACM
978
-
1
-
4503
-
3163
-
0/15/05...$15.00
http://dx.doi.org/10.1145/2737856.2738026
Meghana Marathe
†
, Jacki O’Neill
†
, Paromita Pain
‡
, and William Thies
†
†
Microsoft Research India
‡
University of Texas at Austin
{t
-
meghma, Jacki.ONeill, thies}@microsoft.com
paromita.pain@gmail.com
indeed the results of any impact assessment very much depend on
through whose eyes one is judging the impact.
Perhaps the simplest, but by no means unproblematic, assessments
of impact focus on quantitative measu
res
–
counting numbers (of
devices, users, etc.) or economic impacts (e.g.
,
opportunities for
income generation)
[15]
. However, there has been a growing
awareness that a broader range of more social, less easily
quantifiable, impacts also need to be ta
ken into account, although
there is no simple, single method for how to measure such complex,
situated phenomena. Gomez & Pather
[15]
suggest we should take
lessons from business when assessing ICTs
–
considering them as
enablers of change and transfor
mational, rather than focusing only
on tangible and directly measurable benefits. However, they do not
provide
concrete
guidelines for how one might go about this.
From the development literature, Sen’s capabilities approach
[2
8]
calls for a radical rethink of the idea of development as being about
enabling human freedoms rather than wealth. In particular
,
he calls
for an understanding of what development means from the
perspective of the people involved. When applied to ICT
s this could
involve evaluating how ICTs contribute to freedom and
empowerment
[
20]
. Kleine attempts to operationalize the
capabilities approach through her Choice Framework, which puts
the focus on an
“
individual’s own development outcomes
”
[21]
.
While
both Sen and Kleine cover a much wider space than we are
concerned with here
,
their prioritization of the needs and concerns
of those affected by the project is of key relevance to us.
The importance of this botto
m
-
up approach is illustrated by
Parthasarathy and Srinivasan
[26]
. They present two convincing
examples of how a deep understanding of
a
system
’
s use, gathered
through ethnographic study, can result in radically different
assessments
than more formal, and less rich,
techniques
.
In this paper, we take up these calls from various members of the
ICTD and development communities to understand the impact, or
otherwise, of CGNet Swara from the perspectives of those touched
by the system.
While
we do present quantitative measures of upt
ake
and continued usage, we ‘look behind’ these measures with a
qualitative, primarily interview based study, which aims to
understand what impact means in the terms of those concerned with
or
connected to
the system. To do so we cast our net wide, talking
not just to various user groups (
such as
contributors, listeners and
CGNet Swara staff) but also to others
surrounding
the system (
such
as
government officials, mainstream journalists and former users)
to understand their perspective on the impact or lack
thereof.
Furthermore, we are responding to Heeks and Molla’s call
[20]
to
as
sess the impact of
longer
-
term
projects
, rather than just pilots, as
CGNet Swara has been operational since 2010. The resulting rich
picture of use (and non
-
use) reveals both what might be considered
the tangible, objective impact of the system and a set o
f less
tangible but equally important impacts
on the lives of its users.
2.2
Prior r
esearch
on CGNet Swara
This is not the first paper to consider the usage and even the impact
of CGNet Swara. Mudliar, Donner and Thies examined the initial
usage of CGNet Swara
[24]
. They analyzed posts from the first 20
months of
the project
, and conducted interviews (between the 11
th
and 13
th
months) with a range of people, including listeners,
contributors, mainstream journalists and gove
rnment officials, to
understand how they
“
perceived and used the system, perhaps in
ways that differed from the founders’ expectations
”
.
Mudliar et al
.
highlighted how grievance redressal was an
emergent
and
unexpected
category of activity
which seemed
,
even at
an
early
stage
,
to be the most impactful category of use.
Mudliar and Donner
also
reflected on
CGNet Swara
as a participatory medium
[25]
.
Our current study builds on Mudliar et al. and is distinguished by
(i) the significant expansion and evolution of CGNet Swara in the
3.5 years since the
prior study, and (ii) an explicit focus on
characterizing the impact of the service. When the prior interviews
were conducted, CGNet Swara had shown only modest impact. Its
structure for grievance resolution has since become more
formalized and specific, a
s described in the next section. For
example, the idiom of requesting callers to formally report resolved
grievances did not start until after Mudliar et al. had finished
interviews. Since then, the system has released
287
such reports,
and they serve as a
focal point of our analysis.
Chadha and Steiner
[6]
examined the impact of CGN
et Swara as a
citizen journalism site, by conducting 10 in
-
depth interviews with
regional and national journalists to uncover their opinions on
citizen journalism in general and CGNet Swara in specific. They
found that while journalists were initially posi
tive about citizen
journalism sites such as CGNet Swara, on deeper probing rather
different opinions were revealed. Mainstream journalists reported
not using CGNet Swara to help source stories for a variety of
reasons
,
including
(i
) reports were considered
to be too much about
activism, too partial (one sided) and not vetted for accuracy,
(ii
)
reports on marginal rural issues were not considered newsworthy
for their urban audiences, and
(iii
) there is endemic corruption in
mainstream media in India, includi
ng the partiality of media owners
and journalists and the common practice of paying for stories.
Overall they concluded that CGNet Swara did not succeed in its
aim of giving a voice to tribal communities, as stories were only
rarely taken up by mainstream
news agencies.
In comparison to Chadha and Steiner, our impact study takes
a
broader focus. Firstly we take into account the voices of a wider
group of people
, including
the rural communities for whom the
system has been set up. Secondly, unlike Chadha an
d Steiner, we
do not take an external, pre
-
constructed definition of impact; rather,
we let the various respondents define impact (or lack thereof) for
themselves. Chadha and Steiner’s definition of impact, i.e., to bring
about action through dissemination
of stories by mainstream media,
certainly has validity in that it was one of the stated aims of CGNet
Swara when it was conceived by the founder;
however, our study
reveals that this aim has evolved over time. While the goal of
bringing about action is as
important as ever, the organization relies
more on direct advocacy of officials than on mainstream journalists
as agents of change.
Our findings
challenge Chadha and Steiner’s
assertion that
“
only mainstream journalists can effectively
disseminate stories
and bring about action.
”
2.3
Voice
-
based systems
in developing regions
In addition to CGNet Swara, others have used voice
-
based
system
s
as an inclusive means of accessing, reporting, and sharing
information in rural communities
.
Recent
interactive
voice forum
s
have spanned various domains, including citizen journalism
[18,32]
, agriculture
[27,31]
, feedback on
school meals
[17]
, job
search
[30]
, rural information portals
[1]
, and forums for
immigrants in high
-
resource settings
[4]
.
Together, these forums
have attracted millions of calls and hundreds of thousands of
recordings
[1,31,32]
.
A recent evaluation showed that a voice
-
based agricultural information service offered benefits to farmers
[8]
.
Researchers have also explored how to increase participation
in community radio programs, using mobile phones
[22,23]
,
custom devices
[29]
or other means
[3]
.
When it comes to grievances redressal, perhaps the most common
solution is relatively low
-
tech: a
helpline with live operators.
For
example, the state of Madhya Pradesh launched the CM Helpline
in July 2014
to aid
in
grievance
redressal
[33]
. They report
/http%3A%2F%2Fi.vimeocdn.com%2Fvideo%2F448638542_1280.jpg)
Innovative Approaches in the Global News Ecosystem (Research Panel) Paromita Pain: Finding Their Voice: Examining Some Internet-Based Initiatives that Are Changing the Face of the Indian Media ...