Impact and evaluation in designing social innovation

The 2019 Internatonal Forum on Design to Solve Social Problems has obtained written consent from the speaker to publish the summarized and edited content


SPEAKER: Yoko Akama (Associate Professor, School of Design, RMIT University, Australia)



What is the most important agenda in social innovation design?
*



Designing makes the invisible, visible

This island metaphor (image) is useful to think about designing. This metaphor helps us to pay attention to both the visible things at the surface, like design products, materials, methods and technologies, and also the invisible things under the water, like values, behaviours, mindsets and worldviews. Designing is a way to make what is invisible under the water, visible. 
For designing that pursues social outcomes, it is very important to pay attention to people’s values, behaviours, mindsets and worldviews under the water, and undertake designing that materalises what people regard as valuable outcomes for their social well-being. When designing social innovation, this also means listening to local communities and residents, and understanding what matters to them that may be invisible, under the water, and collaborating with them to materialize those values as outcome of designing. There are various tools, methods and approaches in design that are used to achieve those social outcomes together. 



Traditional evaluation vs designing social innovation

Our network, ‘Designing Social Innovation in Asia-Pacific’ (DESIAP – www.desiap.org), ran an event in Kuala Lumpur in 2017 and gathered together change-makers from Cambodia, Malaysia, Thailand, Philippines, and Myanmar to understand how to evaluate social innovation. Our report can be downloaded here (https://desiap.org/insights/impact-evaluation-in-designing-social-innovation-report/). Our research discovered that alternative models of impact is needed to enable communities to be central in deciding how, why and what is important, so evaluative thinking is built into the project from the very beginning. We discovered that evaluating social innovation needs to be community-led by co-defining aims, criteria and outcomes together. The collaborative process of designing is emergent, rather than pre-set from the start. Evaluation happens iteratively, through learning by doing. This approach values lay, insider knowledges of communities, including their experiences, grounded in their local cultures. The process is adaptive and dynamic, shifting in real-time, according to context. The focus and outcome can benefit all, based on learning, reflection and transformation on a long-term basis. 
In contrast, traditional evaluation models are often determined by an external commissioning body that sets what and how to measure before projects starts, without involving the community. This means project objectives and successes are also pre-defined by the external body, including what issues to address. This top-down approach can have negative power-imbalance, which hinders ways to understand meaningful impacts and outcomes on a community level. Traditional methods of evaluation tends to be linear and quantitative, often based on western knowledges to ensure objectivity, independence and accountability. In addition, long-term impacts cannot be evaluated because the evaluation is carried out after the project is completed. In all, traditional models of evaluating designing social innovation is poorly suited, so DESIAP has created an alternative model, based on the inputs by various change-makers and literatures in Designing Social Innovation. The following are some examples.




Features of effective evaluation in Designing Social Innovation



- Building trust, participatory collaboration, grounded in place, culture and locality 
The most important aspect of designing social innovation is to build trust through enabling people’s participation in the ongoing work. The Sovereign Weaving Treaty is an initiative led by Wiradjuri Nation, an Indigenous community in Australia that are self-determining, renewing culture and building stronger relationships with one another, despite Australia’s colonial history and ongoing negative impact. Gatherings were organized to enable people to connect and learn through weaving, to strengthen cultural practice. Building trust and relationships might mean spending a lot of time together, having cups of tea, walking, visiting and eating meals together. In this work, it is important to follow guidance of Elders, community leaders and learn about cultural protocols, to build mutual respect and reciprocity. 


- Building evidence and capacity 
Point B Design+Training in Mynamar has been developing curriculum and capacity building with teachers. Hten Yin Tun is a human-centred designer that works with Ministry of Social Welfare to embed human-centred values of working with children protection cases. Working with communities in Myanmar, they ask people to reflect on what they noticed as most significant changes to build ground-up evidence. These could be success stories, challenges, problems they overcome, what makes them think, how situation was influenced, behaviour changes, insights, belief systems, perspectives. Htet also uses various design tools, methods and approaches to help people reflect on what influenced their mindset. In the process of asking these questions, they were able to recognize both large and small influences. Listening and sharing their findings with community can also empower the community and help them learn from the collaborative work they are doing together.



Point B Design + Training (Myanmar)

- Adapt process and outcomes 
Kosmesa Khiev at INSTEDD iLab Asia in Cambodia uses design and open-source technology to enhance collaboration and flow of information to better deliver critical services to vulnerable populations. For example, they use mobile technologies with communities for real-time reporting of Malaria. They co-design their processes to consider the capacity and limitations (such as time and skills) of those communities to ensure that feedback and evaluation can happen easily, continuously, dynamically, according to community’s timing and info they need.


InSTEDD iLab Southeast Asia (Cambodia)

- Pilot proposal with stakeholders 
Klaus Oberbauer at Impact Hub Yangon (Myanmar) works on implementing learning and scaling systems for Yangon social enterprise ecosystem. Their design-driven approach is to create spaces for individuals to connect and support each other, through hosting learning environments, and connecting organisations to local and international opportunities that help them grow personally and professionally. This also means Impact Hub acts like an informed cultural broker, bringing together doners and funders, intermediaries, implementers and communities together to shape a project, to build relationships, understandings, including the limitations and opportunities to ensure outcomes and processes are co-designed. They co-designed and implemented a data collection tool and a sensemaking approach to help community members become co-evaluators and co-researchers. This stands out as unique in social innovation space, because it is more common for doners or funders that do not meet communities directly, leading to top-down models of pre-defining deliverables. 


Impact Hub Yangon (Myanmar)

- Synthesis multiple insights 
In order to converge diverse insights, it is necessary to engage with a variety of stakeholders who might find it difficult to participate, including women, children, the elderly and ethnic minorities, and choose methods that are easily accessible to them. Dr. Zeeda Mohamad from Water-warriors Malaysia leads activism, education and social movements on water conservation issues and sustainability. She worked with stakeholders in a water resource conservation movement for the restoration and rehabilitation of water quality on campus. They use social media to crowd source and evaluate what they are doing over time, for example, posting pictures of waterways on facebook engages the community dynamically to listening to different perspectives, enabling them to monitor the water quality of lakes and rivers. Waterways are often affected by pollution, floods, algae blooms so engaging with community helps to prioritise issues and work out solutions together. This can also help form sub-groups of special interests, like volunteer ‘lake inspectors’, that can sustain projects longer-term.


Water-worriers social media post (Malaysia)

- Share learning: take time to record and celebrate effort, progress and achievements
Dr. Emma Rhule at Tandemic (Malaysia) design products, services, and interventions that address business and social challenges in new ways. They partner with organisations to build innovation capabilities internally and do this in a way that considers the bigger picture – systems, incentives, business models – so innovation translates into action and impact. A key capacity building they do is share insights with others from their own work, processes and learning. For example, they look at ways to make data accessible and manageable, such as time management or interview processes. Emma described how staff, within her own team or partners, underestimate how much time certain tasks can take. They devised a method of aggregating this data so it allows project planning and management to be contextually relevant to people and organisations in how long things take. Another example is sharing their approaches to interviews, again in terms of aggregating data, like how long interviews take, with whom, sample size and demographic spreads etc, that help others plan and manage their work more effectively.


- Test ideas frequently and learn from mistakes 
Simon Baldwin from Second Muse, Indonesia was involved with the process of designing and delivering emergency solar-powered light for disasters for it to be a cheap, lightweight, transportable, and renewable light source. Prototyping this product helped to test the ideas with end-users. An unintended consequence of prototyping was also developing a business model based on buy-one, give-one approach. This model means one light sold enabled the company to donate another to the community, resulting in further helping those in most need.

- Build a team to explore unknowns 
We need to admit that we don't know everything. External stakeholders also don’t have time, especially in building trust, relationships and understanding of communities on complex issues like migration and disadvantage. Professor Fadzilah Majid Cook, National University Malaysia, trains community members to do interviews instead of external researchers, so trust, relationship and grass-roots understanding is already guaranteed. The community members benefit being provided the skills and mentoring. This ensures the process and outcomes are practical in ways that can continue or be passed on.


Malaysian National University Community Research


- Create ways to make collective decisions 
Methods for making collective decisions require recognizing, disclosing and understanding individual perspectives, including differences in power. CROSSs works in rural areas of Thailand where they use a participatory approach and co-create ideas with the residents to develop proposals to the government in the town of Chumsaeng. One idea they proposed was a seating arrangement during meetings, which the community adopted since its introduction by CROSSs. Participants sit in a circle so that everyone sees eye to eye with one another. This can help diminish power-differences in social hierarchy. This way of communicating is now standard during meetings for this community. 


CROSSs, Thailand


- Develop shared understanding
In order to form and share a common understanding, it is necessary to pay attention to assumptions based on culture, context, role and life experiences. We need to be aware of our own and others' perspectives and belief systems, and always pay attention to what is not said and what is not heard. It is also necessary to take the time to understand and define what constitutes success, and to understand what it will look like for the community. Dr. Boonanan Natakun from Thammasat University (Thailand) uses co-design and participation to build community resilience for disaster. He uses various objects, maps and visualisations in a fun, accessible, learning action approach to identify and share resources used in people’s every day on a community map. This ensured high levels of participation and accommodate various languages ​used. This mapping was made possible because residents, not just urban planning experts, participated directly in the activities. This allows people to express what they see as important, enabling different living cultures and lifestyles contributing to community resilience. 


Mapping disaster preparedness by communities in Thailand





DESIAP: Designing social innovation in the Asia-Pacific

- DESIAP: an asset-based approach to evaluating and understanding designing social innovation
Through examples of designing social innovation in the Asia-Pacific, DESIAP is a platform to help to make visible the diversity of change-making work taking place in this region. We assist practitioners to understand what designing social innovation is, and to recognise how they are designing in self-taught and intuitive ways with the communities they work with. This approach is politically very important, so we do not ignore, judge or displace grass-roots practices by our expert, external, western frameworks of design. Many change-makers do not self-identify as designers by not being formally trained in design or recognise what they do as designing social innovation. Again, this echoes the problems with traditional methods of evaluation because it is top-down, external and expert-led, rather than listening and being led by communities. We hope our collective work helps to build greater understanding of designing social innovation in the Asia-Pacific and evaluate the successful impact by change-makers and communities in this region.


#Social Innovation Design Evaluation #Core Function #Measurement Standard #Indicator
 
Category related contents
Hashtag related contents