In March 2019, Ushahidi, a Nairobi-based technology company, released a new version of its platform that makes it easy to share community-generated crisis data with Humanitarian Data Exchange (HDX). Users can also use a new module to add HXL hashtags to their data. We think this is a great first step towards engaging an active community in support of HDX’s mission and making this data discoverable by the humanitarian sector. 

Project background 

Ushahidi, which translates to ‘testimony’ in Swahili, was developed to map citizen reports in Kenya after the post-election violence in 2008. Since then, the Ushahidi platform has been used to crowdsource millions of reports from citizens across the world. These are people who may be caught up in the aftermath of hurricanes or earthquakes or may be witnessing violence, corruption or harassment. The Ushahidi platform enables them to be heard by those who can help. 

In 2018, Ushahidi received funding through the EU-funded COMRADES research project to explore how platforms for information sharing can improve community resilience during crisis. Together with our project partners, we undertook extensive research to develop web services that use machine learning and natural language processing to automate or otherwise increase the processing of community-generated data during crises. 

Meeting with community leaders in Nepal to better understand the data they generate and how it could be made more discoverable. (Photo credit: Shadrock Roberts)

As part of this effort, we created a feature to enable Ushahidi users to export data from their deployment onto the HDX platform. The feature also allows data to be tagged with Humanitarian Exchange Language (HXL) hashtags that make it easier to use the data in other systems and to take advantage of web-based tools such as Quick Charts and Data Check. 

This integration ensures that Ushahidi platform users can export standardized datasets to a recognized humanitarian data platform that contains data from over 200 organizations. This makes community-generated data easier to discover by a significant portion of the humanitarian community.

The Ushahidi to HDX workflow

A user of the Ushahidi platform can collect data in many ways including through surveys, Twitter, SMS, email, and an API. Once a Ushahidi platform user runs their campaign, they can take a number of actions with their data such as cleaning it, adding detail or exporting the data. 

To export the data directly to HDX, the user must take the following steps:

  1. Request to create an organization on HDX.
  2. Configure the HDX API so that data goes to the correct HDX organization account.
  3. Select ‘export and tag data’ to start the process of assigning HXL hashtags to the data that has been collected. 
  4. Export a CSV file or upload the file directly to HDX.

Once the data is on HDX, the HDX team will check the dataset for personally identifiable information and any risk of re-identification to the people who have been surveyed.  

The HXL tagging feature developed for Ushahidi as part of the COMRADES platform.

Key learnings

Throughout this process, we have learned the following lessons: 

  1. Find your users. We spent considerable time doing field research with a wide range of stakeholders, including affected communities and humanitarian responders. Those interviewed were not interested in new, stand-alone solutions but in technology that allows them to use tools they are already familiar with.
  2. Focus on integration. HDX and Ushahidi share an emphasis on platform management and data standards that allow for reliable and agnostic services. Integrating these platforms makes it possible for data to be accessed more broadly. 
  3. Design matters. What we assumed would be a relatively simple process ultimately required a significant amount of work to ensure ease of use. The design work created a better product for our users and gave us insight into aspects of Ushahidi and HDX that we did not initially consider. 

Accessing the Code 

The code base for the COMRADES project is contained in GitHub. There are two core modules: a PHP-based API system and a Node.js Web Client. The primary source code can be found here. The repository contains instructions for downloading and configuring the code which is free and open source software that can be redistributed or modified under the terms of the GNU Affero General Public License as published by the Free Software Foundation. 

We encourage open source contributions to this code-base and welcome any feedback or questions from those who use HDX and Ushahidi to improve the discoverability of community data. 

Project Partners

This post was written by Shadrock Roberts, Former Director of Resilience & Research Programs, and Eriol Fox, Design Lead, Ushahidi. 

guest blog HDX HXL ushahidi