Challenges, opportunities, and next steps to build the 21st century AI Powered NGO.
Artificial intelligence (AI) has the potential to help tackle some of the world’s most challenging problems. We are sure you have heard this before, it all sounds good in theory but how do we walk the talk?
To find answers to these questions we organized a 1-hour webinar and panel discussion with some of the leading experts and organizations in the Humanitarian AI space.
What all panelists agreed is one is that despite a lot of attention toward fancy and futuristic AI, there are hundreds of problems that can be addressed with current applications. AI has been around since the 1940s and can come with very pragmatic approaches to address real-world problems and generate value. Another big misconception is that organizations think they need to have a perfectly defined problem and dataset before they can leverage the potential of AI.
The reality is there are no perfect problems and also no perfect data, so better let us start creating value right now by shifting our mind towards pragmatic and collaborative practices.
In this demo day, we took a close look at the tremendous potential AI offers for making communities safer, by helping to reduce, prevent, and respond to crimes. When it comes to public safety, it is often critical to act quickly. AI technologies can supplement the work of people, taking on monotonous and time-consuming tasks that would be impossible for humans to do effectively. Natural language processing can read and analyze public communications and news reports to detect potential problem areas and get-ahead of violence. Of course, this work must be done responsibly and ethically.
Sharing her perspective on the impact that AI can have in keeping people safe was an expert in the field, ElsaMarie D’Silva, the Founder & CEO of the Red Dot Foundation. The Red Dot Foundation’s award-winning platform Safecity crowdsources personal experiences of sexual violence and abuse in public spaces. ElsaMarie is listed as one of BBC Hindi’s 100 Women, and her work has been recognized by numerous UN organizations and the SDG Action Festival.
To go a little deeper into the application of AI for public safety, we shared Omdena projects that took innovative approaches to make communities safer.
Case Study 1: Preventing sexual harassment through a safe-path finder algorithm
“UN Women states that 1 in 3 women face some kind of sexual assault at least once in their lifetime.”
With the first case study, the Omdena team drew upon Safecity’s crowdsourced data about sexual harassment in public spaces and leveraged open-source data to build heatmaps and calculate safe routes through major cities in India. Part of the solution is a sexual harassment category classifier with 93 percent accuracy and several models that predict places with a high risk of sexual harassment incidents to suggest safe routes.
You can learn more about this and related projects here:
Case Study 2: Understanding gang violence patterns and actors through Twitter analysis
Our team worked in partnership with Voice 4 Impact, an award-winning NGO whose solution to violence in our communities addresses the questions people worldwide are asking: “How do we keep missing the signs?”
The Omdena team made use of natural language processing techniques — AI techniques that analyze text to understand what is being communicated. Machine learning algorithms were used to understand gang language and AI models built to detect violent messages on Twitter, without profiling. The aim is to predict and ultimately prevent, gang violence.
You can learn more about this and related projects here:
Case Study 3: Analyzing Domestic Violence through Natural Language Processing (NLP)
Finally, we presented Omdena’s work to uncover domestic violence in India hidden due to COVID lockdowns. This work is part of a project with the award-winning Red Dot Foundation and Omdena’s collaborative platform to build solutions to better understand domestic violence and online harassment patterns during COVID-19. The project used natural language processing techniques with social media, government reports, and other text content to create a dataset with which Safecity could mobilize local efforts to protect and support domestic violence victims.
You can learn more about this and related projects here:
In this demo day, we talked about the inevitable data challenges/roadblocks that come up in real-world AI projects. The insights shared came from our experiences with more than 20 AI projects, working with partners including the UN Refugee Agency (UNHCR), the World Resources Institute, the World Energy Council, and numerous NGOs and corporations.
Omdena is a collaborative platform to build innovative, ethical, and efficient AI solutions to real-world problems. Since our founding in May 2019, over 1250 AI experts from more than 80 countries have come together on Omdena projects to address significant issues related to hunger, sexual harassment, land conflicts, gang violence, wildfire prevention, and energy poverty.
We’ve seen that the way that we approach AI development, via bottom-up collaboration with diverse team members, fosters innovation and creativity which leads to the breakdown of data roadblocks. Innovation is inherent in the Omdena process.
We shared three Omdena projects to act as case studies for these innovative approaches to tackling data challenges.
Data Roadblock 1: Incomplete Data Sets
In the real world, datasets are rarely complete. We find having large teams of dozens of people means that data gathering, cleaning, and wrangling happen at a phenomenal speed. And by taking a bottom-up approach, we have multiple sub-teams looking at data problems from different angles, allowing for innovative approaches to be explored.
In the following case study, the Omdena team worked out ways to identify safe routes in a city in the aftermath of an earthquake, where the relevant data sets were inconsistent and unreliable.
Case Study : Disaster Response: Improving the Aftermath Management of an Earthquake
In collaboration with Istanbul’s Impact Hub innovation center, Omdena data scientists combined satellite imagery of Istanbul with street map data in order to build a tool that facilitates family reunification by indicating the shortest and safest route between two points after an earthquake.
“Omdena´s approach to AI development is by far the best that I have seen in 2019” — Semih Boyaci, Co-Founder Impact Hub Istanbul
We don’t see the lack of data as a showstopper. On those projects without data, the team starts by asking what do we need to know to address the problem? Where might that data live? If it doesn’t exist, how can we create it from something that does exist? Here the diversity of the team members is very powerful.
We’ve seen time and again the impact of bringing together people with vastly different professional and life experiences. Our teams are typically 30% or more female. On any project, we’ll have on average 14 countries represented. Our collaborators range in age from 17 to 65. Not only does this diversity lead to ethical and trusted solutions, but it also fosters creativity and alternative ideas about what data is relevant and where to find it.
In the following project, we looked at how to assess post-traumatic stress disorder among those that have suffered trauma in low-resource environments. In this case, the team started with no data in-hand.
Case Study : Building a chatbot for Post-traumatic-stress-disorder (PTSD) assessment
32 Omdena collaborators developed a machine learning-driven chatbot for PTSD assessment in war and refugee zones.
Through the collaborative efforts of the project community, the team identified and annotated suitable patient data. The teams applied linear classifiers for Natural Language Processing (NLP) for PTSD risk assessment and transfer learning for data augmentation.
Relevant data doesn’t typically come packaged in just one form. We often need to meld disparate data sources to get at a solution. Through collaboration, sub-teams focused on separate data and AI techniques come together to integrate those efforts to derive insights about the problem.
In the following project, the goal was to uncover domestic violence in India hidden due to COVID lockdowns. Among the many challenges the team addressed was the integration of data culled from disparate sources.
Case Study : Analyzing Domestic Violence through Natural Language Processing
This project was done with the award-winning Red Dot Foundation. Within Omdena’s collaborative platform, the team looked craft a dataset to reveal domestic violence and online harassment patterns in India during COVID-19 lockdowns. The AI experts scrapped data from news articles as well as social media to apply various natural language processing (NLP) techniques such as topic modeling, document annotations, and stacking machine learning models.
You can learn more about this and related projects here:
Two Omdena teams with a total of 50 AI experts and data scientists from 25 countries collaborated with the World Energy Council and the Nigerian NGO RA365 in carrying out data-driven analyses and providing AI solutions to address the Global Transition to Clean Energy.
At a recent Omdena Demo Day, team members Amardeep Singh, Julia Wabant, and Simon Mackenzie shared the results and insights gained from these two projects.
The Topic: Energy Transition
One of the Sustainable Development Goals adopted by all United Nations Member States in 2015 aims to ensure access to affordable, reliable, sustainable, and modern clean energy for all by 2030.
Transitioning into a society with cleaner energy is crucial for fighting climate change. Different parts of the world are currently facing different stages of the energy transition. This can be noted both on the implementation of solutions in specific regions as well as in the cultural perception of such transition by societies. Both topics are addressed in the following two Omdena use cases.
1. Use Case: AI for Renewable Energy in Nigeria
Nigeria is one of the countries in the world facing the most severe energy challenges. Over half of the country’s population — 100 million people — lack access to electricity. Some of the problems faced by Nigerians include precarious electricity systems, unstable electricity supply, and electricity available only in certain locations.
An alternative to these problems is investing in local and renewable power solutions. Renewable Africa RA365 is an NGO with the mission to end energy poverty in Nigeria by leveraging innovative clean energy solutions and focusing on providing solar energy to vulnerable populations. In this project, the Omdena team partnered up with RA365 with the goal of identifying communities where solar panels would add the most value.
The first task in this challenge was to define what these areas should be: groups of about 4000 people living within a radius of about 500 m, and that are located more than 15 km away from a power grid. Regions close to schools, healthcare centers, and water locations were considered to have a higher ranking of priority, as they can benefit even more from renewable energy implementation.
One of the biggest challenges in the project was the lack of data for population density, making it hard to identify where people need assistance. In order to find out how the population is distributed in Nigeria and determine who is without access to electricity, the team compared nighttime satellite imagery from NASA Black Marble VIIRS against the geographic location of the population using the Demographic and Health Surveys (DHS) program, ground surveys from WorldPop, and the GRID3 dataset. Also, for identifying the national grid location and, therefore, find regions where people live in relation to existing power lines, the team applied Machine Learning techniques on satellite images from the HV grid from Development Seed/World Bank.
Finally, the team finally worked on finding, among all these towns without electricity supply, which ones would be suitable for the criteria established for the implementation of a local solar energy system. This was done by clustering 4000 people in a 500 m radius using the DbScan clustering technique, leading to the identification of over a thousand high-potential regions.
Clusters of towns with populations between 4–15 thousand people which are suitable for potential off-grid solar navigation in the North of Nigeria.
The Omdena’s team deliverable for this project: A prototype interactive map of the whole of Nigeria identifying the regions with a high demand for electricity and a high potential for solar.
The next steps for this project include a detailed survey for the top target areas in order to identify which locations are most suitable both in terms of infrastructure and cost for implementation of solar systems.
The initiative taken by Omdena and Renewable Africa RA365 has the potential of enabling data-driven investments and policy-making that can change the lives of many people in Nigeria and other African countries.
The data and prototype of this project have been shared with the Lagos State Government agency for solar systems, which is now willing to start the process of mass production already in 2020.
“In order to get this job done, it is not all about providing solutions to these people. We want to make sure that the solutions get to the right people at the right places, and Omdena has really helped us to achieve that.”
Joseph Itopa, Machine Learning Engineer at Renewable Africa RA365
2. Use Case: Sentiment Analysis on Energy Transition
The transition away from dependency on CO2 to a more sustainable society dominates the news headlines worldwide, exposing conflicting opinions and political measures driving towards a future with cleaner energy sources. Understanding the clean energy transition at a human-level is crucial to the effectiveness of whatever steps are taken in the direction of a carbon-free society.
Commissioned by the World Energy Council, the world’s leading member-based global energy network, Omdena explored applications of AI in understanding how people in different regions of the world perceive the energy transition and their role in it.
Using natural language processing (NLP) techniques, the team created tools to collect, scrape, and analyze text about the clean energy transition found on different social media sources (Twitter, YouTube, Facebook, Reddit, and famous newspapers). This text data was analyzed using varying methods, such as sentiment analysis, topic modeling, and clustering to reveal the challenges, reactions, and attitudes of citizens around the world.
Visualizations of the results allow for comparisons of sentiments across nations and societies. The analysis was first focused on English speaking countries, as this provides a common basis for comparing text. For this, the countries representative of different continents and development backgrounds were: USA (America), UK (Europe), Nigeria (Africa), and India (Asia).
The word cloud representation of the results shows that among the 4 countries investigated, only Nigeria has prominent tweets about “electricity supply”. Similarly, “gas prices” are specific to the USA. However, “renewable energy” is present in all 4 countries.
A part of the analysis was also expanded to other countries and languages, gathering and analyzing tweets related to complaints about “renewable energy cost” in more than 20 countries. The results revealed how local conditions and culture can differ significantly from different places. For example, “technology” was the most relevant concern in the complaint tweets in Brazil and France, whereas in Nigeria these tweets were focused solely on “policy”.
Other short and detailed discussions about this project can be found in Omdena publications.
Though broad conclusions cannot be drawn from these isolated collections of data, the results point to models and data sets that are promising for further development. The analysis carried out by the Omdena team allowed for a better understanding of how natural language processing techniques can be used to capture the opinions and concerns of people worldwide about the clean energy transition.
“The Council has been interested in how public sentiment on energy issues might be tracked, or if this were even possible. That is where this project came in — the team at Omdena explored the broad brief and have proven that the conceptual idea is possible.”
Martin Young, Senior Director at the World Energy Council
The demo day recording
Collaborators from this project
We thank our partner organizations, Renewable Africa 365 and the World Energy Council. as well as all Omdena collaborators (listed below) who made the project a success.
Omdenda team members, on the Renewable Energy Nigeria project:
Anastasis Stamatis, Greece
Daniil Khodosko, Canada
Peace Bakare, Nigeria
John Wu, Australia
Siddharth Srivastava, India
Simon Mackenzie, UK
Hoa Nguyen, Vietnam
Takashi Daido, Japan
Jessica Alecci, Netherlands/Italy
Jack David, UK
Shubham Bindal, India
Deborah David, France
Qi Han, Singapore
Stefan Hrouda-Rasmussen, Denmark
Varun G P, India
Ifeoma Okoh (Ify), Nigeria
Suraiya Khan, Canada
Ivan Tzompov, Bulgaria
Henrique Mendonca, Switzerland
Himadri Mishra, India
Sai Praveen, India
Jaikanth J, India
Krithiga Ramadass, India
Omdena team members, on the Energy Transition Social Sentiment project:
By Laura Clark Murray, Joanne Burke, and Rishika Rupam
A team of AI experts and data scientists from 12 countries on 4 continents worked collaboratively with the World Resources Institute (WRI) to support efforts to resolve land conflicts and prevent land degradation.
The Problem: Land conflicts get in the way of land restoration
Among its many initiatives, WRI, a global research organization, is leading the way on land restoration — restoring land that has lost its natural productivity and is considered degraded. According to WRI, land degradation reduces the productivity of land, threatening the economy and people’s livelihoods. This can lead to reduced availability of food, water, and energy, and contribute to climate change.
Restoration can return vitality to the land, making it safe for humans, wildlife, and plant communities. While significant restoration efforts are underway around the world, local conflicts get in the way. According to John Brandt of WRI, “Land conflict, especially conflict over land tenure, is a really large barrier to the work that we do around implementing a sustainable land use agenda. Without having clear tenure or ownership of land, long-term solutions, such as forest and landscape restoration, often are not economically viable.”
And though governments have instituted policies to deal with land conflicts, knowing where conflicts are underway and how each might be addressed is not a simple task. Says Brandt, “Getting data on where these land conflicts, land degradation, and land grabs occur is often very difficult because they tend to happen in remote areas with very strong language barriers and strong barriers around scale. Events occur in a very distributed manner.” WRI turned to Omdena to use AI and natural language processing techniques to tackle this problem.
The Project Goal: Identify news articles about land conflicts and match them to relevant government policies
“We’re very excited that the results from this partnership were very accurate and very useful to us.
We’re currently scaling up the results to develop sub-national indices of environmental conflict for both Brazil and Indonesia, as well as validating the results in India with data collected in the field by our partner organizations. This data can help supply chain professionals mitigate risk in regards to product-sourcing. The data can also help policymakers who are engaged in active management to think about what works and where those things work.” — John Brandt, World Resources Institute.
The Use Case: Land Conflicts in India
In India, the government has committed 26 million hectares of land for restoration by the year 2030. India is home to a population of 1.35 billion people, has 28 states, 22 languages, and more than 1000 dialects. In a land as vast and varied as India, gathering and collating information about land conflicts is a monumental task.
The team looked to news stories, with a collection of 65,000 articles from India for the years 2017–2018, extracted by WRI from GDELT, the Global Database of Events Language and Tone Project.
Identifying news articles about land conflicts
Land conflicts around land ownership include those between the government and the public, as well as personal conflicts between landowners. Other types of conflicts include those between humans and animals, such as humans invading habitats of tigers, leopards, or elephants, and environmental conflicts, such as floods, droughts, and cyclones.
The team used natural language processing (NLP) techniques to classify each news article in the 65,000 article collection as pertaining to land conflict or not. While this problem can be tackled without the use of any automation tools, it would take human beings years to go through each article and study it, whereas, with the right machine or deep learning model, it would take mere seconds.
A subset of 1,600 newspaper articles from the collection was hand-labeled as “positive” or “negative”, to act as an example of proper classification, or example of proper classification. For example, an article about a tiger attack would be hand-labeled as “positive”, while an article about local elections would be labeled as “negative”.
To prepare the remaining 63,400 articles for an AI pipeline, each article was pre-processed to remove stop words, such as “the” and “in”, and to lemmatize words to return them to their root form. Co-referencing pre-processing was used to increase accuracy. A topic modeling approach was used to further categorize the “positive” articles by the type of conflict, such as Land, Forest, Wildlife, Drought, Farming, Mining, Water. With refinement, the classification model achieved an accuracy of 97%.
With the subset of land conflict articles successfully identified, NLP models were built to identify four key components within each article: actors, quantities, events, and locations. To train the model, the team hand-labeled 147 articles with these components. Using an approach called Named Entity Recognition, the model processed the database of “positive” articles to flag these four components.
Matching land conflict articles to government policies
Numerous government policies exist to deal with land conflicts in India. The Policy Database was composed of 19 policy documents relevant to land conflicts in India, including policies such as the “Land Acquisition Act of 2013”, the “Indian Forest Act of 1927”, and the “Protection of Plant Varieties and Farmers’ Rights Act of 2001”.
A text similarity model was built to compare two text documents and determine how close they are in terms of context or meaning. The model made use of the “Cosine similarity” metric to measure the similarity of two documents irrespective of their size.
The Omdena team built a visual dashboard to display the land conflict events and the matching government policies. In this example, the tool displays geo-located land conflict events across five regions of India in 2017 and 2018.
Underlying this dashboard are the NLP models that classify news articles related to land conflict, and land degradation, and match them to the appropriate government policy.
The results of this pilot project have been used by the World Resources Institute to inform their next stage of development.
A team of 28 AI experts and data scientists collaborated to gauge the impact of pandemic policy implemented post-COVID-19 on vulnerable populations to find correlations and encourage data-driven policymaking to lessen the adversity for the most vulnerable populations around the world.
The entire data analysis including a live demonstration can be found in the demo day recording at the end of the article.
COVID-19 pandemic policy impacting the world’s vulnerable populations
At the onset of the pandemic in 2020, the World Health Organization urged governments to take “urgent and aggressive action” against COVID-19. Many governments reacted with strict measures such as closing borders and quarantining entire cities. Governments all over the world enacted these policies without fully analyzing the factors that impact their effectiveness. Nor did they consider how these policies might deepen the problems for vulnerable populations in different regions.
The project goal: Conduct data-driven impact-analyses on how various pandemic policies affect the well-being of vulnerable populations.
An important step of the project was to define “vulnerability” with respect to the particular context. The project focused on the factors of employment and wage loss, access to health, and domestic violence. To identify the vulnerable population for each of these categories, the team looked to the Inequality-adjusted Human Development Index, considered populations above 65 years of age, and women.
Assessing policies and their effects
The team looked at 17 types of policies from the Oxford COVID-19 Pandemic Government Response Tracker, across the categories of containment, economic response, and health systems. The policies explored included closing of public transportation, stay at home requirements, income support, COVID-19 testing policy, and emergency investment in healthcare.
To analyze the effects of these policies, three key aspects were considered:
Time of policy enactment: comparing the time of policy enactment with the effect on a target variable
Stringency metric: the degree of intensity of the policies enacted
Google Mobility Dataset: quantifies the movement of people in places (e.g. grocery stores vs. parks)
Domestic violence as a ‘Shadow pandemic’
It was ascertained that domestic violence is a growing shadow pandemic as countries displayed a relationship between a decrease in mobility and an increase in the google search rates of relevant topics, coupled with an increase in the number of domestic violence-related articles.
The number of news articles related to both Covid19 and domestic violence started to increase a couple of weeks after the first lockdown measures were implemented in Europe (end of February).
The data shows a strong relationship between a decrease in mobility and an increase in Google search rates of domestic abuse topics in many countries. In the countries considered, other than Japan, the peak in search rates has doubled or even tripled, as seen in these graphs of the data from France and India.
The results indicate that the problem of domestic violence could be much bigger than indicated by news stories.
Access to healthcare
The effects of COVID-19 pandemic-response policy measures on access to healthcare, specifically for non-COVID patients was a fascinating angle in this challenge.
The team sought to understand the effects of policy measures on access to healthcare, specifically for non-COVID patients. The vulnerable population was defined based on age, existing chronic medical conditions, and physical access to care facilities. The analysis was focused on England and Wales where there was significant relevant data.
It was found that there was high-mortality among patients with non-COVID chronic diseases during the pandemic as compared to the numbers for the same group in previous years. The data shows a correlation between medical appointment status, such as whether an appointment was kept, changed, or canceled, the stringency of the pandemic policies enacted for the region, and the mobility of the population in that region. In other words, the stringency of pandemic policy and the resulting restrictions on the mobility of a population may cause the medically-vulnerable to miss or avoid regular medical care. And this may be contributing to the increase in non-COVID deaths among this group.
The economic impact of pandemic policies
Closures, lockdowns, and decreased mobility have led to wage and employment loss. Though some governments have instituted income support policies, the timing of that aid correlates to employment loss. In countries where income support policies were put in place at roughly the same time as stringent lockdown policies such as workplace closings, the unemployment rate remained relatively flat. This was the case, for example in Sweden and Belgium. In contrast, a delay in the implementation of income support policies correlates to an increased unemployment rate, as was seen in the United States.
Income support policies may affect individuals in the labor force differently. Many countries have undergone employment and wage loss in the informal economy, wherein enterprises, jobs, and workers are not protected by the state.
The team set out to identify the most economically vulnerable populations in this context. The analysis focused on those countries with stringent lockdowns that have implemented income support policies, and in which the population works in sectors highly-impacted by the pandemic policy, such as accommodation and food service, manufacturing, and retail trade.
Some of the results of this analysis are represented here by a mapping of countries according to the stringency of their pandemic policies and the share of their labor force participation in highly impacted sectors. Each country is represented as a circle, the color, and size of which indicates the vulnerability of the workforce in terms of the share of the workforce involved in informal labor.
Vulnerability ranked by Informality Rate
This type of topology of the vulnerability of labor forces during the pandemic may be useful in indicating which groups to attend to with income support policies.
While government lockdown policies were designed to slow the spread of COVID-19, they had direct and indirect negative effects on their populations.
We found that non-COVID deaths of those with existing health conditions and considerations increased during the pandemic, for the population studied. For this medically-vulnerable population, we found a relationship between the stringency of lockdown pandemic policy and the level of mobility within a locality, with the delivery of non-COVID, and potentially life-saving, healthcare.
Domestic violence emerged as a growing “shadow pandemic”. We found a strong relationship between a decrease in mobility of a population and indicators of domestic violence.
To offset the economic impact of anti-contagion policies, many governments instituted income support policies.
We determined that the timing of income support policies mattered. For the locations studied, when income support policies were implemented at the same time as lockdown measures, unemployment rates stayed flat. In contrast, in countries where income support policies were delayed, unemployment rate curves remained steep even after policy implementation.
The team created economic vulnerability assessments of countries, by considering the stringency of lockdown policies and the share of the labor force involved in highly-impacted sectors and in the informal economy. Income support policies may be more effective when such vulnerability is considered.
Our objective with these results is to support policymakers in finding the most effective ways to minimize the suffering of those most vulnerable.
Find all insights in the demo day recording
All Collaborators from this project
We thank our partner organizations, AI for Peace, SH4P, and PWG. as well as all Omdena collaborators (listed below) who made the project a success.
More about Omdena
Omdena is an innovation platform for building AI solutions to real-world problems through the power of bottom-up collaboration.