How Global Citizens Experience COVID-19 And Why We Need Global Solidarity More Than Ever

How Global Citizens Experience COVID-19 And Why We Need Global Solidarity More Than Ever

From a lack of food to a lack of jobs and no effective support mechanisms where it is most needed.

 

By Rudradeb Mitra 


 

I am worried if killing the economies is worth it if we so laxly approach the support for the most affected populations? — Dawid, Poland

April 7th, 2020:

The economical lockdown around the world has resulted in an increase in violence, a lack of food supply, overloaded healthcare systems, and global panic response that will trigger mental health consequences long after the pandemic is over.

Experts agree that government policies need to balance overcoming both the health and economic crisis. In the short run, economic policies should mitigate the impact of lockdowns and ensure that the current crisis does not trigger financial, debt or currency crises. It should facilitate a quick recovery once the economy is taken out of the deep freeze. Currently, there are no solutions to this.

In order to help policymakers and governments to design data-driven response actions, we decided to run a Coronavirus Policy AI Challenge. More than 70 AI engineers and domain experts from leading organizations are currently finding answers to these questions.

In the meantime, we asked 14 Omdena Collaborators across 5 continents about their Covid19 experiences.

 

Fears, hopes, and expectations from 14 countries

 

Murindanyi Sudi, Uganda

 

 

The lockdown created fear and panic; The country is already in disaster. People cannot get to work or their business meaning they can no longer earn a living. The casual workers who survive on a daily wage also suffer. The government has promised food donations for vulnerable people who have lost their source of livelihood but have yet to fulfill. The cause has led to an increase in thievery and other sorts of violence for survival.

 

Aya Salma, Egypt

 

 

In Egypt, we had more than one uprisings in the past ten years. We have lived through periods of great angst, uncertainty, loss, and curfews. The feelings are not new although the challenge is different this time. Something that did change is that, as others mentioned, we are literally all in this together, this time around, against one invisible “enemy”. Also, during the uprisings we feared for the young, now we fear for the old.

 

Juan Pablo, Chile

 

 

The future is uncertain, however, Chileans are very supportive in times of crisis. Every 5/10 years we either have earthquakes, volcanic eruptions or various natural disasters typical of this narrow and long country. I have entered the world of artificial intelligence with the aim of incorporating it into inclusion projects, and I see no better time than now to start them. Hopefully, many people will work with others in mind, especially in underdeveloped countries.

 

Dev Bharti, UK

 

 

I feel our biggest challenge will be to try and restore balance in nature (not just talk) using innovative means, else nature will find ways such as this pandemic to balance things out. And it is crystal clear that this change needs to come bottom-up from people themselves, as the top-down approach has not worked!

 

Erum Afzal, Pakistan

 

 

Pakistan is suffering from fewer facilities in hospitals, poverty, illiteracy, and psychological depression. The situation is also increasing psychological stress. The media reports on increasing numbers of deaths, people not following government directions, and conditions of underserved communities.”

 

Sabina Meric, Portugal and Serbia

 

 

In Portugal, I like that people go out on their windows or balconies every night at 10 pm to clap to honor medical staff. But in my home country, Bosnia medical staff immigrated to Germany and other countries and the ones who stayed don’t have the equipment to work. A total failure of the health system.

 

Ilias Papadopoulos, Greece

 

During this time, governments are spending hundreds of million euros for redundant advertisements. So my question is, why do we spend so much money on this and don’t care about the vast amount of children dying because of the lack of food and medicines in Africa and other poor regions?

 

Nikhel Gupta, Australia

 

 

There are ample of selfish people who are clearing supermarkets through panic buying, and some who are purchasing guns and ammunition. We are witnessing an extreme level of racism and politics of blame game. What I’m worried about today is not the Covid-19 but how people react to it. Fear is more contagious than Covid-19 and only a united world can defeat them together.

 

Aboli Marathe, India

 

 

I see the domestic help, the construction workers, the vegetable sellers, the laborers who take a 100 rupees home a day, to feed their hungry kids and family. One person’s wage supports 6–10 people. What will they do now?”

 

Mikko Lähdeaho, Finland

 

 

Last weekend a few night clubs and restaurants were still open, but they are all closed now like all public places except markets. My area, Uusimaa, is restricted by police and military. Children go to school remotely and fortunately the Internet is working. I hope we’ll get over this soon and I wish you all strength.

 

Mahzad Khoshlessan, US/Iran

 

One of my big concerns is the economic situation and how the people especially international students like us will be affected by this situation. How this situation is going to affect the job market and hiring process is a concern of many people like me who are in their early career.

 

Ziyad Jappie, South Africa

 

 

I had a verbal confirmation about a job but then the COVID-19 came and I was told they cannot commit. This makes me worry about the possibility of getting jobs in the near future since almost a third of our population is unemployed. Apart from this, the direct effects it has on the people of the country are very worrying given that we have a high number of HIV+ cases. I pray for things to get easier for us all.

 

Dawid Mondrzejewski, Poland

 

 

We are an economy that functions predominately on relatively cheap labor. Unemployment is low, but most salaries are also small. I am worried if killing the economies worth it if we so laxly approach the support for the possibly affected? Hopefully, around the world, fewer people will die from the economic slump, than from the virus (if restrictions have not been so much draconian).

 

Joanne Burke, Orange County, California

 

 

I know there are cases in our city, but neighborhood data is not released. It is a joy to hear my son’s elementary school lessons on Zoom every day while the schools are scrambling to create online content. Prayers to healthcare heroes everywhere!

 

Yash Mahesh Bangera, India

 

 

I was waiting for my MS admits and this corona situation came up suddenly. It will be financially draining and will create a setback for me.

 

Soroush Sarabi, Iran

 

There are grave fears for our people. The severe economic problems in Iran do not allow them to stay at home. Also, Iranians are accustomed to having fun outside the house, and it is tough for them to stay at home, so these two can cause hazardous situations in the next weeks.

 

How Omdena is combating the Coronavirus

A good start to learn more about Omdena’s innovation platform is to read about our Coronavirus Policy AI Challenge, where more than 70 AI and domain experts are collaborating to build AI models that reveal the direct and indirect impact of pandemic policies on the economic health of marginalized communities.

Our aim is to support policymakers in identifying the most effective ways to minimize the economic suffering of those most vulnerable.

 

About Omdena

Solving Challenges Through Collaboration

Omdena is an innovation platform for building AI solutions to real-world problems through the power of bottom-up collaboration.

Learn more about the power of Collaborative AI.

How To Estimate Possibly Undetected COVID-19 Infection Cases

How To Estimate Possibly Undetected COVID-19 Infection Cases

Country-wide estimations for undetected Covid-19 cases and recommendations for enhancing testing facilities.

 

By Nikhel Gupta


Why estimating undetected Covid-19 cases is crucial?

An estimation of the undetected Covid-19 cases is important for authorities to plan economical policies, make decisions around different stages of lockdown and to work towards the production of intensive care units.

 

How far is a Covid-19 testing center from your home? (credit: link)

How far is a Covid-19 testing center from your home? (credit: link)

As we have crossed a psychological mark of 1 million Covid-19 patients around the globe, more questions are popping up regarding the capabilities of our health care systems to contain the virus. One of the major worries is the systematic uncertainty in the number of citizens who have hosted the virus. The major contribution to this uncertainty is possibly due to the small fraction of Covid-19 tests being performed.

The main test to confirm if someone has Covid-19, is to look for signs of the virus’s genetic material in the swab of their nose or throat. This is not yet available for most people. The healthcare workers are morally restricted to reserve the testing apparatus for seriously ill patients in the hospital.

In this article, I will show a simple Bayesian approach to estimate the undetected Covid-19 cases. The Bayes theorem can be written as:

P(A|B) = P(B|A) × P(A) / P(B)

where P(A) is the probability of event A, P(B) is the probability of event B, P(A|B) is the probability of observing event A if B is true, and P(B|A) is the probability of observing event B if A is true.

The quantity of interest for us is P(infected|notTested) i.e. the probability of infections that are not tested. This is equivalent to the percentage of the population infected by Covid-19 but not tested and we can write it as:

P(infected|notTested) = P(undetected|infected)×P(infected)/P(notTested)

Here the other probabilities are:

  • P(notTested|infected): Probability of tests not done on people that are infected or percentage of the population not tested but infected.
  • P(infected): Prior probability of infection or known percentage of the infected population.
  • P(notTested): Probability or percentage of people not tested.

The following plot shows the total Covid-19 tests per million people and the total number of confirmed cases per million people for several countries. This suggests a clear relation between the Covid-19 tests and confirmed positive detections.

 

Figure 1: Tests per million versus positive Covid-19 cases per million as of 20 March 2020 (data source).

 

Assuming that all countries follow this relation between the Covid-19 tests and confirmed cases, we can make a rough estimate of the number of undetected cases in each country (I will come back to this assumption later in this post).

Let’s take Australia as an example:

For example, the plot shows that prior knowledge of infected cases

P(infected) = 27.8/10⁶, and

P(notTested) = (10⁶ — 473)/10⁶.

To estimate the P(notTested|infected), I used the relation between the Covid-19 tests and confirmed cases as in the above Figure 1. This is done by fitting a power law of the form: y = a * x**b, where a is normalization and b is the slope of this power law. The following plot shows a fit to the data points from the above plot, where the best fit a = 0.060±0.008 and b = 0.966±0.014.

 

Figure 2: The relation between Covid-19 tests and confirmed cases and a power-law best fit.

 

Using the best fit parameters, P(notTested|infected) = (10⁶— 4473)/10⁶ / (a * (10⁶ — 4473)**b)/10⁶.

With probabilities 1, 2 and 3, I find P(infected|notTested) = 0.00073 per cent population of Australia. Multiplying this by the population of Australia indicates that there is a possibility of about 18,600 undetected Covid-19 cases in Australia. The following plot shows possible undetected Covid-19 cases as a function of tests per million for different countries as of 20 March 2020.

 

Figure 3: Estimation of undetected Covid-19 cases (see assumptions in the text).

Note that several assumptions and considerations are made to estimate these undetected cases. For instance:

  • I assumed that all countries would follow the same power-law relation to estimate P(notTested|infected). However, this is not an extremely good assumption as there is huge scatter in this relation between different countries.
  • Our prior knowledge of the number of infections can be biased itself as P(infected) depends on the number of tests performed as of 20 March 2020.
  • I haven’t considered the susceptibility of a country’s populations to Covid-19, and the attack rate i.e. the biostatistical measure of the frequency of morbidity, which for Covid-19 is estimated around 50–80% (Verity et al. 2020).
  • The impact of government policies of these countries from 14 days before 20 March and 14 days after is not considered.
  • I haven’t considered how susceptible people are targeted for testing in different countries in the next days.

Figure 4 below shows the total number of confirmed cases versus the tests per million as of 5 April 2020 for several countries (data source).

After 16 days on 5 April, the confirmed positive cases in countries like Ukraine, India and Philipines are consistent with the predictions in Figure 3. These countries performed ≤ 10 tests per million people as of 20 March.

Note that the consistency between estimations as of 20 March and 5 April does not necessarily mean that all undetected cases as of 20 March are confirmed now. Several of the confirmed cases as of 5 April are expected to be new cases due to the spread between 20 March and 5 April (even in the presence of lockdowns).

The estimated undetected cases for countries like Colombia and South Africa are about twice as large (Figure 3) as compared to the total confirmed cases as of 5 April (i.e. about 1,500 for both). Both countries have performed about 100 tests per million people.

Countries like Taiwan, Australia, and Iceland, on the other hand, have shown an order of magnitude small number of confirmed cases as compared to estimated numbers in Figure 3.

This indicates that the countries that have not boosted their testing efficiency to more than 1,000 tests per million people have significantly larger uncertainties on the number of current confirmed cases.

Figure 4: The total number of confirmed cases versus the tests per million as of 5 April 2020.


Given the data in Figure 4 from 5 April 2020, I repeated the whole exercise again to estimate the undetected Covid-19 cases for these countries, cities, and states. The following figure shows the best fit power-law and data points similar to Figure 2 but for the data as of 5 April 2020.

 

Figure 5: Best fit power law for data as of 5 April 2020.

 

The best-fit slope for the power-law relation in Figure 5 (b = 1.281±0.009) is consistent with the slope in Figure 2 at 2-σ confidence level. This helps our assumption of estimating P(notTested|infected) from the best fit power-law relation (the slope is not changing), however, other caveats are the same as before.

Finally, the following plot shows the estimated undetected Covid-19 cases for different countries as of 5 April 2020.

 

Figure 6: Estimated Undetected Covid-19 cases as of 5 April 2020 (see assumptions in the text).

 

As the comparison between the undetected estimations as of 20 March (Figure 3) and confirmed cases as of 5 April (Figure 4) shows that more tests per million people are required to capture the possible undetected cases, thus now is the high time that authorities raise the testing efficiency in order to reduce the systematics from undetected Covid-19 cases. This seems to be the only good way to reduce the death rate of Covid-19 patients as indicated by the large amount of Covid-19 testing in Germany and South Korea.

To make this happen, all countries need at least one testing center within a radius of 20 Km and arrange more drive through testing facilities as soon as possible.

This work was done in collaboration with the people working on the Omdena Coronavirus AI challenge.

You can contact me on LinkedIn and follow my academic research on Orcid.

 

 

How Omdena is combating the Coronavirus

A good start to learn more about Omdena’s innovation platform is to read about our Coronavirus Policy AI Challenge, where more than 70 AI and domain experts are collaborating to build AI models that reveal the direct and indirect impact of pandemic policies on the economic health of marginalized communities.

Our aim is to support policymakers in identifying the most effective ways to minimize the economic suffering of those most vulnerable.

 

About Omdena

Solving challenges through collaboration

Omdena is an innovation platform where changemakers build AI solutions to real-world problems through the power of bottom-up collaboration.

Learn more about the power of Collaborative AI.

Using AI to Enable Data-Driven Response Actions During Pandemics

Using AI to Enable Data-Driven Response Actions During Pandemics

Palo Alto-based startup Omdena wants to use AI to help governments make data-driven decisions when dealing with pandemics like the coronavirus

 

Omdena Logo

 

By Laura Clark Murray 


 

Palo Alto, California, March 30, 2020 –  When travel is restricted, schools closed, businesses shut down, and communities put into quarantine, individuals in those ecosystems lose their sources of income. Omdena, a Palo Alto-based startup that unites AI and domain experts from around the globe, is launching an AI challenge to investigate the impact of such policy decisions on people’s financial stability.

 

In an effort to curb the coronavirus pandemic more than 100 countries have imposed travel restrictions and 2.5 billion people, or 30 percent of the world’s population, have been directed by governments to stay at home. The resulting loss of wages is expected to be disastrous to those already on the economic margins, including wage workers. Omdena’s AI challenge aims to provide analysis of the economic effects of the COVID-19 crisis.

The lockdowns in Europe, the US, and India affect the poorest in those regions and elsewhere. We must think about those hundreds of millions who do not have savings or a pantry full of food. When those people cannot go to work every day to earn a living, the impact is devastating,said Rudradeb Mitra, Founder of Omdena. “We want that impact to be understood and considered by policymakers.”

Omdena, which is a partner of the United Nations’ AI for Global Good Summit 2020, comes with a track record of successfully completed AI projects. Those efforts include using machine learning to identify the safest routes in Istanbul for earthquake victims to reunite with their loved ones. It has also delivered AI solutions which helped detect the outbreak of fires in the Brazilian rainforest with 95 percent accuracy.

“Our goal is to minimize the human suffering that results from pandemic policies. We created this AI challenge to support policymakers with data-driven analyses that will help them make even more informed decisions in the future,” added Mira.

Omdena runs collaborative AI projects in which global teams of 40 or more data scientists and experts build AI solutions to address significant real-world problems. To date, more than 900 people from over 75 countries have participated in Omdena’s challenges.

Omdena’s Coronavirus Policy AI Challenge is supported by the UN AI for Good Global Summit, AI for Peace, PWG, Fruitpunch AI, LabelBox, and Spell. Joining the challenge are economic, health and humanitarian policy experts from around the world, who bring experience with organizations including the World Health Organization, The World Bank, European Commission, and UNICEF USA.

We are excited to join efforts with Omdena to protect those with the least capacity to manage the burdens of this crisis — the impoverished and economically marginalized,” said Branka Panic, Founder of the think tank AI for Peace.We aspire to help governments and international organizations deal with this and future pandemics by taking an AI-enabled and evidence-based approach to policymaking.”

Omdena is a partner of several United Nations organizations, including the UN Refugee Agency and the UN World Food Programme, as well as an official Innovation Partner of the United Nations AI for Global Good Summit 2020.

###

For media inquiries contact: Laura Clark Murray, Omdena, laura@omdena.com 

About Omdena: Founded in May 2019, Omdena is an innovation platform for building AI solutions to real-world problems through global collaboration. The company’s partners include the UN World Food Programme and the UN Refugee Agency. Omdena is an Innovation Partner of the United Nations AI for Good Global Summit 2020. Learn more at Omdena.com

Learn more about Omdena’s Coronavirus Policy AI Challenge at https://omdena.com/challenges/ai-pandemics

About Rudradeb Mitra: The India-born Rudradeb Mitra is a graduate from the University of Cambridge, UK and an international AI expert. He has built six startups in four countries. His primary interest is to build products with social value. He is a mentor and AI advisor at several institutions including Google Launchpad, ImpactHub, MIT Enterprise, Founders Institute and a senior AI advisor of EFMA Banking Group. Mitra founded Omdena in 2019 to address real-world problems through global collaboration.

Fears, Expectations, And Hopes: Voices From 25 Countries As COVID19 Crisis Continues

Fears, Expectations, And Hopes: Voices From 25 Countries As COVID19 Crisis Continues

Why we need a global approach to solve crises like the Coronavirus and what role do fears, expectations, and hopes play in different countries and communities.

 

By Rudradeb Mitra 


 

This crisis shows both the interdependence among countries in the world as well as how countries and communities are impacted on a different scale.

A call for solidarity

After we started a global Coronavirus Policy AI Challenge on our innovation platform at Omdena with more than 70 AI and domain experts, we asked our global community of nearly 1000 collaborators to share their dreams, fears, expectations, and hopes while going through the current crisis.

We have collected 28 stories from 25 countries. This article is part 1 and covers 14 inspiring and thought-provoking stories raising important questions:

What happens to people in the informal economy?

How do experiences differ between developed and developing countries?

How to create awareness for a global approach to solving the crisis?

 

14 countries, 14 perspectives

Matteo Bustreo, Italy

 

 

I am lucky: I can stay in my apartment with my family and I have a job I can do from remote. I went out only twice in the last 10 days, for trashing my bins. My hope is that the hard experience lived by the Italian population can help other governments in taking the most suitable decisions for saving as many lives as possible.

 

Zaheeda Tshankie, South Africa

 

I recently got married again after being widowed for 5 years. Our honeymoon is in Quarantine. A huge disparity in my country is the contrast between the poor and those who are middle-class and above. most poor communities are more concerned about their next meal, not a virus.

 

Anna Lopez, Colombia

 

 

In my city is a resource fund to give food to people that have informal jobs and can’t work now. The mayor picked up all homeless people and created a “refuge camp” where they have food, a ceiling and a clean place to stay for these days. It helps us to feel positive, and that this situation opens our hearts.

Tefy Lucky Rakotomahefa, Madagascar

 

 

I am, in lockdown, at home, for 15 days leveling up my skills. My fear? Are we experienced and disciplined enough to fight this pandemic?

 

Space for optimism

 

 

Serhiy Shekhovtsov, Ukraine

I can see a lot of reasons to stay optimistic here. The whole world is fighting this together. I think one of the best ways to deal with stress and anxiety is to get yourself involved in the fight. I am building a free AI assistant for medical imaging. Hopefully, it will help many doctors someday.

 

Albert Lai, Canada

 

Canada has responded pretty quickly with some sweeping social distancing policies and our government is providing monetary support for those who are unable to support themselves at this time. School’s closed for at least 3 weeks (looks like it’s going to be longer) and we’re looking to start online lessons next week. I’ve been learning about web development in this time and I have to say, it’s time well spent.

Sara EL-ATEIF, Morocco

 

 

People who worked as street vendors or unstable jobs have no income now and are struggling to make ends meet (other people and the government is helping but sadly a lot of people still suffer because we don’t have anything that lists these people as people in need somewhere to make the process quick and efficient).

 

Covid19 in Africa

 

Eric Nzivo, Kenya

 

My fears aren’t actually on how we will handle the situation since I know we have been in close to worse issues and we still pulled through. My fears are on when the world gets corona in control and only Africa is still affected. What will happen then?

 

Colton Magnant, USA

 

There are many who are hoarding resources and completely disappearing from society. At the opposite extreme, there is a disturbing number of people who are confident that it will not affect them and they insist on going about their normal lives regardless. These different groups may coexist except that there seems to be a general lack of compassion and understanding for the opposite group to the point where tensions are high between them.

 

Experiencing a pregnancy

 

Anju Mercian, USA

I am 22 weeks pregnant and it is a very scary time for me right now. I have currently quarantined myself in my home and I haven’t met anyone for almost a month. Usually, my husband comes with me to calm my nervous and hear the baby’s heartbeat but now he isn’t allowed. If the virus doesn’t subside by July I may have to give birth alone which I really do not want to do.

 

Joseph Itopa Abubakar, Nigeria

 

I fear Coronavirus will give birth to a new form of corruption in Nigeria as donations will not be accounted for. My experience so far is that activities are slow, people who came out to purchase food items are chased to go back home whereas, no measures to supply them with food items in their various homes. The price of face masks and sanitizers have increased by 10 fold.

 

Hoa Nguyen, Vietnam

 

I am living in a busy area in HCM with lots of restaurants, and coffee shops. Due to the lockdown law, all of them closed. The street has never been as empty as right now. Some waiters/waitress and employees of theses shops will work as shipper for delivery services. But some I don’t know might be unemployed because they have no other choice.

 

Yashika Sharma, India

 

My fear is, even when the Covid19 will vanish and the world will be corona free, things are not the same. Everyone will be suspecious and look at others fearfully. Public relations would be affected by doubts like ‘What if this person is infected’.

It will take a long time for conditions to become normal again.

 

Leo and Karen, Brazil

 

We are in São Paulo, Brazil. I’m really concerned about all the poor people living on slums — who will take care of them? But most of all I fear we don’t learn anything from this pandemic. Our values must change, this virus is showing us that we are all on the same boat, doesn’t matter how much money you have or which country you live you can be affected. We have to embrace ourselves as humans and treat the environment more respectfully.

We are thanking all Omdena collaborators for the openness to share experiences.

 

How Omdena is combating the Coronavirus

A good start to learn more about Omdena’s innovation platform is to read about our Coronavirus Policy AI Challenge, where more than 70 AI and domain experts are collaborating to build AI models that reveal the direct and indirect impact of pandemic policies on the economic health of marginalized communities.

Our aim is to support policymakers in identifying the most effective ways to minimize the economic suffering of those most vulnerable.

 

About Omdena

Building AI through global collaboration

Omdena is an innovation platform where changemakers build AI solutions to real-world problems through collaboration.

Learn more about the power of Collaborative AI.

The Relation between Covid-19 News Articles and Stock Exchange Prices

The Relation between Covid-19 News Articles and Stock Exchange Prices

A machine learning approach to understand the relation between the news media articles and the downfall of stock exchanges — Panic or Information?

 

By Nikhel Gupta


 

Covid-19 is undoubtedly a cruel virus and we have seen it ripping families apart around the globe. At the time of writing this article, about 470,000 people are infected by this virus in all continents (except Antarctica). So this variety of coronaviruses should be treated with caution and respect. However, if you compare the current daily statistics of the Covid-19 infections to the population of the world, you will find that the probability that any one of us will catch the virus today is super small.

According to the official WHO data, 60 out of 1 million people have hosted the virus until now. This is a very small number and an outlier if you ask a Statistician. So should we really take all the precautions that Governments are asking us to take like hand washing, sanitizing, keeping social distance, etc.? Certainly yes, as even if the probability of getting sick is small, you are not special, it can infect you and you can transmit it to others. Also, you do not want to cripple the health care system which is already overburdened and you want to stay healthy (for a change). But do we really need to panic? No, right?

I believe that all of you reading this article know that we don’t need to panic but still, we’re seeing empty shelves in our supermarkets. So why are those bulk buyers panicking and how is it propagating?

Let me think, what do I see when I turn on any news channel or read a news report today? Firstly, I see news about coronavirus and …, well, there is no ‘and’, I only see news about the coronavirus!

Much of the news these days has disingenuous reporting with sensational claims and flashing scaremongering headlines clearly to attract your attention and clicks. Several media outlets are capitalizing on our fear of losing our dear ones and ignoring to report all other news, which directly or indirectly propagates panic and hysteria. Such panic is not good for our psychological and economical state and we can already see it’s effect on the crashing stock markets around the world.

Several studies in the past have shown that stock markets are directly affected by the everyday news (e.g. Zhou et al. 2018; Hiransha et al. 2018). In this article, I’ll show the predictions of stock prices using the news articles scrapped for the month of January, February, and March of 2020. The following are some of the tasks that are performed for these predictions:

  1. Pulling all news data for all countries and filter articles related to Covid-19.
  2. Combining news data for January, February, and March and scrape them using the URLs in the data.
  3. Applying co-reference resolution to the text, manually labeling economic and non-economic articles and training a random forest/logistic regression model to classify all articles.
  4. Downloading stock exchange data.
  5. Building a neural network to predict stock prices from news articles.

Task 1

Find the full code in Github.

Next, I searched for news headlines that have words related to the coronavirus. For instance, I used the following keywords

relevant_words = [‘corona’, ‘coronavirus’, ‘wuhan’, ‘hubei’, ‘virus’, ‘quarantine’]

The number of articles per day with these keywords is in-between 88,356 (08–03–2020) to 178823 (04–03–2020) for the month of March. This number was just 12,317 on 01–02–2020.

I know that’s a huge rise in the number for English only articles, right?

Note that this number is true only when the above keywords are mentioned in the headlines. There can be several more keywords (e.g. I missed Covid-19) and some articles may be talking about coronavirus in the text and not in the headline.

Task 2

Find the full code in Github.

Task 3

The modern Natural Language Processing (NLP) techniques like neural networks allow us to do this job easily by training a model with a coreference-annotated dataset and use the trained model to perform coreference resolution for all articles. Even better, there are tools available that are trained on such huge datasets and we can just use them to resolve out text data of news articles. One such tool is Neuralcoref, a pipeline extension for Spacy which annotates and resolves coreferences using a neural network.

Here is the working code for co-referencing.

After labeling some of these articles manually, the classification algorithms like Random Forests Classifiers and logistic regression are used to categorize articles into economic and non-economic articles.

First, the co-referenced text is cleaned using the following clean_text() function:

# stop words
stopw = set(stopwords.words(‘english’))
snow = nltk.stem.SnowballStemmer(‘english’)
# lets remove words like not, very from stop words 
reqd_words = set([‘only’,’very’,”doesn’t”,’few’,’not’])
stopw = stopw — reqd_words# text cleaning
def clean_text(article):
 cleaned_article = []
 cleaned_words_list = text_to_word_sequence(article)
 for word in cleaned_words_list:
 if word not in stopw and len(word) > 2:
 cleaned_article.append(snow.stem(word))
 return ‘ ‘.join(cleaned_article)final_df[‘stemmed_articles’] = final_df.text_coref.apply(lambda x: clean_text(x))

The cleaned text is then converted to vectors using TF-IDF bigrams as following

# converting data into vectors using TF-IDF bigram
tfidf = TfidfVectorizer(ngram_range=(1,2), min_df=5, max_features=10000)
tfidf_xtrain_vect = tfidf.fit_transform(train_df.stemmed_articles)tfidf_xtest_vect = tfidf.transform(test_df.stemmed_articles)

And the model is trained using the grid search:

def best_model(x_train, y_train, x_test, y_test):
 pipe = Pipeline([(‘classifier’ , RandomForestClassifier())])
 param_grid = [
 {‘classifier’ : [LogisticRegression()],
 ‘classifier__penalty’ : [‘l1’, ‘l2’],
 ‘classifier__C’ : inverse_lambda,
 ‘classifier__class_weight’ : [None, ‘balanced’],
 ‘classifier__solver’ : [‘liblinear’]},
 {‘classifier’ : [RandomForestClassifier()],
 ‘classifier__n_estimators’ : list(range(10,300,10)),
 ‘classifier__max_features’ : list(range(6,32,5))}
 ]
 clf = GridSearchCV(pipe, param_grid = param_grid, cv = 3, verbose=True, n_jobs=-1)
 best_clf = clf.fit(x_train, y_train)
 print(f’best estimator is {clf.best_estimator_}’)best_logreg_model = clf.best_params_[‘classifier’]
 best_logreg_model.fit(x_train, y_train)unigram_predicts = best_logreg_model.predict(x_test)
 cv_cm = pd.crosstab(y_test, unigram_predicts, rownames=[“True Label”], colnames=[“predicted label”])
 print(“confusion matrix on test data is:”)
 print(cv_cm)
 print(“ “)
 print(“classification report on test data is”)
 print(classification_report(y_true=y_test, y_pred=unigram_predicts))return best_logreg_model

The full code for this is in the following gist.

The model produces the following results on a test dataset:

confusion matrix on test data is:
predicted label  NEGATIVE  POSITIVE
True Label                         
NEGATIVE              405        12
POSITIVE               16       381

classification report on test data is
              precision    recall  f1-score   support    NEGATIVE       0.96      0.97      0.97       417
    POSITIVE       0.97      0.96      0.96       397    accuracy                           0.97       814
   macro avg       0.97      0.97      0.97       814
weighted avg       0.97      0.97      0.97       814

With this trained model, I find approximately 20% of news articles that report economical news related to the coronavirus.

Task 4

And following is the plot showing normalized closing prices of stocks.

 

Stock exchange normalized prices downloaded with Alpha Vantage

Task 5

I will discuss the adapted version of the network in more detail in a future post and here I will present the model predictions for some of the stock exchanges.

I) New York Stock Exchange (NYSE) closing prices from 1st January 2020 to 20 March 2020. The green line shows actual prices and blue lines are the prices predicted from the news articles.

 

NYSE closing prices (green) and predicted prices (blue).

 

II) Same as above but for the Hong Kong Stock Exchange (HKSE).

 

HKSE closing prices (green) and predicted prices (blue).

III) For Australian Securities Exchange (ASX)

 

ASX closing prices (green) and predicted prices (blue)

IV) For Bombay Stock Exchange (BSE)

 

BSE closing prices (green) and predicted prices (blue)

All these stock predictions from the news article data show a correlation between the news and stock prices. Although the correlations are not too strong on a day by day basis as the stock exchange prices depend on several other factors. The downfall trend of stock predictions from news articles is however similar to the actual trend.

What should we do?

On a personal level, I think we need to calm down and keep working. Follow all the precautions. Think twice and crosscheck before believing any news that spreads panic. There is no need to update ourselves with the number of coronavirus cases every hour and keep talking about it in every discussion. Possibly, we need to stop watching/reading the news about the coronavirus and to update and inform ourselves, we can always look into several official platforms developed by the Governments of each country.

Remember, feelings like fear and panic are contagious, probably much more than the Covid-19.


This work was done in collaboration with the community members of Omdena AI. I thank Hoa Nguyen, Yash Mahesh Bangera, Linda and Sadhika Dua for all important contributions.

In future work, I plan to look into the job crisis and the impact of Covid-19 on the informal sector with another Omdena AI challenge.

You can contact me on LinkedIn and follow my academic research on Orcid.

 

About Omdena

Building AI through global collaboration

Omdena is a global platform where changemakers build ethical and inclusive AI solutions to real-world problems through collaboration.

Learn more about the power of Collaborative AI.

Stay in touch via our newsletter.

Be notified (a few times a month) about top-notch articles, new real-world projects, and events with our community of changemakers.

Sign up here