Big Data for Good

Daily life is being digitised and recorded, the result is a vast mass of data.

Big Data is one of today's buzzwords. The mass of data that’s accumulating thanks to the digitalisation of our everyday lives promises greater efficiency, completely new insights, even predictions about the future. This data is generated by us all. More and more of our daily activities now leave a digital trail: shopping for clothes or using online banking, navigating with the Sat-Nav or making a call on our mobile phones. A growing mass of data is also coming from the “internet of things”, from sensors on cargo containers or weather stations. This data also has enormous potential for the social sector and the common good.


Today's economy looks towards Google, Facebook and Amazon, whose empires are all built on a foundation of data. Many entrepreneurs are also asking how they too can better use their data. Governments debate whether they should make the vast volume of data they have on the population available to the public (see trendOpen Data). Digital data is encroaching upon and influencing more and more aspects of our lives. Whereas entrepreneurs get dollar signs in their eyes when they think of optimising processes, increasing efficiency and making predictions about the future, what we're interested in is the question of how we can use data for the common good. How can we make better, data-based decisions in the social sector?

Big Data for Good already helps us better predict events. Google Flu uses the frequency of certain search terms to estimate the probability of flu outbreaks and flu activity. Their predictions are just as accurate as those of the US American Centre for Disease Control (CDC) – but quicker. Natural language analysis of social media in the USA and Ireland showed a full three months before official unemployment figures increased that many people feared for their jobs. The number of tweets from Indonesia mentioning the price of rice correlated extremely closely with the actual price increases, a fact that official statistics were only able to confirm much later on. These predictions of unemployment and fluctuations in the price of rice were down to the work of the Global Pulse Institute. This institute was founded in 2009 by the UN Secretary General, the hope being that instead of waiting months to be able to describe the effects of global crises, we'll actually be able to predict them. Data can be evaluated the moment it’s created, so that there's still time to act. If people in a particular region suddenly purchase less mobile phone credit or start digging into their savings, if weather sensors report extremely low levels of rain, then these are all signs that a crisis is brewing. Robert Kirkpatrick, Director of Global Pulse, believes that “... analysis of patterns within big data could revolutionise the way we respond to events such as global economic shocks, disease outbreaks and natural disasters ... [The lab works on] formulating and testing hypotheses ... to develop methods for harnessing real-time data to gain a real-time understanding of human well being.” Important big data analyses originate from the health sector: an analysis of the mobile phones of 15 million Kenyans revealed the spreading pattern of Malaria, which was then used to develop more exact prevention programmes.

In the aftermath of the Haiti earthquake in 2010, researchers were able to evaluate the data of two million SIM cards in the area, and thereby track the movement patterns of over 600,000 homeless residents of Port-au-Prince. When there was an outbreak of cholera later that year, the same team was able to simulate migratory movements and predict where new outbreaks of the disease could be expected. During the hurricane season of 2011, the US agency FEMA was also able to send out rescue teams much quicker than before to regions where people had posted warnings on social media. The teams had a 12 to 24 hour head start over previous operations, where FEMA would have to wait for verified status reports – a margin that can save lives.

The vision: development organisations and NGOs disclosing their collective knowledge on social progress

In many areas, the potential of big data for the social sector is even greater than for the economic sector. Whereas many companies have long been driven by figures and data, the social sector lacks even basic data in most cases. NGOs, charitable foundations and donors all try to do good, but many lack transparency, meaning they don't learn from one another. They fumble in the dark with their own work, and small organisations in particular (but also some large ones) don’t even understand their own figures. There’s also a lack of a common information infrastructure that could be used to liaise with others, if desired (although there is great fear of having a competitive disadvantage when competing for donations).

Initiatives such as the International Aid Transparency Initiative (IATI) or Washfunders campaign to get international development organisations and NGOs to make their data available to the public. Only with the help of such data can projects be well coordinated, can partners be found, and can abuse and corruption be exposed. To name just a couple of successful examples of how big data has helped put a stop to abuse: in the course of the Subsidios al Campo campaign in Mexico, an abundance of data on agricultural subsidies was revealed to the public on a map, exposing the fact that some high-up government officials and agricultural groups had been illegally siphoning off large sums. Similarly, data analysis exposed systematic failures in the South African medicine trade: it became clear during the Tendai Project that the same medicines were being sold at completely different prices in the different countries, and that they were often unavailable. The analyses are now being used to push political reforms for more effective healthcare policy.

Improved efficiency of NGOs through targeted data analysis

Philanthropy that’s supported by big data can help NGOs identify effective levers for their work and create more fitting programmes. This is why the organisation, which gets young people involved in social projects, continually analyses its own performance and optimises it on the basis of digital data. Evaluating the data usage of 300,000 mobile phone subscribers revealed that amongst their target group, text messages were 30 times more effective than email in recruiting people to get involved in projects. Large volumes of data also help organisations better understand the living conditions of their target groups, as well as to recognise patterns and correlations of social problems.

The Justice Mapping Center in New York aggregated the addresses of all American prisoners. The maps show so-called Million Dollar Blocks – city blocks where so many residents end up in prison that their imprisonment costs over one million US-dollars per year. On the basis of this data analysis, investments in youth centres, drug support or neighbourhood assistance schemes can be precisely aimed at these areas, making social intervention not only better targeted but extremely cost effective.

Who gets involved when, how and why?

Big data analysis also lets us learn about donor behaviour, which means we can become more effective at getting people involved. At present we don't knwo much about why people commit to a particular cause or choose to support a particular organisation. What influence do friends, acquaintances, celebrities and other spokespersons and multipliers have? What does the biography of a donor look like? Does a donor who makes a one-off donation following a catastrophe eventually become a discerning donor, who no longer just responds to an emotional televised appeal, but rather chooses to strategically invest in social change? What is the relationship between volunteer work and donating money?

Data allows us to find out who does voluntary work where and how often, and to use it to create a map of social engagement. This is still only a future scenario, but analyses such as those by Kickstarter and Indiegogo on the use of their platforms all point in a promising direction.


Data has great potential for good deeds. However, this trend is not yet fully developed and there are still many hurdles to overcome. It will be a challenge to understand all the data. Consultants from McKinsey estimate that in the USA alone, the workforce lacks 140,000 to 190,000 people who are qualified to analyse data. Due to this lack of data skills, more and more hackathons are taking place, in which data owners join forces with data analysts. To this end, the NGO Code for America set up a meeting of the city administration and programmers, so that they could jointly consider which digital tools could help the public sector rise to challenges in urban areas. In future, donors must also provide subsidies, since the NGOs alone are already swamped and can’t cover the extra costs.