All Articles

Big Data - Internet's nervous system

From Jallikattu to CalExit, social media has a vital role to play. With big players moving towards a connected world, data (big or small) has been the prime monetizing factor. Apart from the money minting perspective, the data we generate has started influencing our shopping, culinary experience, healthcare, insurance, parenting, and news we read.

Before discussing the pros and cons of big data, let’s look some recent issues and their digital footprints.

Donald Trump

Gender distribution of people speaking about Trump

trump gender

trump geolocation

Chennai Memes


cm header

Posts by time of day

time of day

Followers growth

cm follower

If this is not too much information, take a look at this:

Did facebook automatically suggest you, people, to tag in a photo?

Your amazon’s recommendations are closely related to your recent search history.

Google’s too much location awareness

![] (

The big five of silicon valley - Google, Apple, Facebook, Amazon, and Microsoft have diverged from their original product lines into all sorts of hardware, software, and services.

Take a look at facebook’s recent community update:

50 million people connected?! What does this mean?

So why are these giants interested in your data?

One other potentially unimaginable use of facebook data is in the area of self-censorship.Facebook is actually collecting our keystrokes and it is used for the study of self-censorship.

One other prime usage of our data is its application in facebook marketing tools. If a business/page owner intends to promote his/her brand, facebook can help in choose who sees the advertisement and also define ‘who’ using a bunch of demographic parameters. The concept of curated content has now gone so far that we no longer have control of what we see/what we get to see.

This is exactly what election campaign managers specialize in. And, this is the price we pay for personalization of services and free services. Facebook, apart from gathering our activity based data, it also gathers data from so-called data brokers.

Facebook works with a select group of third-party data providers to help businesses connect with people who might be interested in their products or services. We’ve designed these partnerships with people’s privacy in mind. People using our services have control over the ads they see.

See this official FB help page for more details. They have even listed their data providers.

Facebook has gone a step further and has even hosted the list of government data requests that it has received. You can find it here

Highly recommend to go and see what the government of US, Canada, China, and India requests

How are these big 5 companies interdependent on each other?

One way to visualize these enormous data is to consider the path of a typical internet session, from the user to some revenue-generating action, and then (in some cases) back again to the user

For example, it might seem counterintuitive that Amazon is a major threat to Google’s core search business. But you can see this by following the money through the loop: a significant portion of Google’s revenue comes from search queries for things that can be bought on Amazon, and the buying experience on Amazon (from initial purchasing intent to consumption/unboxing) is significantly better than the buying experience on most non-Amazon e-commerce sites you find via Google searches. After a while, shoppers learn to skip Google and go straight to Amazon.

This is how convoluted the loop is and now it can be more obvious as to why FB stores are becoming a new trend.

Alexa, Cortana, Siri and Google assistant - Created with similar goals but working on the human voice interface. Who knows, with the voice data we build and some machine learning and NLP, a machine can start impersonating you in the digital world(including your voice).

As Google CEO Sundar Pichai recently said:

The next big step will be for the very concept of the “device” to fade away. Over time, the computer itself - whatever its form factor - will be an intelligent assistant helping you through your day. We will move from mobile first to an AI first world.

This AI first world is purely fueled by the big data. Big data, in short, is the internet’s nervous system and AI and machine learning is its brain!