Big Data Problem

Big Data has been the battle cry from computer scientists, economists, CEOs and politicians for the past decade. Its promise of huge insights that could be found by sifting through the Zettabytes of data that the world creates seemed to offer a route to riches and human happiness.

Read More

Unfortunately, the promise of Big Data was heralded without understanding its two inherent flaws:

  1. Firstly, that it is big. Very big. So, while the data from a global bank might contain major and earth enriching insights, extracting them from the noise of data is very hard.
  2. Second, it is data, and data without any context is often worse than useless as it misleads where it should enlighten. A false correlation can lead a CEO to make a completely wrong acquisition, a misunderstood causation can lead a political party into a voter meltdown.

Big data can be actively dangerous as it misleads the recipient to believe in a correlation that might be utterly false but because of the overwhelming size of the data it is judged to be unquestionably true.

In fact, scientists have outlined six different types of bad correlations:

  • Fluke correlations: One of the bedrock truths of statistics is that, given enough trials, almost any possible occurrence can happen, though you might need to wait a long, long time. The more possible events that might take place, the more potential, albeit unlikely, "fluke" events there are.
  • Ephemeral correlations: Some extreme correlations may jump right out at us and scream "Significant!" only to fade upon repeated observations. Though they may not be statistical flukes, such correlations may vanish under the influence of proper causation analysis.
  • Uncorroborated correlations: Statistical correlations that are based purely on number crunching are weak. The arrival of storks and babies in Holland may have seen a correlation, but there is absolutely no causation despite subsequent myths told to small children.
  • Artifactual correlations: Some correlations may be produced by "artifacts" that are entirely separate from "natural" factors being studied. Artifacts might dilute, distort or reverse any "natural” correlations being studied statistically.
  • Wrongheaded correlations: Some correlations proceed from metrics and data that should never have been developed in the first place. This happens a lot in pseudo-science when someone attaches numbers to fundamentally qualitative phenomena, or attaches the wrong numbers to a quantitative phenomenon.
  • Hyped correlations: Hype can infect the narrative that frames any correlation, no matter how well that correlation is grounded in the subject domain, the data and the statistical models.

Read Less

Causal Analytics
History of the Causal Analytics
Big Data Problem

Causal Analytics

Headlamp Software is bringing the benefits of causal analytics to the world. Learn here about how causal analytics works, the insights it delivers, the applications it is being used for… or just more about Headlamp Software.

Read More

Causal Analytics is a part of Etiology; the science of why things occur. While this is an old science it is at the forefront of computer science as computers transition from decision-support to decision-making across many fields in our society.

Computer science is now at the beginning of a Causal Analytics revolution where the ability to record huge quantities of data while varying conditions through automated experimentation can provide unimaginable insights into the cause of events. Leading technology firms like Google, Amazon and Netflix make extensive use of Causal Analytics to understand human behaviour, and in the case of Google’s DeepMind, to reinvent the game of chess at such a profound level that it is almost beyond human understanding.

In economics, statistics are often subjected to regression testing to establish Causal relationships between decisions and outcomes. Without the ability to subject different population samples to different circumstances to prove causation, the science of economics has been troubled by false correlations that seemingly show causality, but are not in truth related.

But Causal Analytics is not new; in medicine it is also known as epidemiology; an approach that in the 1950s showed through a study of London bus drivers versus bus conductors that exercise is good for you (the drivers die young), and proved that smoking causes lung cancer (yes, this was once in doubt; plastics and cars were potentially thought to be the cause).

More recently, Causal Analytics in medicine has been used in molecular epidemiology, where the Causal inferences of molecular experiments are being used to create personalized medical treatments for everything from the cure of cancers to the regrowing of bone cartilage.

Read Less

Inference Engine
About Headlamp Software
Causal Analytics

The insight of the Inference Engine

The inference engine is a component of the Causal Analytics system that applies logical rules to the knowledge base to deduce new information. In practice, data is collected from a source (say the US electrical grid) and stored in a knowledge base. From there the inference engine can used to test different hypotheses against the data and deduce new knowledge.

Read More

The inference engine is a component of the Causal Analytics system that applies logical rules to the knowledge base to deduce new information. In practice, data is collected from a source (say the US electrical grid) and stored in a knowledge base. From there the inference engine is used to test different hypotheses against the data and deduce new knowledge. Going further, the inference engine can then determine new hypotheses to test against, based on the new knowledge. This recursive approach can lead to an automated analysis of data against a huge number of hypotheses and variables.

Inference engines work primarily in one of two modes; either forward chaining and backward chaining. Forward chaining starts with the known facts and asserts new facts. Backward chaining starts with goals, and works backward to determine what facts must be asserted so that the goals can be achieved.

In Causal Analytics, the inference engine is used not just to test different hypotheses against existing datasets, but through the new knowledge gained from the data to establish new lines of experimentation and testing that will create a fresh wave of data. This recursive experimentation is used to gain highly reliable insights into the causation of specific actions.

Read Less

Gainshare Business Model

Big Data Problem

History of the Causal Analytics

Causal Analytics started as an off shoot of probability theory. But while probability theory gained momentum through its application to gaming and then financial markets, the uptake of Causal Analytics had to wait until the arrival of three key elements:

Read More

  1. The availability of sufficient data pools
  2. The on-line environment where real-time experimentation can occur
  3. The access to sufficient computing power to conduct the real-time Analytics and formulate new experimental waves.

Causal computing as a branch of probability mathematics has its own mathematical notation that was created by Judea Pearl, the godfather of Causal Analytics.

Download a white paper on the statistical model and mathematics behind Causal Analytics: Bayesian Causality

Read Less

About Headlamp Software
Potential Applications

Causal Analytics
History of Causal Analytics

About Headlamp Software

Headlamp Software is a new company that is pursuing the use of AI-based Causal Analytics to solve challenges across the medical research, e-commerce, energy and finance sectors. What these sectors all have in common are major structural challenges in improving performance, access to a vast knowledge base of data and the opportunity to introduce automated AI-led experimentation in the advancement of the enterprise.

Read More

Headlamp Software has created a suite of Causal Analytics components including:

  • Knowledge base for the collection and classification of data
  • Inference engine for insight gathering
  • Automated AI led experimentation engine
  • Actionable insight recommendation report

The founder and CEO of Headlamp Software is Chris Cole. Chris has been an entrepreneur and innovator across the IT industry for the past 30 years where his early successes included the co-founding of Peregrine Systems before its acquisition by HP, co-founder of Inference Corporation before its acquisition by eGain and the development of SMP (Symbolic Mathematics Program), the first commercially available program for doing symbolic mathematics that became the basis of Mathematica, the leading mathematics program. Chris has also served as the Chief Architect of Disney Online for the Walt Disney Company and was the developer and author of CD-ROM and Web-based electronic dictionaries and thesauruses for Merriam Webster. More recently Chris has continued his entrepreneurial path with the creation of biotech software company Group IV Biosystems, pioneering the Software-defined mainframe as Chief Architect of LzLabs and founding Headlamp Software.

Chris Cole studied Physics at Harvard University and completed further graduate studies in theoretical particle physics at the California Institute of Technology.

Read Less

Gainshare Business Model
Social Impact

Inference Engine
About Headlamp Software

Gainshare Business Model

Gainsharing is a business approach where a company receives financial reward only as a result of the gains made by its customers. The total financial reward may be based wholly or partially on the success but will often be the major form of remuneration.

Read More

An example would be a search engine optimisation firm whose fees are aligned to the number of web orders received by its client, or a company that rewards its research scientist for the success of products that they have invented.

Gainsharing’s goal is to improve performance across whatever factors are stipulated; increase in revenue, higher profitability, greater innovation, etc.

Gainshare in Causal Analytics works through the gains that organisations make in their key metrics through the adoption software such as that developed by Headlamp Software. For example, an e-commerce company may find that its sales rate for sports products increases 82% per annum through the use of Causal Analytics. A percentage of the resultant extra profit of $22 million is then paid as the fee for the software, with no other licensing fees required.

Read Less

Deployments Today

History of the Causal Analytics

Potential Applications

Discovering the reason “why” has huge benefits across many sectors. Understanding why a treatment works x% better. Understanding why a design change sold y% more products on your site. Understanding how a change in the trading patterns increased income by z%.

Social Impact

About Headlamp Software
Potential Applications

Social Impact of Casual Analytics

To understand how Causal Analytics is about to influence human society it is a good idea to look at a series of chess games that took place towards the end of 2017

Read More

In December 2017 the world changed profoundly when Deepmind’s AlphaZero beat Stockfish. This game between two chess computers could easily have been of no consequence, but these were two very different chess computers.

Stockfish was then the strongest open-source chess engine in the world and a traditionally programmed chess computer that relies on human defined algorithms to decide the next move – although admittedly thinking many moves ahead of what the average human chess player can do. AlphaZero is a Causal Analytics system that has never been programmed with a single chess strategy, just the rules of the game; the rest it has learned through iterative experimentation by playing 100,000 games a day against itself.

The result of the challenge was a clear win to Alpha Zero; in a 100-game match it won 28 games, drew 72 draws, and had zero losses. Perhaps more importantly, along the way to its victory it completely redefined the accepted strategy of chess – what openings to make, what strategies to play – and won by just simply having worked out what truly works best rather what we as humans think works best.

When you extrapolate that concept across society it utterly changes our understanding and reliance on computers. The role of computers will no longer be digital assistants to help humans in their deliberations, but the determinants of strategy through a Causal Analytics algorithm.

What strategy to follow in the finding of a stem cell treatment for Parkinson’s disease? What strategy to employ for an e-commerce site? What approach to the optimisation of the power grid? The answer no longer lies in the domain of human intelligence and experience but in the power of Causal Analytics.

Read Less

Deployments Today

Gainshare Business Model
Social Impact

Deployments Today

Causal Analytics is widely used today but in highly proprietary engagements. Many of the world’s leading software companies today use Causal Analytics to determine their business evolution.

Read More

Ecommerce

When the world’s leading ecommerce site wants to know how to restyle a page or add additional offerings, they decide what to do through Causal Analytics. The positioning of a photo can change buying behavior by 0.25%. The color of a button can increase the chance of being clicked by 0.5%. The location of the checkout can advance sales by 0.75%. All fairly insignificant by themselves but taken together they spell out a company that incrementally increase sales by double digit growth.

The automated AI-driven Causal Analytics leads to millions of experiments per day on the e-commerce site, each producing data that after analysis will lead to the next informed wave of experiments. Wave upon wave, day after day, each learning from the previous wave and each wave determining the ever-greater success of the Ecommerce site.

Medical Research

Medical Research is at the cutting edge of data science for the discovery of new treatments. The use of Causal Analytics in stem cell therapies is enabling a new generation of personalized medicine for treatments such as cancer, Parkinson’s disease and cartilage growth.

The era of molecular epidemiology has created a data rich environment that enables substantial in-vitro experimentation. Through these insights new patient-specific treatments can be created using stem cell therapies and other approaches that will transform the lives of patients.

Over the next 20 years medical research is expected to be the biggest beneficiary of research gains enabled by the introduction of Causal Analytics.

Games

Although trivial compared to the applications in commerce and medicine, Causal Analytics has already made it arrival felt in the world of games. In 2017 DeepMind’s AlphaZero system used Causal Analytics to beat convincingly the world’s leading conventional chess computer. DeepMind also developed AlphaGo, a system that defeated the reigning Go world champion in a game that many described as he most complex on the planet – and in so doing used highly unconventional approaches as it does in chess.

The world of board games is a great proving ground for Causal Analytics as it moves from a test environment to full deployment.

 

Download a Headlamp white paper on the use of Casual Analytics in e- commerce: Moving the world from correlation to causality.

 

 

Read Less