Rarely am I more excited to go to class than I am on Tuesday nights for my weekly Data Mining lecture. While I’m already familiar with some of the concepts, our professor is doing a really good job of tying everything together around the central philosophy of practical machine learning, which is to make predictions and generalize well.
We’ve been told from day one that we don’t care about p-values or traditional statistics—not that they aren’t important, but in this class we can actually keep score of how well our model predicts whatever it’s trying to predict. This represents a fundamental shift in how people perceive and understand the world around them. No longer are we simply testing hypotheses and running basic stats to judge their plausibility; instead, more and more, we are using machines that can learn from data.
A lot of people get tripped up in their everyday thinking by looking at situations as either black or white. Just because there’s only one actual story doesn’t mean that we can’t predict which scenarios were most likely to have happened using what we know from the past. All the time, I listen to people who claim to know *exactly* what happened in a scenario when it’s literally impossible for anybody to know for sure. A lot of these conversations happen on Saturday mornings after a late night out. Others involve government conspiracies. But this kind of thinking happens all the time, and if you look for it, you’ll start to see it everywhere. And it has a name too: Confirmation Bias.
“Confirmation bias is the tendency to search for, interpret, or recall information in a way that confirms one’s beliefs or hypotheses.”
You see, most of the time, us humans tend to find whatever we’re looking for, even when it’s not actually there. It seems like a silly thing to do, but it’s been essential to our survival for a long period of time. In order to achieve your goals, you need to rationalize to yourself that whatever steps you take to achieve those goals are necessary. If you start questioning your beliefs or your motives, then your goals might never get accomplished. People in science might have a problem with that, but people in business and other fast-moving enterprises (like a sports team or the military) really understand the value of good heuristics and guiding principles to make time-sensitive decisions.
Chess is a game where time is (usually) a factor in making decisions. A trait of strong chess players is that they consider only a small set of good moves when making decisions. This is because the advanced player has learned a general idea of what a good move looks like. Beginners, on the other hand, have no concept of what makes a good move. When deciding on a move, beginners are deciding between a massive set of possible moves, because they don’t have the heuristics to narrow down their selection space to moves that generalize well later in the game. As a result, the beginner will generally pick more bad moves than the stronger player who picks from a smaller set of quality moves.
What we’re actually doing by “learning from data” is building better heuristics for our decision-making. We’re getting better at generalizing in situations where the best choice is not obvious, which is the fundamental goal of machine learning. Humans are doing less and less interpretation and hypothesis testing because it’s not a winning strategy—the possible set of explanations for whatever problem you’re trying to solve is simply too complicated for you to rely upon intuitive reasoning. Instead, it’s best to collect and analyze massive amounts of data in order to observe what’s actually happening on a more detailed level. When you do this, you can quickly build better heuristics, and as a result, make better decisions.
Heuristics are moving out of our “gut” and onto our monitors. This is where the world is going, and it’s going to impact a wide range of industries in the coming years. There are data scientists who can easily build excellent models on topics for which they have almost no domain knowledge. And data hotshot Jeremy Howard is excited about the implications of AI progress, but he’s concerned that the labor market will take a pretty big hit. Fortune Magazine recently hypothesized an Algorithmic CEO. I might not know much, but if your plan is to become a high-performing and productive member of society, I’d pay attention to how data is going to impact your industry.