Machines and Metaphors

Introduction: Now that the strange title has attracted you to the article, StOp stands for Stochastic Optimization. This is the first episode in our mini-series. I've been mulling over this article for months now, which is kind of absurd considering that this is meant to be a quick series, but I apologize for my online dormancy. In the meanwhile, I was working on writing content for a course on Machine Learning. If you're still in school (not college), and you want to learn more, check out:  https://code-4-tomorrow.thinkific.com/courses/machine-learning At any rate, let's get started. Expansion and Exploitation: In some ways, the more of this you read about, the more you begin to think of the world as an array of optimization processes - from the bargain you settle on with the grocer to the conversation you had before you sold your company. But an unfortunate side-effect of this kind of outlook, is that you often become a visibly more selfish person. You spend more time exp...

This is part of my new miniseries on Stochastic Optimization. While this is not taught in a lot of Machine Learning courses, it's an interesting perspective, applicable in an incredible number of fields. Nevertheless, this won't be a very long series, and when we exit it, it'll be time to dive straight into our first Machine Learning algorithm! Introduction to Optimization: Ok, so what is Optimization? As the name may suggest, Optimization is about finding the optimal configuration of a particular system. Of course, in the real world, the important question in any such process is this: in what sense? i.e. By what criteria do you intend to optimize the system? However, we will not delve too much into that just yet, but I promise, that will bring about a very strong connection to ML. Introduction to Stochastic Optimization: So far, as part of our blogposts, we have discussed Gradient Descent and the Normal Equation Method . These are both Optimization algorithms, but they di...

Here's a cat - I didn't get a chance to work them into the post. In the previous post, we were discussing Gradient Descent and why it has so many connections to Phase Spaces (check that out over here , if you haven't yet). There, I had dropped a hint about another Optimization Method we could use. Did you catch it? When we discussed some of the calculus involved, I mentioned that the derivative or slope of a function at local minima or maxima is equal to 0. As it turns out, you can simply set the derivative to be equal to 0, and solve for the parameters required. This is sometimes referred to as the Normal Equation Method.  I will talk about a case-specific simplification here.  This method is called Least Squares and it minimizes the following error function, often known as Mean Squared Error, for reasons that will be apparent in a moment: This formula may seem complicated, but don't worry, it is actually quite simple. Before we understand it, though, it is important t...

I'm going to start from where we left off in the last part of this series. If you haven't read that yet, check that out first if you want a more detailed understanding: We explored what a Phase Space is, why it's useful, what it has to do with Machine Learning, and more!  I'm assuming you've read the previous article, or you know what I talked about there: so let's get to it. At the end of the last article, we discovered that it was the power of mathematics that would help us find the best values of the parameters for the lowest cost function. Before we get into what the Math does, however, we'll need to define some things in the math. If you've done Calculus, and in particular, partial derivatives, you can skip this section, but otherwise I would suggest at least a cursory glance. I don't go into too much detail on the subject, but that's only because you won't need it.  Calculus Interlude: Derivatives- The slope of a graph is a concept you...

Phase Spaces One of the least heard of, and most interesting techniques of the sciences, that you rarely realize you’ve used before. Phase spaces are symbolic representations of a particular problem, which you can then use to solve it. Let’s start with a simple problem - in physics maybe. Let’s say we have a car, as all good physics problems do. You’re driving at a set initial speed, and a set acceleration. At what time would you have travelled exactly 15 ft? Let’s look at it in terms of "a phase space". I have a velocity-time graph down here:                                                                                                                                ...

As you can see by now, I love analogies. One thing I have begun to realize about Machine Learning is the structure of the ‘learning’ process. In fact, now that I think about it this way, I’m beginning to find analogs in Philosophy, and even in the more traditional Sciences. When you use Machine Learning, you don’t start with fancy algorithms, abstract conjectures or advanced mathematics. You start with data. This could be anything, depending on the nature of the problem you’re trying to solve: populations over years, inflation rates or number of cat videos per day watched by each person in the USA based on different zones or regions. All these( actually, may be not the last one ) have significant applications or derivations that Machine Learning can aid in achieving. But the important idea here is that no rationale can begin without data, at least in terms of AI. Why is this interesting? It reflects a much more fundamental understanding of our scientific or philosophical pursuits in th...

A depiction of the Cost Curve in Machine Learning, with falling blue balls approaching the optimal minimum, where the algorithm predicts the best model. Science is all about compressing the world. It’s kind of like a .zip file, trying to understand the phenomena and actions of the world around in a few equations that can be scribbled in the corner of a page. As science progressed, and spread, it began, however, to encounter the problem of metaphysics. How could we understand things like joy and happiness, and other abstract things? These two models, of philosophy and physics, were not easily compatible however. One of the two methods of explaining the world had to dominate. While the two definitely still exist, Physics has since taken precedence. This may not seem the case, but that is because the way I’m thinking of physics is different from the standard understanding. To me, the Philosophy of Physics is the idea that all abstract things, ideas and phenomena are just the result of phy...

Machine Learning is quite analogous to the way we learn. One of the most difficult things about learning is preconceptions- on a larger scale, stereotypes. They make it harder to learn something easily, when you have a simple and incorrect idea in mind. ML has not one, but two analogous problems, and this can be very important to make accurate models. So, what are stereotypes? The first way to look at this is that they are training, done from a skewed data set. Let’s take an example. You live in a strange part of the world, where the prevalent flower is a rose. You’ve seen a rose a billion times, and you’ve observed a lot about it: it has thorns, a green stalk and red petals. You know a lot about them. But let’s say that in search of a better job, you set out towards the city. The city seems a bit bewildering, with all the noise and pollution, but the people seem alright, you decide. And then you find the shop. You duck under the huge label that says ‘John’s Floral Arrangements’, and e...

Humans go from guessing to logic, by correction. This is how ML learns. Machine Learning can tend to be a very mystical subject. How a machine can seemingly simulate what is in some way, our greatest ability as living beings- learning- is quite difficult to understand. The best way to do this is by analogy- and that’s what this blog is for. Welcome! This is our first blog, on the philosophy of human learning and how machine learning embodies some part of this. DISCLAIMER: I’m not an expert on Neuroscience. This article is based on my perceptions as a human and a learner.  Learning to talk was probably a pain. I definitely don’t remember it, but watching children around me makes it pretty clear. Relearning a language for school has only accentuated that experience.  But how it happens is very interesting. Here’s how it starts: Someone tells you a couple of rules for the language. Some words are verbs and others are nouns, and if you don’t want to be boring, please use pronouns....