Phase Spaces 1 : Graphs and Geometry

Phase Spaces

One of the least heard of, and most interesting techniques of the sciences, that you rarely realize you’ve used before.

Phase spaces are symbolic representations of a particular problem, which you can then use to solve it. Let’s start with a simple problem - in physics maybe. Let’s say we have a car, as all good physics problems do.

You’re driving at a set initial speed, and a set acceleration. At what time would you have travelled exactly 15 ft?

Let’s look at it in terms of "a phase space". I have a velocity-time graph down here:

                                                                                                                                

Linear Velocity-Time Graph

Nothing very exciting, but it’s a useful analogy. Here, the two variables involved (more on that later), are effectively the speed and the time. What you want to know are the success cases (totally a technical term), where the car travels 15 ft, no more, no less. How could you do that? You could realize that in the phase space that is the graph (this may be technically inaccurate, it’s just to make a point), the point at which the area under the graph reaches 15, the car has travelled 15 ft, and that is the point of time we wanted to find.

What we’ve done here is interesting. We didn’t solve the problem from the actual physical set up. We weren’t thinking through a video of the speedometer, the accelerometer, or even the car really. Instead, we were thinking about a representation of this scenario, in terms of lines and shapes. In the abstract sense, these two are equivalent, but directly or physically, they are not, however, they retain analogous properties. For example, the car must have been moving for a certain amount of time for it to have travelled 15 ft, and a pointer on the graph too, must have reached a certain horizontal coordinate, to have an area of 15, and these quantities can be translated between each other.

Abstract

The most important idea, however, is that we graphed the variables we know. Though the car doesn’t really look like this set of black lines and axes we drew, we symbolically represented the situation by drawing this geometry as a representation of various values of the variables in our control, and the outcome at each of them.

Phase Space is a method of looking at your problem in terms of what you can change, to understand the pattern so that you can ensure the best result. The really cool thing about Phase Spaces is that once the representation has been constructed, properties of geometry can be used to find the answers you want, like how we used the fact that simple areas are proportional to their sides in solving the car problem. This may seem rather random - definitely unrelated to ML - but you’ll see why this is such a useful analogy in a minute.

So, let’s talk about intelligence. As we discussed back in our blog on guessing, it is really based on the idea of refining your guesses, further and further. But how do you decide in what way to push your guesses, which way to bias them? Data. Machine Learning data, in particular, is categorized into what are called features - check out our blog on Philosophy, ML and Physics to really understand that.

To summarize, however, features are attributes of the problem. For example, if you are trying to create an algorithm to classify a word as to whether it is happy or sad. Maybe what you could do, is have each letter be a particular feature. That’s probably not accurate by linguistics, but this works for an example.

Cost Functions

But to make this easy on us, let’s use a more numerical problem. Let’s take an image, which is made of only black and white pixels. We are trying to classify the image to see if there’s a cat in it or not (I don’t know why I put cat analysis in so many articles so far). We will have only 1 feature to start with, to represent our pixel because the image is 1x1, and the feature can be either 1 or 0, for white and black. How do I figure out if this image resembles a cat? (of course one pixel will never look like a cat, the extreme reduction in size is to simplify what will follow)

Earlier, we discussed the idea of a cost function. In Machine Learning, the cost function essentially tells you how wrong your model is. By trying to minimize the cost function, you can improve the model. But how do you do that? By changing the values of the various parameters, which is basically how important each feature is, - Machine Learning problems can have astronomical scales as far as numbers of features go, so don’t expect to be able to visualize so easily - we try to keep changing the model, until it reaches the minimum cost. However, all this sounds really complicated. And what does this have to do with Phase Spaces?

So a Phase Space depicts multiple variables, and geometry and the properties of that space are then used to find the best solution for a particular value of one ‘output’ variable.

Let’s look at this in the context of ML. Instead of velocity and time, let’s plot ML parameters and the cost function.

Back to our example. So let's consider one parameter called θ, and J(θ), which is the cost function. So let’s plot that in a graph:

As you can see, at the red value of the parameter, the cost function is at minimum. In other words, the model that the algorithm came up with is the best it can do. We’ve done it!

But you obviously don’t want to be involved every time - what if you need to move every parameter value until the cost function finally seems minimal? There’s no point in it being Machine Learning then. Here’s the amazing thing that ML does to solve this: Math.

 

Conclusion

In fact, it’s all based on the idea we took on when we were discussing Phase Spaces. Since we’ve converted the problem to a geometric representation, we can try to solve the problem by the properties of shapes and topologies.

That's where we're going to stop this time. In the next half of this series, we'll discuss what these properties are, how we can use them, and why they're useful to a concept called Gradient Descent. We'll also see which person reaches the bottom of the hill first: the normal man, and the drunken one. But more on that next time!