What *are* manifolds? I started down this rabbit hole because of Chris Colah’s excellent Neural Networks / Functional programming blog post:

At their crudest, types in computer science are a way of embedding some kind of data in n bits. Similarly, representations in deep learning are a way to embed a

data manifoldin n dimensions.

So the problem is that people talk about them even when you don’t think you need to care about topology, or even really know what topology means.

The following is a bit of a dive into examples of manifolds with the intention of better understanding what a manifold is.

## Starting somewhere

Opening quote from Wikipedia:

In mathematics, a manifold is a topological space that locally resembles Euclidean space near each point.

Without knowing exactly what topological and Euclidean spaces are that’s hard to understand. Let’s try *simple* Wikipedia:

A manifold is a concept from mathematics. Making a manifold is like making a flat map of a sphere (the Earth).

Ok, kinda. Except this is nicely confused by further reading the Wikipedia page which has:

A ball (sphere plus interior) is a 3-manifold with boundary. Its boundary is a sphere, a 2-manifold.

So, to me, this makes no sense of creating a manifold (a flat map) from a manifold (a sphere is a 2-manifold). Why create a manifold if you’ve already got one? (My thinking here is wrong, but I’m trying to explain all my incorrect thinking as I go and clear it up at the end)

### Bounds checking

Some kind of useful quotes from the section on boundaries:

A piece of paper is a 2-manifold with a 1-manifold boundary (a line)

…

A ball is a 3-manifold with a 2-manifold boundary (a sphere)

## Show me the money

‘Simple’ examples are usually the way for me out of confusion. The list of manifolds helps a lot:

- are
**all**manifolds, so a line is a 1-manifold - A x,y 2D plane is a 2-manifold
- All spheres are manifolds… stop there, I’m confused

### Spheres

This really gets me. Why do we need to create maps of the world if it’s already a manifold. The simple Wikipedia explanation makes sense, i.e. it’s obvious that we want to create a flat map of the world. It also makes sense that where as a triangle on the globe doesn’t add up to 180 degrees (bad) does nicely add up to 180 degrees on a flat map (good). But why, if the earth is *already* a manifold do we want to create another manifold from that. Isn’t being a manifold good enough? Some people are never satisfied.

### n-manifold

A sphere is a 2-manifold, so although it is a 3D object, close to any one point it looks like a 2D *grid*.

The important part is the *n*-manifold. This is what cleared it up for me, you don’t care about creating one manifold from another.

An n-manifold resembles the nth dimension near one point. So a flat map is a 2-manifold and the earth is a 2-manifold because they both represent a 2D grid near one point.

### Manifold Hypothesis

In another post from Chris Colah, he talks explicitly about manifolds. But he talks about them assuming that you know what they are:

The manifold hypothesis is that natural data forms lower-dimensional manifolds in its embedding space.

`*scream*`

### Nash equilibria

Although John Nash is most famous for his Nash equilibrium, but it a lot of his most important work was to do with manifolds:

His famous work on the existence of smooth isometric embeddings of Riemannian manifolds into Euclidean space.

### Differentiation

Interestingly differentiating a curve at a point on that curve gives you the slope of the flat line at that point. So there’s a connection between differentiation and manifolds.

There’s a Calculus on Manifolds book that looks interesting, taken from this talk on The simple essence of automatic differentiation. Then there is a link back to Chris Olah’s NN/FP post based on this Conal Elliot’s Automatic Differentiation paper.

## Euclid forgive me

To be honest even Euclidean space gets me confused. That’s basically just which I can understand better. But is there a specific reason for using Euclidean? Is there some extra property of Euclidean space that isn’t inherent in ? There’s a fundamental principle of parallel lines not converging in Euclidean space, but then if a manifold is in Euclidean space, how can a sphere be a manifold? It doesn’t explain why we want to convert one manifold into another (as in the spheres section above), or even really know what topology means.

### Euclid’s grid

I keep thinking of Euclidean space as basically everything. But it’s not – it’s a n-dimensional grid (with infinite points as it’s the real number line in each direction) and it has straight edges. So a ball isn’t in Euclidean space, or “isn’t Euclidean”, I’m not sure which.

In image analysis applications, one can consider images as functions

on the Euclidean space (plane), sampled on a grid

— Geometric deep learning paper

### Data manifolds (back to Chris)

Perhaps data manifolds are the structure of the data. For example as above an image data is a 2D Euclidean grid.

### What’s not Euclidean

For instance, in social networks,

the characteristics of users can be modeled as signals on the

vertices of the social graph [22]. Sensor networks are graph

models of distributed interconnected sensors, whose readings

are modelled as time-dependent signals on the vertices.…

In computer graphics and vision, 3D objects are

modeled as Riemannian manifolds (surfaces) endowed with

properties such as color texture.

— Geometric deep learning paper

Manifolds don’t have to Euclidean! But at one particular point on a manifold surface it is approximately a grid in the local area. So this holds if it is Euclidian, because locally on a sheet of paper it’s Euclidean as the sheet of paper is already Euclidean.

But certainly I like the explanation that 3D graphics are manifolds. So there’s nothing special that makes the earth a manifold it’s just one example of one. So I think this means that any 3D shape is a manifold.

So what’s not Euclidean *and* not a manifold?

### What’s not a manifold?

That dear reader… is an exercise for you.

## What did this get us?

- Euclidean space means a grid (duh)
- An image of pixels is a 2D grid – possibly the data manifold that Chris Olah was referring to
- A 3D graphic (or the earth) is a 2-manifold. These are not Euclidean but are 2D Euclidean (grid shaped) close to a point on their surface
- When your data is 3D graphics – that is your data manifold