… continuing from the previous lecture

A manifold is an object (like curved surface, embedded into a higher dimension $𝐷$) that locally resembles ${ℝ}^{𝑑}$

t-SNE

Student t-distributed Stochastic Neighbor-Embedding

We take the input vector $𝑋$, and replace it with a matrix $𝑃$ with elements ${𝑝}_{𝑖𝑗}$ that represents the probability of an element ${𝑥}_{𝑖}$ choosing ${𝑥}_{𝑗}$ as its neighbor. This creates the affinity matrix.

Quick note: unsupervised learning is pretty much about treating the label space as another random variable (as we do not know it)

We try to minimize the cost between the embedding’s and the input’s affinity.

The pipeline:

1. Create the affinity matrix from $\mathrm{𝒳︀}$
2. Create embedding space $\mathrm{𝒴︀}$
3. Compute affinity matrix of $\mathrm{𝒴︀}$
4. Get the cost, $\text{KL}\left(𝑃,𝑄\right)$
5. Optimize $\mathrm{𝒴︀}$ based on the cost
6. Go back to step 3.