Selecting the optimal $𝑘$ in $𝑘$-means

Look for diminishing change, and stop before it’s not worth it

$𝑘$-medoids

Almost $𝑘$-means, but the center points must be elements of the datapoints. ($𝑐\subseteq \mathrm{𝒳︀}$)

$𝑘$-medians

Like $𝑘$-medoids, but we are minimizing the ${𝐿}^1$ (taxicab) distance instead of ${𝐿}^2$ (euclidean).

We define the median of a finite set to be the middle element if the set has odd number of elements, and the average of the two middle elements if it has an even number of elements.

Algorithm for the $𝑘$-medians problem is almost the same as the $𝑘$-means algorithm, but updating clusters is done via the median rather than the centroid.

Non-linear dimensionality reduction

• Transforming high-dimensional space into low-dimensional space
• Can be used for e.g. visualization
• Often unsupervised

Dimensionality reduction:

Given a dataset $\mathrm{𝒳︀}=\left\{{𝑥}_1,\dots ,{𝑥}_{𝑛}\right\}\subset {ℝ}^{𝐷}$ and a dimension $𝑑\ll 𝐷$

Find an embedding $\mathrm{𝒴︀}=\left\{{𝑦}_1,\dots ,{𝑦}_{𝑛}\right\}\subset {ℝ}^{𝑑}$ such that is preserves significant structures of $\mathrm{𝒳︀}$