Exam will most likely be multiple choice questions , but cover the same topics as the take home exam

Classification and Regression Trees

Tree-based methods

Simple but powerful, and human interpretable

We focus on CART trees.

We subdivide the space along one coordinate at some threshold. For example, in the IRIS dataset, one coordinate could be “petal length”.

While building the tree:

• If the minimum threshold is met, terminate

  • Minimum as of the size at the tree is allowed to split into categories

• If the classification is pure, terminate

  • Which means at this step, you only have one class left, there is no point in splitting

• Otherwise split by finding the best dimension (parameter) and threshold value

  • see slides for the math
• Repeat for the now created subnodes

Ooh, they have an emoji for the ‘left as an exercise to the reader’

Ensemble Methods

“You train multiple models and then combine them”

If independent classifiers are more often right than wrong AND they make independent errors, then their ensemble average the errors out, and the majority vote tends to outperform the individual classifiers

Limitations

• Single decisions are instable

  • Highly depend on the training
  • Small change can lead to a completely different tree
• Often not accurate

Random Forests

Growing B trees using different subsets of the data, then combine them to give the final prediction (i.e. it’s an ensemble classification tree)