Machine Learning

Machine learning lets analysts find patterns in large, complex and unstructured data sets that traditional regression handles poorly. This module distinguishes supervised, unsupervised, deep and reinforcement learning, explains overfitting and how validation and regularisation control it, and matches the main algorithms (LASSO, support vector machines, KNN, decision trees, ensembles and random forests, PCA and clustering) to the investment problems they suit. Level II questions emphasise choosing the right method and interpreting its output.

Start this lesson free Practice questions

What you will learn

1. Distinguish supervised learning, unsupervised learning and deep learning and the problems each addresses.

Supervised learning uses labelled data: regression for continuous targets, classification for categorical ones.
Unsupervised learning has no target: dimension reduction and clustering.
Neural network nodes apply an activation function to a weighted sum of inputs plus a bias; deep learning stacks many hidden layers.
Deep learning excels at images, speech and text but needs large data sets and is hard to interpret.
Reinforcement learning trains an agent to maximise reward through trial and error.

2. Explain overfitting, the bias-variance trade-off, and methods to control it such as cross-validation and regularisation.

Split data into training, validation and test samples; judge models out of sample.
Overfitting: low training error, high validation error (high variance error); underfitting: high error everywhere (high bias error).
Out-of-sample error = bias error + variance error + base error.
As complexity rises, bias error falls and variance error rises; choose complexity at the minimum validation error.
Control overfitting with regularisation, k-fold cross-validation, more data and fewer features.

3. Match supervised algorithms (penalised regression, support vector machines, k-nearest neighbours, decision trees, ensembles and random forests) to suitable tasks.

LASSO minimises SSE + lambda x sum of |coefficients|, shrinking some coefficients to zero (feature selection).
SVM finds the maximum-margin boundary between classes; soft margins allow some misclassification.
KNN votes among the k nearest training points; results depend on k, scaling and the distance measure.
CART produces interpretable if-then splits but overfits without pruning or depth limits.
Ensembles (voting, bagging) and random forests reduce variance error and improve accuracy at the cost of interpretability.

4. Match unsupervised algorithms (principal components analysis, k-means and hierarchical clustering) to suitable tasks.

PCA creates uncorrelated components; eigenvalues measure variance explained, eigenvectors the weights.
Variance share = eigenvalue / sum of eigenvalues; keep enough components to reach the target share (often 85%-95%).
k-means needs k in advance, assigns points to the nearest centroid and iterates; sensitive to starting points and scaling.
Hierarchical clustering (agglomerative or divisive) needs no preset number of clusters and is shown as a dendrogram.
PCA trades interpretability for compression; clustering finds groups without labels.

Module at a glance

Supervised learning predicts a labelled target (regression or classification); unsupervised learning finds structure without one.
Deep learning stacks layers of nodes that apply activation functions to weighted sums; reinforcement learning maximises reward by trial and error.
Overfitting shows as a large training-validation gap (variance error); underfitting as high error everywhere (bias error).
Control overfitting with regularisation, k-fold cross-validation, simpler models and more data.
LASSO adds lambda x sum of |coefficients| to SSE and can drop features; random forests and other ensembles cut variance error.
KNN votes among nearest neighbours; SVM finds the widest margin; CART gives interpretable splits but overfits.
PCA keeps components by eigenvalue share; k-means needs k in advance; hierarchical clustering yields a dendrogram.

This free module has 4 lessons, 16 practice questions with full explanations.

Start the lesson