Outlier Detection: Isolation Forests and the Local Outlier Factor
Prof. Dr. David B. Blumenthal · 6 concepts · 10 questions
Concept 1 / 6
The What, Why, and Types of Outliers
Outlier detection is the process of identifying data points that deviate from normal data. The hard part is the key question: what counts as 'normal'? Depending on the data, different notions of outlierness apply — e.g. deviation from the center, or deviation from a trend.
Sources of outliers:
- Errors in data collection or entry
- Rare but important events (fraud, system failures)
- Novel or emerging patterns
Why it matters:
- Noise removal — strip out outliers caused by noise or technical errors before analysis → better data quality and more robust results.
- Early detection of critical issues — fraud detection, quality control, cybersecurity, healthcare, finance.
Types of outliers:
| Type | Definition | Example |
|---|---|---|
| Point | A single instance differs from the rest | Sudden temperature spike |
| Contextual | Anomalous only in a specific context | Warm winter day |
| Collective | A group of related points jointly deviate | Sequence of unusual logins |
Global vs. local: a global outlier deviates significantly from the entire dataset; a local outlier deviates only within its neighborhood/cluster and may look perfectly normal globally.
Challenges: curse of dimensionality (distances lose meaning in high dimensions); differing feature scales (need normalization) and within-class variability; noise and missing values; and no clear definition of 'normal', which can drift over time (concept drift).
Four families of detection techniques:
| Family | Core idea | Example |
|---|---|---|
| Statistical | Assume a distribution (e.g. Gaussian); flag low-probability points | Z-scores |
| Distance-based | Outliers lie far from their nearest neighbors | -NN |
| Density-based | Outliers lie in low-density regions vs. neighbors | LOF |
| Model-based | Build a model; outliers fit poorly or deviate from learned behavior | Isolation forests, autoencoders, regression |