Concept 1 / 1
Key Concepts to Memorize
What is a graph (network)?
- Mathematical object G = (V, E): V = nodes, E ⊆ V×V = edges
- Attributes can be stored at both nodes and edges
- Excellent for data where information is encoded in connections
Biomedical graph examples:
- Protein-Protein Interaction (PPI) networks — nodes = proteins, edges = physical binding
- Gene regulatory networks
- Disease-disease networks
BIONETS Lab — 3 research areas:
1. Algorithmic network medicine — mine omics data for molecular disease mechanisms
2. Robust and reproducible AI — algorithms/standards for robust biomedical AI
3. Privacy-preserving federated biomedical AI — decentralized AI for cross-institutional studies on sensitive data
The core problem with PPI prediction benchmarks:
- Many published deep learning results are inflated due to data leakage
- Naive random data splitting introduces 2 types of "shortcuts":
- Shortcut 1 (topology-based): model memorizes which proteins appear only in positive pairs in training → predicts "interaction" for all their test pairs regardless of partner
- Shortcut 2 (sequence similarity): proteins in training and test sets are too similar → model exploits similarity instead of learning true interaction rules
Data leakage (definition from Blumenthal):
> "Performance scores for an ML-based predictor f are inflated due to data leakage if f uses illegitimate information to achieve the reported scores, that is, information in the training data that generalizes to the test data but not to the inference-time data."
Two intended use cases → different evaluation protocols needed:
| Use case | Appropriate split | Result |
|---|---|---|
| Proteins WITH well-characterized homologs (in-distribution) | Random splitting is fine | Deep learning works |
| Proteins WITHOUT well-characterized homologs (out-of-distribution) | Hard split required | No tested DL model outperforms random guessing |
Key insight: High accuracy scores in the literature were often due to shortcuts, not genuine learning. When hard splits are used, models drop to ~random (0.50) performance.
Solution (DataSAIL): Algorithm for leakage-reduced data splitting + guidelines to avoid data leakage in biological ML.