Chapter 01

Machine Learning for Remote Sensing

Machine learning transforms satellite data into actionable intelligence — automating classification, feature extraction, and prediction at planetary scale.

Python (scikit-learn, TensorFlow, PyTorch)Google Earth EngineQGIS Orfeo ToolBoxRasterioLabel Studio

10m globalDynamic World↗ Near-real-time land cover

1.3BMS Building Footprints↗ AI-mapped globally

100MPrithvi Parameters↗ Foundation model

Theory & Foundations

ML algorithms learn patterns from labeled training data to classify, segment, and detect features in geospatial imagery far faster and more consistently than manual methods.

Traditional ML: Random Forest and XGBoost handle tabular spectral features well. Train on pixel spectra → predict land cover classes. Feature importance reveals which bands matter most.

Deep Learning: CNNs automatically learn spatial features from image patches. U-Net architecture dominates semantic segmentation — encoding spatial context then upsampling to full resolution.

Transfer learning: Fine-tune pre-trained models (ResNet, EfficientNet) on satellite data. Foundation models like IBM/NASA Prithvi and Microsoft SatMAE learn general representations from millions of satellite images.

Training data quality is the #1 factor in ML model performance. Garbage in = garbage out. Label verification, class balancing, spatial autocorrelation in train/test splits, and augmentation are critical.

In-Depth Coverage

Algorithm Selection

Choosing the right ML approach for geospatial problems.

Random Forest: Robust, fast, works with small datasets (1000+ samples)
XGBoost: Higher accuracy, needs tuning, good for competitions
CNN: Learns spatial patterns, needs GPU and 10K+ labeled pixels
U-Net: Semantic segmentation, best for mapping applications
Vision Transformers: Latest architectures, massive compute needs

💡 Pro Tip: Start with Random Forest as a baseline. If accuracy is insufficient, move to CNN/U-Net with more training data.

Training Data Best Practices

Data quality determines model quality.

Spatial cross-validation: Don't mix nearby pixels in train/test
Class balance: Over/undersample or use weighted loss
Augmentation: Rotation, flip, brightness for small datasets
Active learning: Model identifies uncertain areas for labeling
Label Studio / CVAT: Open-source annotation tools

Foundation Models

Pre-trained models on massive satellite datasets.

Prithvi (IBM/NASA): 100M parameter model for HLS data
SatMAE (Microsoft): Self-supervised on Sentinel-2
Clay Foundation: Open-source geospatial foundation model
Fine-tune on small labeled datasets for specific tasks
Zero-shot: Classify without task-specific training

Key Techniques

Feature engineeringModel trainingHyperparameter tuningSpatial cross-validationModel deployment

Data Visualization

ML Algorithms in Remote Sensing Publications (%)

Random Forest35

CNN/U-Net30

XGBoost15

SVM10

Transformer10

ML Pipeline

🏷️DataCollect, label, split training data
🧠TrainRF → CNN → U-Net progression
📊EvaluateConfusion matrix, F1, IoU
🚀DeployApply to full study area

Practical Workflow

Collect and label training data with GPS/imagery
Split data: train/validation/test (spatial splitting)
Feature engineering: spectral indices, texture, context
Train model: Random Forest → CNN → U-Net progression
Tune hyperparameters using validation set
Evaluate: Confusion matrix, F1-score, IoU per class
Apply model to full study area
Post-process: smoothing, minimum mapping unit

Software & Tools

TensorFlowFreeFramework — Deep learning model training

Label StudioFreeTool — Training data annotation

Video Tutorials

▶ Deep Learning for Satellite ImagerySatellite Image Deep Learning · YouTube · 60 min ▶ Random Forest Classification in GEEGoogle Earth Engine · YouTube · 40 min

Real-World Application

Google's Dynamic World provides near-real-time 10m land cover maps globally using a deep learning model trained on 5 billion Sentinel-2 pixels, updated every 5 days.

Real-World Projects

Microsoft AI Building Footprints📍 Africa

DL mapped 516M buildings from satellite imagery.

Impact: Baseline data for humanitarian response and development planning.

Case Study

Microsoft's AI for Earth program used deep learning to map every building in Africa (516 million buildings) from satellite imagery, providing baseline data for humanitarian response.

Problem-Based Learning

P1 Map building footprints across an entire country from satellite imagery

200,000 km² country with diverse urban/rural areas. Manual digitization would take years.

Solution: Train U-Net on 5000 manually digitized buildings. Apply to country-wide Sentinel-2 10m imagery. Post-process with morphological operations and size filtering.

TensorFlowSentinel-2PythonQGIS

✅ 2.3 million buildings mapped in 72 hours. 91% IoU accuracy. Database used for census, disaster planning, and tax assessment.

Knowledge Check

1. Best architecture for pixel-wise classification?

2. Why use spatial cross-validation?