A Lightweight 3D Convolutional Network for Hyperspectral–LiDAR Patch Classification

Abstract

We propose a simple yet effective three-dimensional convolutional neural network (3D-CNN) for urban land-cover classification using co-registered hyperspectral imagery (HSI) and LiDAR data. The network treats the entire spectral-LiDAR stack as a three-dimensional volume and uses a series of 3×3×3 convolutions to capture both spectral and spatial context simultaneously. LiDAR elevation data is added as an extra channel in the input. During preprocessing, each hyperspectral band and LiDAR DSM are normalized to zero mean and unit variance. Training uses small local patches (P×P) centered on labeled pixels, with random flips and 90-degree rotations, called dihedral augmentation, applied across all channels. To address class imbalance, inverse-frequency class weighting and label smoothing are included in the cross-entropy loss. Evaluation on the Houston2013 dataset shows that the model achieves high accuracy, a single model reaches an Overall Accuracy (OA) of about 0.90 and an Average Accuracy (AA) of about 0.92 over five runs. An ensemble of five runs improves these results to OA ≈ 0.912, AA ≈ 0.928, and a kappa coefficient (κ) of approximately 0.904. Classes with distinctive spectral and spatial signatures, like water, synthetic grass, and tennis courts, reach nearly 100% recall. Meanwhile, classes with similar appearances, such as highway and road, show higher confusion, with highway recall around 46.9%. These results confirm that combining spectral and three-dimensional structural information significantly enhances accuracy in urban classification.