Railway track surface defect 3D reconstruction and relocalisation through cross-modality fusion

Abstract

Accurate localisation of track surface defects is critical for safe and cost-efficient railway maintenance. However, current reinspection workflows using portable non-destructive testing devices face positioning uncertainty due to the absence of spatial anchors. This research proposes a relocalisation workflow that requires neither ground infrastructure nor depth sensors. In the inspection stage, a three-layer 3D multimodal track model is reconstructed from digital images and thermograms using structure-from-motion and 2D-2D cross-modality registration. During the relocalisation stage, perspective-n-point is employed for 3D–2D spatial registration, projecting the 3D model onto on-site images to achieve accurate defect localisation without reinspection. The method was evaluated using nine shooting positions on seven artificial defects. Projected defects demonstrated improved perceptual separability (edge strength and contrast-to-noise ratio) over on-site images. At optimal imaging positions, the relocated defect centroids exhibited displacement errors range from 0.34 mm to 3.62 mm. The projection accuracy was further evaluated by Intersection over Union values ranging from 0.961 to 0.993 and Structural Similarity Index Measure ranging from 0.845 to 0.968.