A robust machinery action recognition in construction using a mixed graph convolution block

Abstract

Existing computer vision-based approaches struggle to identify machinery actions due to the challenges posed by environmental complexity and various obstructions in construction. This study introduces a novel two-stage framework that benefits from a newly proposed Residual Fusion Graph Convolution Network (RFGCN) to classify machinery actions with enhanced robustness and accuracy. The framework first extracts key machinery components from video data, subsequently transforming them into a graph-based representation. This spatio-temporal graph is then fed into the RFGCN model, specifically designed to overcome issues like partial obstructions and missing information common in busy construction sites. Experimental evaluations reveal the method's high efficacy, achieving an accuracy of up to 96.4 % and outperforming state-of-the-art. Additionally, the proposed RFGCN model achieved state-of-the-art performance on four established benchmark datasets for graph classification using spatial data only. These results suggest the potential of the proposed framework in facilitating the transition towards more intelligent and automated construction sites.