Semi-automatic annotation of emergency events reported on X based on machine learning and evolutionary computing

Abstract

Text corpora related to citizen incidents are scarce and the annotation process, to determine whether they are emergencies, is usually done manually by human annotators. This annotation method yields acceptable results, but it is slower, more expensive, and only feasible for data sets that are small in volume or do not require real-time processing. Annotating emergency corpora with human annotators, which require immediate actions by their nature, does not seem to be ideal, especially when they are reported on social networks such as X. This article proposes an approach for the semi-automatic annotation of emergency events reported on X, based on machine learning and evolutionary computation. The CRISP-DM methodology was applied with six stages: problem understanding, data collection and understanding, modeling, validation and deployment. The computational experiments show a good performance when the linear support vector classifier (LSVC) is used as a model, whose hyper-parameters are optimized with an evolutionary computation algorithm. Average values of 0.976 and 0.963 for the F1-Score, as well as 0.96 and 0.97 for the Matthews correlation coefficient, were obtained to assign the emergency tag to +170 thousand tweets for the binary and multiclass classification, respectively. It is concluded that it is possible to have an efficient approach to assign the emergency label to the texts published on the social network X, thanks to the optimization of the values of the parameters of the machine learning classification models.