Unsupervised Anomaly Detection in OpenStack Logs via Fine-Tuned RoBERTa Embeddings

Abstract

As cloud computing infrastructures increasingly depend on reliable operation, proactively detecting anomalies in system logs becomes indispensable. Conventional log analysis techniques often produce high false-alarm rates and exhibit limited semantic understanding. To address these limitations, we developed an unsupervised anomaly detection framework that leverages fine-tuned RoBERTa-base model embeddings to capture contextual patterns within OpenStack log event sequences. We apply a crucial filtering step to remove high-frequency, non-discriminatory events, ensuring our models learn from nuanced contextual signals rather than simple indicators. From these refined sequences, we construct a custom vocabulary and fine-tune RoBERTa with Parameter-Efficient Fine-Tuning (PEFT) using LoRA. These contextualized embeddings inform unsupervised classifiers, including Isolation Forest and One-Class SVM, trained solely on normal data. Our approach demonstrates excellent and robust performance on a holdout test set (Anomaly F1-Score up to 0.97), significantly outperforming traditional LSTM-based baselines on the same task. These results demonstrate that contextualized transformer embeddings provide a powerful and resilient foundation for log-based anomaly detection, reducing false alarms and improving detection accuracy in complex cloud environments.