NODES 2024 - Enhancing Business Process Anomaly Detection With Neo4j and Graph Neural Networks

Summary

This research presentation explores anomaly detection using neural networks, focusing on graph-based methods. Various encoder-decoder combinations, including dimension-increasing autoencoders, were tested on artificially injected anomalies. Two explanation methods, GNN Explainer and GraphLine, were employed to explain the root causes of anomalies. Key findings include the performance of different graph convolutional architectures and decoders, with Chebyshev convolution-based encoders and convolution-based decoders showing the best results. The study highlighted the importance of transition-based fields in anomaly detection and provided valuable insights into feature importance through graph-based explanations.

Takeaways

• 😀 Anomal Nets is a research project developed by Nex4, focused on detecting anomalies in business process management (BPM) using machine learning and graph-based models.
• 😀 The project leverages graph neural networks (GNNs) and other AI techniques to detect anomalies in business processes represented as graphs, where nodes are states and edges are transitions.
• 😀 Unsupervised learning models, such as Isolation Forest, were initially used for anomaly detection in BPM data, followed by semi-supervised models for improved performance.
• 😀 The incorporation of natural language features, such as word embeddings, enhances the model's ability to detect anomalies, especially in the 2017 dataset.
• 😀 A focus on edge feature reconstruction (detecting anomalies in transitions between states) rather than node features led to improved anomaly detection.
• 😀 Multiple graph convolutional network (GCN) architectures, including GAT, ECC, and GCN, were tested for anomaly detection, with ECC-based decoders providing the best results.
• 😀 Graph neural network explainability methods, such as the GNN explainer and graph-line approach, were employed to offer insights into the root causes of detected anomalies.
• 😀 The study used both encoder-decoder architectures and various graph convolution operators to optimize anomaly detection performance, with certain configurations outperforming others.
• 😀 F1 scores were used as a metric for model performance, with the convolution-based decoder and a shift convolution-based encoder showing the highest F1 scores.
• 😀 The research highlights the importance of transitions and states in detecting anomalies, where user-related fields often have less impact compared to state and transition fields.
• 😀 The team at Nex4 and the Turkish research council (TUBITAK) supported the development and testing of the anomaly detection framework, with ongoing research to improve scalability and real-world applicability.

Q & A

What is the main focus of the Anomal Nets project?

The main focus of the Anomal Nets project is anomaly detection in business processes, leveraging machine learning techniques, particularly graph neural networks, to optimize workflows and identify deviations in business process traces.

Which machine learning techniques were explored in the project for anomaly detection?

The project explored various techniques including isolation forests, unsupervised graph neural networks (GNNs), natural language processing (NLP) with word embeddings, and edge feature-based methods to enhance anomaly detection in business process workflows.

How were business process traces represented in the Anomal Nets project?

Business process traces were represented as graph data, where event logs were transformed into graph structures for analysis and anomaly detection using graph neural networks.

What is the significance of using graph neural networks (GNNs) in this project?

Graph neural networks (GNNs) are significant in this project because they enable the detection of anomalies in complex relationships between various entities in business process data, allowing for a more nuanced and accurate identification of deviations in workflows.

What datasets were used to test the anomaly detection methods?

The project used proprietary business process management (BPM) data from Nex4 as well as public benchmark datasets, such as those from the Business Process Intelligence Challenge (2012 & 2017).

What role did explainable AI (XAI) play in the project?

Explainable AI (XAI) played a crucial role by providing transparency into the anomaly detection process, helping to explain the root causes of detected anomalies through methods like the GNN Explainer and GraphLime, which highlight key features influencing the predictions.

How was the performance of anomaly detection models evaluated in the project?

The performance of anomaly detection models was evaluated using a custom success rate metric, focusing on detecting anomalies across different percentiles, such as the 95th and 90th percentiles, to assess the models' effectiveness.

What were the key findings regarding the encoder architectures used in the study?

The study found that the **chebyshev convolution-based encoder** performed the best in terms of detection accuracy, with an F1 score of 49 at the 95th percentile, demonstrating its strong performance for anomaly detection in business process traces.

What were the key findings regarding the decoder architectures used in the study?

The study showed that the **convolutional-based decoder** outperformed others, achieving the highest F1 score of 49 at the 95th and 90th percentiles, indicating its superior ability to decode and explain anomalies in the data.

What insights were gained from the use of GraphLime and GNN Explainer methods?

GraphLime and GNN Explainer methods provided insights into the relative importance of different features in predicting anomalies. For example, the GNN Explainer highlighted transition-based features as most important, while the GraphLime approach showed that certain user-related fields were less likely to be anomalies.