Time series anomaly detection with a human-in-the-loop [PyCon DE & PyData Berlin 2024]

Summary

The video discusses a scalable anomaly processing initiative using Python and Azure Machine Learning, focusing on expert feedback in analyzing flagged anomalies from time series data. The workflow involves experts classifying anomalies in Label Studio, which integrates seamlessly with Azure functions to update data in real-time. The speaker emphasizes the extensive use of Python in their project and highlights the strengths of Azure Machine Learning Studio while acknowledging concerns about future maintenance. Despite not currently using real-time data applications, the initiative showcases a robust infrastructure for handling anomaly detection and processing.

Takeaways

• 😀 The project focuses on scalable anomaly detection in time series data, allowing experts to evaluate and classify anomalies in real-time.
• 🔗 Experts use Label Studio to review anomaly candidates flagged by an algorithm, enhancing the accuracy of anomaly classification.
• ⚙️ Feedback from experts triggers a web hook that updates the summary project in real-time, ensuring timely processing of feedback.
• 🐍 Python is the primary programming language used throughout the project, comprising approximately 90% of the relevant code, including data handling and machine learning pipelines.
• 📊 The anomaly detection workflow includes the ability for experts to adjust anomaly boundaries based on their classifications, allowing for more precise evaluations.
• 🔄 Label Studio serves as an open-source tool, but much of the project’s functionality is built with custom Python code tailored to specific needs.
• ☁️ Azure Machine Learning Studio is integrated into the project for its ease of use, though there are concerns about the clarity of future feature support.
• 🛠️ The team employs various data processing libraries, including Dask for background processing and scikit-learn for building transformation pipelines.
• 📈 While the current department lacks real-time data applications, the company has the capacity to explore such use cases in other divisions.
• 📅 Future exploration of Snowflake for large time series data is considered, but currently, there are no immediate plans for its implementation.

Q & A

What is the primary goal of the scalable anomaly processing initiative discussed in the video?

The primary goal is to efficiently process and classify anomalies in time series data, allowing experts to provide feedback and improve the accuracy of anomaly detection.

How does the feedback mechanism for anomaly classification work?

When an expert reviews an anomaly and provides feedback, it triggers a webhook that activates an Azure function, transferring the feedback to the summary project in real-time.

What role does Python play in the project described in the video?

Python is extensively used throughout the project, with about 90% of the relevant code written in Python, including data processing, machine learning pipelines, and the backend of Label Studio.

What does the expert interface in Label Studio allow users to do?

The expert interface allows users to view time series data, classify detected anomalies, and adjust the boundaries of anomalies based on their assessments.

What are some of the challenges associated with using Azure Machine Learning Studio mentioned in the video?

Challenges include ambiguity regarding the maintenance of new features and which parts of the service will continue to be supported.

Why is real-time data processing not currently utilized in the department discussed?

The department currently does not have any real-time data applications, but other departments within the company may have relevant use cases.

What libraries are mentioned for data processing in the project?

The project uses libraries such as Scikit-learn for preprocessing data and integrating various transformation techniques.

What is the relationship between Label Studio and open-source software?

Label Studio is an open-source tool used in the project, allowing for customization and integration of specific functionalities tailored to the team's needs.

Are there any plans to integrate Snowflake for large time series data?

Currently, there are no immediate plans to integrate Snowflake, but the team is open to exploring it if a compelling use case arises.

How do experts classify the anomalies detected by the algorithm in Label Studio?

Experts can classify anomalies into three categories: relevant anomalies that require attention, irrelevant anomalies, and known anomalies that do not need further investigation.