How to create custom ASIM parsers for your log sources

Summary

This video highlights the significance of the Advanced Security Information Model (ASIM) in Microsoft Sentinel for normalizing custom network data. It illustrates how to effectively parse and extract key information from custom network logs, such as network protocols and source IP addresses, using functions like parse and split. The presenter emphasizes the importance of adhering to ASIM's preferred field names for consistency and usability. Finally, the video showcases how to integrate custom parsers into the ASIM network session, enabling seamless access to various underlying data sources.

Takeaways

• 🔒 ASIM (Advanced Security Information Model) serves as a critical framework for normalizing data within Microsoft Sentinel.
• 📊 Custom network logs can be ingested with specific fields like 'event info' and 'event message' for better data management.
• 🛠️ The 'parse' function is essential for extracting key components from log entries, creating a structured key-value format.
• 🔍 Regex parsing can be utilized for more complex data extraction needs beyond simple parsing techniques.
• ✂️ The 'split' function effectively breaks down text entries using delimiters, organizing data into clear, usable arrays.
• 🏷️ Renaming parsed fields to match ASIM's preferred conventions improves clarity and consistency in data handling.
• 🌐 Custom event vendor names can be appended to logs for better identification and categorization of data sources.
• 📦 The 'pack' function allows for the neat storage of raw data in a single column while preserving the original log details.
• 💾 Custom parsing functions can be saved and reused, enhancing efficiency in managing multiple data sources.
• 🔗 Integrating custom parsers into the ASIM network session function enables seamless querying across various underlying data sources.

Q & A

What is the Advanced Security Information Model (ASIM)?

ASIM is a guiding framework that normalizes data within Microsoft Sentinel, helping to streamline the processing of security information.

How can ASIM be applied to a custom data source?

ASIM can be leveraged by using functions like 'parse' to extract key-value pairs from custom data sources, making existing detections applicable to that data.

What are the two main fields provided by the hypothetical proprietary network solution's custom network log?

The two main fields are 'event info' and 'event message.'

What is the purpose of the 'parse' function in the context of this transcript?

The 'parse' function is used to extract specific components from the event message, creating a structured output that can be analyzed more easily.

What kind of data can be extracted from the 'event message' field?

Data such as the network protocol (TCP or UDP), source IP address, and port number can be extracted from the 'event message' field.

How does the 'split' function work in this context?

The 'split' function breaks down the 'event info' text entry using a specified delimiter, such as a space, creating an array of components from that text.

What is the significance of renaming the extracted fields?

Renaming extracted fields to terms like 'destination zone' and 'network session ID' aligns them with ASIM's preferred field names, improving clarity and consistency.

How can custom log sources be identified in ASIM?

Custom log sources can be identified by appending a custom name for the event vendor, which helps in categorizing and analyzing the data correctly.

What is the purpose of the 'pack' function mentioned in the transcript?

The 'pack' function is used to neatly store raw data in a single column, facilitating easier access and analysis of that data.

How does the union function contribute to querying different data sources?

The union function integrates the custom parser into the broader ASIM network group, allowing for a comprehensive query across multiple underlying data sources like Palo Alto, Z-Scaler, and others.