Logs and Monitoring - N10-008 CompTIA Network+ : 3.1

Summary

The video script discusses the importance of analyzing logs from network devices like routers, switches, and firewalls for monitoring traffic flows and identifying potential issues. It highlights how logs can provide detailed insights into network activity and be used for real-time monitoring or historical analysis. The script also covers the use of syslog for standardized log collection and the role of SIEM in centralizing log data. Additionally, it touches on environmental monitoring in data centers, the use of NetFlow for traffic analysis, and the value of third-party service status pages for gauging network health.

Takeaways

• 📡 Network infrastructure devices like routers, switches, and firewalls generate logs that provide insights into traffic flows and data summaries.
• 🔥 Firewall logs are particularly detailed, showing every flow of traffic with information about the protocol, hostname, username, ports, and disposition of the traffic.
• 🕵️‍♂️ Logs are useful for real-time monitoring and for retrospective analysis to determine the cause of network events.
• 🗂️ Different devices have different log types, such as audit logs in Active Directory that record login and logout events.
• 🔄 Logs can be consolidated using the syslog protocol, which allows for centralized logging and management.
• 👨‍💻 Syslog assigns facility codes and severity levels to logs, aiding network administrators in prioritizing and taking action on log information.
• 🔍 Administrators can filter logs based on severity levels to focus on critical alerts or view detailed debug information as needed.
• 🛠️ Monitoring individual interfaces on devices can help identify potential network issues such as runts, giants, or CRC errors.
• 🌡️ Environmental factors like temperature, humidity, and electrical systems are crucial for the health and availability of network equipment.
• 📊 NetFlow is a tool for collecting detailed statistics from network traffic, providing insights into conversations, endpoints, applications, and more.
• 🌐 Third-party services and cloud status pages can provide service availability and performance metrics, which can be cross-referenced with internal network data.

Q & A

What types of network devices can provide valuable logs for analysis?

Network devices such as routers, switches, firewalls, and other infrastructure devices can provide logs that detail traffic flows and summaries of data traversing the device.

How detailed are the logs typically found in firewalls?

Firewall logs are often very detailed, showing every single flow of traffic and providing information about what's contained within that traffic flow.

What is the utility of storing historical network logs?

Storing historical network logs allows administrators to go back in time to determine what may have happened before or after a particular event.

What kind of information can be extracted from a highlighted firewall log?

A highlighted firewall log can show the protocol, hostname, username, client field, client port number, server IP address, server port number, and the disposition of the traffic.

How can logs help in searching for specific network events?

Logs allow administrators to run searches for particular IP addresses or port numbers to view all traffic flows on the network that match those values.

What are some examples of logs from an Active Directory infrastructure?

Active Directory infrastructure logs might include audit logs that show who has logged in or out of the network, including details like process ID, file name, username, and login event.

How can syslog help in consolidating logs from various network devices?

Syslog is a standard protocol that can be used to configure different devices to send log information to a centralized logging receiver, often consolidating this data into a SIEM (Security Information and Event Manager).

What is the purpose of severity levels in syslog?

Severity levels in syslog help network administrators filter and take action on the information received from logs, distinguishing between informational details that may not require immediate action and critical details that do.

Why might a network administrator be interested in analyzing individual switch interfaces?

Analyzing individual switch interfaces can help identify potential problems with data traversing the network, such as a large number of runts or giant frames, which may indicate a collision or communication problem.

What are some environmental factors that can affect the health and availability of a network?

Environmental factors such as room temperature, humidity levels, electrical system stability, and the presence of water in cooling systems can all impact the health and availability of a network.

How can NetFlow provide additional network insights beyond SNMP and log files?

NetFlow allows for the collection of statistics and details from the raw traffic traversing the network, providing extensive information on conversations, endpoints, applications in use, and more.

Outlines

00:00

🔍 Analyzing Network Device Logs

This paragraph discusses the importance of analyzing logs from network devices such as routers, switches, and firewalls. It highlights how these logs can provide detailed insights into network traffic flows and summaries, which are crucial for real-time monitoring and historical analysis of network events. The paragraph gives an example of firewall logs that include detailed information about each traffic flow, such as protocol, hostname, username, client and server IP addresses, port numbers, and the disposition of the traffic. It also mentions how different devices, like active directory infrastructure, have different log types, but all can be consolidated using the syslog protocol. The use of a Security Information and Event Manager (SIEM) for centralizing log data is also discussed, along with the significance of syslog's facility codes and severity levels in filtering and acting upon log information.

05:00

📊 Monitoring Network Health and Environment

The second paragraph focuses on monitoring the health and environment of a network, particularly in data centers or computer rooms. It covers the importance of monitoring variables such as temperature, humidity, voltage, and the presence of water to ensure the network's availability and prevent damage to equipment. The paragraph also discusses the use of SNMP and log files for network monitoring and introduces NetFlow for gathering detailed statistics from raw network traffic. It explains the role of a NetFlow probe and collector in monitoring network traffic and creating detailed reports. Additionally, it touches on the use of third-party services for checking the status of cloud-based services and their impact on the network.

10:02

🌐 Network Traffic Analysis and Service Monitoring

The final paragraph delves into the analysis of network traffic using tools like NetFlow, which provides insights into conversations, endpoints, applications, and more. It describes how NetFlow can help in understanding the type of traffic on the network and in interacting with security devices to ensure proper information flow. The paragraph also mentions the utility of third-party services for monitoring the status of cloud services and how they can be correlated with NetFlow or log files to assess their impact on the network.

Keywords

💡Network Infrastructure

Network infrastructure refers to the hardware and software components that support the network's operations. This includes devices such as routers, switches, and firewalls. In the video, the script discusses how these devices generate logs that provide insights into network traffic flows and summaries, which are crucial for monitoring and troubleshooting network issues.

💡Logs

Logs are records of events that occur within a system or network. They are essential for tracking activities, diagnosing issues, and auditing security. The script mentions that logs from firewalls and other devices can be detailed and contain information about traffic flows, protocols, and dispositions of data packets.

💡Firewall

A firewall is a network security system that monitors and controls incoming and outgoing network traffic based on predetermined security rules. The video script highlights that firewall logs can be very detailed, showing every flow of traffic and providing information about the content within that traffic.

💡Traffic Flow

Traffic flow refers to the movement of data packets across a network. The script explains that logs can detail each traffic flow, which is useful for real-time monitoring and historical analysis to understand network status and identify issues.

💡Syslog

Syslog is a standard protocol used for sending log messages from a device to a centralized log management system. The video mentions configuring devices to send syslog messages, which allows for consolidated logging and easier analysis of network events.

💡SIEM

SIEM stands for Security Information and Event Management. It is a system that aggregates logs and events from various sources to provide real-time analysis and historical searching. The script discusses how syslog data is often sent to a SIEM for central management and analysis.

💡Severity Levels

Severity levels classify the importance or urgency of log messages. They help network administrators prioritize issues. The video script describes how syslog assigns severity levels to log messages, allowing administrators to filter and focus on critical alerts.

💡Ethernet Frames

Ethernet frames are the data packets used in Ethernet networks. The script discusses runts (frames smaller than 64 bytes) and giants (frames larger than 1518 bytes) as indicators of potential network issues, such as collisions or communication problems.

💡CRC Errors

CRC errors, or Cyclic Redundancy Check errors, indicate that data corruption has occurred in a network transmission. The video script mentions that CRC errors are often caused by bad cables or interfaces and can be resolved by fixing these hardware issues.

💡Encapsulation Errors

Encapsulation errors occur when there is a mismatch in the expected protocol or frame type between network devices. The script gives an example of mismatched trunk links using ISL versus 802.1Q standards, leading to encapsulation errors.

💡NetFlow

NetFlow is a network protocol for collecting IP traffic information and monitoring the flow of data across a network. The video script explains that NetFlow can provide detailed statistics and insights into network traffic, such as the type of conversations, endpoints, and applications in use.

Highlights

Network infrastructure devices like routers, switches, and firewalls generate logs that provide valuable insights into traffic flows and data summaries.

Firewall logs are particularly detailed, showing every flow of traffic and its contents.

Logs can be used in real-time for network status monitoring and can also be stored for historical analysis of events.

Example firewall log details include protocol, hostname, username, client and server IP addresses, port numbers, and traffic disposition.

Logs from different devices like active directory have unique types but all contain correlatable data.

Audit logs can show login and logout events, and attempts with incorrect credentials.

Standardized log retrieval using syslog protocol allows for centralized logging management.

SIEM consolidates log data into a central database for comprehensive security information and event management.

Syslog assigns facility codes and severity levels to logs, aiding network administrators in prioritizing actions.

Severity levels can be filtered to view only critical alerts or broader debug information as needed.

Analyzing individual interface statistics can reveal potential network issues such as runts or giant frames.

CRC errors, often caused by bad cables or interfaces, can be identified and resolved by monitoring logs.

Encapsulation errors can occur due to mismatched protocols like ISL and 802.1Q on different switches.

Interface statistics provide insights into normal operation and potential errors or issues with network ports.

Environmental factors like temperature, humidity, and electrical systems are crucial for network health and availability.

Flood monitors are necessary in large data centers using water cooling systems to prevent electrical equipment damage.

Creating a baseline of network events helps in understanding normal vs. abnormal network behavior over time.

NetFlow collects detailed raw traffic statistics for deeper network performance analysis.

NetFlow data can be collected using a probe and a collector to monitor network traffic and generate reports.

Third-party services provide status updates on cloud services, which can be correlated with network logs for impact analysis.

Transcripts

If you're working with routers, switches, firewalls,

or other infrastructure devices connected to the network,

there's probably logs inside of that device that can tell you

information such as the traffic flows and traffic

summaries of the data traversing that device.

If you're looking at logs inside of a firewall,

for instance, these logs are often very detailed.

They will show you every single flow of traffic

that's traversing that firewall, and give you

information about what's contained within that traffic

flow.

These logs are not only useful to use in real time

so you can see the status of your network,

but you can also store this information

and go back in time to determine what

may have happened before or after a particular event.

Here, for example, is a set of logs from a firewall.

Each one of the lines here on the left side

is a separate flow of traffic.

On the top line, I have highlighted one particular flow

that shows protocol, in this case,

UDP has a hostname, a username, a client field, a client port

number, a server IP address, a server port number,

and then the disposition of this traffic.

You also get a detailed view of this along the right side,

and as you can see, an extensive number of variables

are stored for each individual traffic flow.

This means if you wanted to run a search

to look for a particular IP address or a particular port

number, you'd be able to view all of the traffic flows

on your network that match that particular value.

Obviously, different devices will

have different types of logs.

For example, if you have an active directory

infrastructure, it probably is going

to have a series of audit logs that

allow you to see who might have logged in

or logged out of the network.

For example, this audit log shows

a message saying that a process has been created.

It shows a process ID.

Shows the file name.

In this case, it was cmd.exe that was used in this process.

And you can see more information about the username

and the login event.

You can also see details on the time, the host name, event IDs,

and other specifics.

This allows us to keep a historical view

of exactly who logged into the network, who logged out,

and perhaps who may have tried to log in

but used incorrect credentials.

When you start looking at the log files

on switches, routers, firewalls, Windows servers, Linux servers,

and other devices, you'll see that each one of those log

types is very, very different.

But all of these diverse log files

contain details that would allow us to correlate

data flows together.

Although all of the logs contain very different information,

we can use a standardized process

to retrieve those log files from every single one

of these devices using a standard protocol called

syslog.

We can configure all of these different devices

to send information to a consolidated logging

receiver using the standard syslog protocol.

In many cases, we're consolidating this information

to SIEM.

This is a security information and event

manager that allows us to bring all

of that data back to one central database. syslog

receives this data from a particular device,

it logs a facility code, which identifies the program that

originally created the log, and it assigns a severity level

to the information contained within that log.

This allows us as the network administrator

to take action on the information

we're receiving from all of these different logs.

Some of these logs will contain informational details

that may not need any type of immediate action,

whereas other logs may have critical details

and alert information that require immediate action.

We can use these severity levels as a filter.

So we can go to a single screen and say

that we'd like to see everything that is a warning

level or higher or maybe we want to view all of the debug

information sent in every log.

You as the network administrator get

to determine exactly what information is important to you

at any particular time.

You may decide during normal operation

that you only want to see alert or emergency log information,

but if you're doing research on a previous event,

you may want to expand this filter out to view debug

information and higher.

It might also be useful to perform

ongoing analysis of individual interfaces on these devices

to see if there may be any problems with the data

traversing the network.

For example, you may be looking at interfaces on a switch,

and you may find that it has a large number of runts.

The minimum size of an Ethernet frame is 64 bytes.

So if you receive a frame that is smaller than 64 bytes,

we qualified that as a runt, and in many cases,

that means that a collision has occurred on the network.

On most of our networks today, we're

using full duplex ethernet so the identification

of a runt and potentially a collision

is something you probably want to be aware of.

If you're not using jumbo frames,

the maximum size of an ethernet frame

is going to be 1,518 bytes.

If your networking device identifies a frame that's

larger than 1,518 bytes, it identifies

that as a giant frame, and again, this

may indicate that some type of communication problem

is occurring on the network.

If any of these ethernet signals are corrupted

as they're being sent across the network,

you may receive a cyclic redundancy check

error, or CRC error.

You might also see this referred to as a frame

check sequence error, or FCS error.

These are commonly caused by a bad cable or bad interface,

and resolving the cable or interface

causes the CRC errors to cease.

And there may be cases where encapsulation errors might

occur, where one switch is expecting one type of frame,

and the other switch is expecting

a different type of frame.

On older switches, for example, you

may see trunk links that are using inner switch link

protocol, or ISL.

These days, we tend to use the 802.1Q standard,

but if one side is set for ISL and the other side is set

for .1Q, then you'll have an encapsulation error.

If you were to log in to a switch

and view the statistics on an individual interface,

you might get a message very similar to this.

And although it seems like there's

a lot of information on the screen,

you tend to start focusing in on very specific areas

to get the information you need.

For example, you may want to look

at the very first line that shows

that fast ethernet interface 0/1 is up

and the line protocol is up and connected,

which means that this interface should be operating normally

on the network.

We can look at another line that shows

this is configured for full duplex and 100

megabits per second, and the media type

is a 10/100 interface.

We can also see that there have been 5,701 packets.

We can see 1,157,662 bytes, and 0, no buffer frames.

We can also see there have been 0 CRC errors, which

means that this particular network should

be performing normally.

However, we can also see a large number of broadcasts.

5,130 broadcasts and 2,269 multicasts.

By looking at the individual interfaces on a switch,

you can start putting together a view

of exactly how each interface is performing

and if there might be any errors or problems

with individual ports.

When you're working with a data center or a computer room,

you also have to be concerned about the environment.

There are many different variables

that can affect the overall health and availability

of the network, such as the temperature in the room.

These devices can get very hot and need constant cooling,

so we want to be sure that the air conditioning that we have

in the room is always working properly,

so we'll want to monitor that temperature.

We might also want to monitor the humidity level.

If we have a lot of humidity in the air,

then we may have some condensation,

and we certainly want to keep that water away

from our electrical equipment.

If we have a low amount of humidity,

we may have an excessive amount of static discharge, which

is also bad for the silicon.

Since we are connecting all of these power devices

to an electrical system, it would also

be useful to know if we're receiving

the proper amount of voltage and that everything is working

as expected, so constant monitoring

of the electrical system can be a very valuable metric.

In a very large data centers, you

may be using water as part of your cooling system.

This means we may want to have flood monitors

in different parts of the room to make sure

that that water never gets close to our electrical equipment.

If you start collecting this information over time,

you can begin to create a baseline of what

may be expected.

For example, you may create an all events baseline that

shows the number of information events

that you have coming through during the day versus the debug

or notice events, and this can be

used to look at event categories, security

events, important events, or other type of metrics

on your network.

SNMP and log files can tell you a lot about what's

happening on the network, but you occasionally

would like to have a lot more detail of information that

may be contained within the ethernet packets themselves.

In those cases, you may need additional collection tools,

such as NetFlow.

NetFlow allows you to gather statistics and details

from the raw traffic traversing the network.

There are many different ways to collect this data.

It may require a separate individual NetFlow probe,

or there may be NetFlow capabilities

built into the equipment that you're already using.

Generally, NetFlow will have a NetFlow probe and a NetFlow

collector.

The probe will sit out on the network

sometimes as part of a tapped connection or it

may be receiving traffic from a switched port analyzer or span

connection, and it's watching all of the packets

traverse the network.

It's gathering those details and exporting

all of that NetFlow traffic back to a central NetFlow collector.

From the NetFlow collector, you can then

create detailed reports of everything

that's been seen over time.

This allows you to get extensive information of what

may be occurring on the network, such as the type

of conversations, the endpoints communicating, the applications

in use, and much more.

For example, you can get a distribution

showing the amount of TLS traffic versus port 0, UDP,

in the clear HTTP, and Microsoft SQL server traffic on one view.

This will give you some insight into exactly what

might be traversing the network and might also

help you interact with your security devices

to make sure that only the proper information is

being sent over the network.

We've talked about a lot of different types of metrics

in this video, but sometimes, all you want to know

is whether a service is up or down.

Fortunately, many third party services, especially services

in the cloud, can provide you with a list of all

of those services and a status of how

that service is performing.

So if you happen to connect to the status page

and you see that a particular problem was recently resolved,

you may be able to track that back to your NetFlow or log

files to determine if that, indeed,

had an impact on your network.