Operationalising Azure Sentinel – from log ingestion to incident detection

Operationalising Azure Sentinel – from log ingestion to incident detection

You may well have heard of Azure Sentinel, Microsoft’s new cloud native SIEM (Security Information and Event Management) and SOAR (Security Orchestration, Automation and Response) with Machine Learning (ML) detection, a powerful querying language and virtually limitless storage. But, like many, you might not yet be sure how to implement it within your organisation.

This blog covers the basics of deploying Azure Sentinel, ingesting your first log sources and enabling detection rules to operationalise and automate your security operations.

Deployment

To deploy your Sentinel instance, simply create an Azure account (if you don’t already have one), type ‘Azure Sentinel’ into the search bar and connect or create a workspace – this is where your logs are going to be stored. Once this has been reviewed and created, select your chosen workspace from Azure Sentinel:

Microsoft have broken the Azure Sentinel workflow into four phases: collect, detect, investigate and respond. Being former military, I like to turn workflows into a continuous cycle, so I have added a fifth phase in the Azure Sentinel workflow – evaluate, which I will explain more on later.

First, let’s take a look at the four Microsoft phases:

1

Collect

1)Collect

Microsoft has really simplified the ingestion process through data connectors. At the time of writing this blog, there are currently 39 live data connectors, not including those in private preview or “coming soon”. Start by searching by name or provider. If found, simply follow the instructions provided by Microsoft on the connector page to start streaming logs into Sentinel. If no data connector is found, Microsoft has provided a list of Syslog, CEF and third-party data connectors here.

NOTE: Try to stick within the recommended guidelines provided by Microsoft where possible. However, there are workarounds if you are unable to do so. For example, if you are unable to forward syslog over CEF, you can create a function to parse the syslog message into a quarriable columns.

2

Detect

2) Detect

Now you have logs streaming into Sentinel’s log analytics workspace, you need to write or enable analytics rules to alert on possible security incidents. There are four types of detection rules in Sentinel, all mapped to ATT&CK tactics:

  1. Fusion This low-volume, high-fidelity rule is enabled by default and is a Microsoft written analytics rule to detect multi-stage attacks based on machine learning. There are currently 35 incident types that include a combination of suspicious Azure Active Directory sign-in events followed by anomalous Office 365 activity. These require the Azure Active Directory Identity Protection and Microsoft Cloud App Security data connectors.

NOTE: This rule cannot be edited and the logic behind the detection cannot be viewed. You can enable/disable this rule only. For more information – click here.

  1. Machine Learning Behaviour Analytics These rules are written by Microsoft and are available to enable under ‘rule templates’. At the time of writing this blog, there is one publicly available ML Behaviour Analytics rule in preview to detect anomalous SSH log ins.

NOTE: This rule cannot be edited and the logic behind the detection cannot be viewed. You can enable/disable this rule only.

  1. Microsoft Security There are currently five Microsoft Security rules to be enabled in Sentinel. These rules simply create an incident from an upstream Microsoft product such as Azure Security Center, Azure AD Identity Protection, Azure ATP, Microsoft Defender ATP (MDATP) and Microsoft Cloud Application Security. Once the appropriate data connector has been connected, simply enable these rules and alerts will be generated in Sentinel matching the Microsoft service. These rules can be configured to exclude specific alerts by name, include only specific events and filter based on alert severity.

    Excluding alerts from Microsoft Security rules is simple. Select the upstream provider analytic and enter the alert name under exclude specific alerts’, then create a scheduled query to query the security alerts or relevant table for the alert name and provide any additional details to exclude using a ‘contains’ parameter.

NOTE: You are not currently able to trigger a playbook based on these analytics, you will need to exclude the specific alert by name then create a new scheduled query for the excluded alert to attach a playbook.

NOTE: You are not currently able to trigger a playbook based on these analytics, you will need to exclude the specific alert by name then create a new scheduled query for the excluded alert to attach a playbook.

NOTE: The O365 data connector currently supports Exchange and Sharepoint logs. It is possible to stream threat management logs into Sentinel with the use of a custom logic application.

  1. Scheduled QueryWithout a doubt, this is my favourite detection method in Azure Sentinel. Log analytics queries are performed using Kusto queries, a read-only request to process data and return results. Understanding what logs are being stored where is crucial to writing effective analytics rules.

    There are several great rule templates provided by Microsoft as well as a built-in Sigma rule converter (to log analytics) in Sentinel Notebooks to get you started. The scheduled queries allow you to write a Kusto query and essentially set it to run on a timer which you can specify to run in minutes, hours or days.

    A result preview is provided in the analytic rule wizard to understand how noisy or effective the rule you are writing is, in real time. You can also map entities from the results which are used later in the investigation graph.

    The difference between pre-written rules in the above detection types and scheduled queries is that you are now able to view the logic behind an analytic, thus being able to modify the query to your environment.

NOTE: There is an incredible community behind Azure Sentinel on Github where scheduled queries are often released before they appear in Sentinel Rule Templates.

NOTE:It isy important to understand how Kusto Query Language (KQL) works to properly leverage the capabilities of Sentinel. A great starting course by Plural Sight can be found here.

3

Investigate

3)Investigate

Triggered alerts are stored in Sentinel under threat management in the ‘Incidents’ tab. From here you can assign a status, owner, severity and comment on individual incidents. Once an incident has been selected, you are presented with two options – ‘Investigate’ and ‘view full details’.

Investigate view brings up an investigation graph, providing entities have been mapped in the analytics rule. From here you can piece together the investigation using a graph view of entities included in the alert. You can go even further by expanding specific entities to include other alerts related to that entity, URL detonation and much more.

View full details will give you the option to view the raw data within log analytics. This is especially useful if entities have not been appropriately mapped to the alert and the investigation view is unavailable, or if you need dig deeper into the logs with additional tabs to query. You can also run playbooks from this view for any preconfigured automation tasks. 

Clicking on the value beneath the AlertID column will provide you the with the alert information such as provider name (where the alerting information was generated from), alert severity, the query which was used to generate the alert, etc. To dig deeper and view the raw logs which were alerted on, select the value represented under the ‘EVENTS’ column.

NOTE: In certain cases, it is much easier to perform deeper analysis within the alerting product. For example, if the alert was generated using a Microsoft Security rule type, you can select the ‘view full details’ option and find the alerting Microsoft product link within the ExtendedLinks option in AlertID, then copy and paste this into your browser.

NOTE: Understanding your log sources and Keyword Query Language (KQL) queries is crucial to perform enhanced investigative techniques against your log analytics workspace.

4

Respond

4)Respond

Automating workflows in Sentinel is achieved by creating ‘playbooks’ using logic apps. There is an incredible community on GitHub, sharing playbooks which can be imported in JavaScript Object Notation (JSON) such as collecting an investigation package from MDATP, importing threat intelligence feeds into log analytics, changing incident severity, raising tickets in ticketing systems, isolating machines in MDATP, and much more.  

5

Evaluate

5) Evaluate

The final phase of the loop is to evaluate your current configuration. What detections are raising in upstream products which haven’t generated an alert within Sentinel? Are you collecting the correct logs? Are you collecting too many logs? Do you have a new detection use case and can this detection be achieved with the current log sources? And so on. 

After the evaluation phase, the cycle can continue from any of the previous steps, providing the correct data and information is present.

Getting the most from Sentinel

It’s worth noting that Sentinel contains many more features which I haven’t touch on in this blog.  I hope to go into more detail on specific areas in the future, such as creating visualisations for stakeholders using Sentinel workbooks, using the built-in Jupyter notebooks to import sigma rules, integrating threat intelligence using the TAXII connector, exploring the built-in hunting queries and testing Kusto queries with live stream.

In summary, there have been some considerable hurdles to overcome in the process of scaling up Sentinel, some of which include: connecting custom third-party connectors, supressing alerts downstream, closing upstream alerts automatically and creating a health query for data connectors.

Overall, I’m very impressed with how quickly Azure Sentinel has taken off, the support we are receiving from Microsoft and the contributions from the wider community on the Microsoft Forums and GitHub to continuously improve Sentinel’s capabilities and I believe this will become a market dominator in the very near future.

Need help getting Azure Sentinel in place?

At Bridewell we have a number of Senior Consultants who are highly skilled Azure Sentinel specialist and if you have any questions or would like further information please get in touch.

Close Menu