YT YouTube2Text

YouTube2Text → Transcripts → Microsoft Sentinel Training | Azure Sentinel Tutorial | Microsoft Sentinel Step-by-Step Guide

Microsoft Sentinel Training | Azure Sentinel Tutorial | Microsoft Sentinel Step-by-Step Guide — Transcript

by CyberPlatter · 42,884 words · 6,863 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00Hey guys, welcome to cyberplatter. I am
0:02Navia. This is a full course on
0:04Microsoft Sentinel which was earlier
0:06known as Azure Sentinel. So we will
0:09cover all the main topics of Sentinel,
0:12explain them in a simple way and show
0:14demos to help you understand how it
0:16works. So by the end of this video, the
0:19goal is to make sure you understand
0:21Microsoft Sentinel and you can use it on
0:23your own. So let's begin with the basic
0:26foundational topic. What is Microsoft
0:28Sentinel and the architecture of
0:31Microsoft Sentinel?
0:33So what is Microsoft Sentinel? Microsoft
0:35Sentinel is a solution security solution
0:38that combines SIM and store solutions.
0:42It has both the capabilities. So it's
0:44designed to provide comprehensive
0:46visibility and enhance security for
0:48organizations by detecting,
0:51investigating and responding to threats
0:54in real time. So I said it is a
0:57combination of SIM and SOR. What is SIM?
1:00SIM stands for security information and
1:04event management.
1:06It combines two core functionalities.
1:09The first one is SIM that is information
1:12management, security information
1:13management and the other one is SEM that
1:16is security event management. SIM
1:19focuses on log collection, storage and
1:22analysis. So this helps organizations
1:25track and manage historical data from
1:28various systems. This can be for
1:30forensics compliance or auditing
1:33purposes. The next core functionality of
1:36SIM is SEM that is uh security event
1:39management. This deals with realtime
1:42monitoring and correlation of events. So
1:45this helps detect suspicious activities
1:48or potential threats across an
1:50organization's IT infrastructure. Now
1:52that we understand what is stemmar
1:55sore stands for security orchestration,
1:58automation and response. This is a
2:02security solution that focuses on
2:03improving the efficiency and
2:05effectiveness of an organization
2:07security uh operations. Like the name
2:10suggests, it combines automation,
2:14incident management and response
2:16orchestration. Because of this, it
2:20streamlines workflows and reduces manual
2:23effort. So, security orchestration
2:25integrates multiple security tools and
2:27systems to work together seamlessly.
2:30Like for example, different platforms
2:32such as firewalls, SIMs, endpoint
2:35security can communicate and coordinate
2:38actions. Next is the automation part. It
2:41automates repetitive tasks such as you
2:44know alert triggering, log analysis and
2:46initial response actions. So this
2:48reduces manual effort enabling security
2:51teams to focus on complex threats. The
2:53next part is response. It is usually
2:55incident response. So this provides
2:58tools for managing and responding to
3:00security uh incidents effectively. This
3:03can include playbooks, workflows to
3:05guide you know teams through
3:07standardized response procedures. So now
3:10we know what is SIM and soore and what
3:13are their capabilities. All these are
3:15provided by Microsoft Sentinel because
3:17it is a combination of both SIM and
3:20soar. And what sets Microsoft Sentinel
3:23apart from the other SIM and soore
3:25solutions is its pay as you go pricing
3:28model. That means you only pay for the
3:32data you ingest and store. So this makes
3:35it a cost effective option for
3:38businesses of all sizes. It's a very
3:40flexible approach and it also ensures
3:43that you are not locked into expensive
3:46upfront costs or licensing fees. So in
3:49the other SIM consour solutions usually
3:51you have to pay an expensive upfront
3:54cost or you know there is licensing that
3:57happens you'll have to pay for that as
3:58well that is not there in Sentinel.
4:01Also, Microsoft Sentinel is a fully
4:04cloudnative solution that is delivered
4:07as software as a service solution that
4:10is SAS solution. This means that you
4:13don't have to worry about maintaining
4:15infrastructure, deploying hardware or
4:17managing updates. It is all handled by
4:20Microsoft. So there are different
4:22service models like SAS, software as a
4:25service, platform as a service, pass and
4:28is that is infrastructure as a service.
4:30And there is on premises and sentinel is
4:33SAS. So if you're using an onremise SIM
4:37then the customer then that is the
4:39organization will have to manage the
4:41entire stack that is you know from
4:44starting networking storage servers
4:46virtualization OS middleware runtime
4:48application identities and data. And
4:52then there is I is that is for example
4:54say you are running a sim on Azure
4:57virtual machines. Then you can see that
5:00the cloud provider or any provider will
5:03manage the virtualization, servers,
5:05storage and networking that is the
5:07infrastructure part and the customer is
5:10responsible for the rest. So management
5:12overhead in is compared to on premises
5:15decreased. Again if you go to pass it
5:17decreases more for the customer because
5:20the customer is maintaining only the
5:22applications identity and data. But then
5:25if you see in SAS that is what the
5:28Microsoft Sentinel model is you are
5:30maintaining only the data and the
5:32identities and accounts part. So the
5:35customer that is the organization is
5:37responsible only for the data that they
5:40ingest and store what kind of
5:42configuration that they require and the
5:45identity identities or accounts that is
5:48that falls under identity and access
5:50management because Sentinel is provided
5:53as a SAS solution. There is no hardware
5:56or software maintenance that the
5:59customer will have to do and also it
6:02automatically scales with the
6:04organization's needs and the team can
6:08focus on security that is it allows
6:11organization to focus on threat
6:13detection and response instead of you
6:15know maintaining infrastructure
6:18and Microsoft Sentinel can be accessed
6:20from two places one is the Microsoft
6:23Azure portal So like you can see it in
6:26this picture. This is in Microsoft
6:28Azure. In Microsoft Azure, you search
6:30for Microsoft Sentinel. So Microsoft
6:33Azure uh URL is portal.asure.com.
6:38If you log into this, you land on the
6:40homepage and then you search for
6:41Microsoft Sentinel. We'll talk about
6:44this more in the next chapters where I
6:47show you how to get a free trial, how
6:49you can get your own Microsoft Sentinel.
6:52The other way that you can log into
6:54Sentinel is from Microsoft Defender
6:56portal. Now you can also access it from
6:59the def defender portal. So the defender
7:02portal aims to combine all the security
7:05capabilities, right? So you can find it
7:07there as well. So in this screenshot it
7:10is in Microsoft Defender and if you see
7:13here there's a separate tab for
7:15Microsoft Sentinel and you can find all
7:17the details here and the URL for this is
7:21security.microsoft.com.
7:23Now that we know what is Microsoft
7:26Sentinel, what are the capabilities, how
7:28it can be accessed, let's go to
7:30Microsoft Sentinel's architecture.
7:32Sentinel integrates with a wide range of
7:34data sources to collect mostly security
7:37related data but you can collect other
7:39data as well. Like you can see here,
7:42Sentinel supports direct integration
7:44with major public cloud platforms uh
7:47like for example Azure obviously AWS,
7:50Google Cloud and more. So you can
7:53collect security logs, events and
7:56telemetry not just from Azure but also
7:58from other public clouds. And then it
8:01supports data injection from on premises
8:04servers and devices like for example you
8:06see Windows, Linux, MacBooks. You can do
8:10this using agents or custom
8:11integrations. It also allows
8:14integrations from custom or third party
8:16data sources using APIs or custom
8:19scripts to expand its capabilities. Not
8:22just APIs, you can also use you know
8:24logic apps or Azure functions to
8:27establish custom log injection. So the
8:30other use cases for you know data
8:32sources would be you know you're sending
8:33data from Azure active directory that is
8:36now Microsoft enter then Azure monitor
8:40Microsoft defender or you know if you
8:42have other security tools like for
8:44example service now you're getting data
8:46from there as well and also Sentinel
8:49itself provides a lot of built-in
8:51connectors for seamless integration. So
8:54all of this data is sent to log
8:56analytics workspace. Sentinel relies on
8:59log analytics workspace for storing and
9:02querying data. It supports both
9:04structured like for example table and
9:08unstructured like for example JSON data.
9:10So all of the data is sent to log
9:13analytics workspace. This acts as the
9:15central repository for logs, metrics and
9:18other security related data and Sentinel
9:21uses KQL to analyze and query this data
9:24for insights. And you can also send data
9:28from threat intelligence sources. So
9:31Sentinel uses Microsoft's uh threat
9:34intelligence but you can also get your
9:37own threat intelligence sources as well
9:39if you have anything in your
9:41organization. Threat intelligence is to
9:43enrich the collected data like it
9:46identifies non-malicious IPs, domains or
9:49file ashes and it can also correlate
9:51data with known threat patterns for
9:53faster detection of attack. And these
9:56are some of the core functionalities
9:58that Sentinel provides for monitoring
10:00and incident response. The first one is
10:03incidents. It is for correlation of
10:05alerts into actionable incidents for
10:08investigation. Next is analytics. This
10:11is built-in and customizable rules to
10:14detect anomalies and suspicious
10:16activities. Then there is notebooks.
10:18These are interactive Jupiterbased
10:20notebooks for advanced threat hunting
10:23and analysis. Next we have workbooks.
10:26These are pre-built and customizable
10:28dashboards to visualize and report on
10:31security metrics. And then there is
10:33hunting. So it is proactive threat
10:36hunting using machine learning and KQL
10:38queries. And then there are playbooks.
10:41These are automated response workflows
10:43built using Azure logic apps to mitigate
10:46threats. So this is just an
10:48introduction. We will talk about all
10:50these capabilities in depth and also you
10:53know demos for them. For now just
10:56understand that these are some of the
10:58core capabilities that are provided by
11:00Sentinel. So you can ingest all the data
11:04and also threat intelligence and store
11:06it in log analytics workspace and then
11:09Sentinel uses KQL to analyze and query
11:12this data for insights and we also saw
11:16some of the Sentinel core components.
11:19Next is chapter 2 and in this we will go
11:22over how to set up Sentinel and manage
11:25permissions using Rback. So let's get
11:28started. If you want to follow along
11:30with the demos, you can create a free
11:32Azure trial account, you can search for
11:35Azure free trial, go to the first link
11:38which is azure.microsoft.com.
11:40Create your free account today. Click on
11:42it. So here you can create a free Azure
11:46account. If I scroll down, it tells you
11:49what is this free account for. It is for
11:51new Azure customers
11:53and you get $200 credit to use Amazon
11:58services within a month and after that
12:01it is pay as you go pricing. So let's
12:04sign up. Click on sign up. Then you
12:06create your profile, your country, your
12:09first name, last name, email address.
12:13Once you enter your phone number, it'll
12:14ask you to verify your identity. You can
12:17get a text or you can ask Azure to call
12:20you with a code. Then enter the
12:22verification code. Then click on verify
12:25code. Then enter your address. Then I'm
12:30going to check I understand that
12:31Microsoft may contact me about my free
12:34account. I represent that I am the
12:36individual identified on this account. I
12:38don't want to receive any emails with
12:41tips and information and offers from
12:43Microsoft. So I'm not going to check
12:45that. and then click on next. And then
12:48it will ask you to verify your identity
12:50by using a credit card or a debit card.
12:53I will enter those details now and then
12:56click on sign up once you enter the
12:58credit card details. After that you
13:00might be asked to set up MFA. Then it
13:03says you are ready to start with Azure.
13:05Go to Azure portal. Okay. Now my free
13:07Azure trial account is ready. Before we
13:10start with the Microsoft Sentinel
13:12deployment, let's look at some
13:14prerequisites for it. First is an Azure
13:17tenant. So you need access to Microsoft
13:20Entra ID license and tenant. Microsoft
13:23Enra ID was formerly called Azure Active
13:26Directory. This is an identity and
13:29access management service used in Azure.
13:31You need access to that. And a tenant is
13:34a dedicated instance of Enra ID that
13:36manages users, groups, and permissions
13:38for an organization. Usually, if you're
13:41part of a company, right, your
13:43organization provides access to an
13:45existing tenant. And if you don't have
13:47an organization
13:49ID tenant, you can still create an
13:52individual Azure account like just we
13:54created a free account, right? You can
13:56create that because you need an
13:58individual account with a valid payment
14:01method. So under this either you need a
14:04Microsoft enter ID license and tenant or
14:07an individual account with a valid
14:09payment method. The valid payment method
14:11that we entered was a credit card and
14:14this is required to access Azure and to
14:17deploy resources. So the next
14:20prerequisite is to have an active Azure
14:23subscription. So to deploy and use
14:26Microsoft Sentinel you must have an
14:28Azure subscription and this Azure
14:31subscription either requires a Microsoft
14:35entry ID tenant that is for businesses
14:37or organizations or an individual
14:40account with payment details. This is
14:42for personal or trial use and an Azure
14:46subscription is used to track resource
14:48creation and billing. So when you're
14:51deploying Microsoft Sentinel, you create
14:53and manage cloud resources such as log
14:55analytics, workspaces, data connectors,
14:58right? For that you need the
15:00subscription to provide a way to
15:02organize, track and manage these
15:05resources. And even if you use a free
15:08trial, Azure needs a subscription to
15:10associate costs with the resources you
15:13deploy. It helps monitor usage and set
15:16up budgets, alerts and cost analysis to
15:19avoid unexpected uh charges.
15:23And then you require a log analytics
15:25workspace. So a log analytics workspace
15:28is essential for Microsoft Sentinel
15:30because it serves as a central location
15:33where data is collected, stored and
15:35analyzed. Sentinel ingests security
15:38logs, threat intelligence and other
15:41telemetry into this workspace and it
15:44later uses it for say threat detection,
15:46security analytics, investigation and
15:49response. Few things that you need to
15:51note is that the workspace must not have
15:54a resource lock applied. Resource logs
15:57prevent modifications which can
15:59interfere with Sentinel's ability to
16:02function properly. And also the
16:04workspace must use a pay as you go or
16:07commitment tier pricing model. So legacy
16:10pricing tiers are not supported and
16:12commitment tiers allow cost savings for
16:15high volume data injections and usually
16:18commitment tiers allow uh cost savings
16:21for high volume data in doing it for an
16:24organization.
16:26And then is a dedicated resource group.
16:28This is a recommendation to simplify
16:31management and security. It is
16:33recommended to create a dedicated
16:35resource group for your log analytics
16:38workspace and other sentinel resources
16:41such as playbooks, workbooks or data
16:43connectors.
16:45So why a dedicated resource group is
16:48recommended? Because it simplifi
16:50simplifies permission management. Like
16:52for example, assigning permissions at
16:55the resource group level ensures they
16:57automatically apply to all included
16:59resources. So this reduces the risk of
17:02improper access configurations
17:05and also you can make sure that only
17:07sentinel related resources are in this
17:10resource group and this makes it easier
17:12to enforce least privilege access.
17:15Right? So a structured approach to
17:17resource grouping enhances security,
17:19simplifies administration and reduces
17:22permission complexity and in turn makes
17:25Sentinel's operation secure and
17:28efficient. So this is how it will look
17:30like. First you have your management
17:32group, then your subscriptions, then
17:34your resource groups. Under that
17:37resource group you will have your
17:38resources for example log analytics
17:41workspace. Now let's see a demo to
17:44create these resources in the Azure
17:46portal. So in the Azure portal, if you
17:49want to look at what subscriptions are
17:51available, you can search for
17:53subscriptions in this uh search bar. So
17:57when I typed SUBS, it gave me services
18:01subscriptions. I'm going to click on
18:03that. And here it tells me I have one
18:06subscription that is the default
18:08subscription that was created. Azure
18:10subscription one and my role is owner
18:14and if you want to create more
18:16subscriptions you can click on add and
18:18then create another subscription. So I
18:21will be using the default one. So I'm
18:23not going to create this. Let me go back
18:25to home. So if you see here we have the
18:28subscriptions. Now let's create a
18:31resource group. Again in the search bar
18:34I'm going to search for resource groups.
18:37Click on resource groups and then click
18:40the create button to start the process.
18:44It gives a basic description of what a
18:46resource group is. It's a container that
18:48holds related resources for an Azure
18:51solution. The resource group can include
18:54all the resources for the solution or
18:57only those resources that you want to
18:59manage as a group. And there is the
19:02subscription. You can select the default
19:04one or the subscription that you have
19:07created and then resource group name.
19:10I'm going to call it Sentinel RG. A
19:14region it's uh East US. You can select
19:17any other uh location as well whichever
19:20suits you. I'm going to go with East US.
19:23Then next. Next. It takes me to the tags
19:27page. So this is not mandatory but a tag
19:30in Azure is a label that consists of a
19:33key value pair. This is used to organize
19:36and manage Azure resources. So tags can
19:39help categorize resources based on
19:41different attributes such as department
19:44cost center project groups and so on.
19:49For example, I can create something like
19:52say cost center and maybe the cost
19:55center number I want to give is 01. I
19:58can create another tag saying technical
20:02owner and then assign the name or you
20:05can also say environment is production
20:07or development or test. So this will
20:10help you group and identify resources
20:13across multiple subscriptions. For now,
20:16I don't want to create any tags for this
20:19resource group. So, I'm going to skip
20:21this step and click on next. Next is the
20:24review and create stage. So, in this
20:27stage, you validate the settings. If
20:30everything looks okay to you, then click
20:32create to finalize the resource group.
20:34So, I'm going to go ahead and click
20:36create. And like you can see the
20:39resource group is created. Sentinel RG
20:42that is in the subscription Azure
20:44subscription one. So I can click on
20:46Sentinel RG and it takes me to the
20:49resource group. So under this you can
20:51see the overview activity log will get
20:54back to this. But for now understand
20:56that the resource group is created. So
20:58going back to our PPD we have the
21:01subscription ready resource group ready.
21:03The next step is to create a log
21:05analytics workspace.
21:08Let's get back to the portal. In the
21:10search bar again you search for log
21:13analytics workspace.
21:16So it comes up here. I'm going to click
21:17on log analytics workspace. Click
21:21create. It tells you a log analytics
21:23workspace is the basic management unit
21:26of Azure monitor logs and it also gives
21:29you a link for more details on Azure
21:32analytics workspace. So it is a logical
21:35storage unit where your log data is
21:38collected and stored. subscription is
21:40already selected. Then the resource
21:42group I'm going to select Sentinel RG
21:45the one we created just now or you can
21:47create a new one as well here. And then
21:50it is asking me for the log analytics
21:53workspace name. I'm going to say
21:55Sentinel workspace
21:58and the region I'm going to leave it as
22:00East US. Next tags I'm going to skip
22:04this. I don't need to tag it. And next
22:07is the review plus creates. Let's
22:10validate our settings. It's taking a
22:13minute. Uh it's done. Validation passed.
22:16And if I scroll down, everything looks
22:19good. I'm going to click on create
22:22button. And then it says initializing
22:24deployment. Initializing template
22:27deployment to resource group Sentinel
22:30RG. Now the deployment is in progress.
22:34And you can click on this notification
22:36button to see what you have done. So
22:39deployment succeeded. Let me close the
22:42notifications.
22:44And if I click on this button which says
22:46go to resource. This is the Sentinel
22:48workspace that we just created.
22:51So we are done with all the
22:53prerequisites to deploy Sentinel. Let's
22:56go back to the portal. Again, in the
22:58search bar, I'm going to search for
23:00Microsoft Sentinel and then click
23:03Microsoft Sentinel. Then I'm going to
23:06click create to begin the deployment.
23:09Here I can choose a workspace that is
23:11already available or create a new
23:15workspace.
23:16So we already created a workspace. I'm
23:19going to click on it and then say add.
23:22It says adding Microsoft Sentinel to
23:25workspace. Sentinel workspace.
23:28It's taking a minute to do that. So you
23:31can click on the notifications to see
23:34what's happening. It also says if you
23:36want more events, you can go to the
23:39activity log and look at them. Now we
23:42have successfully added Microsoft
23:44Sentinel. Let me close the
23:46notifications. So it also gives you a
23:49message that Sentinel uh free trial is
23:52activated. The free trial is active on
23:54this workspace from so and so day that
23:57is for a month and then you can use up
24:00to 10 GB per day data that is free. I'm
24:05going to click okay. You can expand
24:08these options to see what's available.
24:13Threat management there is general then
24:16threat management content management and
24:19configuration. We will go through all of
24:21these one by one in the later chapters.
24:25Now let's see how to set up role based
24:27access control for Microsoft Sentinel.
24:31First, what is role based access control
24:33arbback? Right? So it is a security
24:35model that restricts access to resources
24:39based on a user's role within an
24:41organization.
24:42So instead of assigning permissions to
24:45individual users, roles are created with
24:48specific permissions and users are
24:50assigned to those roles. And Microsoft
24:54Sentinel assigns permissions using Azure
24:57Arbback that is Azure role based access
24:59control to manage access for users,
25:02groups and services. This is to make
25:04sure that security teams have fine grain
25:07control over what users can see and do
25:10within Sentinel.
25:12They have built-in Azure roles. These
25:16are predefined roles provided by
25:18Sentinel that determine user
25:21permissions. So you can assign roles at
25:24the subscription level, resource group
25:26level or the resource level that is the
25:30log analytics workspace, Sentinel
25:33workspace level. Remember that if
25:35assigned at the subscription or resource
25:37group level, Sentinel inherits
25:39permissions from them. Let's look at the
25:41Microsoft Sentinel built-in roles. The
25:44first one is Microsoft Sentinel Reader.
25:47This has view only permissions.
25:50So it can view data, incidents,
25:53workbooks and other Sentinel resources.
25:56So here you can't modify anything. You
25:58only have view access. The next one is
26:02Microsoft Sentinel Responder. This
26:04includes reader permissions but can also
26:08manage incidents like assign them,
26:11dismiss them, update the status, right?
26:14Usually it is given to somebody say if
26:17you have a sock team sock one members
26:20can be given this role where it is
26:22Microsoft sentinel responder. They can
26:24view the data as well as manage
26:27incidents. And then there is Sentinel
26:30contributor and this includes the roles
26:33of Microsoft Sentinel responder. And
26:36then it can also install or update
26:40solutions, create or edit workbooks,
26:43analytic rules and other resources. So
26:46usually SOC 3 higher level sock people
26:50as well as security engineers or
26:52architects who need to configure
26:53sentinel settings are given this access.
26:56And then there is Microsoft sentinel
26:58playbook operator. This role can view
27:02and manually run playbooks. So you can
27:05give this to security analyst say who
27:09need to trigger automation but cannot
27:11modify playbooks. Then there is logic
27:14app contributor who can view and run
27:16playbooks as well as create and edit
27:18playbooks. There's also another role
27:21called Microsoft Sentinel automation
27:23contributor which allows Microsoft
27:25Sentinel to add playbooks to automation.
27:30The best practice is to assign roles at
27:32the resource group level to
27:34automatically apply them to all related
27:37Sentinel resources and make sure that
27:40you use lease privilege that is grant
27:42only the access required for a users
27:45role. So when you're managing Microsoft
27:48Sentinel rules, not just the Sentinel
27:51predefined rules, but you may also
27:54encounter other Azure roles, there are
27:57Azure roles as well as log analytic
27:59roles. So Azure and log analytics roles
28:03provide broader access to resources. So
28:06Azure roles affect all Azure resources
28:09including Sentinel and log analytics.
28:12For example, owner full control
28:14including role assignments. Contributor
28:18can modify resources but not manage
28:21roles. Reader view only access. So these
28:24are broader roles, right? Then there is
28:27log analytics roles. These are
28:29specifically for log analytics
28:31workspace. There is log analytics
28:33contributor who can modify workspaces
28:36and manage settings. And then there is
28:39log analytics reader who has view only
28:41access to logs and analytics data. So
28:45the role overlaps can impact
28:47permissions. You need to keep that in
28:49mind. Suppose say there is a user with
28:51Microsoft Sentinel reader as well as
28:53Azure contributor role. This user can
28:57modify Sentinel resources because of
28:59their contributor role at the Azure
29:02level. So if you only want a user to
29:05have Sentinel specific permissions,
29:08check and remove broader roles like
29:10Azure contributor or owner unless it is
29:13necessary. So the best practice is to
29:15assign permissions at the lowest
29:17possible scope like for example Sentinel
29:20Workspace or Azure resource group to
29:22minimize access.
29:25So let's look at some roles and
29:27permissions recommendations by
29:29Microsoft.
29:30Security analysts are provided with
29:33Microsoft Sentinel responder and
29:35Microsoft Sentinel playbook operator
29:37roles. So they can view and manage
29:40incidents workbooks and other Sentinel
29:43resources because of the responder role
29:46and because of the playbook operator
29:48role they can attach and run playbooks.
29:51Then for a security engineer, Microsoft
29:54Sentinel contributor and this will allow
29:57them to manage incidents, create and
29:59edit workbooks, analytic rules and
30:02install solutions. And they are also
30:05given logic apps contributor. This is to
30:08run and modify playbooks. And then there
30:11are service principles for whom the
30:13recommendations is to recommendation is
30:15to provide Microsoft sentinel
30:17contributor role at the sentinel
30:19resource group level and they can use it
30:22for automated configuration and
30:25management tasks.
30:27Also remember that Microsoft in ID roles
30:30may be required for some integrations.
30:32For example, security administration is
30:35needed to set up data connectors for
30:38external services.
30:40So let's see a demo how to assign these
30:43roles. So the best practice is to assign
30:45at the resource group level or the log
30:49analytics space workspace level but I'm
30:52going to do it at the resource group
30:53level. Search for resource groups and
30:57I'm going to select Sentinel RG the
30:59resource group that we created.
31:02Then in the left hand menu select access
31:05control identity and access management
31:08and then click add add role assignments
31:12and then here search for sentinel. So
31:15you can see there are six predefined
31:18roles
31:20and remember in addition to using
31:22built-in roles Azure custom roles can
31:24also be used. So these custom roles will
31:28let you define specific permissions
31:30beyond the built-in roles. For example,
31:33if you want to create readonly access
31:35for specific Sentinel data, you can use
31:38custom role. Or if you want to restrict
31:41permissions to workbooks, playbooks or
31:43automation roles or even allow third
31:46party integrations with minimal
31:47privilege, you can start using custom
31:50roles. Okay. In the role assignment,
31:52maybe I will select Microsoft Sentinel
31:55contributor and then click next.
31:59Next is the members field. So it's
32:02telling us that the selected role is
32:03Microsoft Sentinel contributor. Then
32:06assign access to user group or service
32:09principle. This is for individual
32:12accounts or groups. So user represents
32:15an individual person with an Azure AD
32:18account or enter ID account like for
32:21example a security analyst uh needs read
32:24only access or Microsoft sentinel
32:26contributor role. Next is the group. A
32:29group is a collection of users and in
32:31ID. So assigning roles to a group
32:33automatically applies those permissions
32:36to all members in that group. Next is a
32:38service principle. A service principle
32:41is a nonhuman
32:43identity representing an application or
32:45service that requires permissions. This
32:47is used when an external or custom
32:50application interacts with Azure
32:52resources like for example API access.
32:55Use case would be like you know a third
32:57party SIM tool needs access to Microsoft
33:01Sentinel. That's about user group or
33:04service principle. Next is managed
33:07identity. Like you can see this is for
33:09automation or services. Azure managed
33:12identities are a special type of service
33:15principle managed by Azure. They are
33:18used to grant secure automated access to
33:21Azure resources without storing
33:23credentials. Usually this is the
33:25recommended practice.
33:28So why do we use managed entities?
33:30because they require no passwords or
33:32secrets like how service principles
33:35require them right these don't require
33:37them automatic entry ID authentication
33:41happens with this and these are more
33:43secure than storing credentials in
33:45applications so that's why you use
33:48managed managed identities there is a
33:51separate video every explaining
33:53everything about managed identity I will
33:55link it in the description box if you
33:57want more details for Now I'm going to
34:00select user group or service principle
34:04and then in the members select members.
34:08You usually have a lot of other users
34:10but here I have only my user name. So
34:14I'm going to select that and then say
34:16select and then scroll down. You can add
34:19a description why this access is
34:21provided as well and then click on next.
34:25It will allow you to validate the
34:27settings. Then you can click on review
34:29and assign. Then if you look at the
34:31notifications, it says that this account
34:34was added as Microsoft Sentinel
34:37contributor for Sentinel RG that is the
34:40resource group name. So that's the end
34:42of chapter 2. We saw how to deploy
34:46Sentinel, what are the prerequisites,
34:48what are the roles and permissions, how
34:50you can assign them.
34:52You are now on chapter three where we'll
34:54discuss content hub and data connectors.
34:57So let's get started. In the previous
35:00chapters, we have discussed about
35:02Sentinel architecture and deployment.
35:04Now, one of the next critical steps is
35:07data ingestion. That is bringing in logs
35:10from various sources for monitoring,
35:12detection, and response. First, let's
35:15look at some common types of data
35:18sources typically integrated with
35:20Sentinel or any other SIM solution. The
35:23first data source is Microsoft services.
35:26This can include Azure AD that is Azure
35:28Active Directory or Enra ID. It can
35:31include sign-in logs, audit logs or
35:33risky users. Microsoft Defender suite
35:37including Defender for Endpoint,
35:39Defender for Office 365, Defender for
35:42identity, Defender for Cloud Apps. This
35:45can include thread detection data. Then
35:47Microsoft 365 including Exchange,
35:50SharePoint and Teams. So, security and
35:53compliance logs.
35:55Then, Microsoft Defender for Cloud
35:58including security alerts and
35:59recommendations.
36:01Next, you can also include Azure
36:03resources like for example Azure virtual
36:07machines and storage including security
36:09logs from cloud hosted VMs. Then, Azure
36:13firewall and NSGS that is network
36:15security groups. You can include network
36:17traffic and firewall logs. Then Azure
36:21monitor and log analytics. This is
36:23general application logs and
36:25diagnostics.
36:27Next data source is from security tools
36:30or applications and network devices.
36:33This can include your firewalls and
36:35proxy logs like for example from
36:38PaloAlto, Forinet, Checkpoint, Zcaler,
36:41right? It can also include web
36:43application firewalls like for example
36:45Azure WF or F5 or Emperwam. Then logs
36:49from IDS or IPS that is intrusion
36:52prevention or detection systems like for
36:54example from Cisco firepower or snot.
36:59Next is EDR that is endpoint detection
37:01and response. This can include tools
37:04like crowd strike semantic Microsoft
37:07defender for endpoint. Then from other
37:10SIM solutions or log man log management
37:13solutions like for example Splunk,
37:16Arcite, Syslog sources.
37:19Next is identity and access management
37:21logs that is AM logs. This can include
37:24Microsoft enter ID that is Azure ID and
37:28conditional access logs octa and ping
37:31identity active directory if you have
37:33one on on premise and from active
37:36directory that is the on-remise one you
37:39can include domain controller logs
37:41kerros authentication logs then you can
37:44also include privileged access
37:46management systems that is spam systems
37:49logs like cyber arc then there is cloud
37:52and hybrid logs logs that is from AWS,
37:56Amazon web services logs including say
37:59cloud trail VPC flow logs, guard duty,
38:02security hub. Then we have Google cloud
38:06platform logs that is GCP logs, cloud
38:09audit logs, security command center
38:11logs. Then we have Kubernetes or
38:15container security logs for example AKS
38:17that is Azure Kubernetes services,
38:20Kubernetes audit logs and Aqua security.
38:24Then data sources from threat
38:26intelligence feeds for example Microsoft
38:29threat intelligence feeds. This is
38:32integrated Microsoft Defender TI. TI is
38:35threat intelligence. You can also get it
38:38from external threat intelligence
38:40providers like for example virus total
38:43or recorded future. Then you can also
38:46have custom TI feeds threat intelligence
38:49feeds example JSON or CSV based threat
38:54intelligence sources. The next data
38:56source could be an application and
38:59database log source. Database security
39:01logs can be from say SQL server, Oracle
39:05DB, MongoDB. Custom application logs can
39:09be you know rest API injection, app
39:11service logs, devop, dev sec ops tools.
39:15You can also include enterprise
39:17application logs from say SAP service
39:21now or work team. Then is the IoT and OT
39:25security logs. This can include IoT
39:28security solutions for example Azure IoT
39:31defender or AWS IoT logs and also
39:35scattera and in industrial control
39:38systems ICS like for example cements and
39:41then you can include your custom data
39:44logs like for example your SIS log and
39:47common event format that is CVF logs.
39:50This is for generic log injection. You
39:52can also include Azure event hub or blob
39:56storage that is streaming log data or
39:59rest api based log injection for
40:01unsupported tools. Here you can also
40:04include the OS logs that is from Windows
40:07event logs including security system and
40:09application logs. Then we have Linux
40:12logs including SIS log, audit dlogs,
40:16secure logs. then Mac OS logs including
40:19system logs, unified logging framework.
40:23So these are only some of the common
40:25data sources used in Microsoft Sentinel
40:28or any other SIM solution. Your setup
40:31might differ but the fundamental topics
40:34remain the same giving you a general
40:36idea of what to include in your security
40:38monitoring strategy. You think of any
40:41log that can most probably be integrated
40:44with Sentinel and it is easy to do it.
40:47Now how do we do this? This is done
40:50using data connectors. Let's go to the
40:52Azure portal. Okay. Now we are in the
40:55Azure portal that is portal.asure.com.
40:58Search for Sentinel. Select the Sentinel
41:01workspace. On the left side if you
41:04scroll down to configuration under that
41:07we have data connectors. Let me click on
41:09that. So it says zero connectors because
41:12I've not connected anything yet. And
41:15also it gives us a message. Let me
41:18increase the font. Gives us the message
41:20that data connector with content source
41:23gallery content have been removed. All
41:26the removed content and more is
41:28available in content hub. So most of the
41:32data connectors now you can find them
41:34under content hub. So this is a
41:37relatively new change where all the data
41:40connectors are moved under content hub.
41:43So if I scroll down on the left
41:45navigation under content management
41:48there is content hub. Let's click on it.
41:51So now you see there are 384 solutions
41:55uh standalone contents that is 3 307 not
41:59uh zero installed because we have not
42:01done that yet. So if you scroll down
42:04here, you can see all the content titles
42:07that are available. Now let's see what
42:09is a content hub, right? Microsoft
42:12Sentinel content hub is a relatively new
42:15update from Microsoft that consolidates
42:18all items under one unified interface.
42:21So, Microsoft Sentinel Content Hub
42:24provides essential SIM components that
42:26enable organizations to ingest data from
42:29various sources, monitor and detect
42:32threats, generate alerts and incidents,
42:36hunt and investigate security events,
42:39respond to threats with automation, and
42:42integrate with different platforms and
42:44services.
42:46Microsoft Sentinel offers a range of
42:48content types designed to enhance
42:50security operations. Let's see what are
42:53those data connectors. This is to enable
42:56log inition from you know diverse
42:58sources like we discussed Microsoft
43:01services third party security tools on
43:03prim systems. This is to provide
43:06seamless integration for realtime
43:08security monitoring. Next parsers. These
43:13convert raw log data into ASM format
43:16that is advanced security information
43:18model format. So it ensures structured
43:21and consistent data across various
43:24content types. Next workbooks. These
43:27provide interactive dashboards for
43:29monitoring and visualizing. This also
43:32helps security teams gain insights from
43:35log data. Then analytics rules. They
43:38generate security alerts and trigger
43:41sock actions. They detect anomalies and
43:44potential threats automatically. Next is
43:46hunting queries. They allow sock
43:49analysts to proactively search for
43:51threats and help detect hidden malicious
43:54activities. Next, notebooks. These
43:57enable advanced security investigations
44:00using Jupiter and Azure notebooks. They
44:03leverage AI and machine learning for
44:06deeper threat analysis.
44:09Next is watch lists. They store custom
44:12data sets like for example VIP users,
44:16blocked IPs, blocked domains. This is
44:19for enhanced threat detection. This can
44:22help reduce alert fatigue by
44:24prioritizing high-risk entities. Next is
44:28playbooks and logic apps. This is to
44:31automate incident response and
44:33remediation. They can be integrated with
44:36thirdparty tools to streamline sock
44:39workflows.
44:41Now if I go back to the Azure portal in
44:44the content type, if I want to filter,
44:46you'll see all the content types that
44:48are available. We will talk about all
44:51these content types in depth in the
44:54coming chapters. But for now, in this
44:56chapter, we will concentrate on data
44:59connectors. How to connect data to
45:02Sentinel?
45:04There are different types of data
45:05connectors that are available in
45:07Microsoft Sentinel. So when you're
45:09selecting data connectors for your
45:10environment, follow a prioritized
45:13approach. Like for example, you can
45:16start with free data connectors and
45:19these provide immediate value in
45:20Microsoft Sentinel while you plan for
45:22additional connectors and budgeting.
45:25Next, you can review the custom data
45:27connectors like you know you assess
45:29whether custom integrations are
45:31necessary for your environment.
45:34Next, you evaluate partner data
45:36connectors. This is to identify any
45:38third-party integrations that align with
45:40your security needs. For both custom and
45:44partner connectors, the recommendation
45:46is to prioritize CF or SIS lock
45:49connectors starting with the most
45:51critical ones. And then you also include
45:54Linux- based devices that require
45:56integration.
45:58So there is a separate chapter which
45:59will talk about how to connect CVF or
46:02syslog connectors Linux based devices to
46:05Sentinel. In that you will learn what
46:08are the best practices related to these
46:10connectors. One of the things that
46:13Microsoft Sentinel recommends is that if
46:15data ingestion costs become excessive,
46:18right? You can consider stopping or
46:21filtering logs forwarded using the Azure
46:24monitor agent to optimize these costs
46:28that that also we will see in the future
46:30chapters.
46:32As of today, these are the data
46:34connectors as well as their data types
46:37that are free in Microsoft Sentinel and
46:39Log Analytics. Like you can see there is
46:42Azure activity logs, health monitoring
46:44for Microsoft Sentinel, Microsoft Entra
46:47ID protection that is Azure AD
46:49protection, Office 365, Microsoft
46:52Defender for Cloud, IoT, XDR, Endpoint,
46:56identity and cloud apps. Like you can
46:59see in this table, there are some
47:01Microsoft services like for example,
47:03Defender for Cloud, IoT, XDR, Endpoint,
47:06identity or cloud apps. The free data
47:09type is security alert. The security
47:12alerts are free. But if you are trying
47:15to ingest raw logs from these Microsoft
47:18services, they may incur charges. So
47:22remember that not all logs need to be
47:25ingested. So you need to prioritize
47:27essential data to control costs. So many
47:30of the data connectors right they
47:32include both free and paid data types.
47:35You can customize settings to enable
47:37only the necessary data types reducing
47:40unnecessary costs.
47:42Let's see how you can customize these
47:44settings only to enable the necessary
47:47data types in Azure uh portal. Search
47:50for Sentinel. Select Microsoft Sentinel
47:53and then Sentinel Workspace
47:56and then under content management,
47:58select content hub. Let's search for
48:01Defender for Cloud Apps. Click on enter.
48:05So there it is. Microsoft defender for
48:07cloud apps. Something you want to note
48:10is under this there are many other items
48:13that are available as well. So if you
48:16see content sources, there is solution
48:19as well as standalone content hub
48:22solutions. These are prepackaged
48:24security solutions that include multiple
48:27components like data connectors,
48:29workbooks, analytics rules, hunting
48:31queries, automation playbooks. These are
48:34designed for specific security use cases
48:37like for example ransomware detection or
48:40compliance monitoring. This is made for
48:42easier deployment. So one-click
48:45installation for comprehensive coverage,
48:47right? And there are standalone content
48:50sources. These are individual connectors
48:53that ingest raw logs from a specific
48:56service like for example defender for
48:58endpoint or enter ID. They are more
49:01flexible. Organizations choose what data
49:04to collect and build their own analytics
49:06rules. These require manual
49:09configuration for alerts, dashboards or
49:11automation. Let me select all and cancel
49:15this for now. So here if you see, let me
49:18scroll to the right. It says content
49:21type data connector workbook analytics
49:24role. The one that we want to check is
49:26for the data connectors. Before that I
49:30need to install this solution Microsoft
49:32Defender for clouds. Let me click on
49:34install and it is getting installed. It
49:37might take a minute. So now installation
49:40is successful. Let me close this
49:42notification bar and let me close this
49:45as well. And now in the status I will
49:48select only the installed data
49:50connectors. Then say apply Microsoft
49:53Defender for cloud apps installed. Now I
49:56will select where the content type is
49:59data connector. So click on that and
50:03then click open connector page. It takes
50:06a minute to load because this is the
50:08first time we have opened this page.
50:10It's verifying all the prerequisites. So
50:13these are the prerequisites. First you
50:15need to have a workspace with read and
50:18write permissions. Tenant permissions
50:20should be global administrator or
50:22security administrator on the workspace
50:25tenants. And then you need to have
50:27license M365 E5 or M365 A5 or any other
50:33Microsoft Defender XTR eligible license.
50:37Note that all of these services like uh
50:40all Microsoft Defender products for
50:42example require different licenses. You
50:45can connect them without an appropriate
50:48license as well. But connecting them
50:50without the appropriate license won't
50:52generate any data. So it makes no sense
50:54connecting it to your Sentinel
50:56workspace. So you need to have licenses
51:00for these products and then connect that
51:02to Sentinel. Now if I scroll down in
51:05configuration you can also see that
51:08after you connect Microsoft Defender for
51:11cloud apps the alerts and discovery logs
51:14are sent to this Microsoft Sentinel
51:16workspace. Here you will get an option I
51:20don't have the license so it is disabled
51:22but if you have the license you'll get
51:24an option to select only alerts and you
51:26can ingest only alerts instead of all
51:28the raw logs. There are many other ways
51:31to you know manage your costs like for
51:34example you can use Microsoft Sentinel's
51:37cost analysis tools. Uh you can review
51:40and adjust retention policies to avoid
51:42unnecessary storage costs or you can
51:44leverage automation or filtering to
51:47manage data efficiently.
51:49So these are the free data connectors
51:51and their data types that you can
51:53connect to Microsoft Sentinel without
51:55any charges. Next is custom data
51:58connectors. So there are many out
52:01ofthe-box data connectors that are
52:03provided by Microsoft Sentinel. But if
52:06you have a tool or an application or a
52:08solution that you cannot connect using
52:10these out ofthe-box connectors, then you
52:13can consider creating your own custom
52:16data connector. Let's see some of the
52:18methods that are available for custom
52:21connectors creation. First one is
52:23codeless connector platform CCP. This
52:27allows customers and partners to create
52:29custom data connectors for Microsoft
52:31Sentinel without writing codes. So it
52:35uses a configuration file for easy
52:37deployment and it can be deployed to
52:40your own workspace or as a solution in
52:42Microsoft Sentinel Hub. It is fully SAS
52:46based. No no need for any additional
52:49service installations. It also includes
52:51health monitoring and full Microsoft
52:53Sentinel support. Say for example a
52:56security team wants to ingest logs from
52:58a third-party SAS application that is
53:01not natively supported by Sentinel.
53:04Instead of building a custom API
53:06integration, you can use CCP to
53:09configure and deploy a codeless
53:11connector like the name says codeless.
53:14This reduces development effort and
53:16maintenance costs. Next one is
53:19connecting with Azure monitor agent. AMA
53:23allows you to collect and send custom
53:25log data from textbased event sources to
53:29Microsoft Sentinel. So there'll be
53:31separate chapters on all of these
53:34connectors, how to connect them, what
53:36you can connect them, what are the
53:38different use cases, examples, right?
53:41For now, this is only a brief
53:43introduction to these connectors. So
53:45this AMA that is Azure monitor agent is
53:48best for textbased log sources like for
53:51example application logs or system logs.
53:54It is more efficient and secure than
53:56legacy agents like log analytics agents.
53:59It provides centralized data collection
54:02and advanced filtering to optimize log
54:04inition costs. So if your company has a
54:08legacy application that generates
54:10security logs as text files, then
54:12instead of writing a custom integration,
54:15you can configure Azure monitor agent to
54:18collect and forward logs to Microsoft
54:20Sentinel. And the next custom data
54:23connector is Log Stash. Log Stash is an
54:26open-source data processing pipeline
54:29that collects, transforms, and routes
54:32log data from various sources. It
54:36supports a wide range of input sources,
54:38for example, files, databases, cloud
54:41services, and event hubs. This also
54:44allows for advanced filtering like parse
54:47logs, remove unnecessary data, and mask
54:49sensitive information. This uses
54:52Microsoft Sentinel log stash output
54:55plug-in to send transform data directly
54:58to Sentinel. This is best for
55:01organizations needing flexibility in log
55:04processing before ingestion. So you
55:06ingest logs from multiple sources using
55:08log stash input plugins, transform and
55:12filter data using log stash filtering
55:15plugins and then send the process data
55:17to Microsoft Sentinel using the log
55:20stash output plug-in. Suppose say a
55:23company collects AWS cloud trail logs
55:26for thread detection. Instead of sending
55:28raw logs, right, you can use log stash
55:31to filter out unnecessary events and
55:34mass sensitive data. This is a use case
55:37example for log stash. And then there is
55:40Azure logic apps. This is a serverless
55:42workflow automation service that enables
55:45you to create custom connectors for
55:47Microsoft Sentinel without managing
55:50infrastructure. So it is best for low
55:53volume data injection or enriching
55:56existing data in Sentinel. It automates
55:59data retrieval from APIs, databases and
56:02files. It supports eventdriven workflows
56:06for realtime data streaming. There is no
56:09coding required for Azure um logic apps.
56:12It's only drag and drop interface for
56:14workflow creation. So something that you
56:17should keep in mind is that logic apps
56:19can become very expensive for high
56:21volume data ingestion. It is only best
56:24suited for ondemand enrichment or
56:27smallcale data collection. For example,
56:30you can use it for recurring tasks like
56:32you know schedule data retrievalss from
56:35APIs, databases or files such as fetch
56:39logs from SAS platform every 15 minutes.
56:42You can use it for ondemand triggering
56:45like manually trigger workflows for ad
56:48hoc uh data collection or testing. You
56:51can also use it for realtime streaming
56:54when the source system can push data to
56:57Sentinel like for example HTTPS
57:00endpoint.
57:02And the next option for custom data
57:04connector is to connect with log
57:06injection API.
57:08Log Analytics data collector API allows
57:11you to send log data directly to
57:14Microsoft Sentinel through a restful
57:16endpoint. This allows for maximum
57:19flexibility that is you can customize
57:21log in from any source. It has realtime
57:25data streaming that is push logs as
57:27events occur and there is no dependency
57:30on other tools. It works independently
57:33of logic apps or AMA or log stash. It
57:37supports structured data that is you
57:40need to send JSON formatted logs without
57:42modification. Remember for this rights
57:45you need programming knowledge to
57:48implement the API calls. You also need
57:52manual log formatting that is logs must
57:55follow log analytics schema. Also
57:58consider rate limits and authentication
58:01which needs to be managed later. So here
58:03you prepare log data in JSON format,
58:06authenticate using an Azure shared key
58:09or Azure AD token and then send logs
58:12using an HTTP post request to the API
58:15endpoint. Suppose an organization wants
58:18to stream realtime security logs from a
58:21custom application. They can develop a
58:23Python script that formats logs as JSON
58:26and then sends them to Sentinel using
58:29the API. And then Azure Functions. Azure
58:33Functions is a serverless computing
58:35service that allows you to execute code
58:37in response to events such as HTTP
58:41requests, timers or messages from other
58:44services.
58:46This is serverless, so you don't need to
58:48manage any infrastructure. This supports
58:50multiple programming languages like for
58:52example Python, PowerShell, JavaScript
58:55and so on. Right? Something that you
58:58need to consider when it comes to Azure
58:59functions is that there is execution
59:01time limits that get applied. Default 5
59:04minutes for consumption plan cost
59:07considerations for high volume data
59:10ingestions because this is not the one
59:12recommended for high volume data
59:14ingestion and it also requires
59:16authentication to interact with
59:18Microsoft Sentinel. So for Azure
59:21functions, you need to develop an Azure
59:23function in PowerShell, Python or any
59:26other language. Configure the function
59:28to receive logs through REST API, HTTP
59:31triggers or event sources. Then
59:34transform and format the logs into the
59:36required schema and then send logs to
59:39Sentinel using the log analytics API.
59:43You can also consider parsing custom
59:45connector data in Microsoft Sentinel. So
59:48when ingesting custom logs right into
59:50Microsoft Sentinel, parsing ensures
59:53structured query ready data that
59:56integrates with Sentinel's built-in
59:58security content. You can use ASIM
1:00:01parsers that is advanced security
1:00:03information model parsers. ASIM
1:00:06standardizes data for use across
1:00:08Sentinel's built-in queries, analytic
1:00:11rules, and workbooks. It simplifies
1:00:13query creation for security analysts.
1:00:16And it also enhances performance by
1:00:18reducing the need for on the-fly query
1:00:21time parsing. So you perform
1:00:23pre-ingestion parsing wherever possible.
1:00:27It reduces query load. And then you
1:00:29implement ASM parsers to align with
1:00:32Sentinel's data schema. And then you
1:00:35normalize fields such as source IP,
1:00:38destination IP, and even time for
1:00:40consistency. And then you deploy this to
1:00:43production. before deploying it to
1:00:45production, make sure that you test your
1:00:47parsing logic. So this is about custom
1:00:50data connectors. Again, there'll be
1:00:52different chapters dedicated to each of
1:00:54these data connectors. It'll mostly be
1:00:57demos. For now, get the brief idea about
1:01:01what these data connectors are. So we
1:01:03saw what are free data connectors and
1:01:05custom connectors. There are also
1:01:07partner data connectors. So, partner
1:01:10data connectors are pre-built
1:01:12integrations developed by Microsoft
1:01:14security partners that allow
1:01:16organizations to ingest third-party
1:01:19security data into Microsoft Sentinel.
1:01:22Let's go back to the portal. Let me
1:01:24close this. Let me delete this search
1:01:27option and then status. I'm going to
1:01:30select all and then apply. Now, if you
1:01:33look at the provider option, there are
1:01:36many providers that are present here.
1:01:38These are the partner data connectors.
1:01:42There are different types of partner
1:01:44data connectors like security solutions.
1:01:46Example include PaloAlto, Cisco,
1:01:49Checkpoint, Forinet, Semantic and
1:01:51Zcaler. Endpoint detection and response.
1:01:55EDR examples include Crowd Strike and
1:01:58Sentinel 1. Identity and access
1:02:01management, Octa, Cyber Arc, Pink
1:02:04identity. And then there is cloud
1:02:06security and casby including net scope,
1:02:10mcafey, bitclass. And then there are
1:02:13threat intelligence connectors as well
1:02:15like recorded future. So if I go to the
1:02:19search bar and search for threat, you
1:02:22can see all the uh data connectors that
1:02:25include threat in their title. If I
1:02:28scroll down, there is threat defense,
1:02:31then luminar threat intelligence, then
1:02:34another intelligence source. So you can
1:02:37install all of these, get licenses for
1:02:39them and you can install these as well.
1:02:42These are the partner data connectors.
1:02:45So there are free custom and partner
1:02:47data connectors. So that's it for this
1:02:50chapter.
1:02:52This is chapter four. In this we will
1:02:55discuss how to connect Entra ID that is
1:02:57formerly known as Azure Active Directory
1:03:00to Sentinel. In the previous chapters we
1:03:03learned about SE Sentinel architecture
1:03:06and deployment and what are data
1:03:08connectors what is content hub different
1:03:11types of data that you can connect to
1:03:13Sentinel. what are the different data
1:03:15sources, different types of data
1:03:17connectors and in this chapter we will
1:03:21cover one of the data connectors that is
1:03:23entra ID. Let's go to the Azure portal.
1:03:27So I've logged into portal.asure.com.
1:03:30Here let's search for entra ID. So there
1:03:34it is Microsoft enter ID. Let's click on
1:03:36it. So Microsoft Entra ID formerly known
1:03:40as Azure Active Directory or Azure AD is
1:03:43Microsoft's cloud-based identity and
1:03:46access management service. It is used to
1:03:48manage user authentication, single sign
1:03:51on that is SSO and access control for
1:03:53Microsoft 365, Azure and other cloud
1:03:56applications.
1:03:58There is a separate video which talks
1:04:00about Azure Active Directory in depth
1:04:02covering single sign on conditional
1:04:04access identity protection privileged
1:04:07identity management. So I will link that
1:04:10in the description box.
1:04:12So if I scroll down here
1:04:15you can see monitoring there is signin
1:04:18logs, audit logs, provisioning logs. If
1:04:21I click on signin logs
1:04:24you can see who logged in. This is the
1:04:28date. This is a request ID, user name,
1:04:31application was Azure portal and were
1:04:34they successful in logging in? Were they
1:04:37just trying failed loginins? All these
1:04:39details you get in signin logs.
1:04:42Similarly, we have audit logs.
1:04:45Audit logs provide a detailed record of
1:04:48activities related to user and admin
1:04:51actions.
1:04:52Now let's see how to connect Entra ID to
1:04:55Microsoft Sentinel.
1:04:57Let's search for Sentinel.
1:05:01Click on Microsoft Sentinel and then the
1:05:04Sentinel workspace and then under
1:05:07content management go to content hub.
1:05:10You can directly search for Entra ID.
1:05:14There is Entra ID protection
1:05:17and then there is Microsoft Entra ID.
1:05:20This is the one we want to install. So
1:05:23suppose say you don't know the name, you
1:05:25can also search with say content type.
1:05:28You need the data connector or it
1:05:31includes analytic rules. Then say apply
1:05:35and then support.
1:05:37It's needs to be supported by Microsoft
1:05:41provider. I'm going to uncheck all of
1:05:43these and select only Microsoft
1:05:47and then say apply
1:05:50and category. I'm going to uncheck all
1:05:52of these and search for identity. Yeah,
1:05:56identity and access management falls
1:05:58under identity. Say I'll say apply. So
1:06:02now you can see here that it has only
1:06:04three results. Now we have narrowed it
1:06:06to only three. Microsoft Entra ID, SAP
1:06:10applications and zero trust. The one
1:06:14that we want is Microsoft Entra ID.
1:06:17Let's click on it. So it says Microsoft
1:06:20Entra ID solution for Microsoft Sentinel
1:06:23enables you to ingest Microsoft Entra ID
1:06:26audits, signin, provisioning, risk
1:06:29events, and risky user or service
1:06:32principle logs using diagnostic settings
1:06:35into Microsoft Sentinel.
1:06:38So it has one data connector,
1:06:42two workbooks,
1:06:4463 analytics rules and 11 playbooks in
1:06:49this one solution.
1:06:51I'm going to say install.
1:06:54Now the installation is in progress. Now
1:06:57it's uh installed successfully. Let me
1:07:00close this. Let me minimize this as
1:07:03well.
1:07:04Now in the status if I select for
1:07:07installed
1:07:08let me select all of these
1:07:13I mean I'm just resetting the filter
1:07:17and only status I've selected installed
1:07:20and everything else is all.
1:07:24So now you can see I have only two
1:07:26solutions installed that is Microsoft
1:07:28Enra ID and Microsoft Defender for cloud
1:07:32apps which we did it in the previous
1:07:34chapters.
1:07:36Now if I go to configuration and go to
1:07:38data connectors
1:07:41here as well I can see that two
1:07:44connectors are connected to Microsoft
1:07:46Sentinel. If I click on Microsoft Entra
1:07:50ID and open connector page. Okay, first
1:07:54of all to connect Microsoft Entra ID to
1:07:57uh Microsoft Sentinel, there are a few
1:08:00prerequisites. First is that your
1:08:02workspace should have read and write
1:08:04permissions. This we covered in chapter
1:08:072 how to do this. Then diagnostic
1:08:10settings, read and write permissions to
1:08:12Microsoft Entra ID diagnostic settings.
1:08:16Then we need to have tenant permission,
1:08:19global administrator or security
1:08:21administrator on the workspace tenant.
1:08:24If you have created a free trial using
1:08:26your ID and logged in using the same ID,
1:08:30then you should be a global
1:08:31administrator by default.
1:08:35Let me open enter ID in a different tab.
1:08:39Search for enter ID. Microsoft enter ID.
1:08:43Here I just wanted to show you the
1:08:45diagnostic settings. This will allow you
1:08:48to send audit logs, signin logs and
1:08:51provisioning logs to external monitoring
1:08:54tools like log analytics, Microsoft
1:08:56Sentinel, Azure storage and event hubs.
1:09:00So if I click on add diagnostic setting,
1:09:04a diagnostic setting specifies a list of
1:09:06categories of platform logs and metrics
1:09:10that you want to collect from a resource
1:09:13and one or more destinations that would
1:09:15stream them to. So you can select what
1:09:18logs you want and what is the
1:09:20destination that you want to send to. So
1:09:23if I select send to log analytics
1:09:26workspace, it is telling me this is a
1:09:28subscription and it has selected my
1:09:30sentinel workspace. If you have other
1:09:32workspaces, you'll get an option to
1:09:34select the other one as well. So I just
1:09:38wanted to show this because we talked
1:09:40about the prerequisite in here that is
1:09:43diagnostic settings. Okay. Now under
1:09:46configuration you can actually select
1:09:49what kind of logs you want in Sentinel.
1:09:54For this example let's connect audit
1:09:56logs and signin logs and say apply
1:09:58changes and then let's go back to the
1:10:01homepage.
1:10:02Now let's try to create a user in Enra
1:10:05ID and see if you can see the logs in
1:10:07Sentinel. Let's go back to Enra ID.
1:10:11Under manage go to users
1:10:15and here click new user create new user
1:10:21and then provide the user principle
1:10:23name. I'll say testing audit logs.
1:10:30I'll provide the same display name
1:10:33and then say review and create.
1:10:37So it's created a user. Now if I refresh
1:10:40this page,
1:10:42there's another user called testing
1:10:44audit logs. Now if you go back to the
1:10:47Azure portal, then click on enter ID in
1:10:51data connectors. Click open connector
1:10:54page. If I scroll down on this, you can
1:10:58see that sign in logs and audit logs are
1:11:01getting ingested. This is one of the
1:11:03ways to verify that the logs are getting
1:11:06ingested to Sentinel. You can also see
1:11:09here that there is a line right audit
1:11:12log one sign in log one. So we are
1:11:15getting logs. Now let's see if you can
1:11:18see the user that we created audit logs.
1:11:20For that you can either go to go to
1:11:24analytics here. So it takes you to the
1:11:27logs page
1:11:29or you can go back to Sentinel and then
1:11:33click logs under general
1:11:38and here I'm going to search for audit
1:11:41logs. There is tab completion. So I can
1:11:44just click tab and it will complete my
1:11:47query to audit logs. And I'm going to
1:11:49run this.
1:11:52And here I can see the log. There is one
1:11:55log that is generated.
1:11:58It gives you the tenant ID, source
1:12:01system, resource ID, operation name is,
1:12:06let me actually zoom in a little bit.
1:12:11So there is tenant ID the time what time
1:12:15it was generated what time this action
1:12:17was taken then operation name add user
1:12:21we added a user category is user
1:12:24management if I scroll down
1:12:28you can still find additional details
1:12:32and then if I go to target resources and
1:12:35then expand this as well
1:12:39you can see that this account was added
1:12:42that is testing audit logs which we just
1:12:45created. So there is audit log that is
1:12:48getting generated and it is pushed to
1:12:50Sentinel. Let me go back to the query.
1:12:53Now let's search for signin logs.
1:12:57So there is sign-in log as well that is
1:13:00getting generated.
1:13:03There's a lot of information that you
1:13:05can find from these logs.
1:13:08If you know the KQL language that is
1:13:10used here, then it is very easy to query
1:13:13for the information that you're looking
1:13:15for.
1:13:16There will be a separate chapter on KQL
1:13:19as well. For now, we have verified that
1:13:22both of these logs that signin logs and
1:13:24audit logs are sent to Sentinel.
1:13:28You are now on chapter five. We will
1:13:30discuss the threat detection and
1:13:32mitigation workflow in Sentinel.
1:13:35This chapter that is the workflow
1:13:37chapter serves as the foundation for
1:13:39upcoming topics. We will go over all the
1:13:42key components of thread detection and
1:13:44mitigation in Microsoft Sentinel and in
1:13:47the following chapters we will discuss
1:13:49all of these components individually in
1:13:52more depth. The first component is data
1:13:55connectors. We have already covered data
1:13:58connectors in detail back in chapter 3.
1:14:00If you need a refresher, you can find
1:14:02the link to the video in the description
1:14:05box. So to quickly recap, data
1:14:08connectors are responsible for ingesting
1:14:10logs from various sources like for
1:14:12example Azure, AWS, GCP, Office 365,
1:14:16firewalls, endpoint security solutions,
1:14:19other security solutions, thirdparty
1:14:22services, and onremise systems.
1:14:25This is the first step in the Microsoft
1:14:28Sentinel workflow ensuring that raw
1:14:30security data is continuously collected
1:14:33and fed into the system for further
1:14:35processing.
1:14:37So previously as we discussed data
1:14:40connectors had a dedicated section in
1:14:42Sentinel. Now they are integrated under
1:14:45the content hub which centralizes
1:14:47connectors, analytic rules, workbooks
1:14:50and hunting queries in one place. And
1:14:53this update is to simplify deployment,
1:14:55management and discovery of new
1:14:57connectors.
1:14:59Once you have the raw logs from data
1:15:01connectors, the next step is to parse
1:15:03them. What are parsers?
1:15:06Parsers transform raw ingested logs into
1:15:10a structured format for analysis and
1:15:13thread detection. They help normalize,
1:15:16extract and enrich security data to make
1:15:21it usable in analytics rules, hunting
1:15:24queries and incident response.
1:15:27How parsers work? First receive logs
1:15:31from data connectors that is in the raw
1:15:33format and then extract and standardize
1:15:37fields like for example timestamps, IP
1:15:39addresses and user activity. Then format
1:15:42the data to align with Microsoft
1:15:44Sentinel's analytics engine. Then store
1:15:48parse data in log analytics making it
1:15:51accessible for analytics rules to detect
1:15:53threats, hunting queries for proactive
1:15:56investigation and workbooks and
1:15:58dashboards for visualization.
1:16:01There are different types of parsers in
1:16:04Sentinel. The first one is KQL parsers
1:16:07that is custoquery language parsers. As
1:16:10the name suggests it is written in KQL.
1:16:14Next one is normalization parsers. These
1:16:17transform logs into standard schemas.
1:16:20For example, advanced security
1:16:22information model ASM. And then we also
1:16:25have custom parsers. This allow
1:16:27organizations to define their own data
1:16:30transformation rules. Why parsers are
1:16:33important? Because they improve thread
1:16:35detection accuracy. That is they ensure
1:16:38rules analyze correctly formatted data.
1:16:41They also help with faster
1:16:43investigations. That is security
1:16:45analysts can query structured logs
1:16:47efficiently.
1:16:49And parsers align logs from different
1:16:51sources into a common schema for better
1:16:54correlation.
1:16:56So after parsers the next step is
1:16:59analytics rules. So data connectors and
1:17:02then parsers and then analytics rules.
1:17:05Analytics rules are the queries that are
1:17:08written in KQL language that
1:17:11continuously scan ingested log data for
1:17:14suspicious activities or anomalies.
1:17:17When certain conditions or thresholds
1:17:20are met, these rules trigger alerts to
1:17:23signal potential threats. So analytic
1:17:26rules convert past log data into
1:17:29meaningful alerts. You can customize
1:17:32analytics rules to fit specific threat
1:17:35scenarios or compliance requirements.
1:17:38Next, alerts. Like I mentioned, whenever
1:17:41an analytics rules conditions are met,
1:17:45like for example, say repeated failed
1:17:47login or anomalous network traffic, an
1:17:50alert is generated.
1:17:52Alerts are individual events indicating
1:17:55something suspicious or malicious might
1:17:58be happening. So these provide initial
1:18:01visibility into potential threats and
1:18:04then these alerts often serve as
1:18:06building blocks for incidents.
1:18:09Next, hunting queries. So analytic rules
1:18:12automatically generate threats, right?
1:18:15Hunting queries are manually executed by
1:18:17security analysts to uncover threats
1:18:20that might not yet have a defined rule.
1:18:23So these help in proactive threat
1:18:25hunting. Analysts use KQL to look for
1:18:29anomalies or patterns across large data
1:18:32sets to proactively discover advanced or
1:18:35hidden threads. Findings from hunting
1:18:38queries can also be escalated or
1:18:41converted into incidents if a threat is
1:18:43discovered. Next, watch lists. Watch
1:18:47listlets are usually used to store known
1:18:49indicators of compromise IOC's such as
1:18:53malicious IPs domains or user accounts.
1:18:57Sentinel automatically checks new events
1:18:59against these watch lists. If there's a
1:19:02match, it can raise an alert or flag it
1:19:05for investigation.
1:19:06When watch list items appear in logs,
1:19:09right, they can quickly generate or
1:19:11enhance existing incidents. They help
1:19:14prioritize alerts by correlating them
1:19:17with non-malicious indicators. So hence
1:19:20they reduce a lot of noise. Next
1:19:24incidents this is the central point of
1:19:26investigation. So what Sentinel does it
1:19:29correlates alerts uh relevant data from
1:19:32hunting queries or watch lists into
1:19:34incidents. This provides us a holistic
1:19:37view of potentially related threats.
1:19:41Incidents group together multiple alerts
1:19:43or suspicious activities that might be
1:19:46part of the same attack chain. So
1:19:48instead of investigating each alert in
1:19:51isolation, analysts can review an entire
1:19:54incident to see all related alerts,
1:19:57making it easier to see the bigger
1:19:59picture. Incidents are often assigned
1:20:01severity levels to help teams focus on
1:20:04the most critical threats first. Next,
1:20:07manual investigations.
1:20:10So this is human analysis. Security
1:20:12analysts dig into the details of an
1:20:15incident to validate the threat, gather
1:20:17context, and decide on remediation
1:20:20steps. Analysts may coordinate with
1:20:23other teams, run additional queries, or
1:20:25gather forensic evidence to confirm the
1:20:28scope of the threat. Some complex
1:20:30threats require human expertise to
1:20:33interpret subtle indicators and avoid
1:20:35false positives. Analysts can craft a
1:20:38custom response if the situation is
1:20:40unique or requires careful handling.
1:20:44Next is workbooks. They're used for
1:20:46dashboard and reporting. They create
1:20:48visual representations like for example
1:20:51graphs, charts on tables of incident
1:20:54data, trends, and overall security
1:20:56posture. Security teams can tailor
1:20:59workbooks to display the KPIs or metrics
1:21:02most relevant to their organization.
1:21:04So this provides immediate insights into
1:21:07ongoing uh incidents and detection
1:21:09trends. It's also useful for management
1:21:13or board level reporting as the data is
1:21:15displayed in a more intuitive format.
1:21:18Next, automation rules. Automation rules
1:21:22can tag, assign or prioritize incidents
1:21:24based on predefined criteria. So they
1:21:27help with automatic triage. They also
1:21:31serve as a gateway to run playbooks when
1:21:34certain conditions are met, like for
1:21:36example, high severity incidents. As the
1:21:40volume of alerts grow, automation rules
1:21:43help manage incidents at scale without
1:21:46overwhelming human analysts. They ensure
1:21:49that standard procedures are followed
1:21:51every time a certain type of threat is
1:21:54detected. Next, playbooks. This is the
1:21:57source solution. Playbooks are built on
1:22:00Azure logic apps. They define automated
1:22:03workflows that can respond to threats in
1:22:06real time. For example, blocking an IP
1:22:08address, disabling a compromised user
1:22:11account, sending a notification to a
1:22:13Slack or team's channel or creating a
1:22:16ticket in a service management platform
1:22:18like for example service. Now playbooks
1:22:20can interact with various internal
1:22:23systems like for example firewalls or
1:22:25endpoint solutions and also external
1:22:28services like for example ticketing,
1:22:31threat intelligence APIs and so on.
1:22:34These automated responses reduce
1:22:36incident resolution time significantly.
1:22:40Predefined workflows ensure consistent
1:22:42and correct actions minimizing mistakes
1:22:45during high pressure situations.
1:22:48Automation rules and playbooks work
1:22:50together. Automation rules you can use
1:22:52it for handling the initial triage for
1:22:55example you know the tagging or
1:22:58assigning an incident to somebody or
1:23:00closing an incident if it's a known
1:23:02false positive and playbooks handle the
1:23:05complex or external facing actions like
1:23:08I mentioned blocking an IP disabling an
1:23:11account integrating with ticketing
1:23:13systems. Then we have notebooks. These
1:23:17are often Jupiter notebooks. They allow
1:23:19deeper forensic analysis, machine
1:23:22learning and data correlation beyond
1:23:24standard dashboards. Analysts can
1:23:27document their workflow in a notebook,
1:23:30share it with team members, and iterate
1:23:33quickly on queries or scripts. So when
1:23:36the investigation requires advanced
1:23:38techniques such as behavioral analysis
1:23:41or machine learning, notebooks provide
1:23:43the necessary flexibility and power.
1:23:46Notebooks maintain a clear step-by-step
1:23:49record of the investigation which is
1:23:51invaluable for auditing or knowledge
1:23:54sharing. So putting it all together,
1:23:56analytics rules scan data from data
1:23:59connectors and trigger alerts upon
1:24:01detecting suspicious activities. Hunting
1:24:04queries and watch lists provide parallel
1:24:07ways to uncover and correlate additional
1:24:10threats. Alerts and any data from
1:24:13hunting queries or watch lists are
1:24:15grouped into incidents for a holistic
1:24:17investigation.
1:24:19from an incident. Security teams can
1:24:21launch manual investigations, use
1:24:24notebooks for deeper analysis, or
1:24:27consult workbooks for visual summaries.
1:24:30Automation rules categorize and manage
1:24:33incidents, triggering playbooks that
1:24:36enact automated responses.
1:24:39So, this is the workflow that is
1:24:41followed in Sentinel for threat
1:24:43detection and mitigation. I hope this
1:24:46detailed walkthrough clarifies how each
1:24:48component in the Microsoft Sentinel
1:24:50workflow fits together. Starting with
1:24:53data connectors and continuing through
1:24:56alerts, incidents and response
1:24:58mechanisms.
1:24:59Let's go to the next chapter. In this
1:25:02chapter, we will discuss threat
1:25:03intelligence in Sentinel. What options
1:25:06are available for threat intelligence?
1:25:08What is threat intelligence? How to use
1:25:10them? How can we integrate them with
1:25:13Sentinel? What are the different options
1:25:15and why they matter?
1:25:18First, what is threat intelligence?
1:25:20Threat intelligence has three key
1:25:22components. First one is data
1:25:24collection. Organizations gather raw
1:25:27data from multiple sources such as
1:25:30open-source intelligence, commercial
1:25:32feeds, internal logs, and information
1:25:35sharing communities.
1:25:37The next key component is analysis.
1:25:41To understand that key component first
1:25:43let's see what are TTPs. TTP stands for
1:25:47tactics techniques and procedures in
1:25:49cyber security. These describe how
1:25:53threat actors operates. Tactics is the
1:25:56why behind an attack like for example
1:25:58persistence, privilege escalation,
1:26:01exfiltration.
1:26:02Techniques is the how an actor achieves
1:26:05a goal. For example, fishing, lateral
1:26:08movement, credential dumping
1:26:11and procedures is the specific
1:26:13implementation of a technique by a
1:26:16particular adversary. For example, using
1:26:19mimikads for credential dumping. So
1:26:22analysis is where the collected data is
1:26:26processed and analyzed to uncover
1:26:28patterns TTPs used by thread actors. The
1:26:32result is actionable intelligence that
1:26:35informs security measures. The next key
1:26:38component is dissemination. The
1:26:40actionable intelligence is then shared
1:26:42with security teams, decision makers and
1:26:45sometimes automated systems enabling
1:26:48proactive defense and rapid response.
1:26:51So combining all list, threat
1:26:54intelligence is a process of gathering,
1:26:56analyzing and disseminating information
1:26:59about potential or active cyber threats.
1:27:02It helps organizations understand
1:27:05adversary behaviors, identify
1:27:07vulnerabilities, and make informed
1:27:09decisions to improve their security
1:27:12posture. To talk more about why threat
1:27:15intelligence matters, first one is
1:27:17proactive defense. By understanding
1:27:20adversary tactics, organizations can
1:27:23anticipate and prepare for potential
1:27:25attacks rather than reacting after a
1:27:28breach occurs. Enhanced detection.
1:27:31Integrating threat intelligence into
1:27:33security tools improves the ability to
1:27:36detect and block threats early. Informed
1:27:39decision making. Strategic and
1:27:41operational intelligence help shape
1:27:44security policies and investments.
1:27:47Ensuring resources are focused on the
1:27:49most pressing risks. Then improved
1:27:52incident response. Detailed technical
1:27:55intelligence equips security teams with
1:27:57the necessary data to quickly identify,
1:28:01contain, and remediate incidents,
1:28:03reducing the overall impact.
1:28:06There are different ways that you can
1:28:08import threat intelligence into
1:28:10Sentinel.
1:28:11The first solution is Microsoft Defender
1:28:14threat intelligence connector.
1:28:17This is the direct integration with
1:28:19Microsoft's curated threat intel.
1:28:22Microsoft defender threat intelligence
1:28:24that is defender TI provides public
1:28:27open-source and highfidelity indicator
1:28:30indicators of compromise that is IoC's
1:28:32to enhance threat detection and response
1:28:35in Microsoft Sentinel. What are IOC's?
1:28:38Indicators of compromise. These are
1:28:41forensic artifacts that signal a
1:28:43potential security breach, malware
1:28:46infection or unauthorized access in an
1:28:48organization's network. There are
1:28:51different types of IoC's like for
1:28:53example network-based IOC's include IP
1:28:56addresses that is malicious or
1:28:58suspicious IPs involved in attacks.
1:29:01domain names that is fishing or command
1:29:04and control domains and URL links that
1:29:07is malicious website pages used for
1:29:11driveby downloads or fishing. There are
1:29:14also host based IOC's like file hashes
1:29:17that is unique identifiers for malware
1:29:20files MD5 S shaft 256 all these are file
1:29:26hashes then registry changes
1:29:29unauthorized modifications indicating
1:29:31malware persistence then there are
1:29:33processes and services this is unusual
1:29:37processes or services running on a
1:29:39machine we also have email based IOC's
1:29:42like for example spoofed or suspicious
1:29:44email sources that is sender email
1:29:47addresses the spoofed or suspicious ones
1:29:50executables macros or infected files
1:29:54URLs directing users to fishing sites
1:29:57all these are IoC's indicators of
1:30:00compromise
1:30:02defender has a standard connector as
1:30:04well as a premium connector standard
1:30:07connector is free it ingests public and
1:30:09open-source threat intelligence premium
1:30:12connector is is the paid one which
1:30:14ingests Microsoft enriched and curated
1:30:17IOC's for deeper analytics.
1:30:20The premium connector right it is also
1:30:22available for a limited time free like a
1:30:25month or something that you can use and
1:30:27check out if that works for you. Now
1:30:30let's see a demo of how to connect this
1:30:32in Sentinel. Let's go to our portal.
1:30:36Search for Sentinel. Select Microsoft
1:30:39Sentinel and then click Sentinel
1:30:42Workspace.
1:30:44Then under content management,
1:30:46click content hub,
1:30:49search for threat intelligence.
1:30:55The first one which is Microsoft
1:30:57Defender for Threat Intelligence. That
1:30:59is the one we are looking for. Let's
1:31:01click on it.
1:31:02It says Microsoft centralizes numerous
1:31:05data sets into a single platform,
1:31:07Microsoft Defender Threat Intelligence,
1:31:10MDTI, making it easier for Microsoft's
1:31:13community and customers to conduct
1:31:15infrastructure analysis.
1:31:17If I scroll down, the pricing is free.
1:31:20There are eight playbooks and one
1:31:23workbook in this. Let me install this.
1:31:27Click install. Installation is pro is in
1:31:29progress now. Now it's completed. It
1:31:33says that the Microsoft Defender threat
1:31:35intelligence is installed
1:31:37and once this is installed you can find
1:31:40the new intelligence that is provided by
1:31:42defender in the management interface or
1:31:45if you go to logs
1:31:48under general you can query the threat
1:31:52intel indicators table.
1:31:55Right now I don't have any intelligence
1:31:57that is ingested but you will be able to
1:32:01find data here. Defender TI provides who
1:32:06is DNS records, SSL certificates and
1:32:11malware insights to give context to
1:32:13suspicious domains, IPs and hosts.
1:32:17It enables cross referencing IOC's with
1:32:20articles, attack patterns and
1:32:22vulnerabilities that is CVE data to
1:32:25understand the threat impact.
1:32:28So that's about defender TE. The next
1:32:30one is threat intelligence taxi. First
1:32:34let's see what is taxi. Taxi stands for
1:32:38trusted automated exchange of indicator
1:32:41information. It is a standardized
1:32:43protocol for sharing cyber security
1:32:46threat intelligence that is CTI.
1:32:49It shares this information in a very
1:32:51structured and automated way. It works
1:32:54alongside sticks that is structured
1:32:57threat information expression which
1:32:59provides the format for thread data.
1:33:03Think of sticks as the language and taxi
1:33:06as the delivery method for sharing
1:33:08threat intelligence between security
1:33:10tools.
1:33:12It provides automated threat
1:33:13intelligence sharing. It reduces manual
1:33:16effort in integrating threat fields.
1:33:19Now, Sentinel provides taxi data
1:33:22connector that is it supports industry
1:33:24standard feeds for external TI in
1:33:27ingestion.
1:33:28This data connector in Sentinel enables
1:33:31organizations to import threat
1:33:33intelligence data in sticks format from
1:33:36taxi toxervers.
1:33:39It uses the sticks format and taxi
1:33:43protocol that is widely adopted for
1:33:46sharing threat intelligence. Microsoft
1:33:48Sentinel has a native taxi client that
1:33:51can pull data from taxi 2.0 and 2.1
1:33:55servers eliminating the need for custom
1:33:57injection scripts. Let's see how to
1:34:00connect and configure a taxi connector
1:34:03in Sentinel. Now let's go back to our
1:34:05portal. In content hub under content
1:34:08management, I'm again searching for
1:34:10threat intelligence.
1:34:12Let me close this. So the first one is
1:34:15Microsoft Defender threat intelligence
1:34:17which we have already installed. But if
1:34:20I scroll down, there is also threat
1:34:22intelligence. This is the connector that
1:34:24we are looking for. Let's select this
1:34:28and say install. Installation is in
1:34:31progress.
1:34:33Now it is installed. Now let's say
1:34:36manage.
1:34:38It says that the threat intelligence
1:34:40solution contains data connectors for
1:34:42import of threat indicators into
1:34:45Microsoft Sentinel analytics rules for
1:34:48matching TI data with event data
1:34:51workbook and hunting queries. Threat
1:34:54indicators can be malicious IPs, URLs,
1:34:57file hashes, domains, email addresses
1:35:00and so on. So there are five data
1:35:03connectors, one workbook, 52 analytic
1:35:06rules and five hunting queries
1:35:10and then you also can see the different
1:35:13types of content that is available here.
1:35:16Let's go back one step. Let me minimize
1:35:19this. And if you see under threat
1:35:22intelligence, there is threat
1:35:25intelligence taxi. This is the connector
1:35:27that we need. So we'll say open
1:35:29connector
1:35:31and it is asking us for some information
1:35:34to add a taxi server. For this demo we
1:35:38will use pulse type threat feed. This is
1:35:41a very popular threat intelligence
1:35:43source.
1:35:44Let's sign up and then register.
1:35:52Provide your email address.
1:35:55registration link is sent to your email
1:35:58address
1:36:00and then here we need to provide a
1:36:02username.
1:36:04I'll say Sentinel
1:36:07Taxi demo.
1:36:10Once you fill this information, you can
1:36:12continue to the next page.
1:36:16Okay. Now I'm registered and logged in.
1:36:19Here you can see your account and then
1:36:21select account. here
1:36:24this is the API key that we need let's
1:36:27go back to our Sentinel portal
1:36:31friendly name I'm providing it as
1:36:33pulseive taxi
1:36:36and then it is asking for API root URL
1:36:39that is provided in the documentation I
1:36:42just searched for pulseive API root URL
1:36:46if I scroll down there is API root URL
1:36:49I'm going to copy this paste in API root
1:36:53URL. Then there is collection ID and
1:36:56collection ID is also provided in their
1:36:59documentation. If I scroll down, there
1:37:01is collections.
1:37:03There are three types of collections.
1:37:06Pulse type indicator data, thread data
1:37:08and test data. We will connect the test
1:37:11data. This is a test collection
1:37:14containing live sample indicator
1:37:15objects. It is available to anyone for
1:37:18free for the purposes of testing,
1:37:20debugging or even just learning about
1:37:23sticks and taxim. So I'm going to copy
1:37:25this, go back to Sentinel and provide
1:37:28this
1:37:30and then provide a username.
1:37:33I'll say taxi test and the password is
1:37:37from pulse. You see your API key here
1:37:41under your account. Copy that. Paste it
1:37:44in the password field. Import
1:37:47indicators. All available at most one
1:37:50day old, one week. I'm going to say all
1:37:52available. It also gives you polling
1:37:54frequency.
1:37:56How many types it has to pull with pulse
1:37:59type. I'm going to say once uh an hour
1:38:02and then say add. And then once it is
1:38:05connected, it might take some time for
1:38:08it to reflect data. You can query for it
1:38:10using the threat intelligence indicator
1:38:14table and then where provider name is
1:38:18equal to taxi
1:38:20and then where source system contains
1:38:23pulse dive.
1:38:25That's how you connect a taxi threat
1:38:28intelligence data connector. The next
1:38:30option is threat intelligence upload
1:38:33API.
1:38:34Many organizations rely on threat
1:38:37intelligence platforms that is tips to
1:38:40aggregate threat indicator feeds from
1:38:43multiple sources. A threat intelligence
1:38:46platform is a security solution designed
1:38:48to aggregate, analyze and operationalize
1:38:51threat intelligence from various
1:38:53sources. It collects threat intelligence
1:38:56from open-source, commercial and threat
1:38:58intel sources. converts raw data into a
1:39:02structured format that is sticks or taxi
1:39:05and enhances it with contextual
1:39:07information and then it identifies
1:39:10patterns, relationships and trends in
1:39:12thread data and then shares intelligence
1:39:15with SIMs for example in this case
1:39:18Microsoft Sentinel, EDR, XDR, firewalls
1:39:21and soore tools for automated security
1:39:24actions.
1:39:26The threat intelligence upload API
1:39:28enables organizations to import sticks
1:39:31objects directly into Microsoft Sentinel
1:39:34streamlining the integration of
1:39:36structured threat intelligence into
1:39:38security operations.
1:39:40This does not require any data
1:39:42connector. So it simplifies the
1:39:45integration process. It uses the widely
1:39:48adopted sticks format for better
1:39:50interoperability.
1:39:52It requires the Microsoft Sentinel
1:39:54contributor role for the Microsoft Entra
1:39:56application. We'll have to create a
1:39:58Microsoft Entra application for this.
1:40:01And then API permissions allow fine
1:40:03grain control over threat intelligence
1:40:06injection at the workspace level.
1:40:09For this the first step is to register a
1:40:12Microsoft Entra application. For that
1:40:15first sign into Microsoft Entra admin
1:40:18center that is entra.microsoft.com
1:40:22then I under identity go to applications
1:40:26and then app registrations
1:40:29in this new registration
1:40:32provide a name I'll say sentinel
1:40:36upload API test and then register it's
1:40:41created my application now this is the
1:40:44app ID that we need later. So let's keep
1:40:48this tab open. Once that is done, the
1:40:51next step is to assign a role to the
1:40:53application. Let's go back to the Azure
1:40:56portal.
1:40:58Let's go to the log analytics workspace.
1:41:02I selected Sentinel workspace. In this
1:41:04go to access control
1:41:07and then add and then add role
1:41:10assignments.
1:41:11In the role tab, search for Microsoft
1:41:14Sentinel contributor.
1:41:18Select it. Next. Click select members.
1:41:23Search for the app that we created.
1:41:25Sentinel upload test API. Select that
1:41:27application. Then say select
1:41:31and then review and assign.
1:41:34So the role is getting added to the app
1:41:36app registration that we just created.
1:41:40And once you're done with this, we need
1:41:42to submit the threat intelligence to the
1:41:44upload API. For that, you need the
1:41:47application client ID, Microsoft Entra
1:41:50Access token with OOTH 2.0
1:41:53authentication and Microsoft Sentinel
1:41:55Workspace ID. And this method is
1:41:58particularly useful for organizations
1:42:00that maintain proprietary threat
1:42:02intelligence. Companies using internal
1:42:05threat feeds that need to be injected
1:42:08into Sentinel can make use of this or
1:42:11security teams that are using custom
1:42:13scripts or automation pipelines for
1:42:15thread data can use this upload API
1:42:18method.
1:42:19The next method is thread intelligence
1:42:22platform. This data connector is on its
1:42:24path for deprecation.
1:42:27So this will be very soon removed by
1:42:29Microsoft Sentinel. So, Microsoft
1:42:31recommends switching to the threat
1:42:33intelligence upload API for improved
1:42:36functionality.
1:42:37So, if you're already using a tip data
1:42:40connector, start migrating to the upload
1:42:43API.
1:42:45So, this was designed to import threat
1:42:47intelligence indicators from external
1:42:49threat intelligence platforms into
1:42:51Sentinel. So, now it's going to be
1:42:54removed. So, better start using upload
1:42:57API. There are other ways to import
1:43:00threat intelligence as well. You can
1:43:02upload a file in Azure portal or you can
1:43:06do it also in uh defender portal under
1:43:09threat management threat intelligence.
1:43:12In Sentinel go to threat management.
1:43:15Under threat intelligence
1:43:18say import in this you can import using
1:43:21a file. So the format is CSV and JSON as
1:43:26well. You can select one of those file
1:43:28formats and then upload your um threat
1:43:32intelligence file. You can also add
1:43:35entity to threat indicators in an
1:43:37incident. You can also use threat
1:43:39indicators in analytics rules. You can
1:43:42use matching analytics to detect
1:43:44threats. All of this we'll see in the
1:43:47later chapters.
1:43:49So that is it about threat intelligence.
1:43:52Now we'll move on to the next chapter.
1:43:54In this chapter, we will discuss
1:43:56analytics rules that are required for
1:43:58thread detection. So, what are analytics
1:44:01rules? Analytics rules in Microsoft
1:44:04Sentinel are automated detection
1:44:06mechanisms that continuously analyze
1:44:08security data to identify potential
1:44:11threats. These rules help security teams
1:44:14to detect, investigate, and respond to
1:44:16suspicious activities in an
1:44:18organization's environment. As a recap,
1:44:22Microsoft Sentinel ingest data from
1:44:24various sources such as firewalls,
1:44:27endpoint protection tools, cloud
1:44:30services, and identity platforms.
1:44:32Analytics rules scan this data for
1:44:35threat indicators, anomalies, or
1:44:37suspicious behavior patterns. When an
1:44:40analytics rule detects a security issue,
1:44:42it triggers an alert. This alert
1:44:45contains details about the detected
1:44:47event such as affected users, devices,
1:44:51IP addresses, and timestamps. Then
1:44:54alerts are grouped into incidents which
1:44:57act as case files for investigation.
1:45:00This helps analysts see the bigger
1:45:02picture and determine if multiple alerts
1:45:05are related to a coordinated attack. And
1:45:09then playbooks that are automated
1:45:11workflows can be triggered to contain
1:45:14threats, notify teams or gather more
1:45:17data. Security analysts can also
1:45:20manually investigate incidents and take
1:45:22necessary actions to mitigate risks.
1:45:25There are different types of analytics
1:45:28rules. Let's look at the main ones
1:45:30first. Schedule rules. These run at
1:45:34predefined intervals scanning collected
1:45:37logs for specific patterns. These are
1:45:40useful for detecting known attack
1:45:42techniques such as repeated failed login
1:45:45attempts or unauthorized access. Next
1:45:48one near realtime NRT rules. These
1:45:52execute queries at high frequency for
1:45:54rapid thread detection. I mean these are
1:45:57getting executed every 1 minute. These
1:46:00are ideal for time-sensitive threats
1:46:02like brute force attacks or unusual
1:46:05administrative activity. Then there are
1:46:08anomaly rules. These use machine
1:46:10learning to detect unusual behaviors
1:46:13that may indicate threats. For example,
1:46:16a user accessing a system from an
1:46:18unexpected location or logging in at odd
1:46:21hours. Then we have Microsoft security
1:46:24rules. These rules integrate with
1:46:27Microsoft security products for example
1:46:30Microsoft Defender or Entra ID to
1:46:33generate alerts in Sentinel. This allows
1:46:36seamless correlation of security signals
1:46:38from across the Microsoft security
1:46:40ecosystem.
1:46:42Then there are three more specialized
1:46:44analytics rules in these. So the first
1:46:47four that we discussed are the primary
1:46:49rule types. Then Sentinel offers
1:46:51specialized rule templates with limited
1:46:54customization options. You can't
1:46:56customize them so much but they they are
1:46:58there. First specialized rule is uh
1:47:01threat intelligence rules. These compare
1:47:04ingested security locks against known
1:47:07threat indicators like malicious IP
1:47:09addresses or domains. These are useful
1:47:13for identifying compromised accounts,
1:47:15fishing attempts or command and control
1:47:17that is C2 activity. Next one is fusion
1:47:20rules. Advanced multi-stage attack
1:47:23detection. These correlate multiple low
1:47:26confidence signals to detect complex
1:47:30multi-step cyber attacks. This helps
1:47:32reduce false positives and identify
1:47:36advanced persistent threats that is APS.
1:47:39Next, machine learning behavior
1:47:41analytics. These leverage AIdriven
1:47:45behavioral analytics to spot evolving
1:47:48threads. These are useful for detecting
1:47:50insider threats, account takeovers or
1:47:53privilege abuse.
1:47:55Now let's see demos for these types of
1:47:58analytics roles. How to create them? How
1:48:00you can utilize them? Let's go to our
1:48:02Azure portal. Let's search for Sentinel.
1:48:07Click on Microsoft Sentinel and then the
1:48:10workspace. Sentinel workspace.
1:48:13Here go to configuration and then
1:48:15analytics.
1:48:18This gives you the overview of the rules
1:48:20that are available. There is one active
1:48:23rule. As you can see here, the active
1:48:26rule section in analytics tab refers to
1:48:29the currently enabled analytic rules
1:48:32that are actively running and generating
1:48:34security alerts based on some defined
1:48:37conditions.
1:48:39This rule says the severity is high.
1:48:42Name of the rule is advanced multi-stage
1:48:45attack detection.
1:48:47rule type is fusion. It is enabled and
1:48:51the tactics that it is using. If I click
1:48:54on it,
1:48:59you can find more details about this
1:49:02what this uh rule does. This rule covers
1:49:06the following detections. Fusion for
1:49:08emerging threats, ransomware,
1:49:10scenariobased fusion detections.
1:49:13To enable these detections, we recommend
1:49:15you to configure the following data
1:49:17connectors for best results. Azure
1:49:20defender, Microsoft defender for
1:49:22endpoint, cloud app security, defender
1:49:25for identity.
1:49:27So even if you don't manually create
1:49:30them right there, there will be some
1:49:32active rules because in the content
1:49:34management under content hub, we enabled
1:49:37some of the solutions which have these
1:49:39analytics rules in them. And the next
1:49:42tab is rule templates. These templates
1:49:46are pre-built detection rules that use
1:49:49KQL to analyze logs and identify
1:49:51suspicious activities across your
1:49:54systems. If I click on one of the alert
1:49:57rule templates, it's give it is giving
1:50:00you the description severity data
1:50:03sources
1:50:05uh Microsoft 365 threat intelligence
1:50:07platforms threat intelligence uh taxi
1:50:11data sources Microsoft 365 threat
1:50:14intelligence platforms tax defender TI
1:50:18and then MITER attack which one it is
1:50:20using it is fishing then you have the
1:50:22rule query
1:50:25And then rule frequency when it is run
1:50:28it this one is run every 1 hour. Rule
1:50:31period last 14 days. So it is running on
1:50:34the data that is in the last 14 days
1:50:38every 1 hour. Rule threshold event
1:50:42grouping. We will talk about all these
1:50:44in a minute. So let me close this. So
1:50:48let's try to create a scheduled rule.
1:50:52Right now you can create an analytics
1:50:55rule from a template that is use the
1:50:58query logic and the scheduling look back
1:51:00settings that is already provided in the
1:51:02template and you can customize it a
1:51:05little bit or you can create an
1:51:07analytics rule from scratch. You can
1:51:10build your own query and rule from
1:51:12scratch.
1:51:14Scheduled rules are the most common type
1:51:16of analytics rules in Microsoft
1:51:18Sentinel. They're based on KQL that run
1:51:21at regular intervals.
1:51:24These queries analyze raw data from a
1:51:27defined loop back period. They can
1:51:30perform complex statistical operations
1:51:32to identify baselines and outliers.
1:51:36Let's create a scheduled rule. So go to
1:51:39configuration analytics in that create
1:51:43scheduled query rule name. So we are
1:51:47creating a rule to identify any new uh
1:51:50users are added.
1:51:53So we'll say new user creation.
1:51:57This is a unique identifier for the rule
1:51:59and it is plain text only. And then
1:52:02provide the description. I'm going to
1:52:05say the same. Severity. This defines the
1:52:08potential impact that the rule triggers
1:52:10a true positive. There are four options.
1:52:14Informational is no impact but you could
1:52:17indicate future threat activity.
1:52:20Low is minimal immediate impact more
1:52:24steps needed for a real threat. Medium
1:52:27some impact but limited in scope or
1:52:30requires additional actions.
1:52:33High is significant impact allowing
1:52:35threat actors broad access. I will go
1:52:39with medium and then is miter attack.
1:52:42This is to select relevant tactics and
1:52:45techniques to categorize the threat
1:52:47activity.
1:52:48So I will select initial access and
1:52:52privilege escalation. And the status you
1:52:55can keep it disabled or enabled. Enabled
1:52:58rule runs immediately or at a scheduled
1:53:01time. Disabled rule is created but
1:53:03inactive until manually enabled. Once
1:53:07this is done, let's go set rule logic.
1:53:11Then there is rule query. Uh here you
1:53:14can paste the KQL query you created and
1:53:16tested. So I will paste mine. Any errors
1:53:21in the query will be validated
1:53:23instantly. Suppose say I'm going to
1:53:25remove this type it wrong spelling
1:53:27mistake. It says fail to resolve this
1:53:30expression named res. So it immediately
1:53:34tells you that something is wrong. So
1:53:36let me retype it which is a very useful
1:53:40uh feature I feel. So audit logs this is
1:53:44the table. So this KQL query retrieves
1:53:48and filters Azure ID that is enter ID
1:53:50audit logs to find events where a user
1:53:53was added. So it fetches data from the
1:53:57audit log table.
1:54:00Here let me zoom this a little bit. So
1:54:03here it fetches data from the audit logs
1:54:06table in Microsoft Sentinel or log
1:54:08analytics.
1:54:10So this table contains Azure AD audit
1:54:13logs which track administrative actions.
1:54:16It filters the logs to only include
1:54:19events where the active display name is
1:54:22equal to add user. This ensures that we
1:54:26only see user creation events.
1:54:30Then it gets the user name that is email
1:54:33of the newly added user from the target
1:54:36resources array. There is a different uh
1:54:40video on just KQL. If you want to learn
1:54:44more or if you don't know KQL, I
1:54:46recommend referring that. I will link
1:54:48that in the description box. And this
1:54:51zero target resources zero refers to the
1:54:55first object in the array. And this two
1:54:58string ensures the value is treated as a
1:55:01string.
1:55:03And then extend created by this extracts
1:55:07the username email of the person who
1:55:10performed the action. So this one is who
1:55:13got added created by is the person who
1:55:16performed the action.
1:55:18initiated by user user principle name
1:55:22holds the initiator's UPN email address
1:55:26and two string ensures it is treated as
1:55:29a string then we are selecting specific
1:55:32columns to display like the time
1:55:35generated timestamp of when the data was
1:55:38added activity display name of the event
1:55:42that is add user user principle name the
1:55:45newly added users UPN created by the
1:55:49admin or user who performed the action
1:55:52and the result indicates if the action
1:55:54was successful or failed. There is also
1:55:58view query results. Click this. It you
1:56:02can actually execute you can actually
1:56:04execute your query and see if it is
1:56:07fetching results. Run it does fetch
1:56:11results. Add user and the event is here.
1:56:15So let's go back. Then there is alert
1:56:18enhancement tab. The first one is entity
1:56:22mapping. This helps you define up to 10
1:56:25entity types recognized by Sentinel.
1:56:28This helps integrate identified entities
1:56:32into the fields into the entities field
1:56:34in alert schema. Entity mapping enables
1:56:38correlation across other security logs.
1:56:41enhances visibility in sentinel
1:56:43investigations and it also improves
1:56:45automated responses using playbooks. So
1:56:49let's click add new entity. Select the
1:56:52entity. I will select account and then
1:56:56we need to add identifier.
1:56:59Full name is user principle name. And
1:57:03then if I want I can add more
1:57:05identifiers. This is enough for me.
1:57:07After entity mapping, there is custom
1:57:09details.
1:57:11When creating or modifying scheduled
1:57:13query analytics rules, right, in
1:57:15Sentinel, you can enhance alerts by
1:57:17adding custom details. This allows
1:57:20security analysts to quickly view
1:57:22critical event information without
1:57:24needing additional investigation. So, it
1:57:27helps with faster triage and
1:57:30investigation. So, normally alerts only
1:57:33contain highle metadata. With custom
1:57:35details, you can extract and display
1:57:37specific events like for example
1:57:40usernames, IP addresses, device names
1:57:42directly in the alert properties. So if
1:57:46I click add new, I can select the value
1:57:49as user principle name and say created
1:57:54user.
1:57:56It has a key value pair and then there
1:57:59is alert details. So all alerts and
1:58:03incidents inherit the name, description,
1:58:06severity and miter attack tactics set in
1:58:09the rule. However, this does not account
1:58:12for dynamic details of each specific
1:58:14alert instance. So you can do that using
1:58:18alert details. You can override default
1:58:20alert properties. So you can you can do
1:58:24that using two ways. One is custom
1:58:26variable alert names and descriptions
1:58:29and the other one is dynamic severity
1:58:31tactics and other properties. So instead
1:58:34of saying new user created an entra id
1:58:38as the alert name you can say new user
1:58:40created and the user principle name who
1:58:44was the user that created and who
1:58:46created it. You can create your custom
1:58:48alert names like this. You can create
1:58:52custom alert name format as well as
1:58:54alert description format too. This is
1:58:57only an example. I'll remove this. And
1:59:00you can also use KQL fields to set
1:59:04priority dynamically. Like for example,
1:59:07extend severity is equal to case. User
1:59:10principle name contains admin, then it
1:59:13will be a high severity alert. user
1:59:15principle name contains test then it is
1:59:18a low pri priority or medium priority
1:59:21alert.
1:59:23So let me delete this as well. This was
1:59:26an example. So I will say alert name
1:59:29format new user created user principle
1:59:33name and the alert description format a
1:59:37new user so and so was created in entra
1:59:40ID at whatever time.
1:59:42Next is query scheduling. This run query
1:59:47every 5 hours. You can customize this.
1:59:50I'm going to leave it as is. Look up
1:59:52data from the last 5 hours. I'm going to
1:59:55leave this as well as is. You can set to
1:59:58start it automatically right now or at a
2:00:01specific time. It can start running from
2:00:04then. Then alert threshold. Generate
2:00:07alert when number of query results is
2:00:09greater than zero. Even if there is one
2:00:12alert, one event, it will generate an
2:00:15alert if you set it to zero. Then there
2:00:18is event grouping.
2:00:21It says configure how rule how rule
2:00:24query results are grouped into alerts.
2:00:26So all you I usually they select group
2:00:30all events into a single alert or you
2:00:33can also say trigger an alert for each
2:00:35event.
2:00:37So if you select group all events into a
2:00:39single alert that means all results from
2:00:41the query execution will be grouped into
2:00:43one single alert. This is best for batch
2:00:46processing where you want a single alert
2:00:49per run.
2:00:51And in the second option each alert each
2:00:54result generates a separate alert and
2:00:56separate incident. It is best if you if
2:01:00each event requires individual
2:01:02investigation. So it also gives you a
2:01:05warning saying a single analytics rule
2:01:07can generate up to 150 alerts. If you
2:01:10choose trigger an alert for each event
2:01:13and the rule query returns more than 150
2:01:16events, the rule will generate 150
2:01:19alerts.
2:01:20And the last alert will include a
2:01:22summary of all the events. I will go
2:01:25with group all events into a single
2:01:27alert.
2:01:29Next one is separation. You can turn
2:01:31this on or off. If you want to turn it
2:01:34on, the rule will stop running for a
2:01:37defined period after an alert is
2:01:39generated. Once the separation period
2:01:42expires, the rule will resume execution
2:01:45at the scheduled interval. This is
2:01:47useful for reducing noise from repeated
2:01:51uh detections of the same activity. I
2:01:54will have to set this to 5 hours or
2:01:56greater than 5 hours, not less than
2:01:58five. So if I set this to four, it says
2:02:01suppression must be equal to or greater
2:02:04than the frequency.
2:02:06So I will not turn this on for now. I'll
2:02:09switch it off. And then there is result
2:02:12simulation feature. This allows you to
2:02:14test and validate your analytics rule
2:02:17before fully deploying it. It helps
2:02:20ensure that your KQL query works as
2:02:23expected and generates meaningful
2:02:25results.
2:02:27Uh let me go to the incident settings
2:02:30now. So it is asking Microsoft Sentinel
2:02:33alerts can be grouped together into an
2:02:35incident. So I'm going to enable this
2:02:37saying create incidents from alerts
2:02:40triggered by this analytic rules. And
2:02:42then there is alert grouping. So if you
2:02:45want multiple related alerts to be
2:02:47grouped into a single incident, enable
2:02:50this group related alerts.
2:02:53So you see this it limit the group to
2:02:56alerts created within the selected time
2:02:58frame. You can select that as well. And
2:03:01then there is group alerts triggered by
2:03:03this analytics rule into a single
2:03:05incident. There are three options by
2:03:08grouping alerts into a single incident
2:03:10if all entities match. This is the
2:03:12recommended one. and grouping all alerts
2:03:15triggered by this rule into a single
2:03:17incident or grouping alerts into a
2:03:20single incident if the selected entity
2:03:22types and details match. You can select
2:03:25the entities and then group them. Also,
2:03:28there is also an option to reopen closed
2:03:31matching incidents. Okay. Next, we'll go
2:03:33to automated response. So, if I click
2:03:37add,
2:03:38so this will help you create an
2:03:40automation rule. uh we will cover this
2:03:42in the later chapters. So for now I will
2:03:45not create this. The next step is to
2:03:49review and create the alert. So this is
2:03:51how you create a schedule query rule.
2:03:55Now let's create another rule which is
2:03:58near realtime query rule. Select create
2:04:02and nrt query rule. We will use the same
2:04:06example new user creation
2:04:10but this is for near real time.
2:04:14I will paste the same description.
2:04:18Keep the severity medium
2:04:21my technique I'm going to say initial
2:04:24axis status enabled. Then go to set rule
2:04:28logic. We will use the same query that
2:04:32is audit logs
2:04:34where operation name is equal to
2:04:40add user and then we'll keep everything
2:04:44as is.
2:04:46Then go to incident settings. This also
2:04:48we will leave it as is. And then go to
2:04:51automated response. We will not do this.
2:04:54And then review and create. It is
2:04:57validating. and then say validation
2:05:00passed and then save it. It says
2:05:04analytics rule saved and then you can
2:05:07see that the alert is created. This is
2:05:10how you create NRT rule. So we saw
2:05:13scheduled rules near realtime rules. Now
2:05:16we'll see anomaly rules. Let's go back
2:05:19to the portal. So in the Azure portal
2:05:22analytics if I go to anomalies you can
2:05:26see what are the rules that are enabled.
2:05:29All of these are in enabled status.
2:05:32So if I right click on one of these
2:05:35alerts I can edit them, duplicate them
2:05:37or disable them. So I can say edit
2:05:41and I can edit them either disable them
2:05:43or enable them.
2:05:46You can use if you want to search for
2:05:48these in KQL that is logs you can search
2:05:52for anomalies. So if I go back to locks.
2:05:56So here under general if I go to
2:05:59analytics close this let me delete this.
2:06:03Search for anomalies.
2:06:07This is the table. And the next rule is
2:06:10Microsoft security rules.
2:06:13Let's go back to the portal. Click
2:06:15create. Microsoft incident creation
2:06:18rule. Rule name I'll say defender.
2:06:23Same description status enabled.
2:06:26Analytics rule logic. You just need to
2:06:29select the Microsoft security service
2:06:32that is Microsoft's defender for cloud
2:06:34apps. Let's actually make this defender
2:06:37for cloud apps. And then scroll down.
2:06:41Filter by severity any. So all the
2:06:45alerts from cloud apps will be forwarded
2:06:47to Sentinel.
2:06:50Include any specific alerts or exclude
2:06:52any specific alerts. You can do both.
2:06:55And then automated response. I don't
2:06:57want to create this here now. Then
2:06:59review and create.
2:07:02Save. So if I go to this,
2:07:08the rule is created.
2:07:10Let me go back to anomalies once. So if
2:07:13I want to search for this rule ID,
2:07:16right, all I have to do is copy this
2:07:18rule ID, go to general, go to locks,
2:07:23cancel this, look for anomalies,
2:07:28that is the table where
2:07:32rule ID contains
2:07:36this.
2:07:37And then you can run your uh query to
2:07:40see if there are any results. I don't
2:07:42have any but if there is anything you
2:07:44will get the results here as well. And
2:07:47then there are three more specialized
2:07:49analytics rules. Threat intelligence
2:07:51rules, fusion rules and ML rules. This
2:07:54you cannot customize a lot. Usually you
2:07:57will get the option to enable or disable
2:08:00them. You can do it from the rule
2:08:02template tab. So if I go to the portal
2:08:07and go back to configuration analytics
2:08:10in rule templates you can filter them
2:08:13using rule type say fusion ML and threat
2:08:19intelligence apply and then you can
2:08:22click on them and create the rule. So as
2:08:25you see this is a ML behavior analytics
2:08:29rule. You don't have a lot of options to
2:08:32edit them other than you know you can
2:08:34enable or disable them. Set up automated
2:08:38response. So if I go to automated
2:08:40response, you can set this up and create
2:08:42them. Let me go back. And it's the same
2:08:46with threat intelligence rule as well.
2:08:49You can enable or disable them and
2:08:51select what MITER attack strategy you
2:08:54want to map this to.
2:08:57Let me go back. Fusion rule is already
2:09:00in use here that is it is an active rule
2:09:05which you can find it here. So not a lot
2:09:08of customization in the last three that
2:09:10is the specialized analytics rules that
2:09:13you can do. So that is all about
2:09:15analytics rules guys. In this next
2:09:18chapter we will discuss UIBA user and
2:09:22entity behavior analytics. What is UEBA?
2:09:25How it can be used in Microsoft
2:09:27Sentinel? What are the benefits of UEIBA
2:09:30and how to configure UEIBAB in Sentinel?
2:09:35First, let's learn what is UEIBAB, user
2:09:38and entity behavior analytics and how
2:09:41UIBBA works in Microsoft Sentinel.
2:09:44UBABBA in Microsoft Sentinel is a
2:09:47security feature.
2:09:49It detects suspicious activities by
2:09:51analyzing user and entity behaviors over
2:09:54time. Instead of relying solely on
2:09:58static rules or signatures, UEIBA uses
2:10:02machine learning and anomaly detection
2:10:04to identify potential threats.
2:10:08Now let's see how actually UEIBA works.
2:10:11First one is builds baselines that is
2:10:14learns the normal behavior of users,
2:10:17hosts, IP addresses and applications
2:10:20over time.
2:10:22Suppose say there is a user name called
2:10:25John. He is from the finance department
2:10:28and typically logs in from New York
2:10:31between 900 a.m. to 5:00 p.m. And he
2:10:35does that only on weekdays. So now
2:10:38Sentinel recognizes this as John's
2:10:42normal behavior and creates a pattern
2:10:45that is building baselines. It does this
2:10:48for all the users, hosts, IP addresses,
2:10:52domains and so on. Next detecting
2:10:55anomalies that is it identifies
2:10:58deviations from this normal behavior
2:11:01using machine learning techniques.
2:11:04Suppose say now John suddenly logs in at
2:11:062:00 a.m. from Russia and then downloads
2:11:10a large number of sensitive files. This
2:11:13will be detected by UEIBBM.
2:11:16It can also detect deviations at a
2:11:18broader level. For example, John is a
2:11:21finance user, right? Suddenly if he
2:11:23starts performing IT admin tasks, that
2:11:26will also be detected. So this deviates
2:11:30from usual activity triggering an
2:11:32anomaly alert.
2:11:35Next correlating events that is it
2:11:38connects unusual activities across
2:11:41multiple data sources for better thread
2:11:44detection. Say John attempts multiple
2:11:47failed login and all of these login are
2:11:50from a new location and after successful
2:11:54login his account creates new
2:11:56administrator accounts in Azure active
2:11:59directory or enter ID and a new device
2:12:02is enrolled under his credentials. So
2:12:05individually these actions may not seem
2:12:08suspicious but correlating them reveals
2:12:11a potential account takeover attack.
2:12:14Next is assessing sensitivity and
2:12:17impact. This evaluates the relative
2:12:20importance of an asset, its peer group
2:12:23and its blast radius.
2:12:26What is blast radius? Blast radius
2:12:29refers to the potential impact of a
2:12:31compromised asset in a security
2:12:34incident. So it helps security analysts
2:12:37understand how far an attack can spread,
2:12:40what other assets could be affected and
2:12:42how critical the compromised asset is
2:12:45within the organization. Let's see some
2:12:47of the key aspects of blast radius
2:12:50analysis. First one is asset
2:12:53sensitivity. It determines the
2:12:55criticality of an asset based on its
2:12:57function, access level and importance.
2:13:00Peer grouping identifies similar assets
2:13:04to assess abnormal behavior or common
2:13:07attack targets.
2:13:09Lateral movement analysis evaluates how
2:13:12an attacker could move from one asset to
2:13:15another. Potential impact assessment
2:13:18that is helps prioritize response
2:13:20actions based on how much damage a
2:13:23compromise could cause. Say for example
2:13:26there are two users. Alice is an HR
2:13:30staff and Bob is a cloud administrator.
2:13:34Now Alice logs in from a different city
2:13:36than usual
2:13:38and Bob logs in from an untrusted device
2:13:41and attempts to change firewall rules.
2:13:44Now what UEBA does it prioritizes the
2:13:49risk. Bob's role has higher privileges.
2:13:54So the blast radius is larger if
2:13:57compromised.
2:13:58So Sentinel prioritizes Bob's alert as a
2:14:02higher risk than Alys's. This is how
2:14:05assessing sensitivity and impact works
2:14:08works in UIBM.
2:14:10This is a picture from Microsoft
2:14:12documentation. which represents how
2:14:14UEIBA process is in Microsoft Sentinel
2:14:18and how it gradually reduces the volume
2:14:21of raw events by applying intelligence
2:14:24and analytics.
2:14:26So first here is the ingestion that is
2:14:29all security related logs and events are
2:14:32ingested from various sources like
2:14:34endpoints, networks and cloud services.
2:14:37This stage collects all the raw uh data
2:14:41without filtering. So it is 100% of raw
2:14:44events.
2:14:46Next security researchbased filtering is
2:14:49elim eliminating irrelevant data and
2:14:52only 30% of raw events are left. That is
2:14:56only events that have potential security
2:14:58significance are retained.
2:15:02Then events are enhanced with additional
2:15:04context such as user behavior,
2:15:07geoloccation, device attributes and
2:15:10historical patterns.
2:15:12This helps in identifying abnormal
2:15:15behavior. Then the security analytics
2:15:18build profiles for users, devices and
2:15:21entities. This establishes a baseline of
2:15:25normal behavior to detect uh deviations.
2:15:29Then events are mapped to the MITER
2:15:31attack framework to identify tactics,
2:15:34techniques and procedures that is TTPs.
2:15:37This helps in detecting sophisticated
2:15:40attacks by correlating multiple signals.
2:15:44And by this time only 10% of the raw
2:15:46events remain. And then the final step
2:15:50security alerts are triggered only on
2:15:53anomalies and confirmed threats. That is
2:15:57less than 1% of raw events. This reduces
2:16:02false positives ensuring security
2:16:04analysts focus on the most critical
2:16:07threats. Now let's try to understand why
2:16:10UEBA matters. First is that it reduces
2:16:13false positives. So traditional rules
2:16:16may flag all failed login as threats but
2:16:20UEIBA understands context like for
2:16:22example a traveling employee. So,
2:16:25Microsoft Sentinel provides security
2:16:28analysts with artifacts and insights to
2:16:31understand anomalous activities in
2:16:33context. Actions performed by a user,
2:16:36host or IP address are evaluated
2:16:40contextually where a true anomaly is
2:16:43identified based on a few key factors.
2:16:47First one, geographical locations,
2:16:49devices and environments. That is detect
2:16:52suspicious login from unexpected places,
2:16:55devices or environments.
2:16:58And then time and frequency, that is it
2:17:01compares current actions to the user's
2:17:04past behavior. For example, logging in
2:17:06at unusual hours.
2:17:10Then there is peer behavior comparisons.
2:17:12that is it identifies anomalies by
2:17:15comparing actions to others in the same
2:17:18department or role.
2:17:20And then organizationwide behavior that
2:17:23is it detects deviations at a broader
2:17:25level like for example as we discussed a
2:17:28finance user suddenly performing IT
2:17:31admin tasks like you can see in this
2:17:34small picture there is the context user
2:17:38then peer then organization all of this
2:17:41will be compared. So it reduces false
2:17:44positives and then it detects advanced
2:17:47threats that is identifies insider
2:17:50threats, compromised accounts and
2:17:52lateral movement by attackers
2:17:54and then it also enhances response
2:17:57prioritization that is help security
2:17:59teams focused on critical threats first
2:18:02and this reduces alert fatigue. So all
2:18:06in all, UEIBA goes beyond rule-based
2:18:09detection by learning behaviors,
2:18:12detecting anomalies and correlating
2:18:14security events.
2:18:16So this explains the UEIBA architecture
2:18:19in Microsoft Sentinel. So it is showing
2:18:23how data flows from customer sources
2:18:26that is for example SAS applications and
2:18:29on premises environments to Sentinel for
2:18:32analysis.
2:18:34So first the customer data sources SAS
2:18:37applications for example say Microsoft
2:18:39365 or third party cloud apps and then
2:18:42there is onremise infrastructure as well
2:18:44like servers, network devices, security
2:18:47appliance. So from this raw data is
2:18:50ingested like for logs, events and
2:18:52alerts are ingested from this customer
2:18:55data to Microsoft Sentinel and then
2:18:59Microsoft Sentinel onboards data sources
2:19:02to analyze user and entity behaviors. It
2:19:06performs behavior analytics, identity
2:19:08analytics, user peer analytics and
2:19:11access analytics.
2:19:13And it performs behavior analytics,
2:19:16identity analytics, user be peer
2:19:20analytics, user access analytics.
2:19:24And then it detects anomalies and
2:19:26security threads. There is also active
2:19:30directory that is entra ID integration
2:19:33with sentinel.
2:19:35So this provides user and group
2:19:37information to enhance identity based
2:19:41thread detection. It helps detect
2:19:43privileged account misuse, unauthorized
2:19:45access attempts and lateral movements.
2:19:49This entire integration allows proactive
2:19:52thread detection by identifying
2:19:54deviations from normal behavior.
2:19:58Now let's see how to configure UBA in
2:20:00Microsoft Sentinel. Let's go to the
2:20:03portal.
2:20:05Let's search for Sentinel.
2:20:07Select Microsoft Sentinel. Select the
2:20:10workspace
2:20:11here. Go to threat management.
2:20:15Under threat management, there is entity
2:20:17behavior. And then you can click on set
2:20:21UIBA. That is enable UEBA to benefit
2:20:24from a much richer set of entities,
2:20:27profiles, insights into the behavior of
2:20:30your users, hosts, and more. So you can
2:20:32click set UIBA or if you don't get this
2:20:36popup close this and go to entity
2:20:39behavior settings
2:20:41and then here it says what is it that is
2:20:44what is UIBBA and then how to enable it.
2:20:48Let's click set UIBA. Here it says turn
2:20:51on the UIBA feature. You must complete
2:20:54step two for UIBBA functionalities to
2:20:57start. So this is step one. It's it also
2:21:00gives you a prompt saying only a global
2:21:02administrator or a security
2:21:04administrator in Microsoft Entra ID can
2:21:08turn this feature on or off. This is the
2:21:10permissions that you need. So I'm going
2:21:12to turn it on.
2:21:14And then it says sync Microsoft Sentinel
2:21:18with at least one of the following
2:21:20directories.
2:21:21This will create profiles for the users
2:21:23and entities in your organizations and
2:21:26also create data stores in Microsoft
2:21:29Sentinel. So I will select Microsoft
2:21:32Enra ID because we don't have active
2:21:35directory that is the onremise one and
2:21:39then I'll hit apply. It's validating. It
2:21:42says entities synced successfully and
2:21:45then it says select the existing data
2:21:48sources you want to enable for entity
2:21:50behavior analytics. I will say audit
2:21:53logs
2:21:54and then sign in logs as well and then
2:21:57say apply. It says data sources update
2:22:00was successful.
2:22:02And then we are back to the main page
2:22:04here. If I scroll down anomalies is
2:22:07enabled.
2:22:09You need to enable this if it is not
2:22:11already enabled.
2:22:13These anomalies are stored in the
2:22:15anomalies table within your Sentinel
2:22:18workspace. So anomalies table stores
2:22:21detected anomalies for use in detection
2:22:23rules, hunting queries and
2:22:25investigation. This helps security
2:22:27analysts track unusual behaviors over
2:22:30time. It also says that you can tune
2:22:33parameters of machine learning models to
2:22:36improve detection accuracy.
2:22:39Something to note is that it uses 30
2:22:42days of active data for training. So for
2:22:44UEIBA you need to give some time. It
2:22:47says 30 days for it to start
2:22:49understanding or mapping the normal
2:22:52behavior and understanding how the
2:22:54entities work and also data is cached
2:22:58and encrypted with Microsoft managed
2:23:00keys. So this needs to be enabled
2:23:02anomalies.
2:23:04Once this is done, let's go back to the
2:23:06Sentinel homepage.
2:23:08and then scroll down to configuration.
2:23:11In that select analytics,
2:23:14we briefly touched on this in previous
2:23:16chapters, but go to anomalies
2:23:19and then select data sources Microsoft
2:23:23enter ID and then apply. So only the
2:23:27data sources with Microsoft Entra ID are
2:23:30showing up here the anomalies the uh
2:23:32rules
2:23:34and if you see it says UIBA anomalous
2:23:36account deletion password reset sign in
2:23:39account manipulation failed sign in
2:23:42account create creation privilege
2:23:44granted so these are the rules let's
2:23:47select one of these so it gives you the
2:23:50rule ID it is enabled and a description
2:23:53of what it does rule frequency and rule
2:23:57status that is rule is enabled. So let
2:24:00me close this. So if you see all the
2:24:02anomalies rules are enabled. If they
2:24:05were not enabled by default, you can
2:24:07enable them. So if I can select this. So
2:24:10there's a button to disable it as well.
2:24:13And then if I go to general and logs, we
2:24:16can also query the behavior analytics
2:24:19data. Let me minimize this. Close this.
2:24:25The table is called behavior analytics.
2:24:28This is the table that stores user and
2:24:31entity behavior analytics derived from
2:24:33machine learning models in Microsoft
2:24:35Sentinel. So it contains insights into
2:24:38login attempts, access patterns and
2:24:40anomalies based on historical data. And
2:24:43then we will say where activity type
2:24:48contains
2:24:50failed log on that is it filters the
2:24:54data set to include only failed login
2:24:57attempts. So it helps us to focus on
2:24:59unsuccessful access attempts which might
2:25:02indicate a brute force attack or
2:25:04compromised credentials.
2:25:06And then we will say where
2:25:10this is another filter activity insights
2:25:16dot first time user connected from
2:25:20country is equal to equal to true.
2:25:25That means it checks if the user is
2:25:27logging in from a country for the first
2:25:29time. So a first login first time login
2:25:33from a new country may indicate an
2:25:35unauthorized access attempt or a
2:25:37compromised account. So that is what we
2:25:39are trying to find. Next
2:25:42we'll apply another filter where
2:25:45activity insights
2:25:47country uncommonly connected from among
2:25:50peers is equal to equal to
2:25:54true. So this checks if the login
2:25:57attempt comes from a country that is
2:25:59uncommon among the user's peer group.
2:26:02Suppose say if most employees from an
2:26:05organization login from the US and UK.
2:26:08But if this user logs in from Russia or
2:26:11China, it raises a security flag.
2:26:15This can indicate geoloccation based
2:26:17anomalies and potential account takeover
2:26:19attempts.
2:26:21So you see when I try to run this there
2:26:23is a problem.
2:26:25And even if I don't run it, KQL tells me
2:26:28what needs to be done and where there is
2:26:31an error. It also gives you a
2:26:34recommendation. Avoid using the contains
2:26:36operator as it has a high compute price.
2:26:40So I'm going to just say type is equal
2:26:44to equal to and then run it. I don't
2:26:49have any results. But if there is any
2:26:52anomalous behavior in your organization,
2:26:55this should fetch results.
2:26:58So there is also another table called
2:27:01user peer analytics.
2:27:04This stores top 1 to 20 ranked peers. So
2:27:09Microsoft what Microsoft does it
2:27:11calculates and ranks peers based on
2:27:13factors like Microsoft entra security
2:27:16group membership mailing list members
2:27:19and departmental roles. Based on this it
2:27:22creates a list. In that top 1 to 20
2:27:26ranked peers are stored in this peer
2:27:28analytics table.
2:27:31That's it about this topic. Let's move
2:27:33on to the next chapter. Now in this
2:27:36chapter we will discuss how to ingest
2:27:38Windows event logs using AMA that is
2:27:42Azure monitor agent. Before we learn how
2:27:45to ingest logs using AMA, let's see what
2:27:48it is. Azure monitor agent is the
2:27:52primary telemetry collection agent in
2:27:55Azure monitor. It is designed to collect
2:27:58and send logs and metrics from various
2:28:00resources to Azure monitor log analytics
2:28:04workspaces event hubs or Azure storage.
2:28:08It replaces the log analytics agent the
2:28:11legacy one MMA agent providing better
2:28:15performance security and flexibility
2:28:18with data collection rules DCRs. This is
2:28:22something important that you need to
2:28:23remember that AMA Azure monitor agent
2:28:26uses data collection rules. So DCRS
2:28:31define what to collect like specify log
2:28:34types like event logs, SIS log or
2:28:37performance counters. They can target
2:28:40specific resources like apply to
2:28:42specific VMs, resource groups or
2:28:44subscriptions. They transform data
2:28:47before sending that is filter, aggregate
2:28:50or modify log data and they can send
2:28:53data to multiple destinations for
2:28:56example log analytics, event hubs or
2:28:58Azure storage. Okay, going back to Azure
2:29:02monitor agent. Let's see what resources
2:29:05can AMA collect data from. First one is
2:29:08Azure virtual machines VMs. It can
2:29:11collect logs, metrics, custom logs.
2:29:14Azure VM scale sets on premises or
2:29:18multicloud VMs. For this you need to
2:29:22have Azure Arc enabled servers and Azure
2:29:26Kubernetes service AKS also Azure
2:29:30functions and app services. Now let's
2:29:33see what data can AMA collect. Windows
2:29:37event logs for this we'll see the demo.
2:29:39SIS log that is Linux logs performance
2:29:43metrics. Azure monitor logs and
2:29:46container logs AKS. So this picture is
2:29:50from Microsoft documentation. The shows
2:29:53how Azure monitor processes incoming
2:29:55data from clients through a cloud
2:29:58pipeline before storing it in designated
2:30:01destinations. To break it down, the data
2:30:04is ingested from different sources using
2:30:07direct ingest or an agent. These clients
2:30:11send raw data to Azure monitor for
2:30:14processing and then data flows into a
2:30:18transformation process within the
2:30:20pipeline. So cloud pipeline is one of
2:30:22the core components of Azure monitor
2:30:25pipeline. It has two components. Cloud
2:30:27pipeline and edge pipeline. We will talk
2:30:30about cloud pipeline. Now cloud pipeline
2:30:32provides a standardized way to process
2:30:35and route incoming data within Azure
2:30:38monitor. And then a DCR is applied to
2:30:41specify what data is collected, define
2:30:44the schema of the incoming stream, apply
2:30:47transformation such as filtering or
2:30:49manipulation and then determine the
2:30:51destination for for the process data.
2:30:54Once the data is transformed by DCR, it
2:30:59is sent either to log analytics
2:31:01workspace that is used for querying and
2:31:04analyzing logs or it is sent to Azure
2:31:07monitor workspace. This stores and
2:31:10manages monitoring data. Now let's see a
2:31:13demo of how to ingest Windows event logs
2:31:16from a VM using AMA to Sentinel that is
2:31:21log analytics workspace. Let's go to the
2:31:24portal. First, we need to create a
2:31:26virtual machine. Let's do that. Search
2:31:30for virtual machines. Select virtual
2:31:33machines. Then click create. Azure
2:31:36virtual machine. Create a virtual
2:31:38machine hosted by Azure. Click that.
2:31:41Then it asks you for details.
2:31:44Subscription. The same subscription. I
2:31:47will select the resource group as the
2:31:49same one that is existing. virtual
2:31:51machine name. I'll say Windows logs
2:31:54demo. I'll keep the region same.
2:31:57Availability zone. I'm going to go with
2:31:59all the default options. It's asking for
2:32:02zone options. Self- selected zone. Zone
2:32:05uh availability zone. Zone one. Uh I'm
2:32:08going to keep everything the same. I
2:32:09have to select the image. I will select
2:32:12Windows Server 2019 data center. Select
2:32:15the size and then enter admin user ID
2:32:20and password whatever you want to set it
2:32:22as. I'll use the username Windows log.
2:32:26I've entered the password and then I
2:32:28will keep all the default settings. Go
2:32:30to review and create. Okay, there is
2:32:34something that I have missed to select
2:32:35in networking. Let me see. So if you see
2:32:38by default a virtual network and subnet
2:32:41will be created new ones. It says select
2:32:44a load balancer. Load balancing options.
2:32:47I've selected Azure load balancer by
2:32:50default. So I'm going to go with none.
2:32:52And then review and create. Validation
2:32:55passed and then I will say create. Now
2:32:58it is creating the virtual machine.
2:33:01Initializing deployment.
2:33:04Deployment is in progress and you can
2:33:07also see the deployment details here.
2:33:10Okay, the deployment is complete. Let's
2:33:12go to resource.
2:33:15Okay, now VM is ready. Let's go back to
2:33:18Sentinel. Select Microsoft Sentinel and
2:33:20then the workspace.
2:33:23Here let's go to content management
2:33:25content hub and then search for Windows
2:33:29event logs. There is this solution
2:33:32called Windows security events. Let's
2:33:34select that. Let me minimize this.
2:33:38So it says the Windows security event
2:33:41solution for Microsoft Sentinel allows
2:33:44you to ingest security events from your
2:33:47Windows machines using the Windows agent
2:33:50into Microsoft Sentinel. This solution
2:33:53includes two data connectors to help
2:33:56ingest the logs.
2:33:58First one is Windows security events
2:34:00using AM. This is what we need. And the
2:34:04other one is security events using a
2:34:07legacy agent. The legacy agent we
2:34:09already mentioned that it is log
2:34:11analytics agent. It is getting replaced
2:34:14by AMA now. So we don't need this. The
2:34:17first data connector is what we are
2:34:19looking for. So it has two data
2:34:22connectors, two workbooks, 20 analytic
2:34:25rules and 50 u hunting queries. So I'm
2:34:29going to hit install.
2:34:32It's installing like you can see it is
2:34:35installed. Now let me minimize this and
2:34:39here let me search for the data
2:34:41connector. So if you see the content
2:34:44type you can select the data connector
2:34:46here. So this one says windows security
2:34:49events that is the AMA one that we are
2:34:52looking for and then say open connector
2:34:55page.
2:34:57To integrate with Windows security
2:34:59events using AMA, you have to have read
2:35:02and write permissions. Like I mentioned
2:35:05before, for non- Aure VMs, to collect
2:35:08data from non- Aure VMs, they must have
2:35:11Azure Arc installed and enabled.
2:35:14Once you meet all the prerequisites, the
2:35:17next part is configuration. Enable data
2:35:20collection rule. Create data collection
2:35:23rule.
2:35:25Rule name Windows log demo. And then the
2:35:29subscription uh resource group. Click
2:35:32next. Resources.
2:35:34Here you can select the resources.
2:35:37So once you select the machines here it
2:35:40says Azure monitor agent will
2:35:42automatically be installed on those
2:35:45resources. So let me expand the
2:35:48subscription then resource group. We
2:35:51have our resource group here. I'm going
2:35:54to select everything under the resource
2:35:56group or you can select individual
2:35:59machines as well and then hit collect.
2:36:02You can either collect all security
2:36:04events or the common ones minimal or
2:36:08custom. So if you hover over this, it
2:36:10says all events, all Windows security
2:36:14and app locker events. Common means a
2:36:16standard set of events for auditing
2:36:18purposes. Minimal, a small set of events
2:36:22that might indicate potential threats.
2:36:24By enabling this option, you won't be
2:36:26able to have a full audit trail. Then
2:36:28you also have custom. This allows you to
2:36:31filter and select the security events to
2:36:34stream by using XPath queries. Okay. So
2:36:37if I say custom here you can choose to
2:36:42stream specific custom events instead of
2:36:44you know selecting all security events
2:36:47common ones or minimal ones. So I can
2:36:50add something like this for RDP and
2:36:53select add for now I I will select all
2:36:56security events and then create and
2:36:59review. Validation is passed. I'm going
2:37:02to say create.
2:37:05It says successfully installed uh
2:37:08extension create data collection rule
2:37:11association succeeded and then
2:37:12installation of the agent is also
2:37:14successful. Let me close these
2:37:16notifications.
2:37:18Let me close this as well. And that is
2:37:20how you ingest Windows security events
2:37:23using AMA Azure monitor agent to
2:37:27Sentinel to look at the logs. I think
2:37:30we'll have to wait for some time and
2:37:32after that you will see an increase in
2:37:35this graph. You can also go to locks and
2:37:38search for security events to see if the
2:37:41locks are getting ingested. It's been
2:37:44like 5 minutes and I can already see
2:37:46that the security event locks are
2:37:49getting accumulated here. This is one
2:37:52way to see it. Also, you can go to
2:37:54configuration data connectors. This
2:37:56might be easier to find for it. and then
2:37:59scroll down to windows security events
2:38:02and then here as well there is an
2:38:05increase in the graph. So there are logs
2:38:08getting ingested. Another way to check
2:38:11is go to logs under general close this
2:38:15and then query for the table security
2:38:18events run
2:38:21and then here as well you can see the
2:38:24security events. You can expand them to
2:38:29see what exactly has happened. That's it
2:38:32about this topic. Let's move on to the
2:38:34next chapter. Now, in this chapter,
2:38:37we'll discuss how to automate threat
2:38:39responses in Sentinel. So, Microsoft
2:38:42Sentinel is not only a SIM system but
2:38:44also serves as a platform for sore that
2:38:47is security orchestration, automation
2:38:49and response. One of its main function
2:38:52is to streamline and automate security
2:38:55operations.
2:38:56First let's see why do we need to
2:38:59automate responses responses to threats.
2:39:02First one is it reduces alert fatigue.
2:39:04So automation helps to prioritize and
2:39:07filter alerts right. So this reduces the
2:39:10noise and false positives that overwhelm
2:39:12analysts. So by automating the triage
2:39:15process only critical threats are
2:39:17escalated.
2:39:19Next, it improves incident response
2:39:21time. Automating response actions like
2:39:24blocking IP addresses or isolating
2:39:26affected systems. Speeds up containment
2:39:29and remediation. This allows
2:39:32organizations to respond more swiftly,
2:39:35limiting the damage caused by incidents
2:39:37and reducing recovery time. So, next is
2:39:40to free up analysts for threat hunting
2:39:42and investigations. So when you automate
2:39:45routine tasks, right, analysts can focus
2:39:48on more strategic activities like
2:39:51investigating sophisticated threats or
2:39:53proactively hunting for potential
2:39:56vulnerabilities.
2:39:57This enhances the effectiveness of the
2:39:59security team and improves overall
2:40:02threat detection capabilities.
2:40:05Now let's see what are the options to
2:40:07automate responses in Sentinel. The
2:40:10first one is using automation rules. The
2:40:13second one is playbooks Azure logic
2:40:15apps. This is the actual source
2:40:17solution. With automation rules, you can
2:40:20manage incident handling centrally. For
2:40:23example, using Microsoft Sentinel's
2:40:26centralized dashboard, all alerts
2:40:29related to a potential fishing attack
2:40:31are automatically collected,
2:40:33categorized, and prioritized. This
2:40:36ensures that analysts handle them
2:40:38efficiently and in a consistent manner.
2:40:41Then you can use it to assign playbooks
2:40:43to incidents and alerts. Suppose say if
2:40:46a ransomware alert is triggered, an
2:40:48automated playbook is assigned to the
2:40:51incident that includes steps for
2:40:54isolating affected systems, notifying
2:40:56the team and blocking malicious IP
2:40:58addresses. So this ensures a quick
2:41:01response. You can also use automation
2:41:04rules to automatically tag, assign or
2:41:07close incidents.
2:41:09For example, say that a low priority
2:41:11alert for a network scan that is done by
2:41:14an internal team is created. So when
2:41:17this is done, the system can
2:41:19automatically tag it as low priority,
2:41:22assign it to a specific analyst or close
2:41:26it if no further action is needed. This
2:41:29reduces manual tasks. Then you can use
2:41:32it to automate responses for multiple
2:41:34analytics rules. For example, if
2:41:37multiple suspicious login attempts are
2:41:40detected across different systems, an
2:41:42automation rule can modify the incident
2:41:45severity or status, assign the incident
2:41:48to a specific team, add relevant tags to
2:41:51it for easier categorization, send
2:41:53notifications to the security team. This
2:41:56allows multiple rules to be addressed
2:41:58simultaneously without manual
2:42:00intervention. Next, you can use
2:42:03automation rules to create structured
2:42:05workflows for security analysts. Say a
2:42:08malware detection alert is triggered.
2:42:11Just like how we discussed for
2:42:13suspicious login attempts, the same
2:42:15actions can be taken for this also that
2:42:17you know first it changes the incident
2:42:19classification based on some predefined
2:42:22criteria. Then it is assigning to a
2:42:25specific security analyst who is an
2:42:28expert in malware detection and
2:42:30response. And then it is adding comments
2:42:32with investigation steps. And then it is
2:42:36tagging similar incidents for
2:42:38correlation. So these actions help
2:42:41ensure a streamlined and consistent
2:42:43incident management process. Next is the
2:42:46playbooks option. So automation rules
2:42:49also help with automation, right? But
2:42:51they are mostly for uh simpler tasks.
2:42:54But playbooks you can use it for more
2:42:57complex items. And this is the actual
2:43:00source solution that is provided by um
2:43:03Sentinel. First item it can do is to
2:43:06automate and orchestrate threat
2:43:08response. For instance, a playbook
2:43:10triggered by a DDoS attack automatically
2:43:13triggers traffic filtering at the
2:43:15firewall, updates the monitoring
2:43:18dashboard and sends notifications to the
2:43:21response team. And all this is done
2:43:23without any manual intervention.
2:43:26Next, integrate with external and
2:43:28internal systems. For example, a
2:43:31playbook designed for an advanced
2:43:33persistent threat that is AP could
2:43:36integrate with email security, firewall,
2:43:39and endpoint detection systems. And it
2:43:42can automatically block the attacker's
2:43:44IP, quarantine infected devices, and
2:43:47block fishing emails.
2:43:50Next, you can run playbooks
2:43:52automatically or manually on new alerts.
2:43:56For example, for a suspected data
2:43:58exfiltration attempt, an automated
2:44:02playbook could immediately trigger steps
2:44:04like isolating the affected server and
2:44:07notifying the admin. But a security
2:44:10analyst can also manually run the
2:44:13playbook to execute specific actions
2:44:16based on further investigation.
2:44:19Next, playbooks are built on Azure logic
2:44:22apps for scalability and reliability.
2:44:26So, a playbook built on Azure logic apps
2:44:29can automatically respond to an
2:44:31increased number of alerts during a
2:44:33large scale attack such as say a
2:44:36coordinated botnet attack. So, this is
2:44:39done without compromising performance
2:44:41and ensuring the system remains
2:44:43responsive even under heavy load. Now
2:44:47let's go to the demo part. We will start
2:44:49with first creating an automation role.
2:44:52Let's go to the Azure portal. Let's
2:44:55search for Sentinel. Select Microsoft
2:44:58Sentinel and then the Sentinel
2:45:00workspace.
2:45:02Here scroll down to configuration and
2:45:05then go to automation here and then
2:45:08click create and then there is
2:45:11automation rule. So create automation
2:45:13rule. The first thing that you need to
2:45:16do when designing an automation rule is
2:45:19to identify the specific incidents or
2:45:22alerts it should apply to. Once you're
2:45:25done with this, then next is you have to
2:45:27decide what you want to achieve with
2:45:29this automation. For example, assign
2:45:32task to security analysts for triaging,
2:45:35investigating, and remediating
2:45:37incidents,
2:45:39suppressing noisy incidents, managing
2:45:42incident workflows by updating the
2:45:44status from new to active, and assigning
2:45:47a owner, or do you want to tag incidents
2:45:50to improve classification and tracking?
2:45:53Do you want to escalate incidents by
2:45:55reassigning ownership? You can also
2:45:59close uh resolved incidents and also
2:46:02specify a reason and add comments to
2:46:04this. You can analyze incident details
2:46:07like alerts, entities, properties and
2:46:10take further action using playbooks. You
2:46:14can also handle or respond to alerts
2:46:16that do not have an associated incident.
2:46:19So first thing is that you identify what
2:46:22kind of specific incidents or alerts
2:46:25this automation rule should apply to and
2:46:27then you can decide what you want to
2:46:30achieve with this automation rule. Let's
2:46:33go back to the portal. First is the
2:46:36automation rule. I will say this is for
2:46:39so in the previous demos we created a
2:46:42rule in analytics saying that if there
2:46:45is any new ID created in enter ID new
2:46:49user created in enter ID trigger an
2:46:51alert for the same thing we will create
2:46:54an automation rule say new user
2:46:57automation
2:46:59first one is trigger trigger is when
2:47:02this automation rules should run so
2:47:05there are three option when an incident
2:47:07is created. So a new incident created
2:47:11within the defender portal or Microsoft
2:47:13Sentinel and then when an incident is
2:47:16updated. So you can run an automation
2:47:19rule when incident status changes like
2:47:22closed, reopened or triaged. Incident
2:47:25ownership is assigned or updated.
2:47:27Incident severity is modified. new
2:47:31alerts are added to an existing incident
2:47:34or comments tags or tactics are added to
2:47:37a new incident and then when an alert is
2:47:41created this is generally done by using
2:47:44Microsoft Sentinel scheduled or near
2:47:46realtime NRT analytics roles for this we
2:47:50will select when incident is created
2:47:53next one is conditions so you can also
2:47:56specify conditions to determine when a
2:47:58rule should execute
2:48:00These uh conditions are case sensitive
2:48:03and they also change based on the
2:48:06trigger. So if I select something else
2:48:08in the trigger when incident is updated
2:48:11this also changes when alert is created
2:48:15again see you can see the change. So
2:48:18let's go back to an incident is created.
2:48:21First one is if incident provider equals
2:48:25we will select Microsoft Sentinel. There
2:48:27are two options defender XDR and
2:48:29Microsoft Sentinel. We'll go with
2:48:31Microsoft Sentinel. Then analytic rule
2:48:34name contains and does not contain. Even
2:48:37here incident provider it says equals
2:48:39and does not equal to. So I will say
2:48:42contains and in this I will say new user
2:48:46creation. So it's an and operator here
2:48:50if you can see it. If incident provider
2:48:53is Microsoft Sentinel and analytic rule
2:48:57name is equal to new user creation that
2:49:01means both of these conditions should
2:49:03meet for this automation rule to get
2:49:06triggered. You can add more conditions
2:49:08by clicking add. It says condition and
2:49:11and condition group or. So for and all
2:49:15conditions must be met for this
2:49:17automation rule to trigger and for or at
2:49:20least one condition must be met for this
2:49:22rule to trigger. So if I click any of
2:49:25this and click the arrow, you can see
2:49:28that incident properties using title,
2:49:31description, severity, status, tactics,
2:49:33tag and custom details key. You can
2:49:36trigger this automation role and you can
2:49:39also use the entity properties in the
2:49:41incident or alert for triggering the
2:49:44this one. So you can use what file name
2:49:47can be there and you know for false
2:49:49positives if this file name you know
2:49:52repeats in the entity you can close it
2:49:54some it's just an example or host name
2:49:58IP address you can select you get a lot
2:50:01of options to select from here so for
2:50:04now I'll delete this next is actions in
2:50:07actions you can run a playbook change
2:50:10status change severity assign owner add
2:50:14tax add task. Let's first select change
2:50:18status and change it to active
2:50:23and then you can add another action. Say
2:50:26that change severity and then change it
2:50:29to high and then add another action. You
2:50:33can also assign an owner to this. I'll
2:50:36assign my own name. And then add another
2:50:39action. You can add tags. I will say
2:50:43testing automation rule apply and then
2:50:47add another action you can add tasks. So
2:50:51if I provide the incident task title
2:50:54sorry
2:50:56saying testing and then you can also add
2:50:59a description to this saying once you
2:51:02see an incident like this validate the
2:51:06scope of the alert run so queries stop
2:51:09suspicious act process and isolate the
2:51:12affected machines block so and so IP
2:51:15addresses and URLs so you can add add
2:51:18tasks also here. So for now I'll uh
2:51:21delete this. And then you can also add
2:51:24rule expiration. It says the rule will
2:51:26be automatically disabled at the given
2:51:28expiration date. I'm going to leave it
2:51:31as indefinite. And something that I also
2:51:34wanted to show is in actions you can
2:51:36actually decide which one should happen
2:51:39first. You can move it using these arrow
2:51:41marks. Next is order. It says automation
2:51:45rules will be run in the order that is
2:51:47defined in the order field. Each
2:51:49automation rule will be executed
2:51:52serially after the previous rules in an
2:51:54ascending order. If you want this rule
2:51:58to run before an existing rule, you can
2:52:00also change this number and then say
2:52:03apply. It says saving automation rule.
2:52:05The automation rule is now saved. And
2:52:08this is how you create an automation
2:52:10rule. Now let's see how to trigger this.
2:52:14Let's go to entra ID. Add user. Create
2:52:19new user. User principle name. Testing
2:52:23automation rule. Same display name.
2:52:27Password is autogenerated. That is fine.
2:52:29Let's go ahead and review and create and
2:52:32then say create. Successfully created
2:52:36new user.
2:52:38Now let's go back to Sentinel
2:52:41Microsoft Sentinel Sentinel Workspace
2:52:44and then incidents. It will take a few
2:52:47minutes. Now under incidents, you can
2:52:50see that there are incidents created. I
2:52:53was testing something else. So there are
2:52:55other incidents as well. But let's go to
2:52:57the one that was triggered by the
2:52:58automation rule. Let's select this. It's
2:53:02near real time near you new user
2:53:04creation. Let's scroll down. view full
2:53:08details. So if you see what we did in
2:53:11the automation rule was to change the
2:53:13severity to high. This has happened and
2:53:16to change the status from new to active
2:53:19that has also happened and then assign
2:53:21it to an owner and then tag this
2:53:25incident. So you can see the tagging as
2:53:27well. Testing automation role. You can
2:53:30also see it under activity log. If I
2:53:32click activity log, it clearly says so
2:53:36you'll have to scroll down. First
2:53:37incident was created incident status was
2:53:40changed to active by automation rule.
2:53:43new user incident automation
2:53:46and then severity was changed to high
2:53:48again by the automation role and the
2:53:51owner was changed to Navia by automation
2:53:54rule and then there was also a tag that
2:53:57was attached to this incident and then I
2:54:00will close this. So this is how you can
2:54:03use automation rules. You can also try
2:54:06running playbooks with this. Once you
2:54:09create playbooks, you can attach it to
2:54:12the automation rules like we saw before.
2:54:15Let me go back to Sentinel. Now we will
2:54:17see how to create and configure
2:54:19playbooks.
2:54:21Before we start with creating playbooks,
2:54:23let's see what are some of the
2:54:24recommended use cases for Microsoft
2:54:27Sentinel playbooks.
2:54:29First one is enrichment. Playbooks can
2:54:31collect additional data and attach it to
2:54:34an incident, helping security teams make
2:54:37better informed decisions.
2:54:39For example, a playbook automatically
2:54:41enriches an alert about a suspicious IP
2:54:45by querying external threat intelligence
2:54:47sources and then appends the reputation
2:54:50score to the incident. External threat
2:54:53intelligence sources like virus total,
2:54:56Microsoft defender threat intelligence.
2:54:58You can get data from these. Next use
2:55:01case is birectional sync. So playbooks
2:55:04help synchronize Microsoft Sentinel
2:55:06incidents with external ticketing
2:55:09systems to ensure incidents are tracked
2:55:11properly. For example, when an incident
2:55:14is created in Microsoft Sentinel, a
2:55:16playbook automatically opens a
2:55:18corresponding ticket in Service Now,
2:55:20Jira or other IT service management
2:55:23tool. And when an incident is updated or
2:55:26resol resolved in Service Now, the same
2:55:29update is reflected back in Microsoft
2:55:32Sentinel. The next one is orchestration.
2:55:35Playbooks facilitate better coordination
2:55:37and communication among security teams
2:55:40by integrating with collaboration tools.
2:55:43For example, a playbook detects a high
2:55:45severity security alert. Then it sends
2:55:48an automated message to the sock teams,
2:55:51Microsoft Teams or Slack channel,
2:55:54ensuring analysts are immediately
2:55:56informed. The playbook assigns incidents
2:55:58to specific analysts based on predefined
2:56:01criteria, ensuring an efficient
2:56:04workflow. The next recommended use case
2:56:07is response. Playbooks enable automatic
2:56:10or semi-automatic responses to mitigate
2:56:13threats quickly. Example, if a playbook
2:56:16detects a compromised user account, that
2:56:18is say multiple failed login attempts
2:56:21followed by an unusual access. If it
2:56:24detects something like this, it can
2:56:26automatically disable the account in
2:56:28Entra ID and notify administrators.
2:56:31Also, during an investigation, an
2:56:34analyst can manually trigger a playbook
2:56:36to isolate a suspected uh compromised
2:56:39device from the network while gathering
2:56:41forensic data for further analysis. So,
2:56:44these are the recommended use cases of
2:56:47Sentinel playbooks. So if you go back to
2:56:50the portal and then under configuration
2:56:54go to automation and then click create.
2:56:57Here you can create playbook with
2:56:59incident trigger, alert trigger, entity
2:57:02trigger or you can also create a blank
2:57:05playbook. Usually the recommended
2:57:08practice is to go to playbook templates
2:57:11and start using the templates that is
2:57:13already provided by Microsoft Sentinel.
2:57:17For example, like the use case that we
2:57:19discussed, let's search for virus total.
2:57:23There is one under that. There are so
2:57:26many of them.
2:57:28Let's select this.
2:57:30Let's click view details. So, it talks
2:57:33about virus total what it can help you
2:57:36with. It also gives you the details that
2:57:39the virus total solution for Microsoft
2:57:42Sentinel contains playbooks that can
2:57:44help enrich incident creation with
2:57:47threat information and intelligence for
2:57:50IPs, file hashes and URLs from virus
2:57:54total. Let's go back and there are nine
2:57:58playbooks in this. Let's say install.
2:58:02Installation is in progress. Now it is
2:58:04successfully installed. Now let me click
2:58:08manage.
2:58:09It says nine of them have been installed
2:58:12but it requires configuration because
2:58:15you'll have to connect your virus total
2:58:17account with this. So these are the
2:58:19playbooks that are available. So it says
2:58:22get virus total domain report domain
2:58:25report for alert trigger domain report
2:58:28for incident triggered file info for
2:58:30both of them. IP report then URL report.
2:58:35If you select one of these. Let me
2:58:38select IP report for incident triggered.
2:58:41This is the playbook. Click create. So
2:58:44then there is subscription resource
2:58:46group like any other resource in Azure.
2:58:49Then the region, it is set to the same
2:58:52region as your log analytics workspace.
2:58:56So it is grayed out. Then provide the
2:58:58playbook name. So then there is an
2:59:01option to enable diagnostics logs and
2:59:04log analytics. This says push runtime
2:59:07events to log analytics to help monitor
2:59:09your playbook. If you want to do that,
2:59:12you can select this. For now, I'm going
2:59:15to unselect it. Then say connections.
2:59:20It says for each connector this playbook
2:59:22uses, you can choose to use an existing
2:59:25connection from another playbook.
2:59:26Otherwise, you must create a new
2:59:28connection and authenticate when you are
2:59:31brought to the log logic apps designer
2:59:34page. So, I'm going to review and
2:59:37create. So, it's giving me a warning
2:59:40sign saying authorize this connection
2:59:42after deployment in the logic app
2:59:44designer. It's for Azure Log Analytics
2:59:47data collector, Azure, Microsoft
2:59:50Sentinel, and Virus Total. Let me create
2:59:52the playbook. Deployment in progress.
2:59:56So the deployment is complete and here
2:59:58is the logic app for this. You can click
3:00:01on these blocks to see what it is doing.
3:00:04Connected to get virus total report
3:00:06incident trigger what it is trying to
3:00:09do. So the trigger is Microsoft
3:00:11Sentinel. The playbook starts when a
3:00:14Sentinel incident is triggered and then
3:00:17get IPs. It extracts IP addresses
3:00:21related to the incident and then for
3:00:24each IP address
3:00:26it that is extracted from the incident
3:00:29it can do these actions. First is get an
3:00:32IP report that is from threat
3:00:34intelligence sources like in this case
3:00:37virus total it's going to do that and
3:00:40then there is a condition in this
3:00:42condition check a decision this is a
3:00:44decision point based on the incident
3:00:46report that is checking if an IP is
3:00:49malicious
3:00:51if it is malicious it say takes an
3:00:54action such as blocking the IP or
3:00:56escalating it and then adding a comment
3:00:59to the incident if If it is false that
3:01:03if the IP is non-malicious, you can set
3:01:06it to mark it possibly safe and then log
3:01:10nodes in the incident. So the invalid
3:01:13error connection that I'm getting is
3:01:15because that these connectors are
3:01:17unauthorized. That means I'm not
3:01:19connected anything yet. So with this
3:01:22logic app that is the playbook you can
3:01:25ensure that all extracted IPs are
3:01:28checked categorized and logged
3:01:30appropriately in Microsoft Sentinel.
3:01:32Let's go back to Sentinel. Under content
3:01:35hub I will say content type only
3:01:38playbooks and then like the example we
3:01:42discussed we can search for service now.
3:01:45There is one for service now as well. So
3:01:47create service now record alert
3:01:50triggered and then create an update
3:01:52service now record. Let me click service
3:01:55now. It also tells us what are the
3:01:58things that it can do. The service now
3:02:01ITSM solution for Microsoft Sentinel
3:02:04makes it easy to synchronize incidents
3:02:06between Microsoft Sentinel and Service
3:02:09Now. So there are two options that it
3:02:11provides to do this. The first option is
3:02:13a recommended one where birectional
3:02:16incident sync happens that is retrieve
3:02:19Microsoft Sentinel incidents and
3:02:21automate the creation of incidents in
3:02:23service now. Birectional sync of status,
3:02:27severity, owner, comments, work nodes,
3:02:30entities and alerts. Details of alerts
3:02:33and entities added to work nodes.
3:02:36filtering of Microsoft Sentinel
3:02:38incidents based on tax or custom
3:02:40filters.
3:02:42So this is the uh playbook that you see
3:02:45in most of the organizations that you
3:02:47use because it's very easy to maintain
3:02:50incidents with this birectional sync.
3:02:53And then there is also another option
3:02:55which you can do unidirectional sync
3:02:58from Microsoft Sentinel to service now
3:03:01only from Sentinel to service now not
3:03:03the other way around. There are three
3:03:06playbooks for this like you can see
3:03:09here. Let me say install and then click
3:03:12view details and then click manage.
3:03:17Again the same error because we have not
3:03:20configured it. Service now details.
3:03:23There's one saying create and update
3:03:25service now record. Create snow record
3:03:27alert trigger. Create snow record
3:03:30incident trigger. Let me click create
3:03:33snow record incident trigger.
3:03:36Select this and say create playbook.
3:03:40Again you get the subscription and
3:03:42resource group. Playbook name.
3:03:46Go to connections. You'll have to
3:03:48connect service. Now here for now we'll
3:03:51say review and create. We do have the
3:03:54warning signs. For now we'll just go
3:03:57ahead and say create playbook. The
3:03:59deployment is successful. So Microsoft
3:04:02Sentinel incident is the trigger. your
3:04:04initializing variable and then switch
3:04:07case severity high case severity medium
3:04:10medium you can add more as well and then
3:04:14you can create the records.
3:04:16This is done in service now. For that
3:04:19you need to enter the details. You can
3:04:22also run any of the playbooks manually
3:04:24as well. You can try testing the
3:04:26connection using run. This is a logic
3:04:28app. So you can try running it manually
3:04:30as well.
3:04:32Let's go back. So here if you see in the
3:04:35prerequisites right, it also says you
3:04:38need existing snow instance and also
3:04:42snow credentials, username and password
3:04:44for this. Let me close this. Let me go
3:04:47to the content hub. There are other
3:04:49useful playbook templates as well like
3:04:52for example enter ID.
3:04:55See this. Let's select this.
3:04:58Let's click manage. It's already
3:05:01installed for one of the previous
3:05:03chapters. So you don't have to do it
3:05:05again. So if you see here, there are 11
3:05:08playbooks in this solution.
3:05:11Let me go next. Yeah, here we do have
3:05:14the playbook. So block enter ID alert,
3:05:18block enter ID user entity trigger, uh
3:05:21prompt user alert, prompt user inter
3:05:24incident, reset enter ID, password. So
3:05:28there are so many uh playbooks available
3:05:31here as well and you can click on it to
3:05:33see what is the description how you can
3:05:36do it. So for block Microsoft Enra ID
3:05:39user, it says for each account entity
3:05:42included in the alert, this playbook
3:05:44will disable the user in Microsoft Enra
3:05:47ID. Add a comment to the incident that
3:05:50contains this alert and then notify
3:05:53manager if a wheel will. It also gives
3:05:56you a note saying this playbook will not
3:05:58disable admin user.
3:06:01And then if you go back to the content
3:06:02hub and search for soore, there is
3:06:07Sentinel sore essentials. This is a
3:06:09solution. Let's install this.
3:06:13It's in progress. Okay, it got
3:06:15installed. Let me go back to it.
3:06:19It says the Microsoft Sentinel Sore
3:06:22essential solution for Microsoft
3:06:24Sentinel contains playbooks that can
3:06:27help you get started with basic
3:06:28notification and orchestration scenarios
3:06:31for common use cases.
3:06:34These include playbooks for sending
3:06:36notifications over email collaboration
3:06:39platform such as Microsoft Teams and so
3:06:41on. It has uh 18 playbooks. Let's hit
3:06:46manage.
3:06:48So first one is create incident from
3:06:50Microsoft forms response what this will
3:06:52do it will create a Microsoft sentinel
3:06:55incident. When Microsoft forms response
3:06:58is submitted.
3:07:00This is a very common use case. Usually
3:07:03the customers of your organization might
3:07:05be using a form to create their
3:07:07complaints or you know something is not
3:07:10working and that you can use that form
3:07:13to create sentinel alerts or incidents
3:07:16and then there is a playbook to create a
3:07:19new incident when a new email arrives.
3:07:22So you can send an email to a shared
3:07:25mailbox and that will create a Microsoft
3:07:29Sentinel incident but you have to share
3:07:32send an email with incident keyword in
3:07:35the subject.
3:07:37So there are also playbooks like this
3:07:40which will give you instructions of what
3:07:42to do when there is a ransomware attack.
3:07:45So it says this playbook will walk the
3:07:47analysts through four stages of
3:07:50responding to a ransomware incident that
3:07:52is containment, investigation,
3:07:55eradication, recovery, eradication and
3:07:57recovery and then prevention.
3:07:59So when you're giving step-by-step
3:08:01instructions to the analysts, you're
3:08:04making them more efficient and use their
3:08:06time more wisely. And then there is
3:08:09notifications that you can send when an
3:08:12incident is closed. So this playbook is
3:08:15utilizing new update trigger to notify
3:08:17person or group on Microsoft Teams or
3:08:20Outlook when incident is closed
3:08:23or incident is reopened. Incident
3:08:26severity is changed notification and
3:08:29then you can post message on Slack.
3:08:33Whenever there is an incident created
3:08:35you can post it on Slack channel.
3:08:38same for an alert and then post messages
3:08:40on teams for incident as well as an
3:08:43alert. There is also a playbook that
3:08:47looks for other alerts with the same IP
3:08:49as the triggered incident. When such an
3:08:52alert is found, this playbook will add
3:08:55the alert to the incident.
3:08:57And this will do it only if that
3:08:59incident is not already related to sorry
3:09:02that alert is not already related to
3:09:04another incident.
3:09:07There's also you can send emails when an
3:09:11incident is created with basic details
3:09:13like incident title, severity, tactics,
3:09:15what is the link.
3:09:18There is another one which says incident
3:09:20assignment uh shift. This is interesting
3:09:22because this playbook will assign an
3:09:24incident to an owner based on the shift
3:09:28scheduled in Microsoft Teams. So when an
3:09:31incident is assigned the incident owner
3:09:33will be notified you know with an email
3:09:37and then there is update trigger notify
3:09:40owner this playbook sends a team message
3:09:42to the new incident owner. Let's go back
3:09:46to the homepage
3:09:48under configuration automation
3:09:51playbook templates. You can customize
3:09:53all of these templates as well. You can
3:09:56add your own details to all of them. And
3:09:59one more setting that I wanted to show
3:10:01was under configuration
3:10:03uh settings here go to settings again
3:10:07and then if you scroll down there is
3:10:10playbook permissions.
3:10:12This controls who and what can execute
3:10:15and modify logic app playbooks in
3:10:17Microsoft Sentinel.
3:10:20So these permissions ensure secure
3:10:22automation and integration with external
3:10:24services.
3:10:26So, Microsoft Sentinel automation rules
3:10:28can trigger play uh logic app playbooks
3:10:31right for incident response. But for a
3:10:34playbook to be used, specific
3:10:35permissions must be granted to the
3:10:38appropriate users or services.
3:10:41So, if I click configure permissions,
3:10:44it says choose the resource groups that
3:10:47contain the playbooks you want to give
3:10:49Microsoft Sentinel permissions to run.
3:10:52So I'm going to choose Sentinel RG and
3:10:54then say apply. It is adding
3:10:57permissions. Says finished adding
3:11:00permissions.
3:11:02That is one thing you need to set. If at
3:11:04all you are having permissions problems,
3:11:06maybe this is a place that you want to
3:11:08check once. So that's it for this
3:11:11chapter guys. In this chapter, we
3:11:13discussed what are automated responses
3:11:16to threats in Sentinel, how you can
3:11:19configure them, why do you need them,
3:11:21what are the different ways that is
3:11:23automation rules and playbook. Playbook
3:11:26using Azure logic apps that is the
3:11:28source solution of Sentinel. How to
3:11:31create automation rules? How to create
3:11:33playbooks? What are the usually used uh
3:11:37recommended use cases for both of these?
3:11:40Let's move on to the next topic. Now in
3:11:43this chapter, we will discuss Microsoft
3:11:45Sentinel workbooks. What are workbooks?
3:11:48How are they used? What are the use
3:11:50cases? How to create them? How to
3:11:53customize them? We'll learn all of this
3:11:55in this chapter. So, what are Microsoft
3:11:58Sentinel workbooks? Workbooks are a
3:12:01powerful tool within Microsoft Sentinel.
3:12:05They offer interactive and customizable
3:12:07dashboards and organizations can
3:12:10visualize and analyze security data from
3:12:13various sources using these dashboards.
3:12:16So these workbooks provide security
3:12:18teams with realtime insights
3:12:22and hence improve their ability to
3:12:24detect and respond to threats
3:12:25efficiently. So by integrating many
3:12:29different data sets and fostering
3:12:31collaboration, Microsoft Sentinel
3:12:33workbooks play a crucial role in
3:12:36strengthening an organization security
3:12:38posture. Now let's see some of the key
3:12:40use cases for Microsoft Sentinel
3:12:42workbooks. First one is sock monitoring.
3:12:46So security operation centers that is
3:12:48socks rely on Microsoft Sentinel
3:12:51workbooks to track security events in
3:12:53real time. Analysts can monitor key
3:12:56performance indicators that is KPIs such
3:13:00as incident response times, alert
3:13:02volumes, and thread detection rates.
3:13:05Workbooks provide a centralized view of
3:13:08security metrics, optimizing workflows,
3:13:11and improving incident management. Next
3:13:14use case is compliance and audit
3:13:16reporting. So, organizations must comply
3:13:19with regulatory standards and audit
3:13:21requirements, right? So, Sentinel
3:13:23workbooks help generate compliance
3:13:25reports by visualizing data on security
3:13:28controls, user activities, and policy
3:13:30enforcement. Then the then these reports
3:13:33can be customized to align with specific
3:13:36frameworks, making it easier to
3:13:38demonstrate compliance during audits.
3:13:41Next use case is threat hunting. So
3:13:44workbooks support proactive threat
3:13:45hunting by visualizing anomalous
3:13:48behaviors, attack patterns and
3:13:50indicators of compromise that is IoC's.
3:13:53Security teams can leverage these
3:13:55insights to uncover hidden threats,
3:13:58investigate suspicious activities and
3:14:00take preventive actions before an
3:14:03incident escalates. So these are some of
3:14:06the key use cases of Sentinel workbooks.
3:14:10Now let's see how to create a workbook.
3:14:13Let's go to our Sentinel portal. In
3:14:16Sentinel, go to threat management.
3:14:18Actually, let me minimize this first and
3:14:21then go to workbooks.
3:14:24Here you can choose whether to use a
3:14:26pre-built template or create a new
3:14:29workbook from scratch. So for templates,
3:14:32you can go to templates. Here you can
3:14:34browse through the available workbook
3:14:36templates and select one that meets your
3:14:39needs. So these templates offer
3:14:41predefined visualizations and metrics
3:14:44making it easier to get started.
3:14:47If you want to create a workbook from
3:14:49scratch then select add workbook button.
3:14:52Once you click add workbook a blank
3:14:55workbook opens. You can add your custom
3:14:59queries and visualizations here.
3:15:02So if you see there are some metrics
3:15:04parameters that are already available
3:15:06for us. So let's click edit.
3:15:10If you want to rename this add your own
3:15:12uh description then you know edit this
3:15:15and you'll have to edit it in markdown
3:15:18text. So if you want to remove
3:15:20everything
3:15:22and then say my workbook
3:15:25so you can preview it in the preview
3:15:28box. So markdown language if it is
3:15:31double star that means it should be in
3:15:33bold. If I remove this
3:15:37and go back to preview it's not bold.
3:15:40You can also change color. You can
3:15:43change the font. You have many
3:15:45customization options. For example, if I
3:15:48say style color yellow and then font
3:15:52size I'm increasing to 20. Maybe I'll
3:15:54make it 40 for now. And then if I
3:15:57preview this is how it looks. And if you
3:16:00go back uh say I want to make it bold as
3:16:03well. Then say font
3:16:09colon bold. And then if you preview this
3:16:12it's in bold. It's in the color that I
3:16:15wanted and it's in the size that I
3:16:16wanted. I'm going to say done editing.
3:16:19So now this is how you can customize
3:16:21your description, your title
3:16:24and then you can customize the dashboard
3:16:26as well. So I can say edit. So this is
3:16:29the query. So it's showing me a bar
3:16:31chart. If I want to change it to say pie
3:16:34chart, I can do that.
3:16:36I'll just have to say pie chart and run
3:16:39query. It's working. So I'm going to
3:16:42scroll down. So if you see there is a
3:16:45pie chart now. Before it was a bar
3:16:47chart. So there is customization options
3:16:49for you and you can also choose the size
3:16:53if you want it medium. So you can see
3:16:56that the size changes and then large
3:17:00or small tiny full you have all the
3:17:04options here. If you don't want to do it
3:17:07in KQL you can also choose set by query.
3:17:11Say for example bar chart. So for bar
3:17:14chart it changed to this. I'm going to
3:17:17make it small so that it's easier to
3:17:19view. So this is the bar chart view,
3:17:22line chart,
3:17:24scatter chart. So you can choose
3:17:26whichever you want, whichever is easier
3:17:29for the people to analyze it. And also
3:17:32you can choose the time range. Do you
3:17:34want it for last 24 hours, 30 days? And
3:17:37you can also use custom dates. If you
3:17:39click here, you can see the custom time
3:17:41range that you can pick from.
3:17:43And if you check chart settings, you
3:17:47have other customization options. For
3:17:49example, custom number formatting. That
3:17:51means apply custom number formatting.
3:17:54This will let you set units style
3:17:57formatting options for number values.
3:18:00And then series setting. You can choose
3:18:03you can add more values here. For now,
3:18:06I'll close this.
3:18:08I'll also say here done editing. And if
3:18:12you see there is an add button where you
3:18:14can add text, parameters, links or tabs,
3:18:17add query, add metric and add group. So
3:18:21if you want to add a query for which you
3:18:23want the visualization, you can just add
3:18:26the query here and then run it to test
3:18:28if it is working. And if you are not
3:18:31comfortable with KQL, let's close this
3:18:34for now and then add again. Say add
3:18:37metric.
3:18:39So you can add metrics here. So if you
3:18:42say add metric, you have the name space
3:18:45and then metrics. You can choose
3:18:47whatever you want here. If at all you're
3:18:50ingesting data, then it will show you in
3:18:52the graph in the charts.
3:18:55And if you want a display name, you can
3:18:57use that. And also there is aggregation
3:18:59option. So different metrics have
3:19:01different available aggregation methods.
3:19:04Let me cancel this as well.
3:19:06And you can also publish your workbook.
3:19:09So publishing a workbook in Microsoft
3:19:11Sentinel means making it available for
3:19:13other users, teams or organizations to
3:19:16use either within your envir environment
3:19:19or as part of shared gallery template.
3:19:22So when you publish a workbook, you save
3:19:24it in a structured format, usually JSON,
3:19:27and share it in a repository such as
3:19:30GitHub or Azure. This allows others to
3:19:32easily deploy and use a workbook for
3:19:35security monitoring, compliance
3:19:37reporting or threat detection.
3:19:39So in the editing, if you go here, this
3:19:43is the advanced uh editor. Click on it.
3:19:47There is gallery template JSON that is
3:19:49available. You copy it, save it in JSON
3:19:54file on your local machine. Before that
3:19:57you need to add the properties to the
3:20:00JSON file like this style setting
3:20:03template ID and then schema
3:20:07and then you save the JSON file with a
3:20:09meaningful name like you know my
3:20:12Sentinel workbook uh my Sentinel
3:20:15workbook for sock something like that so
3:20:17other people know also. Then you upload
3:20:20the JSON file to GitHub. So navigate to
3:20:23your solution repository in GitHub.
3:20:26Locate the workbooks folder. If the
3:20:28folder does not exist, you'll have to
3:20:29create one. Then upload your JSON file
3:20:31to the workbooks folder. You also have
3:20:35options to upload optional logo. This is
3:20:38just an option. You can upload logos.
3:20:41You can upload capture and upload
3:20:43preview images.
3:20:45And you can also update the workbooks
3:20:48metadata JSON file. You can include
3:20:51relevant details such as workbook name,
3:20:53description, template ID, preview, image
3:20:56links, logo if at all you've added one.
3:20:59So that's how you publish your workbook
3:21:01also. It's very easy. First you go to
3:21:04advanced editor, copy the JSON,
3:21:08make it a file, add the properties.
3:21:10Before that, upload it to your GitHub.
3:21:14Add other preview images, optional logo,
3:21:18and update your workbooks metadata JSON
3:21:20file. So once you review all uploaded
3:21:23files to ensure correct formatting and
3:21:26naming conventions, validate your
3:21:28changes before submitting to pull
3:21:30request or integrating the workbook into
3:21:32your Microsoft Sentinel gallery. Let me
3:21:36close this. I'm not going to create this
3:21:38one because it might charge me. So I'm
3:21:42going to cancel all of that. And then if
3:21:44I go to templates here, let me say I'm
3:21:47going to select event analyzer. You can
3:21:50create your workbooks from here as well.
3:21:52The event analyzer workbook allows to
3:21:55explore, audit, and speed up analysis of
3:21:58Windows event logs including all event
3:22:02details and attributes such as security
3:22:04application system setup directory
3:22:07services DNS and others. It also tells
3:22:11you content source, Windows security
3:22:14events. So if I say view templates,
3:22:18you can select the subscription
3:22:20workspace and the time range that you
3:22:22want the data for. So that is how you
3:22:25customize your templates that are
3:22:27already available. And if you say save,
3:22:32it will ask you to set a location where
3:22:34you want to save this. Say East US. Yes.
3:22:39And now if you see in my workbooks,
3:22:42there is one that is already saved. If I
3:22:44go to my workbooks, there is event
3:22:46analyzer. You can view the template. You
3:22:48can view saved workbook and then also
3:22:51edit it. So I clicked on saved workbook.
3:22:54View saved workbook. You have the option
3:22:56to edit it as well. Let me go back. Let
3:23:00me delete this for now. Now let's see
3:23:03some of the key Microsoft Sentinel
3:23:05workbooks and their use cases. First one
3:23:08is security monitoring and incident
3:23:10investigation. For that we have
3:23:13analytics, health and audit. This
3:23:15ensures security rules are functioning
3:23:17correctly and tracks any changes made to
3:23:20analytics rules. Then there is incident
3:23:23overview provides detailed insights into
3:23:26security incidents helping analysts with
3:23:28triage and investigation.
3:23:31Investigation insights help security
3:23:34teams analyze incidents, bookmarks and
3:23:36suspicious activities through visual
3:23:39reports. MITER attack workbook map
3:23:42security events to the MITER attack
3:23:44framework providing better threat
3:23:46visibility.
3:23:48Then next one is identity and access
3:23:50management. In that we have Microsoft
3:23:53Entra audit logs. This tracks user
3:23:56operations including password and device
3:23:59management activities.
3:24:01Then there is Microsoft Entra Signin
3:24:04logs. This monitors user login activity
3:24:07including failed attempts and access
3:24:09locations.
3:24:11Next is identity and access workbook.
3:24:13This analyzes identity related security
3:24:16logs to provide insights into uh
3:24:19authentication activities.
3:24:21Next we have cloud and compliance
3:24:24reporting. Azure security benchmark.
3:24:27This evaluates cloud security posture
3:24:29across Microsoft 365, Azure, thirdparty
3:24:33and hybrid workloads. Cyber security
3:24:36maturity model certification CMMC.
3:24:40This assesses security compliance based
3:24:42on CMMC control frameworks.
3:24:46Zero trust. This visualizes zero trust
3:24:49security principles and aligns them with
3:24:52compliance frameworks. Next one is
3:24:54threat intelligence and hunting. Threat
3:24:58intelligence workbook. This provides
3:25:00insights into threat indicators,
3:25:02malicious domains and suspicious
3:25:04activities across cloud and hybrid
3:25:07environments.
3:25:09Microsoft Defender for cloud apps. This
3:25:11tracks cloud application usage,
3:25:14highlighting potential security risks.
3:25:17Security alerts workbook. This displays
3:25:20alerts across different security sources
3:25:23and helps analysts respond to threats in
3:25:26real time. Next one is workspace and
3:25:29data monitoring, data collection, health
3:25:32monitoring. This ensures data injection
3:25:34is running properly, identifying
3:25:37anomalies and log collection.
3:25:40Workspace usage report. This provides
3:25:42visibility into long consumption, data
3:25:46latency and cost management.
3:25:49Event analyzer. This enables in-depth
3:25:52analysis of Windows event logs including
3:25:55security system and application events.
3:25:59So in summary, Microsoft Sentinel
3:26:01workbooks provide realtime insights,
3:26:04streamline security operations, and
3:26:06enhance thread detection. So that's it
3:26:09about workbook guys. Microsoft Sentinel
3:26:11workbooks. Let's go to the next chapter.
3:26:15In this chapter, we will discuss
3:26:17Microsoft Sentinel watch lists. What are
3:26:20Microsoft Sentinel watch lists? So,
3:26:22these are a powerful tool that help
3:26:24security teams to organize and manage
3:26:27important data. Important data such as
3:26:30IP addresses, domains, user accounts,
3:26:34and device names. So these lists assist
3:26:38in detecting threats, investigating
3:26:40security incidents and responding
3:26:43effectively. So by using this watch
3:26:46lists right analysts can quickly filter
3:26:48information, correlate alerts and
3:26:51enhance their threat and threat hunting
3:26:54processes and then finally strengthening
3:26:56security operations.
3:26:59Watch lists in Microsoft Sentinel allow
3:27:01organizations to bring in external data
3:27:04and compare this data with security
3:27:07events within their Sentinel
3:27:08environment. Like for example, a watch
3:27:11list might contain high value assets,
3:27:14recently terminated employees or service
3:27:17accounts. These lists can be used in
3:27:20searches, detection rules, analytics
3:27:23rules, threat hunting activities, and
3:27:26automated response playbooks. Watch
3:27:29lists are stored within Sentinel as name
3:27:31value pairs. Watch lists are designed
3:27:34for fast query performance and low
3:27:37latency. This ensures quick access to
3:27:40critical security data. Now let's see
3:27:43when to use watch lists. First, rapid
3:27:47threat investigation and response.
3:27:50Security teams can quickly important
3:27:53data from CVS files and use them in
3:27:56alert rules, hunting queries, workbooks,
3:27:59and notebooks to detect and respond to
3:28:02threats more efficiently.
3:28:04So important data such as IP addresses
3:28:07or file hashes. For example, a company
3:28:10identifies a list of malicious IPs from
3:28:13an external threat intelligence feed. By
3:28:16adding these IPs to a watch list,
3:28:18Sentinel can automatically flag any
3:28:20network traffic from these sources for
3:28:23further investigation. Next use case is
3:28:26to manage business data. Organizations
3:28:30can import data like lists of privileged
3:28:33users or former employees to track
3:28:36potential risks. This allows for the
3:28:38creation of allow lists and block lists
3:28:42to control access and detect
3:28:44unauthorized login attempts. For
3:28:46example, a company creates a watch list
3:28:49of employees who recently left the
3:28:51organization. If any of these accounts
3:28:54show activity in security locks,
3:28:56Sentinel triggers an alert to
3:28:58investigate potential unauthorized
3:29:00access. The next use case is to reduce
3:29:03alert fatigue. By creating allow lists,
3:29:07security teams can suppress unnecessary
3:29:10alerts from trusted users or known safe
3:29:13activities. So this prevents security
3:29:15teams from being overwhelmed by false
3:29:18positives and allows them to focus on
3:29:21real threats. For example, a security
3:29:24team notices that their network scanning
3:29:26tool frequently triggers alerts. They
3:29:29add the tool's IP address to a watch
3:29:31list and hence preventing unnecessary
3:29:34alerts while still monitoring real
3:29:37threats. Next use case is to enhance
3:29:40event data. Watch lists provide valuable
3:29:43context by enriching security event logs
3:29:46with external data sources and this
3:29:50helps security analysts better
3:29:52understand potential threats and take
3:29:54appropriate action. For example, a watch
3:29:57list of known fishing domains is used to
3:30:00cross-ch checkck URLs found in company
3:30:03emails. If a match is detected, Sentinel
3:30:06automatically flags the email as a
3:30:08potential fishing attempt. So, these are
3:30:11some of the key use cases of when to use
3:30:15watch lists. Now, let's go to the Azure
3:30:18portal and see how to create them. In
3:30:20Sentinel, go to configuration and then
3:30:23watch list. Click new.
3:30:27I will add the name as internal scanner
3:30:31IP addresses
3:30:33and you can add a description. And then
3:30:36there is alias and alias is a unique
3:30:39identifier assigned to a watch list
3:30:41column while creating the watch list. So
3:30:44it allows security anallets to reference
3:30:46specific columns easily when uh writing
3:30:49queries in KQL. I will just say scanner
3:30:53IPS and then next. The next one is
3:30:57source type local file or Azure storage.
3:31:00I'll select a local file here. And then
3:31:03the file take file type is supposed to
3:31:05be CSV. This is number of lines before
3:31:09uh row with headings. I will enter zero.
3:31:12Then upload the file that you have. I
3:31:14created this file with example IP
3:31:18addresses
3:31:20and I uploaded it here. Drag and drop it
3:31:23or you can browse for files and then you
3:31:26select the search key. Search key in
3:31:29Microsoft Sentinel watch list is the
3:31:31column that you will be using frequently
3:31:34for searching, filtering and joining
3:31:36with other data in Sentinel. So this
3:31:40acts as the primary key for referencing
3:31:42the watch list and queries. So in this
3:31:45example I will select IP address column
3:31:48as the search key. This is the first
3:31:50column and this uniquely identifies each
3:31:53row and then say create. Next create
3:31:57again and it says you successfully
3:32:00created internal scanner IPs watch list.
3:32:03Let me refresh the page. And here is the
3:32:07internal IPs watch list. If I expand
3:32:10this show panel, it says it was created
3:32:13by whom when it was last updated, search
3:32:16keys, IP address. You can view and log.
3:32:19So if I click view and logs, it takes
3:32:22some time to reflect. I'll select KQL
3:32:25mode here. This is the query that is
3:32:28getting run that is get watch list
3:32:30scanners IPs. It'll list the IPs that we
3:32:33just created for this watch list, but
3:32:36it's going to take some time to reflect
3:32:37here. For now, let me close this and go
3:32:40back to the watch list.
3:32:43And then again, show panel. I can also
3:32:45update this watch list either edit watch
3:32:49list items or bulk update. So, if I
3:32:51click edit watch list items,
3:32:54it will let me add new uh rows or you
3:32:58can also delete them. If I select one of
3:33:00these IP addresses, I can either delete
3:33:02them or I can add new ones with the new
3:33:06button. So, let me close this.
3:33:11And then there was bulk update option as
3:33:13well. If I go to bulk update,
3:33:15I can upload a file again. So, it also
3:33:19gives me a message saying the updated
3:33:21watch list data will be appended to the
3:33:23existing list of items. Items with same
3:33:26properties will be dduplicated.
3:33:29Let me close it. Now let me see if we
3:33:33can see it in the logs. If we go to view
3:33:35and logs and then select KQL mode. So
3:33:40this is how it shows. When was it last
3:33:42updated
3:33:44ID search key description IP address and
3:33:48the threat level.
3:33:50So you're using this query get watch and
3:33:53the scanner IPS. This is the alias of
3:33:55that watch list. And if you want to see
3:33:58all the aliases of watch list, you can
3:34:01just say get watch list alias and run.
3:34:04There's only one that I've created and
3:34:06it comes up here. Scanner IPS. Now let's
3:34:09see how to use watch lists in analytic
3:34:12rules. So if you go to configuration and
3:34:15then analytics or either you can go to
3:34:18content hub and search for high count
3:34:24and then content type. I'm going to
3:34:26select only analytics rules
3:34:29and then select high counts of
3:34:31connections by client IP on many ports.
3:34:34Let's say install. And then it says
3:34:38description identifies when 30 or more
3:34:41ports are used for a given client IP in
3:34:4310 minutes occurring on the IAS server.
3:34:47And now the installation is successful.
3:34:49Then let's say create rule.
3:34:53And this is the rule query here. Let me
3:34:56close this and then go to configuration
3:34:59analytics.
3:35:02And if you see in the rule templates,
3:35:06this rule must be there. I'm going to
3:35:09right click and say create rule and then
3:35:12go to set logic. And let me expand this
3:35:16rule query. So we just need to add this
3:35:20line where get watchless scanner IPS is
3:35:23the function that retrieves the contents
3:35:25of a watch list named scanner IPS and
3:35:28then project IP address. The project
3:35:31operator is used to select specific
3:35:33columns from the watch list. And in this
3:35:35case, it extracts only the IP address
3:35:38column from the watch list and it
3:35:40filters out unnecessary data. And then
3:35:43there is let watch list and then there
3:35:47is let watch list. This let statement
3:35:50creates a temporary variable called
3:35:52watch list that stores the extracted IP
3:35:55addresses. And this variable can now be
3:35:57used in other queries to compare, filter
3:36:00or join with other security logs. And
3:36:03that's it. You don't have to do anything
3:36:05else. I just change the product project
3:36:08IP address to project IP space address
3:36:12because that is how I have given it in
3:36:14my watch list. Just have to go automated
3:36:17review create and it is validating the
3:36:21fields.
3:36:22Now it says validation passed and you
3:36:25can save the analytics role and in the
3:36:27active rules you can see that it is
3:36:30already saved and that's it about watch
3:36:33list guys. Let's go to the next chapter.
3:36:36In this chapter we will discuss
3:36:38Microsoft Sentinel notebooks.
3:36:41Let's get started. What are Microsoft
3:36:44Sentinel notebooks? Notebooks are a
3:36:47powerful tool in Microsoft Sentinel that
3:36:49help security teams analyze and
3:36:52investigate threats using code, data
3:36:55visualization, and machine learning.
3:36:58They allow analysts to write and run
3:37:01scripts for threat hunting, deep data
3:37:04analysis, and automation. Microsoft
3:37:07Sentinel stores a large amount of
3:37:09security data. And this data can be
3:37:11accessed through an API. And this API
3:37:15lets tools like Jupiter notebooks and
3:37:18Python connect to Sentinel. This makes
3:37:21it easier to explore security data, find
3:37:24threats, and automate investigations.
3:37:27What are Jupyter Notebooks? Jupiter
3:37:30Notebooks is an open-source tool that
3:37:33allows users to create and share
3:37:35documents that contain live code,
3:37:38visualizations, and text explanations.
3:37:42It supports multiple programming
3:37:44languages including Python and is widely
3:37:46used for data analysis, machine learning
3:37:49and security investigations.
3:37:52If you go to jupiter.org, you can find
3:37:54more documentation on Jupiter notebooks.
3:37:57If you scroll down, it has a Jupiter
3:38:00lab, Jupiter notebook. It supports over
3:38:0440 programming languages including
3:38:06Python. And you can also provide
3:38:09documentation for Jupyter notebooks
3:38:12here. So this is the documentation how
3:38:15to use it. Now let's see when to use
3:38:18Jupyter notebooks and Microsoft
3:38:20Sentinel. So like I said, Jupyter
3:38:22notebooks offer more flexibility and
3:38:24advanced capabilities beyond what is
3:38:27available in the Microsoft Sentinel
3:38:29portal. First use case for Jupyter
3:38:32Notebooks and Sentinel is advanced data
3:38:35analytics. So you can perform custom
3:38:38threat analysis using Python based
3:38:40machine learning techniques that are not
3:38:43available in Sentinel by default. You
3:38:45can detect anomalies and security logs
3:38:48with AIdriven models.
3:38:50Next use case is custom data
3:38:52visualizations.
3:38:54So you can create interactive timelines
3:38:57and process trees to analyze security
3:38:59incidents more effectively. You can
3:39:01build custom dashboards with unique
3:39:04visual elements not provided in Sentinel
3:39:07by default.
3:39:09Then the third key use case of Jupyter
3:39:11notebooks is integrating external data
3:39:14sources.
3:39:15So you can correlate on premises
3:39:18security data with cloud-based Sentinel
3:39:21logs. You can also analyze third party
3:39:23threat intelligence feeds alongside
3:39:26Microsoft Sentinel alerts.
3:39:29So, Microsoft Sentinel seamlessly
3:39:31integrates Jupiter notebooks into the
3:39:33Azure portal and this allows security
3:39:36analysts to query, analyze, and
3:39:38visualize data directly within a
3:39:41notebook environment. The KQL magic
3:39:44library enables KQL queries from
3:39:46Sentinel to run inside Jupyter
3:39:49notebooks. We will see that in the demo
3:39:51soon. Like any other module, if I go to
3:39:54the Azure portal and then go to threat
3:39:57management notebooks,
3:40:00you have readytouse notebooks that are
3:40:02created by security experts. If you go
3:40:04to templates,
3:40:06so some notebooks are scenario specific
3:40:09and can be used immediately. But others
3:40:12are templates demonstrating useful
3:40:15techniques that analysts can modify and
3:40:17customize for their needs.
3:40:20So if you go to templates there are
3:40:22templates that are available. It says
3:40:25notebook types investigation and
3:40:27hunting. You can filter it here as well.
3:40:30Configuration getting started hunting
3:40:33investigation. Let's close this for now.
3:40:37Also right if you go to this uh
3:40:39github.com Azure Azure Sentinel
3:40:42notebooks you can import notebooks from
3:40:45this repository as well. So you can use
3:40:49Jupiter notebook functions using
3:40:51browserbased interface or kernel that is
3:40:55execution engine. Browserbased interface
3:40:59this is where you write and run code
3:41:01such as Python and KQL queries and the
3:41:05results including text output tables and
3:41:08visualizations are displayed on the
3:41:10browser. And in the kernel execution
3:41:13engine, the kernel processes the code
3:41:15and returns results. And in Microsoft
3:41:18Sentinel, the kernel runs on an Azure
3:41:20virtual machine that is VM. The VM can
3:41:23handle multiple notebooks at the same
3:41:26time. And if complex machine learning
3:41:28models are used, more powerful VM
3:41:30options are available. We can use Python
3:41:34packages as well in Microsoft Sentinel
3:41:36notebooks. These are reusable libraries
3:41:39that help simplify coding by providing
3:41:42pre-written functions for various tasks.
3:41:46Microsoft Sentinel notebooks rely on
3:41:48several popular Python packages like for
3:41:50example pandas. These are used for uh
3:41:54data processing and manipulation.
3:41:57There is also something called as Mystic
3:41:59Pi. This is a security focused Python
3:42:02package and Microsoft Sentinel notebooks
3:42:05use Mystic Pi and this is a specialized
3:42:08package designed for threat hunting and
3:42:11security investigations, data enrichment
3:42:14and analysis visualizing security data.
3:42:18Now we will create a Sentinel notebook.
3:42:21For that it's the same location under
3:42:23threat management notebooks templates.
3:42:27Under this, we'll select get started
3:42:30guide for Microsoft Sentinel ML
3:42:32notebooks.
3:42:34So this notebook serves as a beginner's
3:42:36guide to using Jupiter notebooks for
3:42:39security investigations in Microsoft
3:42:41Sentinel. So it basically walks you
3:42:43through the fundamental steps needed to
3:42:45start working with Sentinel notebooks
3:42:48and provides references to additional
3:42:50documentation for advanced capabilities.
3:42:54So why we recommend to run this notebook
3:42:56first? This notebook helps you to
3:42:59understand the core features of Sentinel
3:43:01notebooks. Learn how to analyze security
3:43:04data using pre-built notebooks and get
3:43:07familiar with data visualization and
3:43:09threat hunting techniques. So what this
3:43:13uh notebook basically does is that it
3:43:15attempts to use sign-in logs data from
3:43:18your Microsoft Sentinel workspace.
3:43:21But note that realtime data is not
3:43:24available. It will automatically switch
3:43:25to sample data for demonstration
3:43:28purposes
3:43:29and then you click create from template
3:43:33and we get this error saying Azure
3:43:36machine learning workspace is required.
3:43:39So this workspace is for providing the
3:43:43necessary compute resources and
3:43:44infrastructure to execute Jupyter
3:43:46notebooks.
3:43:48So for that first let's go to the search
3:43:52bar and search for Azure machine
3:43:55learning
3:43:57and select Azure machine learning. Click
3:44:00create new workspace for ML projects and
3:44:04teams subscription is the same. I will
3:44:07select the same Sentinel group uh
3:44:10resource group name
3:44:15Sentinel notebooks demo
3:44:21region same it has a storage account key
3:44:24vault and there is application insights
3:44:27as well. So these will be newly created.
3:44:30If you don't want them uh to create new
3:44:32ones, you can use the already existing
3:44:35uh storage accounts or keywords or
3:44:37application insights, but you need to
3:44:39specify them here.
3:44:41Next, networking. I will continue with
3:44:44the public one. Workspace is accessed
3:44:47via public endpoint. So, I'll go with
3:44:49the same. And then next is uh
3:44:53encryption. So, this says your data is
3:44:56encrypted by default using Microsoft
3:44:58manage keys. So that is great. You there
3:45:01is also an option that you can say
3:45:03encrypt data using a customer managed
3:45:05key but I'll stick with Microsoft manage
3:45:08keys for now. And it also says once the
3:45:11workspace is created you cannot change
3:45:13the configuration.
3:45:15So this is good. I'll go next.
3:45:19I will go with the default settings here
3:45:21as well. Access type storage account
3:45:23credential based access. Next one is
3:45:25tagging. I don't need any tagging here.
3:45:28Next click review and create. It is
3:45:32running final validation. Now the
3:45:34validation is passed. I'm going to click
3:45:36create to create this resource and it is
3:45:41deployment is in progress. Now you can
3:45:44see that different resources are getting
3:45:46created. Application insight storage
3:45:48account key wall log analytics
3:45:50workspace.
3:45:52Now you can see that your deployment is
3:45:54complete. We'll go to resource and this
3:45:57is how your machine learning workspace
3:46:00looks. Now let's go back to Sentinel.
3:46:03Search for Sentinel.
3:46:05Sentinel workspace. Let me minimize
3:46:08this. Go to threat management notebooks.
3:46:12In that templates, we'll select a
3:46:14getting started guide for Microsoft
3:46:17Sentinel ML notebooks. Create from
3:46:20template. We don't get that error
3:46:23message anymore saying that you need
3:46:25Azure machine learning workspace. It's
3:46:27already selected the one that is
3:46:29available. Sentinel notebooks demo. Now
3:46:32let's say save.
3:46:35And now creating notebook.
3:46:38Cloning notebook. Notebook was saved
3:46:40successfully. Let's launch the notebook.
3:46:44And then we land here Azure AI Machine
3:46:46Learning Studio. Notebooks require a
3:46:49compute environment to run Python
3:46:51scripts, perform security investigations
3:46:53and process large data sets and that is
3:46:57why machine learning compute is needed.
3:47:00So if you see here there is compute
3:47:03and there's automatically serverless
3:47:06compute available. If you don't want to
3:47:08use this, you can create a new one. Let
3:47:11me create a new one. So click that add
3:47:14mark. Compute name. I'm going to keep it
3:47:17as Sentinel demo. Virtual machine type,
3:47:20I will go with CPU. The size
3:47:23recommendation virtual machine size.
3:47:27The recommended one is this one, memory
3:47:30optimized,
3:47:32but you can select any other one as
3:47:34well. I'll go with the cheaper one for
3:47:37now. And then say review and create.
3:47:41It'll ask you to review it and then you
3:47:45can click create. It says compute
3:47:47creation is in progress. You can now see
3:47:50it in your list of computes. So we will
3:47:52have to wait for this to complete. So
3:47:55here you can see that Sentinel demo the
3:47:58one that we created right now is getting
3:48:00created.
3:48:03Now you see that compute Sentinel demo
3:48:06uh provisioning succeeded. It took me
3:48:09around 5 minutes to get this completely
3:48:11created. Okay. So now it says that the
3:48:14computer is running. Now let's see what
3:48:17the notebook says. First it is talking
3:48:20about the prerequisites, some
3:48:21permissions which we already have. Then
3:48:23the contents of this notebook. What is
3:48:26Jupyter notebook? We went through it.
3:48:28And then running code in notebooks.
3:48:31We'll see that in a minute. Let me
3:48:33minimize this so that it's easier to
3:48:37read.
3:48:38And then we initialize the notebook and
3:48:41mysticpy query data from Microsoft
3:48:43Sentinel. We also configure and test
3:48:47external data providers. We connect to
3:48:49virus total and geolite 2.
3:48:53This is only an introduction. What is a
3:48:55notebook? Jupyter notebooks. We already
3:48:57went through it. So I'm going to skip
3:48:59this part. What is Jupyter notebook?
3:49:02Then the next part is running code in
3:49:04notebooks. This part explains how
3:49:08markdown and code cells work in Jupyter
3:49:10notebooks particularly in Azure ML
3:49:13notebooks and VS code. So if I scroll
3:49:17down you see add code cell as well as
3:49:20add markdown cell. Markdown cells it is
3:49:24used for writing text explanations
3:49:26documentation and even HTML formatting.
3:49:30It does not execute code but helps
3:49:32organize and explain notebook content.
3:49:36And then there is code cells that
3:49:38contain Python or KQL code for
3:49:41execution. And you can run queries,
3:49:43calculations, machine learning, modules
3:49:45and visualizations.
3:49:48And there is a run button that is play
3:49:50button here. If you want to run this
3:49:52code, you can just say run cell and it
3:49:55says congratulations. You can just you
3:49:58just ran this code cell 2 + 2 is equal
3:50:00to 4. You can also edit this. Suppose
3:50:03say I'm going to say 2 + 22 and also in
3:50:07the print statement I'll make it 22. If
3:50:10I run this, it says 2 + 22 is 24. So you
3:50:15can modify this as well. And there are
3:50:18other options as well available here.
3:50:20You can say clear output. If I click on
3:50:22clear output, the output that we got is
3:50:25cleared.
3:50:26and the output. So let me run this again
3:50:29then the output 2 + 22 is stored in this
3:50:33variable y now.
3:50:36So next if I scroll down so in Jupyter
3:50:39notebooks variable set within a code
3:50:42cell persist across other cells as well
3:50:44meaning you can use and modify them
3:50:47throughout your notebook. So this allows
3:50:50you to chain cells together and build on
3:50:53previous computation. So this value y
3:50:55the variable y is carried on to the next
3:50:58cell. And if I say y + 2 run this it
3:51:02becomes 26. 24 is taken from here 26.
3:51:0726 at the bottom. Next one is
3:51:10initializing the notebook and mystic pi.
3:51:13And the next step is initializing
3:51:16notebooks. So initializing the notebook
3:51:18and mystic pi. So let me scroll down.
3:51:22For this we will have to run this code.
3:51:25I'm going to hit run.
3:51:28It says checking upgrade to latest
3:51:30Mystic Pi version.
3:51:32And in the note you already see that
3:51:35don't be alarmed if you see
3:51:36configuration warning such as missing
3:51:38Mystic Pi uh config YAML. So this is
3:51:42completely fine if you get errors like
3:51:44this. So let me scroll down. It is still
3:51:46checking upgrade to latest Mystic Pi
3:51:48version. Uh it says starting AML
3:51:51notebook pre-checks.
3:51:54Azure CLI authentication recommended.
3:51:57Okay, we'll look into it. Notebook
3:52:00pre-checks complete.
3:52:02Once this is done, next scroll down
3:52:05again and see you can see the warning
3:52:08could not find mistake by config. That
3:52:10is fine. So after initializing notebooks
3:52:14we are getting this error and the note
3:52:17says you may need to restart the kernel
3:52:19to use updated packages.
3:52:22So let me restart it. So if you see here
3:52:24at the top there is restart kernel and
3:52:27run all cells and there is interrupt
3:52:30kernel then restart kernel. Let's do
3:52:32restart kernel. It says do you want to
3:52:35restart the current kernel? All
3:52:37variables will be lost. That is fine.
3:52:39Let's restart it. I will also
3:52:41authenticate here because we changed the
3:52:44computer. It asked me to authenticate.
3:52:46Let me authenticate there as well. It
3:52:49takes me to the another window.
3:52:52Now it says successfully authenticated
3:52:54to the compute. You can now use any
3:52:56Azure SDK.
3:52:59Now we will verify Microsoft Sentinel
3:53:02settings. For this, scroll down and run
3:53:04this query. And it is giving us a lot of
3:53:08errors.
3:53:10And it says the error is here where it
3:53:12is printing JSON.dumps.
3:53:19Uh if I go back that line is here.
3:53:23MP config dot settings is not a standard
3:53:26dictionary uh that can be converted to
3:53:29JSON. So instead it is likely a custom
3:53:32object that needs to be converted before
3:53:34using JSON dump. So let's try doing
3:53:37that. So I'm going to use
3:53:41this with this function an additional
3:53:44one. So I'm going to comment the
3:53:47previous one and let's see if that
3:53:49works. Let me run this.
3:53:56Yes, now the sample data is changed to
3:53:59the data that I have that is Sentinel RG
3:54:02is my resource group, my resour uh
3:54:04subscription ID, tenant ID, workspace ID
3:54:07and the workspace name. So this is
3:54:09working well. Let's scroll down. The
3:54:11next one is to configure your Azure uh
3:54:14cloud. This is optional and this needs
3:54:17to be done only if you're running in
3:54:20government cloud. So we most of the
3:54:22times it is not true. So we will skip
3:54:24this section and the next part is load a
3:54:27query provider for Microsoft Sentinel.
3:54:31So it says the query provider is the
3:54:33object you use to querying data from uh
3:54:36MS uh Microsoft Sentinel and make it
3:54:39available to view and analyze in the
3:54:42notebook. So for that you need to run
3:54:44this query and the next part is to
3:54:46authenticate to the Microsoft Sentinel
3:54:49workspace.
3:54:51Let's execute this code.
3:54:53Now it says connected and remember we
3:54:57authenticated here before we had this
3:54:59authentication label or error message or
3:55:02warning message and we clicked
3:55:04authenticate. It is exact. If you have
3:55:06not done that then you'll get an error
3:55:08message here. We did that previously. So
3:55:11it says connected now and then says test
3:55:14your connection using a mystic pi
3:55:16built-in Microsoft Sentinel query. Let
3:55:18me run this.
3:55:20And now as you see it returned results
3:55:23that means the connection is working.
3:55:25There is nothing the query uh returned
3:55:28no rows for this time range that is
3:55:30completely fine but it is still working.
3:55:33The connection is good. And the next
3:55:35step is to configure and test external
3:55:38data providers virus total and goite.
3:55:42In that the first option is to configure
3:55:45Azure key vault to store secrets. This
3:55:48is an optional thing. I would recommend
3:55:50doing it if you're doing it in the
3:55:51production environment. For now, we will
3:55:54not. We will skip this section. And then
3:55:57it says configure and test virus total.
3:56:01It also gives you a link to virus total.
3:56:04Let's open this in a new tab.
3:56:07It goes to virus total.com.
3:56:11And then uh this is a powerful threat
3:56:13intelligence source. So to connect this
3:56:17you need a virus total account and an
3:56:20API key. To sign up there is another
3:56:23link. Let's go open that to virus total
3:56:27and let's create an account.
3:56:30Oh okay. This is just a documentation.
3:56:32We don't need this. Let's just go to
3:56:35virus total homepage and then say sign
3:56:38up. Enter your name and then say join
3:56:42us. It's creating my account. I am not a
3:56:46robot.
3:56:48Signed up successfully. So, let's go
3:56:50back. Sign in. So, now I've signed up
3:56:55and signed in to virus total. And then
3:56:58if you click on your profile, there is
3:56:59API key. This is the API key that we
3:57:02need later. Let's go back to notebooks
3:57:05now. And let's run this. It is just
3:57:09initializing.
3:57:10And here select TI providers.
3:57:15It says loading. And then here add
3:57:18provider. I'm going to select virus
3:57:20total. And then add. And this is the
3:57:23place where you provide your API key.
3:57:26Let's go back to virus total. Copy the
3:57:29API key. Go back. Add the value here.
3:57:34And then if you scroll down there is
3:57:36update and then say save settings and
3:57:41then scroll down and run this again. And
3:57:45then here we go to goip providers. It's
3:57:48loading. Let's give it a minute. Here
3:57:50you select goip lights.
3:57:54And this is the goite to sign up page.
3:57:57You need to sign up here. Provide your
3:57:59name, your company name. Once you select
3:58:03go IP light click add and it's loading
3:58:08now you provide the API key now once you
3:58:11sign up and create your account in J
3:58:14light 2 and then once that is done it is
3:58:17the same thing you'll have to run these
3:58:20um cells and then you can test virus
3:58:25total lookup by running this
3:58:29and this is a sample data that is run
3:58:31there is an IP address that is selected
3:58:33it is running and it's gives you the
3:58:36data from virus total.
3:58:38Uh it's the same thing for test go
3:58:41location as well. So you get all the
3:58:43details here. You can also see the raw
3:58:46details of that IP address. And then if
3:58:49you scroll down there is test IP geol
3:58:52location lookup. I have not added the
3:58:54geoloccation, API key and the account
3:58:56details but you can do that for geoloite
3:58:592 as well and you will get similar
3:59:02results. It'll fetch the data and that's
3:59:04it about Microsoft Sentinel notebooks
3:59:07guys. Let's go to the next chapter. In
3:59:10this chapter we will discuss incident
3:59:12response and Microsoft Sentinel. This is
3:59:15a critical aspect of security
3:59:17operations.
3:59:18What is incident response? Incident
3:59:20response is a structured approach that
3:59:22organizations take to detect,
3:59:26investigate, and respond to cyber
3:59:28security threats and security breaches.
3:59:32The goal of the incident response is to
3:59:34minimize damage, recover quickly, and
3:59:38prevent future incidents.
3:59:40Now, let's look at some of the key
3:59:42phases of incident response.
3:59:45First phase is preparation. So before
3:59:48any incident occurs right, organizations
3:59:51must be ready to handle potential
3:59:53threats.
3:59:55So this involves developing an incident
3:59:57response plan, defining roles and
4:00:01responsibilities of the IR team. Next,
4:00:04setting up security tools, for example,
4:00:06SIM solutions, EDR solutions, and
4:00:09firewalls.
4:00:10And then conducting training and running
4:00:13tabletop exercises.
4:00:15and then creating playbooks for
4:00:18automated responses. So these are some
4:00:20of the key steps in preparation.
4:00:23Next phase is detection and
4:00:25identification. So this phase focuses on
4:00:28identifying suspicious activities that
4:00:31could indicate an attack. Security
4:00:33tools, logs and monitoring systems play
4:00:36a crucial uh role in this step. So this
4:00:40basically includes threat detection that
4:00:42is using SIMs like Microsoft Sentinel,
4:00:45Splunk to analyze logs and alerts.
4:00:50Next one is incident triage. So this is
4:00:52classifying incidents based on severity
4:00:55and impact. So in the previous chapters
4:00:57we saw that we can use automation rules
4:01:00in Microsoft Sentinel to do this.
4:01:03Next is verification that is determining
4:01:05whether it is a false positive or an
4:01:08actual threat. So an example for this
4:01:11phase would be say Sentinel detects
4:01:14multiple failed login attempts from an
4:01:16unusual location and then triggers an
4:01:18alert for potential credential theft.
4:01:22Next phase is containment. So once an
4:01:24incident is confirmed, the goal is to
4:01:26prevent it from spreading and limit
4:01:28damage. So this can include short-term
4:01:31containment that is isolating affected
4:01:34systems or accounts
4:01:36and forensic analysis like collecting
4:01:38evidence before making changes like for
4:01:41example if ransomware is detected the
4:01:44affected machine is quarantined to stop
4:01:46further encryption. Next step is uh
4:01:50eradication.
4:01:52After containment the root cause of the
4:01:54incident must be removed. So this can
4:01:56involve items like removing malware or
4:02:00back doors that is cleaning the infected
4:02:03systems,
4:02:05patching vulnerabilities like fixing
4:02:07security gaps that allowed the attack,
4:02:10updating security policies, for example,
4:02:13adjusting firewall rules or access
4:02:15controls.
4:02:16An example would be if an attacker
4:02:19exploited a vulnerability in outdated
4:02:21software, the software is patched and
4:02:24all affected endpoints are scanned for
4:02:26residual uh threats.
4:02:29Next phase of incident response is
4:02:31recovery. So the the focus here is to
4:02:35restore normal operations while ensuring
4:02:38no threats remain. This includes system
4:02:41restoration that is bringing cleaned and
4:02:43secured systems back online.
4:02:46Monitoring for reoccurrence that is
4:02:49watching for signs of reinfection.
4:02:52Credential rotation that is resetting
4:02:55passwords and strengthening access
4:02:57controls. An example of recovery stage
4:03:00would be that after removing malware,
4:03:02all business applications are restored
4:03:04from backups and logs are closely
4:03:07monitored for anomalies.
4:03:10The next key phase is lessons learned.
4:03:14So once the incident is fully resolved,
4:03:16a post incident review is conducted to
4:03:19improve future responses.
4:03:22So this includes incident report that is
4:03:25documenting what happened impact and
4:03:27response steps. Next process improvement
4:03:31that is updating policies and security
4:03:34tools.
4:03:35Next training and awareness educating
4:03:38employees on how to prevent similar
4:03:40incidents. So an example of lessons
4:03:43learned phase would be that if an attack
4:03:45occurred due to fishing, organizations
4:03:48implement stronger email security
4:03:50filters and conduct employee training on
4:03:54recognizing fishing emails.
4:03:56So these are the key phases of incident
4:04:00response. Now we'll go to the Azure
4:04:03portal to see how we can do this in
4:04:06Microsoft Sentinel tool. So we have
4:04:09already discussed thread detection. I'm
4:04:11not going to go in depth on that.
4:04:15So as soon as you log into Microsoft
4:04:17Sentinel in the overview tab, you
4:04:20already see how many incidents are
4:04:22available. I mean how many incidents are
4:04:24open, they are active
4:04:26or they are closed in the past uh 24
4:04:30hours.
4:04:31So you can get that information here as
4:04:34well. So if you want to check more data
4:04:36on incidents, you go to threat
4:04:38management under that incidents. Let me
4:04:42minimize this and close this too. So
4:04:46here it gives you a basic information
4:04:48saying there are four open incidents.
4:04:52There is one new incident, one, three
4:04:55active incidents. So new, active,
4:04:58closed, all these are the statuses. So
4:05:01here if you see the status of an
4:05:04incident can be new, active or closed.
4:05:09Let me close this for now. So you can
4:05:12see this from here. And it also tells
4:05:15you the severity of those incidents.
4:05:18There are three high incidents and one
4:05:21medium severity incident. There are zero
4:05:23low ones and zeroformational.
4:05:26So that also if you go to this any
4:05:29incident for that matter you can see
4:05:32here high uh that is the severity high
4:05:35medium low andformational
4:05:38and this is where you manage
4:05:42this top panel is the one where you
4:05:45manage all the incidents like bulk
4:05:47manage incidents you can create
4:05:49incident. So if you click here, you can
4:05:52also manually create an incident if it
4:05:54is not triggered. You provide the title,
4:05:58description, severity, status and owner.
4:06:01You can assign it to one of the users.
4:06:05You can also assign a tag to it and then
4:06:08create one.
4:06:10And the next one is to refresh.
4:06:12You can also enable this auto refresh
4:06:15incident. So you don't have to use this
4:06:17button. And then there is this time
4:06:20where you can say last 30 days, 14 days,
4:06:247 days or you can also use custom range
4:06:27to check your incidents if you're
4:06:29looking for something particular.
4:06:32And then actions.
4:06:34So in actions you can bulk change the
4:06:38severity of incidents and assign it to
4:06:41some owner and status. Bulk change them
4:06:45also the tax. And if you don't want any
4:06:48incident, you can delete them as well.
4:06:51Remember that this action is not
4:06:53reversible, it will not show up again
4:06:55here. But you can see it in the security
4:06:58incident table in the log screen. For
4:07:01now, I'll say no. So what it is talking
4:07:04about is if you go to general and then
4:07:06under that logs, let's close this. Let's
4:07:10select KQL mode and search for security
4:07:15incidents.
4:07:17This is the table say run.
4:07:20So this table has all the details of the
4:07:24incidents also. So you can find it here
4:07:27once it is deleted but it will not come
4:07:30back in the portal back there in the
4:07:32overview or you know incidents tab. And
4:07:35then there is security efficiency
4:07:38workbook.
4:07:39So this is a built-in dashboard designed
4:07:42to help sock teams measure and optimize
4:07:45their efficiency.
4:07:48So it tells us how many incidents
4:07:50created over time incidents by closing
4:07:53classification that we will see when you
4:07:55close a um an incident you can choose
4:07:59the classification
4:08:01and then there is incident created by
4:08:03severity by owner status. It also gives
4:08:08you the meanantime to triage, meanantime
4:08:10to closure.
4:08:13So this is a very helpful uh workbook
4:08:16for sock team. Let me go back. So let me
4:08:20select one of these incidents. So here
4:08:23you can assign it to the user. First is
4:08:27owner that is unassigned here. You can
4:08:30assign it to one of the users or there
4:08:32is status which is new, active and
4:08:35closed. You can change it from new to
4:08:37say active or closed. Severity again
4:08:41this is set up with medium but you can
4:08:44change it as well. And here it also
4:08:47tells you how many events uh do you have
4:08:50as evidence? How many alerts are there
4:08:52in this incident? When was it last
4:08:55updated and when was it created? It also
4:08:58gives you entities. We don't have any so
4:09:00it is not here.
4:09:03It also gives you an incident workbook.
4:09:05Let let's open it and see what that is.
4:09:09Okay. So, this is giving us details
4:09:11about that title, time created,
4:09:13severity, status, owner, products,
4:09:17alert, bookmarks, labels, if there are
4:09:19any classifications.
4:09:21It gives you all the details.
4:09:25Okay, let's go back. Then it also tells
4:09:28you how this uh got generated. It is
4:09:31because of the analytics rule rule
4:09:33called new user creation NRT. We created
4:09:36this in our previous chapters.
4:09:40So this rule was created just to
4:09:42identify if there was any new user
4:09:45created in enter ID. We use the KQL
4:09:48query for this that is we grabbed
4:09:51operations is equal to add user in audit
4:09:54logs. So there next there is tagging. If
4:09:57you if you want to tag it, you can also
4:10:00do that.
4:10:02And then you can write your comments
4:10:04here. And then you can go view full
4:10:07details.
4:10:09So here alert product names. It says
4:10:12Microsoft Sentinel here. And this refers
4:10:15to the sources or security solutions
4:10:18that generated these alerts.
4:10:21It can be Microsoft Defender for
4:10:23Endpoint, Office 365, identity, cloud,
4:10:27identity protection, right? So right now
4:10:30it is Microsoft Sentinel. And in the
4:10:33status, if you want to close this, you
4:10:35can select classification. This is what
4:10:37I was talking about before. So it is
4:10:40true positive that it is a suspicious
4:10:43activity. Then it is benign positive
4:10:46that is suspicious but expected. Next
4:10:49one is false positive which is incorrect
4:10:51alert logic.
4:10:53Another one is false positive which is
4:10:56incorrect inaccurate data. So either
4:11:00there can be false positive because of
4:11:02inaccurate data or incorrect alert logic
4:11:05and the other one is undetermined. You
4:11:08can select one of these while closing an
4:11:10incident.
4:11:12And then let's see what happens if I go
4:11:14to logs.
4:11:17So logs it runs the query for security
4:11:20incident. If I expand one of these you
4:11:23can see the tenant ID, time it
4:11:25generated, the incident title and the
4:11:29name that is the ID, severity, status,
4:11:33owner
4:11:34and a lot of other details.
4:11:37Okay, let me close this.
4:11:40And then there are tasks.
4:11:43One of the most critical aspects of
4:11:45running an efficient security operation
4:11:48center is the standardization of
4:11:50processes. That is all analysts follow a
4:11:54consistent and structured approach.
4:11:57That is to enhance incident response
4:12:00effectiveness, reduce human error and
4:12:03improve compliance with security best
4:12:05practices.
4:12:06So one of the ways to achieve that is
4:12:09creating a task. So you can add task,
4:12:12give a title and then provide the
4:12:15description what needs to be done. This
4:12:17is to ensure that no critical steps are
4:12:19overlooked in handling security threats.
4:12:24Actually let me close this and go back
4:12:27to another incident. So this incident
4:12:29was created before I created automation
4:12:32rules. Let me go to one of the other
4:12:35incidents and open them. view full
4:12:39details and then in the activity log if
4:12:42you see it says first incident was
4:12:46created at so and so time and then
4:12:48incident status was changed to active by
4:12:52automation rule automation rule demo. So
4:12:55I created an automation rule demo which
4:12:58changes the status of an incident from
4:13:01new to active whenever an incident is
4:13:04created. Then it changes the severity to
4:13:07high. And then it changes the owner to
4:13:11anybody that you want to.
4:13:14And then it also applies a tag saying
4:13:17created by automation rule. This is the
4:13:20tag that I added in the uh automation
4:13:23rule. All of this gets recorded and all
4:13:26the data is available in the activity
4:13:28log. Here you can also add different
4:13:32comments.
4:13:34Let me close this and go to tasks.
4:13:37There's already a task that is added
4:13:40here.
4:13:42Why? Because it was in the automation
4:13:45rule. So if I go to the automation rule,
4:13:48we have already gone through this in the
4:13:50previous section. So I'm not going to
4:13:53talk more about it. But the automation
4:13:55rule is automation rule demo. When an
4:13:58incident is triggered from sentinel with
4:14:02all the analytic rule names I've
4:14:04selected here the action change status
4:14:07to active change severity to high then
4:14:10assign owner as Navia and then add tag
4:14:13saying created by automation rule and
4:14:16then I have added a task saying sock
4:14:19user creation checks and then the
4:14:23checklist of what needs to be done when
4:14:25a new user is identified first I'm
4:14:28saying that the user request should
4:14:30verification should be done that is
4:14:32validate the request source is it an
4:14:35email is it a service now ticket or if
4:14:38there is manager approval for this and
4:14:41then request follows the organization's
4:14:44acqu uh access request procedures
4:14:47confirm that the requesttor has the
4:14:49authority to approve user creation so I
4:14:52have added the checklist which the sock
4:14:55analyst whoever comes here can go
4:14:57through it and make sure all of this is
4:15:00followed. That is how you can add tasks
4:15:03in incidents or or you can add directly
4:15:06as well. But I used automation rule to
4:15:09add this. You can add other tasks too.
4:15:13Suppose same I'm just adding something
4:15:16random saying verify with the one who
4:15:18created the user account and in the
4:15:21description you can same check the
4:15:24reason for the user creation
4:15:28and then I'm adding let me make this
4:15:30bullet points and then I can also add
4:15:34make make sure it is according to the
4:15:36principle of least privilege so you get
4:15:39an idea of what can be done with adding
4:15:42tasks
4:15:43and then I'll say save. So there's
4:15:45another task that is created. So it says
4:15:48this is created by me and the other one
4:15:50was created by the automation rule.
4:15:52Automation rule demo. So I told you I
4:15:56used automation rule to add a task. So
4:15:59you can also use uh playbooks in
4:16:02Microsoft Sentinel to add incident
4:16:04tasks. You'll have to select add task
4:16:07action in the Microsoft Sentinel con
4:16:09connector and this automatically
4:16:12attaches a task to the incident that get
4:16:14triggered because of that playbook.
4:16:17So how do you decide do if you want to
4:16:19use automation rule or playbooks to add
4:16:22tasks to incidents. So for automation
4:16:26rules right it is recommended for simple
4:16:29static tasks that don't require external
4:16:32conditions or automated actions. So it
4:16:35is best for predefined steps that apply
4:16:37to all incidents of a certain type. It
4:16:40is faster and more efficient as they are
4:16:43built in and require no additional
4:16:45configurations. Like for example, you
4:16:48can say automatically add a review
4:16:50incident details task to all high
4:16:52severity incidents or you can add a
4:16:55check affected users activity logs. Uh
4:16:58this is also a task to an incident
4:17:00tagged as identity compromise that you
4:17:03can do using automation rule.
4:17:06And when you want to use playbooks, you
4:17:09can use that for advanced scenarios
4:17:12where tasks depend on conditions or
4:17:15external system responses. And also the
4:17:18playbooks support dynamic decision
4:17:21making such as evaluating an incident
4:17:23context before assigning tasks. So for
4:17:26example, a playbook queries a threat
4:17:29intelligence uh source to check an IP
4:17:32reputation. Then if it is malicious, it
4:17:35adds a task saying block the IP address.
4:17:38But if it is not, it takes no action on
4:17:41it. So that's how you can use playbooks
4:17:44and automation rules to create tasks for
4:17:47incidents.
4:17:49And if you go to logs and uh go to KQL
4:17:53mode, search for security incidents.
4:17:58Run. Oh, sorry. I need to remove the
4:18:01pipe. Run.
4:18:04And then if I scroll more to the right,
4:18:08there is this field called tasks.
4:18:12There is a column under this which there
4:18:14is task. And if you want to search only
4:18:17for the incidents that have tasks
4:18:19assigned to them. So if I scroll to the
4:18:22right again to the tasks column,
4:18:25you see it is showing me only the
4:18:27incidents that have tasks assigned to
4:18:29them. So how to read this KQL query?
4:18:33Security incident. This is the table
4:18:36that contains incident data in Microsoft
4:18:39Sentinel. And then there is extent.
4:18:43This creates a new column called task
4:18:46count without modifying the original
4:18:49data.
4:18:51Two dynamic tasks. This converts the
4:18:54tasks field into a dynamic JSON like
4:18:58array. And this one array length to
4:19:02dynamic tasks counts the number of tasks
4:19:05within the tasks array.
4:19:08And then it is storing this data in task
4:19:11count. So next is where task count is
4:19:16greater than zero. This filters
4:19:18incidents where the task count is
4:19:20greater than zero. That is incidents
4:19:22that contain at least one task. If an
4:19:25incident has no tasks, it will be ex
4:19:29excluded from the results. So if you
4:19:31want to know more about KQL, there is a
4:19:33separate chapter on it, a big video. So
4:19:37I will link that in the description box.
4:19:40Let me close this now. Okay. And then
4:19:44there is incident actions. You can
4:19:46either run playbook, create automation
4:19:49rule or create team. Actually let me
4:19:52close this and go back to the incidents.
4:19:54Here also there is actions where there
4:19:57is investigation run playbook create
4:20:01automation rule and create team in
4:20:04investigate.
4:20:06This is actually the investigation graph
4:20:09uh that provides a visual approach to
4:20:11analyzing incidents. So we don't have it
4:20:13for our incidents because this is a
4:20:16simple analytics rule that is creating
4:20:18these incidents for the default
4:20:20incidents that are created in uh
4:20:23sentinel or the analytics rules where
4:20:26you have provided all the entities how
4:20:28it needs to be projected you will get
4:20:31the incident investigate option for now
4:20:35let's use the images provided by
4:20:37Microsoft to see how it works so invest
4:20:42investigation graph enables security
4:20:45analysts to explore alerts, entities and
4:20:47their relationships in an intuitive way.
4:20:51So while this method enhances
4:20:53situational awareness, it may require
4:20:55frequent context switching compared to a
4:20:58structured query based investigation.
4:21:01So you just need to go to in actions and
4:21:04investigate to launch the graph view and
4:21:07the graph will provide you you know
4:21:10details about the entities that are
4:21:12directly connected to the alert and each
4:21:16resource connected further and then you
4:21:19can select the entities and open the
4:21:22entities page where you can review
4:21:24information on that entity. Here it is
4:21:27for VM admin and you can hover over any
4:21:31entity in the graph to uncover a set of
4:21:34expert designed questions tailored to
4:21:37that specific entity type. So these
4:21:40questions are created by Microsoft
4:21:42security professionals and analysts to
4:21:45help you guide deeper analysis. They are
4:21:48referred to as exploration queries. For
4:21:51example, using those exploration
4:21:53queries, you can get something like
4:21:55related alerts. And by selecting these
4:21:59events here, you can actually get the
4:22:02raw event results and the query used in
4:22:04log analytics for these. And see, it is
4:22:08also giving you a timeline for these.
4:22:11And if you hover over the timeline, you
4:22:14can see which things on the graph occur
4:22:17during that time. So that's how you can
4:22:21in use the investigate tab to get the
4:22:23investigation graph. Let me go back to
4:22:26the portal.
4:22:28So we saw investigate. The next option
4:22:30is run playbook. You can execute a
4:22:33playbook on this incident to perform
4:22:35automated actions such as enrichment,
4:22:37response or collaboration. So if I go
4:22:40there and run playbook, I can select the
4:22:43playbooks and run them manually. Let me
4:22:46close this.
4:22:48And the next option is to set up an
4:22:51automation rule to apply specific
4:22:53actions automatically to future
4:22:56incidents generated by the same
4:22:58analytics rule and hence reducing the
4:23:01manual lo workload. So if I click create
4:23:04automation rule,
4:23:07it opens a page where it has already
4:23:09selected some uh parameters
4:23:13and you can modify this.
4:23:16Let me close that. And then the last
4:23:19option is create team. This is still in
4:23:22preview. It will help you to create a
4:23:25Microsoft Teams channel to facilitate
4:23:28realtime collaboration across different
4:23:30teams or departments involved in
4:23:32incident response. So if I click create
4:23:36sorry if I click create team it takes me
4:23:40to the team name. It tells you the
4:23:42incident number and then title then team
4:23:45description and you can add members to
4:23:48that group or groups to that uh team's
4:23:52channel and then create the team. So
4:23:54this is how an actual incident looks in
4:23:57Microsoft Sentinel mostly. This is how
4:24:00it does. So initial it is giving you the
4:24:03title saying initial access incident
4:24:06involving one user. So incident serves
4:24:09as a centralized case file that compiles
4:24:12all relevant evidence for a specific
4:24:15security investigation.
4:24:17So these incidents are built from
4:24:19alerts. So these alerts can be generated
4:24:22by analytics rules within Sentinel or
4:24:25imported from third party security tools
4:24:27that produce alerts. Each of these
4:24:30incidents inherit key attributes from
4:24:33alerts uh such as entity uh entities
4:24:37involved that is users, IP addresses,
4:24:40devices and so on like you can see here.
4:24:43And then severity levels to prioritize
4:24:47the response. Here it is high. And then
4:24:50incident status like we discussed
4:24:52before, new, active and closed. And it
4:24:55also provides you tactics and
4:24:57techniques. This is from MITER attack
4:25:00for contextual analysis. This is initial
4:25:03axis. So if you see here it is giving
4:25:06you the incident timeline all the
4:25:08different alerts in this and then
4:25:12entities there's an account there are
4:25:15like four IP addresses and then the
4:25:18similar incidents. So they it gives you
4:25:20similar incident so that you know you
4:25:22can go back and check the comments and
4:25:24find out what was done before. And in
4:25:26this if you see evidence there are no
4:25:28events that it says not applicable and
4:25:31there are 35 alerts in this. I feel like
4:25:35what are these bookmarks? These
4:25:37bookmarks actually it's called hunting
4:25:39bookmarks in Microsoft Sentinel and this
4:25:41allows security analysts to save
4:25:44important queries and query results for
4:25:46future reference. So if I go back to
4:25:49logs in Microsoft Sentinel right I've
4:25:52run this query saying audit logs where
4:25:54operation name contains add usually we
4:25:57don't prefer using contains you can use
4:25:59equal to equal to add user. So here if
4:26:03you select one of these or you know a
4:26:06couple of them you can add them. So
4:26:09click on add bookmark. It says, "Hunting
4:26:12bookmarks enable Microsoft Sentinel
4:26:14users to save, tag, annotate, share, and
4:26:19investigate results from a log analytics
4:26:21query." So this is the bookmark name.
4:26:24You can obviously change it. And it says
4:26:27event time mapping. Do you want to do it
4:26:29as of now or time generated or activity
4:26:32date time? Then you can add an entity.
4:26:35If you say add entity, you can select IP
4:26:39URL or whatever you want here. Close it.
4:26:43Add tactics and techniques. This is from
4:26:46my attack tank. You can select one of
4:26:49those. So our example incident was
4:26:51initial access. I'm going to select
4:26:53that.
4:26:55And then there are tax as well that you
4:26:57can add and you can create add your
4:27:01notes and create the bookmark. Let me
4:27:03just create this as an example. I'll
4:27:07close this. If I go to threat management
4:27:09under that hunting, I'm going to say
4:27:11okay. Here, if I go to bookmarks, I can
4:27:15see the one that I saved right now. It
4:27:18tells me this is the audit logs and
4:27:21audit logs bookmark event time raw
4:27:25result of this. And then I can
4:27:27investigate on it. say investigate
4:27:30and it'll take me to the investigation
4:27:33graph. I'll close this and then go back
4:27:37to incidents and then in incident
4:27:39timeline it is showing us what is the
4:27:41alert that you know is attached to this
4:27:44incident. You can also click on these
4:27:47three dots and remove this alert if you
4:27:50think it is not appropriate with this
4:27:52incident. You can also add alerts to
4:27:55incidents and you can run suppose say
4:27:58you click these three dots you can run
4:28:01playbook on this alert and also do the
4:28:04same with the entities as well. So I
4:28:06don't have any entities because I have
4:28:08not mapped them using my um analytics
4:28:11role. That is the also the warning that
4:28:14I'm getting here that you need to do
4:28:16that. But I have not done that here. But
4:28:19if I had done that, I would have got an
4:28:22option like this for my entities as well
4:28:24saying you can add to threat
4:28:27intelligence or you can run a playbook
4:28:30on this saying you know I want to block
4:28:32this IP address. So that's it about
4:28:34incident response in Sentinel guys.
4:28:38Let's go to the next chapter. In this
4:28:40next chapter, we will discuss threat
4:28:42hunting in Microsoft Sentinel. What is
4:28:44threat hunting? How is it different from
4:28:47analytic rules? We will learn KQL and
4:28:50also how to use hunting queries for
4:28:52threat hunting. In this chapter, we will
4:28:55cover the basics and the foundation for
4:28:58KQL that is required for threat hunting.
4:29:01But if you want to know KQL in depth,
4:29:04there is another video which I will link
4:29:06it in the description box. So what is
4:29:09threat hunting? Right. First is threat
4:29:12hunting is a proactive cyber security
4:29:14approach. In threat hunting, security
4:29:17analysts manually search for hidden
4:29:20threats that automated security tools
4:29:23may have missed. So instead of relying
4:29:25on alerts and incidents analysts use
4:29:29hypothesis,
4:29:30past data that is historical data and
4:29:34advanced queries to uncover hidden
4:29:36threats.
4:29:38In Microsoft Sentinel threat hunting is
4:29:41done using custoquy language that is KQL
4:29:44in the hunting blade. So let me go to
4:29:46the portal, select Microsoft Sentinel
4:29:50here and then go to the workspace and
4:29:53then here under threat management if I
4:29:56select hunting. This is where we usually
4:29:59do threat hunting. And this enables
4:30:02analysts to analyze logs, spot anomalies
4:30:06and investigate suspicious activity
4:30:08proactively. And if I scroll down here
4:30:11in configuration there is analytics.
4:30:15This is the analytic rules. This on the
4:30:18other hand, we discussed already that
4:30:20these are automated detection mechanisms
4:30:23that generate alerts when predefined
4:30:26conditions are met. So these rules
4:30:29continuously monitor logs and telemetry
4:30:32data to detect known threats or
4:30:35suspicious patterns. Now let's look at
4:30:38the difference between threat hunting
4:30:40and analytic rules. So threat hunting is
4:30:43a proactive approach whereas analytic
4:30:46rules is a reactive approach. Execution
4:30:49method in threat hunting in is manual
4:30:52and in analytics rule it is automated.
4:30:56The objective of threat hunting is to
4:30:58find unknown threats and analytic rules
4:31:01is to detect known threats. In threat
4:31:05hunting, it is interactive in uh
4:31:07investigation and analytic rules is
4:31:10based on predefined detection logic. The
4:31:14use case for thread hunting is mainly to
4:31:16search for hidden attack patterns and
4:31:20analytic rules is continuously monitor
4:31:22for threats. The tool used for uh threat
4:31:26hunting is hunting queries and for
4:31:30analytic rules are the rules that we
4:31:32defined previously in the other chapters
4:31:34like scheduled queries or near realtime
4:31:37queries. So for threat hunting or for
4:31:40analytic rules KQL is the foundation. So
4:31:43let's see what is KQL and the basics of
4:31:46KQL.
4:31:48All of us know that KQL stands for
4:31:50custoto query language.
4:31:53It is a readonly query language. So what
4:31:56is a readonly query language? Readonly
4:31:59query language is a type of query
4:32:01language that allows users to retrieve
4:32:04and analyze data.
4:32:07It does not allow to modify, update or
4:32:10delete the data. Okay. So that's why it
4:32:13is called readonly query language. It's
4:32:16mainly used for data exploration,
4:32:18reporting and analysis.
4:32:21So you can use KQL to write queries in
4:32:24Azure data explorer, Azure monitor, log
4:32:27analytics, Microsoft Sentinel and other
4:32:30Azure uh services.
4:32:32KQL follows a structured format for
4:32:35writing queries. So it makes it easier
4:32:37to read and understand. So now we will
4:32:40go to general and then under that logs
4:32:44and try to write some KQL queries
4:32:47understand KQL basics using security
4:32:50events table. So first let me switch
4:32:53from simple mode to KQL mode. Minimize
4:32:56this. So first in any KQL query is the
4:33:00table name. So table name represents the
4:33:04data source that is a table from which
4:33:07you are querying data. for example,
4:33:10security events. So as soon as I start
4:33:13writing right, you can see that KQ the
4:33:16query language starts giving me options
4:33:19and it also has uh tab completion.
4:33:24I'll select security events. There are
4:33:27other many other tables actually like
4:33:29signin logs, device network events,
4:33:32audit logs. So security event table is a
4:33:35built-in log table in Microsoft Sentinel
4:33:38and Azure monitor and it stores Windows
4:33:42security event logs collected from the
4:33:44connected machines. So if you remember
4:33:47in one of the previous chapters we
4:33:48connected a Windows machine uh VM to
4:33:53Microsoft Sentinel. So we will look at
4:33:55the data from that machine. So you can
4:33:58remove this pipe first. the table name.
4:34:01You can just run the table name and see
4:34:03if it has data. So, it does have data.
4:34:08And this is the time range. You can
4:34:10change the time range from here as well.
4:34:1230 minutes, 4 hours, 7 days or custom
4:34:16date also.
4:34:18And then if I use this pipe operator,
4:34:21right? This is used to chain commands
4:34:24together. that is passing the output of
4:34:26one operation as an input to the next.
4:34:30So we will pass the output of security
4:34:33event table as an input to the next one
4:34:36where we will say count.
4:34:39And also here if you click on this arrow
4:34:42you can see what the count operator
4:34:45does. It says it returns the number of
4:34:47records in the input record set and it
4:34:50also gives you the syntax and how to use
4:34:52it.
4:34:54So for now we are using pipe operator to
4:34:57get the output of security event table
4:35:01and send that as an input to count. So
4:35:04each command processes data and then
4:35:08passes it to the next command using the
4:35:11pipe operator. So if I run this now, so
4:35:14it says that these are the number of
4:35:16rows and count is an operator that
4:35:19simply returns the total number of rows
4:35:22or events in the security event table or
4:35:25any other table.
4:35:27And then the next operator is take. So
4:35:32take is used to retrieve a random subset
4:35:35of rows from a table. So it is useful
4:35:38for quickly previewing data without
4:35:41running expensive queries on large data
4:35:44sets. So I can say take five. So it'll
4:35:48return random five rows. As you can see
4:35:52here, these are random. Next time you
4:35:55run, you might get different result.
4:35:59And instead of take, you can also use
4:36:02limit. This does exactly the same thing.
4:36:06and take and limit can be used
4:36:09interchangeably. So if I say five, it
4:36:12will randomly return five random rows.
4:36:16So limit is just an alias for take. So
4:36:19you can use either with stake and limit
4:36:22it is random uh rows that are you know
4:36:26retrieved. But with sort is another
4:36:29operator you can order results based on
4:36:32one or more columns. You can say sort by
4:36:37time generated one of the columns in
4:36:40security event like you can see here and
4:36:42then run this. The default salt order is
4:36:46descending.
4:36:48But if you want it in ascending order
4:36:51you can say asc and then run it. So
4:36:55ascending is from smallest to largest or
4:36:58oldest to newest for time. uh in
4:37:01descending it is from largest to
4:37:04smallest that is newest to oldest for
4:37:08time. So if I use ASC I will get the
4:37:12oldest uh time first. You see this it is
4:37:18March 20th. If I just remove this or
4:37:22write descending and then run it will be
4:37:26March 23rd that is the newest data
4:37:28first.
4:37:30Similar to sort by, you can also use
4:37:33order by. It's going to do the same. So
4:37:36if you scroll down here, it exactly
4:37:39tells you what is the syntax and how you
4:37:42can use it, what is ascending, what is
4:37:44descending, where should you provide the
4:37:46column name, all the details. So if I
4:37:49run this, it's going to fetch me the
4:37:51same result as sort by. Next operator is
4:37:56project. This is used to select only
4:37:59specific columns from a table. So it
4:38:02helps reduce unnecessary data and
4:38:04improve query efficiency. So I can say
4:38:07project and it will start giving me the
4:38:11column names here. Uh I can say project
4:38:15account,
4:38:17then time generated comma and then I can
4:38:21say event ID and then run. So if you see
4:38:25you'll get only these three columns
4:38:27project is for that
4:38:30and if you don't want the default column
4:38:32names you can also change it like for
4:38:35example I can say project user is equal
4:38:38to account time is equal to time
4:38:42generated
4:38:44and ID is equal to event ID. So you see
4:38:49this is showing the previous result was
4:38:52showing us the default column names
4:38:54account time generated event ID. Now I
4:38:57will run this. So it changes to the user
4:39:01time and ID name. So you can use project
4:39:04to rename columns as well. And you can
4:39:08also use project to create a new column
4:39:12based on a calculated value. Let me
4:39:15remove all of this
4:39:18and project only event data
4:39:22and then run. So the this is the event
4:39:25ID data and then I'll say event category
4:39:32is equal to two string.
4:39:36Two string like you can see converts
4:39:39input to a string value. It is a data
4:39:42type conversion function. So it converts
4:39:45a value into a string that is text
4:39:48format and then I will say event ID
4:39:53multiplied by 10. If I run this, so this
4:39:57is something random that I'm trying to
4:39:59explain. So you have the event ID and
4:40:01then you created a new column called
4:40:03event category and then multi in that
4:40:06you're storing event ID multiplied by
4:40:1010. So this is how you can do it. So we
4:40:13got the event category and event
4:40:15category we're storing it as a text
4:40:18because you're using two string data
4:40:20type here. And the next operator that we
4:40:23will see is distinct. So distinct is
4:40:26used to remove duplicate values from a
4:40:29column or a combination of columns. So
4:40:32it returns only unique records from a
4:40:35data set. So right now you see event ID
4:40:38there are duplicate numbers 4 625 4625
4:40:41you know it goes on. So if I want to get
4:40:44only the unique values from it I can say
4:40:47distinct event id and then run you will
4:40:51get only the distinct values in that
4:40:54column. Next operator is where. So this
4:40:59is used to filter data based on specific
4:41:02conditions.
4:41:03It works like the where clause in SQL
4:41:07allowing you to retrieve only relevant
4:41:09rows. Like for example, I will say where
4:41:12security event pipet to where event ID
4:41:16is equal to is equal to equal to 4625.
4:41:23If I run this, so this will filter
4:41:26events where event ID is 4625.
4:41:30that is it is representing failed login
4:41:32attempts. You can also you know combine
4:41:36multiple com conditions using and or or
4:41:39operator. Like for example, if I say
4:41:42event ID is equal to equal to 26 4625
4:41:47and
4:41:49account is equal to admin and then if I
4:41:53run there are no results but if I change
4:41:57so if I'm using equal to equal to it'll
4:42:00exactly check for the same thing but I
4:42:03can say contains and then run we can
4:42:06still see results because it's not just
4:42:10admin it is administrator
4:42:12or you know capital admin. So you can
4:42:15use multiple conditions using e uh
4:42:18security event using and or or operator.
4:42:23So if you're using and you're getting
4:42:25event you're getting columns that event
4:42:29ID is equal to 4625 and also the account
4:42:32contains admin. If I do or it is either
4:42:37one of them. And you can also use where
4:42:41condition by to filter by time range
4:42:44like for example where time generated is
4:42:48more than a go 24 hours. So you can use
4:42:53it to filter by time range as well. uh
4:42:57also something that I wanted to mention
4:42:59suppose say if I'm using account
4:43:01contains
4:43:03admin this filters events where account
4:43:06column contains the word admin anywhere
4:43:10like what we saw before that it can
4:43:13contain anywhere administrator
4:43:16somewhere you know windows log demo and
4:43:19then administrator
4:43:21but if I use has and run it again it
4:43:25matches whole words only not the others
4:43:29like it is getting only admin right not
4:43:32the other things but if I say equal to
4:43:36equal to and then run it I will not get
4:43:38any results because with has it was in
4:43:42capitals and I am writing admin small.
4:43:45So top is the next operator. It is used
4:43:48to return the highest or lowest n
4:43:51records based on a specified column. So
4:43:54if I say top four by
4:43:59account and then run it. So it is
4:44:03returning the number of rows to return
4:44:06is four and account is the column to
4:44:09sort by and default is descending that
4:44:12is largest values first. So you see it
4:44:16started wy.
4:44:19So if I say top 10
4:44:23by time generated and then enter this is
4:44:28going to retrieve the top 10 latest
4:44:31security events uh that is sorted by
4:44:34time generated. You can also get the
4:44:37oldest by using ascend operator. So with
4:44:41sort it was sorting it but it was
4:44:44returning all rows without limiting
4:44:46results. But in top it returns only the
4:44:50top n rows based on sorting.
4:44:54Next operator is summarize. So this is
4:44:57used for aggregation that is allowing
4:44:59you to group data and apply functions
4:45:02like count, sum, average and so on. It
4:45:06works similarly to group by in SQL. So
4:45:09here if I scroll down it is giving you
4:45:12the syntax. It says summarize then the
4:45:15summarize parameter and then aggregation
4:45:18a call to an aggregation function such
4:45:21as count or average. So let's try this
4:45:24summarize. Then I will say count and
4:45:28then run and this will give you the
4:45:30total number of events in security
4:45:33events. So we are here counting the
4:45:36number of security events. You can also
4:45:38store this in an other column same total
4:45:41events
4:45:44is equal to count and then it stores
4:45:47that in a column called total events and
4:45:50then you can also use summarize
4:45:54event count
4:45:57is equal to count
4:46:00by event ID and then run. So this counts
4:46:05the number of occurrences for each event
4:46:08ID. So before we did it for the entire
4:46:11table, now we are doing it for each
4:46:13event ID. And if you want say only fail
4:46:17login count, you can do that as well. So
4:46:20I'm going to remove this. Say where
4:46:23event ID is equal to equal to sorry
4:46:27event ID is equal to equal to 4625
4:46:34that is to filter only failed login
4:46:36attempts. Then again we will pipe it and
4:46:40send it to summarize. Summarize failed
4:46:45login count
4:46:52is equal to count by account.
4:46:56Let me run this. So this counts failed
4:46:58login attempts for each account. So all
4:47:02of these accounts failed login attempts
4:47:04is going on. So mine is a test VM. So it
4:47:07doesn't matter. I usually keep it
4:47:09closed. So that is how you see the
4:47:12failed login attempts the number
4:47:16and you can do the same thing using
4:47:18event ID 4 624 to get the successful
4:47:23login.
4:47:24And then we will summarize saying last
4:47:27login time
4:47:32is equal to max
4:47:35time generated
4:47:39by account and this finds the most
4:47:42recent that is maximum login at login
4:47:46time for each user. If I run this, so
4:47:48you can see that the most recent login
4:47:52time for each user. And you can also use
4:47:56summarize to group multi group by
4:47:58multiple columns. So let me remove this.
4:48:01And then summarize
4:48:04event count
4:48:06is equal to count by event ID,
4:48:14comma, account. So if I run this, this
4:48:18groups by both event ID and account
4:48:20showing the number of events per user
4:48:23like this. Now that we know some basics
4:48:26of KQL, let's go back to Microsoft
4:48:29Sentinel the homepage. Let's close this.
4:48:32Okay. And under threat management, let's
4:48:35go to hunting. And there you can create
4:48:38a new hunt. Give the hunt name,
4:48:41description, owner. You can select the
4:48:44status and then hypothesis. Is it
4:48:47validated, invalidated, known? We'll do
4:48:50this in a minute. But first, let's go to
4:48:52queries. It also says the hunting query
4:48:56is moved under content hub. So you see
4:49:00there are few hunting queries because we
4:49:02have connected Windows machine to this.
4:49:06So if I go to content management and
4:49:10content hub, let me close this. And here
4:49:13content type I'm going to select only
4:49:16hunting query and then say apply. And
4:49:19status I'm going to select only
4:49:22installed and then apply. It gives me
4:49:24Windows security events. So here you can
4:49:27see there are many solutions installed
4:49:30under this. If I scroll to the right it
4:49:33says content type hunting query is also
4:49:36included.
4:49:37So there are many of these. Let's take
4:49:40one for example, least common processes
4:49:43by command line. So either I can run
4:49:46query or view results. So this is the
4:49:50query that is getting executed here. And
4:49:53it says it looks across your environment
4:49:55for least common process command lines
4:49:59may be noisy and require allow listing.
4:50:02And if I scroll down, it also tells you
4:50:04the tactic is execution. And then let's
4:50:08go and run this query. Click run query.
4:50:11When I run query, notice that the
4:50:13results go from zero. Right? So there
4:50:17are zero results. But still we will go
4:50:19ahead and view results here. So no
4:50:22results for the specified time range. I
4:50:25will go to KQL mode. So this is the
4:50:28hunting query. So let's try to
4:50:31understand what it is doing here. So
4:50:35this query analyzes process creation
4:50:38events 4688
4:50:41from security event locks and it filters
4:50:45suspicious executions while ignoring
4:50:48allowed processes and then it summarizes
4:50:51occurrences and applies sensitivity
4:50:53thresholds to detect anomalies.
4:50:57So the let operator here will define the
4:51:00variables. So these are the variables
4:51:03that are getting defined. First is the
4:51:06start time and then there is end time.
4:51:08Both of them together they define the
4:51:11search time range. So you can see that
4:51:14let me actually maximize zoom this more.
4:51:18So this two date time is a data type
4:51:22conversion function that converts a
4:51:24value into a datetime format. Okay. So
4:51:28we are taking the start time end time.
4:51:30This is to define the search time range.
4:51:34And then we are saying look back is
4:51:36equal to start time minus. So if you
4:51:40hover over it, it tells you what this
4:51:42is. So subtract and then 7D that is 7
4:51:47days. So it expands the search to 7 days
4:51:50before the start time. And then we are
4:51:54saying allow list. This is a list of
4:51:57known safe processes. So we are uh
4:52:01defining that as well. These are the
4:52:03safe processes.
4:52:05And then the sensitivity this is a
4:52:08threshold. The default is five to filter
4:52:10out frequent events. Next we are
4:52:13querying the security event table where
4:52:17time generated between lookup to end
4:52:20time that is it keeps the logs within
4:52:23the defined time range. And then it says
4:52:26event id is equal to equal to 4688
4:52:30that is it focuses on process creation
4:52:33events. And then new process name does
4:52:37not end with this is for do not the
4:52:42exclamatory mark ends with cornhost.exe.
4:52:46So it ignores anything with
4:52:49cornhost.exe.
4:52:51This is often a legitimate Windows
4:52:53process.
4:52:54And then we are using the extend
4:52:56operator. Extend operator is used to
4:52:59create new calculated columns or modify
4:53:02existing ones by adding computed values
4:53:06to a data set. So it does not remove any
4:53:08existing columns but it will simply
4:53:11append the new ones for analysis. Let me
4:53:14open a new query. So if I say security
4:53:20event pipe it extend event time
4:53:26is equal to
4:53:28to date time and then time generated.
4:53:32Then if I run this this will create a
4:53:35new column called event time like you
4:53:38can see here with date time format. So
4:53:41you can use extend operator to create
4:53:45new calculated columns or you know
4:53:48modify the existing ones. Let's go back
4:53:50to our query here. So you're using
4:53:54extend and creating all these new
4:53:56columns.
4:53:58This extracts the actual process name
4:54:01from a full path. Okay. So what we are
4:54:04trying to do so if there is a full path
4:54:07like this we are trying to extend uh
4:54:10extract only cmd.exe exe. Okay, let's go
4:54:15back here. This split. So, it splits the
4:54:19path into an array by using this
4:54:22operator
4:54:23backslash. Then array length. This
4:54:27counts the folder depth. And this one,
4:54:30this is proc array. And then we are
4:54:33creating another column where we are
4:54:36storing this value in it minus one that
4:54:39is proc array length. And then the proc
4:54:42is proc array and the last index which
4:54:45selects the last item the actual
4:54:47executable name. And then we are using
4:54:50where to filter and this excludes known
4:54:54safe executables. This is the variable
4:54:57we defined here allow list where we gave
4:55:01the safe execution names and that will
4:55:03be removed here. And then we are using
4:55:06summarize to aggregate suspicious
4:55:09commands and collect relevant metadata.
4:55:13So it is getting us start time that is
4:55:16the first time that the command appeared
4:55:18and the last time the command appeared
4:55:21and then this count will give you the
4:55:25number of times this command was
4:55:27executed. And then we are using dount
4:55:30computer that will give you the number
4:55:32of unique computers affected that is
4:55:36distinct count. And then make set
4:55:39computer max size is equal to 500. This
4:55:43stores up to 500 affected computers.
4:55:47The hosts we are storing up to 500
4:55:51affected computers.
4:55:53Then we are calculating the number of
4:55:55unique users who executed it that is d
4:55:59count subject username that is distinct
4:56:02count and then again we are storing up
4:56:06to 500 user accounts using make set and
4:56:10also again we are filtering it using
4:56:12where command and using sensitivity. So
4:56:16this will only keep low frequency
4:56:18events. So if a command appears too
4:56:21frequently it's likely a normal process
4:56:24that's the concept and then we are
4:56:26extending it that is it adds a timestamp
4:56:30column start time UTC for easier
4:56:34analysis. So in summary what this entire
4:56:37hunting query is doing it is detecting
4:56:40rare rare processes executions over a
4:56:437-day look back period. So we provided
4:56:47this and then it ignores safe or known
4:56:52processes using this allow list and then
4:56:56we are using extract to get the process
4:56:59names from full file paths and then it
4:57:03groups executions by command line
4:57:05arguments to find unique attacks. So we
4:57:09are using this. then it counts affected
4:57:12hosts and users while filtering out uh
4:57:16the frequent events. So this is how this
4:57:19hunting query works. I just wanted to
4:57:22show you this and explain how hunting
4:57:24queries are written. It's usually huge
4:57:27queries that you write. So you have to
4:57:29get used to KQL to understand the
4:57:32hunting queries.
4:57:34Now let me close this and then go to
4:57:37hunting under threat management and
4:57:39create a new hunt and this hunting we
4:57:43will do it for entra ID
4:57:46audit logs
4:57:49I will use the same description and then
4:57:52select an owner and the status is new as
4:57:55of now and the hypothesis is unknown
4:57:58because we haven't validated or
4:58:00invalidated it. You'll have to run it to
4:58:03see if it is validated or invalidated.
4:58:05So I'm going to say unknown and then
4:58:08create. So it created the new hunt. Let
4:58:12me go to this hunt. So again you can
4:58:15select the time here last 24 hours, 48
4:58:18hours, whatever the custom range that
4:58:21you want. Then there is query action.
4:58:23Either you can create a new query or add
4:58:26queries to your hunt. So for now we will
4:58:29select query actions. New query and the
4:58:32name for this we will use the same
4:58:34example like before new user creation
4:58:39and the query is audit locks where
4:58:42operation name is equal to equal to
4:58:47add user.
4:58:49So that's the query. If I scroll down
4:58:52tactics and techniques you can also add
4:58:55entity mapping which I'm not going to do
4:58:57here because we're keeping it simple.
4:58:58You'll have to select the tactics and
4:59:00techniques here. I'm going to select
4:59:02initial access and persistence and then
4:59:06create. So our new query is created. New
4:59:09user uh creation. And then we let's
4:59:12create another hunting query. This is
4:59:15for user deletion.
4:59:20And the query would be audit logs where
4:59:24operation name is equal to equal to
4:59:28delete user.
4:59:30And then scroll down tactics and
4:59:32techniques. So this is deletion. So it
4:59:35is related to evasion of techniques. I
4:59:39will say evasion masquerading and then
4:59:42create. Now we will try creating and
4:59:45deleting a user ID. Let's go to entra id
4:59:49and then add user create new user
4:59:56test user 4 and I will give the same
5:00:00display name and then review and create
5:00:04create okay successfully created user
5:00:07and then if I go to manage and then
5:00:09users I will delete the test user one so
5:00:13delete select it and say delete delete
5:00:17delete the selected user. Yes, user
5:00:19successfully deleted. And then now let's
5:00:22go back to our Sentinel. Let's show this
5:00:24panel. And then here we will do run
5:00:28selected. Okay, we'll select both and
5:00:29say run uh selected queries.
5:00:33You can see that new user creation the
5:00:35result is one. It might take a little
5:00:37bit more time for the other one to pop
5:00:40up. So let's get back to that later. But
5:00:42if I click on this user new user
5:00:45creation, you can also see that the
5:00:47result is one. And you can click the
5:00:49result to see what exactly happened. And
5:00:52here it says add user operation name.
5:00:56New user was added. Let's go back to the
5:00:59hunt. Refresh it again. Let's see if the
5:01:02deletion comes up. And if you also see
5:01:04here, right, it says initial access. The
5:01:07tactics that we selected previously, it
5:01:09shows up here. We selected initial
5:01:11access and persistence it is coming up.
5:01:14So it's still taking time but results
5:01:16should be uh appearing here where user
5:01:19deletion there is one and new user
5:01:22creation. We already saw new user
5:01:24creation it's taking time for it to load
5:01:27and then once you have validated the uh
5:01:30results you can come here say actions
5:01:33and you can create an incident for this
5:01:35or add new queries to hunt. So you can
5:01:40create an incident. Click create
5:01:42incident. And it will create the title,
5:01:44description. It's getting it from the
5:01:47previous hunt page. Severity, medium,
5:01:50you can change it. Status is new. And
5:01:53you can assign the owner. And then go to
5:01:55bookmarks. If there are any bookmarks to
5:01:58your incident, we already talked about
5:02:00bookmarks. You can add them here. So you
5:02:03can say add bookmarks. And then search
5:02:06for your bookmark and add them here. and
5:02:08then create new. So successfully created
5:02:12an incident
5:02:14and then if I open this panel I can also
5:02:17say status is closed because we already
5:02:21did validate this and also we validated
5:02:25the query. So I can select validated and
5:02:29then say apply for the hypothesis.
5:02:32And then if I go back to Microsoft
5:02:35Sentinel under incidents, our new
5:02:38incident that is entra ID audit logs is
5:02:41created. And also if you notice new user
5:02:44creation is also created because we
5:02:47created NRT rules and also a schedule
5:02:51rule which will run every 5 hours for
5:02:55that as well. An incident is created
5:02:57because we created a new user. So if I
5:03:00go to enter ID audit logs and say view
5:03:03fuel full details and then go to
5:03:06activity log it says incident was
5:03:09created and then because of the
5:03:11automation rules that we have set up
5:03:13before incident status was changed to
5:03:16active by the automation rule and then
5:03:19the severity was changed to high and
5:03:22then it was assigned to an owner and
5:03:25also a tag was created saying automation
5:03:28rule. Let me close this. Go back to our
5:03:31hunting.
5:03:33And the other options here, you can
5:03:35delete this hunting if you don't want
5:03:36this. And then you can add your comments
5:03:40to this as well. I will close this. So
5:03:43that's it about threat hunting guys.
5:03:46Now let's see how miter attack is
5:03:48integrated in Microsoft Sentinel. So if
5:03:51you go to Microsoft Sentinel in that
5:03:53threat management and under that there
5:03:55is MITER attack it is still in preview.
5:03:58In this let me zoom it a little bit. In
5:04:02this you can see legend. Uh this legend
5:04:05is to see how many detections are
5:04:07currently active in your work workspace
5:04:10for a specific technique. And you see
5:04:12here active scheduled active NRT query
5:04:16active anomaly. So all these are active
5:04:18rules in my work uh workspace.
5:04:22So then you can use this search bar to
5:04:24find a technique by its name or ID and
5:04:28this will show your organization
5:04:30security status for that technique.
5:04:33Suppose say I search for account
5:04:35manipulation.
5:04:38You get account manipulation here. So
5:04:40I'm going to select this and then expand
5:04:43this panel. Here you can view full
5:04:46technique details in the miter attack
5:04:48knowledge base. So you can see that
5:04:51there is also a link. So you see here
5:04:54view full technique details on the
5:04:55official miter attack website. So if I
5:04:58click this, it goes to miter.org and
5:05:02gives you the full description of what
5:05:04that technique is. Let me go back to
5:05:06Microsoft Sentinel and it also tells you
5:05:09what tactic it is and recommended
5:05:13coverage details. So what are the active
5:05:15rules covered under this? So if I click
5:05:17view, it tells me that these three rules
5:05:20anomaly rules are related to this
5:05:23technique. I'll go back. So actually as
5:05:26soon as you open, you can see that there
5:05:28are three detections associated with
5:05:30account manipulation. There are no
5:05:32incidents created for this or zero
5:05:35alerts.
5:05:36And then there is something called as
5:05:38simulated rules. Simulated coverage in
5:05:42MITER metrics shows detections that are
5:05:44available but not yet set up in your
5:05:47Microsoft Sentinel workshop. So it helps
5:05:50you see your potential security status
5:05:53if all available detections were
5:05:56configured. So if I go to simulated
5:05:58rule, select one of these rules.
5:06:01Actually first let me remove this search
5:06:04query and then come here, select for
5:06:07analytics rule. So you see here right
5:06:10143. So if I click this go back. So it
5:06:15is first giving you the description and
5:06:18then the active coverage but now it is
5:06:20also giving you simulated coverage. So
5:06:23if I click this view it will tell me
5:06:27what needs to be enabled or what needs
5:06:30to be scheduled for this specific
5:06:32technique. And then you can review it.
5:06:36See if you need to enable anything. Let
5:06:39me go back. And we have also seen if I
5:06:42go back to Microsoft Sentinel main page.
5:06:44Let me close this and go to the
5:06:46homepage. Let me zoom in and then go to
5:06:51configuration and then analytics. If you
5:06:54remember when we were creating rules
5:06:56here, you can set up these scheduled uh
5:06:59rules with MITER techniques. Like for
5:07:02example here it gives you a miter
5:07:04technique option and you can expand
5:07:06them. There are techniques too that you
5:07:09can select from. So this is how you do
5:07:11it in analytics rules. Let me see in
5:07:14incidents. Open one of these incidents
5:07:18and view full details. So here also
5:07:21there is a tab for tactics and
5:07:23techniques which is not filled here
5:07:25because it is created from an analytics
5:07:27role where we did not provide the
5:07:30technique. If so, it would come up here
5:07:33as well in the incident. So, there is an
5:07:35option in the incidents as well to show
5:07:38this.
5:07:39And then if I go to hunting under threat
5:07:42management, select this hunt and then go
5:07:46to the query. If I scroll down, there is
5:07:48technique. I can add the technique here
5:07:50as well. So, while creating a new query
5:07:53from query actions and scroll down, you
5:07:56can map it to the MITER attack
5:07:59techniques and tactics. So that's how
5:08:01you can use miter attack in Microsoft
5:08:04Sentinel. So basically what this miter
5:08:07attack page does. So if I go to miter
5:08:10attack, it helps you to view active
5:08:13detections in your workshop, identify
5:08:16available detections that you can
5:08:18configure using simulated rules and
5:08:21understand security coverage based on
5:08:23MITER attack tactics and techniques. So
5:08:27this is a very helpful page for sock and
5:08:30incident response teams.
5:08:33Now let's see how to enable workspace
5:08:35manager and configure it. So first step
5:08:38in that is for you to select which one
5:08:41is going to be your central workspace.
5:08:44in that central workspace in Microsoft
5:08:47Sentinel. If you scroll down to settings
5:08:51under configuration and then scroll down
5:08:54and then go to settings here and then
5:08:57scroll down there is workspace manager
5:09:00configuration. In this it is telling us
5:09:03what is this workspace manager. So
5:09:05workspace manager enables users to
5:09:08centrally manage multiple Microsoft
5:09:11Sentinel workspaces within one or more
5:09:14Azure tenants. The central workspace
5:09:18that is with the workspace manager
5:09:20enabled can consolidate content items to
5:09:24be published at scale to member
5:09:26workspaces. So this is the central
5:09:29workspace that I will be selecting. So
5:09:32then how to enable it? Also there is a
5:09:35note on that. For this you need at least
5:09:38two sentinel workspaces because this is
5:09:41the one that is used for multiple
5:09:43workspaces. So one workspace to manage
5:09:46from that is the central um workspace
5:09:50and at least one other workspace to be
5:09:52managed. So this is the the one other me
5:09:56so the at least one other workspace
5:09:59needs to be the member workspace. Then
5:10:02it says this is the role that is
5:10:03required Microsoft Sentinel contributor
5:10:06role and then if at all it is across
5:10:09multiple Microsoft entra ID tenants then
5:10:13you need to enable Azure Lighthouse. We
5:10:16will discuss Azure Lighthouse
5:10:18architecture as well in the future
5:10:20chapters. And then here if this is your
5:10:23central workspace you need to enable
5:10:25this make this workspace a central
5:10:28workspace. Okay, I've enabled it. And
5:10:31then it says go to workspace manager.
5:10:34Let me click on it. You can either do
5:10:37this, go to workspace manager from that
5:10:39page, settings page or you can find it
5:10:42under configuration as well. So here
5:10:45configuration and then workspace
5:10:47manager.
5:10:49Now let's see how to onboard member
5:10:51workspaces in Microsoft Sentinel. So
5:10:54member workspaces are workspaces managed
5:10:57by the workspace manager.
5:11:00So you can onboard some or all
5:11:03workspaces within a single tenant.
5:11:06Or if you have Azure Lighthouse enabled,
5:11:09you can get workspaces across multiple
5:11:12tenants as well. So if I go back to the
5:11:14portal, you can say add then workspace.
5:11:18There are no workspaces here. Let's
5:11:20create a couple of workspaces. So I will
5:11:23select log analytics workspace
5:11:27and the name I'll say workspace member
5:11:34one and then create running final
5:11:38validation validation passed and then
5:11:41again create. So it is initializing
5:11:44deployment. Submitting deployment.
5:11:46Deployment in progress
5:11:49and it says deployment is succeeded.
5:11:52Let's go back to log analytics workspace
5:11:56again and then create another one. So I
5:11:59will select the same resource group name
5:12:01of the instance workspace number two and
5:12:05then review same running final
5:12:08validation validation past and then
5:12:10create it deployment in progress
5:12:15and now it is created.
5:12:18So if you see here we've created the
5:12:20workspaces. Now let's add add both of
5:12:23these workspace members to Sentinel. So
5:12:26I'm going to go to Microsoft Sentinel.
5:12:29I've added one here already. Workspace
5:12:31member one. I'm going to create another
5:12:34one as well and select workspace member
5:12:362. Add. It is adding Microsoft Sentinel
5:12:40to workspace so and so. And now it is
5:12:42successfully added Microsoft Sentinel.
5:12:45And it also says it might take a few
5:12:47minutes for your workspace to appear in
5:12:49Microsoft Sentinel workspaces list.
5:12:51That's fine. I'm going to say okay. Then
5:12:54go back to Microsoft Sentinel. Refresh
5:12:57it. And then we will go to our central
5:13:00workspace.
5:13:02This is the main workspace where we
5:13:04enable workspace manager. So let me go
5:13:07to workspace manager. And now here try
5:13:10to add workspace. One of the workspaces
5:13:13is reflecting. The other one might take
5:13:16a little bit of time. That will also
5:13:18come up here.
5:13:20Let me refresh it again. and then go to
5:13:23add workspace. So you can see both the
5:13:26worker members are here. Now to onboard
5:13:29these select both of them or you can
5:13:32select whichever you want and then add
5:13:35and then you can see that the members
5:13:37here is from zero to it's increased from
5:13:400 to two the count and you can see your
5:13:43workspaces added here as well under
5:13:45workspaces
5:13:48and then also something that you notice
5:13:50here when I click add is group. So
5:13:53workspace manager groups help you
5:13:55organize workspaces based on business
5:13:57units, industries, regions or other
5:14:00categories. So these groups make it
5:14:03easier to distribute and manage security
5:14:05content across related workspaces. So to
5:14:09create a group, click add and then
5:14:11select group. Group name, I'll say demo
5:14:17workspace group.
5:14:21same description
5:14:23then there is an ID as well which is
5:14:26generated by Sentinel then I will say
5:14:29select workspaces then add workspaces
5:14:33I'm going to select both of these and
5:14:35then click add and then select content
5:14:39so there are two methods to add content
5:14:42one is to go to add and then select all
5:14:46content so this adds all active content
5:14:49currently deployed in your central
5:14:51workspace and it's only includes
5:14:55active contents not the template. So
5:14:57keep that in mind. So it is a point in
5:15:01time snapshot that means newly added
5:15:04content won't be automatically included
5:15:06later on. So if you want that you have
5:15:08to again do this. So either you can
5:15:11select all content or you can select
5:15:14content
5:15:16and then a select content window opens
5:15:18like this and from here you can custom
5:15:21choose which content items to add.
5:15:25I don't have a lot of content in my
5:15:27workspace but this is how it'll show up
5:15:30if you have a lot of them enabled. This
5:15:32is from Microsoft documentation page. So
5:15:35this option that is if I go back to the
5:15:38sentinel let me close this and then
5:15:41select content add this only content one
5:15:46will let you efficiently manage and
5:15:48distribute analytics rules hunting
5:15:50queries workbooks and more across member
5:15:53workspaces or you can go with all
5:15:56content option. Uh let me select all
5:15:59content for now and then go to review
5:16:02plus create. It'll tell me what
5:16:05workspaces are selected, what content is
5:16:08selected. I've selected everything and
5:16:11then I will say create submitting group
5:16:14and then successfully we have one group
5:16:17here that has created demo workspace
5:16:20group. It has two workspaces and 45
5:16:24content items.
5:16:26And remember that once you create the
5:16:29group, it doesn't mean that the content
5:16:31items are published to the member
5:16:33workspaces. So you'll have to select the
5:16:36group like this and then say publish
5:16:39content. So I'm going to click publish
5:16:41content and then it says in progress.
5:16:44It's going to take some time. And then
5:16:46if I click actions either I can edit it
5:16:49or remove this uh group itself. Let me
5:16:53close this. So you can see it is in
5:16:54progress and you can also publish from
5:16:58here.
5:16:59So if you have multiple groups here and
5:17:01if you want to publish all those group
5:17:03definitions then you can select all of
5:17:05them from here and then hit publish. And
5:17:10then it took about 2 to 3 minutes. Now
5:17:12it has succeeded. If I click again on
5:17:16this expand this panel it says it is pub
5:17:20last published date. And then if you
5:17:22want to edit it, you can go to edit and
5:17:25then edit the content and update it.
5:17:29When you want to manage custom content
5:17:31in Microsoft Sentinel, there are two
5:17:33ways to do it. One is directly in your
5:17:36Microsoft Sentinel workspaces like we
5:17:38saw before or through an external source
5:17:42control repository like under
5:17:45repositories. So if you go to Microsoft
5:17:47Sentinel content management and
5:17:49repositories, you can add your external
5:17:52repositories here. So if you see here it
5:17:55says what is it? Microsoft Sentinel
5:17:59repositories help you automate the
5:18:01deployment and management of your
5:18:04Microsoft Sentinel content through
5:18:06central repositories like GitHub or
5:18:09Azure DevOps. So if I scroll down, it
5:18:11says supported source controls, GitHub
5:18:14and Azure DevOps. And also the supported
5:18:18content types include analytics rules,
5:18:20hunting query, playbooks, workbooks,
5:18:24parsers, and automation rules.
5:18:27And why do you want to use external
5:18:30repositories? Because first one, version
5:18:33control. It helps you track and manage
5:18:36changes uh efficiently.
5:18:39Then automation that is automatically
5:18:42deploy your updates to your workspaces
5:18:45and then also helps with collaboration
5:18:48that is it enables multiple users to
5:18:50contribute and review content.
5:18:53Now to create a repository you go to
5:18:56repositories under content management
5:18:59add new and then provide the name
5:19:03demo external repositories
5:19:08and then under source control you can
5:19:10either choose GitHub or Azure DevOps. So
5:19:13if I select GitHub and then say
5:19:15authorize, it will take you to the
5:19:17GitHub page to provide your user ID
5:19:20password or it's the same with Azure
5:19:22DevOps. If I say Azure DevOps and
5:19:25authorize, it's going to take you to the
5:19:27Microsoft page login page to authorize
5:19:30that is provide your user ID and
5:19:32password. Let me close this. Once you
5:19:35authorize say for example for GitHub if
5:19:38you authorize it properly then you will
5:19:40get the repository to select you can add
5:19:43your repositories branch and then
5:19:46content type content type like analytics
5:19:48rules or hunting queries or so on. I'm
5:19:51not going to go ahead with this so I'm
5:19:53going to close it. Once you have added
5:19:55your connections you can edit them,
5:19:57delete them, update them, right? And
5:20:00this also talks about the permission
5:20:02requirements. Creating a connection in
5:20:04Microsoft Sentinel repositories
5:20:06authorizes Microsoft Sentinel to
5:20:09automatically deploy content from the
5:20:12source control repositories. That is the
5:20:15external repositories we are connecting.
5:20:17Microsoft Sentinel is automatically
5:20:20authorized to deploy content from those
5:20:23external repositories. So as a result,
5:20:26Microsoft Sentinel requires the owner
5:20:29role in the resource group containing
5:20:31the workspace from any user creating a
5:20:34connection. So make sure you have the
5:20:36necessary permissions as well. We have
5:20:39now come to the end of the full
5:20:41Microsoft Sentinel course. If there are
5:20:44any topics that I missed or if you have
5:20:46any questions or comments, please leave
5:20:48them in the comment section below. Thank
5:20:50you so much for watching. We hope this
5:20:52video was helpful. If it was, please
5:20:54don't forget to like, subscribe, and
5:20:56share our videos. Thank you. Goodbye.

About this transcript

This page contains the full transcript of Microsoft Sentinel Training | Azure Sentinel Tutorial | Microsoft Sentinel Step-by-Step Guide by CyberPlatter, generated from the public captions YouTube serves with the video. The transcript has 42,884 words across 6,863 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.