Create Flow Connection - Azure NSG Flow Logs

Network security group (NSG) flow logs is a feature of Azure Network Watcher that allows you to log information about IP traffic flowing through an NSG. Flow data is sent to Azure Storage accounts from where you can access it as well as export it to any visualization tool, SIEM, or IDS of your choice.

For more information on Azure NSG Flow Logs, please refer to Azure's documentation.

Common Use Cases

Network Monitoring: Identify unknown or undesired traffic. Monitor traffic levels and bandwidth consumption. Filter flow logs by IP and port to understand application behavior. Export Flow Logs to analytics and visualization tools of your choice to set up monitoring dashboards.

Usage monitoring and optimization: Identify top talkers in your network. Combine with GeoIP data to identify cross-region traffic. Understand traffic growth for capacity forecasting. Use data to remove overtly restrictive traffic rules.

Compliance: Use flow data to verify network isolation and compliance with enterprise access rules

Network forensics & Security analysis: Analyze network flows from compromised IPs and network interfaces. Export flow logs to any SIEM or IDS tool of your choice.

Prerequisites

1. Enabling flow collection is required in order to ingest flow data in the Armor SIEM. Refer Enable Flow Collection by Account

2. An Azure account with an active subscription

3. NSG flow log data is written to an Azure Storage account. To learn how to create and configure Storage account refer here

Create a VM with a network security group

Refer to below Microsoft docs link to create a VM with a network security group:

• Linux Virtual Machine

• Windows Virtual Machine

VM deployment is completed as shown in the following picture:

Enable Network Watcher

If you already have a network watcher enabled in the East US region, skip to Register Insights provider.

1. In the portal, select All services (https://portal.azure.com/#allservices). In the Filter box, enter Network Watcher. When Network Watcherappears in the results, select it.
   

2. Select Regions, to expand it, and then select ...to the right of East US, as shown in the following picture:
   
   
   

Register Insights provider

NSG flow logging requires Microsoft.Insights provider. To register the provider, complete the following steps:

1. In the top, left corner of the portal, select All services (https://portal.azure.com/#allservices). In the Filter box, type Subscriptions. When Subscriptionsappear in the search results, select it.
   

2. From the list of subscriptions, select the subscription you want to enable the provider for.
   

3. Select Resource providers, under SETTINGS.

4. Confirm that the STATUSfor the microsoft.insightsprovider is Registered, as shown in the picture that follows. If the status is Unregistered, then select Register, to the right of the provider.
   
   
   

Enable NSG flow log

1. NSG flow log data is written to an Azure Storage account. Refer to the article here to learn more.

2. In the top, left corner of the portal, select All services (https://portal.azure.com/#allservices). In the Filterbox, type Network Watcher. When Network Watcher appears in the search results, select it.

3. Under LOGS, select NSG flow logs, as shown in the following picture:
   

4. From the list of NSGs, select the NSG named myVm-nsg.

5. Under Flow logs settings, select On.

6. Select the flow logging version. Version 2 contains flow-session statistics (Bytes and Packets)
   

7. Select the storage account that you created in step 1.

Open

Open

8. Set Retention (days)to 5, and then select Save.
   
   

Configure the Azure Custom Deployment Template

You can use these instructions to send logs to Armor from the storage account configured above.

1. Click on the "Deploy to Azure" link from the README link mentioned above. This will redirect to the Azure portal. Enter, or select, the following information mentioned in README. Below is the screen with the sample configuration.
   

2. Read and acknowledge TERMS AND CONDITIONS and click Purchase

3. Wait for the deployment to complete.
   
   
   

Verify Connection In AMP

To learn more about "Search For Collected Logs In Kibana (BETA)" see the article here

1. Click Discoverto take you to the log search screen

2. Search with your Network security group name configured e.g. myVm-nsg

Webhook Tagging

To learn more about Webhook Tagging for Flow logs, see the article here.

Troubleshooting common issues

Refer to Microsoft docs here for common issues.

For specific issue related to the above-mentioned Armor Configuration

1. Validate Flow logs are stored in Storage Account configured

Refer to below Microsoft docs links to verify flow logs are stored in the storage account configured from the above steps and to learn about the NSG Flow log format.

• Verify Flow logs stored in the storage account

• View Flow log. Armor support only the Version 2 flow log event.

2. Validate Flows logs are transmitted to Armor

Using the below steps, you can verify that configured logs are been transmitted to Armor.

• In the top, left corner of the portal, select All services (https://portal.azure.com/#allservices). In the Filterbox, type App Service Plan. When App Service Plan appears in the search results, select it.
  

• In the App Service plans screen, search for the plan created, select it
  

• Under Settings, select Select the function app.
  

• Under Functions, select functions and then select BlobTriggerIngestAndTransmit
  

• In the BlobTriggerIngestAndTransmit, click on Monitor to view the Success Count and Error Count.
  

• If any Error operation is reported as showing in the below picture. You view the details by clicking on the respective Date (UTC)