Content Comparison

Event Hubs is a fully managed, real-time data ingestion service that's simple, trusted, and scalable. Stream millions of events per second from any source to build dynamic data pipelines and immediately respond to business challenges. Keep processing data during emergencies using thegeo-disaster recoveryand geo-replication features.

Integrate seamlessly with other Azure services to unlock valuable insights. Allow existing Apache Kafka clients and applications to talk to Event Hubs without any code changes—you get a managed Kafka experience without having to manage your own clusters. Experience real-time data ingestion and microbatching on the same stream.

How It Works

Image Added

You can use this document to learn how to create Microsoft Azure Event Hubs as an External Log Source.

Armor supports log collection for Microsoft Azure services listed below:

• Azure Event Hubs
• Azure Application Gateway

In this article:

Prerequisites:

Event Hubs and Storage Account Creation

Azure Event Hubs is a big data streaming platform and event ingestion service. It can receive and process millions of events per second.

Subscription

Create Subscription (if not available) - This should be controlled by the billing administrator of the Azure Directory.

Resource Group

Create Resource Group (if not available) - A resource group is a container that holds related resources for an Azure solution. The resource group can include all the resources for the solution, or only those resources that you want to manage as a group.

Create an Event Hubs Namespace

1. Log in to the Azure Portal. (https://portal.azure.com/#allservices)
2. In search box next to All services, type event hub when Event Hub appear in the search results, select it.
3. Event Hubs -> +Add
   Image Added
   
4. On the Create namespace page, take the following steps:
5. Select the subscription in which you want to create the namespace.
6. Select the resource group you created in the previous step.
7. Enter a name for the namespace. The system immediately checks to see if the name is available.
8. Select a location for the namespace.
9. Choose the pricing tier(Standard (can not be basic as we are required to provide a named ConsumerGroup)).
10. Throughput Units: 1
    1. Throughput units are explicitly selected by the customer, either through the Azure portal or event hub management APIs. Throughput units apply to all event hubs in a namespace, and each throughput unit entitles the namespace to the following capabilities:
    2. Up to 1 MB per second of ingress events (= events send into an event hub), but no more than 1,000 ingress events, management operations, or control API calls per second.
    3. Up to 2 MB per second of egress events (= events consumed from an event hub).
    4. Up to 84 GB of event storage (sufficient for the default 24-hour retention period).
    5. Throughput units are billed hourly, based on the maximum number of units selected during this hour.
11. Setup Features
    1. Provides enhanced availability by spreading replicas across availability zones within one region at no additional cost. Learn more
       Image Added
12. Add tags: Tags can be any additional metadata used to describe the resource
13. Review and create
    Image Added
    

Create an Event Hub

1. On the Event Hubs Namespace page, select Event Hubs in the left menu.
2. At the top of the window, click + Event Hub.
   Image Added
3. Type a name - armor-logs for your event hub, then click Create.
4. Click Create
   Image Added
   

Create Shared access policies

1. On the Event Hubs Namespace page, select Event Hubs in the left menu.
2. Select Event Hub created above, armor-logs.
3. Select Shared access policies in Settings in the left menu and click +Add
   1. Enter Policy name → armor-logs
   2. Select Manage
   3. Click Create
      Image Added

Create a Consumer Group

1. On the Event Hubs Namespace page, Click Event Hubs in the left menu.
2. Select Event Hub created above, armor-logs.
   
3. Select Consumer groups in Entities and click +Consumer group
   1. Name → armor-logs
   2. Click Create
      Image Added
      

Retrieve Event Hub Connection String

1. Select All services, then type event hub when Event hub appear in the search results, select it.
2. On the Event Hubs Namespace page, select Event Hubs.
3. Select Event Hub created above, armor-logs.
4. Select Shared access policies in settings.
5. Select the Shared access policy created above, armor-logs.
6. Copy Connection string—primary key
   Image Added
   

Create Storage Account

An Azure storage account contains all of your Azure Storage data objects: blobs, files, queues, tables, and disks. The storage account provides a unique namespace for your Azure Storage data that is accessible from anywhere in the world over HTTP or HTTPS. Data in your Azure storage account is durable and highly available, secure and massively scalable.

1. Log in to the Azure Portal. (https://portal.azure.com/#allservices )
2. In search box next to All services, type storage account when Storage accounts appear in the search results, select it. All Services ->Storage accounts -> +Add
   Image Added
   
3. Basic :
   1. Storage account name
   2. Performance: Depending on the type of storage account you create, you can choose between standard and premium performance tiers.Learn more
   3. Account kind: Azure Storage offers several types of storage accounts. Each type supports different features and has its own pricing model.Learn more
      Image Added
4. Networking:
   1. Public endpoint (all networks)
      Image Added
      
5. Advanced:
   1. Secure transfer required: Enabled
   2. Large file shares: Disabled
   3. Blob soft delete: Disabled
   4. Versioning: Disabled
   5. Hierarchical namespace: Disabled
      Image Added
6. Tags: Tags are name/value pairs that enable you to categorize resources and view consolidated billing by applying the same tag to multiple resources and resource groups
   Image Added
   
7. Review and Create
8. Click Create

Retrieve Storage Account Connection String

1. Select All services, then type storage account when Storage account appear in the search results, select it. Click on the storage account → Settings → Access Keys
2. From the key 1 or key 2 section copy the Connection string
   Image Added
   

Create Armor Azure Event Hubs Log Source

1. In the Armor Management Portal (AMP), in the left-side navigation, click Security.
2. Click Log & Data Management.
3. Click External Sources.
4. Click the plus ( + ) sign.
   • If you do not have any log sources already created, then click Add a New Log Source.
5. Complete the missing fields:
   • In Endpoint, select the available Armor Endpoint.
   • In Log Source Type, select the Microsoft Azure Platform.

   • In HostName, enter a hostname

     Info
     The hostName must be unique for the selected log source type. Hostnames are case sensitive, validation is case insensitive.

6. In Protocol, confirm that the Azure Platform is selected.
7. Enter your Azure platform details:
   • Consumer Group, by default, armor-logs will be populated.
   • In Event Hub Connection String, paste your event hub connection string
   • In Storage Acc. Connection String, paste your storage account connection string
8. Click Save Log Source.
9. A message will display at the bottom of the screen, indicating that the log source has been created.
   Image Added