Page Comparison

...

Table of Contents

maxLevel	3
minLevel	3

Understanding the Datalake

...

The Armor data lake is a centralized repository for storing Armor collected data. With regards to vulnerabilities, the data lake contains all the data for every report created for an environment and all the historical data from when the reports are run. This can be a lot of data so narrowing down the scope of information is critical to making sense of it all.

Accessing the Datalake

...

Users can access the Datalake in two ways:

...

Expand

title	Option 2: Log Search in AMP

Select a Report from the Report List and click the report name to access the details.
Image RemovedImage Added
Copy its unique report Id by navigating into the report's detail page.
Image RemovedImage Added
Navigate to Security -> Log Search and SSO into Chaos Search.
Image RemovedImage Added
Create a filter by doing the following:
1. Click on Add filter.
2. In Field select event.ReportId
3. Select is for Operator.
4. Paste the report Id from the report details page into the Value field.
5. Click Save.
  Image RemovedImage Added
Now set the date range to encompass the report date or dates to show and click Refresh.
Image RemovedImage Added

Data Presentation

...

Expand

title	Table Example

Field	Values
@timestamp	Nov 25, 2020 @ 07:32:27.480
#@version	1
_id	47741608
_index	1_1024_customer
#_score	1
_type	doc
armor_metrics.input_port	5445
armor_metrics.latency.processing	0.857
armor_metrics.processing_chain	["KVN_V4_collector_i-0908b8b2b53868dc0\|2020-11-25T13:32:27Z","KVN_V4_processor_i-0aa172c88f440b715\|2020-11-25T13:32:28Z"]
document_size	3,926
event_uuid	6d820110-73e5-45c9-945e-10c281fd4cb4
external_id	4f5b9ab7-8e57-4993-b0fb-440cd44d11e5
host.hostname	ip-10-0-0-8.us-west-2.compute.internal
host.ip	10.0.0.8
host.os.full	Amazon Linux 2
host.os.name	Linux
host_asset_id	75424166
index_type	ecs-1.5.0-vulnerability
labels.parent_id	1
logsource.origin	unknown
message_size	0
original_timestamp	Nov 25, 2020 @ 07:32:27.233
received_timestamp	Nov 25, 2020 @ 07:32:27.480
tags	["customer","flow_source_data_miss","default_parent_id","cached_parent_metadata"]
tenant_id	1024
vulnerability.category	["AMAZON LINUX","PCI"]
vulnerability.consequence	Allows unauthorized disclosure of information; allows unauthorized modification; allows disruption of service.
vulnerability.cve	http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-5188
vulnerability.description	Amazon Linux Security Advisory for e2fsprogs: ALAS2-2020-1509
vulnerability.diagnosis	<DIV> Issue Overview: <P>An exploitable code execution vulnerability exists in the quota file functionality of E2fsprogs 1.45.3. A specially crafted ext4 partition can cause an out-of-bounds write on the heap, resulting in code execution. An attacker can corrupt a partition to trigger this vulnerability. (<A HREF="https://access.redhat.com/security/cve/CVE-2019-5094" TARGET="_blank">CVE-2019-5094 </A>)</P><P>A code execution vulnerability exists in the directory rehashing functionality of E2fsprogs e2fsck 1.45.4. A specially crafted ext4 directory can cause an out-of-bounds write on the stack, resulting in code execution. An attacker can corrupt a partition to trigger this vulnerability. (<A HREF="https://access.redhat.com/security/cve/CVE-2019-5188" TARGET="_blank">CVE-2019-5188 </A>)</P> </DIV>
vulnerability.discovery	0
vulnerability.enumeration	135
vulnerability.id	352127
vulnerability.last_modification	Oct 29, 2020 @ 07:29:25.000
vulnerability.patchable	1
vulnerability.pci_flag	1
vulnerability.published	Oct 29, 2020 @ 07:29:25.000
vulnerability.reference	https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html
vulnerability.report_id	20201125.133227
vulnerability.results	Package Installed Version Required Version e2fsprogs 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2 e2fsprogs-libs 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2 libcom_err 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2 libss 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2 e2fsprogs 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2
vulnerability.scanner.vendor	Powered by Qualys
vulnerability.score.base	6.7
vulnerability.score.environmental	0.0
vulnerability.score.temporal	5.4
vulnerability.score.version	3.0
vulnerability.severity	3
vulnerability.solution	Please refer to Amazon advisory <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509</A> for affected packages and patching details, or update with your package manager. <P>Patch:<BR> Following are links for downloading patches to fix the vulnerabilities: <P> <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on x86_64)</A><P> <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on aarch64)</A><P> <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on src)</A><P> <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on i686)</A>
vulnerability.status	Active
vulnerability.vulnerability_type	VULNERABILITY

Expand

title	JSON Example

Code Block

{
  "_score": 1,
  "_type": "doc",
  "_source": {
    "vulnerability.enumeration": "135",
    "document_size": 3926,
    "@timestamp": "2020-11-25T13:32:27.480Z",
    "vulnerability.published": "2020-10-29T12:29:25.000Z",
    "vulnerability.results": "Package\tInstalled Version\tRequired Version\ne2fsprogs\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2\ne2fsprogs-libs\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2\nlibcom_err\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2\nlibss\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2\ne2fsprogs\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2",
    "tenant_id": "1024",
    "vulnerability.cve": "http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-5188",
    "host.os.name": "Linux",
    "message_size": 0,
    "vulnerability.description": "Amazon Linux Security Advisory for e2fsprogs: ALAS2-2020-1509",
    "vulnerability.scanner.vendor": "Powered by Qualys",
    "_id": 47741608,
    "tags": "[\"customer\",\"flow_source_data_miss\",\"default_parent_id\",\"cached_parent_metadata\"]",
    "armor_metrics.processing_chain": "[\"KVN_V4_collector_i-0908b8b2b53868dc0|2020-11-25T13:32:27Z\",\"KVN_V4_processor_i-0aa172c88f440b715|2020-11-25T13:32:28Z\"]",
    "vulnerability.score.temporal": "5.4",
    "vulnerability.solution": "Please refer to Amazon advisory <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509</A> for affected packages and patching details, or update with your package manager.\n<P>Patch:<BR>\nFollowing are links for downloading patches to fix the vulnerabilities:\n<P> <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on x86_64)</A><P> <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on aarch64)</A><P> <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on src)</A><P> <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on i686)</A>",
    "armor_metrics.input_port": "5445",
    "original_timestamp": "2020-11-25T13:32:27.233Z",
    "logsource.origin": "unknown",
    "vulnerability.score.environmental": "0.0",
    "vulnerability.status": "Active",
    "vulnerability.category": "[\"AMAZON LINUX\",\"PCI\"]",
    "host.ip": "10.0.0.8",
    "vulnerability.discovery": "0",
    "vulnerability.reference": "https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html",
    "vulnerability.report_id": "20201125.133227",
    "received_timestamp": "2020-11-25T13:32:27.480Z",
    "host.os.full": "Amazon Linux 2",
    "vulnerability.pci_flag": "1",
    "vulnerability.patchable": "1",
    "vulnerability.score.version": "3.0",
    "event_uuid": "6d820110-73e5-45c9-945e-10c281fd4cb4",
    "vulnerability.last_modification": "2020-10-29T12:29:25.000Z",
    "vulnerability.diagnosis": "<DIV>\n                            Issue Overview:\n                            <P>An exploitable code execution vulnerability exists in the quota file functionality of E2fsprogs 1.45.3. A specially crafted ext4 partition can cause an out-of-bounds write on the heap, resulting in code execution. An attacker can corrupt a partition to trigger this vulnerability. (<A HREF=\"https://access.redhat.com/security/cve/CVE-2019-5094\" TARGET=\"_blank\">CVE-2019-5094 </A>)</P><P>A code execution vulnerability exists in the directory rehashing functionality of E2fsprogs e2fsck 1.45.4. A specially crafted ext4 directory can cause an out-of-bounds write on the stack, resulting in code execution. An attacker can corrupt a partition to trigger this vulnerability. (<A HREF=\"https://access.redhat.com/security/cve/CVE-2019-5188\" TARGET=\"_blank\">CVE-2019-5188 </A>)</P>\n                        </DIV>\n\n                        ",
    "labels.parent_id": "1",
    "host_asset_id": "75424166",
    "vulnerability.vulnerability_type": "VULNERABILITY",
    "external_id": "4f5b9ab7-8e57-4993-b0fb-440cd44d11e5",
    "vulnerability.score.base": "6.7",
    "armor_metrics.latency.processing": 0.8566529750823975,
    "vulnerability.severity": "3",
    "vulnerability.consequence": "Allows unauthorized disclosure of information; allows unauthorized modification; allows disruption of service.",
    "@version": 1,
    "host.hostname": "ip-10-0-0-8.us-west-2.compute.internal",
    "index_type": "ecs-1.5.0-vulnerability",
    "vulnerability.id": "352127"
  },
  "_id": "47741608",
  "_index": "1_1024_customer"
}

...

Helpful Fields for Searching the Datalake

...

Field

FIELD

Filter By

FILTER BY
hostname	the hostname of the machine on which the event was sent
data_type	the type of the data being searched for, trend-hids in this instance

Adding a Filter

...

To add additional filters, click on the Add Filter Button.

...

Then set the field to one of the helpful fields above, select the operator, put in the value and hit save. The data is now filtered on a specific reportId, rPolicy Policy or other field selected.

Viewing Datalake Aggregations

...

Please refer to Reports for custom aggregations, visualizations and custom reports.

...

Versions Compared

Old Version 5

New Version Current

Key

Understanding the Datalake

Accessing the Datalake

Data Presentation

Helpful Fields for Searching the Datalake

Adding a Filter

Viewing Datalake Aggregations