...
...
...
...
...
...
...
...
...
id | 1914987854 |
---|
...
background-color | $lightGrayColor |
---|---|
id | 1914987831 |
Table of Contents | ||||
---|---|---|---|---|
|
...
...
id | 1914987844 |
---|
Understanding the Datalake
...
The Armor data lake is a centralized repository for storing Armor collected data. With regards to vulnerabilities, the data lake contains all the data for every report created for an environment and all the historical data from when the reports are run. This can be a lot of data so narrowing down the scope of information is critical to making sense of it all.
Accessing the Datalake
...
Users can access the datalake in two ways:
Expand | ||
---|---|---|
| ||
|
Expand | ||
---|---|---|
| ||
|
Data Presentation
...
Data consists of documents stored in the datalake. Each document contains all the data related to that particular rule and resource. Below are examples of the table and JSON views:
Expand | ||
---|---|---|
| ||
Field | Values | |
@timestamp | Nov 25, 2020 @ 07:32:27.480 | |
#@version | 1 | |
_id | 47741608 | _index | 1_1024_customer
#_score | 1 | |
_type | doc | |
armor_metrics.input_port | 5445 | |
armor_metrics.latency.processing | 0.857 | |
armor_metrics.processing_chain | ["KVN_V4_collector_i-0908b8b2b53868dc0|2020-11-25T13:32:27Z","KVN_V4_processor_i-0aa172c88f440b715|2020-11-25T13:32:28Z"] | |
document_size | 3,926 | |
event_uuid | 6d820110-73e5-45c9-945e-10c281fd4cb4 | |
external_id | 4f5b9ab7-8e57-4993-b0fb-440cd44d11e5 | |
host.hostname | ip-10-0-0-8.us-west-2.compute.internal | |
host.ip | 10.0.0.8 | |
host.os.full | Amazon Linux 2 | |
host.os.name | Linux | |
host_asset_id | 75424166 | |
index_type | ecs-1.5.0-vulnerability | |
labels.parent_id | 1 | |
logsource.origin | unknown | |
message_size | 0 | |
original_timestamp | Nov 25, 2020 @ 07:32:27.233 | |
received_timestamp | Nov 25, 2020 @ 07:32:27.480 | |
tags | ["customer","flow_source_data_miss","default_parent_id","cached_parent_metadata"] | |
tenant_id | 1024 | |
vulnerability.category | ["AMAZON LINUX","PCI"] | |
vulnerability.consequence | Allows unauthorized disclosure of information; allows unauthorized modification; allows disruption of service. | |
vulnerability.cve | ||
vulnerability.description | Amazon Linux Security Advisory for e2fsprogs: ALAS2-2020-1509 | |
vulnerability.diagnosis | <DIV> Issue Overview: <P>An exploitable code execution vulnerability exists in the quota file functionality of E2fsprogs 1.45.3. A specially crafted ext4 partition can cause an out-of-bounds write on the heap, resulting in code execution. An attacker can corrupt a partition to trigger this vulnerability. (<A HREF="https://access.redhat.com/security/cve/CVE-2019-5094" TARGET="_blank">CVE-2019-5094 </A>)</P><P>A code execution vulnerability exists in the directory rehashing functionality of E2fsprogs e2fsck 1.45.4. A specially crafted ext4 directory can cause an out-of-bounds write on the stack, resulting in code execution. An attacker can corrupt a partition to trigger this vulnerability. (<A HREF="https://access.redhat.com/security/cve/CVE-2019-5188" TARGET="_blank">CVE-2019-5188 </A>)</P> </DIV> | |
vulnerability.discovery | 0 | |
vulnerability.enumeration | 135 | |
vulnerability.id | 352127 | |
vulnerability.last_modification | Oct 29, 2020 @ 07:29:25.000 | |
vulnerability.patchable | 1 | |
vulnerability.pci_flag | 1 | |
vulnerability.published | Oct 29, 2020 @ 07:29:25.000 | |
vulnerability.reference | ||
vulnerability.report_id | 20201125.133227 | |
vulnerability.results | Package Installed Version Required Version e2fsprogs 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2 e2fsprogs-libs 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2 libcom_err 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2 libss 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2 e2fsprogs 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2 | |
vulnerability.scanner.vendor | Powered by Qualys | |
vulnerability.score.base | 6.7 | |
vulnerability.score.environmental | 0.0 | |
vulnerability.score.temporal | 5.4 | |
vulnerability.score.version | 3.0 | |
vulnerability.severity | 3 | |
vulnerability.solution | Please refer to Amazon advisory <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509</A> for affected packages and patching details, or update with your package manager. <P>Patch:<BR> Following are links for downloading patches to fix the vulnerabilities: <P> <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on x86_64)</A><P> <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on aarch64)</A><P> <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on src)</A><P> <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on i686)</A> | |
vulnerability.status | Active | |
vulnerability.vulnerability_type | VULNERABILITY | |
Expand | |||
---|---|---|---|
| |||
Code Block | |||
theme | Midnight | ||
firstline | 1 | linenumbers | true|
{
"_score": 1,
"_type": "doc",
"_source": {
"vulnerability.enumeration": "135",
"document_size": 3926,
"@timestamp": "2020-11-25T13:32:27.480Z",
"vulnerability.published": "2020-10-29T12:29:25.000Z",
"vulnerability.results": "Package\tInstalled Version\tRequired Version\ne2fsprogs\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2\ne2fsprogs-libs\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2\nlibcom_err\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2\nlibss\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2\ne2fsprogs\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2",
"tenant_id": "1024",
"vulnerability.cve": "http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-5188",
"host.os.name": "Linux",
"message_size": 0,
"vulnerability.description": "Amazon Linux Security Advisory for e2fsprogs: ALAS2-2020-1509",
"vulnerability.scanner.vendor": "Powered by Qualys",
"_id": 47741608,
"tags": "[\"customer\",\"flow_source_data_miss\",\"default_parent_id\",\"cached_parent_metadata\"]",
"armor_metrics.processing_chain": "[\"KVN_V4_collector_i-0908b8b2b53868dc0|2020-11-25T13:32:27Z\",\"KVN_V4_processor_i-0aa172c88f440b715|2020-11-25T13:32:28Z\"]",
"vulnerability.score.temporal": "5.4",
"vulnerability.solution": "Please refer to Amazon advisory <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509</A> for affected packages and patching details, or update with your package manager.\n<P>Patch:<BR>\nFollowing are links for downloading patches to fix the vulnerabilities:\n<P> <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on x86_64)</A><P> <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on aarch64)</A><P> <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on src)</A><P> <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on i686)</A>",
"armor_metrics.input_port": "5445",
"original_timestamp": "2020-11-25T13:32:27.233Z",
"logsource.origin": "unknown",
"vulnerability.score.environmental": "0.0",
"vulnerability.status": "Active",
"vulnerability.category": "[\"AMAZON LINUX\",\"PCI\"]",
"host.ip": "10.0.0.8",
"vulnerability.discovery": "0",
"vulnerability.reference": "https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html",
"vulnerability.report_id": "20201125.133227",
"received_timestamp": "2020-11-25T13:32:27.480Z",
"host.os.full": "Amazon Linux 2",
"vulnerability.pci_flag": "1",
"vulnerability.patchable": "1",
"vulnerability.score.version": "3.0",
"event_uuid": "6d820110-73e5-45c9-945e-10c281fd4cb4",
"vulnerability.last_modification": "2020-10-29T12:29:25.000Z",
"vulnerability.diagnosis": "<DIV>\n Issue Overview:\n <P>An exploitable code execution vulnerability exists in the quota file functionality of E2fsprogs 1.45.3. A specially crafted ext4 partition can cause an out-of-bounds write on the heap, resulting in code execution. An attacker can corrupt a partition to trigger this vulnerability. (<A HREF=\"https://access.redhat.com/security/cve/CVE-2019-5094\" TARGET=\"_blank\">CVE-2019-5094 </A>)</P><P>A code execution vulnerability exists in the directory rehashing functionality of E2fsprogs e2fsck 1.45.4. A specially crafted ext4 directory can cause an out-of-bounds write on the stack, resulting in code execution. An attacker can corrupt a partition to trigger this vulnerability. (<A HREF=\"https://access.redhat.com/security/cve/CVE-2019-5188\" TARGET=\"_blank\">CVE-2019-5188 </A>)</P>\n </DIV>\n\n ",
"labels.parent_id": "1",
"host_asset_id": "75424166",
"vulnerability.vulnerability_type": "VULNERABILITY",
"external_id": "4f5b9ab7-8e57-4993-b0fb-440cd44d11e5",
"vulnerability.score.base": "6.7",
"armor_metrics.latency.processing": 0.8566529750823975,
"vulnerability.severity": "3",
"vulnerability.consequence": "Allows unauthorized disclosure of information; allows unauthorized modification; allows disruption of service.",
"@version": 1,
"host.hostname": "ip-10-0-0-8.us-west-2.compute.internal",
"index_type": "ecs-1.5.0-vulnerability",
"vulnerability.id": "352127"
},
"_id": "47741608",
"_index": "1_1024_customer"
} | |||
The schema for these documents is based on Elastic Common Schema, please refer to the below links for the details and explanation of the fields:
Vulnerability schema - https://www.elastic.co/guide/en/ecs/1.5/ecs-vulnerability.html
Custom Fields:
src_ip - the event's source IP
src_port - the event's source port
parsed.trendmicro.name - the name of the event signature
dst_ip - the event's destination IP
dst_port - the event's destination port
parsed.trendmicro.severity - the severity of the event
Helpful Fields for Searching the Datalake
...
Field | Filter By |
---|---|
hostname | the hostname of the machine on which the event was sent |
data_type | the type of the data being searched for, trend-hids in this instance |
Adding a Filter
...
To add additional filters, click on the Add Filter Button.
...
Then set the field to one of the helpful fields above, select the operator, put in the value and hit save. The data is now filtered on a specific reportId, rPolicy or other field selected.
Viewing Datalake Aggregations
...
Please refer to Reports for custom aggregations, visualizations and custom reports.