Versions Compared

Old Version 5

changes.mady.by.user Former user

Saved on

compared with

New Version 6

changes.mady.by.user Former user

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Understanding the Datalake

...

The Armor data lake is a centralized repository for storing Armor collected data. With regards to vulnerabilities, the data lake contains all the data for every report created for an environment and all the historical data from when the reports are run. This can be a lot of data so narrowing down the scope of information is critical to making sense of it all.

Accessing the Datalake

...

Users can access the datalake in two ways:

...

Data consists of documents stored in the datalake. Each document contains all the data related to that particular rule and resource. Below are examples of the table and JSON views:

Expand
titleTable Example
FieldValues

@timestamp

Nov 25, 2020 @ 07:32:27.480
#@version

1

_id

47741608

_index

1_1024_customer

#_score

1

_type

doc

armor_metrics.input_port

5445

armor_metrics.latency.processing

0.857

armor_metrics.processing_chain

["KVN_V4_collector_i-0908b8b2b53868dc0|2020-11-25T13:32:27Z","KVN_V4_processor_i-0aa172c88f440b715|2020-11-25T13:32:28Z"]

document_size

3,926

event_uuid

6d820110-73e5-45c9-945e-10c281fd4cb4

external_id

4f5b9ab7-8e57-4993-b0fb-440cd44d11e5

host.hostname

ip-10-0-0-8.us-west-2.compute.internal

host.ip

10.0.0.8

host.os.full

Amazon Linux 2

host.os.name

Linux

host_asset_id

75424166

index_type

ecs-1.5.0-vulnerability

labels.parent_id

1

logsource.origin

unknown

message_size

0

original_timestamp

Nov 25, 2020 @ 07:32:27.233

received_timestamp

Nov 25, 2020 @ 07:32:27.480

tags

["customer","flow_source_data_miss","default_parent_id","cached_parent_metadata"]

tenant_id

1024

vulnerability.category

["AMAZON LINUX","PCI"]

vulnerability.consequence

Allows unauthorized disclosure of information; allows unauthorized modification; allows disruption of service.

vulnerability.cve

http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-5188

vulnerability.description

Amazon Linux Security Advisory for e2fsprogs: ALAS2-2020-1509

vulnerability.diagnosis&lt;DIV&gt; Issue Overview: <P>An exploitable code execution vulnerability exists in the quota file functionality of E2fsprogs 1.45.3. A specially crafted ext4 partition can cause an out-of-bounds write on the heap, resulting in code execution. An attacker can corrupt a partition to trigger this vulnerability. (<A HREF="https://access.redhat.com/security/cve/CVE-2019-5094" TARGET="_blank">CVE-2019-5094 </A>)</P><P>A code execution vulnerability exists in the directory rehashing functionality of E2fsprogs e2fsck 1.45.4. A specially crafted ext4 directory can cause an out-of-bounds write on the stack, resulting in code execution. An attacker can corrupt a partition to trigger this vulnerability. (<A HREF="https://access.redhat.com/security/cve/CVE-2019-5188" TARGET="_blank">CVE-2019-5188 </A>)</P> &lt;/DIV&gt;
vulnerability.discovery

0

vulnerability.enumeration

135

vulnerability.id

352127

vulnerability.last_modification

Oct 29, 2020 @ 07:29:25.000

vulnerability.patchable

1

vulnerability.pci_flag

1

vulnerability.published

Oct 29, 2020 @ 07:29:25.000

vulnerability.reference

https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html

vulnerability.report_id

20201125.133227

vulnerability.results

Package Installed Version Required Version e2fsprogs 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2 e2fsprogs-libs 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2 libcom_err 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2 libss 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2 e2fsprogs 1.42.9-12.amzn2.0.2.x86_64 1.42.9-19.amzn2

vulnerability.scanner.vendor

Powered by Qualys

vulnerability.score.base

6.7

vulnerability.score.environmental

0.0

vulnerability.score.temporal

5.4

vulnerability.score.version

3.0

vulnerability.severity

3

vulnerability.solutionPlease refer to Amazon advisory <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509</A> for affected packages and patching details, or update with your package manager. <P>Patch:<BR> Following are links for downloading patches to fix the vulnerabilities: <P> <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on x86_64)</A><P> <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on aarch64)</A><P> <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on src)</A><P> <A HREF="https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html" TARGET="_blank">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on i686)</A>
vulnerability.status

Active

vulnerability.vulnerability_type

VULNERABILITY

Expand
titleJSON Example
Code Block
themeMidnight
firstline1
linenumberstrue
{
  "_score": 1,
  "_type": "doc",
  "_source": {
    "vulnerability.enumeration": "135",
    "document_size": 3926,
    "@timestamp": "2020-11-25T13:32:27.480Z",
    "vulnerability.published": "2020-10-29T12:29:25.000Z",
    "vulnerability.results": "Package\tInstalled Version\tRequired Version\ne2fsprogs\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2\ne2fsprogs-libs\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2\nlibcom_err\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2\nlibss\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2\ne2fsprogs\t1.42.9-12.amzn2.0.2.x86_64\t1.42.9-19.amzn2",
    "tenant_id": "1024",
    "vulnerability.cve": "http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-5188",
    "host.os.name": "Linux",
    "message_size": 0,
    "vulnerability.description": "Amazon Linux Security Advisory for e2fsprogs: ALAS2-2020-1509",
    "vulnerability.scanner.vendor": "Powered by Qualys",
    "_id": 47741608,
    "tags": "[\"customer\",\"flow_source_data_miss\",\"default_parent_id\",\"cached_parent_metadata\"]",
    "armor_metrics.processing_chain": "[\"KVN_V4_collector_i-0908b8b2b53868dc0|2020-11-25T13:32:27Z\",\"KVN_V4_processor_i-0aa172c88f440b715|2020-11-25T13:32:28Z\"]",
    "vulnerability.score.temporal": "5.4",
    "vulnerability.solution": "Please refer to Amazon advisory <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509</A> for affected packages and patching details, or update with your package manager.\n<P>Patch:<BR>\nFollowing are links for downloading patches to fix the vulnerabilities:\n<P> <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on x86_64)</A><P> <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on aarch64)</A><P> <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on src)</A><P> <A HREF=\"https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html\" TARGET=\"_blank\">ALAS-2020-1509: Amazon Linux 2 (e2fsprogs (1.42.9-19.amzn2) on i686)</A>",
    "armor_metrics.input_port": "5445",
    "original_timestamp": "2020-11-25T13:32:27.233Z",
    "logsource.origin": "unknown",
    "vulnerability.score.environmental": "0.0",
    "vulnerability.status": "Active",
    "vulnerability.category": "[\"AMAZON LINUX\",\"PCI\"]",
    "host.ip": "10.0.0.8",
    "vulnerability.discovery": "0",
    "vulnerability.reference": "https://alas.aws.amazon.com/AL2/ALAS-2020-1509.html",
    "vulnerability.report_id": "20201125.133227",
    "received_timestamp": "2020-11-25T13:32:27.480Z",
    "host.os.full": "Amazon Linux 2",
    "vulnerability.pci_flag": "1",
    "vulnerability.patchable": "1",
    "vulnerability.score.version": "3.0",
    "event_uuid": "6d820110-73e5-45c9-945e-10c281fd4cb4",
    "vulnerability.last_modification": "2020-10-29T12:29:25.000Z",
    "vulnerability.diagnosis": "<DIV>\n                            Issue Overview:\n                            <P>An exploitable code execution vulnerability exists in the quota file functionality of E2fsprogs 1.45.3. A specially crafted ext4 partition can cause an out-of-bounds write on the heap, resulting in code execution. An attacker can corrupt a partition to trigger this vulnerability. (<A HREF=\"https://access.redhat.com/security/cve/CVE-2019-5094\" TARGET=\"_blank\">CVE-2019-5094 </A>)</P><P>A code execution vulnerability exists in the directory rehashing functionality of E2fsprogs e2fsck 1.45.4. A specially crafted ext4 directory can cause an out-of-bounds write on the stack, resulting in code execution. An attacker can corrupt a partition to trigger this vulnerability. (<A HREF=\"https://access.redhat.com/security/cve/CVE-2019-5188\" TARGET=\"_blank\">CVE-2019-5188 </A>)</P>\n                        </DIV>\n\n                        ",
    "labels.parent_id": "1",
    "host_asset_id": "75424166",
    "vulnerability.vulnerability_type": "VULNERABILITY",
    "external_id": "4f5b9ab7-8e57-4993-b0fb-440cd44d11e5",
    "vulnerability.score.base": "6.7",
    "armor_metrics.latency.processing": 0.8566529750823975,
    "vulnerability.severity": "3",
    "vulnerability.consequence": "Allows unauthorized disclosure of information; allows unauthorized modification; allows disruption of service.",
    "@version": 1,
    "host.hostname": "ip-10-0-0-8.us-west-2.compute.internal",
    "index_type": "ecs-1.5.0-vulnerability",
    "vulnerability.id": "352127"
  },
  "_id": "47741608",
  "_index": "1_1024_customer"
}

...

Helpful Fields for Searching the Datalake

...

Field

Filter

By

vulnerability.category

The type of system or architecture that the vulnerability affects. See https://qualysguard.qualys.com/qwebhelp/fo_portal/knowledgebase/vulnerability_categories.htm#P for a listing of potential categories

vulnerability.severity

1

through

5

host.hostname

the

hostname

of

any

servers

in

your

account

vulnerability.report_id

a

scan

ID

that

can

be

used

to

show

only

the

vulnerabilities

associated

with

a

specific

scan


Adding a Filter

...

To add additional filters, click on the Add Filter Button.

...

Then set the field to one of the helpful fields above, select the operator, put in the value and hit save. The data is now filtered on a specific reportId, rPolicy or other field selected.

Viewing Datalake Aggregations

...

Please refer to Reports for custom aggregations, visualizations and custom reports.

...