Log Search

Getting Started


Visualization tools and dashboards such as those in the Armor Management Portal use values of data to help security teams visualize what is occurring in each environment. Users can then process that data to develop reports and graphs, making it easier to share with others. Once data is gathered, users can then take advantage of the virtualization and reporting capabilities with just a few clicks.

With Log Search and Data Visualization capabilities, users can build customer dashboards within the Armor Management Portal. With just a few clicks, users can visualize log alerts and incident information within any environment. For example, teams may want to see where a certain malware has surfaced across multiple environments. Searches can show patterns and include artifacts for analysis. Searches can also be saved and are designed to return results based on a current time range.

Please make sure to review ChaosSearch's documentation on Log Search data and visualization.



A list of Standard Visualizations has been prepared for users, including steps to configure and examples of each visualization.

For more information on the Log Search data and visualization tool, please see Kibana Guide [7.9] | Elastic

 

Exporting Data from Log Search


Users can export small quantities of documents (logs, events, vulnerabilities, security incidents, cspm alerts, edr alerts) via a Data Table | Kibana Guide [6.8] | Elastic visualization within Log Search.

  1. In the Armor Management Portal (AMP), in the left-side navigation, click Log Search.

  2. In Log Search, click the Visualize tab.

  3. Click on the Create New Visualization button.

  4. In the New Visualization popup window, click on Data Table.

  5. In the New Data Table / Choose a source popup window, select the appropriate source for the query.

  6. Customize the visualization as needed.




    Armor recommends that users add a bucket in the Buckets dropdown and configure its settings to match the screenshot above.

  7. When finished, click the Blue Triangle just above metrics to Apply Changes.

    1. Users can use the +Add filter link (see screenshot) to limit the results that are returned to contain only the events to be exported.

    2. Query date functionality works as it does in the 'Discover' page.

    3. Export links i(see screenshot) can be used to export the results in CSV format via browser download.

While filtering for the index-pattern, the behavior of the search box can be confusing. The Search Box will automatically append a wildcard to the end of a filter, but not to the beginning. To ensure that a search will return data users should only filter from Page 1 and prepend a wildcard character to the search.

e.g. *_5797 will search for *_5797* and return 5441_5797_customer as long as users are on Page 1

Never filter from any pages other than Page 1.

 

Log Search Field Glossary


Log Search allows for the use of both scripted and custom field names. For a complete list of all scripted field names, please see ECS Field Reference | Elastic Common Schema (ECS) Reference [8.11] | Elastic .

The list below contains custom fields created by Armor. This list is constantly growing, so if you are unable to find what you're looking for, please reach out to your Customer Success Manager or Support.

Name

Description

Name

Description

@timestamp

Represents the time extracted from the original event.

@version

The document's version.

_id

The document’s ID.

_index

The index to which the document belongs.

_score

The original JSON representing the body of the document.

_type

The document’s mapping type.

armor_metadata

Contains information about the Armor Account. 

beat.hostname

Alias to agent.hostname.

beat.name

Alias to host.name.

beat.version

Elastic Filebeat version.

data_type

The data source or type of Armor data.

destination.address

The request's destination IP address.

document_size

The document’s size.

dst_geo.city_name

 Destination Geo IP, or user-supplied destination Geo city name. 

dst_geo.continent_code

 Destination Geo IP, or user-supplied destination Geo continent code.

dst_geo.country_code2

 Destination Geo IP, or user-supplied destination Geo country code 2.

dst_geo.country_code3

 Destination Geo IP, or user-supplied destination Geo country code 3.

dst_geo.country_name

 Destination Geo IP, or user-supplied destination Geo country name.

dst_geo.dma_code

 Destination Geo IP, or user-supplied destination Geo dma code.

dst_geo.latitude

 Destination Geo IP, or user-supplied destination Geo latitude.

dst_geo.longitude

 Destination Geo IP, or user-supplied destination Geo longitude.

dst_geo.postal_code

 Destination Geo IP, or user-supplied destination Geo postal code.

dst_geo.region_code

 Destination Geo IP, or user-supplied destination Geo region code.

dst_geo.region_name

 Destination Geo IP, or user-supplied destination Geo region name.

dst_geo.timezone

 Destination Geo IP, or user-supplied destination Geo timezone.

dst_ip

dst_ip IP addres of the destination. Can be one or more IPv4 or IPv6 addresses.

This field is available in armor ingestion supported logs for;

AWS VPC Flow Logs
Web Application Firewall (WAF)
Brocade - Virtual Traffic Manager
Imperva - SecureSphere
Juniper

dst_port

Port of the destination.

This field is available in armor ingestion supported logs for;

AWS VPC Flow Logs
Web Application Firewall (WAF)
Brocade - Virtual Traffic Manager
Imperva - SecureSphere
Juniper
event.ReportId

event.ReportId

The Report ID of the CSPM report. 

event_timestamp

Represents the date when the event started or when the activity was first observed.

event_uuid

The event's universally unique identifier (UUID).

events.count

The total count of events.

events.rate_15m

The per-second event rate in a 15-minute sliding window.

events.rate_1m

The per-second event rate in a 1-minute sliding window.

events.rate_5m

The per-second event rate in a 5-minute sliding window.

external_id

A unique id assigned to the armor agent installed on a customer host machine.

host.name

The event source's hostname.

http.request.body.bytes

The size of the request body sent to the server in bytes.

http.request.method

The method of the HTTP request (GET, POST, PUT).

http.request.referrer

The referrer for the logged HTTP request.

http.response.body.bytes

The size of the server's response in bytes.

http.response.status_code

The HTTP response status code.

http.version

The version of the HTTP protocol used in the request.

iis.access.server_name

The name of the server on which the log file entry was generated.

iis.access.site_name

The site name and instance number.

iis.access.sub_status

The substatus code of the HTTP request.

iis.access.win32_status

The Windows status code returned by IIS.

index_type

Differentiates the type of Trend data (e.g AV/FIM/IDS).

input.type

The document's input type.

keywords

 

labels.parent_id

Contains the customer's parent Armor Account Number. 

log.file.path

The path to the log file.

logsource.hostname

The hostname of the logsource.

logsource.origin

The origin of the logsource.

logsource.relay_port

The relay port of the logsource.

logsource.timestamp

The timestamp of the logsource.

message

Raw test message of entire event.

message_size

The size of the message.

nginx.access.remote_ip_list

An array of remote IP addresses relevant to the request; can include IP address from HTTP headers.

original_timestamp

The original timestamp of the message.

parentId

The document's parent account identifier.

parsed.sshd.event

OpenSSH server process event.

parsed.sshd.message

OpenSSH server process message.

parsed.sshd.message_code

OpenSSH server process message code.

parsed.sudo.command

The command executed using sudo.

parsed.sudo.error

The resulting error from command using sudo. 

parsed.sudo.pwd

The print working directory (pwd) where command using sudo was executed. 

parsed.sudo.tty

The name of the device file used when command using sudo was executed.

parsed.sudo.username

The username of the sudoer.

parsed.trendmicro.action

The action performed by the Anti-Malware engine or detected by the integrity rule. Possible values are: Deny Access, Quarantine, Delete, Pass, Clean, Terminate, and Unspecified. Can contain: created, updated, deleted or renamed.

parsed.trendmicro.category

Event category.

parsed.trendmicro.cn1

The agent computer's internal unique identifier.

parsed.trendmicro.cn1_label

The name label for the field cn1.

parsed.trendmicro.cn2

The size of the quarantine file. This extension is included only when the "direct forward" from agent /appliance is selected.

parsed.trendmicro.cn2_label

The name label for the field cn2.

parsed.trendmicro.cn3

Position within packet of data that triggered the event.

parsed.trendmicro.cn3_label

The name label for the field cn3.

parsed.trendmicro.count

The number of times this event was sequentially repeated.

parsed.trendmicro.cs1

(Optional) A note field which can contain a short binary or text note associated with the payload file. If the value of the note field is all printable ASCII characters, it will be logged as text with spaces converted to underscores. If it contains binary data, it will be logged using Base-64 encoding.

parsed.trendmicro.cs1_label

The name label for the field cs1.

parsed.trendmicro.cs2

(For the TCP protocol only) The raw TCP flag byte followed by the URG, ACK, PSH, RST, SYN and FIN fields may be present if the TCP header was set.

parsed.trendmicro.cs2_label

The name label for the field cs2.

parsed.trendmicro.cs3

 

parsed.trendmicro.cs3_label

The name label for the field cs3.

parsed.trendmicro.cs5

Position within stream of data that triggered the event.

parsed.trendmicro.cs5_label

The name label for the field cs5.

parsed.trendmicro.cs6

A combined value that includes the sum of the flag values:

1 - Data truncated - Data could not be logged.
2 - Log Overflow - Log overflowed after this log.
4 - Suppressed - Logs threshold suppressed after this log.
8 - Have Data - Contains packet data
16 - Reference Data - References previously logged data.

parsed.trendmicro.cs6_label

The name label for the field cs6.

parsed.trendmicro.description

Event description.

parsed.trendmicro.ds_frame_type

Connection ethernet frame type.

parsed.trendmicro.ds_tenant

Deep Security tenant name.

parsed.trendmicro.ds_tenant_id

Deep Security tenant ID number.

parsed.trendmicro.dst_ip

IP address of the destination computer.

parsed.trendmicro.dst_mac

Destination MAC Address.

parsed.trendmicro.dst_port

(For TCP and UDP protocol only) Port number of the destination computer's connection or session.

parsed.trendmicro.dvchost

The hostname or IPv6 address for cn1.

Does not appear if the source is an IPv4 address. (Uses dvc field instead.)

parsed.trendmicro.file_path

The location of the malware file or integrity rule target entity. May contain a file or directory path, registry key, etc.

parsed.trendmicro.in

(For inbound connections only) Number of inbound bytes read.

parsed.trendmicro.message

The type of scan. Possible values are: Realtime, Scheduled, and Manual.

parsed.trendmicro.name

Event name.

parsed.trendmicro.out

(For outbound connections only) Number of outbound bytes read.

parsed.trendmicro.proto

Name of the connection transport protocol used.

parsed.trendmicro.severity

The severity of the event. 1 is the least severe; 10 is the most severe.

parsed.trendmicro.src_ip

IP address of the source computer.

parsed.trendmicro.src_mac

Source computer network interface MAC address.

parsed.trendmicro.src_port

(For TCP and UDP protocol only) Source computer connection port.

parsed.trendmicro.suser

Deep Security Manager administrator's account.

parsed.trendmicro.target

The subject of the event. It can be the administrator account logged into Deep Security Manager, or a computer.

parsed.trendmicro.trend_micro_ds_file_sha1

The SHA1 hash of the file

parsed.trendmicro.trend_micro_ds_malware_target

The file, process, or registry key (if any) that the malware was trying to affect. If the malware was trying to affect more than one, this field will contain the value "Multiple."

Only suspicious activity monitoring and unauthorized change monitoring have values for this field.

parsed.trendmicro.trend_micro_ds_malware_target_type

The type of system resource that this malware was trying to affect, such as the file system, a process, or Windows registry.

Only suspicious activity monitoring and unauthorized change monitoring have values for this field.

parsed.trendmicro.username

(If parse-able username exists) The name of the target user initiated the log entry.

prospector.type

The type of Filebeat prospector used.

proto

Name of the connection transport protocol used.

received_timestamp

The timestamp of when Elasticsearch received document.

source

The document's source.

source.address

The event source's IP address.

src_geo.city_name

 Source Geo IP, or user-supplied source Geo city name. 

src_geo.continent_code

 Source Geo IP, or user-supplied source Geo continent code.

src_geo.country_code2

 Source Geo IP, or user-supplied source Geo country code 2. 

src_geo.country_code3

 Source Geo IP, or user-supplied source Geo country code 3. 

src_geo.country_name

 Source Geo IP, or user-supplied source Geo country name. 

src_geo.dma_code

 Source Geo IP, or user-supplied source Geo dma code. 

src_geo.latitude

 Source Geo IP, or user-supplied source Geo latitude. 

src_geo.longitude

 Source Geo IP, or user-supplied source Geo longitude. 

src_geo.postal_code

Source's postal code.

src_geo.region_code

Source's region code.

src_geo.region_name

Source's region name.

src_geo.timezone

 Source Geo IP, or user-supplied source Geo timezone.

src_ip

Source's IP address.

src_port

Source's port.

syslog_facility

Syslog facility levels.

syslog_facility_code

Syslog facility level code.

syslog_pid

Syslog process identification number (pid).

syslog_program

Syslog program name.

syslog_severity

Syslog severity level.

syslog_severity_code

Syslog severity level code.

syslog_timestamp

Syslog timestamp.

syslog5424_pri

The name of field which passes in the extracted PRI part of the syslog message.

tenant_id

The document's tenant ID.

trendmicro.dsm.syslog_hostname

The syslog hostname used to forward logs to the Trend Micro DSM.

trendmicro.dsm.syslog_message

The syslog message sent to the Trend Micro DSM.

type

The document's type

url.original

The unmodified original url as recorded in the event source.

url.path

The path of the request.

url.query

The request's query string.

user.name

The user making the request if the request is authenticated.

user_agent.device.name

The name of the device recorded in the user agent string.

user_agent.name

The name of the client's user agent.

user_agent.original

The unparsed user_agent string of the request.

user_agent.os.name

The operating system from which the client sent the request.

username

The document's username.

vulnerability.cve

Contains the URL related to a vulnerability and provides more information for the customer to read.

vulnerability.published

Contains the year that a vulnerability was first announced. 

vulnerability.solution

Contains the solution for a given vulnerability. 

vulnerability.vulnerability_type

Contains information about the type of vulnerability. 

wineventlog.activity_id

The globally unique identifier (GUID) for the activity in process for which the event is involved.

wineventlog.computer_name

Gets the name of the computer on which this event was logged.

wineventlog.event_data.target_user_name

The TargetUserName of the Windows user event logged. 

wineventlog.event_id

The identifier for this event.

wineventlog.level

The level of the event. The level signifies the severity of the event.

wineventlog.log_name

The name of the event log where this event is logged.

wineventlog.opcode

The opcode of the event. The opcode defines a numeric value that identifies the activity or a point within an activity that the application was performing when it raised the event.

wineventlog.process_id

The process identifier for the event provider that logged this event.

wineventlog.provider_guid

The globally unique identifier (GUID) of the event provider that published this event.

wineventlog.record_number

The event record identifier of the event in the log.

wineventlog.source_name

The source of the event in the log.

wineventlog.task

The display name of the task for the event.

wineventlog.thread_id

The thread identifier for the thread that the event provider is running in.

wineventlog.user.domain

The domain of the user whose context is used to publish the event.

wineventlog.user.identifier

The security descriptor of the user whose context is used to publish the event.

wineventlog.user.name

The name of the user whose context is used to publish the event.

wineventlog.user.type

The type of user whose context is used to publish the event.

wineventlog.version

The version number for the event.