Security Logging

Waratek agents use the CEF format when logging security events, wrapped around in a Syslog header envelope. The joint format (Syslog + CEF) is always transmitted regardless of the destination (security log file, Syslog server, Elasticsearch), split by a white space (SP).

[SYSLOG PREFIX] SP [CEF MESSAGE]

CEF Message Format

CEF stands for Common Event Format - this format is used in every security message event created by agents. Waratek CEF messages follow the CEF specification from ArcSight.

CEF:0

PIPE

Device vendor

PIPE

Device Product

PIPE

Device Version

PIPE

Device Event

PIPE

Event Name

PIPE

Event Severity

PIPE

Extensions

Logging Devices

There are 2 types of devices logging security messages:

• ARMR mod: For specific ARMR mods log security events and, at a more granular level, an ARMR rule.

  • An example of a Load Rule event for an ARMR patch rule where the ARMR mod is named CVE-2017-10102:

    CEF:0|ARMR:ARMR|ARMR|2.2|RegistryImpl_Skel|Load Rule|Low|rt=May 05 2020 15:02:23.053 +0100 dvchost=I-dev05 procid=46210 outcome=success

• Waratek agent: The Waratek agent itself acts as a logging device for all other security events not sent by an ARMR mod.

  • An example of new security policies applied.

    CEF:0|ARMR:ARMR|Secure Agent|23.0.0|Engine|Reload Rules|Low|rt=Jul 03 2020 01:44:30.199 +0100 dvchost=I-dev05 procid=1130132 outcome=success msg=New ARMR policy has been applied

  • An example of no security policies applied.

    CEF:0|ARMR:ARMR|Secure Agent|23.0.0|Engine|Reload Rules|Low|rt=Jul 03 2020 01:44:30.172 +0100 dvchost=I-dev05 procid=1130132 msg=Waratek rules file '/tmp/armr.rules' defined in 'com.waratek.rules.local' does not exist or is inaccessible. No security rules were loaded!

  • An example of invalid syntax in security log files.

    CEF:0|ARMR:ARMR|Secure Agent|23.0.0|Engine|Syntax Error|Very-High|rt=Jul 03 2020 01:44:30.172 +0100 dvchost=I-dev05 procid=1130132 msg=duplicate app name detected in rules file. Offending app name: "SQL mod" reason=Error loading ARMR App at line 38 : app("SQL mod"):

Device Vendor

For both the ARMR mod and the Waratek agent the value is ARMR:ARMR.

Device Product

• ARMR mod - the value is ARMR

  CEF:0|ARMR:ARMR|ARMR|2.2|RegistryImpl_Skel|Load Rule|Low|rt=May 05 2020 15:02:23.053 +0100 dvchost=I-dev05 procid=46210 outcome=success

• Waratek agent - the value is Secure Agent.

  CEF:0|ARMR:ARMR|Secure Agent|23.0.0|Engine|Syntax Error|Very-High|rt=Jun 30 2020 19:26:28.703 +0100 dvchost=I-dev05 procid=840923 msg=invalid location specifier, patch must contain only one of [call, callreturn, callsite, entry, error, exit, instruction, line, read, readreturn, readsite, write, writesite, writereturn] reason=Error loading ARMR App at line 20 : app("myapp"):

Device Version

• ARMR mod: The version of the ARMR mod as declared in the requires declaration
• Waratek agent: The version of the Waratek agent (23.0.0)

Device Event

The particular part of the device where the event occurred. In the case of :

• ARMR mod: this is set to the name of the rule that triggered the security event
• Waratek agent: this is simply set to Engine

Event Name

The name of the security event type

• ARMR mod: different event names in the context of ARMR rule lifecycle events and upon execution: Load Rule, Link Rule, Execute Rule, Unload Rule, Unlink Rule.

• Waratek agent:

  • Syntax Error: when an ARMR mod with invalid syntax is not recognized by the agent:

    CEF:0|ARMR:ARMR|Secure Agent|23.0.0|Engine|Syntax Error|Very-High|rt=Jun 30 2020 19:26:28.703 +0100 dvchost=I-dev05 procid=840923 msg=invalid location specifier, patch must contain only one of [call, callreturn, callsite, entry, error, exit, instruction, line, read, readreturn, readsite, write, writesite, writereturn] reason=Error loading ARMR App at line 20 : app("myapp"): n, line, read, readreturn, readsite, write, writesite, writereturn] reason=Error loading ARMR App at line 20 : app("myapp"):

  • Reload Rules: events related to loading/unloading rules in general, but not specifically associated with any ARMR mod:

    CEF:0|ARMR:ARMR|Secure Agent|23.0.0|Engine|Reload Rules|Low|rt=Jun 30 2020 19:26:28.703 +0100 dvchost=I-dev05 procid=1356675 outcome=success msg=No ARMR policy is in effect

Event Severity

One of the five severity values defined by CEF: Unknown, Low, Medium, High, Very-High.

Extensions

An extension is a key-value pair, where these are separated by whitespace. To avoid ambiguity between different extensions and for security reasons some characters need to be escaped (=, \, \n, \r), as mentioned in the CEF specification.

The order of extensions is not guaranteed and these can appear shuffled in a future release. The following table lists all supported extensions and their usage scope.

Extension

Description

rt

Time of the triggered event. It has the following format MM DD YYYY HH:mm:ss.SSS zzzz, as follows :

• MM

  • English language abbreviation for the month of the year with the first character in uppercase and the other two characters in lowercase (Jan, Feb, Mar, Jun)

• DD

  • day of the month, e.g., (03, 12, 30, …)

• YYYY

  • full year, e.g., 1975

• HH

  • hour represented in a 24-hour format

• mm

  • minutes from 00 to 59

• ss

  • seconds from 00 to 59

• SSS

  • milliseconds from 000 to 999

• zzzz

  • timezone comprised of a “+“/”-” sign and other four digits for the time difference to GMT:

  • first 2 digits indicate the hour difference

  • last 2 digits indicate the minute difference

dvchost

The hostname of the machine the Java process is running on. It is the hostname as seen from the local host configuration or the FQDN depending on the agent configuration

procid

The process ID is taken from the Operating System where the process is running

nodeid

The ID of the Java process relative to the Portal. This is the id assigned by the Portal to the agent upon registration

outcome

Value is either success or failure

act

The action as defined in the ARMR rule. Examples of actions are: protect, detect or allow

msg

User-defined message as declared in the ARMR rule

When defining an action in the ARMR rule it is always possible to specify the message attribute. If provided, this will control the value of the msg extension in the security event. It is noted that:

• if the message attribute is omitted entirely then no security event will be generated;
• if the message attribute is defined as an empty string ("") then a default message will be included in the security event msg extension;
• if the message attribute is defined and is not empty then the provided message will be included in the security event msg extension.

If unable to retrieve values for any of the extensions dvchost, procid or nodeid then these extensions are omitted from the CEF message.

The rt extension is mandatory in every CEF message logged.

Syslog Format

Syslog format wraps the actual log message, just like a network frame/datagram and it’s widely used in enterprise IT. Unix systems have adopted this format for their internal logging events when they are logging to locals files.

Syslog Header Format

PRI

SP

VERSION

SP

TIMESTAMP

SP

HOSTNAME

SP

APP-NAME

SP

PROCID

SP

MSGID

SP

STRUCTURED-DATA

SP

MSG

SP

Space Character

PRI

The PRI part must have three digits and will be bound with angle brackets as the first and last characters. The number contained within these angle brackets is known as the ‘Priority’ value and is computed by combining both the Facility and Severity.

Facility

The Waratek agent only uses a Facility of USER for its security logging.

Numerical Code

Facility

1

user-level messages

Severity

The following Syslog severities are used for the Waratek’s agent security logging. They map directly to the CEF message severity if the message payload is indeed in the CEF format. For other message types, the Syslog severity of Notice is chosen.

Syslog Numerical Code

Syslog Severity

CEF Severity

1

Alert: action must be taken immediately

Very-High

2

Critical: critical conditions

High

4

Warning: warning conditions

Medium

5

Notice: normal but significant condition

Unknown

6

Informational: informational messages

Low

The Priority value is calculated by first multiplying the Facility number by 8 and then adding the numerical value of the Severity. For example, a user-level message (Facility=1) with a Severity of Notice (Severity=5) would have a Priority value of 13.

VERSION

The VERSION field denotes the version of the Syslog protocol specification. VERSION equals '1' indicating that the standard documented by RFC 5424 is being followed.

TIMESTAMP

The TIMESTAMP is of the following format :

`DATE-FULL-YEAR`-`DATE-MONTH`-`DATE-MDAY`T`TIME-HOUR`:`TIME-MINUTE`: `TIME-SECOND`.`TIME-MILLISECOND`[Z][+/-`UTC-TIME-HOUR`: `UTC-TIME-MINUTE`]

The TIMESTAMP value must follow these restrictions:

• The "T" and "Z" characters in this syntax must be upper case.

• Usage of the "T" character is required.

• Leap seconds must not be used.

• DATA-FULL-YEAR

  • 4 digits.

• DATE-MONTH

  • 2 digits; 01-12

• DATE-MDAY

  • 2 digits; 01-28, 01-29, 01-30, 01-31 based on month and year.

• TIME-HOUR

  • 2 digits; 00-23

• TIME-MINUTE

  • 2 digits; 00-59

• TIME-SECOND

  • 2 digits; 00-59

• TIME-MILLISECOND

  • 3 digits.

• UTC-TIME-HOUR

  • timezone difference from UTC; 2 digits; 00-23

• UTC-TIME-MINUTE

  • timezone difference from UTC; 2 digits; 00-59

Examples

Example 1

1985-04-12T23:20:50.52Z

This represents 20 minutes and 50.52 seconds after the 23rd hour of 12 April 1985 in UTC.

Example 2

1985-04-12T19:20:50.52-04:00

This represents the same time as in example 1 but expressed in US Eastern Standard Time (observing daylight savings time).

Example 3

2003-10-11T22:14:15.003Z

This represents 11 October 2003 at 10:14: 15 pm, 3 milliseconds into the next second. The timestamp is in UTC. The timestamp provides millisecond resolution. The creator may have actually had a better resolution, but providing just three digits for the fractional part of a second does not tell us.

Example 4

2003-08-24T05:14:15.003-07:00

This represents 24 August 2003 at 05:14: 15 am, 3 microseconds into the next second. The timestamp indicates that its local time is -7 hours from UTC. This timestamp might be created in the US Pacific time zone during daylight savings time.

Example 5: An Invalid TIMESTAMP

2003-08-24T05:14:15.000000003-07:00

This example is nearly the same as Example 4, but the second is followed by nanoseconds which is invalidate.

HOSTNAME

The HOSTNAME field identifies the machine that originally sent the Syslog message or - if not available.

APP-NAME

The APP-NAME field should identify the device or application that originated the message. It is intended for filtering messages on a relay or collector.

In the Waratek agent, the APP-NAME value is simply java.

PROCID

The PROCID field is often used to map log entries to the process from where they were logged. The Waratek agent prints the corresponding process ID from the Operating System. If the process ID cannot be identified, a random one is generated and used for as long as the process is alive.

MSGID

The hyphen (-) is used for all Waratek Agents.

STRUCTURED-DATA

The hyphen (-) is used for all Waratek Agents.

MSG

The MSG field is, in fact, the CEF message. Examples:

<13>1 2020-05-11T19:02:53.188+01:00 I-dev05 java 394700 - - CEF:0|ARMR:ARMR|Secure Agent|23.0.0|Engine|Reload Rules|Low|rt=May 11 2020 19:02:53.188 +0100 dvchost=I-dev05 procid=394700 outcome=success msg=No ARMR policy is in effect

<9>1 2020-05-11T14:39:23.965+01:00 I-dev05 java 433109 - - CEF:0|ARMR:ARMR|ARMR|2.2|RegistryImpl_Skel|Execute Rule|Very-High|msg=java.lang.Exception outcome=failure procid=433109 dvchost=I-dev05 rt=May 11 2020 14:39:23.965 +0100

<14>1 2020-07-07T16:39:52.540+01:00 I-dev05 java 1398713 - - CEF:0|ARMR:ARMR|ARMR|2.2|block file.txt|Load Rule|Low|rt=Jul 07 2020 16:39:52.538 +0100 dvchost=I-dev05 procid=1398713 nodeid=1 outcome=success <14>1 2020-07-07T16:39:52.558+01:00 I-dev05 java 1398713 - - CEF:0|ARMR:ARMR|ARMR|2.2|block file.txt|Link Rule|Low|rt=Jul 07 2020 16:39:52.557 +0100 dvchost=I-dev05 procid=1398713 nodeid=1 outcome=success