Arcsight руководство администратора

About the connector

ArcSight Enterprise Security Manager (ESM) is a threat detection, analysis, triage, and compliance management SIEM platform.

This document provides information about the Micro Focus ArcSight connector, which facilitates automated interactions, with an ArcSight ESM server using FortiSOAR™ playbooks. Add the Micro Focus ArcSight connector as a step in FortiSOAR™ playbooks and perform automated operations, such as annotating events, running a report based on a report ID, and uploading an ArcSight report file as an attachment in FortiSOAR™.

You can configure ArcSight ESM and FortiSOAR™ so that FortiSOAR™ ingests correlated events from ArcSight ESM and converts them into an alert in FortiSOAR™. For more information, see the ArcSight ESM and FortiSOAR™ integration section.

Version information

Connector Version: 3.0.0

FortiSOAR™ Version Tested on: 6.4.1-2133

Micro Focus ArcSight Version Tested on: 7.0

Authored By: Fortinet

Certified: Yes

Release Notes for version 3.0.0

Following enhancements have been made to the Micro Focus ArcSight connector in version 3.0.0:

  • Added the «Delete Active List Entries» operation and playbook.
  • Updated the «Get Active List Entries» operation by making the ‘Active List ID’ parameter optional.
  • Removed the ‘Do Not Fail Connector Function On API Error’ parameter from the connector’s «Configuration» page.
  • Enhanced the data ingestion playbooks.
  • Updated the «Configuration Parameters» by moving the «Active List ID» configuration parameter out of the Enable Pull ArcSight Events Service onchange parameters. 

Installing the connector

From version 5.0.0 onwards, use the Connector Store to install the connector. For the detailed procedure to install a connector, click here.
You can also use the yum command to install connectors. Connectors provided by FortiSOAR™ are delivered using a FortiSOAR™ repository. Therefore, you must set up your FortiSOAR™ repository and run the yum command as a root user to install connectors:

yum install cyops-connector-arcsight

Important: Upgrading the connector from 2.x to 3.x is not backward compatible because explicit changes are needed in the existing playbooks to delete the ‘Active List Entries» after Alerts are created in FortiSOAR™.

Prerequisites to configuring the connector

  • You must have the IP Address or FQDN of ArcSight ESM server and credentials to access the server.
  • To access the FortiSOAR™ UI, ensure that port 443 is open through the firewall for the FortiSOAR™ instance.

Configuring the connector

For the procedure to configure a connector, click here.

Configuration parameters

In FortiSOAR™, on the Connectors page, click the Micro Focus ArcSight connector row (if you are in the Grid view on the Connectors page) and in the Configurations tab enter the required configuration details: 

Parameter Description
Server URL IP Address or FQDN of the ArcSight ESM server to which you will connect and perform automated operations.
ESM Port REST API port of the ArcSight ESM server.
Defaults to 8443.
Username Username to access the ArcSight ESM server.
Password Password to access the ArcSight ESM server.
Active List ID Resource ID of the Active List for which you want to retrieve events from ArcSight ESM.
Verify SSL Specifies whether the SSL certificate for the server is to be verified or not.
By default, this option is set as True.
Enable Pull ArcSight Events Service If you select this option, i.e., set this option as True (default), then ArcSight events will be pulled.
If you select this option, then you must specify the following parameters:

  • Playbook Trigger: The string used to trigger the playbook.
    Note: The playbook authentication method should be set as HMAC.
  • Reader Port: Port used for communication between FortiSOAR™ and the ArcSight Active List Reader. 10012 is the default port number. You can specify any unused port number if the default port is unavailable. You can also use a similar port number for multiple ArcSight connector configurations as the ArcSight Active List Reader process is capable of communicating with multiple ArcSight servers.
  • Poll Interval: Poll Interval, in seconds, that determines how frequently the ArcSight Active List Reader polls the active list for event IDs.
    Note: 300 seconds is the default poll interval.

Important: To run the Enable Pull ArcSight Events Service, you can import the FortiSOAR_ArcSight.arb in ArcSight. Steps to import the FortiSOAR_ArcSight.arb in ArcSight, is mentioned in the «Importing the FortiSOAR_ArcSight.arb package in ArcSight» section.
Also, ensure that the playbook whose trigger you have specified in the Playbook Trigger parameter is in the Active state. This ensures that you experience seamless event reading from ArcSight ESM.

Actions supported by the connector

The following automated operations can be included in playbooks, and you can also use the annotations to access operations from version 4.10.0 onwards:

Function Description Annotation and Category
Annotate Event Updates an ArcSight Event Stage, assigns it to a user and adds a comment. annotate_event
Investigation
Get Event Details Retrieves information for events from the ArcSight ESM server, based on event IDs and other input parameters you have specified. get_event_info
Investigation
Run Report with Default Parameters Runs a report based on an ID or URI and default inputs on the ArcSight ESM server. run_report
Investigation
Run Report Runs a report based on an ID and custom user inputs on the ArcSight ESM server. run_report
Investigation
Delete Report Deletes an archived report from the ArcSight ESM server, based on the Resource ID you have specified. delete_report
Remediation
Create Case Creates a case in ArcSight ESM, based on the input parameters you have specified. create_case
Investigation
Update Case Updates an existing case in ArcSight ESM, based on the input parameters you have specified. update_case_info
Investigation
Get Case Information Retrieves information about a case from ArcSight ESM, based on the case ID you have specified. get_case_info
Investigation
Add Events to Case Adds the specified events to an existing case in ArcSight ESM, based on the case ID you have specified. add_events
Investigation
Delete Case Events Deletes the specified events from an existing case from ArcSight ESM, based on the case ID you have specified. delete_events
Remediation
Search Query Searches ArcSight ESM records based on the query you have specified. search_query
Investigation
Download Report Downloads a report based on an ID from ArcSight ESM and then upload that report as an attachment in the Attachment Module. upload_report
Investigation
Get Active List Information Retrieves information about an active list from ArcSight ESM, based on the Active List ID you have specified. get_active_list_info
Investigation
Update Active List Adds new items to a specified active list on ArcSight ESM, based on the Active List ID and other input parameters you have specified. update_active_list
Investigation
Get Active List Entries Retrieves entries for a specified active list, based on the Active List ID you have specified. get_active_list_entries
Investigation
Clear Active List Entries Clear entries for a specified active list, based on the Active List ID you have specified. clear_active_list_entries
Remediation
Delete Active List Entries Deletes entries from a specified active list in ArcSight ESM, based on the Active List ID and other input parameters you have specified. delete_active_list_entries
Investigation
Get Fields Retrieves details of all fields from ArcSight ESM. get_fields
Investigation
Get Query Viewer Data Retrieves data of a specific query viewer from ArcSight ESM, based on the Query Viewer ID you have specified. get_query_viewer_data
Investigation

operation: Annotate Event

You can annotate ArcSight Events using the ArcSight Console to update the Stage and Assignee of the event and to add comments to the event.

 ArcSight Console: Update Stage and Add Comments to the event

 ArcSight Console: Update Assignee of the event

You can also perform similar operations using the Annotate Event function in FortiSOAR™ playbooks.

Input parameters

Parameter Description
Event ID The ID of the ArcSight Event that you want to annotate.
Stage The Stage to be set for the Event. You can choose from one of the following values:
Queued/Initial/Monitoring/Rule Created/Follow-Up/Final/Flagged as Similar/Closed
User An existing ArcSight user to whom you want to assign the event. For example, admin.
Comment The comment that you want to add to the event.

Output

The JSON output returns a Success message if the ArcSight ESM event is annotated successful or an Error message containing the reason for failure.

The output contains a non-dictionary value.

operation: Get Event Details

Input parameters

Parameter Description
Event IDs IDs of ArcSight Events whose details you want to retrieve from ArcSight. You can add multiple IDs using the CSV or list format.
Replace Null Values with Empty String? If an event field is not set, the ArcSight APIs return the following values. Use this option to replace these values with an empty string. Note that, by default, the Replace Null Values with Empty String? field is set to True.
Field Type integer: Returned value in place of NULL: -2147483648 (Integer.MIN_VALUE)
Field Type long: Returned value in place of NULL: -9223372036854775808 (Long.MIN_VALUE)
Field Type double: Returned value in place of NULL: 5e-324 (Double.MIN_VALUE
IP Address Keys to Parse (Optional) ArcSight API returns the IP address fields in decimal format. Provide a comma-separated list of field names you want to convert from decimal to IP address format.
Defaults to address.
MAC Address Keys to Parse (Optional) ArcSight API returns the MAC address fields in decimal format. Provide a comma-separated list of field names you want to convert from decimal to MAC address format.
Defaults to macAddress,translatedAddress.
Fields Name (Optional) Specify field names if you want to retrieve a specific set of columns from Micro Focus ArcSight.
Time Field Names (Optional) Specify a comma-separated list or array of field names for which you want to perform time conversion in the output of this operation.
Date Time Format (Optional) Specify the DateTime format for converting the time fields. You must specify a DateTime format that is supported by the arrow library. For more information on the arrow library, see https://arrow.readthedocs.io/en/latest/

Output

The JSON output contains the details of the event, based on the specified event ID and other input parameters, retrieved from ArcSight ESM.

The output contains the following populated JSON schema:
{
     "endTime": "",
     "ttl": "",
     "severity": "",
     "locality": "",
     "domainFp5": "",
     "domainFp2": "",
     "domainNumber3": "",
     "name": "",
     "domainFp6": "",
     "domainFp1": "",
     "deviceCustomFloatingPoint2": "",
     "deviceCustomString2": "",
     "domainDate6": "",
     "originalAgent": {
         "name": "",
         "assetName": "",
         "address": "",
         "translatedAddress": "",
         "assetId": "",
         "version": "",
         "type": "",
         "addressAsBytes": "",
         "macAddress": "",
         "mutable": "",
         "hostName": "",
         "zone": {
             "referenceID": "",
             "managerID": "",
             "uri": "",
             "isModifiable": "",
             "referenceString": "",
             "referenceType": "",
             "referenceName": "",
             "id": ""
         },
         "assetLocalId": "",
         "id": ""
     },
     "domainNumber8": "",
     "domainNumber6": "",
     "deviceCustomFloatingPoint4": "",
     "type": "",
     "startTime": "",
     "domainIpv4addr1": "",
     "destination": {
         "translatedPort": "",
         "assetName": "",
         "address": "",
         "translatedAddress": "",
         "geo": {
             "mutable": "",
             "longitude": "",
             "latitudeLong": "",
             "longitudeLong": "",
             "latitude": ""
         },
         "assetId": "",
         "processId": "",
         "addressAsBytes": "",
         "port": "",
         "mutable": "",
         "hostName": "",
         "zone": {
             "referenceID": "",
             "managerID": "",
             "uri": "",
             "isModifiable": "",
             "referenceString": "",
             "referenceType": "",
             "referenceName": "",
             "id": ""
         },
         "assetLocalId": "",
         "macAddress": ""
     },
     "domainIpv4addr3": "",
     "domainNumber5": "",
     "domainFp8": "",
     "domainFp4": "",
     "deviceEventClassId": "",
     "agentReceiptTime": "",
     "persistence": "",
     "managerId": "",
     "deviceSeverity": "",
     "deviceCustomFloatingPoint3": "",
     "deviceCustomString1": "",
     "flexNumber1": "",
     "domainDate1": "",
     "domainNumber12": "",
     "deviceReceiptTime": "",
     "deviceEventCategory": "",
     "category": {
         "outcome": "",
         "object": "",
         "mutable": "",
         "significance": "",
         "behavior": "",
         "deviceGroup": ""
     },
     "deviceCustom": {
         "mutable": "",
         "number1Label": "",
         "string1Label": "",
         "string2Label": ""
     },
     "domainIpv4addr2": "",
     "deviceCustomDate2": "",
     "modelConfidence": "",
     "correlatedEventCount": "",
     "deviceCustomDate1": "",
     "managerReceiptTime": "",
     "baseEventCount": "",
     "domainNumber13": "",
     "aggregatedEventCount": "",
     "deviceCustomFloatingPoint1": "",
     "domainDate3": "",
     "domainFp7": "",
     "deviceCustomNumber3": "",
     "assetCriticality": "",
     "deviceDirection": "",
     "domainIpv4addr4": "",
     "agentSeverity": "",
     "eventAnnotation": {
         "endTime": "",
         "stage": {
             "referenceID": "",
             "managerID": "",
             "uri": "",
             "isModifiable": "",
             "referenceString": "",
             "referenceType": "",
             "referenceName": "",
             "id": ""
         },
         "version": "",
         "modificationTime": "",
         "eventId": "",
         "managerReceiptTime": "",
         "flags": "",
         "auditTrail": "",
         "stageUpdateTime": ""
     },
     "agent": {
         "name": "",
         "assetName": "",
         "address": "",
         "translatedAddress": "",
         "assetId": "",
         "version": "",
         "type": "",
         "addressAsBytes": "",
         "macAddress": "",
         "mutable": "",
         "hostName": "",
         "zone": {
             "referenceID": "",
             "managerID": "",
             "uri": "",
             "isModifiable": "",
             "referenceString": "",
             "referenceType": "",
             "referenceName": "",
             "id": ""
         },
         "assetLocalId": "",
         "id": ""
     },
     "domainDate5": "",
     "deviceProcessId": "",
     "relevance": "",
     "finalDevice": {
         "product": "",
         "assetName": "",
         "address": "",
         "translatedAddress": "",
         "assetId": "",
         "version": "",
         "addressAsBytes": "",
         "vendor": "",
         "mutable": "",
         "hostName": "",
         "zone": {
             "referenceID": "",
             "managerID": "",
             "uri": "",
             "isModifiable": "",
             "referenceString": "",
             "referenceType": "",
             "referenceName": "",
             "id": ""
         },
         "assetLocalId": "",
         "macAddress": ""
     },
     "domainNumber11": "",
     "sessionId": "",
     "deviceCustomNumber2": "",
     "flexDate1": "",
     "bytesIn": "",
     "concentratorDevices": {
         "product": "",
         "assetName": "",
         "address": "",
         "translatedAddress": "",
         "assetId": "",
         "version": "",
         "addressAsBytes": "",
         "vendor": "",
         "mutable": "",
         "hostName": "",
         "zone": {
             "referenceID": "",
             "managerID": "",
             "uri": "",
             "isModifiable": "",
             "referenceString": "",
             "referenceType": "",
             "referenceName": "",
             "id": ""
         },
         "assetLocalId": "",
         "macAddress": ""
     },
     "domainDate2": "",
     "deviceCustomNumber1": "",
     "device": {
         "product": "",
         "assetName": "",
         "address": "",
         "translatedAddress": "",
         "assetId": "",
         "version": "",
         "addressAsBytes": "",
         "vendor": "",
         "mutable": "",
         "hostName": "",
         "zone": {
             "referenceID": "",
             "managerID": "",
             "uri": "",
             "isModifiable": "",
             "referenceString": "",
             "referenceType": "",
             "referenceName": "",
             "id": ""
         },
         "assetLocalId": "",
         "macAddress": ""
     },
     "originator": "",
     "domainNumber9": "",
     "domainNumber1": "",
     "priority": "",
     "domainDate4": "",
     "flexNumber2": "",
     "domainNumber7": "",
     "eventId": "",
     "domainFp3": "",
     "domainNumber2": "",
     "domainNumber4": "",
     "concentratorAgents": {
         "name": "",
         "assetName": "",
         "address": "",
         "translatedAddress": "",
         "assetId": "",
         "version": "",
         "type": "",
         "addressAsBytes": "",
         "macAddress": "",
         "mutable": "",
         "hostName": "",
         "zone": {
             "referenceID": "",
             "managerID": "",
             "uri": "",
             "isModifiable": "",
             "referenceString": "",
             "referenceType": "",
             "referenceName": "",
             "id": ""
         },
         "assetLocalId": "",
         "id": ""
     },
     "bytesOut": "",
     "domainNumber10": ""
}

operation: Run Report with Default Parameters

You can get the ID for a report (Resource ID) from the ArcSight Console, as shown in the following image:

ArcSight Console - Resource ID

You can get the URI for a report from the ArcSight Console. To get the URI, you must add the report name to the parent resource, as shown in the following image:

ArcSight Console - Parent Groups: Resources

Input parameters

Parameter Description
Run Report By Parameter of the report based on which you want to run a report on ArcSight ESM. You can choose between Report ID or Report URI.
Report URI or Report ID ID or URI of the ArcSight report that you want to run on ArcSight ESM.

Output

The JSON output returns the Download ID of the report. You can use this Download ID to download the report subsequently when the report is ready. You can use the «Download Report» operation to download the report and add it as an attachment in FortiSOAR™.

The output contains a non-dictionary value.

operation: Run Report

Input parameters

Parameter Description
Report Id ID of the ArcSight report that you want to run.
Input parameters Input parameters in the JSON format.
For example, {'StartTime': '$Now - 3h', 'Report Format': '0'}.
The keys are the same as seen on the ArcSight console. Note that the values for the drop-down fields are their integer positions. For example, the Report Format should be specified as 0, 1, 2, etc., and not as pdf, csv, html, etc.

Output

The JSON output returns the Download ID of the report. You can use this Download ID to download the report subsequently when the report is ready. You can use the «Download Report» operation to download the report and add it as an attachment to FortiSOAR™.

The output contains a non-dictionary value.

operation: Delete Report

Input parameters

Parameter Description
Report ID ID of the archived ArcSight report that you want to delete from ArcSight ESM.

Output

The JSON output returns a Success message if the specified report is deleted from ArcSight ESM, or an Error message containing the reason for failure.

The output contains a non-dictionary value.

operation: Create Case

Input parameters

Parameter Description
Parent Group ID Parent Group ID of the case you want to create.
Case Name Name of the case that you want to create.
Alias (Display Name) (Optional) Alias or Display Name of the case that you want to create.
Ticket Type (Optional) Ticket type of the case you want to create.
You can choose from the following options: INTERNAL, CLIENT, or INCIDENT.
Stage (Optional) Stage that you want to assign to the created case.
You can choose from the following options: QUEUED, INITIAL, FOLLOW_UP, FINAL, or CLOSED.
Frequency (Optional) Frequency that you want to assign to the created case.
You can choose from the following options: TEN_TO_FIFTEEN, NEVER_OR_ONCE, FIFTEEN, LESS_THAN_TEN, or MORE_THAN_FIFTEEN.
Operational Impact (Optional) Operational Impact that you want to assign to the created case.
You can choose from the following options: NO_IMPACT, NO_IMMEDIATE_IMPACT, LOW_PRIORITY_IMPACT, HIGH_PRIORITY_IMPACT, or IMMEDIATE_IMPACT.
Security Classification (Optional) Security Classification that you want to assign to the created case.
You can choose from the following options: UNCLASSIFIED, CONFIDENTIAL, SECRET, or TOP_SECRET.
Consequence Severity (Optional) Consequence Severity that you want to assign to the created case.
You can choose from the following options: NONE, INSIGNIFICANT, MARGINAL, CRITICAL, or CATASTROPHIC.
External ID (Optional) Unique ID of the case you want to create.
Description (Optional) Description of the case you want to create.
Deprecated (Optional) Whether or not the created case is deprecated.
Additional attributes in json format (Optional) Use this field to set values that are not displayed in FortiSOAR™.

Output

The JSON output contains the case ID and the details of the case created on ArcSight ESM.

The output contains the following populated JSON schema:
{
     "modifierName": "",
     "creatorName": "",
     "type": "",
     "reference": {
         "referenceType": "",
         "uri": "",
         "referenceString": "",
         "referenceName": "",
         "isModifiable": "",
         "id": "",
         "managerID": ""
     },
     "displayID": "",
     "createdTimestamp": "",
     "attributeInitializationInProgress": "",
     "deprecated": "",
     "isAdditionalLoaded": "",
     "state": "",
     "estimatedRestoreTime": {
         "day": "",
         "minute": "",
         "year": "",
         "hour": "",
         "timezoneID": "",
         "second": "",
         "milliSecond": "",
         "month": ""
     },
     "modifiedTimestamp": "",
     "name": "",
     "localID": "",
     "description": "",
     "URI": "",
     "initialized": "",
     "disabled": "",
     "reportingLevel": "",
     "numberOfOccurences": "",
     "inCache": "",
     "inactive": "",
     "resourceid": "",
     "typeName": "",
     "modificationCount": "",
     "createdTime": {
         "day": "",
         "minute": "",
         "year": "",
         "hour": "",
         "timezoneID": "",
         "second": "",
         "milliSecond": "",
         "month": ""
     },
     "modifiedTime": {
         "day": "",
         "minute": "",
         "year": "",
         "hour": "",
         "timezoneID": "",
         "second": "",
         "milliSecond": "",
         "month": ""
     }
}

operation: Update Case

Input parameters

Parameter Description
Case ID ID of the case you want to update.
Case Name (Optional) Updated case name, if you want to update the name of an existing case.
Alias (Display Name) (Optional) Alias or Display Name of the case that you want to update.
Ticket Type (Optional) Ticket type of the case you want to update.
You can choose from the following options: INTERNAL, CLIENT, or INCIDENT.
Stage (Optional) Updated stage, if you want to update the stage of an existing case.
You can choose from the following options: QUEUED, INITIAL, FOLLOW_UP, FINAL, or CLOSED.
Frequency (Optional) Updated frequency, if you want to update the frequency of an existing case.
You can choose from the following options: TEN_TO_FIFTEEN, NEVER_OR_ONCE, FIFTEEN, LESS_THAN_TEN, or MORE_THAN_FIFTEEN.
Operational Impact (Optional) Updated operational impact, if you want to update the operational impact of an existing case.
You can choose from the following options: NO_IMPACT, NO_IMMEDIATE_IMPACT, LOW_PRIORITY_IMPACT, HIGH_PRIORITY_IMPACT, or IMMEDIATE_IMPACT.
Security Classification (Optional) Updated security classification, if you want to update the security classification of an existing case.
You can choose from the following options: UNCLASSIFIED, CONFIDENTIAL, SECRET, or TOP_SECRET.
Consequence Severity (Optional) Updated consequence severity, if you want to update the consequence severity of an existing case.
You can choose from the following options: NONE, INSIGNIFICANT, MARGINAL, CRITICAL, or CATASTROPHIC.
Estimated Restore Date Time (Optional) Updates the Date and time for restoring the case, if required.
External ID (Optional) Updated External ID, if you want to update the Unique ID of the case.
Description (Optional) Updated description of the case.
Deprecated (Optional) Updates whether or not the case is deprecated.
Notification Group IDs (Optional) IDs of groups that should be notified when the case is updated.
Custom Fields (Optional) Use this field to set or update values that are not displayed in FortiSOAR™.

Output

The JSON output contains the details of the case updated on ArcSight ESM.

The output contains the following populated JSON schema:
{
     "modifierName": "",
     "creatorName": "",
     "alias": "",
     "type": "",
     "reference": {
         "referenceType": "",
         "uri": "",
         "referenceString": "",
         "referenceName": "",
         "isModifiable": "",
         "id": "",
         "managerID": ""
     },
     "displayID": "",
     "initialized": "",
     "createdTimestamp": "",
     "attributeInitializationInProgress": "",
     "deprecated": "",
     "isAdditionalLoaded": "",
     "state": "",
     "estimatedRestoreTime": {
         "day": "",
         "minute": "",
         "year": "",
         "hour": "",
         "timezoneID": "",
         "second": "",
         "milliSecond": "",
         "month": ""
     },
     "modifiedTimestamp": "",
     "name": "",
     "localID": "",
     "description": "",
     "URI": "",
     "estimatedStartTime": {
         "day": "",
         "minute": "",
         "year": "",
         "hour": "",
         "timezoneID": "",
         "second": "",
         "milliSecond": "",
         "month": ""
     },
     "disabled": "",
     "reportingLevel": "",
     "numberOfOccurences": "",
     "inCache": "",
     "inactive": "",
     "resourceid": "",
     "typeName": "",
     "modificationCount": "",
     "createdTime": {
         "day": "",
         "minute": "",
         "year": "",
         "hour": "",
         "timezoneID": "",
         "second": "",
         "milliSecond": "",
         "month": ""
     },
     "modifiedTime": {
         "day": "",
         "minute": "",
         "year": "",
         "hour": "",
         "timezoneID": "",
         "second": "",
         "milliSecond": "",
         "month": ""
     },
     "eventIDs": "",
     "detectionTime": {
         "day": "",
         "minute": "",
         "year": "",
         "hour": "",
         "timezoneID": "",
         "second": "",
         "milliSecond": "",
         "month": ""
     }
}

operation: Get Case Information

Input parameters

Parameter Description
Case ID ID of the case for which you want to retrieve the information from ArcSight ESM.

Output

The JSON output contains the details of the case, retrieved from ArcSight ESM, based on the specified case ID.

The output contains the following populated JSON schema:
{
     "modifierName": "",
     "creatorName": "",
     "type": "",
     "reference": {
         "referenceType": "",
         "uri": "",
         "referenceString": "",
         "referenceName": "",
         "isModifiable": "",
         "id": "",
         "managerID": ""
     },
     "displayID": "",
     "initialized": "",
     "createdTimestamp": "",
     "attributeInitializationInProgress": "",
     "deprecated": "",
     "isAdditionalLoaded": "",
     "state": "",
     "estimatedRestoreTime": {
         "day": "",
         "minute": "",
         "year": "",
         "hour": "",
         "timezoneID": "",
         "second": "",
         "milliSecond": "",
         "month": ""
     },
     "modifiedTimestamp": "",
     "name": "",
     "localID": "",
     "description": "",
     "URI": "",
     "estimatedStartTime": {
         "day": "",
         "minute": "",
         "year": "",
         "hour": "",
         "timezoneID": "",
         "second": "",
         "milliSecond": "",
         "month": ""
     },
     "disabled": "",
     "reportingLevel": "",
     "numberOfOccurences": "",
     "inCache": "",
     "inactive": "",
     "resourceid": "",
     "typeName": "",
     "modificationCount": "",
     "createdTime": {
         "day": "",
         "minute": "",
         "year": "",
         "hour": "",
         "timezoneID": "",
         "second": "",
         "milliSecond": "",
         "month": ""
     },
     "modifiedTime": {
         "day": "",
         "minute": "",
         "year": "",
         "hour": "",
         "timezoneID": "",
         "second": "",
         "milliSecond": "",
         "month": ""
     },
     "eventIDs": "",
     "detectionTime": {
         "day": "",
         "minute": "",
         "year": "",
         "hour": "",
         "timezoneID": "",
         "second": "",
         "milliSecond": "",
         "month": ""
     }
}

operation: Add Events to Case

Input parameters

Parameter Description
Case ID ID of the case in which you want to add events.
Events IDs IDs of the events that you want to add to the specified case.
You must provide the Event IDs in the list format.

Output

The JSON output returns a Success message if the events are successfully added to the specified case ID, or an Error message containing the reason for failure.

The output contains a non-dictionary value.

operation: Delete Case Events

Input parameters

Parameter Description
Case ID ID of the case from which you want to delete events.
Events IDs IDs of the events that you want to delete from the specified case.
You must provide the Event IDs in the list format.

Output

The JSON output returns a Success message if the events are successfully deleted from the specified case ID, or an Error message containing the reason for failure.

The output contains a non-dictionary value.

operation: Search Query

Input parameters

Parameter Description
Query Query using which you want to search ArcSight ESM.
Start Position Position from where you want to start the search.
By default, this is set to 0.
Page Size Number of result records that you want to display on one page.
By default, this is set to 10.

Output

The JSON output contains the search results retrieved from ArcSight ESM, based on the specified query.

The output contains the following populated JSON schema:
{
     "elapsed": "",
     "queryStr": "",
     "hitCount": "",
     "statusString": "",
     "searchHits": [
         {
             "uri": "",
             "score": "",
             "uuid": "",
             "name": ""
         }
     ],
     "queryTerms": [],
     "rewrittenQueryString": ""
}

operation: Download Report

Input parameters

Parameter Description
Report ID Download ID of the ArcSight report that you want to upload as an attachment in FortiSOAR™.
Note: You can get the ID of the report using the Run Report function.
Name of the file when added as an attachment in Cybersponse Name of the file when it is added as an attachment in FortiSOAR™.
If you do not specify any name, then the file by default is named as ‘ArcSight Report’.

Output

The JSON output contains the details of the attachment in FortiSOAR™.

The output contains a non-dictionary value.

operation: Get Active List Information

Input parameters

Parameter Description
Active List ID Resource ID of the Active List for which you want to retrieve details from ArcSight ESM.

Output

The JSON output contains the details of the active list, retrieved from ArcSight ESM, based on the specified active list ID.

The output contains the following populated JSON schema:
{
     "modifierName": "",
     "multiMap": "",
     "creatorName": "",
     "inactive": "",
     "type": "",
     "reference": {
         "referenceType": "",
         "referenceString": "",
         "isModifiable": "",
         "referenceName": "",
         "id": "",
         "managerID": "",
         "uri": ""
     },
     "caseSensitiveType": "",
     "capacity": "",
     "isAdditionalLoaded": "",
     "keyFields": "",
     "createdTimestamp": "",
     "modifiedTimestamp": "",
     "deprecated": "",
     "timePartitioned": "",
     "state": "",
     "optimizeData": "",
     "activeListType": "",
     "attributeInitializationInProgress": "",
     "name": "",
     "localID": "",
     "fieldTypes": "",
     "fieldSubTypes": {},
     "URI": "",
     "initialized": "",
     "entryTimeToLive": "",
     "disabled": "",
     "partialCache": "",
     "inCache": "",
     "fieldNames": "",
     "resourceid": "",
     "typeName": "",
     "modificationCount": "",
     "createdTime": {
         "day": "",
         "timezoneID": "",
         "milliSecond": "",
         "month": "",
         "second": "",
         "year": "",
         "minute": "",
         "hour": ""
     },
     "modifiedTime": {
         "day": "",
         "timezoneID": "",
         "milliSecond": "",
         "month": "",
         "second": "",
         "year": "",
         "minute": "",
         "hour": ""
     }
}

operation: Update Active List

Input parameters

Parameter Description
Active List ID Resource ID of the Active List that you want to update on ArcSight ESM.
Column Names List (Optional) List of column names that you want to update, i.e., columns in which you want to add entries.
By default, all the column names are included.
Entry List List of entries to add to the specified active list.
You must add the values in the same sequence as the columns specified.
For example, [[“val1”, “val2”], [“val3”, “val4”]]

Output

The JSON output returns a Success message if the active list is successfully updated on ArcSight or an Error message containing the reason for failure.

The output contains a non-dictionary value.

operation: Get Active List Entries

Input parameters

Parameter Description
Active List ID (Optional) Resource ID of the active list for which you want to retrieve entries from Micro Focus ArcSight.
Clear Active List Entries Select this option, i.e., set it to True (default), to clear the entries of the specified active list after the active list is read.

Output

No output schema is available at this time.

operation: Clear Active List Entries

Input parameters

Parameter Description
Active List ID Resource ID of the Active List for which you want to clear entries from ArcSight ESM.

Output

The output response appears as follows if the entries are cleared successfully from the specified Active List:
Active List: <Active List ID> entries are cleared successfully.

operation: Delete Active List Entries

Input parameters

Parameter Description
Active List ID Resource ID of the Active List from which you want to delete entries on ArcSight ESM.
Note: The default ID is taken from the connector configuration. For more information, see the Configuring the connector section.
Entry List List of entries that you want to delete from the specified active list. You must add the values in the same sequence as the columns specified. For example, [[“val1”, “val2”], [“val3”, “val4”]]

Output

The output contains a non-dictionary value.

operation: Get Fields

Input parameters

None.

Output

The output contains the following populated JSON schema:
{
     "sei.getFieldsResponse": {
         "sei.return": [
             {
                 "fieldDisplayName": "",
                 "fieldType": {
                     "type": "",
                     "javaTypeName": "",
                     "name": ""
                 },
                 "sidetable": "",
                 "derived": "",
                 "groupDisplayName": "",
                 "simple": "",
                 "fieldName": "",
                 "reference": "",
                 "fieldIndex": "",
                 "copyOfFieldName": "",
                 "groupName": ""
             }
         ]
     }
}

operation: Get Query Viewer Data

Input parameters

Parameter Description
Query Viewer ID Resource ID of the query viewer for which you want to retrieve details from ArcSight ESM.

Output

No output schema is available at this time.

Included playbooks

The Sample - Micro Focus ArcSight - 3.0.0 playbook collection comes bundled with the Micro Focus ArcSight connector. This playbook contains steps using which you can perform all supported actions. You can see the bundled playbooks in the Automation > Playbooks section in FortiSOAR™ after importing the Micro Focus ArcSight connector.

  • Active List : Clear Active List Entries
  • Active List : Delete Active List Entries
  • Active List : Get Active List Entries
  • Active List : Get Active List Information
  • Active List : Update Active List
  • ArcSight > Fetch 
  • > Arcsight > Fetch Events And Create Record
  • >> ArcSight > Get Base Events
  • ArcSight > Ingest
  • Case : Add Events to Case
  • Case : Create Case
  • Case : Delete Case Events
  • Case : Get Case Information
  • Case : Update Case
  • Event : Annotate Event 
  • Event : Get Fields
  • Get Query Viewer Data
  • Report : Delete Report
  • Report : Download Report
  • Report : Run Report
  • Report : Run Report with Default Parameters
  • Search Query

Note: If you are planning to use any of the sample playbooks in your environment, ensure that you clone those playbooks and move them to a different collection since the sample playbook collection gets deleted during the connector upgrade and delete.

Troubleshooting

For troubleshooting, any issues with the Pull ArcSight Events Service, see the /var/log/cyops/cyops-integrations/arcsight/arcsight_reader.log log file.

Connection refused while requesting to run the wrapper

This generally occurs in the case of self-signed SSL certificates. If you are using self-signed certificates for testing or staging, keep in mind this problem will not occur in production, and you might need to switch the certificates on or off.

Resolution:

Ensure that the SSL certificates are trusted or that SSL checking is turned off in the wrapper script. This is not advised for production instances.

Playbook fails after the ingestion is triggered

There are many reasons for a playbook failure, for example, if a required field is null in the target module record, or there are problems with the Playbook Appliance keys.

Resolution:

Investigate the reason for failure using the Running Playbooks tab in the Playbook Administration page. Review the step in which the failure is being generated and the result of the step, which should contain the trace of the error. Once you have identified the error and if you cannot troubleshoot the error, contact Fortinet support for further assistance.

ArcSight ESM and FortiSOAR™ integration

ArcSight ESM and FortiSOAR™ integration is achieved by the following simple steps:

  • Setup an Active List in ArcSight ESM
  • Create a user in ArcSight ESM 
  • Setup the ArcSight ESM connector in FortiSOAR™
  • Configure Data Ingestion

Setup an Active List in ArcSight ESM

An Active List in ArcSight ESM holds correlated events, which can be read by FortiSOAR™ and then converted into alerts.

To ingest data from ArcSight, you need to create an “Active List” and configure “Rules” in ArcSight ESM, so that events from ArcSight ESM can be pulled into FortiSOAR™ as described in the following sections.

Use the “FortiSOAR_ArcSight.arb” package to create the Active List in ArcSight ESM and configure the Rule that forwards desired events to the created active list. You have to create and configure a rule to define the type of events you want to forward and investigate in FortiSOAR™. Once the active list is added and the rule is configured, FortiSOAR™ monitors the active list pulls the desired events from ArcSight ESM, and creates alerts in FortiSOAR™. 

Download the FortiSOAR_ArcSight.arb package, which is attached to this article, and then import the same into ArcSight ESM, as described Importing the FortiSOAR_ArcSight.arb package in ArcSight section.

Alternatively, you can manually set up the active list and the rules using the standard ArcSight interface. Points to be considered while manually setting up rules:

Rule:

  • Create a rule (generally a “Lightweight Rule”) that would populate events in the FortiSOAR Active List.
  • Specify conditions to filter out undesired events, such as add a rule to exclude low priority events so that unnecessary alerts would not be pushed to FortiSOAR™.

The following image displays a sample ArcSight Rule to Forward Events to an Active list:

The following image displays an Active List populated with desired events and the Resource ID is highlighted in the right pane: 

Create a user in ArcSight ESM 

FortiSOAR™ requires a user account and password to connect to ArcSight ESM. You could use an existing user, or create a new standard user for this purpose. This user account will be used by FortiSOAR™ to fetch and/or update events and invoke other supported actions. Ensure that the user has the following permissions: 

  • Read and write access to the FortiSOAR Active List (“FortiSOAR AL”). 
  • Access to all required events in ArcSight ESM, including the base events.

The following image displays a FortiSOAR user in ArcSight ESM with «Read» and «Write» access to FortiSOAR AL:

Setup the ArcSight ESM connector in FortiSOAR™

Install and configure the Micro Focus ArcSight Connector in FortiSOAR™ as described in the Installing the connector and Configuring the connector sections.

Configure Data Ingestion

Configure data ingestion using the “Data Ingestion Wizard” to seamlessly map the incoming ArcSight ESM correlated event data into a FortiSOAR™ alert.

The Data Ingestion Wizard enables you to configure scheduled pulling of data from ArcSight ESM into FortiSOAR™. It also lets you pull some sample data from ArcSight ESM using which you can define mapping of data between ArcSight ESM and FortiSOAR™. Mapping of common fields are generally already done by the Data Ingestion Wizard; users mostly require to only map any custom fields that are added in the ArcSight event.

  1. To begin configuring data ingestion, click Configure Data Ingestion on the ArcSight ESM connector’s Configurations page. 

    Click Let’s Start by fetching some data, to open the “Fetch Sample Data” screen.
    Sample data is required to create a field mapping between ArcSight ESM data and FortiSOAR™. The sample data is pulled from connector actions or ingestion playbooks.
  2. On the Fetch Data screen, provide the configurations required to fetch ArcSight ESM data, i.e., specify the Event IDs of sample events to be fetched to create a mapping between ArcSight ESM events and FortiSOAR™ alerts.
    You can also specify the maximum number of base events to be fetched per event and then click Fetch Data.

    The Data Ingestion Wizard uses the specified event IDs to pull sample data from ArcSight ESM into FortiSOAR™. The specified event IDs are used only as sample data and not used for subsequent data ingestion.
  3. On the Field Mapping screen, map the fields of an ArcSight event to the fields of an alert present in FortiSOAR™. 
    To map a field, click the key in the sample data to add the “jinja” value of the field. For example, to map the path parameter of an ArcSight event to the File Path parameter of a FortiSOAR™ alert, click the Field Path field and then click the path field to populate its keys:  

    For more information on field mapping, see the Data Ingestion Wizard chapter see the «Connectors Guide» in the FortiSOAR™ product documentation.
    Once you have completed mapping fields, click Save Mapping & Continue.
    It is recommended to schedule the pull data from ArcSight ESM using the Poll Interval parameter in the “Enable Pull ArcSight Service” section of the Micro Focus ArcSight connector Configurations page as described in the Configuring the connector section. Therefore, just click Save Setting & Continue.
  4. The Summary screen displays a summary of the mapping done, and it also contains links to the Ingestion playbooks. Click Done to complete the data ingestion and exit the Data Ingestion Wizard.

For additional information about the “Data Ingestion Wizard” and installing and configuring connectors, see the «Connectors Guide» in the FortiSOAR™ product documentation.

Importing the FortiSOAR_ArcSight.arb package in ArcSight

Note: The ‘FortiSOAR_ArcSight.arb’ package included with this version has been updated to remove the ‘Active List Rule’ from the package.

  1. Download the FortiSOAR_ArcSight.arb file that is attached to this document.
  2. To import the FortiSOAR_ArcSight.arb package in ArcSight, navigate to the Packages tab in ArcSight as shown in the following image:
  3. Click Import and select the FortiSOAR_ArcSight.arb.
    The FortiSOAR_ArcSight.arb package contains the Active List (FortiSOAR_Event_collector).
    Once the FortiSOAR_ArcSight.arb package is imported successfully, the FortiSOAR Active List will appear in ArcSight as follows:

FortiSOAR_ArcSight.arb

ArcSight is a cyber security product that offers big data security analytics and intelligence software for SIEM and log management. It is designed to help clients discover and prioritize security risks, organize and manage incident response activities, and ease audit and compliance tasks. This article will cover all you need to know to get started with ArcSight ESM.

ArcSight ESM — Table of contents

  • What is ArcSight ESM?
  • What is the Use of ArcSight ESM?
  • ArcSight ESM Overview
  • ArcSight ESM Architecture
  • ArcSight ESM Key Features
  • ArcSight ESM Event Ingestion for Security Operations integration
  • ArcSight ESM Supported Versions
  • MID Server

What is ArcSight ESM?

ArcSight Enterprise Security Manager (ESM) is a Big Data analytics-based enterprise security solution that turns Big Data into actionable insight. ArcSight ESM is a market-leading security event information collection, correlation, and reporting system.ArcSight ESM evaluates and analyses every login, logoff, file access, and database query in the organization to give actual security risk ranking and breach of enforcement. 

What is the Use of ArcSight ESM?

ArcSight ESM is a market-leading security event information collection, correlation, and reporting system. ArcSight ESM aids you in the following areas:

  • Real-time correlation of data from any source to discover issues before they become a breach.
  • Building Security Use Cases with ArcSight ESM gives you a thorough understanding of ArcSight’s security problem-solving approach in the context of ESM.

If you want to enrich your career and become a professional in ForgeRock, then enroll in «ArcSight Training«. This course will help you to achieve excellence in this domain.

ArcSight ESM Overview

ESM uses ArcSight ESM Overview ArcSight ESM Architecture SmartConnectors to collect event data from your network.

SmartConnectors transform device event data into a standard format that may use to correlate.

The Manager in the CORR Engine is in charge of Processing and storing event data. Users may monitor events, run reports, produce resources, conduct investigations, and control the system using the ArcSight Console or the ArcSight Command Center.

ESM’s underlying architecture is used to power additional ArcSight products that control event flow, simplify event analysis, and offer security warnings and incident response.

ArcSight ESM Architecture

Several components make up the ESM for the Fusion environment, allowing it to receive and show data from sources like ESM. The following picture will help you comprehend the software and components that make up your ESM for Fusion setup. 

ArcSight ESM Architecture

ArcSight ESM Key Features

The following are critical features of ArcSight:

Layered Security Analytics.

It’s a one-stop solution for real-time correlation, hypothesis-based threat hunting, and behavioural analytics.

Native SOAR Out-of-the-Box 

Security Orchestration Automation and Response offers automated, coordinated, and expedited incident response.

Log Management and Reporting

Unified storage, quick big-data search, rich analytics, visualization, and reporting speed up threat hunting and make compliance easier.

MITRE ATT&CK Integration

Extensive coverage of MITRE ATT&CK methodologies and tactics, with tiered analytics and threat monitoring content packages.

Security Data Operating Platform

Real-time data collection and enrichment Device, connector, and destination management have been streamlined.

 MindMajix YouTube Channel

ArcSight ESM Event Ingestion for Security Operations integration

Security incident analysts may gather associated events and automate the development of security incidents with the ServiceNow platform thanks to the ArcSight ESM event ingestion interface with the Security Incident Response solution. Data is continuously absorbed depending on a polling schedule, and analysts utilise it to identify and respond to possible cyber security risks.

Correlated events that are candidates for security incidents can be ingested regularly using this integration. You may map fields in associated events to security incident fields, preview the configuration of an event as a security incident, and schedule event ingestion to automatically produce security incidents.

This connection gives a security operations centre (SOC) analyst access to ArcSight ESM correlation events. This data may be linked to Now Platform Security Incident Response (SIR) security incidents for further analysis and repair. Different correlation event types are produced and made available via correlation query viewers in ArcSight ESM, and your Now Platform instance profiles are built to manage them.

These profiles control the appearance of specific ArcSight ESM associated event fields for SIR security events.
This integration includes the following critical functionalities:

  • Create several event intake profiles to generate SIR security events for various risks such as malware and unauthorised access attempts.
  • Drag-and-drop mapping of ArcSight ESM correlation event field values to corresponding SIR security incident fields
  • To verify event mapping information, a preview of the SIR security incident layout based on example correlation events is supplied.
  • Input prior correlation events as well as new significant occurrences at predefined intervals.
  • Remove correlation events that do not meet SIR incident creation criteria, such as low priority events.

ArcSight ESM Supported Versions

The ArcSight ESM Manager version 7.0.0.2436 was used to test this integration. The integration supports ArcSight ESM on-premises and Cloud/Hosted service environments.

MID Server

When the ArcSight ESM server is deployed within your corporate network, this integration requires an installed and configured MID Server in your Now Platform instance to connect to the ArcSight ESM service. A MID Server is unnecessary if you use the ArcSight ESM cloud service. 

Conclusion:

With this, we have come to the end of this blog of ArcSight ESM. We hope the information covered is valuable and helps you gain a thorough grasp of ArcSight ESM.

About Author

Remy Sharp

SaiKumar Kalla

Kalla Saikumar is a technology expert and is currently working as a content associate at MindMajix. Write articles on multiple platforms such as ServiceNow, Business Analysis, Performance Testing, Mulesoft, Oracle Exadata, Azure, and other courses. And you can join him on LinkedIn.

Готов к работе

Скачать установщик

1. Загрузите установщик Arcsight HA и установщик Arcsight ESM с официального сайта HP.
2. Загрузите файл лицензии Arcsight, установщик Arcsight HA и установщик Arcsight ESM в каталог / tmp uatemsha1.
Введение в установщик
Структура каталогов ArcSightESMSuite-7.0.0.2234.1 выглядит следующим образом:

D:ArcSightArcSightESMSuite-7.0.0.2234.1>tree /F
D:.
│  ArcSightESMSuite.bin
│
├─ESMComponents
│  │  component.properties
│  │
│  ├─logger
│  │      ArcSightLogger.bin
│  │
│  ├─manager
│  │      ArcSightManager.bin
│  │      ArcSightManager_Documentation_Pack.iam.zip
│  │
│  └─service
│          boxster-services.tgz
│
└─Tools
    │  prepare_scripts.sh
    │  prepare_system.sh
    │  stop_services.sh
    │
    └─highavail
            functions
            prepareHA.sh
            setupESM.sh
            template.properties

Требования к разделу жесткого диска

При установке операционной системы оставшееся пространство не выделяется. После установки операционной системы выполните следующую команду, чтобы разделить пространство.

1. Создать PV

fdisk -l
partprobe /dev/sda
pvcreate /dev/sda4
#pvdisplay 
   --- Physical volume ---
  PV Name               /dev/sda4
  VG Name               vg00
  PV Size               500.39 GiB / not usable 4.96 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              128099
  Free PE               0
  Allocated PE          128099
  PV UUID               eTLmWv-QaK2-qHPJ-wJ3e-JfvQ-pw2I-anYUoT

Во-вторых, создать VG

vgcreate vg00 /dev/sda4
vgdisplay

Три, создай LV

1. Создайте раздел / tmp и отформатируйте его в формате ext2.

lvcreate -L 10G -n tmp vg00
lvdisplay
mkfs.ext4 /dev/vg00/tmp

2. Создать / matadate раздел без форматирования.

lvcreate -L 64M -n metadate vg00

3. Создайте раздел / opt и отформатируйте его в формате xfs.

lvcreate -l 100%FREE -n opt vg00
mkfs.xfs /dev/mapper/vg00-opt

4. Создайте файл автоматического монтирования при загрузке.
Отредактируйте / etc / fstab и добавьте следующие 2 строки

/dev/mapper/vg00-tmp /tmp ext4 defaults 1 2
/dev/mapper/vg00-opt /opt xfs defaults,inode64 1 2

[[email protected] ~]# fdisk -l

Disk /dev/sda: 600.1 GB, 600093712384 bytes, 1172058032 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 262144 bytes / 262144 bytes
Disk label type: dos
Disk identifier: 0x0006bfae

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1026047      512000   83  Linux
/dev/sda2         1026048   105883647    52428800   83  Linux
/dev/sda3       105883648   122660863     8388608   82  Linux swap / Solaris
/dev/sda4       122660864  1172058031   524698584   83  Linux

Конфигурация, связанная с операционной системой

1. Настройте IP-адрес и имя хоста

Конфигурация хоста 1:
1. Настройте IP-адрес и имя хоста.

Конфигурация хоста 1:
Имя хоста: uatesmha1
IP-адрес: 192.168.1.143
Шлюз: 192.168.1.254
/ etc / hosts:
192.168.79.143 uatesmha1
192.168.1.144 uatesmha2
192.168.1.145 uatesm

Конфигурация хоста 2:
Имя хоста: uatesmha2
IP-адрес: 192.168.1.144
Шлюз: 192.168.1.254
/ etc / hosts:
192.168.1.143 uatesmha1
192.168.1.144 uatesmha2
192.168.1.145 uatesm

Разархивируйте установщик

tar -xvf ArcSightESMSuite-7.0.0.2234.1.tar 
tar -xvf ArcSight-Highavail-7.0.0.1104.1.tar

Результаты распакованного каталога следующие:

файл highavail.properties

1. Файл свойств Arcsight HA

Перейдите в каталог / tmp / Tools / highavail и выполните команду:

cp ./template.properties highavail.properties

Используйте редактор vi для настройки следующего содержимого в highavail.properties.

# Сервисное имя хоста, т.е. uatesm
service_hostname=
# Общий диск, вот / opt
shared_disk=
Объем метаданных, здесь / dev / mapper / vg00-meta
metadata_volume=
# IP первичного сервера
primary_cable_ip=
# Имя основного сервера
primary_hostname=
# Резервный IP-адрес сервера
secondary_cable_ip=
# Имя хоста резервного сервера
secondary_hostname=

2. Запустите файл подготовки установки HA и выполните следующую команду:

/tmp/Tools/highavail/prepareHA.sh

Действуйте в соответствии с шагами, и выполнение будет завершено без ошибок. После завершения операции перезапустите сервер.

3. Скопируйте сгенерированные свойства highavail.properties на резервный компьютер и выполните также /tmp/Tools/highavail/prepareHA.sh. Перезагрузите сервер без ошибок.

scp ./highavail.properties [email protected]:/tmp/Tools/highavail/

4. Установите модуль Arcsight HA

Выполните следующие команды на главном сервере:

/tmp/ ArcSight-Highavail-7.0.0.1104.1.bin –i console

Следуйте инструкциям.

5. Проверьте, успешна ли установка

/usr/lib/arcsight/highavail/bin/arcsight_cluster status

Tue Jan 29 10:22:49 CST 2019 OK
uatesmha1: online Primary
uatesmha2: online

Disk: Connected UpToDate/UpToDate

OK   Network-uatesmha1
OK   Network-uatesmha2

Started   ESM
Started   Failover-Check-uatesmha1
Started   Failover-Check-uatesmha2
Started   Filesystem
Started   Ping-uatesmha1
Started   Ping-uatesmha2
Started   STONITH-SSH-uatesmha1
Started   STONITH-SSH-uatesmha2
Started   Service-IP

Установите Arcsight ESM

1. Запустите от имени пользователя arcsight: /tmp/ArcSightESMSuite.bin –i console и следуйте инструкциям
2. Запустите мастер первого запуска Arcsight ESM
/opt/arcsight/manager/bin/arcsight firstbootsetup -boxster -soft -i console
, обратите внимание, чтобы выбрать компактный режим и установить модуль HA Monitor.

Следуйте инструкциям и обратите внимание на имя хоста ESM при uatesm.
В соответствии с приглашением запустите /opt/arcsight/manager/bin/setup_services.sh под root, чтобы установить ESM в качестве службы.

3. Проверьте установку Arcsight ESM

/etc/init.d/arcsight_services status


Пока что установлен Arcsight ESM HA.
Проверьте состояние кластера:

/usr/lib/arcsight/highavail/bin/arcsight_cluster status

Troubleshoot

Сообщение об ошибке: ПРЕДУПРЕЖДЕНИЯ: arcsight-monit настроен, но не работает.

При выполнении «/etc/init.d/arcsight_services status», WARINGS: arcsight-monit настроен, но не запущен. Появляется сообщение об ошибке.

Подходить:

1. Выполните команду: / usr / lib / arcsight / highavail / bin arcsight_cluster диагностировать

, если нет других ошибок.
2. Переустановите службу и выполните следующую команду

/opt/arcsight/manager/bin/remove_services.sh
/opt/arcsight/manager/bin/setup_services.sh

3. Выполните: /etc/init.d/arcsight_services status. Сообщение об ошибке исчезает.

Microfocus / ArcSight Data Platform / ArcSight ESM

Content Player 2020-03-11 12-38-08

arcsight_securedata_add_on_for_adp_enabling_privacy_compliance_flyer.pdf 2020-03-26 10-48-37

arcsight_enterprise_security_manager_ds.pdf 2020-03-26 10-51-16

Reference Architecture

  1. Connector -> Logger -> ESM (Ideally.)
  2. Connector -> ESM -> Logger
  3. Connector -> Logger & ESM

Youtubes

Here’s the video showing what is possible with that CIRCP MISP integration

How ArcSight, CIRCL MISP and MITRE ATT&CK matrix can be used to provide realtime protection against these attacks capitalizing on Corona/COVID-19 fears.

Achieving True Zero-Day Protection with ArcSight, MITRE ATT&CK, and MISP CIRCL

How To: Configure MISP & ESM to address COVID-19 & Coronavirus threats

  • Amazing Community post (with ESM content: dashboards, etc..) by Steve Maxwell
  • How To: Configure MISP & ESM to address COVID-19 & Coronavirus threats – https://community.microfocus.com/t5/ArcSight-Tips-Information/How-To-Configure-MISP-amp-ESM-to-address-COVID-19-amp/ta-p/2771353
  • Security Operations and How to Defend Against COVID-19-themed Cyber Threats – https://community.microfocus.com/t5/Security-Blog/Security-Operations-and-How-to-Defend-Against-COVID-19-themed/ba-p/2771721
  • COVID-19 Security Package from SOC Prime – https://marketplace.microfocus.com/arcsight/content/covid-19-security-package-from-soc-prime
  • http://mitre.microfocus.com/
  • Here is an example of Mitre ArcSight Active Content –https://sec.microfocus.com/foswiki/bin/view/ArcSightActivate/L1EntityMonitoring

ArcSight Family

  • ArcSight ESM Express
  • ArcSight ESM Software
  • ArcSight ESM Appliance
  • ArcSight FPE (Voltage Format-Preserving Encryption (FPE) )
    • SecureData Add-on for SODP Enabling Privacy Compliance
    • https://www.microfocus.com/media/flyer/arcsight_securedata_add_on_for_adp_enabling_privacy_compliance_flyer.pdf
  • ArcSight ESM High Availability Module
  • ArcSight Logger
    • Logger (standalone)
      • use without ADP
      • Raw devices or Connectors
    • ArcSight ADP Logger
      • ArcMC central management
      • Raw devices, Event Broker or Connector
  • ArcSight Investigate (Powered by Vertica Big Data Analytics and Machine Learning.)
  • ArcSight Interset UEBA
  • Security Open Data Platform
  • ArcSight Transformation Hub
  • ArcSight Reputation Security Monitor
  • ArcSight ThreatDetector
    • Reputation-based intelligence to security information and event management, delivering even greater protection and further reducing business risk.
  • ArcSight Compliance Insight Package for IT Governance
  • ArcSight Event Broker / HUB
    • Based on Apache Kafka distributed event streaming platform capable of handling trillions of events a day integrated with Public Cloud platforms for Horizontal and Vertical scaling.
  • ArcSight Command Centre
  • ArcSight Compliance Insight Package – https://www.microfocus.com/media/data-sheet/security_arcsight_compliance_insight_package_for_it_governance_ds.pdf
  • ArcSight Activate Framework
  • ArcSight Interactive Discovery
  • ArcMC – Management Console
    • Focus on threats, not on tools. ArcSight Management Center (ArcMC) is a centralized security management center that manages large deployments of ArcSight solutions
  • ArcSight Console
  • Vertica / IDOL
  • ArcSight Distributed Analytics
  • Arcsight
  • NetIQ Sentinel
  • Arcsight SMARTConnectors Frameork
    • (Load Balancer) with Pacer https://clusterlabs.org/
    • TLS Voltage SecureDataEncryption at Application level FPE and FIPS compliant.
    • WiSC Windows Event Log SMARTConnector
      • Linux/64-bit -.NET
      • Microsoft Windows EventLog
      • Normal Windows User, Remote, No reboot required.
    • Windows Native SMARTconnector
      • WiNC
      • Windows/64-bit .NET
    • Microsoft Windows Event Log – Native
      • Windows Unified SMARTconnector
      • WUC
      • Legacy FCIFS
      • Connector Appliances, Virtual and Binary
    • NetFlow Collector
      • https://community.microfocus.com/t5/ArcSight-User-Discussions/ArcSight-and-NetFlow/td-p/1517147
      • https://community.microfocus.com/t5/ESM-and-ESM-Express-Previous/ArcSight-ESM-Netflow-Monitoring-Foundation-Guide-v1-1/ta-p/1583654
    • Smart Collectors – Integrate with EvenHub or Transformation Hub to offload enrichment off the Smart Collector agent.
      • Linux
      • Windows
    • Smart Connectors
      • Linux
      • Windows
      • Solaris
      • OpenSource
      • AWS CloudWatch
      • Azure Monitor EventHub
      • MispModelConnector – linux
      • MispModelConnector – Windows
    • ArcSight SmartConnector Parser
  • Logger SmartMessage Pools
  • FlexConnectors
  • Quick Flex Parser Tool
  • ArcSight Risk Insights
  • Reputation Security Monitor (RepSM) – http://www.hp.com/hpinfo/newsroom/press_kits/2013/rsa2013/DataSheet_RepSM.pdf
  • MISP threat intelligence with ArcSight ESM
    • https://www.circl.lu/services/misp-malware-information-sharing-platform/
  • DNS Malware Analytics (DMA)
  • Domain Generation Algorithm
    • https://community.microfocus.com/t5/ArcSight-User-Discussions/How-does-quot-HPE-DNS-Malware-Analytics-v2-4-quot-works/td-p/1512889
    • https://marketplace.microfocus.com/arcsight/content/dns-malware-analytics

Difference between a Smart Connector and Smart Collector

To undersand the Collectors v.s Connectors, we need to step back and look at what the SmarConnectors do.

Conceptually, the standard SmartConnectors have two main responsibilties: “Collect” raw data from various sources, and “Process” the collected data to become enriched security events and post them to a destination.

Introduced in ADP 2.30, customers can take advantage of the massive scalabilty and robustness of the Event Broker infrastructure, and move the computationaly intensive “Process” step to the highly scalable and more robust Event Broker streaming infrastructure.

This is done by using syslog Colelctors and syslog CEBs: Collectors are standalone compnents very similar to the SmartConenctors, but they only “Collect” raw syslog data like the syslog SmartConnectors do, wrap it up and post it to a dedicated eb-con-syslog topic in Event Broker.

At that point, the Event Broker’s CEB stream processors (CEB stands for Connector in Event Broker) read the data from the eb-con-syslog topic, do the parsing/normalization/enrichment/filtering processing (as the standalone SmartConnectors destination pipelines do) and post the security events on the EB topics for consumption.

In other words: as their name suggests, the syslog Collectors are lightweight component responsible for collecting raw syslog data and passing it to Event Broker for processing.

Main advantages of the new architecture:

  1. Potential for hardware consolidation and data throughput increase in the data collection layer where the Collectors are deployed: due to moving the processing to the EB streaming infrastructure.
  2. Improved stabilty and easy horizontal scalability as the data flows increase with time, or fluctuate during operations: CEBs are deployed or undeployed on the EB nodes with a single click in the ArcMC UI.
  3. Reduced network traffic due to a single data feed to Event Broker, instead of having tmultiple destinations coming from SmartConnectors
  4. The raw Syslog data is now available on the EB topic for any system that customer would like to share it with.

Note that at this time Colectors and CEBs are only available for Syslog data.

SmartConnector formats;

https://community.microfocus.com/t5/ArcSight-Connectors/ArcSight-SmartConnector-User-Guide-7-15-0/ta-p/1586784

  • Log File Readers (including text and log file)
  • Syslog
  • SNMP
  • Database
  • XML
  • Proprietary protocols, such as OPSEC

Connector Types

  • API Connectors
  • Database Connectors
    • Database connectors use SQL queries to periodically poll for events. Connectors support major database types, including
    • MS SQL, MS Access, MySQL, Oracle, DB2, Postgres, and Sybase.
    • IBM DB2 connectors: DB2 drivers are no longer provided in the connector installation due to licensing requirements.
    • Microsoft SQL Server Multiple Instance DB connector
    • McAfee Vulnerability Manager DB.
    • Time-Based Queries use a time field to retrieve events found since the most recent query time until the current time.
    • ID-Based Queries use a numerically increasing ID field to retrieve events from the last checked ID until the maximum ID.
    • Job ID-Based Queries use Job IDs that are not required to increase numerically. Processed Job IDs are filed in such a way that only new Job IDs are added. Unlike the other two types of database connector, Job IDs can run in either Interactive mode or Automatic mode
  • FlexConnectors
  • File Connectors
    • Real Time
    • Folder Follower:
  • Microsoft Windows Event Log Connectors
    • SmartConnector for Microsoft Windows Event Log
    • SmartConnector for Microsoft Windows Event Log – Native
    • SmartConnector for Microsoft Windows Event Log – Unified
  • Model Import Connectors
    • Rather than collecting and forwarding events from devices, Model Import Connectors import user data from an Identity Management system into ArcSight ESM. See individual configuration guides for Model Import Connectors on Protect724 for information about how these connectors are used
    • Model Import Connectors extract the user identity information from the database and populate the following lists in ESM with the data:
    • Identity Roles Session List
    • Identity Information Session List
    • Account-to-Identity Map Active List
  • Scanner Connectors
  • SNMP Connectors
    • SNMP Traps contain variable bindings, each of which holds a different piece of information for the event. They are usually sent over UDP to port 162, although the port can be changed. SNMP connectors listen on port 162 (or any other configured port) and process the received traps. They can process traps only from one device with a unique Enterprise OID, but can receive multiple trap types from this device. SNMP is based upon UDP, so there is a slight chance of events being lost over the network. Although there are still some SNMP connectors for individual connectors, most SNMP support is provided by the SmartConnector for SNMP Unified. Parsers use the knowledge of the MIB to map the event fields, but, unlike some other SNMP-based applications, the connector itself does not require the MIB to be loaded
  • Syslog Connectors
    • Syslog messages are free-form log messages prefixed with a syslog header consisting of a numerical code (facility + severity), timestamp, and host name. They can be installed as a syslog daemon, pipe, or file connector. Unlike other file connectors, a syslog connector can receive and process events from multiple devices. There is a unique regular expression that identifies the device.
    • Syslog Daemon connectors listen for syslog messages on a configurable port, using port 514 as a default. The default protocol is UDP, but other protocols such as Raw TCP are also supported. It is the only syslog option supported for Windows platforms.
    • Syslog Pipe connectors require syslog configuration to send messages with a certain syslog facility and severity. The Solaris platform tends to under perform when using Syslog Pipe connectors. The operating system requires that the connector (reader) open the connection to the pipe file before the syslog daemon (writer) writes the messages to it. When using Solaris and running the connector as a nonroot user, using a Syslog Pipe connector is not recommended. It does not include permissions to send an HUP signal to the syslog daemon.
    • Syslog File connectors require syslog configuration to send messages with a certain syslog facility and severity. For high throughout connectors, Syslog File connectors perform better than Syslog Pipe connectors because of operating system buffer limitations on pipe transmissions
    • Raw Syslog connectors generally do no parsing and takes the syslog string and puts it in the rawEvent field as-is . The Raw Syslog destination type takes the rawEvent field and sends it as-is using whichever protocol is chosen (UDP, Raw TCP, or TLS). The Raw Syslog connector is always used with the Raw Syslog destination. The event flow is streamlined to eliminate components that do not add value (for example, with the Raw Syslog transport the category fields in the event are ignored, so the categorization components are skipped). If you are transporting data to ArcSight Logger, you can use specific configuration parameters to provide minimal normalization of the syslog data (for source and timestamp)
    • Syslog NG Daemon connectors support Syslog NG version 3.0 for BSD syslog format. Support is provided for collection of IETF standard events. This connector is capable of receiving events over a secure (encrypted) TLS channel from another connector (whose destination is configured as CEF Syslog over TLS), and can also receive events from devices
    • CEF Encrypted Syslog (UDP) connectors allow connector-to-connector communication through an encrypted channel by decrypting events previously encrypted through the CEF Encrypted Syslog (UDP) destination. The CEF connector lets ESM connect to, aggregate, filter, correlate, and analyze events from applications and devices that deliver their logs in the CEF standard, using the syslog transport protocol.
    • UNIX supports all types of syslog connector. If a syslog process is already running, you can end the process or run the connector on a different port. Because UDP is not a reliable protocol, there is a slight chance of missing syslog messages over the network. Generally, TCP is a supported protocol for syslog connectors. There is a basic syslog connector, the connector for UNIX OS Syslog, which provides the base parser for all syslog sub-connectors. For syslog connector deployment information, see the connector Configuration Guide for UNIX OS Syslog. For device-specific configuration information and field mappings, see the connector configuration guide for the specific device. Each syslog sub-connector has its own configuration guide. During connector installation, for all syslog connectors, choose Syslog Daemon, Syslog Pipe, or Syslog File. The names of the syslog sub-connectors are not listed
  • IP NetFlow (NetFlow/J-Flow) Retrieves data over TCP in a Cisco-defined binary format.
  • ArcSight Streaming Connector Retrieves data over TCP from Logger in an ArcSight-proprietary format
  • Connectors for Transformation Hub
    • Connectors in =Transformation Hub supports ArcSight customers who want to have large-scale distributed ingestion pipelines with 100% availability, where data from any existing or new source at any scale can be ingested while maintaining enterprise level robustness. Transformation Hub can take messages with raw data collected from any source the ArcSight connector framework understands and automatically perform the data ingestion processing currently done by connectors, but deployed and managed at scale as Transformation Hub processing engines. Users deploy the Transformation Hub using the ArcSight Installer and Management Center to achieve the desired layout. New topics can be created in Management Centerand designated to process raw data from a particular technology framework with output into a specific format.
    • The connector technology in Transformation Hub performs all processing a connector would normally do: parser selection, normalization, main flow, destination specific flows, and categorization, as well as applying network zoning and Agent Name resolution. For more information, see the ArcSight Transformation Hub Administrator’s Guide and the ArcSight Management Center Administrator’s Guide.
Note: If you are using the Linux Red Hat 6.x or later platforms, ensure that you have these libraries or packages installed before installing a connector: 
X libraries 
glibc
libXext
libXrender
libXtst
unzip
fontconfig  dejavu-sans-fonts 
When installing the 32-bit SmartConnector executable on 64-bit machines, the 32-bit versions of glibc, libXext, libXrender, and libXtst must be installed as well as the 64-bit versions

ESM Install

Hyper-V Configuration

Partitions Sizes

Capture

partion

  • /tmp – more than 6 GB
  • /opt – more than 100 GB

CentOS Software Selection

  • GNOME Desktop
    • Compatibility Libraries
    • Development Tools
    • System Administration Tools

CentOSInstall

ESM bin

<ARCSIGHT_HOME>

/opt/arcsight/manager/bin

Log files /

opt/arcsight/var/logs/

Properties files

/opt/arcsight/manager/config


  • Make sure that the partition in which your /tmp directory resides has at least 6 GB of space. Make sure that the partition in which your /opt/arcsight directory resides has at least 100 GB of space.
  • Specifying a Global Event ID Generator ID, Global event IDs uniquely identify events across the ArcSigh
  • The Manager host name is used to generate a self-signed certificate. The Common Name (CN) in the certificate is the host name that you specify when prompted
  • The Manager host name is the IP address (for IPv4 only) or the fully-qualified domain name of the machine where the Manager is installed. All clients (for example, the ArcSight Console) use this name to connect to the Manager. For flexibility, Micro Focus recommends using a fully-qualified domain name instead of an IP address.
  • Make sure that the IP address 127.0.0.1 is resolved to localhost in the /etc/hosts file, otherwise, the ESM installation will fail. This applies to IPv4 and IPv6 systems.

  • If you do not want the host name on your DNS server, add a static host entry to the /etc/hosts file to resolve the host name locally.
  • 8443/tcp 22/tcp (ssh)
  • TCP ports used internally for inter-component communication: 1976, 28001, 2812, 3306, 5555, 6005, 6009, 7777, 7778, 7779, 7780, 8005, 8009, 8080, 8088, 8089, 8666, 8765, 8766, 8881, 8808, 8880, 8888, 8889, 9095, 9090, 9123, 9124, 9999, 45450
  • 8443/TCP – SmartConnectors and consoles 9000/TCP – Peering694/UDP – High Availability module 7789/TCP – High Availability module 22/TCP – SSH login

  • Open the following TCP ports for inter-component communication:

  • 1976, 2812, 3306, 5555, 6005, 6009, 7777, 7778, 7779, 7780, 8005, 8009, 8080, 8088, 8089, 8666, 8765, 8766, 8808, 8880, 8881, 8888, 8889, 9000, 9090, 9095, 9123, 9124, 9999, 28001, 45450

  • The information repository uses ports 3179, 3180, 3181, and 3182.

  • Port

    Flow

    Description

    22/TCP

    Inbound

    SSH log in (Unix only)

    53/UDP

    Inbound/Outbound

    DNS requests and responses

    8443/TCP

    Inbound

    SmartConnectors and Consoles

    25/TCP

    Outbound

    SMTP to mail server

    110/TCP

    Outbound

    POP3 to mail server, if applicable

    143/TCP

    Outbound

    IMAP to mail server, if applicable

    1645/UDP

    Inbound/Outbound

    RADIUS, if applicable

    1812/UDP

    Inbound/Outbound

    RADIUS, if applicable

    389/TCP

    Outbound

    LDAP to LDAP server, if applicable

    636/TCP Outbound LDAP over SSL to LDAP server, if applicable

  • <ARCSIGHT_HOME>/config/jetty/keystore (to prevent the ArcSight Manager private key from being stolen)
  • <ARCSIGHT_HOME>/config/jetty/truststore (with SSL Client authentication only, to prevent injection of new trusted CAs)
  • <ARCSIGHT_HOME>/config/server.properties (has database passwords)
  • <ARCSIGHT_HOME>/config/esm.properties (has cluster configuration properties and SSL properties common to persistor, correlator, and aggregator services on the node) This properties file is present on each node in a distributed correlation cluster.
  • <ARCSIGHT_HOME>/config/jaas.config (with RADIUS or SecurID enabled only, has shared node secret)
  • <ARCSIGHT_HOME>/config/client.properties (with SSL Client authentication only, has keystore passwords)
  • <ARCSIGHT_HOME>/reports/sree.properties (to protect the report license)
  • <ARCSIGHT_HOME>/reports/archive/* (to prevent archived reports from being stolen)
  • <ARCSIGHT_HOME>/jre/lib/security/cacerts (to prevent injection of new trusted CAs)
  • <ARCSIGHT_HOME>/lib/* (to prevent injection of malicious code) l <ARCSIGHT_HOME>/rules/classes/* (to prevent code injection)
  • The xmlrpc.accept.ips property restricts access for ArcSightConsoles.

  • The agents.accept.ips property restrict saccess for SmartConnectors.

  • For registration, the SmartConnectors need to be in xmlrpc.accept.ips as well, so that they can be registered. (Being “registered” does not mean you can then remove them.)

    • The format for specifying subnets is quite flexible, as shown in the following example:

    • xmlrpc.accept.ips=192.0.2.0 192.0.2.5

    • agents.accept.ips=10.*.*.*,192.0.0.0-192.0.255.255

System Requirements for ESM 7.2;

  • Community Enterprise Operating System (CentOS) 7.6 and 6.10

Minimum

Mid-Range

High Performance

Processors

8 cores (16 preferred)

32 cores

40 cores

Memory

48 GB RAM (64 preferred)

192 GB RAM

512 GB RAM

Hard Disk

Six 600 GB disks (1.5 TB) (RAID 10)

10,000 RPM

20 1 TB disks (10 TB) (RAID 10)

15,000 RPM

12 TB (RAID 10)

Solid state

Linux Install

Download Install CentOS 7.6 http://ftp.iij.ad.jp/pub/linux/centos-vault/7.6.1810/isos/x86_64/

[code="bash"]
//Use CentOS 7.6 - http://ftp.iij.ad.jp/pub/linux/centos-vault/7.6.1810/
Boot intro Troubleshooting —&gt; install CentOS 7 in basic graphics mode

Download the ArcSightESMSuite- 7.0.0.xxxx.1.tar from <a href="https://softwaresupport.softwaregrp.com/.">https://softwaresupport.softwaregrp.com/.</a>

scp  [email protected]:tmp/esminstall

//Install TMUX for remote installations

yum install tmux
tmux list-sessions
tmux attach -t number-of-session

// USB Mount
fdisk -l
mkdir /mnt/usb
mount -v -t auto /dev/sdf1 /mnt/usb
cd /mnt/usb/
umount /dev/sdf1

//Nic on laptop enp0s31f6
nmtui edit enp0s31f6

// Add hostanme to IP address in hosts file
nano /etc/host

//Mount USB
fdisk -l mkdir
/mnt/usb mount -v -t auto /dev/sdf1 /mnt/usb
cd /mnt/usb/
umount /dev/sdf1

// Unarchive installer

Create arcsight user with GUID and SU rights
Create a folder called esm_installer
chown arcsight: esm_installer
<span style="color:var(--color-text);">tar xvf ArcSightESMSuite-7.0.0.xxxx.1.tar</span>
cd Tools
sudo ./prepare_system.sh
reboot

// Copy the license files to same location

ulimit -a (<span style="color:var(--color-text);">open files 65536/</span><span style="color:var(--color-text);">max user processes 10240)</span>

// Download and set Timezone
wget tzdata-2019b-1.el7.noarch.rpm <span style="color:var(--color-text);">/opt/work/
rpm -Uvh /opt/work/</span>

sudo yum install tzdata -y
timedatectl list-timezones
timedatectl list-timezones | egrep -o “*Australian*.*”
timedatectl set-timezone “Asia/Kolkata”
timedatectl set-timezone America/Los_Angeles
timedatectl set-timezone UTC
timedatectl set-time 15:58:30
timedatectl set-time 20151120
timedatectl status
timedatectl | grep local
timedatectl set-local-rtc 1
timedatectl set-local-rtc 0
timedatectl set-ntp true

su arcsight | Pwd
Login under user account: arcsight into Console and install
/etc/init.d/arcsight_services stop all
/opt/arcsight/manager/bin/arcsight tzupdater /opt/arcsight /opt/arcsight/manager/lib/jre-tools/tzupdater
/etc/init.d/arcsight_services start all

//Starting the installer

chmod +x /tmp/esm_install/ArcSightESMSuite.bin

chown -R arcsight:arcsight ../Tools

// Error: You are installing this product on an unsupported platform.
// If you are install on later version you might need to downgrade the version manual then update it later
sudo nano /etc/centos-release
sudo nano /etc/redhat-release
CentOS Linux release 7.7.1908 (Core)
CentOS Linux release 7.6 (Core)

// LOGIN into CONSOLE as arcsight
./ArcSightESMSuite.bin -i console

/opt/arcsight/manager/bin/arcsight firstbootsetup -boxster -soft -i console

/opt/arcsight/kubernetes/scripts/cdf-updateRE.sh &gt; /tmp/ca.crt

//To install the time zone update package after you complete the ESM
installation:
/etc/init.d/arcsight_services stop all

/opt/arcsight/manager/bin/arcsight tzupdater /opt/arcsight
/opt/arcsight/manager/lib/jre-tools/tzupdater
/etc/init.d/arcsight_services start all

// As arcsight user

// Install ESM Login under user account: arcsight into Console and install
/opt/arcsight/manager/bin/arcsight firstbootsetup -boxster -soft -i console
/opt/arcsight/manager/bin/setup_services.sh
/var/spool/mail/root

IMPORTANT: The root user must run the following script to start up required services:
/opt/arcsight/manager/bin/setup_services.sh.

// START SERVICES as arcsight user
/etc/init.d/arcsight_services start
/etc/init.d/arcsight_services stop all
/etc/init.d/arcsight_services start all

//Set the hostname in local hosts file

//Applications/Google Chrome.app/Contents/MacOS/Google Chrome --ignore-certificate-errors &amp;&gt; /dev/null &amp;

// Access https://arcsight:8443

/Chrome SSL Error type "thisisunsafe"

// Remove ESM
./Uninstall_ArcSight_ESM_Suite_7.2.0.0
/opt/arcsight/manager/bin/remove_services.sh
su arcsight
./opt/arcsight/suite/UninstallerData/Uninstall_ArcSight_ESM_Suite_7.2.0.0 

Remove all files in /tmp and /opt/arcsight rm -r *
[/code]

The volume or partition required for installation of the /opt/arcsight directory does not contain the minium of 50GB of space to successfully install arcsight

Capture

[code]

df
df /opt/arcsight

df /opt/arcsight 	50GB
df/tmp	 		6GB

lsblk
df -Th
pvs
vgs
lvs

parted
echo 1 &gt; /sys/block/sda/device/rescan
resizepart
pvresize

pvresize /dev/sda3
lvextend -l +100%FREE -r /dev/mapper/centos-root

Resizing the Linux Root Partition in a Gen2 Hyper-V VM
[/code]
[code="bash"]

/home/arcsight/arcsight_services_status.sh

/opt/arcsight/connector/replay_pd/current/bin/arcsight agents

/opt/arcsight/connector/replay/current/bin/arcsight agents

/sbin/service arcsight_services start

/sbin/service arcsight_services start manager

/sbin/service arcsight_services stop

/sbin/service arcsight_services stop manager

tail -f /opt/arcsight/var/logs/manager/default/server.std.log

/opt/arcsight/manager/bin/arcsight deploylicense

[/code]
Installations Options
0- ArcSight Content Management - This package contains resources to track content that is being managed across multiple ESM systems.
1- ArcSight ESM HA Monitoring - This package contains resources to track High Availability (HA) status and changes.
2- ArcSight Transformation Hub Monitoring - This package contains resources for monitoring Transformation Hub.
3- Security Threat Monitoring - This package contain default security threat monitoring content.
4- Threat Intelligence Platform - This package contains default content for threat intelligence platform.

Install ArcSight Console

  • Download software

Tune BIOS

  1. DisableHyperThreading.This setting exists on most server class processors (for example, Intel processors) that support hyper threading. AMD processors do not have an equivalent setting.
  2. DisableIntelVT-d.This setting is specific to Intel processors and is likely to be present on most recent server class processors. AMD processors have an equivalent setting called AMD- Vi.
  3. SetPowerRegulatortoStaticHighPerformance.This setting tells the CPU(s) to always run at high speed, rather than slowing down to save power when the system senses that load has decreased. Most recent CPUs have an equivalent setting.
  4. SetThermalConfigurationtoIncreasedCooling.This setting increases the server fan speed to avoid issues with the increased heat that results from constantly running the CPU(s) at high speed.
  5. EnabletheMinimumProcessorIdlePowerPackageStatesetting.This setting tells the CPU not to use any of its C-states (various states of power saving in the CPU).
  6. SetPowerProfiletoMaximumPerformance. This setting results in the following changes:
    • QPI power management (the link between physical CPU sockets) is disabled.
    • PCIe support is forced to Gen 2.
    • C-states are disabled.
    • Lower speed settings on the CPUs are disabled so that the CPUs constantly run at high speed.

Silent Deployment using Terraform

  • https://github.com/mitchese/docker-arcsight-esm
  • https://community.microfocus.com/t5/ArcSight-User-Discussions/esm-silent-install/td-p/1640585

ArcSight SmartConnector Install

//Insure FULL Java version on CentOS
//http://www.java.com/en/download/linux_manual.jsp
[[email protected] ~]$ java -version
openjdk version "1.8.0_181"
OpenJDK Runtime Environment (build 1.8.0_181-b13)
OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)
[[email protected] ~]$

Microfocus has many product lines that is very interesting for cyber security intergrations;

  • Predictive Analytics and Machine Learning
    • Vertica
    • IDOL
    • ArcSight Investigate

Architecture

ESM 101 2020-02-12 11-09-36

ESM 101 2020-02-03 15-35-31

User Roles

ESM_101_7.0P1.pdf (page 14 of 161) 2020-02-03 14-36-18

ESM 101 2020-02-12 12-57-40

ArcSight Connectors

automate the process of collecting and managing logs from any device and in any format through normalization and categorization of logs into a unified format known as Common Event Format (CEF), which is now an industry standard for log format. You can use this unified data for searching, reporting, analyzing or storing logs. ArcSight Connectors also manage ongoing updates, upgrades, configuration changes and administration of distributed deployments through a centralized web-based interface. They can be deployed as software or on an appliance

ESM 101 2020-02-03 15-39-55

ArcSight Connectors helps you with:

  • Scale easily to manage extreme machine data across IT
  • Reduce the cost of handling large volumes of logs and events in various formats
  • Automate the process of managing connectors to collect audit-quality log data
  • Share, upload, or download connectors within your ArcSight community
  • Seamlessly integrate with the ArcSight platform
  • Broadest set of built-in connectors that collect, aggregate, filter, and parse the logs
  • Managing log records in hundreds of different formats from hundreds of vendors
  • Patented technology to normalize and categorize logs that enables full-text English searching on rich metadata
  • High compression of log data up to 10:1 to reduce your storage costs significantly
  • Automate bandwidth management with low footprint

FlexConnector The FlexConnector framework is a software development kit (SDK) that enables you to create your own SmartConnector tailored to the nodes on your network and theirspecific event data. FlexConnector typesinclude file reader, regular expression file reader, time-based database reader, syslog, and Simple Network Management Protocol (SNMP) readers.

Forwarding Connector The Forwarding Connectorsforward events between multiple Managersin a hierarchical ESM deployment, and/or to one or more Logger deployments.

ArcSight Manager

The ArcSight Manager isthe heart of the solution. It is a Java-based server that drives analysis, workflow, and services. It also correlates output from a wide variety of security systems. The Manager writes eventsto the CORR-Engine asthey stream into the system. Itsimultaneously processesthem through the correlation engine, which evaluates each event with network model and vulnerability information to develop real-time threatsummaries. ESM comes with default configurations and standard foundation use cases consisting of filters, rules, reports, data monitors, dashboards, and network modelsthat make ESM ready to use upon installation

CORR-EngineStorage

The Correlation Optimized Retention and Retrieval (CORR) Engine is a proprietary data storage and retrieval framework that receives and processes events at high rates, and performs high-speed searches

Security Use Case and Activate Framework Marketplace

ArcSightActivate Framework

ArcSight Activate Framework is a modular content development framework that allows you to implement ArcSight SIEM quickly and effectively. The framework provides a standard way of creating content. Standardized content means new analysts and engineers can easily review and understand existing content reducing the ramp-up time for new employees. It also opens up the possibility of sharing content with other ArcSight users. Best of all, the base content has been created from 10 years of experience implementing ArcSight in thousands of environments. What does this mean? It is proven and it works! ArcSight Activate Framework makes implementing SIEM easy. It helps you with:

  • Deploy modular content and standardized use cases to implement ArcSight quickly and effectively in your environment with minimal setup required.
  • Enable inexperienced users to create content quickly. Content created is easier to understand reducing training and maintenance costs.
  • Provide a standardized approach to creating content that can be shared between ArcSight installations and within the community to easily keep up on the latest IT security threats. This results in a robust SIEM that is easier to set up and maintain.
  • Leverage proven use cases developed by ArcSight SIEM experts to provide a robust implementation to increase your effectiveness and deployment success.

Downloads 309

  • https://h41382.www4.hpe.com/gfs-shared/downloads-309.pdf

Interactive Discovery

ArcSight Interactive Discovery (AID) is a separate software application that augments Pattern Discovery, dashboards, reports, and analytical graphics. AID provides enhanced historical data analysis and reporting capabilities using a comprehensive selection of pre-built interactive statistical graphics. You can use AID to: l Quickly gain visibility into your complex security data l Explore and drill down into security data with precision control and flexibility l Accelerate discovery of hard-to-find eventsthat may be dangerous l Presentstate of security in compelling visualsummaries l Build a persuasive, non-technical call to action l Prove IT Security value and help justify budgets

Pattern Discovery

Pattern Discovery can automatically detectsubtle, specialized, or long-term patternsthat might otherwise go undiscovered in the flow of events. You can use Pattern Discovery to: l Discover zero-day attacks—Because Pattern Discovery does not rely on encoded domain knowledge (such as predefined rules or filters), it can discover patternsthat otherwise go unseen, or are unique to your environment. l Detect low-and-slow attacks—Pattern Discovery can process up to a million eventsin just a few seconds(excluding read-time from the disk). This makes Pattern Discovery effective to capture even low-and-slow attack patterns. l Profile common patterns on your network—New patterns discovered from current network traffic are like signaturesfor a particularsubset of network traffic. By matching against a repository of historical patterns, you can detect attacksin progress. The patterns discovered in an event flow that either originate from or target a particular asset can be used to categorize those assets. For example, a pattern originating from machinesthat have a back door (unauthorized program that initiates a connection to the attacker) installed can all be visualized as a cluster. If you see the same pattern originating from a new asset, it is a strong indication that the new asset also has a back door installed. l Automatically create rules—The patterns discovered can be transformed into a complete rule set with a single mouse click. These rules are derived from data patterns unique to your environment, whereas predefined rules must be generic enough to work in many customer environments. Pattern Discovery is a vital tool for preventive maintenance and early detection in your ongoing security management operations. Using periodic, scheduled analysis, you can always be scanning for new patterns over varying time intervalsto stay ahead of new exploitative behavior

Logger ArcSight Logger is an event data storage appliance that is optimized for extremely high event throughput. Loggerstoressecurity events on board in compressed form, but can alwaysretrieve unmodified events on demand for historical analysis-quality litigation data. Logger can be deployed stand-alone to receive eventsfrom syslog messages or log files, or to receive eventsin Common Event Format from SmartConnectors. Logger can forward selected events assyslog messagesto ESM. Multiple Loggers work together to scale up to support high sustained input rates. Event queries are distributed across a peer network of Loggers.

Content, Solutions, and CIPs for ESM and Logger

ArcSight ESM Compliance Insight Package for the Payment Card Industry (PCI) version 4.1 is now generally available. It can be downloaded by licensed customers from the HP support web site. The solution guide and release notes can be found here.

What’s New?

ESM Compliance Insight Package for PCI 4.1 contains the following important updates:

  • Support for PCI requirements specified in Payment Card Industry Data Security Standard 3.2 (PCI DSS 3.2)
  • Support for logs generated by applications subject to Payment Application Data Security Standard 3.2 (PA DSS 3.2)

About ESM Compliance Insight Package for PCI:

The ESM Compliance Insight Package for PCI provides a system of reports and real-time checks specifically designed to monitor systems that contain cardholder data, manage vulnerability and access control, monitor networks, and maintain security policies to help demonstrate to stakeholders and auditors that the controls over your company’s credit card data systems expose little or no risk.

  • https://community.microfocus.com/t5/Content-Solutions-and-CIPs-for/tkb-p/sol

Resources

ESM uses objects called resources to manage event-processing logic. A resource defines the properties, values, and relationships used to configure the functions that ESM performs. Resources can also be the output of such a configuration (such as archived reports, or Pattern Discovery snapshots and patterns).

ESM has more than 30 different types of resources and comes with hundreds of these resources already configured to give you functionality as soon as the product is installed. These resources are presented in the Navigator panel of the ArcSight Console.

Modeling Resources “The Network Model” on page 120 enables you to build a businessoriented view of data derived from physical information systems. These distinctions help ESM to clearly identify events in your network, providing additional layers of detail for correlation. “The Actor Model” on page 146 creates a real-time user model that maps humans or agents to activity in applications and on the network. Once the actor model is in place, you can use category models to visualize relationships among actors, and correlation to determine if their activity is above board. l Assets l Asset Ranges l Asset Categories l Zones l Networks l Customers l Vulnerabilities l Locations l Actors l Category Models

Correlation Resources Correlation is a process that discovers the relationships between events, infers the significance of those relationships, prioritizes them, then provides a framework for taking action. l Filters l Rules l Data Monitors l Active Lists l Session Lists l Integration Commands l Pattern Discovery

Monitoring and Investigation Resources Active channels and dashboards are tools that monitor all the activity that ESM processes for your network. Each of these views enables you to drill down on a particular event or series of events in order to investigate their details. Saved searches are those you run on a regular basis. They include query statements, the associated field set, and a specified time range. Search filters contain only the query statements. You define and save searches and search filters in the ArcSight Command Center, and export these resources as packages in the ArcSight Console. l Active Channels l Field Sets l Saved Searches and Search Filters l Dashboards l Query Viewers

Workflow and User Management Resources Workflow refers to the way in which people in your organization are informed about incidents, how incidents are escalated to other users, and how incident responses are tracked. l Annotations l Cases l Stages l Users and User Groups l Notifications l Knowledge Base l Reference Pages

Reporting Resources Reporting resources work together to create batch-oriented functions used to analyze incidents, find new patterns, and report on system activity. l Reports l Queries l Trends l Templates l Focused Reports

Administration Resources Administration resources are tools that manage ESM’s daily maintenance and long-term health. l Packages l Files l Storage and storage volumes l Retention periods

Standard Content Standard content is a series of coordinated resources that address common enterprise network security and ESM management tasks. Many of these resources are installed automatically with ESM to provide essential system health and status operations. Others are presented as install-time options organized by category. l ArcSight Administration l ArcSight System

Content Synchronization and Management Content synchronization provides the ability to publish content from one ESM instance to multiple ESM instances. Synchronization is managed through the creation of supported packages, establishment of ESM subscribers, and scheduling the publication of content. Packages

Normalising Event Data

Normalize meansto conform to an accepted standard or norm. Because networks are heterogeneous environments, each device has a different logging format and reporting mechanism. You may also have logsfrom remote sites where security policies and procedures may be different, with different types of network devices, security devices, operating systems and application logs. Because the formats are all different, it is difficult to extract information for querying without normalizing the eventsfirst. The following examples are logsfrom differentsourcesthat each report on the same packet traveling acrossthe network. These logsrepresent a remote printer buffer overflow that connectsto IIS servers over port 80.

Check Point:

“14” “21Nov2016” “12:10:29” “eth-s1p4c0” “ip.of.firewall” “log” “accept” “www-http” “192.0.2.0” “192.0.2.1” “tcp” “4” “1355” “” “” “” “” “” “” “” “” “” “firewall” “len 68”

Cisco Router:

Nov 21 15:10:27: %SEC-6-IPACCESSLOGP: list 102 permitted tcp 192.0.2.0(1355) -> 192.0.2.1(80), 1 packet Cisco PIX: Nov 21 2016 12:10:28: %PIX-6-302001: Built inbound TCP connection 125891 for faddr 192.0.2.0/1355 gaddr 192.0.2.1/80 laddr 10.0.111.22/80

Snort:

[**] [1:971:1] WEB-IIS ISAPI .printer access [**] [Classification: Attempted Information Leak] [Priority: 3] 11/21-12:10:29.100000 192.0.2.0:1355 -> 192.0.2.1:80 TCP TTL:63 TOS:0x0 ID:5752 IpLen:20 DgmLen:1234 DF ***AP*** Seq: 0xB13810DC Ack: 0xC5D2E066 Win: 0x7D78 TcpLen: 32 TCP Options (3) => NOP NOP TS: 493412860 0 [Xref => http://cve.mitre.org/cgi-bin/cvename.cgi?name=CAN2001-0241] [Xref => http://www.whitehats.com/info/IDS533]

In order to productively store this diverse data in a common data store, SmartConnectors evaluate which fields are relevant and arrange them in a common schema. The choice of fields are content driven, ESM 101 Chapter 4: Data Collection and Event Processing Micro Focus ESM (7.0 Patch 1) Page 31 of 161 not based on syntactic differences between what Checkpoint may call target address and what Cisco calls destination address. To normalize, SmartConnectors use a parser to pull out those valuesfrom the event and populate the corresponding fieldsin the schema. Here is a very simple example of these same alerts after they have been normalized.

ESM 101 2020-02-12 11-29-43

Time stamp

Another factor in normalization is converting timestampsto a common format. Since the devices may all use different time zones, ESM normalization convertsthe timestampsto UTC (GMT).

Event Severity

During the normalization process, the SmartConnector collects data about the level of danger associated with a particular event asinterpreted by the data source that reported the event to the connector. These data points, device severity and agentseverity, become factorsin calculating the event’s overall priority described in “Evaluate the Priority Formula” on page 41.

Device severity capturesthe language used by the data source to describe itsinterpretation of the danger posed by a particular event. For example, if a network IDS detects a DHCP packet that does not contain enough data to conform to the DHCP format, the device flagsthis as a high-priority exploit.

Agent severity is the translation of the device severity into ESM-normalized values. For example, Snort uses a device severity scale of 1-10, whereas Checkpoint uses a scale of high, medium and low. ESM normalizesthese valuesinto a single agentseverity scale. The default ESM scale is Low, Medium, High, and Very High. An event can also be classified as AgentSeverity Unknown if the data source did not provide a severity rating.

Event Categories

Like the logsthemselves, differentsecurity devices also include a model for describing the characteristics of the eventsthey process. But no two devices or vendors use the same eventcharacteristic model. To solve this problem, ArcSight has also developed a common model for describing events, which enables you to understand the realsignificance of a particular event asreported from different devices. This common model also enables you to write device-independent content that can correlate events with normalized characteristics. This model is expressed as event categories, and the SmartConnector assignsthem using default criteria, which can be configured during connectorsetup. Event categories are a series of six criteria that translate the core meaning of an event from the system that generated it into a common format. These six criteria, taken individually or together, are a central tool in ESM’s analysis capability.

ESM 101 2020-02-12 11-41-23ESM 101 2020-02-12 11-40-53

Correlation is a four-dimensional processthat draws upon the network model, the priority formula, and optionally, Pattern Discovery to discover, infer meaning, prioritize, and act upon eventsthat meet specific conditions. For example, varioussystems on a network may report the following events: l UNIX operating system: multiple failed log-ins l IDS: Attempted brute force attack l Windows operating systems: multiple failed log-ins A correlation rule putsthese data pointstogether and detectsfive or more failed log-insin a oneminute period targeting the same source. Based on these facts, this combination of eventsis considered an attempted brute force attack. The Windows operating system next reports a successful log-in from the same source. The attempted brute force attack followed by a successful login from the same source elevatesthe risk that the attack may have been successful. To verify whether an attack wassuccessful, you can analyze the volume of traffic going to the Windows target. In this case, a sudden spike in traffic to thistarget can verify that a brute force attack was successful. ESM’s correlation tools use statistical analysis, Boolean logic, and aggregation to find events with particular characteristics you specify. Rules can then take automated action to protect your network.

Sizing

  • Configuring the HPE Proliant DL380 Gen9 24-SFF CTO Server as a Vertica Node – https://www.vertica.com/kb/Configuring-the-Proliant-DL380-Gen9-24-SFF-CTO-Server-as-a/Content/Hardware/Configuring-the-Proliant-DL380-Gen9-24-SFF-CTO-Server-as-a.htm
  • Recommendations for Sizing Vertica Nodes and Clusters – https://www.vertica.com/kb/Recommendations-for-Sizing-Vertica-Nodes-and-Clusters/Content/Hardware/Recommendations-for-Sizing-Vertica-Nodes-and-Clusters.htm

EVENT BROKER and KAFKA

  • https://sookocheff.com/post/kafka/kafka-in-a-nutshell/
  • https://www.slideshare.net/rahuldausa/introduction-to-kafka-and-zookeeper
  • https://www.youtube.com/watch?v=gFul1Gw0CjA
  • https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines

Reference

  • ArcSight ESM documenation Binder
    • https://community.microfocus.com/t5/ArcSight-User-Discussions/ESM-7-3-Documentation-Consolidated/m-p/2817070
  • ArcSight Product Documentation – https://community.microfocus.com/t5/ArcSight-Product-Documentation/ct-p/productdocs
  • SmartConnector Configuration Guide
    • https://community.microfocus.com/t5/ArcSight-Connectors/SmartConnector-Config-Guides-7-14-3-8270-0/ta-p/2769113
  • ArcSight ESM 101 – ESM_101_7.0P1
  • ArcSight in 4 hours – https://community.microfocus.com/t5/ArcSight-User-Discussions/ArcSight-ESM-in-4-Hours/m-p/1588067
  • ArcSight Resources – https://community.microfocus.com/t5/ArcSight-User-Discussions/ArcSight-Videos-resources-locations-and-links/td-p/1501953
  • HP ArcSight Logger in 2 Hours – HP Arcsight Logger
  • NetFlow – https://community.microfocus.com/t5/ArcSight-User-Discussions/ArcSight-and-NetFlow/td-p/1517147

ArcSight is developing Open and Extensible integration with BigData Analytics Kafka and Hadoop technology;

Tools

  • ArcSight-Sysmon-FlexConnector – https://github.com/S3COPS
  • SQRRL Threat Hunting – https://marketplace.microfocus.com/arcsight/content/sqrrl-threat-hunting-arcsight
  • ArcSight Ideas Exchange – https://community.microfocus.com/t5/ArcSight-Idea-Exchange/idb-p/ArcSightIdeas?utm_campaign=00164298

Понравилась статья? Поделить с друзьями:
  • Стили руководства организацией кратко
  • Паглюферал для кошек инструкция по применению
  • Инструкции по тб для учащихся школы скачать 2022
  • Стиральная машинка беко 5 кг инструкция время стирки
  • Ночная сказка мануал женского здоровья