A malware policy is a set of rules that scans the traffic across your network, and determines how to respond to malware detected in the network. An effective policy is customized to the network environment being monitored.
A Trellix malware policy is a set of rules that define the traffic to be scanned for the detection of malware and how you want to respond if the malicious activity is detected. Creating a policy enables you to define an environment to protect by the different operating systems, applications, and protocols in your network. These parameters, or rules, relate to all of the attacks defended against by Trellix IPS.
.jpg "11_1M10-AdvancedMalwareDetection_GUID-DFF22EA5-1CAA-4BB1-A74E-D08DC0D8ED0F.jpg")
A malware policy has various components. The malware policy allows you to select the Protocols to Scan. You can select the protocol streams which are monitored by the Sensor according to the configured malware policy. The Sensor can extract files from the HTTP, FTP, and SMTP traffic for scanning.
In general, when a user downloads a file, it might either be a file that was downloaded as a complete file or one that was downloaded as many segments and then reassembled to form the original file. Files are, at times, split by browsers or download managers to speed up the download. The Sensor can scan and analyze files that are downloaded as a complete file or as many segments.
Specific File Type in the HTTP, FTP, and SMTP data streams can be scanned, selected based on the kind of traffic your network experiences. Various Malware Engines are supported to scan the selected file types in the network traffic. You can configure one or more supported engines for a specific file type.
Note
Blocking malware over FTP is not always possible. For blocking malware, the Sensor needs to know the file size or the end of the file. If file size is not known when using FTP utility, the Sensor raises an alert with Inconclusive results in the Manager when FTP file blocking/alert/tcp-reset option is enabled in the malware policy. If the file size is known, the file transfer is blocked and the Sensor raises an alert with the result as Attack Blocked in the Manager.
There is, however, an exception to this case. If the file size is not known but end of file is read by Trellix Advanced Research Center, the file transfer is blocked in that case, and Attack Blocked is displayed in the alert generated in the Manager.
Trellix IPS performs malware analysis on APK files in the following sequence:
NS-series: Threat Feed / Local Block List → TIE/GTI File Reputation → Trellix IPS Analysis → Gateway Anti-Malware → IVX → Trellix Intelligent Sandbox → NTBA
Virtual IPS: Threat Feed / Local Block List → TIE/GTI File Reputation → Trellix IPS Analysis (PDF/Flash files only) → IVX → Trellix Intelligent Sandbox → NTBA
Trellix IPS performs malware analysis on Executables and PDF files in the following sequence:
NS-series: Threat Feed / Local Block List → TIE/GTI File Reputation → Trellix IPS Analysis (PDF/Flash/Microsoft Office files) → Gateway Anti-Malware → IVX → Trellix Intelligent Sandbox → NTBA
Virtual IPS: Threat Feed / Local Block List → TIE/GTI File Reputation → Trellix IPS Analysis (PDF/Flash/Microsoft Office files) → IVX → Trellix Intelligent Sandbox → NTBA
After the scanning is complete, these engines report a certain confidence level for the scanned file. The confidence level is based on the specificity and severity of the malware and is indicative of the extent to which the file is infected, for example, a high confidence level indicates a high probability of the file being infected. The Action Thresholds are set to be triggered based on the confidence level returned. Since, dynamic analysis is a time taking process, there is a need to carefully employ this process for improved user experience. Trellix IPS submits files to Trellix Intelligent Sandbox for dynamic analysis only if other engines that are enabled report back a malware confidence lower than medium.
You can remediate the threat through configured response actions like blocking and quarantining infected hosts, stopping malicious file download and identifying hosts with a high threat score through the dashboard. You can prevent the malware from reaching the host by blocking it or sending TCP resets. You can analyze malware using the malware dashboards. The malware dashboards allow you to drill down into each piece of detected malware. The Malware Files page provides you with more details of the detected malware. You can also save the malicious file and submit it for analysis.
The supported parameters for a malware policy are:
File Type: Executables, Microsoft Office files, PDF files, compressed files, Android application package, Java archive, flash files and script files.
Malware Engines: Threat Feed / Local Block List, GTI File Reputation, Trellix IPS Analysis, Skyhigh Gateway Anti-Malware (GAM), Intelligent Virtual Execution (IVX), and Trellix Intelligent Sandbox.
Action Thresholds: Alert, Block, Send TCP Reset, Add to Block list, and Save File.
Consider the following example.
The Sensor is deployed in the inline mode. Monitoring ports G0/1 and G0/2 inspect traffic flowing between router A and router B.
Create an Advanced Malware policy, Malware Policy_1 with the following PDF malware rule enabled for SMTP traffic and push the policy to the Sensor.
File Type: PDF
Malware Engine: Trellix IPS Analysis
Action Thresholds: For Very low, low and medium confidence level returned, the configured action is to generate an Alert.
For High and Very high confidence levels, the configured response action is to Block and Send TCP Reset.
Configure the malicious file to be saved for analysis.
Assign the policy to both inbound and outbound traffic and to the available interfaces (in this case to ports G0/1 and G0/2), according to your requirement.
Advanced malware detection using Trellix IPS Analysis.png "GUID-22B1411F-3999-47E0-8622-59D69315C214-low.png")
The PDF files are extracted from the SMTP data streams and scanned by the PDF-JavaScript component of the Trellix IPS Analysis engine. The engine detects a Base64 encoded PDF file.
Based on the severity and the extent to which the file is infected the Trellix IPS Analysis engine returns a confidence level. In this case, the confidence level returned is high. An alert is raised in the Attack Log.
As defined by the malware policy, the response action is triggered for a high confidence level. The detected malware infected data packets are dropped. Thus, malicious files are prevented from reaching the host.
You can drill down and analyze the infected hosts and the malware details through the various dashboards.
Consider some variation to the above example and have multiple engines configured, namely, Skyhigh Gateway Anti-Malware, Trellix IPS Analysis, and TIE / GTI File Reputation. Multiple engines report varying confidence levels; Trellix IPS Analysis and TIE / GTI File Reputation report a low confidence level while Gateway Anti-Malware reports a high confidence level. In such a scenario, the highest confidence level returned is considered by the Sensor when evaluating its response action. In this case, based on the high confidence level returned by the NTBA engine, traffic is blocked.
.png "GUID-34A23C02-C23C-4345-8DAF-BA66E81C9FDD-low.png")
Trellix advises you to create multiple, specific policies that focus on the specific needs of unique zones in your network, rather than a one-size-fits-all policy for the entire network.
TIE / GTI File Reputation
Trellix IPS integrates with Threat Intelligence Exchange (TIE) and Trellix GTI File Reputation. While Trellix GTI is a cloud‑based service that provides real‑time protection from malicious file downloads, Threat Intelligence Exchange is an enterprise repository for file reputation which is provided by several individual security products.
For more information on both these products, see Trellix Intrusion Prevention System Integration Guide.
Sources, such as Intelligent Sandbox and Trellix GTI, provide file reputation for several file hashes to Threat Intelligence Exchange. You, as a security administrator, have the discretion to override this file reputation with an enterprise specific reputation to suit your environment. This management is carried out through the ePO console. In future, when the same file hash is detected by the Sensor, it queries Threat Intelligence Exchange which responds with the customized file reputation. You then have the ability to make a decision based on that file reputation response.
GTI File Reputation scans the selected file type for potential malware including encoded files. The Sensor creates a fingerprint (MD5 hash value) of the file that is seen as potentially malicious, embeds the fingerprint in a standard HTTPS request, and sends it to a Trellix GTI cloud server. The cloud server compares the fingerprint against the threat database maintained by Trellix ARC. If the fingerprint is identified as a known malware, the cloud server notifies the Sensor and it enforces a response action for the malware.
Threat Feed / Local Block List
Trellix IPS also provides users the option to upload custom fingerprints to the Manager, which can be used for file reputation instead of GTI lookups or to complement them. The Manager then sends these fingerprints to the Sensors.
You can add the MD5 or SHA256 hash values of known malicious files to the block list and MD5 or SHA256 hash values of trusted files to the allow list. The Sensor scans the specified file types for potential malware, including encoded files, and compares it with custom fingerprints. This enables the Sensor to immediately identify known malware and also identify the trusted files, without having to analyze such files further. This saves both time and valuable Sensor resources.
Note
The Manager running on 11.1 Update 1 or later releases supports addition of up to 400,000 hash entries (allowed and blocked combined) with a limit of 200,000 per each hash type. Manager prior to 11.1 Update 1 release supports addition of only MD5 hashes up to 100,000 entries (allowed and blocked combined).
Sensors prior to 11.1 Update 1 release do not support SHA256 hashes. The maximum number of hashes supported (cumulative of Blocked Hashes and Allowed hashes) by these Sensors is 100,000.
Sensors running on 11.1 Update 1 or later releases support both SHA256 and MD5 hashes. NS-series Sensors support a maximum of 200,000 hashes for each hash type while Virtual IPS Sensors support a maximum of 100,000 hashes for each hash type. If the Manager has both NS-series and virtual Sensors, entries over 100,000 in each hash type are pushed only to the NS-series Sensors. The push fails on virtual Sensors and a fault is raised which can be noticed in the Faults (Manager → Troubleshooting → Logs → Faults) tab.
In case of heterogeneous environments, if the total MD5 hash entries exceed 100,000:
A limit exceed error can be seen in filetransfer.log during a bulk (full) update
A fault will be raised in the Faults tab and error count will be incremented at the Sensor level during an incremental update. Refer to
show ab statscommand for more information.Note
A Full update is triggered when the total entries are more than 4000; else, an incremental update is triggered to all the Sensors connected to the Manager.
In case MD5 and SHA256 hashes of the same file are added, the MD5 hash takes precedence over SHA256 hash of the file during analysis.
Note
The Sensor checks the allow list before the block list.
If a match is found in the block list, it enforces a response action. It can also be configured per interface. Note the following when custom fingerprints is configured among multiple engines.
When multiple engines are selected, allow list and block list get the highest priority. It is the first engine to scan the file.
If the scanned file finds a match in the allow list, the file is considered to be clean.
If the scanned file finds a match in the block list, none of the other configured engines scans the file. A confidence level of Very High is returned and the configured response action is triggered.
If the scanned file does not find a match in the allow or block list, other applicable engines scan the file. The highest confidence level returned is considered for the response action.
From 11.1 Update 10 release onwards, You can also import IoC (Indicator of Compromise) file hashes from threat feeds configured in the Manager (using Policy → <Admin Domain Name> → Threat Intelligence → Feed Configuration) and create a malware policy for the Sensors to scan those file hashes and raise alerts, if a match is found. The supported file hashes are MD5 and SHA-256. You can exclude specific file hashes while configuring the threat feed to manage false positives. See STIX-based threat intelligence feed support for enhanced protection for more information.
Trellix IPS Analysis
On board the Sensor are various individual engines that make it possible to scan for advanced malware threats emerging through several vectors. These engines, although individual, focus on a common objective and are called Trellix IPS Analysis. Trellix IPS Analysis is capable of analyzing the following files for threats:
JavaScript in PDF files
Shell code in Flash files and those that are embedded in PDF files
Shell code in Microsoft Office files
Trellix IPS Analysis is enabled by default with scanning enabled for Microsoft Office files, PDF files, flash files, and script files.
PDF-JavaScript: Trellix IPS Analysis extracts JavaScript in the PDF and executes the extracted JavaScript. It uses heuristics to detect attacks. If any malicious content is found, the Sensor sends an alert to the Manager. Based on the confidence level returned, the configured response action takes place.
For encrypted PDF files, the engine locates the encryption dictionary and generates the master key.
If the file is in XDP format, the Base64 stream is decoded and the JavaScript extraction process begins.
The PDF engine searches for objects containing embedded files. If embedded files are found, those are extracted. If the extracted embedded file is a PDF file, flash file (.cws, .fws, or .zws), Portable Executable (PE) file, or Microsoft Office file, the JavaScript extraction process begins.
Shell code in flash files: The Sensor supports detection of malicious flash files using heuristic analysis rather than signatures. The Sensor detects various flash exploitation techniques, such as Vector spraying, presence of shell code, and similar exploitation techniques.
The Sensor transfers files to the flash engine after the flash file is successfully extracted. The flash engine scans incoming flash files. If any malicious content is found, the Sensor sends an alert to the Manager. Based on the confidence level returned, the configured response action takes place.
Shell code in Microsoft Office files: Microsoft Office files sent over HTTP, SMTP, and FTP are examined for shell code that might be embedded. Further, if the shell code is XOR encrypted, the Sensor decodes the file. If the file contains shell code it is considered suspicious, and sent to the next engine for further analysis.
Skyhigh Gateway Anti-Malware Engine
The NS-series Sensors and NTBA appliances are equipped with the Skyhigh Gateway Anti-Malware Engine, which consists of the Gateway Anti-Malware DAT and engine, Anti-Virus DAT, and Anti-Malware Engine.
Gateway Anti-Malware Engine operates on several platforms and detects and blocks malware threats — everything from viruses and worms to adware, spyware, and riskware. To further protect end users against emerging malware threats, zero-day threats, and targeted attacks, Gateway Anti-Malware Engine focuses on generic and heuristic detection of malware.
If your deployment consists of an NS-series Sensor and an NTBA appliance, the files are scanned by Gateway Anti-Malware Engine present on the Sensor. The illustration shows you scenarios in which files are sent to Gateway Anti-Malware
The illustration shows you scenarios in which files are sent to Gateway Anti-Malware.
Selection criteria to send files to Skyhigh Gateway Anti-Malware Engine.jpg "11_1M10-Advanced_Malware_Policy_GAM2014_GUID-B15589CE-C87C-47DD-B61B-A51E28DCDBAE_7_en-us_Source.jpg")
The file is scanned by Gateway Anti-Malware Engine which returns results (confidence level) to the Sensor. The Sensor sends the alert to the Manager and the response action configured in the Manager is acted upon.
The Sensor and NTBA appliances are configured to use a specific version of Gateway Anti-Malware Engine. This version depends on the Manager software version you are running to manage Sensors and NTBA Appliances. The table below illustrates different versions of Gateway Anti-Malware Engine that are compatible with different Sensor, Manager, and NTBA versions.
With support for Gateway Anti-Malware 2017 on NS-series Sensors, the engine supports proxy server for GTI integration with file reputation enabled. The Anti-Malware engine version available with Gateway Anti-Malware is 59xx.
Skyhigh Gateway Anti-Malware
Anti-Malware Engine Version
Manager
Sensor
NTBA
2014
5700
NA
NA
9.1.3.3 or later
2015
NA
NA
9.1.3.1 or later
2017
5900
10.1.7.4 or later
10.1.5.3 or later
NA
2019
5900
10.1.7.29 or later
10.1.5.41 or later
NA
2021
5900
10.1.7.55 or later
10.1.5.153 or later
NA
2023
6600
11.1.7.84 or later
11.1.5.84 or later
NA
Intelligent Virtual Execution (IVX)
Intelligent Virtual Execution (IVX) Engine is a signature-less, dynamic analysis engine that inspects suspicious network traffic to identify attacks that evade traditional signature-based and policy-based defenses. The IVX engine detects zero-day, multiflow, and other evasive attacks with dynamic, signature-less analysis in a safe, virtual environment. It stops infection and compromise phases of the cyberattack kill chain by identifying never-before-seen exploits and malware.
Trellix IPS offers integration capability with Trellix Intelligent Virtual Execution - Server and Trellix Intelligent Virtual Execution - Cloud which utilize IVX engine's technology to perform malware analysis.
When you integrate Trellix IPS with IVX, the Sensor initiates a communication channel with the Trellix VX or Trellix IVX Cloud. This channel is open unless the Sensor is down, the Trellix VX (IVX) or Trellix IVX Cloud (IVX Cloud) is down, or you disable the integration. By default, this communication channel is over HTTPS protocol and the IVX and IVX Cloud listen on port 443 which cannot be changed.
The Manager accesses the RESTful APIs of IVX for its communication. When a connection is required, the Manager establishes an HTTPS connection. IVX and IVX Cloud listen on a fixed port number 443 for such connections.
When you integrate Trellix IPS with IVX and the authentication is successful, IVX serves as an additional malware engine for all the supported file types in the Advanced Malware Policies. You can select this engine along with any of the other malware engines.
For more information on IVX integration, see Trellix Intrusion Prevention System Integration Guide.
Trellix Intelligent Sandbox
Trellix Intelligent Sandbox is an on-premise appliance that facilitates detection and prevention of malware. Trellix Intelligent Sandbox provides protection from known, near-zero day, and zero-day malware without compromising on the quality of service to your network users.
The Trellix Intelligent Sandbox solution primarily consists of the Trellix Intelligent Sandbox appliance and its pre-installed software. The Trellix Intelligent Sandbox appliance is available in two models. The low-end model is the ATD-3000. The high-end model is ATD-6000. For complete information on Trellix Intelligent Sandbox, see the Trellix Intelligent Sandbox Product Guide.
You can integrate Trellix Intelligent Sandbox with Trellix IPS. After you integrate, both the Sensor and the Manager communicate with Trellix Intelligent Sandboxseparately to augment your defense against malware. This integration enables you to analyze files, for at least known malware, before they are downloaded into your network. For detailed information on how to configure this integration, see the Trellix Intrusion Prevention System Integration Guide.
Integration with Trellix Intelligent Sandbox.png "GUID-152E4387-FE63-4F8E-B290-2A85C970FEA0-low.png")
When you integrate with Trellix Intelligent Sandbox, Intelligent Sandbox is available as an additional malware engine for all the supported file types in the Advanced Malware Policies. You can select this engine along with any of the other malware engines except NTBA. Because Skyhigh Gateway Anti-Malware Engine is available in both Trellix Intelligent Sandbox and NTBA appliance, you can only select either of these engines for a file type.
Based on the configuration, an inline Sensor detects a file download. If the file is not listed in the block or allow list, it sends a copy of the file to the applicable malware engines, including Trellix Intelligent Sandbox. If Trellix Intelligent Sandbox is able to detect a malware in real time, the Sensor blocks the download. If Trellix Intelligent Sandbox requires more time for analysis, the Sensor allows the file to be downloaded. If Trellix Intelligent Sandbox detects a malware after the file has been downloaded, it informs Trellix IPS, and you can use the Sensor to quarantine the host. Then, you can enforce remediation before allowing the host to come back online. If the same malicious file is downloaded again, the Sensor itself blocks it since it now has the information about that file. For detailed information on how this integration prevents known and unknown malware from entering you network, see the Trellix Intrusion Prevention System Integration Guide.
Note
GTI File Reputation is available both in the Advanced Malware policies in the Manager as well as in Trellix Intelligent Sandbox. However, Trellix recommends that you use the Advanced Malware policy for this feature. The Sensor can respond quicker if it is configured in the Advanced Malware policy because, in this case, it directly communicates with GTI.