The new docs.trellix.com features a modernized UI and AI-powered conversational search. Content is currently available in English, with additional languages launching in early November 2026. We hope you enjoy the updated experience.

How the Sensor detects domain name exceptions and C&C server domains?

Prev Next

The high-level flow of how the Sensor detects domain name exceptions and C&C server domains in the DNS traffic is explained in this section.

GUID-3AA470AA-347F-452B-B1AA-DDE7F3C7B373-low.png

For the sake of explanation, assume the following:

  • Inline Sensor monitoring ports are upstream to the local (recursive) name servers. Therefore, all traffic from the name servers pass through the Sensor.

  • You have configured the network of the name server as the inside network. The Sensor inspects DNS response packets for known C&C server domains. So, you must enable the following options on the Advanced Callback Detection tab of Inspection Options policy.

    • Callback Detectors and Heuristic Callback Discovery — Enable this option in the outbound direction.

    • DNS Sinkholing — Enable this option. The Sensor checks the DNS packets for C&C server domains. If the Sensor detects a C&C server domain, it sends a crafted DNS response packet to the corresponding bot.

    • Domain Name Exclusion List Processing — Enable this option. The Sensor checks the DNS packets for user-defined Domain Name Exceptions.

    Note

    Recall that the Sensor inspects the DNS response packet for the C&C server domains. If the Sensor inspects a response packet, you must enable the corresponding feature in the direction of the request packets. In the scenario discussed here, the DNS requests are outbound since the DNS server and clients are defined as inside network. So, for this scenario, you enable the above-mentioned feature for outbound.

  • You have downloaded the latest callback detector in the Manager and deployed it on the Sensor as well.

  1. In step 4 of the above diagram, the Sensor receives the DNS response packet. The Sensor checks if the source or destination IP address in the DNS response is part of the CIDRs Excluded from Advanced Callback Detection list. If true, the Sensor exempts the DNS flow from inspection for callback activities. If not, the Sensor parses the DNS response as follows.

  2. The Sensor checks the domain name in the DNS response against the Callback Detection Exclusions. If present, the Sensor forwards the packet and does not perform any other DNS-based analysis (for FFSN and DGA). However, the Sensor subjects the subsequent L7 traffic to IPS as applicable.

  3. If the domain name is present in the callback detector's C&C server, the Sensor performs the following:

    1. If the DNS response contains A records, the Sensor crafts a DNS response with 127.0.0.1 as the resolved IP address and 720 minutes as the TTL for this record. The Sensor sends this crafted DNS response through the peer monitoring port. When the corresponding host (bot) receives this loopback address as the resolved IP address, it routes the traffic to itself. This way, you can restrict the malicious bot traffic to the infected host. For the next 12 hours, the recursive name server provides this loop back address to any host trying to resolve the corresponding C&C server domain.

      This loopback IP address and TTL are default values. You can configure them in the Protocol Settings page for the Sensor (Devices → <Admin Domain Name> → Devices → <Device Name> → Setup → Advanced → Protocol Settings). So, if you want to redirect the bot traffic to a specific server, you can provide the IPv4 address in the Protocol Settings page with a TTL value of up to 720 minutes. For example, you might want to sinkhole the bot traffic to a Linux server with socat and packet sniffer installed. Redirecting the bot traffic to a separate server enables you to analyze the bot traffic instead of simply sinkholing the traffic.

    2. If the DNS response contains AAAA records, the Sensor crafts a DNS response with ::1 as the IP address with the TTL value as configured in the Protocol Settings page.

      Note

      For AAAA records, you cannot configure a different sinkhole IP address. Only the default ::1 loop-back IP address is allowed.

    If the Sensor is inline between the DNS clients and the recursive name server, the Sensor sends the crafted DNS response to the bot through the monitoring port. Therefore, depending upon the operating system, the bot might or might not cache the sinkhole IP address according to the TTL in the Protocol Settings page.

  4. If the domain is not present in the botnet detector's C&C server, the Sensor parses the DNS response for FFSN and DGA detection. If you configure both FFSN and DGA, the Sensor analysis the DNS packets concurrently for both.

DNS-based detection of excluded and C&C server domains
DNS-based detection of excluded and C&C server domains


When the Sensor detects a C&C server domain, it raises the Callback Detectors: High Confidence C&C Server Name Match alert.

Note the following for interfaces in SPAN and tap modes:

  • You must enable Callback Detectors and Heuristic Callback Discovery in inbound and outbound direction. That is, you must enable C&C server detection in both the directions.

  • The Sensor might not be able to block the DNS traffic related to C&C server domains. As a result, sinkholing of bot traffic might not be possible.

  • Even if an interface is in SPAN or tap mode, it can still reduce the load on the FFSN and DGA heuristic engines by identifying the domain name exceptions and C&C server domains.