Nmap/Short Course/Lecture 3

Main page: Nmap/Short Course

Lab: Nmap/Short Course/Lab 3

Summary and Objective

Summary:

• This 30-minute lecture will introduce more nuanced scanning techniques, delve into operating system identification, and cover Nmap's versatile output options.

Objective:

• To equip students with knowledge of advanced TCP scanning techniques useful for firewall analysis and stealth, the ability to perform and interpret OS detection, and proficiency in managing and utilizing Nmap's various output formats, including scan comparison with Ndiff.

Notes

Recap and Introduction to Advanced Scanning

Quick Recap of Essential Scans

In our previous lecture, we covered fundamental port scanning techniques, including TCP SYN (-sS) and Connect (-sT) scans for identifying TCP services, and UDP scans (-sU) for UDP services. We also emphasized the importance of interpreting port states (open, closed, filtered) and performing service and version detection (-sV) to understand exactly what software is running on discovered ports. These skills form the bedrock of network service enumeration.

Why Advanced Scans? Beyond the Basics

While SYN, Connect, and basic UDP scans are powerful, there are scenarios where more sophisticated techniques are needed. Advanced scans can help in:

• Stealth: Attempting to gather information while minimizing the chances of detection by Intrusion Detection Systems (IDS) or vigilant administrators.
• Firewall Analysis: Understanding firewall rule sets, such as determining if a firewall is stateful or stateless, and identifying which ports are specifically being blocked or allowed.
• Evasion: Bypassing certain types of packet filters or less sophisticated IDS. These techniques often involve sending TCP packets with unusual flag combinations or leveraging indirect scanning methods.

Understanding Target and Firewall Reactions

The effectiveness of advanced scans hinges on how target operating systems and intermediary devices (like firewalls and IDS) respond to non-standard network probes. Different systems adhere to or deviate from TCP/IP RFCs (Request for Comments – the technical standards documents for the internet) in unique ways. Nmap exploits these differences to infer port states or device characteristics. For example, some scans are designed to elicit responses only from closed ports, or to see if a firewall drops certain packets silently versus rejecting them with an error.

Advanced TCP Scan Types

These scans manipulate TCP flags to probe targets in ways that can be less obvious or provide specific insights into filtering mechanisms.

Fin (-sF), Xmas (-sX), and Null (-sN) Scans

These are "stealth" scans because they don't use the SYN flag, which is how firewalls and logging systems typically monitor connection attempts.

Mechanics:

• FIN Scan (-sF): Sends a TCP packet with only the FIN (finish) flag set.
• Xmas Scan (-sX): Sets the FIN, PSH (push), and URG (urgent) flags simultaneously (lighting the packet up like a Christmas tree).
• Null Scan (-sN): Sends a TCP packet with no flags set.

Expected Behavior (RFC 793):

• According to TCP RFC 793, if a port is closed, the target system should respond with a TCP RST (reset) packet.
• If the port is open, the target should drop these malformed packets without a response.

Real world behavior:

• If an RST packet is received, Nmap marks the port as closed.
• If no response is received (after retransmissions), the port is marked open|filtered. This ambiguity means it could be open, or a firewall could be dropping the probe.
• If an ICMP unreachable error (e.g., type 3 code 1, 2, 3, 9, 10, or 13) is received, the port is marked filtered. These scans are generally effective against UNIX-based systems. However, Microsoft Windows systems (and some others like Cisco devices) don't strictly follow RFC 793 in this regard; they tend to send RST packets for both open and closed ports in response to these probes, making these scan types ineffective against them for determining open ports.

# Perform a FIN scan (requires sudo/root)
sudo nmap -sF <target_IP>

# Perform an Xmas scan
sudo nmap -sX <target_IP>

# Perform a Null scan
sudo nmap -sN <target_IP>

[Instructor: Run 'sudo nmap -sF <your_lab_Linux_target_IP_with_known_ports>' and show output]
Starting Nmap X.XX ( https://nmap.org ) at YYYY-MM-DD HH:MM ZONE
Nmap scan report for <target_IP>
Host is up (0.0XXs latency).
Not shown: XXX closed ports
PORT    STATE         SERVICE
22/tcp  open|filtered ssh
80/tcp  open|filtered http
111/tcp closed        rpcbind
Nmap done: 1 IP address (1 host up) scanned in X.XX seconds

ACK Scan (-sA)

The ACK scan is primarily used to map out firewall rule sets and determine if a firewall is stateful or stateless. It does not typically determine if a port is open or closed.

Mechanics:

• Nmap sends TCP packets with only the ACK flag set to the specified ports.

Purpose & Port States:

• If an RST packet is returned, the port is classified as unfiltered. This means the target host received the ACK packet and responded, indicating that a firewall (if present) did not block it.
• If nothing is returned, or if an ICMP unreachable error is received, the port is classified as filtered. This suggests a firewall is likely dropping the ACK packets. A stateful firewall will usually drop unsolicited ACK packets (marking ports as filtered), whereas a simple stateless firewall might pass them through, leading to RSTs from the target host (marking ports as unfiltered).

# Perform an ACK scan (requires sudo/root)
sudo nmap -sA <target_IP>

# ACK scan common web ports to test firewall rules
sudo nmap -sA -p 80,443,8080 <target_IP>

This scan is valuable for reconnaissance against firewalls. For example, if -sS shows a port as filtered, but -sA shows it as unfiltered, it implies a stateful firewall is blocking incoming SYN packets but not necessarily other types.

Idle Scan (-sI zombiehost)

TCP Idle Scan (-sI <zombie host> [options]):

• This is an advanced and truly blind TCP port scanning method. It doesn't send packets to the target from your true IP address. Instead, it leverages an idle "zombie" host on the internet that has a predictable IP fragmentation ID (IPID) sequence. Nmap sends SYN packets to the target, spoofing the zombie's IP. If the target port is open, it sends a SYN/ACK back to the zombie. This causes the zombie's IPID to increment. Nmap then probes the zombie's IPID to see if it changed.
  • Pros: Ultimate stealth regarding the target (it never sees your IP).
  • Cons: Difficult to find suitable zombie hosts (must be idle, predictable IPID), can be slow, and complex to set up correctly. This is more of an expert technique.

This is a very sophisticated scan type, often discussed more in penetration testing courses, but good for students to be aware of its existence.

OS Detection (-O)

Why Detect the Operating System?

Identifying the operating system and its version on a target host is a crucial piece of information for several reasons:

• Vulnerability assessment: Known vulnerabilities are often OS-specific or even version-specific. Knowing the OS helps narrow down potential exploits.
• Tailoring attacks: Payloads or exploits might need to be customized for the target OS.
• Network inventory: Understanding the OS landscape of a network is vital for asset management.
• Policy compliance: Verifying that only authorized operating systems are present on the network.

How Nmap Performs OS Detection

Nmap's OS detection (-O option) is a sophisticated process that involves sending a series of TCP, UDP, and ICMP probes to open and closed ports on the target host. Nmap analyzes the responses to these probes, looking for subtle differences in network protocol implementations across various operating systems. Some of the characteristics Nmap examines include:

• TCP Initial Sequence Number (ISN) sampling and predictability.
• TCP options support and ordering (e.g., Window Scale, Timestamps, Selective ACK).
• IP ID (Identification) field sampling (e.g., all zeros, incremental, random).
• Responses to various ICMP probes.
• Explicit Congestion Notification (ECN) support.
• Characteristics of TCP initial window sizes. Nmap compares the collected "fingerprint" of responses against its extensive database (nmap-os-db) of known OS fingerprints.

Interpreting Results and Options

The output will usually provide a best guess for the OS, often including vendor, OS family, version, and sometimes even device type (e.g., router, printer).

--osscan-guess or --fuzzy: If Nmap can't find an exact match, these options make Nmap try harder and report the closest matches, which can be less accurate.

--max-os-tries <number>: Set the maximum number of OS detection attempts. Nmap might also estimate the system uptime based on TCP timestamp options. If you have a fingerprint for an OS not in Nmap's database, you can submit it to help improve Nmap.

# Perform OS detection (requires at least one open and one closed TCP port)
sudo nmap -O <target_IP>

# Perform OS detection with aggressive guessing
sudo nmap -O --osscan-guess <target_IP>

[Instructor: Run 'sudo nmap -O <your_lab_target_IP_with_open_closed_ports>' and show output]
Starting Nmap X.XX ( https://nmap.org ) at YYYY-MM-DD HH:MM ZONE
Nmap scan report for <target_IP>
Host is up (0.0XXs latency).
Not shown: XXX closed ports
PORT   STATE SERVICE
22/tcp open  ssh
80/tcp open  http
Device type: general purpose
Running: Linux 5.X
OS CPE: cpe:/o:linux:linux_kernel:5.X
OS details: Linux 5.4 - 5.10
Network Distance: X hops
Uptime guess: X.XXX days (since YYYY-MM-DD HH:MM:SS)

Nmap done: 1 IP address (1 host up) scanned in X.XX seconds

Discuss the Device type, Running, OS CPE, and OS details fields. Mention that accuracy can vary and sometimes Nmap might only provide a broad family.

Output Formats & Ndiff

Saving and managing scan results is crucial for analysis, reporting, and tracking changes over time. Nmap offers several output formats.

Common Output Formats

Normal Output (-oN <filespec>): This is the human-readable format you see on the screen by default. It's useful for quick review and manual inspection.

nmap -sS -A -oN normal_scan_results.txt <target_IP>

XML Output (-oX <filespec>): This is a machine-readable format using Extensible Markup Language. It's the most comprehensive and flexible format, ideal for parsing by other tools, scripts, or Nmap GUIs like Zenmap. Most Nmap data can be represented in XML.

nmap -sS -A -oX xml_scan_results.xml <target_IP>

Grepable Output (-oG <filespec>): This format puts all information for a host on a single line, with fields separated by tabs. It's designed to be easily parsed by standard command-line tools like grep, awk, or cut. While simple, it's less comprehensive than XML and sometimes considered deprecated for complex parsing, but still very handy for quick scripting.

nmap -sS -A -oG grepable_scan_results.gnmap <target_IP>

Script Kiddie Output (-oS <filespec>): This format presents results in "l33t sp34k." It's mostly for amusement and not practical for professional use, but good to know it exists.

nmap -sS -A -oS skiddie_scan_results.txt <target_IP>

All Formats (-oA <basename>): This convenient option saves the scan results in Normal (.nmap), XML (.xml), and Grepable (.gnmap) formats simultaneously, all using the provided basename.

nmap -sS -A -oA all_formats_basename <target_IP>
# This creates:
# all_formats_basename.nmap
# all_formats_basename.xml
# all_formats_basename.gnmap

NOTE: XML is generally the preferred format for programmatic interaction and detailed reporting.

Ndiff Utility: Comparing Scans

Ndiff is a tool packaged with Nmap that compares two Nmap XML scan files and highlights the differences. This is extremely useful for:

• Network Monitoring: Running scans periodically and using Ndiff to see what has changed (e.g., new hosts came online, ports opened or closed, services changed versions).
• Auditing: Verifying that security changes have taken effect or that no unauthorized changes have occurred.
• Penetration Testing: Tracking changes on a target network throughout an engagement. Ndiff produces a human-readable text output showing hosts and ports that were added, removed, or changed state/service information between the two scans.

# First scan (e.g., last week)
nmap -sV -oX scan_last_week.xml 192.168.1.0/24

# Second scan (e.g., today)
nmap -sV -oX scan_today.xml 192.168.1.0/24

# Compare the two scans
ndiff scan_last_week.xml scan_today.xml

[Instructor: Prepare two slightly different XML scan files and run 'ndiff scan1.xml scan2.xml' to show example output.]
-192.168.1.25:
-  Host is up
-  Ports:
-    80/tcp   open  http    Apache httpd 2.4.29 ((Ubuntu))
+192.168.1.25:
+  Host is up
+  Ports:
+    80/tcp   open  http    nginx 1.14.0 (Ubuntu)
+    443/tcp  open  ssl/http nginx 1.14.0 (Ubuntu)

192.168.1.50:
   Host is up
   Ports:
-    22/tcp open  ssh     OpenSSH 7.6p1 Ubuntu 4ubuntu0.3
+    22/tcp open  ssh     OpenSSH 8.2p1 Ubuntu 4ubuntu0.1

NOTE: Lines starting with - are from the first file (old), and lines with + are from the second file (new). This shows changes like a new port (443) opening on .25 and the Apache server being replaced by Nginx, and an SSH version update on .50.

Lab

Perform OS detection on key systems in each scenario. Save scans in multiple formats. If previous scans exist (or can be simulated), use Ndiff to identify changes in the Web/Cloud Infrastructure (Scenario 3).

Nmap/Short Course/Lab 3