Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: enable Nmap to scan a different set of ports on some hosts than on others #1217

Open
dmiller-nmap opened this issue May 15, 2018 · 8 comments

Comments

@dmiller-nmap
Copy link

NOTE: This is most likely a wontfix issue, as it would require lots of major changes to how Nmap does things. This issue entry is mostly for discussion of the idea and alternatives.

The feature

Many users have asked for capability for Nmap to scan differing sets of ports on separate targets within a single scan. This capability is described in a couple of different ways, depending on the use case:

  1. The user has the results of a previous scan and wants to verify if the opened ports are still open.
  2. The user has a list of hostnames and port numbers and wants to run version detection or a NSE script against each of them.

The common feature of these use cases is that some port numbers will be scanned for some targets, but not for all targets. Regardless of input mechanism (usually described as from a file, either one target & port per line or previous Nmap output), the result is the same: not all ports are scanned on all targets.

The challenges

The primary difficulty here is that Nmap's core scan engine, ultrascan, takes one set of ports and one list of targets as input. It would have to be reworked generally to accept per-target port lists, and the data structures would have to be modified. The port lists are not small (see the struct scan_lists structure; each of the lists pointed to is heap-allocated 64 KB) when considering duplicating them for each target.

An additional difficulty is clarity in output. With current Nmap output, one can look at the invocation (or the list of ports scanned in the XML or verbose Grepable formats) to see which ports were scanned for any target; if the "Not shown: 998 closed ports" line shows, you can subtract the 2 shown ports and arrive at which ports are closed. If the set of scanned ports is different for each target, we will have to update the output to include the list of scanned ports for every target.

The alternatives

A big reason to avoid the difficulty of implementing this is that in most cases described, simply scanning the "extra" ports is not that much of an additional burden. The default Nmap scan is 1000 ports per host, and it finishes fairly quickly for most targets, simply because most networks only have a small number of those ports open. So using some other means (command line processing, usually) to extract the list of open port numbers from the desired input will nearly always yield far fewer than 1000 ports, meaning shorter scan times overall.

Another point to consider is that in addition to some previously open ports being discovered as closed (the reason given for "verifying old scans"), it is just as likely that some previously closed ports will have become open, and limiting a scan to the particular ports that were previously open on each target will miss these.

A final alternative is to perform post-processing on the original scan output (or use the reverse-index NSE script) to get a list of targets for each port number, then run a series of single-port scans to arrive at the desired results. This will usually result in fewer, simpler scans than scanning all the previously-open ports on a single target at a time, and can be faster, too, since the scan load is spread across many targets at once.

@roycewilliams
Copy link

roycewilliams commented May 15, 2018

Very thorough and insightful response - appreciate the effort, and totally understood that it may be a wontfix.

For the first two alternatives - the "not much of an additional burden to scan them all again" argument, and the "you might miss some" argument - I'd like to argue that there are other use cases in play for some users. Specifically:

  • It may sometimes be the noise / detectability of the scan that the user wants to minimize.

  • It may not be "again". 😄 For some use cases, there is value in assembling a list of targets & ports to be used for an initial scan.

The workaround - lots of single-port scans - would be the same for these, of course. :) But it may at least slightly shift the value proposition of bringing such functionality into nmap itself. :)

Also, naively: instead of modifying ultrascan, could a different structure entirely be invoked when importing this kind of target set?

@dmiller-nmap
Copy link
Author

@roycewilliams Yes, I hadn't considered that use case: validating external results

Regarding using a different scan engine besides ultrascan: yes, there are a few other parts of Nmap that are built as separate scan engines, namely idle scan and FTP bounce scan. As a whole, though, the goal of Nmap's developers has been to unify scan engines into ultrascan, since it has some excellent properties like global congestion control and tunable timing that are tricky to work into new code.

@djcater
Copy link

djcater commented May 24, 2018

I've had situations where I've wanted this feature, but most of the time I've managed to work around it.

Let's say you've run an asynchronous port scanner such as masscan at a fixed rate across 1024 IP addresses (/22) with all 65536 TCP ports. Assume that in this scenario you don't mind sacrificing some accuracy for quicker results.

You then know (roughly) what ports are open on what IP addresses.

You could then run Nmap only against the specific set of open ports per IP address (what this feature request is about).

Instead, I just get the union of all open port numbers, and then run Nmap with that set of ports against the full range again (with service detection, version detection, scripts etc.). Usually, that set is less than 100 unique port numbers, and so the Nmap port scanning phase finishes fairly quickly. For me, this has worked well enough that I don't really feel the need for this feature anymore.

@roycewilliams
Copy link

@djcater Totally - that's an excellent approach for some of the use cases. But since a masscan against all ports isn't exactly "minimal noise" ;), and since that technique makes the nmap phase effectively a rescan, it's less of a match for the use case points that I mention in my follow-up note above.

@dmiller-nmap
Copy link
Author

Perl script to accumulate unique ports and targets from an input file and launch one nmap scan against all of them:

perl -lanE'END{$,="\n";open$i,">ips";say$i keys%h;exec"nmap -iL ips -p".join(",",keys%p)}$h{$F[0]}=$p{$F[1]}=1' input.txt

@roycewilliams
Copy link

That's not a Perl script - that's a Perl one-liner that wants to be a script someday. ;)

Kidding aside - interesting! This reduces the scan to a superset of all ports seen for any IP in the list? If so, that's a useful middle ground on the way to the desired use case.

What's the expected input format?

@dmiller-nmap
Copy link
Author

@roycewilliams Yes, that's what it does. Input is whitespace-separated IP and port number, one per line. It was a bit of a joke in response to some folks elsewhere recommending 15-line bash scripts to launch one scan per IP/port pair, which would of course take much too long.

@dmiller-nmap
Copy link
Author

A patch has been proposed for this feature on the Nmap-dev mailing list: https://seclists.org/nmap-dev/2019/q2/2

I have not reviewed it yet, but if anyone is interested in trying it out I would love to hear their thoughts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants