Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OS Detection CPU Performance Issues #2050

Open
fyodor opened this issue May 21, 2020 · 2 comments
Open

OS Detection CPU Performance Issues #2050

fyodor opened this issue May 21, 2020 · 2 comments
Labels

Comments

@fyodor
Copy link
Member

fyodor commented May 21, 2020

As Nmap's OS detection system has expanded over the years, to the point it now has more than 5,600 fingerprints, some of the algorithms are not scaling as well as they could be and performance has bogged down in some cases. We received a report recently with CPU graphs showing that OS detection seemed to be by far the most CPU intensive part of their scan. @dmiller-nmap has at least partially addressed this already, but I'm creating this issue to publicly document it and the fix.

@fyodor fyodor added the Nmap label May 21, 2020
@fyodor
Copy link
Member Author

fyodor commented May 21, 2020

I'm leaving this open in case Dan is working on other improvements to this as well, but here are some notes he sent with an initial fix:

a) The CPU usage increases during OS fingerprinting because of the work of comparing each observation fingerprint with each print in the database, which currently contains 5600+ fingerprints of 119 tests each.

b) The time during which no packets are being sent appears to correspond with the fingerprint matching CPU spike. During this phase, Nmap is not doing any network task, but is instead processing results and outputting them.

The period immediately before this shows relatively little network utilization, which is because of the more conservative timing settings that OS scanning uses; if a probe is dropped, the entire fingerprint must be reattempted, so we have less tolerance for network drops than during port scan. Still, we are always open to opportunities to improve performance, so we will look into improving throughput during OS scanning.

Finally, in response to this report I profiled Nmap during an OS detection scan and found that nearly half the CPU time was being spent in one function, expr_match(), which is used to match an observed value with one from the database. I found I was able to considerably improve this function by removing unused functionality and avoiding a string copy operation, which reduced overall CPU time spent on OS matching by 50%. Other improvements may occur prior to next release, but for now, this change can be seen here: df1874c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants
@fyodor and others