Thanks for the bug report! It would be helpful to have the core dump file from the crash. Also, can you try using the nmap-service-probes file from 7.40 with Nmap 7.60, and the file from 7.60 with Nmap 7.40, and tell us if either one of them crashes? This will tell us if the problem is with one of the service scan patterns.

Apart from this, the output from this command with -d2 --version-trace --packet-trace could also help identify the problem.

(EDITED to fix link to stackoverflow question regarding OS X core dumps)

Ok, I've successfully reproduced this bug. Whatever the service is, it responds to some probes (tor-versions and RTSPRequest in my test) with HTTP that doesn't contain carriage returns. The pattern that is causing the problem for me is:

match http m|^HTTP/1\.0 \d\d\d (?:(?!\r\n\r\n).)*\r\nServer: WildFly/(\d[\w._-]*)\r\n|s p/JBoss WildFly Application Server/ v/$1/ cpe:/a:redhat:jboss_wildfly_application_server:$1/

Specifically, we're getting into a really deep recursion in trying to match that "any number of single characters not followed by '\r\n\r\n'" group. I don't quite understand why it's recursing; we used this pattern in a lot of places to match "some number of HTTP headers, but not the message body." And it's only affecting some versions of libpcre, including the one Nmap includes in its source tree. Apparently #1108 shows the same crash affecting CentOS.

I'm going to try some things to improve the performance of these match lines, but the better solution might be to upgrade libpcre. Here are some workarounds for now:

1. Use a different libpcre. Build from source, and be sure to install the development headers for libpcre on your system.
2. Comment out the 19 or so matches that use this pattern: sed -i bak '/(?:(?!\\r/ s/^/#/' nmap-service-probes

Thanks for reporting!

Ok, I figured out a good solution for now: we need to make these groups non-greedy.

Our intent was to allow the patterns to match within the headers only, and stop when they get to the end of the headers. This is because the body of a HTTP response could be quite large, and we'd like to fail quickly. But what was actually happening is that the regex engine was seeing the greedy .* and skipping all the way to the end of the (probably large) string, then working backwards to see if it could get a match. This backtracking is what makes regular expressions so expressive, but also what was causing the deep recursion.

To take the example from before, this is the new match line:

match http m|^HTTP/1\.0 \d\d\d (?:(?!\r\n\r\n).)*?\r\nServer: WildFly/(\d[\w._-]*)\r\n|s p/JBoss WildFly Application Server/ v/$1/ cpe:/a:redhat:jboss_wildfly_application_server:$1/

Note the addition of the ? which makes the * non-greedy. Now the engine will progress one byte at a time until it either matches the "Server" part or until it finds the double-CRLF that marks the end of the headers. Watch for this fix in a commit soon.