Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nsock: engine_iocp cannot handle PCAP read events, may have other structural problems #2126

Closed
dmiller-nmap opened this issue Sep 21, 2020 · 2 comments · Fixed by res0nat0r/nmap#1

Comments

@dmiller-nmap
Copy link

The IOCP Nsock engine for Windows appears to have some problems. We need to go over it carefully and figure out whether everything is correct. First indication was an assertion failure.

Describe the current behavior
Easy reproduction on Windows:

> nping google.com --nsock-engine iocp -dddddddd
Trying to initialize Windows pcap engine
npcap service is already running.
wpcap.dll present, library version: Npcap version 0.9997, based on libpcap version 1.9.1
Nping will send packets at raw ethernet level

Starting Nping 0.7.80 ( https://nmap.org/nping ) at 2020-09-18 18:26 Central Daylight Time
Resolving specified targets...
Determining target 172.217.6.14 MAC address or next hop MAC address...
    > Checking system's ARP cache...
    > Success: Entry found [XX:XX:XX:a2:a2:78]
+-----------------TARGET-----------------+
Device Name:            eth0
Device FullName:        eth0
Device Type:            Ethernet
Directly connected?:    no
Address family:         AF_INET
Resolved Hostname:
Supplied Hostname:      google.com
Target Address:         172.217.6.14
Source Address:         192.168.80.10
Next Hop Address:       192.168.80.1
Target MAC Address:     00:00:00:00:00:00
Source MAC Address:     80:ee:73:f8:f8:48
Next Hop MAC Address:   a0:04:60:a2:a2:78
1 target IP address determined.
libnsock nsock_set_loglevel(): Set log level to FULL DEBUG
libnsock nsock_iod_new2(): nsock_iod_new (IOD #1)
Next target returned by getNextTarget(): Targets[0/1] --> 172.217.6.14

BPF-filter: (not src host 192.168.80.10 and dst host 192.168.80.10) and ((icmp and icmp[icmptype] = 0) or (icmp and (icmp[icmptype] = 3 or icmp[icmptype] = 4 or icmp[icmptype] = 5 or icmp[icmptype] = 11 or icmp[icmptype] = 12)) )
Opening pcap device eth0
libnsock nsock_pcap_open(): PCAP requested on device '\Device\NPF_{CEA181E4-93AB-4FA8-BCDD-F72B397FD50A}' with berkeley filter '(not src host 192.168.80.10 and dst host 192.168.80.10) and ((icmp and icmp[icmptype] = 0) or (icmp and (icmp[icmptype] = 3 or icmp[icmptype] = 4 or icmp[icmptype] = 5 or icmp[icmptype] = 11 or icmp[icmptype] = 12)) )' (promisc=0 snaplen=8192 to_ms=1) (IOD #1)
libnsock nsock_pcap_open(): PCAP created successfully on device '\Device\NPF_{CEA181E4-93AB-4FA8-BCDD-F72B397FD50A}' (pcap_desc=-1 bsd_hack=0 to_valid=0 l3_offset=14) (IOD #1)
Pcap device eth0 open successfully
Next target returned by getNextTarget(): Targets[0/1] --> 172.217.6.14

fillPacket(target=015B2D30, port=0, buff=012EFB8C, bufflen=65535, filledlen=012E8B6C rawfd=-1)
fillPacketICMP(target=015B2D30, buff=012EFB9A, bufflen=65521, filledlen=012E8AF4)
libnsock event_new(): event_new (IOD #1) (EID #13)
libnsock nsock_pcap_read_packet(): Pcap read request from IOD #1  EID 13
libnsock nsock_pool_add_event(): NSE #13: Adding event (timeout in 1000ms)
libnsock nsock_pool_add_event(): PCAP NSE #13: Adding event to PCAP_READ_EVENTS
libnsock event_new(): event_new (IOD #1) (EID #21)
libnsock nsock_pcap_read_packet(): Pcap read request from IOD #1  EID 21
libnsock nsock_pool_add_event(): NSE #21: Adding event (timeout in 1000ms)
libnsock nsock_pool_add_event(): PCAP NSE #21: Adding event to PCAP_READ_EVENTS
libnsock event_new(): event_new (IOD #NULL) (EID #28)
libnsock nsock_timer_create(): Timer created - 1ms from now.  EID 28
libnsock nsock_pool_add_event(): NSE #28: Adding event (timeout in 1ms)
libnsock nsock_loop(): nsock_loop() started (timeout=2ms). 3 events pending
libnsock iocp_loop(): wait for events
libnsock do_actual_pcap_read(): PCAP do_actual_pcap_read TEST (IOD #1) (EID #21)
libnsock do_actual_pcap_read(): PCAP do_actual_pcap_read TEST (IOD #1) (EID #13)
libnsock process_event(): Processing event 28 (timeout in -14ms, done=0)
libnsock process_event(): NSE #28: Sending event
libnsock nsock_trace_handler_callback(): Callback: TIMER SUCCESS for EID 28
nping_event_handler()
nping_event_handler(): Received callback of type TIMER with status SUCCESS
SENT (0.2650s) ICMP [192.168.80.10 > 172.217.6.14 Echo request (type=8/code=0) id=22655 seq=1] IP [ver=4 ihl=5 tos=0x00 iplen=28 id=52976 foff=0 ttl=64 proto=1 csum=0xe856]
0000   a0 04 60 e1 a2 78 80 ee  73 53 f8 48 08 00 45 00  ..`..x..sS.H..E.
0010   00 1c ce f0 00 00 40 01  e8 56 c0 a8 50 0a ac d9  ......@..V..P...
0020   06 0e 08 00 9f 7f 58 7f  00 01                    ......X...
libnsock event_delete(): event_delete (IOD #NULL) (EID #28)
Next target returned by getNextTarget(): Targets[0/1] --> 172.217.6.14

fillPacket(target=015B2D30, port=0, buff=012EFB8C, bufflen=65535, filledlen=012E8B6C rawfd=-1)
fillPacketICMP(target=015B2D30, buff=012EFB9A, bufflen=65521, filledlen=012E8AF4)
libnsock event_new(): event_new (IOD #1) (EID #37)
libnsock nsock_pcap_read_packet(): Pcap read request from IOD #1  EID 37
libnsock nsock_pool_add_event(): NSE #37: Adding event (timeout in 1000ms)
libnsock nsock_pool_add_event(): PCAP NSE #37: Adding event to PCAP_READ_EVENTS
libnsock event_new(): event_new (IOD #1) (EID #45)
libnsock nsock_pcap_read_packet(): Pcap read request from IOD #1  EID 45
libnsock nsock_pool_add_event(): NSE #45: Adding event (timeout in 1000ms)
libnsock nsock_pool_add_event(): PCAP NSE #45: Adding event to PCAP_READ_EVENTS
libnsock event_new(): event_new (IOD #NULL) (EID #52)
libnsock nsock_timer_create(): Timer created - 1000ms from now.  EID 52
libnsock nsock_pool_add_event(): NSE #52: Adding event (timeout in 1000ms)
libnsock nsock_loop(): nsock_loop() started (timeout=1001ms). 5 events pending
libnsock iocp_loop(): wait for events
libnsock do_actual_pcap_read(): PCAP do_actual_pcap_read TEST (IOD #1) (EID #45)
libnsock do_actual_pcap_read(): PCAP do_actual_pcap_read TEST (IOD #1) (EID #37)
libnsock do_actual_pcap_read(): PCAP do_actual_pcap_read TEST (IOD #1) (EID #21)
libnsock do_actual_pcap_read(): PCAP do_actual_pcap_read TEST (IOD #1) (EID #13)
libnsock iocp_loop(): wait for events
libnsock do_actual_pcap_read(): PCAP do_actual_pcap_read TEST (IOD #1) (EID #45)
libnsock do_actual_pcap_read(): PCAP do_actual_pcap_read READ (IOD #1) (EID #45) size=60
libnsock iocp_loop(): wait for events
libnsock do_actual_pcap_read(): PCAP do_actual_pcap_read TEST (IOD #1) (EID #45)
Assertion failed: fs_length(&(nse->iobuf)) == 0, file src\nsock_pcap.c, line 418

Example of bad area of code, iocp_loop() calls pcap_read_on_nonselect() to check if there's any available PCAP data, and if not, it's supposed to do the appropriate IOCP stuff to get any other events, but the else is dangling. The engine_select.c code where this appears to be copied from has the subsequent code indented in a { block }.

Furthermore, the function iterate_through_event_lists() does not appear to do anything related to PCAP. It should probably call handle_pcap_read_result() either directly or via process_iod_events().

@gvanem
Copy link

gvanem commented Sep 22, 2020

What does GetQueuedCompletionStatusEx() == -1 and WSAGetLastError() == WAIT_TIMEOUT signify?

Maybe the associated iobuf for the event should be cleared. Or could there be several events fighting for the same iobuf?
I've given up trying to understand the messy internal event-structures in Nmap.

res0nat0r added a commit to res0nat0r/nmap that referenced this issue Oct 7, 2020
* Implement Ncat proxy creds via environment variable. Fixes nmap#2060, closes nmap#2073

* Fix --resume from IPv6 scans

* Use correct default buffer position. Closes nmap#2084

* Clarify upper boundary for variable-length numerical fields

* Make maximize_fdlimit return rlim_t on appropriate platforms. Closes nmap#2085. Fixes nmap#2079

* Credential object is creds.Account, not brute.Account. See nmap#2086

* Clarify location of the Error object

* Use correct default buffer position. Closes nmap#2086

* Minor optimization of url.parse_query()

* Output of matched fingerprints in http-default-accounts. Fixes nmap#2077

* Document that --open implies --defeat-rst-ratelimit since 7.40

* SNMP scripts are enabled on non-standard ports. See nmap#1473

* Increases SQL Server version resolution

* Eliminate reflection false positives in http-shellshock. Closes nmap#2089

* Unify AFP pathname serialization

* Correct AFP name extraction from responses. Closes nmap#2091
FPGetFileDirParms and FPEnumerateExt2 could crash due to unpacking from
out-of-bounds positions. This latent issue got exposed by converting from
bin.unpack to more stringent string.unpack

* Clarified parsing of the volume list in AFP FPGetSrvrParms

* Add cross references between the 2 whois scripts

* Streamline Boolean expressions

* Centralize AFP timestamp conversion to string

* Fix a word-wrapping issue

* Prevent SSH2 KEX confusion. Fixes nmap#2105

* Add ssh2.fetch_host_key() support for group 16

* Handle case of corrupted TCP options with length 0. Fixes nmap#2104

* Add iDRAC9 fingerprint to http-default-accounts. Closes nmap#2096

* fix license url: http -> https

* Implementation of TLS SNI override in Ncat
Closes nmap#2087, closes nmap#1928, fixes nmap#1927, fixes nmap#1974

* Fix off-by-one issue in last change. Fixes nmap#2107

* Be more strict with TCP options parsing, avoid reading off the end of TCP options. See nmap#2107

* Remove nmap-update

This feature was never publicly released, and has not been distributed
in our binary builds for a couple versions now. It needed to be removed
in order to reduce the number of places Nmap looks for data files. See nmap#2051

* If fetchfile didn't find the XSL, use a relative path on all platforms.

* Do not search NMAPDATADIR on Windows as it is not defined. See nmap#2051

* Remove an unused variable

* Require trailing '/' to match a directory name with --script. See nmap#2051

* Stop using Shellshock in header name. Fixes nmap#1983

* Fix line wrapping

* Speed improvement for script afp-ls. Closes nmap#2098

* New option --discovery-ignore-rst. Closes nmap#1616

* Nbase is needed for __attribute__ on Windows

* include string_pool in Windows build

* Use larger buffer size for socket errors (WSAETIMEDOUT was longer).

* Allow multiple UDP payloads per port. Closes nmap#1859 (payloads to be committed later)

* New UDP payloads. Closes nmap#1860

* Use ASCII chars for some payload data where it makes sense

* Pass error along instead of printing (link error)

* OpenSSL 1.1.X renamed libs: libeay32->libcrypto ssleay32->libssl

* More OpenSSL DLL name changes

* One last libeay32->libcrypto name change

* Fix loopback detection on Windows with new Npcap

* Add some popular favicon hashes

* Update nmap-mac-prefixes

* Update nmap-services from IANA

* Handle too-short response in s7-info. See nmap#2117

* Remove a todo item that is done (--resolve-all)

* Update dated 'class' network terms to CIDR. Closes nmap#2054

* Call superclass's init method from derived class

* Use signed value for tcp header offset and option lengths to detect underflow

* Correctly check for unsigned subtraction underflow.

* Add some missing changelog entries

* Tell LGTM to use the correct version of Python (2)

* Process new Linux and OpenBSD fingerprints

* Only get SSL options if we use them, currently for NO_SSLv2

* Process a few service fingerprint submissions

* Add a requested feature

* Try to make sure enough data is present before parsing. See nmap#2117

* Replace hyphens in the client SSH banner
Hyphen is not allowed in the software version string (RFC 4253, section 4.2)

* Update the SSH protocol flow. Closes nmap#1460
Allows the server to start the key exchange before the protocol version
exchange (banner exchange) is completed

* Silence static analysis warning

LGTM points out that since comparison with sizeof(buf) coerces n to
unsigned, all negative values become very large values, which are
necessarily larger than sizeof(buf), so the test is redundant. We still
want the test in our code to be explicit that we are checking for it, so
reordering the comparisons should silence the warning. A good optimizing
compiler should be able to combine the two conditions anyway.

See github/codeql#4249

* Be explicit about truncating division (timeout is in whole milliseconds)

* Improve docs on -Pn and host discovery

"Host discovery" is the preferred term over "ping scan" because of
confusion with ICMP Echo Request, a.k.a. "ping" as used by the "ping"
utility. Warn when users use -Pn because it has negative impact on scan
times since ultrascan timing parameters fall back to slow initial
defaults.

* Fix a config issue with LGTM (libverbs not linked in libpcap)

* Update IPv6 classifier based on new submissions through 2020-09-14

* Fix a meaningless error message when parsing IPv6 extension headers.

* Allow %F date format to mean YYYY-mm-dd like GNU date

* Remove duplicate test conditionals already tested in enclosing block

* Properly handle pcap reads in iocp engine. Fixes nmap#2126

Still has an odd code smell, but this fixes my test case with Nping.

* Add missing prototype

* Make IOCP the default Nsock engine on Windows. See nmap#2126

* Update macosx build to OpenSSL 1.1.1h, use jhbuild for all build steps

* Default rule base for script mysql-audit. See nmap#2125

* Avoid masked use of date before 1/1/1970 UTC. Fixes nmap#2136, closes nmap#2137

* Fix a CHANGELOG typo

* Reintegrate Nmap 7.90 release branch

* Bump version and regen docs for 7.90SVN post-release

* Only warn about protocol specs in port list with -p. Fixes nmap#2135

* Handle a weird IOCP error for UDP sockets. Fixes nmap#2140

Co-authored-by: nnposter <nnposter@e0a8ed71-7df4-0310-8962-fdc924857419>
Co-authored-by: dmiller <dmiller@e0a8ed71-7df4-0310-8962-fdc924857419>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants
@gvanem @dmiller-nmap and others