> For the complete documentation index, see [llms.txt](https://pcastagnaro.gitbook.io/pentest-bug-bounty-resources/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://pcastagnaro.gitbook.io/pentest-bug-bounty-resources/pentest-bounty-resources/networking/parsing.md). # Parsing ## Nmap to CSV ```bash git clone https://github.com/maaaaz/nmaptocsv cd nmaptocsv ./nmaptocsv.py -i InputFile.nmap -f ip-port-protocol-service-version -o OutputFile.csv ``` {% hint style="warning" %} Use python 2.7. {% endhint %} ## [**nmap2md**](https://github.com/vdjagilev/nmap2md) {% code overflow="wrap" lineNumbers="true" %} ```bash git clone https://github.com/vdjagilev/nmap2md cd nmap2md ./nmap2md.py nmapOutput.xml > nmapOutput.md ``` {% endcode %} ### Nmap Scan and nmap2md {% code overflow="wrap" lineNumbers="true" %} ```bash nmapoutput=NmapResult ; sudo nmap --top-ports 10000 -v --max-retries 2 --max-rtt-timeout 400ms -sV -oA ${nmapoutput} domain1.com domainN.com; tools/nmap2md/./nmap2md.py ${nmapoutput}.xml > ${nmapoutput}.md ``` {% endcode %} *** ## Nmap Grep Comprehensive parsing script for grepable Nmap output files. Provides a summary table, split hosts files, and URLs for web and SMB hosts. **nmap-grep.sh** is meant for parsing grepable Nmap output files (-oG). The file must be the first parameter. **--out-dir** can be used to specify a custom output directory. If not output directory is given, **nmap-grep-YYYY-MM-DD-HH-MM-SS** will be created. * git clone * ./nmap-grep.sh **output-nmap.gnmap** \[--out-dir \[outputdirectory]] \[\[options]] * This script performs the following actions, which each have different options to flag them as disabled. * Create a summary table for open ports as summary.txt, including the IP, port, tcp/udp, protocol, and any version information. This can be disabled with **--no-summary**. * Create files for each open port, listing each IP with that port open on a separate line. By default, these files will be named \[port]-\[tcp/udp]-hosts.txt. This can be disabled with **--no-split**. * Rename split hosts files for common ports and services. For example, **21-tcp-hosts.txt** becomes **ftp-hosts.txt**. This can be disabled with **--no-label-split**. * Create **web-urls.txt**, with URLs for every open TCP 80, 443, 8080, and 8443 service. This can be disabled with **--no-web-urls**. * Create **smb-urls.txt**, with URLs for every open TCP 445 service. This can be disabled with **--no-smb-urls**. * Create **up-hosts.txt**, listing every host that reported as "up". This can be disabled with **--no-up**. ## awesome-nmap-grep A collection of awesome, grep-like commands for the nmap greppable output (-oG) format. This repository aims to serve as a quick reference to modify the output into readable formats. All of the below commands assume the output was saved to a file called **output.grep**. The example command to produce this file as well as the sample outputs was: **nmap -v --reason 127.0.0.1 -sV -oG output.grep -p-**. Finally, the **NMAP\_FILE** variable is set to contain **output.grep**. ### count number of open ports * NMAP\_FILE=**output-nmap.gnmap** * egrep -v "^#|Status: Up" $NMAP\_FILE | cut -d' ' -f2,4- | sed -n -e 's/Ignored.\*//p' | awk -F, '{split($0,a," "); printf "Host: %-20s Ports Open: %d\n" , a\[1], NF}' | sort -k 5 -g #### output Host: 127.0.0.1 Ports Open: 16 ### print the top 10 ports * NMAP\_FILE=**output-nmap.gnmap** * egrep -v "^#|Status: Up" $NMAP\_FILE | cut -d' ' -f4- | sed -n -e 's/Ignored.\*//p' | tr ',' '\n' | sed -e 's/^\[ \t]\*//' | sort -n | uniq -c | sort -k 1 -r | head -n 10 #### output 1 9001/open/tcp//tor-orport?/// 1 9000/open/tcp//cslistener?/// 1 8080/open/tcp//http-proxy/// 1 80/open/tcp//http//Caddy/ 1 6379/open/tcp//redis//Redis key-value store/ 1 631/open/tcp//ipp//CUPS 2.1/ 1 6234/open/tcp///// 1 58377/filtered/tcp///// 1 53/open/tcp//domain//dnsmasq 2.76/ 1 49153/open/tcp//mountd//1-3/ ### top service identifiers * NMAP\_FILE=**output-nmap.gnmap** * egrep -v "^#|Status: Up" $NMAP\_FILE | cut -d ' ' -f4- | tr ',' '\n' | sed -e 's/^\[ \t]\*//' | awk -F '/' '{print $7}' | grep -v "^$" | sort | uniq -c | sort -k 1 -nr #### output 2 Caddy 2 1-3 (RPC 100005) 1 dnsmasq 2.76 1 Redis key-value store 1 OpenSSH 6.9 (protocol 2.0) 1 MySQL 5.5.5-10.1.14-MariaDB 1 CUPS 2.1 ### top service names * NMAP\_FILE=**output-nmap.gnmap** * egrep -v "^#|Status: Up" $NMAP\_FILE | cut -d ' ' -f4- | tr ',' '\n' | sed -e 's/^\[ \t]\*//' | awk -F '/' '{print $5}' | grep -v "^$" | sort | uniq -c | sort -k 1 -nr #### output 2 mountd 2 http 1 unknown 1 tor-orport? 1 ssl|https 1 ssh 1 redis 1 mysql 1 ipp 1 http-proxy 1 domain 1 cslistener? ### hosts and open ports * NMAP\_FILE=**output-nmap.gnmap** * egrep -v "^#|Status: Up" $NMAP\_FILE | cut -d' ' -f2,4- | sed -n -e 's/Ignored.\*//p' | awk '{print "Host: " $1 " Ports: " NF-1; $1=""; for(i=2; i<=NF; i++) { a=a" "$i; }; split(a,s,","); for(e in s) { split(s\[e],v,"/"); printf "%-8s %s/%-7s %s\n" , v\[2], v\[3], v\[1], v\[5]}; a="" }' #### output Host: 127.0.0.1 Ports: 16 open tcp/22 ssh open tcp/53 domain open tcp/80 http open tcp/443 https open tcp/631 ipp open tcp/3306 mysql open tcp/4767 unknown open tcp/6379 open tcp/8080 http-proxy open tcp/8081 blackice-icecap open tcp/9000 cslistener open tcp/9001 tor-orport open tcp/49152 unknown open tcp/49153 unknown filtered tcp/54695 filtered tcp/58369 ### banner grab * NMAP\_FILE=**output-nmap.gnmap** * egrep -v "^#|Status: Up" $NMAP\_FILE | cut -d ' ' -f2,4- | awk -F, '{split($1,a," "); split(a\[2],b,"/"); print a\[1] " " b\[1]; for(i=2; i<=NF; i++) { split($i,c,"/"); print a\[1] c\[1] }}' | xargs -L1 nc -v -w1 #### output found 0 associations found 1 connections: 1: flags=82\ outif lo0 src 127.0.0.1 port 52224 dst 127.0.0.1 port 3306 rank info not available TCP aux info available Connection to 127.0.0.1 port 3306 \[tcp/mysql] succeeded! Y 5.5.5-10.1.14-MariaDB�uds9^MIf��!?�EgVZ>iv7KTD7mysql\_native\_passwordfound 0 associations nc: connectx to 127.0.0.1 port 54695 (tcp) failed: Connection refused nc: connectx to 127.0.0.1 port 58369 (tcp) failed: Connection refused