DNS & OSINT
Let's try to get subdomains from the DNS records. We should also try for Zone Transfer (If vulnerable, you should report it).
dnsrecon
dnsrecon -a -d tesla.com# subdomains
bbot -t tesla.com -f subdomain-enum
# subdomains (passive only)
bbot -t tesla.com -f subdomain-enum -rf passive
# subdomains + port scan + web screenshots
bbot -t tesla.com -f subdomain-enum -m naabu gowitness -n my_scan -o .# Subfinder, use -silent to only have subdomains in the output
./subfinder-linux-amd64 -d tesla.com [-silent]# findomain, use -silent to only have subdomains in the output
./findomain-linux -t tesla.com [--quiet]python3 oneforall.py --target tesla.com [--dns False] [--req False] [--brute False] runassetfinder --subs-only <domain># It requires that you create a sudomy.api file with API keys
sudomy -d tesla.comvita -d tesla.comtheHarvester -d tesla.com -b "anubis, baidu, bing, binaryedge, bingapi, bufferoverun, censys, certspotter, crtsh, dnsdumpster, duckduckgo, fullhunt, github-code, google, hackertarget, hunter, intelx, linkedin, linkedin_links, n45ht, omnisint, otx, pentesttools, projectdiscovery, qwant, rapiddns, rocketreach, securityTrails, spyse, sublist3r, threatcrowd, threatminer, trello, twitter, urlscan, virustotal, yahoo, zoomeye"Uses the API https://sonar.omnisint.io to obtain subdomains
# Get list of subdomains in output from the API
## This is the API the crobat tool will use
curl https://sonar.omnisint.io/subdomains/tesla.com | jq -r ".[]"curl https://jldc.me/anubis/subdomains/tesla.com | jq -r ".[]"RapidDNS free API
# Get Domains from rapiddns free API
rapiddns(){
curl -s "https://rapiddns.io/subdomain/$1?full=1" \
| grep -oE "[\.a-zA-Z0-9-]+\.$1" \
| sort -u
}
rapiddns tesla.comFetches known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl for any given domain.
# Get subdomains from GAUs found URLs
gau --subs tesla.com | cut -d "/" -f 3 | sort -uThey scrap the web looking for JS files and extract subdomains from there.
# Get only subdomains from SubDomainizer
python3 SubDomainizer.py -u https://tesla.com | grep tesla.com
# Get only subdomains from subscraper, this already perform recursion over the found results
python subscraper.py -u tesla.com | grep tesla.com | cut -d " " -f# Get info about the domain
shodan domain <domain>
# Get other pages with links to subdomains
shodan search "http.html:help.domain.com"Quick and Dirty script to use the Censys API to query subdomains of a target domain
export CENSYS_API_ID=...
export CENSYS_API_SECRET=...
python3 censys-subdomain-finder.py tesla.compython3 DomainTrail.py -d example.comsecuritytrails.com has a free API to search for subdomains and IP history
This project offers for free all the subdomains related to bug-bounty programs. You can access this data also using chaospy or even access the scope used by this project https://github.com/projectdiscovery/chaos-public-program-list
You can find a comparison of many of these tools here: https://blog.blacklanternsecurity.com/p/subdomain-enumeration-tool-face-off
amass
Repository: https://github.com/OWASP/Amass
Installation
# Kali Linux
#OWASP Amass is installed by default and can be managed like any other Kali package:
sudo apt update && sudo apt install amassThe 'enum' Subcommand
This subcommand will perform DNS enumeration and network mapping while populating the selected graph database. All the setting available in the configuration file are relevant to this subcommand.
#Command Line
amass enum -d tesla.com | grep tesla.com # To just list subdomains
# Oneliner
project=<NAME>; domain1=<DOMAIN1.COM>; domainN=<DOMAINN.com>; amass enum -active -ipv4 -brute -o ${project}.amass.txt -json ${project}.amass.json -d ${domain1},${domainN}#Docker
domain=example.com; sudo docker run -v OUTPUT_DIR_PATH:/.config/amass/ caffix/amass enum -active -ipv4 -brute -o amass.${domain}.txt -json amass.${domain}.json -d ${domain}See the extended documentation: https://github.com/OWASP/Amass/blob/master/doc/user_guide.md
The 'intel' Subcommand
The intel subcommand can help you discover additional root domain names associated with the organization you are investigating. The data source sections of the configuration file are utilized by this subcommand in order to obtain passive intelligence, such as reverse whois information.
project=proj_name; domain1=domain1.com; amass intel -o ${project}.amass.txt -whois -d ${domain} -ipv4 -srcInfo: Automated Pentest Recon Scanner
STEALTH
Quickly enumerate single targets using mostly non-intrusive scans to avoid WAF/IPS blocking.
sniper -t <TARGET> -m stealth -o -re -w workspace_name
sniper -t www.domain.com -m stealth -o -re -w domainblacksheepwall
Repo: https://github.com/tomsteele/blacksheepwall
Info: blacksheepwall is a hostname reconnaissance tool
Examples
~/tools./blacksheepwall_linux_amd64.1 -censys -crtsh -vt -srv -axfr -headers -tls -domain www.domain.com 200.123.123.123dns-parallel-prober
Repo: https://github.com/lorenzog/dns-parallel-prober
Info: This is a parallelised domain name prober to find as many subdomains of a given domain as fast as possible.
PLEASE NOTE this script probes DNS servers actively, so please use at your own risk. You are likely to get blacklisted and/or saturate your bandwidth. Whatever you do, it's your responsibility to make sure you have approval for it.
subdomain_scanner
Repo: https://github.com/cr4shcod3/subdomain_scanner
Info: Subdomain Scanner is a python tool designed to enumerate subdomains on a target domain through a wordlist.
Examples
python subdomain.py -u http://example.com
python subdomain.py -u http://example.com -l 100
python subdomain.py -u http://example.com -cl filename.txt
python subdomain.py -u http://example.com -l 100 -o filename.txt
Leviathan Framework
Repo: https://github.com/leviathan-framework/leviathan
Info: Leviathan is a mass audit toolkit which has wide range service discovery, brute force, SQL injection detection and running custom exploit capabilities. It consists open source tools such masscan, ncrack, dsss and gives you the flexibility of using them with a combination.
The main goal of this project is auditing as many system as possible in country-wide or in a wide IP range.
Trape
Repo: https://github.com/boxug/trape
Info: Trape is a recognition tool that allows you to track people, the information you can get is very detailed. We want to teach the world through this, as large Internet companies could monitor you, obtaining information beyond your IP.
Sublist3r
Repo: https://github.com/aboul3la/Sublist3r
Info: Sublist3r is a python tool designed to enumerate subdomains of websites using OSINT. It helps penetration testers and bug hunters collect and gather subdomains for the domain they are targeting. Sublist3r enumerates subdomains using many search engines such as Google, Yahoo, Bing, Baidu, and Ask. Sublist3r also enumerates subdomains using Netcraft, Virustotal, ThreatCrowd, DNSdumpster, and ReverseDNS.
subbrute was integrated with Sublist3r to increase the possibility of finding more subdomains using bruteforce with an improved wordlist. The credit goes to TheRook who is the author of subbrute.
Installation
git clone https://github.com/aboul3la/Sublist3r.git
sudo pip install -r requirements.txtExamples
# Specify engines (-e)
python ~/tools/Sublist3r/sublist3r.py -e google,yahoo,virustotal -d example.com -o output.txt
# With bruteforce (-b)
python ~/tools/Sublist3r/sublist3r.py -b -d example.com -o output.txt
# Scan the found subdomains against specified tcp ports
python ~/tools/Sublist3r/sublist3r.py -d example.com -p 80,443 -o output.txt
# Most complete (engines, ports and brute force)
python ~/tools/Sublist3r/sublist3r.py -b -d example.com -p 80,443 -o output.txt
cloudflare_enum
Deprecated
Repo: https://github.com/mandatoryprogrammer/cloudflare_enum
Info: A simple tool to allow easy querying of Cloudflare's DNS data written in Python.
HostileSubBruteforcer
Repo: https://github.com/nahamsec/HostileSubBruteforcer
Info: This app will bruteforce for exisiting subdomains and provide the following information:
IP address
Host
if the 3rd party host has been properly setup. (for example if site.example.com is poiting to a nonexisiting Heroku subdomain, it'll alert you) -> Currently only works with AWS, Github, Heroku, shopify, tumblr and squarespace.
There may be some false positives depending on the host configurations. (Tried to take them out as much as possible) Also works recursively at the end to get the subdomains under the ones that it has already found and dumps all your data into an output.txt file just in case (fresh one gets created at the beginning of each process)
More info http://labs.detectify.com/post/109964122636/hostile-subdomain-takeover-using
AQUATONE
Repo: https://github.com/michenriksen/aquatone/
Info: AQUATONE is a set of tools for performing reconnaissance on domain names. It can discover subdomains on a given domain by using open sources as well as the more common subdomain dictionary brute force approach. After subdomain discovery, AQUATONE can then scan the hosts for common web ports and HTTP headers, HTML bodies and screenshots can be gathered and consolidated into a report for easy analysis of the attack surface.
Install
gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
\curl -sSL https://get.rvm.io | bash -s stable
sudo gem install aquatoneAquatone One-liner
echo "aquatone-discover -d \$1 && aquatone-scan -d \$1 --ports huge && aquatone-takeover -d \$1 && aquatone-gather -d \$1" >> aqua.sh && chmod +x aqua.sh
/aqua.sh domain.comGitrob
Repo: https://github.com/michenriksen/gitrob
Info: Gitrob is a command line tool which can help organizations and security professionals find sensitive information lingering in publicly available files on GitHub. The tool will iterate over all public organization and member repositories and match filenames against a range of patterns for files that typically contain sensitive or dangerous information.
Looking for sensitive information in GitHub repositories is not a new thing, it has been known for a while that things such as private keys and credentials can be found with GitHub's search functionality, however Gitrob makes it easier to focus the effort on a specific organization.
Truffle Hog
Repo: https://github.com/dxa4481/truffleHog
Info: Searches through git repositories for secrets, digging deep into commit history and branches. This is effective at finding secrets accidentally committed.
DNSdumpster
Repo: https://dnsdumpster.com/
Info: DNSdumpster.com is a FREE domain research tool that can discover hosts related to a domain. Finding visible hosts from the attackers perspective is an important part of the security assessment process.
PublicWWW
Repo: https://publicwww.com/
Info: Ultimate solution for digital marketing and affiliate marketing research, PublicWWW allow you to perform searches this way, something that is not possible with other regular search engines.
Reverse IP Lookup
Repo: http://reverseip.domaintools.com/
Info: Enter an IP address and our patented Reverse IP Lookup tool will show you all of the domains currently hosted there. Results include all gTLD domains and any known ccTLD domains. Lookups are free for DomainTools Personal and Enterprise Members, for IPs with up to 2,000 hosted domains.
ThreatCrowd
Repo: https://threatcrowd.org/
Info: A Search Engine for Threats. Search by Domain, IP, Email or Organization
MX Lookup
Repo: https://mxtoolbox.com/
Info: This test will list MX records for a domain in priority order. The MX lookup is done directly against the domain's authoritative name server, so changes to MX Records should show up instantly. You can click Diagnostics , which will connect to the mail server, verify reverse DNS records, perform a simple Open Relay check and measure response time performance. You may also check each MX record (IP Address) against 105 DNS based blacklists . (Commonly called RBLs, DNSBLs).
IPv4info
Repo: http://ipv4info.com/
Info: Key features: ipv4 networks registration data; ipv4 address allocation table; all domains for IP address; all domains in registered network block; Autonomous systems registration data, its originated prefixes and peers.
VirusTotal domain information
Repo: https://www.virustotal.com/en-gb/domain/google.com/information/
Info: VirusTotal's passive DNS only stores address records. This domain has been seen to resolve to the following IP addresses.
crt.sh Certificate Search
Repo: https://crt.sh/
Info: Enter an Identity (Domain Name, Organization Name, etc), a Certificate Fingerprint (SHA-1 or SHA-256) or a crt.sh ID.
# Get Domains from crt free API
crt(){
curl -s "https://crt.sh/?q=%25.$1" \
| grep -oE "[\.a-zA-Z0-9-]+\.$1" \
| sort -u
}
crt tesla.comInforme de transparencia de Google
Repo: https://transparencyreport.google.com/https/certificates
Info: Busca todos los certificados de un dominio que estén presentes en los registros de transparencia de certificados públicos activos. Esto devuelve los subdominios de un dominio.
Pentest-Tools
Repo: https://pentest-tools.com/information-gathering/google-hacking
Info: Every penetration test should start with a passive reconnaissance phase. Since public search engines have gathered huge amounts of information about almost every website from the Internet, it is a good idea to make some queries and get this information from them. Very often you will find sensitive information or data that is not supposed to be public.
Wayback Machine
Repo: https://archive.org/web/
Info: Explore more than 327 billion web pages saved over time
gitgrepper
Repo: https://gist.github.com/EdOverflow/a9aad69a690d97a8da20cd4194ca6596
Info: Simple Bash script to find interesting data in GIT logs.
Recon-ng
Repo: https://bitbucket.org/LaNMaSteR53/recon-ng
Info: econ-ng is a full-featured Web Reconnaissance framework written in Python. Complete with independent modules, database interaction, built in convenience functions, interactive help, and command completion, Recon-ng provides a powerful environment in which open source web-based reconnaissance can be conducted quickly and thoroughly.
Recon-ng has a look and feel similar to the Metasploit Framework, reducing the learning curve for leveraging the framework. However, it is quite different. Recon-ng is not intended to compete with existing frameworks, as it is designed exclusively for web-based open source reconnaissance. If you want to exploit, use the Metasploit Framework. If you want to social engineer, use the Social-Engineer Toolkit. If you want to conduct reconnaissance, use Recon-ng! See the Usage Guide for more information.
Recon-ng is a completely modular framework and makes it easy for even the newest of Python developers to contribute. Each module is a subclass of the "module" class. The "module" class is a customized "cmd" interpreter equipped with built-in functionality that provides simple interfaces to common tasks such as standardizing output, interacting with the database, making web requests, and managing API keys. Therefore, all the hard work has been done. Building modules is simple and takes little more than a few minutes. See the Development Guide for more information.
enumall.py
Repo: https://github.com/jhaddix/domain
Info: Script to enumerate subdomains, leveraging recon-ng. Uses google scraping, bing scraping, baidu scraping, yahoo scarping, netcraft, and bruteforces to find subdomains. Plus resolves to IP.
./enumall.py -a domain1.comAltdns - Subdomain discovery through alterations and permutations
Repo: https://github.com/infosec-au/altdns
Info: Altdns is a DNS recon tool that allows for the discovery of subdomains that conform to patterns. Altdns takes in words that could be present in subdomains under a domain (such as test, dev, staging) as well as takes in a list of subdomains that you know of.
From these two lists that are provided as input to altdns, the tool then generates a massive output of "altered" or "mutated" potential subdomains that could be present. It saves this output so that it can then be used by your favourite DNS bruteforcing tool.
Alternatively, the -r flag can be passed to altdns so that once this output is generated, the tool can then resolve these subdomains (multi-threaded) and save the results to a file.
Altdns works best with large datasets. Having an initial dataset of 200 or more subdomains should churn out some valid subdomains via the alterations generated.
Maltego
Repo: https://www.paterva.com/web7/
Info: Maltego CE is the community edition of Maltego and is available for free for everyone after a quick registration. It offers the same functionality as Maltego Classic with a few limitations.
Last updated