3 weeks ago
10 Views

How to Use the Shodan API with Python to Automate Web Scanning « Null Byte :: WonderHowTo

Shodan calls itself “The search engine for Internet-connected devices.” With so many devices connected to the internet featuring varying levels of security, the special capabilities of of which search engine mean the item can provide a list of devices to test as well as also attack. In of which tutorial, we’ll use Python to target specific software vulnerabilities as well as also extract vulnerable target IP addresses coming from Shodan.

Any device connected to the internet must reveal some sort of information regarding itself. of which can be relatively limited, as clever system configurations can block most undesired requests. On some devices, one might be able to scan ports to reveal things such as the services running on a web server, or the name of a webcam connected to a wireless network.

from the first episode of the third season of USA Network’s Mr. Robot, the main character uses the Shodan search engine in order to gather information about his corporate adversary.

A Shodan search coming from “Mr Robot.”. Image by Mr. Robot/USA Network

A search like the one used from the show can reveal essential information about a potential target. Using of which same technique, we’ll look at exactly what can be found using Shodan’s search function, as well as also how the item can be used to execute a hack.

Step 1: Using Shodan

Shodan can be accessed like most some other search engines, by navigating to http://shodan.io in a web browser.

Rather than using traditional search terms to search the content of a publicly indexed website, when searching Shodan we’ll generally look for the information found in device headers, or some other information besides the device’s HTTP web content, which is usually indexed by traditional search engines.

In Mr. Robot, the protagonist searches the string below.

org:”Evil Corp” product:”Apache Tomcat”

While we could search of which same search command, the fictional company “Evil Corp” most likely will not return any results. The second component of the string, the “product” filter, is usually still a functional as well as also useful search. of which search string does require usage of filters, an option only available to registered users. A Shodan account can be registered by clicking on the “Login/Register” button at the top right of the homepage, or by visiting https://account.shodan.io/register directly.

After a Shodan account is usually registered, numerous additional search capabilities will become available from the form of filters. Some of these filters are shown from the list below.

  • country: Filter to a specific country
  • city: Filter to a specific city
  • geo: Filter by coordinates
  • hostname: Look for a matching hostname
  • net: Limit to an IP/Prefix
  • os: Filter based on operating system
  • port: Filter based on open ports

These filters can be applied using the same format as from the example used in Mr. Robot, where the filter is usually included from the search followed by a colon as well as also the search term. The format shown below can be used for any of the filters available within Shodan.

filter:”Keyword”

In of which example, “filter” would certainly be the name of the filter used, as well as also “Keyword” would certainly be the search term which is usually sought within the filter’s category. Multiple filters can be applied, so long as they are separated by spaces.

Don’t Miss: How to Find Vulnerable Targets Using Shodan—the globe’s Most Dangerous Search Engine

The “Apache Tomcat” search, as shown from the show, will indeed return legitimate results when used on Shodan. We can test of which by searching the string shown below.

product:”Apache Tomcat”

After searching, we can see of which over 1.4 million results are returned. of which search provides some interesting data regarding the locations as well as also organizations which are using Apache Tomcat, however to a hacker these results can have a different sort of utility.

An attacker might specifically search for servers or web-connected devices using out-of-date software with known vulnerabilities in order to find devices to exploit. of which process could be completed manually by copying results coming from a Shodan search in a web browser as well as also choosing addresses to attack manually, however the process can also be automated by using scripting languages as well as also Shodan’s API.

Step 2: Retrieving a Shodan API Key

In order to use Shodan’s API to directly request as well as also receive data while bypassing the web interface, we’ll need to use our API key. of which API key can be retrieved by navigating to the “My Account” section of the Shodan website, linked at the upper right of the homepage, or simply by opening https://account.shodan.io/.

of which key will be inserted into the Python code used to make API calls, so the item may be useful to copy the item to your clipboard or save the item to a file.

Step 3: Calling the Shodan API with Python

In order to use Python to make requests using the Shodan API, we’ll need to have a functional Python environment as well as the Shodan Python module installed. from the examples used in of which tutorial, Python 2.7 is usually used. Python 3 also works using the module however would certainly require numerous syntax alterations to be functional with the scripts shown in of which tutorial. On Debian-based Linux operating systems, Python 2.7 can be installed by opening a terminal emulator as well as also running the command below.

sudo apt-get update && sudo apt-get install python2.7

With Python installed, we can also install the Shodan Python module. of which can be done using Pip or by using Easy Install. Pip can also be installed using apt-get using the command below.

sudo apt-get install python-pip

After pip is usually installed, we can use pip to install the Shodan Python module.

sudo pip install shodan

If you have multiple versions of Python present on your device, as well as also potentially multiple versions of pip, you may need to specify you wish to install the module for Python 2.7 by using the command below instead, with pip2.7 specified.

sudo pip2.7 install shodan

If neither of these techniques succeed, the library can also be installed by running the command below.

easy_install shodan

Once Python as well as also the Shodan library are installed, we can begin writing a brand-new Python script. On the Linux command line, we can create a brand-new file as well as also begin editing the item using nano. Be sure to choose a filename some other than “shodan” to ensure of which there are no conflicts between referencing the library as well as also the script itself. Below, we’ll create a file called “search.py”.

nano search.py

The first thing we’ll want to add to of which file is usually a line which will load the Shodan library. We can use the “import” function of Python to do of which, as seen below.

import shodan

Next, we can define our Shodan API key to ensure of which the script can use the item to make API queries. Add the following lines to do so.

SHODAN_API_KEY = “insert your API key here”

api = shodan.Shodan(SHODAN_API_KEY)

Replace “insert your API key here” with your API key retrieved coming from Shodan’s website, leaving the quotation marks enclosing the field.

Next, we can use a “try” declaration to define what the script should attempt. Following of which, we can add a command which uses the “api.search” unit of the Shodan API to actually request a search’s results.

try:
results = api.search(‘apache’)

In of which example, the search string is usually simply “apache,” however of which can be replaced with any search desired, including searches with filters such as those shown earlier from the tutorial. We can return the results of of which search using the set of print commands shown near the end of the code below.

# Show the results
print ‘Results found: %s’ % results’total’
for result in results’matches’:
print ‘IP: %s’ % result’ip_str’
print result’data’
print ”
except shodan.APIError, e:
print ‘Error: %s’ % e

The script should currently appear similar to the code displayed from the image below. More information on the Shodan API as well as also of which code can be found at its documentation page.

The script can currently be saved as well as also tested. Within nano, we can save the script with Ctrl+O, as well as also exit nano with Ctrl+X. coming from within the same directory, we can run the script using the command below.

python2 script.py

Running the script should return numerous IP addresses as well as also some information associated with them, including HTTP status, location, as well as also some other device information indexed by Shodan. of which information is usually formatted very similarly to the data shown when searching within the web interface.

While of which additional information may allow for additional criteria to be processed by some other scripts as well as also tools, if one wished to automate the process of gathering as well as also testing attacks against IP addresses of which format is usually largely unnecessary.

To only return IP addresses, we can change the formatting of our Python script. First, we can remove the “IP:” prefix coming from the line shown below.

print ‘IP: %s’ % result’ip_str’

Such of which the item looks like of which line instead.

print ‘%s’ % result’ip_str’

We can also delete the line which precedes the item, as well as also the two lines which follow the item.

print ‘Results found: %s’ % results’total’
print result’data’
print ”

The script should currently appear like the one shown below.

When we run of which script, the item will instead return a list of IP addresses without any some other unnecessary content.

of which list is usually much more useful for automating attacks against the list, however we need to have an effective way to save the item. We can use shell operations in order to send the output directly to a log file. When running the script, include the “>>” operator followed by the name of the file you wish to send the output to.

python2 search.py >> log.txt

currently we have a text file containing a list of IPs which we can use to test various attacks, depending on what search terms we’ve used to identify particular kinds of vulnerable systems.

Step 4: Automating Tasks with Shodan Results

For an example of a command line utility which can use an IP address as an argument, we’ll use ping. Using something which actively attempts to attack or exploit the devices found by Shodan would certainly be illegal as well as also irresponsible, so make sure to only run tools you have permission to use on a target if you’re actually exploiting anything. First, we’ll create a brand-new shell script using nano the same way we did earlier.

nano ping.sh

We can begin the script with the “crunchbang” (the “#!” symbols) as well as also shell declaration. of which states of which the item is usually a shell script, to be run by the bash shell.

#!/bin/bash

Next, we can add a statement which allows us to do something with each line of our IP list file individually.

cat log.txt | while read line
do
ping $line
done

The script should currently look like the one shown from the image below.

We can currently save of which script with Ctrl+O, as well as also exit nano Once more with Ctrl+X. To be able to run the script, we’ll need to mark the item as executable by our operating system by granting the item of which privilege using chmod.

chmod +x ping.sh

currently, we can run the script coming from the command line.

./ping.sh

The script should iterate through each IP from the address as well as also send a ping to each IP.

If of which works, you’ve currently successfully retrieved Shodan results as well as also individually processed them! Ping is usually hardly an attack vector, however that has a few minor alterations, a similar script could be used by an attacker for malicious purposes.

Step 5: Weaponizing the Attack

The string which initiated the “ping” within the shell script would certainly be one easy area to manipulate the action which is usually done with the IPs retrieved coming from Shodan. The original Python script could also be updated for more complicated exploits or tests. The basic ping command of the “while” iteration of the shell script is usually shown below.

ping $line

The “$line” variable in of which command represents each line of the IP list file, log.txt. We can replace of which ping command with any some other string which would certainly include an IP address as an argument. We could use nmap to port-scan the target IPs by using the command below, using the -sS argument to conduct a service scan.

nmap -sS $line

While each internet connected device incorporates a variety of ways in which the item could be indexed by scanning services such as Shodan, one can check the security of their local network as well as also router by checking their external IP at a website like http://www.whatsmyip.org/ as well as also searching of which IP on Shodan to see what sort of information is usually available.

some other Applications

of which format could be expanded to practically any some other attack which could be launched coming from a command line as well as also includes an IP. of which sort of scanning as well as also attacking of multiple targets is usually an extremely effective method for discovering vulnerable systems without having to take the time to individually find as well as also attack them manually.

of which methodology can be applied to all sorts of different attacks, using Shodan, Python, shell scripting, or some other tools, so long as they develop the capability of finding devices as well as also attacking them without user input.

I desire of which you enjoyed of which tutorial on Shodan! If you have any questions about of which tutorial or Shodan usage in general, feel free to leave a comment below, or reach me on Twitter @tahkion.

Cover photo as well as also screenshots by TAKHION/Null Byte

Leave a Comment

Your email address will not be published. Required fields are marked *

16 − 10 =