Network Hacking - Anonymous Reconnaissance and Browsing (Python for Hackers Part-9.2)

Using the Mechanize Library to Perform Anonymous Reconnaissance 

Most computer users use a web browser to navigate websites and view content over the Internet. Each website has a different features, but will usually read a particular text document, analyze it, and then display it to a user, just like the way a source file interacts with the Python interpreter. 

Using Python, you can browse the internet by getting and parsing the HTML source code of a website. There are different libraries that come with this programming language that can handle web content, but for this hack, you will be using Mechanize, which includes the primary class called Browser. Take a look at this sample script that will show you how to get a source code of a website: 

When you run this script, you will see syngress.com’s HTML code for their index page, which will look like this:

Ensuring Anonymity While Browsing

Now that you know how to get a webpage, you will want to create a script that will allow you to anonymously retrieve information from a website. As you may already know, web servers see to it that they log the IP addresses of different users that view their websites in order to identify them. This can usually be prevented by using a VPN (virtual private network), or by using Tor. What happens when you use a VPN is that all traffic gets routed to the private network automatically. With this concept, you get the idea that you can use Python to connect to the proxy servers instead, which will give your program an added layer of anonymity.

You can use the Browser class to specify a proxy server that will be used by a particular program. For this script, you can use the HTTP proxy provided by www.hidemyass.com. Just in any case this proxy is not available to be used anymore, you can simply go to the website and select an HTTP proxy that you can use. You can also get other great proxies for your codes at http://rmccurdy.com/scripts/proxy/good.txt. 

You will then see that the website you are trying to access believes that you are using the 216.155.139.115 IP address, which is actually the IP address that your proxy provided you. Now, continue building your script: 

At this point, your browser already contains a single layer of anonymity. However, websites do use a string called user-agent in order to identify unique users that log in to their site. This string will usually allow the website to get useful information about a user in order to provide a tailored HTML code, which then provides a better user experience. However, malicious websites can also use that information to exploit the browser that is being used by a targeted user. For example, there are certain user-agent strings that some travel websites use to detect users that browse using Macbooks, which then proceed to give these users more expensive options. 

Since you are using Mechanize, you can change the user-agent string just like how you change the proxy. You can make use of available user-agent strings from 

http://www.useragentstring.com/pages/useragentstring.php that you can use for the next function that you are going to make. Now, you will be creating a script that will allow you to test a change on your user-agent string to the Netscape browser:

When you run this code, you will be able to see that you are able to browse a webpage using a false user-agent string. The website that you are browsing now thinks that you are using a Netscape 6.01 browser instead of simply using Python to fetch the page. 

What happens after is that websites that you are going to visit will attempt to present cookies that they can use as a unique identifier in order to identify you as a repeat visitor when you go back to their site the next time. To prevent these websites from identifying you, you will need to see to it that you clear all the cookies from your browser whenever you perform functions that you want to be anonymous. Another built-in library in Python, called the Cookelib, will allow you to make use of various container types that will allow you to deal with cookies that website present you. For this script, you will be using a container type that will allow you to save cookies to disk, and then print out the cookies that you received during your session:

When you run this script, you will see your session ID cookie for browsing the Syngress site:

Finalize Your Anonymous Browser into a Python Class

At this point, you have an idea of all the functions that you want to include in your anonymous browser, and that in order to make the entire process of importing all these functions to all files that you will be creating in the future, you will need to turn that into a class. This will allow you to simply call the class using a browser object in the future. This script will help you do this:

This class now contains user-agents list, as will as proxy server list that you may want to use when you browse. It also contains the functions that you were able to create earlier, which you can call individually or all at once using the anonymize function. The anonymize function will also allow you to select the option to wait for 60 seconds which will increase the time of requests that you send. While this will not change anything in the information that you submit to the website, this step will decrease the chance that the websites that you are visiting will recognize that the information being sent to them comes from a single source. You will also notice that the file anonBrowser.py includes this class, and should be saved in a local directory containing scripts that will call it. 

Now, you can write a script where you can use the class that you have just created. In this example, you will be entering votes for an online competition on the website kittenwar.com where you have to vote for kittens based on their cuteness. Because the votes on the website will be tabulated according to a user’s session, you will need to have unique visits to the website in order for your votes to be counted. Using this script, you should be able to visit the targeted website anonymously five times, which will allow you to enter five votes using the same computer: 

After running this script, you will be able to fetch the targeted web page using five different unique sessions, which means that you are using different cookies every time you visit.