Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changing the HTTP endpoints to HTTPS where ever applicable. #158

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Changing the HTTP endpoints to HTTPS where ever applicable. #158

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .vscode/settings.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"python.linting.pylintEnabled": false
}
8 changes: 4 additions & 4 deletions LICENSE
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007

Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Crazy. @konarkmodi has even changed the links to https in License.

Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.

Expand Down Expand Up @@ -645,7 +645,7 @@ the "copyright" line and a pointer to where the full notice is found.
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
along with this program. If not, see <https://www.gnu.org/licenses/>.

Also add information on how to contact you by electronic and paper mail.

Expand All @@ -664,11 +664,11 @@ might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<http://www.gnu.org/licenses/>.
<https://www.gnu.org/licenses/>.

The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<http://www.gnu.org/philosophy/why-not-lgpl.html>.
<https://www.gnu.org/philosophy/why-not-lgpl.html>.
2 changes: 1 addition & 1 deletion base.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<html>
<head>
<!--Import Google Icon Font-->
<link href="http://fonts.googleapis.com/icon?family=Material+Icons" rel="stylesheet">
<link href="https://fonts.googleapis.com/icon?family=Material+Icons" rel="stylesheet">
<!--Import materialize.css-->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/materialize/0.98.1/css/materialize.min.css">
<!--Let browser know website is optimized for mobile-->
Expand Down
2 changes: 2 additions & 0 deletions domain/domain_checkpunkspider.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ class style:

def checkpunkspider(reversed_domain):
time.sleep(0.5)

# Tried putting on HTTPS, but the CERT is broken on this website.
req = requests.post("http://www.punkspider.org/service/search/detail/" + reversed_domain, verify=False)
try:
return json.loads(req.content)
Expand Down
2 changes: 1 addition & 1 deletion domain/domain_emailhunter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ def main(domain):
def output(data, domain=""):
if type(data) == list and data[1] == "INVALID_API":
print colored(
style.BOLD + '\n[-] Emailhunter API key not configured, skipping Email Search.\nPlease refer to http://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
style.BOLD + '\n[-] Emailhunter API key not configured, skipping Email Search.\nPlease refer to https://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
else:
for x in data:
print str(x)
Expand Down
2 changes: 1 addition & 1 deletion domain/domain_forumsearch.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ class style:
def boardsearch_forumsearch(domain):
time.sleep(0.3)
req = requests.get(
'http://boardreader.com/index.php?a=l&q=%s&d=0&extended_search=1&q1=%s&ltype=all&p=50' % (domain, domain))
'https://boardreader.com/index.php?a=l&q=%s&d=0&extended_search=1&q1=%s&ltype=all&p=50' % (domain, domain))
soup = BeautifulSoup(req.content, "lxml")
text = soup.findAll('bdo', {"dir": "ltr"})
links = {}
Expand Down
2 changes: 1 addition & 1 deletion domain/domain_github.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ def output(data, domain=""):
if type(data) == list and not data[0]:
if data[1] == "INVALID_API":
print colored(
style.BOLD + '\n[-] Github Access Token not configured, skipping Github Search.\nPlease refer to http://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
style.BOLD + '\n[-] Github Access Token not configured, skipping Github Search.\nPlease refer to https://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
else:
print colored("Sad! Nothing found on github", 'red')
else:
Expand Down
1 change: 1 addition & 0 deletions domain/domain_googletracking.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
Original Idea: @jms_dot_py

Original article -
# Tried putting on HTTPS, but the CERT is broken on this website.
http://www.automatingosint.com/blog/2017/07/osint-website-connections-tracking-codes/

Original code -
Expand Down
2 changes: 1 addition & 1 deletion domain/domain_history.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ class style:
def netcraft_domain_history(domain):
ip_history_dict = {}
time.sleep(0.3)
endpoint = "http://toolbar.netcraft.com/site_report?url=%s" % (domain)
endpoint = "https://toolbar.netcraft.com/site_report?url=%s" % (domain)
req = requests.get(endpoint)

soup = BeautifulSoup(req.content, 'html.parser')
Expand Down
2 changes: 1 addition & 1 deletion domain/domain_pagelinks.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ class style:
def pagelinks(domain):
time.sleep(0.3)
try:
req = requests.get('http://api.hackertarget.com/pagelinks/?q=%s' % (domain))
req = requests.get('https://api.hackertarget.com/pagelinks/?q=%s' % (domain))
page_links = req.content.split("\n")
return page_links
except:
Expand Down
2 changes: 1 addition & 1 deletion domain/domain_pastes.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ def output(data, domain=""):
if not data[0]:
if type(data) == list and data[1] == "INVALID_API":
print colored(
style.BOLD + '\n[-] google_cse_key and google_cse_cx not configured. Skipping paste(s) search.\nPlease refer to http://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
style.BOLD + '\n[-] google_cse_key and google_cse_cx not configured. Skipping paste(s) search.\nPlease refer to https://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
else:
print "Error Message: %s" % data[1]['error']['message']
print "Error Code: %s" % data[1]['error']['code']
Expand Down
2 changes: 1 addition & 1 deletion domain/domain_shodan.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ def main(domain):
def output(data, domain=""):
if type(data) == list and data[1] == "INVALID_API":
print colored(
style.BOLD + '\n[-] Shodan API Key not configured. Skipping Shodan search.\nPlease refer to http://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
style.BOLD + '\n[-] Shodan API Key not configured. Skipping Shodan search.\nPlease refer to https://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
else:
if 'matches' in data.keys():
for x in data['matches']:
Expand Down
6 changes: 4 additions & 2 deletions domain/domain_subdomains.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@ def subdomains(domain, subdomain_list):
def subdomains_from_netcraft(domain, subdomain_list):
print colored(' [+] Extracting subdomains Netcraft\n', 'blue')
target_dom_name = domain.split(".")
req1 = requests.get("http://searchdns.netcraft.com/?host=%s" % domain)
req1 = requests.get("https://searchdns.netcraft.com/?host=%s" % domain)

#The page still shows HTTP in the HTML body, so this regex needs to staty at HTTP.
link_regx = re.compile('<a href="http://toolbar.netcraft.com/site_report\?url=(.*)">')
links_list = link_regx.findall(req1.content)
for x in links_list:
Expand All @@ -76,7 +78,7 @@ def subdomains_from_netcraft(domain, subdomain_list):
next_page = 21

for x in range(2, num_pages):
url = "http://searchdns.netcraft.com/?host=%s&last=%s&from=%s&restriction=site%%20contains" % (
url = "https://searchdns.netcraft.com/?host=%s&last=%s&from=%s&restriction=site%%20contains" % (
domain, last_item, next_page)
req2 = requests.get(url)
link_regx = re.compile('<a href="http://toolbar.netcraft.com/site_report\?url=(.*)">')
Expand Down
4 changes: 2 additions & 2 deletions domain/domain_zoomeye.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ def search_zoomeye(domain):
time.sleep(0.3)
zoomeye_token = get_accesstoken_zoomeye(domain)
authData = {"Authorization": "JWT " + str(zoomeye_token)}
req = requests.get('http://api.zoomeye.org/web/search/?query=site:%s&page=1' % domain, headers=authData)
req = requests.get('https://api.zoomeye.org/web/search/?query=site:%s&page=1' % domain, headers=authData)
return req.text


Expand All @@ -51,7 +51,7 @@ def main(domain):
def output(data, domain=""):
if type(data) == list and data[1] == "INVALID_API":
print colored(
style.BOLD + '\n[-] ZoomEye username and password not configured. Skipping Zoomeye Search.\nPlease refer to http://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
style.BOLD + '\n[-] ZoomEye username and password not configured. Skipping Zoomeye Search.\nPlease refer to https://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
else:
if 'matches' in data.keys():
print len(data['matches'])
Expand Down
4 changes: 2 additions & 2 deletions emails/email_basic_checks.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ class style:
def basic_checks(email):
if re.match('[^@]+@[^@]+\.[^@]+', email):
if cfg.mailboxlayer_api != "" and cfg.mailboxlayer_api != "XYZ" and cfg.mailboxlayer_api != "" and cfg.mailboxlayer_api != "XYZ":
url = "http://apilayer.net/api/check?access_key=%s&email=%s&smtp=1&format=1" % (cfg.mailboxlayer_api, email)
url = "https://apilayer.net/api/check?access_key=%s&email=%s&smtp=1&format=1" % (cfg.mailboxlayer_api, email)
req = requests.get(url)
resp = json.loads(req.text)
return resp
Expand All @@ -33,7 +33,7 @@ def output(data, email=""):
if data == -1:
print colored(style.BOLD + '\n[-] Please pass a valid email ID.\n' + style.END, 'red')
elif data == -2:
print colored(style.BOLD + '\n[-] MailBoxLayer_API Key not configured. Skipping basic checks.\nPlease refer to http://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
print colored(style.BOLD + '\n[-] MailBoxLayer_API Key not configured. Skipping basic checks.\nPlease refer to https://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
else:
print "Is it a free Email Address?:",
print "No" if not data['free'] else "Yes"
Expand Down
2 changes: 1 addition & 1 deletion emails/email_clearbit.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ def output(data, email=""):
print data
if type(data) == list and data[1] == "INVALID_API":
print colored(
style.BOLD + '\n[-] Clearbit API Key not configured. Skipping Clearbit Search.\nPlease refer to http://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
style.BOLD + '\n[-] Clearbit API Key not configured. Skipping Clearbit Search.\nPlease refer to https://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
else:
for x in data.keys():
print '%s details:' % x
Expand Down
2 changes: 1 addition & 1 deletion emails/email_fullcontact.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ def banner():
def output(data, email=""):
if type(data) == list and data[1] == "INVALID_API":
print colored(
style.BOLD + '\n[-] Full-Contact API Key not configured. Skipping Fullcontact Search.\nPlease refer to http://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
style.BOLD + '\n[-] Full-Contact API Key not configured. Skipping Fullcontact Search.\nPlease refer to https://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
else:
if data.get("status", "") == 200:
if data.get("contactInfo", "") != "":
Expand Down
2 changes: 1 addition & 1 deletion emails/email_pastes.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ def output(data, email=""):
if type(data) == list and not data[0]:
if data[1] == "INVALID_API":
print colored(
style.BOLD + '\n[-] google_cse_key and google_cse_cx not configured. Skipping paste(s) search.\nPlease refer to http://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
style.BOLD + '\n[-] google_cse_key and google_cse_cx not configured. Skipping paste(s) search.\nPlease refer to https://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
else:
print "Error Message: %s" % data[1]['error']['message']
print "Error Code: %s" % data[1]['error']['code']
Expand Down
4 changes: 2 additions & 2 deletions emails/email_slideshare.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ def banner():


def main(email):
req = requests.get('http://www.slideshare.net/search/slideshow?q=%s' % (email))
req = requests.get('https://www.slideshare.net/search/slideshow?q=%s' % (email))
soup = BeautifulSoup(req.content, "lxml")
atag = soup.findAll('a', {'class': 'title title-link antialiased j-slideshow-title'})
slides = {}
Expand All @@ -33,7 +33,7 @@ def output(data, email=""):
if data:
print "Found %s published slides\n" % len(data)
for tl, lnk in data.items():
print str(tl).strip() + " : http://www.slideshare.net" + str(lnk).strip()
print str(tl).strip() + " : https://www.slideshare.net" + str(lnk).strip()
else:
print colored('[-] No Associated Slides found.', 'red')

Expand Down
2 changes: 1 addition & 1 deletion emails/email_whoismind.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ def banner():


def main(email):
req = requests.get('http://www.whoismind.com/email/%s.html' % (email))
req = requests.get('https://www.whoismind.com/email/%s.html' % (email))
soup = BeautifulSoup(req.content, "lxml")
atag = soup.findAll('a')
domains = []
Expand Down
2 changes: 1 addition & 1 deletion ip/ip_shodan.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ def main(ip):
def output(data, ip=""):
if type(data) == list and data[1] == "INVALID_API":
print colored(
style.BOLD + '\n[-] Shodan API Key not configured. Skipping Shodan search.\nPlease refer to http://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
style.BOLD + '\n[-] Shodan API Key not configured. Skipping Shodan search.\nPlease refer to https://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
else:
if 'error' in data.keys():
print 'No information available for that IP.'
Expand Down
4 changes: 2 additions & 2 deletions ip/ip_virustotal.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ def main(ip):
print ip
api = cfg.virustotal_public_api
params = "{'ip': '%s', 'apikey': '%s'}" % (ip, api)
url = "http://www.virustotal.com/vtapi/v2/ip-address/report?ip=%s&apikey=%s" % (ip, api)
url = "https://www.virustotal.com/vtapi/v2/ip-address/report?ip=%s&apikey=%s" % (ip, api)
req = requests.get(url, params)
return req
else:
Expand All @@ -37,7 +37,7 @@ def output(data, ip=""):
# Use the data variable to print out to console as you like
if type(data) == list and data[1] == "INVALID_API":
print colored(
style.BOLD + '\n[-] VirusTotal API Key not configured. Skipping VirusTotal Search.\nPlease refer to http://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
style.BOLD + '\n[-] VirusTotal API Key not configured. Skipping VirusTotal Search.\nPlease refer to https://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
else:
for i in data:
print i
Expand Down
2 changes: 1 addition & 1 deletion mkdocs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ extra:
site_name: DataSploit
site_description: All about DataSploit.
site_author: Shubham Mittal
site_url: http://datasploit.github.io/datasploit/
site_url: https://datasploit.github.io/datasploit/

# Repository
repo_name: datasploit/datasploit
Expand Down
2 changes: 1 addition & 1 deletion username/username_gitscrape.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ def output(data, username=""):
if type(data) == list:
if data[1] == "INVALID_API":
print colored(
style.BOLD + '\n[-] Github Access Token not configured. Skipping Gi Search.\nPlease refer to http://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
style.BOLD + '\n[-] Github Access Token not configured. Skipping Gi Search.\nPlease refer to https://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
else:
print "[+] Found %s repos for username %s\n" % (len(data), username)
counter = 1
Expand Down
2 changes: 1 addition & 1 deletion username/username_twitterdetails.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ def main(username):
def output(data, username=""):
if data[1] == "INVALID_API":
print colored(
style.BOLD + '\n[-] Twitter API Keys not configured. Skipping Twitter search.\nPlease refer to http://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
style.BOLD + '\n[-] Twitter API Keys not configured. Skipping Twitter search.\nPlease refer to https://datasploit.readthedocs.io/en/latest/apiGeneration/.\n' + style.END, 'red')
else:
if data:
hashlist = data[0]
Expand Down