Login Register






Tor_Crawler. filter_list
Author
Message
Tor_Crawler. #1
Hello,
I downloaded the Tor_Crawler from below URL:
Code:
https://github.com/alex-miller-0/Tor_Crawler
But I can't use it and got below error:
Code:
$ python3.7m Python 3.7.3 (default, Dec 20 2019, 18:57:59) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from TorCrawler import TorCrawler >>> my = TorCrawler() Traceback (most recent call last): File "/home/jason/.local/lib/python3.7/site-packages/socks.py", line 809, in connect negotiate(self, dest_addr, dest_port) File "/home/jason/.local/lib/python3.7/site-packages/socks.py", line 444, in _negotiate_SOCKS5 self, CONNECT, dest_addr) File "/home/jason/.local/lib/python3.7/site-packages/socks.py", line 533, in _SOCKS5_request raise SOCKS5Error("{:#04x}: {}".format(status, error)) socks.SOCKS5Error: 0x05: Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in _new_conn (self._dns_host, self.port), self.timeout, **extra_kw) File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 80, in create_connection raise err File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 70, in create_connection sock.connect(sa) File "/home/jason/.local/lib/python3.7/site-packages/socks.py", line 47, in wrapper return function(*args, **kwargs) File "/home/jason/.local/lib/python3.7/site-packages/socks.py", line 814, in connect raise GeneralProxyError("Socket error", error) socks.GeneralProxyError: Socket error: 0x05: Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen chunked=chunked) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 354, in _make_request conn.request(method, url, **httplib_request_kw) File "/usr/lib/python3.7/http/client.py", line 1244, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1290, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1239, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output self.send(msg) File "/usr/lib/python3.7/http/client.py", line 966, in send self.connect() File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in connect conn = self._new_conn() File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 168, in _new_conn self, "Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f02e10a5278>: Failed to establish a new connection: Socket error: 0x05: Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/jason/.local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 638, in urlopen _stacktrace=sys.exc_info()[2]) File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 398, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='www.icanhazip.com', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f02e10a5278>: Failed to establish a new connection: Socket error: 0x05: Connection refused')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/jason/.local/lib/python3.7/site-packages/TorCrawler.py", line 139, in __init__ self.ip = self.check_ip() File "/home/jason/.local/lib/python3.7/site-packages/TorCrawler.py", line 256, in check_ip return requests.get("http://www.icanhazip.com").text[:-2] File "/home/jason/.local/lib/python3.7/site-packages/requests/api.py", line 75, in get return request('get', url, params=params, **kwargs) File "/home/jason/.local/lib/python3.7/site-packages/requests/api.py", line 60, in request return session.request(method=method, url=url, **kwargs) File "/home/jason/.local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request resp = self.send(prep, **send_kwargs) File "/home/jason/.local/lib/python3.7/site-packages/requests/sessions.py", line 646, in send r = adapter.send(request, **kwargs) File "/home/jason/.local/lib/python3.7/site-packages/requests/adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPConnectionPool(host='www.icanhazip.com', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f02e10a5278>: Failed to establish a new connection: Socket error: 0x05: Connection refused'))
Tor is running:
Code:
# service tor status tor ● tor.service - Anonymizing overlay network for TCP (multi-instance-master) Loaded: loaded (/lib/systemd/system/tor.service; enabled; vendor preset: enabled) Active: active (exited) since Sun 2020-02-23 12:00:45 +0330; 10min ago Process: 17252 ExecStart=/bin/true (code=exited, status=0/SUCCESS) Main PID: 17252 (code=exited, status=0/SUCCESS) Feb 23 12:00:45 Xen-aka-X3n systemd[1]: Starting Anonymizing overlay network for TCP (multi-instance-mast Feb 23 12:00:45 Xen-aka-X3n systemd[1]: Started Anonymizing overlay network for TCP (multi-instance-maste
How can I solve it?

Thank you.

Reply

RE: Tor_Crawler. #2
iirc tor isn't configured as a socks proxy out of the box
https://tor.stackexchange.com/questions/...cks5-proxy

Reply

RE: Tor_Crawler. #3
No, it is configured.
Can I share "torrc" content?

Reply

RE: Tor_Crawler. #4
Yeah im late to this party apparently , but.. tor is a socks proxy out of the box yes on port 9050 by default. I have been using it lately and auto switching ip every 10 seconds rather than using proxys for some cracking configs.


Requests does not come with out of the box socks support so:
pip install pysocks

if all else fails , its a tiny script and its4 years old so it is probably made for a early version of python 3 , I  would just re-code the web request code to use requests over tor heres a little wrapper I made:

Code:
import requests import shutil requests.packages.urllib3.disable_warnings() class HttpWrapper:     def __init__(self,proxy = "localhost:9050"):         self.headers =    headers = {         "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",         "Accept-Encoding": "gzip, deflate, sdch, br",         "Accept-Language": "it-IT,it;q=0.8,en-US;q=0.6,en;q=0.4",         "Connection": "keep-alive",         "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36",     }         self.s = requests.Session()         # self.s.cookies=thecookie         self.useproxy = useproxy         self.proxyDict = {             "http": "socks5://" + proxy,             "https": "socks5://" + proxy         }     def load_cookies_from_lwp(self, filename):         lwp_cookiejar = cookielib.LWPCookieJar()         lwp_cookiejar.load(filename, ignore_discard=True)         return lwp_cookiejar     def setuseragent(self, theagent):         self.headers = {'user-agent': theagent}     def save_cookies_lwp(self, filename):         lwp_cookiejar = cookielib.LWPCookieJar()         for c in self.s.cookies:             args = dict(vars(c).items())             args['rest'] = args['_rest']             del args['_rest']             c = cookielib.Cookie(**args)             lwp_cookiejar.set_cookie(c)         lwp_cookiejar.save(filename, ignore_discard=True)     def get(self, theurl, refferer=""):         if not refferer == "":             self.headers["referer"] = refferer         if self.useproxy == True:             r = self.s.get(theurl, headers=self.headers, proxies=self.proxyDict, verify=False, timeout=15.000)         else:             r = self.s.get(theurl, headers=self.headers, verify=False, timeout=15.000)         return r.text     def post(self, theurl, data, refferer=""):         if not refferer == "":             self.headers["referer"] = refferer         if self.useproxy == True:             r = self.s.post(theurl, data=data, headers=self.headers, proxies=self.proxyDict, verify=False,                             timeout=10.000)         else:             r = self.s.post(theurl, data=data, headers=self.headers, verify=False, timeout=10.000)         return r.text     def download(self,url,file):         response = requests.get(url, stream=True)         with open(file, 'wb') as out_file:             shutil.copyfileobj(response.raw, out_file)         del response


Then we can just use something like

get:
Code:
from HttpWrapper import HttpWrapper wrapper = HttpWrapper() html = wrapper.get("https://www.somelink.onion") print(html)



post
Code:
postdata = {"username" : "MyUsername", "password": "Mypassword"} html = wrapper.post("https://www.somelink.onion",postdata)
(This post was last modified: 03-23-2020, 05:43 PM by Darkbyte.)

Reply