Scraping Dark Web Sites with Python

This video demonstrates how to automate interactions and scrape data from Tor hidden services using command-line tools and Python, enabling efficient collection of threat intelligence from the dark web 0:06-0:29.

Key Takeaways:
• To interact with Tor hidden services, install the Tor service and configure the control port in the torrc file, which allows tools like torify to tunnel traffic through the network 0:51-1:30.
• Direct command-line access to onion addresses is possible using curl by specifying a SOCKS5 proxy on port 9050, allowing users to retrieve HTML content without a graphical browser 8:00-9:02.
• Automation in Python is achieved using the requests_tor library, where a session object is initialized with the proxy and control ports to make standard HTTP requests to onion URLs 9:35-12:08.
• This technique is practical for monitoring ransomware leak sites and threat intelligence feeds, such as tracking updates from groups like ALPHV or Black Cat to identify data breaches or scams 13:00-14:02.

The tutorial provides a foundational guide for accessing and scraping Tor hidden services, essential for tracking cyber threats without manual browsing.

Sources:

• 0:06-0:29 Introduction to automating interactions with Tor hidden services
• 0:51-1:30 Configuring Tor service and using torify
• 8:00-9:02 Using curl with SOCKS5 proxy to access onion sites
• 9:35-12:08 Python automation using requests_tor
• 13:00-14:02 Monitoring