OSINT Toolkit

OSINT Toolkit is one of my first and also most popular personal projects, which I started around 2019/2020 and published in 2022 as an open-source project on GitHub. It is a full-stack application developed using React and Python. It quickly outgrew its initial OSINT feature set, making the name somewhat obsolete. Yet it remains unchanged for now.

It is a self-hostable, on-demand analysis platform designed for cybersecurity specialists. It consolidates various security tools into a single, easy-to-use environment, streamlining everyday tasks. Optimized for single-user operation, OSINT Toolkit runs locally in a Docker container and is not intended for long-term data storage or management. Instead, it focuses on accelerating daily workflows, such as news aggregation and analysis, IOC and email investigations, and more. To further enhance efficiency, OSINT Toolkit also integrates generative AI capabilities, providing additional support for analysis and decision-making.

Screenshot of osint toolkits newsfeed reader using the new full screen design with a sidebar and a navbar on top

Core Features

Newsfeeds: An RSS reader that can extract IOCs and keywords from news articles. It can also analyze articles for relevance using AI and generate a detailed cybersecurity news report summarizing the most important events from the past seven days.
IOC Tools:
- IOC Lookup: Look up IOCs from multiple sources and display results in an easy-to-read table view. Lookup is available for individual IOCs or as a bulk-lookup.
- IOC Extractor: Extracts IOCs from files, removes duplicates, sorts by IOC type, and allows users to analyze IOCs with threat intelligence platforms in one click.
- Defang / Fang: Fang or defang IOCs for save sharing. Works in bulk and removes the manual effort.
Email Analyzer: Analyze potential phishing emails with one click, integrating various threat intelligence services and AI.
Domain Finder: Search for domain patterns to identify potential new phishing domains.
AI Templates: Create prompt templates following prompt engineering best practices, with an option to let AI generate prompts based on the title and description of the template. It’s possible to add static context or fetch context dynamically from websites. Creating templates for repetetive tasks allows users to save time while also receiving high quality and consistant results in a predefined format.
CVSS Calculator: Calculate CVSS scores. Currently support CVSS 3.1 and CVSS 4.0.
Detection Rules: A graphical user interface for creating detection rules. Auto complete makes it easy to create consistent detection rules following best practices.
- Sigma
- Yara
- Snort/Suricate

GitHub repository

The evolution of OSINT Toolkit

The Initial Idea and the Very First Version
2020
A Lockdown project
OSINT Toolkit was a typical lockdown project started in 2019/2020. The initial goal was to simplify IOC lookups across multiple OSINT platforms. That’s why the project was originally named OSINT Buddy. It was my very first attempt at developing a full-stack application on my own.
Adding More Features
2021
More than just OSINT
While developing the application, I came up with more ideas and gradually added new modules to the platform. I also integrated a settings section to configure API keys. The first features were all about IOCs.

Going Public

2021

Popular by Accident

After a big design overhaul and the integration of many more threat intelligence services I created a public GitHub repository. I never promoted the project anywhere and did not expect other people to find it, but the project grew rapidly and gained attention on GitHub and social media soon. I continued implementing integrations with more and more services, including those listed in the table below:

IPs	Domains	URLs	Emails	Hashes	CVEs
AbuseIPDB	Alienvault	Alienvault	Emailrep.io	Alienvault	GitHub
Alienvault	Checkphish.ai	Checkphish.ai	GitHub	GitHub	NIST NVD
Checkphish.ai	GitHub	GitHub	Hunter.io	Maltiverse
CrowdSec	Maltiverse	Google Safe Browsing	Have I Been Pwnd	Pulsedive
GitHub	Pulsedive	Maltiverse	Reddit	Reddit
IPQualityScore	Shodan	Pulsedive	Twitter	ThreatFox
Maltiverse	ThreatFox	Shodan		Twitter
Pulsedive	Reddit	ThreatFox		Virustotal
Shodan	Twitter	Reddit
Reddit	URLScan	Twitter
ThreatFox	Virustotal	URLScan
Twitter		Virustotal
Virustotal

I also introduced dark mode and added a couple more features. One significant addition was a visual editor for creating Yara and Sigma detection rules. OSINT Toolkit was the first tool ever to offer this feature. Around the same time, I started to integrate AI capabilities as GPT-3 gained popularity. screenshot of osint toolkit showing the analysis of an ip adress with results from different threat intelligence services

screenshot of osint toolkit showing the analysis of an ip adress with results from different threat intelligence services

Expanding AI Capabilities
2022
The rise of AI also affected OSINT Toolkit
Generative AI is taking over and also affects OSINT Toolkit. After exploring what GenAI is capable of I implemented some more AI features. In the newsfeed module it now was possible to analyze news articles for relevance based on a company profile that can be configured and also create fully automated news reports showing the most relevant news of the last 7 days and analyze them without human interaction. Also the AI Assistant module was renamed to AI Templates allowing users to create and manage their very own AI automations for automating repetetive tasks in an easy way.
Another Redesign
2024
Making More Space
As the list of modules grew and more AI features were added, the old boxed design became unsuitable. Too many things on too little space made the application more and more uncomfortable to use. Therefore I transitioned to a full-screen layout, giving the application more space in the browser. I also added a sidebar to better organize the growing set of features.
The Future
202_?
The Development continues...
OSINT Toolkit will remain open source. However, since the name OSINT Toolkit no longer accurately reflects its growing functionality, I’m thinking about a new name that better fits its purpose. But I didn't make a final decision yet. Additionally, I’d like to develop a more advanced version with an improved codebase and an upgraded tech stack. This could eventually serve as a commercial cloud platform. But nothing is set in stone yet. Let’s see what the future holds.