OPEN SOURCE INTELLIGENCE
TOOLS AND RESOURCES HANDBOOK
0
Aleksandra Bielska
Natalie Anderson, Vytenis Benetis, Cristina Viehman
OPEN SOURCE INTELLIGENCE
TOOLS AND RESOURCES HANDBOOK
2018
1
Foreword
I am delighted to share the latest version of our OSINT Tools and Resources Handbook. This
version is almost three times the size of the last public release in 2016. It reflects the changing
intelligence needs of our clients in both the public and private sector, as well as the many
areas we have been active in over the past two years.
No list of OSINT tools is perfect, nor is it likely to be complete. Indeed, such is the pace of
change that by the time you read this document some of our suggestions may have been
surpassed or have ceased to exist. Regrettably, today's tool might also be tomorrow's
vulnerability.
To counter the first problem, we have included a list of toolkits provided by other OSINT
practitioners working to improve the state-of-the-art. To manage the second, we
recommend that all tools be tested in a secure computing environment whenever possible.
Work on the next iteration of the Handbook has already begun. For now, I hope this version
contributes to improving your efficiency and effectiveness as a researcher, analyst,
investigator or general OSINT practitioner. Please feel free to share it with your colleagues.
To encourage its broadest possible dissemination, we are publishing the Handbook under a
Creative Commons CC BY License.
I would like to end by thanking my colleagues at i-intelligence for their efforts in compiling
the Handbook. Particular credit is due to Aleksandra Bielska who led the project and has
coordinated the inputs from different contributors. I am also indebted to Natalie Anderson
of Mercyhurst University for her excellent research and for the time spent checking these
links. Kris Wheaton and Brad Gleason from Mercyhurst’s Department of Intelligence Studies
are also thanked for their support.
Finally, I would like to thank our community of followers on Twitter for sharing their
recommendations with us, as well as the many practitioners who have suggested new tools
to us while participating in our courses.
Chris Pallaris
Director, i-intelligence GmbH
2
Table of Contents
Foreword .............................................................................................................................. 2
Search ................................................................................................................................... 9
General Search.................................................................................................................. 9
Meta Search .................................................................................................................... 11
Visual Search and Clustering Search Engines .................................................................. 12
National Search Engines ................................................................................................. 13
Code Search ................................................................................................................... 16
FTP Search ...................................................................................................................... 17
Similar Sites Search ......................................................................................................... 18
IoT Search Engines .......................................................................................................... 19
Kids-Friendly Search Engines .......................................................................................... 20
Other Specialty Search Engines ...................................................................................... 21
Other Search Engines and Tools ..................................................................................... 23
Social Media ....................................................................................................................... 25
Major Social Networks .................................................................................................... 25
Real-Time Search, Social Media Search, and General Social Media Tools ...................... 27
Social Media Tools: Twitter ............................................................................................. 30
Social Media Tools: Facebook ........................................................................................ 35
Social Media Tools: Google+ .......................................................................................... 37
Social Media Tools: Instagram ........................................................................................ 38
Social Media Tools: Pinterest .......................................................................................... 40
Social Media Tools: Reddit ............................................................................................. 41
Social Media Tools: VKontakte ....................................................................................... 43
Social Media Tools: Tumblr ............................................................................................. 44
Social Media Tools: LinkedIn ........................................................................................... 45
Social Media Tools: Snapchat ......................................................................................... 46
Social Media Tools: WhatsApp ....................................................................................... 47
Social Media Tools: Skype .............................................................................................. 48
Social Media Tools: Telegram ......................................................................................... 49
Other Instant Messaging and Chat Tools ........................................................................ 50
Dating Apps .................................................................................................................... 51
Blogs, Forums, Discussion Boards, and Q&A Sites ............................................................. 52
Blog Search ..................................................................................................................... 52
3
Forums and Discussion Boards Search ............................................................................ 53
Q&A Sites ....................................................................................................................... 54
People Investigations ......................................................................................................... 55
Main People Search Tools and Engines .......................................................................... 55
Address & Contact Information Search ........................................................................... 58
Public Records ................................................................................................................ 60
Ancestry Research ........................................................................................................... 62
Phone Search .................................................................................................................. 63
E-mail Search / E-mail Check .......................................................................................... 66
Username Check ............................................................................................................. 70
CV and Resume Search ................................................................................................... 71
Expert Search .................................................................................................................. 72
Salary Search ................................................................................................................... 74
Creating a Sock Puppet .................................................................................................. 76
Company Research ............................................................................................................. 77
Main Company Search Tools and Engines ...................................................................... 77
Company Research (EU) .................................................................................................. 80
Company Research (UK).................................................................................................. 81
Company Research (US) .................................................................................................. 84
Company Research (Switzerland) .................................................................................... 86
Company Research (Middle East) ................................................................................... 87
Company Research (Africa) ............................................................................................. 88
Company Research (China) ............................................................................................. 90
Business Registers (EU + EFTA) ...................................................................................... 91
Business Registers (US) ................................................................................................... 93
Business Registers (Other) .............................................................................................. 95
Business Registers Lists and Directories .......................................................................... 97
Job Search Resources ..................................................................................................... 99
Company Reviews ......................................................................................................... 101
Patent Research ............................................................................................................ 102
Tenders ......................................................................................................................... 104
Charity, NGO, and Nonprofit Research ........................................................................ 106
Real Estate Research ........................................................................................................ 108
Real Estate Research ..................................................................................................... 108
Online Marketplaces......................................................................................................... 111
Classifieds and Online Marketplaces ............................................................................ 111
Cryptocurrencies and Financial Information ..................................................................... 114
Cryptocurrencies ........................................................................................................... 114
4
Financial Information ..................................................................................................... 115
Legal Research ................................................................................................................. 116
Legal Research Reources and Tools .............................................................................. 116
Researching Countries ...................................................................................................... 119
Country Profiles ............................................................................................................ 119
Researching Terrorism, Crime, and Cybersecurity ............................................................ 120
Researching Terrorism .................................................................................................. 120
Researching Crime ........................................................................................................ 122
Researching Cybersecurity ............................................................................................ 123
Researching Stolen Items ................................................................................................. 125
Researching Stolen Items .............................................................................................. 125
Researching Human Trafficking ........................................................................................ 127
Researching Human Trafficking ..................................................................................... 127
Transportation .................................................................................................................. 131
Airplanes ....................................................................................................................... 131
Drones .......................................................................................................................... 133
Cars ............................................................................................................................... 135
Railways ........................................................................................................................ 137
Public Transport ............................................................................................................ 141
Ships ............................................................................................................................. 144
Container and Cargo Tracking ...................................................................................... 146
Web Intelligence .............................................................................................................. 147
Domain and IP Research, Websites Analysis ................................................................. 147
Web History and Website Capture ............................................................................... 155
Wifi Search .................................................................................................................... 157
Researching Dark Web ..................................................................................................... 158
Researching Dark Web.................................................................................................. 158
Events Search ................................................................................................................... 160
Events Search ................................................................................................................ 160
Working with Images ........................................................................................................ 161
Image Search and Reverse Image Search ..................................................................... 161
Image Analysis .............................................................................................................. 164
Stock Images ................................................................................................................. 167
Image and Photo Editing .............................................................................................. 169
OCR Tools ..................................................................................................................... 171
Working with Videos ........................................................................................................ 172
5
Video Search and Reverse Video Search ....................................................................... 172
Live Streaming and Webcams ....................................................................................... 174
Other Video Tools ......................................................................................................... 176
Video Editors ................................................................................................................ 179
Working with Audio .......................................................................................................... 181
Radio ............................................................................................................................. 181
Podcasts ........................................................................................................................ 183
Other Audio Tools ........................................................................................................ 185
Working with Documents ................................................................................................. 186
Documents and Slides Search ....................................................................................... 186
Pastebins ....................................................................................................................... 188
Document and Reference Management ....................................................................... 189
PDF Management ......................................................................................................... 190
Working with Document Metadata ............................................................................... 191
Academic Resources and Grey Literature ......................................................................... 192
Academic and Grey Literature Research Tools ............................................................. 192
Lectures, Talks, Conference Presentations, Speeches ...................................................... 196
Lectures, Talks, Conference Presentations, Speeches ................................................... 196
Books and Reading ........................................................................................................... 198
Books Search and Reading Tools .................................................................................. 198
Working with Data and Statistics ...................................................................................... 201
Data and Statistics ........................................................................................................ 201
Infographics and Data Visualization .............................................................................. 206
Dashboard Tools ........................................................................................................... 209
Data Analysis................................................................................................................. 210
Data Scrapers ................................................................................................................ 212
Working with News .......................................................................................................... 214
News Sources ............................................................................................................... 214
News Digest and Discovery Tools ................................................................................. 216
Fact Checking ............................................................................................................... 217
Working with Foreign Language Content ........................................................................ 219
Translation Tools ........................................................................................................... 219
Web Monitoring ............................................................................................................... 222
RSS Readers .................................................................................................................. 222
Other RSS Tools ............................................................................................................ 224
Website Monitoring ...................................................................................................... 225
Social Media Monitoring ............................................................................................... 226
6
Other Monitoring Tools ................................................................................................ 227
Data Collation................................................................................................................... 228
Bookmarking ................................................................................................................. 228
Notetaking .................................................................................................................... 231
Annotation and Highlighting......................................................................................... 233
Screenshot and Screen Capture Tools .......................................................................... 234
Browsing and Browsers .................................................................................................... 236
Browsers ....................................................................................................................... 236
Startpages ..................................................................................................................... 238
Customising New Tab ................................................................................................... 239
Tab Management .......................................................................................................... 241
Extension Managers ...................................................................................................... 243
Offline Browsing ........................................................................................................... 244
Browsing History ........................................................................................................... 245
Other Browsing Tools ................................................................................................... 246
Writing and Dissemination ............................................................................................... 247
Writing and Office Tools ............................................................................................... 247
Slideshow and Presentation Tools ................................................................................ 249
Digital Publishing .......................................................................................................... 251
Newsletter Tools ........................................................................................................... 252
Digital Storytelling ........................................................................................................ 253
Wikis, Websites, and Web Apps ....................................................................................... 255
Wikis ............................................................................................................................. 255
Website Builders ........................................................................................................... 256
App Builders ................................................................................................................. 258
Mind Mapping, Concept Mapping and Idea Generation ................................................. 259
Mind Mapping, Concept Mapping and Idea Generation Tools .................................... 259
Productivity and File Management ................................................................................... 261
Main Work Management and Productivity Tools .......................................................... 261
E-mail Management ...................................................................................................... 264
Cloud Storage and File Sharing .................................................................................... 266
Web Automation .......................................................................................................... 268
Collaboration and Communication ................................................................................... 269
Collaboration and Project Management ....................................................................... 269
Communication Tools ................................................................................................... 273
Other Utility Tools ............................................................................................................ 276
File Converters .............................................................................................................. 276
7