Open Source Surveillance ## Sections • [Open Source Surveillance API](https://app.theneo.io/offensive-osint/oss/open-source-surveillance-api.md): This is documentation regarding Open Source Surveillance system - https://os-surveillance.io Search Social media Events Crimes Critical Infrastructure Public Cameras Internet Exposed Devices Wifi networks Transportation system News and more based on coordinates Search Telegram Twitter Instagram Threads Youtube Tiktok based on hashtag • [Authentication](https://app.theneo.io/offensive-osint/oss/open-source-surveillance-api/authorization.md): Every request must contain Authorization Bearer, find it on settings page. • [Workflow](https://app.theneo.io/offensive-osint/oss/open-source-surveillance-api/workflow.md): Add new Workspace (i.e. territory to research). Provide latitude, longitude, radius and name - save returned workspace id for later Maximum radius is 3000000 meters. You can create one big workspace and keep all information there Run search based on provided coordinates Check progress of your task If task is done, make request to get results Or if parameter “completed” increases, it means that module is finished Check if any module failed (Optional) Next time you search, use previously obtained workspace id • [Modules](https://app.theneo.io/offensive-osint/oss/open-source-surveillance-api/modules.md): Open Source Surveillance system supports following sources and categories Social Media airbnb alltrails bikemap booking diveboard findpenguins flickr instagram outdooractive snapchat sports_tracker strava twitter vkontakte weibo wikiloc youtube Events amber_alerts craigslist crimes crimes2 eventbrite experiences events1 events2 events3 events4 events5 facebook_events facebook_marketplace global_incident news offenders ticketmaster weather zillow Critical infrastructure water power emergency military transportation communication sacral Cameras traffic worldcam windy surveillance Networks network bluetooth cell Transportation incidents vessel planes amtrak waze Shodan Cameras screenshot geohttpserver blueiris motioneye dahua hikvision axis vivotek netwave amcrest rtsp netcam Industrial Control Systems nmea siemens crestron moxa iec omron red ipc@chip nordex tank bluekeep modbus bacnet tridium building Internet of Things printers mqtt snmp kasa Vulnerabilities proxyshell bluekeep Other stores newspapaer trestle (address lookup) ai_satellite_mapbox ai_satellite_bing ai_satellite_maptiler ai_satellite_arcgis ai_satellite_here ai_satellite_nasa ai_satellite_landsat ai_satellite_google timelapse Hashtag instagram telegram threads twitter tiktok youtube Username flickr sports_tracker instagram twitter youtube • [Location](https://app.theneo.io/offensive-osint/oss/main-endpoints.md): Below script Creates new workspace with coordinates in New York by making request to /addCoordinates Search Flickr and Vkontakte posts in provided coordinates and bounding box by making request to /search Ask for task status by making request to /getProgress/{search_id} Retrieve search results when it's ready by making request to /getSearchResults/{search_id} More informatoin here https://github.com/woj-ciech/os_surveillance-api Plain text import requests import time headers = { "Authorization": "Bearer XXXX", "Content-Type": "application/json" } BASE_URL = "https://osint.os-surveillance.io/api" modules_to_run = ["flickr", "vkontakte"] #Add new workspace to the database def add_workspace(): url = BASE_URL + "/addCoordinates/" payload = { "title": "New York", "position": { "lat": 40.72020796700515, "lng": -74.0401684945897 }, "radius": 30000 } response = requests.post(url, headers=headers, json=payload) workspace_id = response.json()['workspace_id'] return workspace_id #Search for data in the workspace #pass workspace_id from add_workspace() #include latitude, longitude and bounding box of the area def search(wokspace_id): url = BASE_URL + f"/search/" payload = { "options": modules_to_run, "keywords":None, "date_from":None, "date_to":None, "only_new":False, "coordinates_id":wokspace_id, "lat":40.71846412371451, # center latitude "lng":-73.99740740984262, # center longitude "ne_lat":40.74334880686837, # north east latitude "ne_lng":-73.81237074710944, # north east longitude "sw_lat":40.67945984569057, # south west latitude "sw_lng":-74.16341826297858 # south west longitude } response = requests.request("POST", url, headers=headers, json=payload) search_pk = response.json()['search_pk'] return search_pk #Check progress of the search request def get_progress(search_pk): url = BASE_URL + f"/getProgress/{search_pk}/" response = requests.request("GET", url, headers=headers) return response.json() #Get results of the search request def get_results(search_pk): url = BASE_URL + f"/getSearchResults/{search_pk}/" response = requests.request("GET", url, headers=headers) r_json = response.json() return response.json() def check_errors(): url = BASE_URL + "/checkErrors/" response = requests.request("GET", url, headers=headers) return response.json()['errors'] pp = get_progress("21") workspace_id = add_workspace() search_pk = search(workspace_id) # next time when searching in this area, use same workspace_id while True: progress = get_progress(search_pk) print(progress['progress']) if progress['progress'] == 100.0: results = get_results(search_pk) break else: time.sleep(1) for result in results: for item in result['objects']: print(item) # print(item['title']) # print(item['timestamp']) # print(item['photo_url']) # print(item['location']) • [Add Workspace](https://app.theneo.io/offensive-osint/oss/main-endpoints/add-workspace.md): Create new workspace based on latitude, longitude and radius. Title is also mandatory. Maximum radius is 3000000m. • [Search](https://app.theneo.io/offensive-osint/oss/main-endpoints/search.md): Run modules for given workspace and coordinates. You must provide centre of your search and bounding box. Bounding box must be inside of the bounding box as defined in Workspace. Options parameter contains list of modules to run. flickr vkontakte All modules are described in Modules section. Possible error messages: Please active your account. Coordinates are not in the bounding box Bounding box is not inside the workspace bounding box • [Get progress](https://app.theneo.io/offensive-osint/oss/main-endpoints/get-progress.md): Get progress of the running modules. • [Get search results](https://app.theneo.io/offensive-osint/oss/main-endpoints/get-search-results.md): Get search results after all or some modules finished • [Check errors](https://app.theneo.io/offensive-osint/oss/main-endpoints/check-errors.md): Some modules might not run properly, to find out which ones; endpoint checkErrors needs to be queried, after get progress returned all tasks as done. Include previously obtained “search_pk” • [Additional endpoints](https://app.theneo.io/offensive-osint/oss/main-endpoints/additional-endpoints.md): Below endpoints provide additional features like get workspaces, user information, database search or querying particular item. • [Search database](https://app.theneo.io/offensive-osint/oss/main-endpoints/additional-endpoints/search-database.md): Search across all gathered items in database • [Show all items](https://app.theneo.io/offensive-osint/oss/main-endpoints/additional-endpoints/show-all-items.md): Show all items that belong to the given workspace • [Remove Workspace](https://app.theneo.io/offensive-osint/oss/main-endpoints/additional-endpoints/remove-workspace.md): Remove workspace and associated items • [Get user info](https://app.theneo.io/offensive-osint/oss/main-endpoints/additional-endpoints/get-user-info.md): Get information about current user and credits usage • [Get items](https://app.theneo.io/offensive-osint/oss/main-endpoints/additional-endpoints/get-items.md): Get items for given module and workspace • [Get item](https://app.theneo.io/offensive-osint/oss/main-endpoints/additional-endpoints/get-item.md): Gets item based on workspace id, module type and item id. • [Get Workspaces](https://app.theneo.io/offensive-osint/oss/main-endpoints/additional-endpoints/get-workspaces.md): Gets all workspaces associated with user. • [Hashtag monitoring](https://app.theneo.io/offensive-osint/oss/hashtag-monitoring.md): Search Telegram Twitter Instagram Threads Youtube Tiktok based on hashtag • [Add Hashtag](https://app.theneo.io/offensive-osint/oss/hashtag-monitoring/check-errors-copy-1.md): Add new hashtag to monitor • [Search](https://app.theneo.io/offensive-osint/oss/hashtag-monitoring/add-hashtag-copy-1.md): Search added hashtag by ID on different platforms. • [Get all Hashtags](https://app.theneo.io/offensive-osint/oss/hashtag-monitoring/search-copy-1.md): Get all added hashtags and associated IDs • [Get Hashtag posts by type](https://app.theneo.io/offensive-osint/oss/hashtag-monitoring/get-all-hashtags-copy-1.md): Get collected posts by platfom and ID Example - /getHashtagPosts/twitter/16