Offensive OSINT

Open Source Surveillance API

Authorization

Workflow

Modules

Main endpoints

Add Workspace

Search

Get progress

Get search results

Check errors

Additional endpoints

Search database

Show all items

Remove Workspace

Get user info

Get items

Get item

Get Workspaces

Default

Open Source Surveillance

Production

Sandbox

<p>This is documentation regarding Open Source Surveillance system - <a href="https://os-surveillance.io" target="_blank">https://os-surveillance.io</a></p><p>Search</p><ul><li><div>Social media</div></li><li><div>Events</div></li><li><div>Crimes</div></li><li></li></ul>

This is documentation regarding Open Source Surveillance system - 

<p class="slate-p " >This endpoints will make your geo search possible.</p><p class="slate-p " >Below code creates a new workspace, searches for Flickr and Vkontakte in specific coordinates, in the added workspace, then checkes for progress of the search and gets results at the end.</p><div class="code-block-wrapper" style="width:60%"><div class="code-block-header">Plain text</div><pre class="slate-CodeBlockElement slate-code_block"><code class="language-python keep-markup drop-tokens"><div class="slate-code_line">import requests
import time</div><div class="slate-code_line">headers = {
"Authorization": "Bearer XXXX",
"Content-Type": "application/json"
}
BASE_URL = "https://osint.os-surveillance.io/api"</div><div class="slate-code_line">modules_to_run = ["flickr", "vkontakte"]</div><div class="slate-code_line">#Add new workspace to the database</div><div class="slate-code_line"></div><div class="slate-code_line">def add_workspace():</div><div class="slate-code_line">  url = BASE_URL + "/addCoordinates/"</div><div class="slate-code_line">  </div><div class="slate-code_line">  payload = {</div><div class="slate-code_line">    "title": "New York",</div><div class="slate-code_line">    "position": {</div><div class="slate-code_line">      "lat": 40.72020796700515,</div><div class="slate-code_line">      "lng": -74.0401684945897</div><div class="slate-code_line">    },</div><div class="slate-code_line">    "radius": 30000</div><div class="slate-code_line">  }</div><div class="slate-code_line">  </div><div class="slate-code_line">  response = requests.post(url, headers=headers, json=payload)</div><div class="slate-code_line">  </div><div class="slate-code_line">  workspace_id = response.json()['workspace_id']</div><div class="slate-code_line">  </div><div class="slate-code_line">  return workspace_id</div><div class="slate-code_line"> </div><div class="slate-code_line"> </div><div class="slate-code_line">#Search for data in the workspace</div><div class="slate-code_line">#pass workspace_id from add_workspace()</div><div class="slate-code_line">#include latitude, longitude and bounding box of the area</div><div class="slate-code_line">def search(wokspace_id):
  url = BASE_URL + f"/search/"</div><div class="slate-code_line">  payload = {</div><div class="slate-code_line">      "options":</div><div class="slate-code_line">          modules_to_run,</div><div class="slate-code_line">      "keywords":None,</div><div class="slate-code_line">      "date_from":None,</div><div class="slate-code_line">      "date_to":None,</div><div class="slate-code_line">      "only_new":False,</div><div class="slate-code_line">      "coordinates_id":wokspace_id,</div><div class="slate-code_line">      "lat":40.71846412371451, # center latitude</div><div class="slate-code_line">      "lng":-73.99740740984262, # center longitude</div><div class="slate-code_line">      "ne_lat":40.74334880686837, # north east latitude</div><div class="slate-code_line">      "ne_lng":-73.81237074710944, # north east longitude</div><div class="slate-code_line">      "sw_lat":40.67945984569057, # south west latitude</div><div class="slate-code_line">      "sw_lng":-74.16341826297858 # south west longitude</div><div class="slate-code_line">  }</div><div class="slate-code_line">  </div><div class="slate-code_line">  response = requests.request("POST", url, headers=headers, json=payload)</div><div class="slate-code_line">  </div><div class="slate-code_line">  search_pk = response.json()['search_pk']</div><div class="slate-code_line">  </div><div class="slate-code_line">  return search_pk</div><div class="slate-code_line"></div><div class="slate-code_line">#Check progress of the search request</div><div class="slate-code_line">  def get_progress(search_pk):
  url = BASE_URL + f"/getProgress/{search_pk}/"</div><div class="slate-code_line">  response = requests.request("GET", url, headers=headers)</div><div class="slate-code_line">  </div><div class="slate-code_line">  return response.json()</div><div class="slate-code_line"></div><div class="slate-code_line">#Get results of the search request</div><div class="slate-code_line">def get_results(search_pk):
  url = BASE_URL + f"/getSearchResults/{search_pk}/"</div><div class="slate-code_line">  response = requests.request("GET", url, headers=headers)</div><div class="slate-code_line">  r_json = response.json()</div><div class="slate-code_line">  </div><div class="slate-code_line">  return response.json()</div><div class="slate-code_line"></div><div class="slate-code_line">def check_errors():
  url = BASE_URL + "/checkErrors/"</div><div class="slate-code_line">  response = requests.request("GET", url, headers=headers)</div><div class="slate-code_line">  </div><div class="slate-code_line">  return response.json()['errors']</div><div class="slate-code_line">  pp = get_progress("21")</div><div class="slate-code_line"></div><div class="slate-code_line">workspace_id = add_workspace()
search_pk = search(workspace_id) # next time when searching in this area, use same workspace_id</div><div class="slate-code_line">while True:
  progress = get_progress(search_pk)
  print(progress['progress'])
    if progress['progress'] == 100.0:
      results = get_results(search_pk)
      break
    else:
      time.sleep(1)</div><div class="slate-code_line"></div><div class="slate-code_line">for result in results:
  for item in result['objects']:
    print(item)
    # print(item['title'])
    # print(item['timestamp'])
    # print(item['photo_url'])
    # print(item['location'])</div></code></pre></div><p class="slate-p " ></p><p class="slate-p " ></p>

This endpoints will make your geo search possible.

Below code creates a new workspace, searches for Flickr and Vkontakte in specific coordinates, in the added workspace, then checkes for progress of the search and gets results at the end.

<p class="slate-p " >This is initial endpoint which creates a new workspace based on latitude, longitude and radius. Title is also mandatory.</p><p class="slate-p " >Maximum radius is 3000000m.</p><p class="slate-p " ></p><p class="slate-p " ></p><p class="slate-p " ></p><p class="slate-p " ></p><p class="slate-p " ></p><p class="slate-p " ></p><p class="slate-p " ></p><p class="slate-p " ></p>

This is initial endpoint which creates a new workspace based on latitude, longitude and radius. Title is also mandatory.

title

position

radius

Content-Type

workspace_id

error

<p class="slate-p " >Run modules for given workspace and coordinates.</p><p class="slate-p " >You must provide centre of your search and bounding box.</p><p class="slate-p " >Bounding box must be inside of the bounding box as defined in Workspace.</p><p class="slate-p " ><strong class="slate-bold">Options</strong> parameter contains list of modules to run.</p><ul class="slate-ul " ><li class="slate-li " ><div style="position:relative">flickr</div></li><li class="slate-li " ><div style="position:relative">vkontakte</div></li></ul><p class="slate-p " >All modules are described in section Modules</p><p class="slate-p " >Possible error messages:</p><ul class="slate-ul " ><li class="slate-li " ><div style="position:relative">Please active your account.</div></li><li class="slate-li " ><div style="position:relative">Coordinates are not in the bounding box</div></li><li class="slate-li " ><div style="position:relative">Bounding box is not inside the workspace bounding box</div></li><li class="slate-li " ><div style="position:relative">Bounding box area exceeds the limit of 1500 km²</div></li><li class="slate-li " ><div style="position:relative">Too many modules chosen</div></li></ul><p class="slate-p " ></p><p class="slate-p " ></p>

Run modules for given workspace and coordinates.

You must provide centre of your search and bounding box.

Bounding box must be inside of the bounding box as defined in Workspace.

 parameter contains list of modules to run.

All modules are described in section Modules

options

keywords

date_from

date_to

only_new

coordinates_id

ne_lat

ne_lng

sw_lat

sw_lng

search_pk

List of blocked modules due to lack of credits

blocked_modules

message

<p class="slate-p" placeholder="Type / for widgets">Get progress of the running modules.</p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p>

The unique identifier of the task for which progress is being requested.

task_id

completed

total

progress

<p class="slate-p" placeholder="Type / for widgets">Get search results after all or some modules finished</p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p>

Get search results after all or some modules finished

The unique identifier for the search request that was made in order to retrieve the search results.

<p class="slate-p" placeholder="Type / for widgets">Some modules might not run properly, to find out which ones; endpoint check_errors needs to be queried after get progress returned all tasks as done</p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p><p class="slate-p" placeholder="Type / for widgets"></p>

Some modules might not run properly, to find out which ones; endpoint check_errors needs to be queried after get progress returned all tasks as done

errors

<p>Below endpoints provide additional features like get workspaces, user information, database search or querying particular item.</p><p></p>

Below endpoints provide additional features like get workspaces, user information, database search or querying particular item.