S3 Table Integration
This guide provides a step-by-step walkthrough for using Similarweb’s Batch API to create an S3 integration, manage tables, and efficiently extract data.
1
Create a new S3 connection
Start by creating a new S3 integration to store and access data. This setup will generate AWS access and secret keys that allow you to interact with your data securely in the specified S3 bucket.
--url https://api.similarweb.com/batch/v5/s3-connector/setup \
2
Save your credentials
After creating the connection, we’ll provide you with private credentials, including AWS access keys. These keys are essential to access the shared S3 bucket securely. Make sure to store them in a safe place.
{
"aws_access_key": "S3-ACCESS-KEY",
"aws_secret_key": "S3-SECRET-KEY",
"expiration_date": "2025-10-22",
"integration_name": "s3_default",
"s3_bucket": "my_bucket",
"s3_prefix": "account_id/my_new_integration/"
}
3
Generate a report using 'create table' url
Creating a table is similar to requesting a one-time report, but this time, you will modify the delivery information and use the create-table URL. This step allows you to set up a table to store data continuously.
--url https://api.similarweb.com/batch/v5/integration/create-table \
4
Modify delivery information
Adjust the delivery_information section to define where and how the data will be delivered. In this case, set the delivery method to bucket_access to store the data in your S3 bucket.
"delivery_information": {
"response_format": "csv",
"delivery_method": "bucket_access",
"delivery_method_params": {
"integration_name": "s3_default",
"table_name": "test"
}
5
Specify the integration name
The integration_name parameter should always be set to s3_default to direct data to the appropriate integration.
"integration_name": "s3_default",
6
Name your table
Provide a unique name for your table. This name will be used to identify and manage the table in your S3 bucket. Choose a meaningful name that corresponds to the data you will store.
"table_name": "test"
7
Run the request
Once you have filled out the request body, submit the request to create the table. After submitting, the table will be set up in your S3 bucket, and the requested data will be added to it.
8
Retrieve the table path
Upon successful creation of the table, you will receive a location that represents the S3 path where the table data is stored. This path is used to access the stored data.
"location": "s3://web-bulk-api-reports-production-us-east-1/12345678/s3_default/test",
9
Add data to the table
To add more data to the table after it has been created, use the request report URL. This step allows you to append new data to the table.
--url https://api.similarweb.com/batch/v5/request-report \
10
Send data to the table
By specifying the integration name and table name, you can add new data to your existing table. This allows you to keep your data up-to-date with additional information as needed.
"delivery_information": {
"response_format": "csv",
"delivery_method": "bucket_access",
"delivery_method_params": {
"integration_name": "s3_default",
"table_name": "test"
11
Note
"response_format": "csv", "delivery_method": "bucket_access" "integration_name": "s3_default", "table_name": "test" Must be the same as when you created the table
"response_format": "csv",
"delivery_method": "bucket_access",
"delivery_method_params": {
"integration_name": "s3_default",
"table_name": "test"
On this page
- S3 Table Integration