API and BigQuery export

On Pro, Studio and Scale you can take your data out two ways: a read-only API, and a daily BigQuery export. An owner or admin sets both up under Account → API & export.

API keys

Create a key and give it a name. The key is shown once, so copy it then. It reads every site in your account. Remove it any time; anything using it stops working at once.

Send the key in the Authorization header. Every endpoint is under https://sensegap.com/api/v1 and answers JSON.

curl https://sensegap.com/api/v1/sites \
  -H "Authorization: Bearer sg_api_YOUR_KEY"

Endpoints

GETWhat it returns
/sitesYour sites, with their id, domain and time zone.
/sites/{id}One site.
/sites/{id}/dailyBot visits per day. by=bot (default), page, purpose or bot_page (one row per bot, verification result and page).
/sites/{id}/referralsPeople sent by AI, per day, source and landing page.
/sites/{id}/eventsRaw requests, newest first. kind=bot (default), human or all; bot=gptbot for one bot.

daily and referrals take from and to as days (2026-10-01, in the site's time zone, both included). The default is the last 30 days, and they go back as far as your plan's history. events takes from and to as ISO times or Unix milliseconds. The default is the last 24 hours; raw requests are kept 90 days.

curl "https://sensegap.com/api/v1/sites/site_abc123/daily?by=bot&from=2026-10-01&to=2026-10-28" \
  -H "Authorization: Bearer sg_api_YOUR_KEY"

Purposes are the same as in the dashboard: ai_training, ai_search, ai_user, ai_agent, search and other. Fake bots (failed verification) are counted in fake.

Pages of results

Each list returns up to limit rows in data (1,000 by default; 500 for events). When there are more, next_cursor is set: send it back as cursor with the same parameters. When it's null, you have everything. For events, set from and to yourself while paging, so the window doesn't move.

curl "https://sensegap.com/api/v1/sites/site_abc123/events?from=2026-10-28T00:00:00Z&to=2026-10-29T00:00:00Z&cursor=eyJ0cyI6..." \
  -H "Authorization: Bearer sg_api_YOUR_KEY"

Limits and errors

Each key can make 120 requests a minute. Past that you get 429 with a Retry-After header. Errors look like {"error": {"code": "not_found", "message": "…"}}: 401 for a missing or removed key, 403 when your plan doesn't include the API, 404 for a site that isn't yours, 400 for a bad parameter.

BigQuery export

Every day we write yesterday's data into a dataset in your own Google Cloud project. You keep it as long as you like and query it with SQL or Looker Studio.

  1. In BigQuery, create a dataset, for example sensegap.
  2. In IAM & Admin → Service accounts, create a service account. Give it BigQuery Job User on the project and BigQuery Data Editor on the dataset.
  3. Under the service account's Keys, add a JSON key and download it.
  4. In SenseGap, open Account → API & export, paste the key, check the project ID, enter the dataset, and pick all sites or one. Click Test and save.

Saving checks the key, opens the dataset and creates three tables: daily_bot_pages, ai_referrals and events (raw requests). Each is partitioned by day and clustered by site_id. If something is missing, the error says what to fix.

The export runs from 14:00 UTC, once the day is over in every time zone. Each run replaces that day in the tables, so a day is never counted twice. Export now writes the latest day right away.

We store the key encrypted and use it only to write to your dataset. If a run fails three times in a row, the account owner gets an email. Once it works again, we catch up on missed days, up to a week back. To stop, click Remove and delete the key in Google Cloud. Data already in BigQuery stays there.

Still stuck? Write to us.

Email hello@sensegap.com with your site's domain and what you tried. A person reads every message.

Email support

Missing bot or a wrong verdict? The bot list is open source. Open an issue on GitHub.

Open an issue