Bot Management
Clickport automatically detects and blocks bot traffic before it reaches your analytics data. Every incoming event passes through a multi-layer detection system that identifies crawlers, scrapers, monitoring tools, and other non-human visitors. Blocked events are counted separately and surfaced in the Bot Center dashboard so you can see exactly what is being filtered.
A note on detail. This page describes what the bot detection system does, not how it works internally. Publishing the exact detection logic would make it easier for bad actors to craft bots that bypass filtering. If you have questions about a specific detection method, reach out through the
contact page.
How bot detection works
When a tracking event arrives at the API, it passes through multiple detection layers. If any layer identifies the visitor as a bot, the event is blocked immediately and never written to your analytics tables.
Detection layers
- Client-side checks: The tracker runs several checks in the browser before sending any data. If it detects signs of browser automation or a non-production environment, it silently exits without sending events.
- User agent pattern matching: The server matches the request's User-Agent header against a curated list of known bot patterns. This covers search engine crawlers, AI bots, SEO tools, social media preview bots, monitoring services, HTTP libraries, and vulnerability scanners.
- Datacenter IP detection: The visitor's IP address is checked against a blocklist of datacenter and cloud provider IP ranges. Real visitors browse from residential or mobile networks, not from AWS or Google Cloud. To avoid false positives, known VPN provider ranges are whitelisted so legitimate users on VPNs are not blocked.
- Spam referrer filtering: The referrer URL is checked against a maintained list of thousands of known spam domains.
- Viewport and header analysis: Additional server-side heuristics examine request metadata for patterns that indicate non-browser clients.
120
bots blocked in last 30 days (6.5% of traffic)
+ 22 manually flagged
Detection Breakdown
Blocklists
The bot detection system relies on multiple blocklists sourced from established open-source projects. These lists are automatically downloaded, cached on the server, and refreshed periodically. If a remote fetch fails, the previous cached version is used as a fallback.
Blocklist Statistics
Auto-updated from open-source sources
Datacenter IPs
Cloud provider IP ranges
3,430 ranges
VPN Ranges (whitelisted)
Excluded from datacenter blocking
5,400 ranges
Spam Domains
Known referrer spam sources
2,322 domains
Bot UA Patterns
Built-in pattern list
79 patterns
- Datacenter IPs: IP ranges for major cloud providers. Real visitors browse from residential or mobile networks, so traffic from cloud infrastructure is a strong bot signal.
- VPN ranges: Known VPN provider IP ranges. These are whitelisted to prevent false positives, since legitimate visitors sometimes use VPN services that route through datacenter IPs.
- Spam referrers: Domains known for referrer spam. Both exact domain matches and parent domain lookups are performed.
- Bot UA patterns: A curated pattern list covering search engine crawlers, AI bots, SEO tools, social media bots, monitoring services, HTTP libraries, and more.
Bot Center dashboard
The Bot Center is a tab inside the Settings panel on the dashboard. It shows a summary of bot blocking activity for the last 30 days and provides visibility into what is being filtered.
What the Bot Center shows
- Bot Protection status: A green "Active" indicator confirming that detection is running.
- Blocked count: The total number of bot events blocked in the last 30 days, along with the percentage of total traffic this represents.
- Manually flagged count: The number of sessions that were manually flagged as bots by you (see below).
- Detection breakdown: A visual bar chart showing which detection method caught the most bots. Common breakdowns include Bot User Agents, Datacenter IPs, Spam Referrers, Headless Browsers, and No Viewport.
- Top blocked: The most frequently blocked sources. For user agent detections, this shows the bot name (Googlebot, AhrefsBot). For datacenter IP detections, this shows the provider (Apple Inc., Amazon).
- Blocklist stats: Current counts for each blocklist source with the last update date.
Note: Bot statistics are stored with a 90-day retention period. Older data is automatically cleaned up, so the Bot Center always reflects recent activity.
Manual session flagging
Sometimes a bot slips through automated detection. When you spot a suspicious session in the Sessions panel, you can manually flag it as a bot.
How flagging works
- Open the Sessions panel from the Content tab on the dashboard.
- Find the suspicious session. Look for signs like: a single pageview with zero scroll depth, very short duration, or a datacenter-like browsing pattern.
- Expand the session to see its detail view with page-by-page data.
- Click Flag as Bot in the session actions.
Session detail with Flag as Bot action
Entry page
/pricing
Duration
0s
Scroll depth
0%
Pageviews
1
Browser
Chrome 120
Country
US
Flag as Bot
Delete Session
When you flag a session as a bot, Clickport marks it internally so that all dashboard queries exclude it. The session's events are also updated to prevent them from appearing in page-level panels. This means the flagged session is effectively removed from all dashboard metrics retroactively.
You can also delete a session entirely using the Delete Session action. This permanently removes both the session record and all its associated events from ClickHouse.
Important: Flagging a session as a bot is permanent. There is no "unflag" action. If you flag a session by mistake, you would need to delete the session data manually. For most cases, flagging is the preferred approach because it preserves the data for auditing while excluding it from your analytics.
IP exclusion
In addition to bot detection, you can exclude specific IP addresses from tracking entirely. This is useful for filtering out your own visits and those of your team. Excluded IPs are checked at ingestion time before any event processing occurs.
IP exclusion is managed in the Dashboard tab of the Settings panel. The interface shows your current IP address and lets you add or remove it with a single click. All excluded IP addresses are stored in your site settings and take effect immediately.
Unlike bot flagging, IP exclusion is preventive. Events from excluded IPs are silently dropped before they enter the analytics pipeline. No data is recorded and no bot statistics are counted for these requests.
Client-side detection
The tracker includes its own first line of defense before any data is sent to the server. It checks for common signs of browser automation, headless environments, and non-production contexts. If any check triggers, the tracker silently exits without sending events.
The tracker also does not fire on localhost or local file URLs, so development and testing environments never generate tracking data.
How blocked events are recorded
When a bot is detected at the server level, the event is not inserted into your analytics tables. Instead, the detection is counted and categorized so the Bot Center can display aggregate statistics. Blocked events are stored with the detection method, the source that triggered the block (e.g., the bot name or IP provider), and a daily counter. This data is retained for 90 days and cleaned up automatically.