Log In · Signup for Free
EN  DE  FR  ES  IT  ะ ัƒััะบะธะน  ํ•œ๊ตญ์–ด  ๆ—ฅๆœฌ่ชž  ็ฎ€ไฝ“ไธญๆ–‡  ุนุฑุจูŠ

Bot Internet Indexing

What Bots Index Websites?

This page shows the bots and automated agents visiting our secret root domain endpoint since 18 June 2026. To make it harder to find, the domain name was an obscure random string of letters and numbers on a rarely used TLD. The website loaded JavaScript, Images and Media and premium web technologies to determine how deep the site was being indexed.

Where are the competitors? ๐Ÿ˜ณ

Since our secret website was created on 18th June 2026, not a single company that considers themselves a technology indexing competitor found the website in the 64 days the experiment has run for.

 

BuiltWith found the website within 2 days of the site existing without any insider knowledge from ourselves. This means if you were using a competitor for technology tracking it will miss sites by a very, very large margin if it finds them at all.

183 identified user agents made 84,857 requests across 53,653 sessions from 8,926 IP addresses. Data last generated 20 August 2026 00:20 UTC.

google.com was identified through 27 user agents, making 1,781 requests from 72 IP addresses.

Its first visit was 19 hours, 35 minutes after the site was made live.

Clear Filter
August 2026
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/151.0.7922.137 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Requests: 2 · IPs: 2 · Pages: 2 · Assets: 0
2026-08-18 02:08:28 UTC
60 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/151.0.7922.108 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

โ˜•
Requests: 21 · IPs: 1 · Pages: 1 · Assets: 4
2026-08-14 04:43:25 UTC
57 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/151.0.7922.108 Safari/537.36

๐Ÿ”— โ˜•
Requests: 21 · IPs: 1 · Pages: 1 · Assets: 5
2026-08-14 04:43:24 UTC
57 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/150.0.7871.186 Safari/537.36

๐Ÿ”— ๐Ÿ–ผ โ˜•
Requests: 32 · IPs: 4 · Pages: 1 · Assets: 11
2026-07-31 15:42:29 UTC
43 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.7871.186 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

๐Ÿ”— ๐Ÿ–ผ โ˜•
Requests: 228 · IPs: 6 · Pages: 7 · Assets: 53
2026-07-30 05:39:01 UTC
42 days since site launch
July 2026
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Googlebot/2.1 (+http://www.google.com/bot.html)

โ˜•
Requests: 176 · IPs: 2 · Pages: 140 · Assets: 18
2026-07-24 08:42:32 UTC
36 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.7871.128 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

๐Ÿ”— ๐Ÿ–ผ โ˜•
Requests: 112 · IPs: 19 · Pages: 6 · Assets: 32
2026-07-20 16:04:26 UTC
32 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.7871.124 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

๐Ÿ”— ๐Ÿ–ผ โ˜•
Requests: 29 · IPs: 13 · Pages: 3 · Assets: 9
2026-07-16 14:25:23 UTC
28 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.7871.114 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Requests: 2 · IPs: 2 · Pages: 2 · Assets: 0
2026-07-16 05:42:14 UTC
28 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/150.0.7871.46 Safari/537.36

โ˜•
Requests: 9 · IPs: 4 · Pages: 1 · Assets: 1
2026-07-14 17:15:47 UTC
26 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +https://www.google.com/bot.html) Safari/537.36

Requests: 1 · IPs: 1 · Pages: 1 · Assets: 0
2026-07-05 03:15:12 UTC
16 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.7871.46 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

๐Ÿ”— ๐Ÿ–ผ โ˜•
Requests: 184 · IPs: 26 · Pages: 6 · Assets: 45
2026-07-03 01:52:50 UTC
14 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/149.0.7827.200 Safari/537.36

๐Ÿ”— ๐Ÿ–ผ โ˜•
Requests: 52 · IPs: 11 · Pages: 1 · Assets: 10
2026-07-02 23:05:16 UTC
14 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/149.0.7827.200 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

๐Ÿ”— ๐Ÿ–ผ โ˜•
Requests: 87 · IPs: 14 · Pages: 3 · Assets: 11
2026-07-01 14:09:41 UTC
13 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)

Requests: 1 · IPs: 1 · Pages: 1 · Assets: 0
2026-06-30 00:31:00 UTC
11 days since site launch
June 2026
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (iPhone; U; CPU iPhone OS) (compatible; Googlebot-Mobile/2.1; http://www.google.com/bot.html)

Requests: 1 · IPs: 1 · Pages: 1 · Assets: 0
2026-06-30 00:24:38 UTC
11 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Requests: 1 · IPs: 1 · Pages: 1 · Assets: 0
2026-06-29 04:07:21 UTC
10 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; http://www.google.com/bot.html)

Requests: 1 · IPs: 1 · Pages: 1 · Assets: 0
2026-06-29 02:20:11 UTC
10 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; http://www.google.com/bot.html)

Requests: 1 · IPs: 1 · Pages: 1 · Assets: 0
2026-06-25 16:33:11 UTC
7 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
AdsBot-Google ( http://www.google.com/adsbot.html)

Requests: 2 · IPs: 2 · Pages: 2 · Assets: 0
2026-06-25 16:00:55 UTC
7 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Googlebot-Image/1.0

๐Ÿ–ผ
Requests: 19 · IPs: 15 · Pages: 1 · Assets: 18
2026-06-20 23:02:08 UTC
2 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/148.0.7778.96 Safari/537.36

๐Ÿ”— ๐Ÿ–ผ โ˜•
Requests: 66 · IPs: 17 · Pages: 1 · Assets: 16
2026-06-20 23:01:45 UTC
2 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.84 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Requests: 15 · IPs: 11 · Pages: 1 · Assets: 0
2026-06-20 23:01:42 UTC
2 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Googlebot-News

โ˜•
Requests: 4 · IPs: 3 · Pages: 3 · Assets: 1
2026-06-20 00:57:55 UTC
1 day since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/148.0.7778.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

๐Ÿ”— ๐Ÿ–ผ โ˜•
Requests: 216 · IPs: 22 · Pages: 5 · Assets: 64
2026-06-19 17:50:38 UTC
1 day since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

๐Ÿค– โ˜•
Requests: 387 · IPs: 56 · Pages: 151 · Assets: 22
2026-06-19 00:06:05 UTC
0 days since site launch
google.com
google.com Search

Google operates crawlers including Googlebot to discover and index public content for Search and related services.
Mozilla/5.0 (compatible; Google-Extended/1.0; +http://www.google.com/bot.html)

โ˜•
Requests: 111 · IPs: 4 · Pages: 99 · Assets: 2
2026-06-19 00:05:58 UTC
0 days since site launch
Site Launched on 18 June 2026