Log In · Signup for Free

Bot Internet Indexing

What Bots Index Websites?

This page shows the bots visiting our secret root domain endpoint between 30th October 2022 and 30th October 2023. To make it harder to find, the domain name was an obscure random string of letters and numbers on a rarely used TLD. The website loaded JavaScript, Images and Media and premium web technologies to determine how deep the site was being indexed.

Where are the competitors? 😳

Since our secret website was created on 30th October 2022 not a single company that considers themselves a technology indexing competitor found the website in the 365 days the experiment ran for.

BuiltWith found the website within 5 days of the site existing without any insider knowledge from ourselves. This means if you were using a competitor for technology tracking it will miss sites by a very, very large margin if it finds them at all.

AHREFs and Bing are excluded from the results as they indexed the site multiple times per day.

NETCRAFT has visited our secret endpoint 33 times since the website was created on 30th October 2022, 364 days ago. That is on average once every 11 days.

NETCRAFT's first visit was 64 days after the site was made live.

October 2023

25 days, 20 hours, 24 minutes, 44 seconds ago

25 days, 20 hours, 27 minutes, 3 seconds ago

28 days, 6 hours, 11 minutes, 55 seconds ago

30 days, 18 hours, 6 minutes, 16 seconds ago

53 days, 13 hours, 52 minutes, 15 seconds ago

September 2023

55 days, 5 hours, 22 minutes, 8 seconds ago

58 days, 21 hours, 14 minutes, 2 seconds ago

84 days, 19 hours, 18 minutes, 12 seconds ago

85 days, 9 hours, 27 minutes, 57 seconds ago

90 days, 27 minutes, 39 seconds ago

July 2023

111 days, 6 hours, 31 minutes, 59 seconds ago

117 days, 14 hours, 50 minutes, 57 seconds ago

118 days, 7 hours, 19 minutes, 43 seconds ago

121 days, 8 hours, 46 minutes, 55 seconds ago

147 days, 3 hours, 56 minutes, 52 seconds ago

June 2023

147 days, 12 hours, 49 minutes, 7 seconds ago

152 days, 5 hours, 29 minutes, 23 seconds ago

173 days, 12 hours, 41 minutes, 54 seconds ago

179 days, 5 hours, 46 minutes, 58 seconds ago

181 days, 7 hours, 41 minutes, 37 seconds ago

May 2023

205 days, 20 hours, 24 minutes, 9 seconds ago

211 days, 9 hours, 29 minutes, 47 seconds ago

213 days, 15 hours, 55 minutes, 48 seconds ago

237 days, 6 hours, 29 minutes, 29 seconds ago

240 days, 22 hours, 21 minutes, 53 seconds ago

March 2023

243 days, 2 hours, 23 minutes, 47 seconds ago

265 days, 11 hours, 6 minutes, 33 seconds ago

269 days, 4 hours, 16 minutes, 25 seconds ago

271 days, 22 hours, 34 minutes, 1 second ago

295 days, 4 hours, 21 minutes, 2 seconds ago

January 2023

298 days, 1 hour, 38 minutes, 13 seconds ago

299 days, 19 hours, 3 minutes, 32 seconds ago

299 days, 22 hours, 17 minutes, 14 seconds ago

Site Launched on 30 October 2022