Bot Internet Indexing

What Bots Index Websites?

This page shows the bots visiting our secret root domain endpoint between 30th October 2022 and 30th October 2023. To make it harder to find, the domain name was an obscure random string of letters and numbers on a rarely used TLD. The website loaded JavaScript, Images and Media and premium web technologies to determine how deep the site was being indexed.

Where are the competitors? ๐Ÿ˜ณ

Since our secret website was created on 30th October 2022 not a single company that considers themselves a technology indexing competitor found the website in the 365 days the experiment ran for.

 

BuiltWith found the website within 5 days of the site existing without any insider knowledge from ourselves. This means if you were using a competitor for technology tracking it will miss sites by a very, very large margin if it finds them at all.

AHREFs and Bing are excluded from the results as they indexed the site multiple times per day.

CENSYS has visited our secret endpoint 61 times since the website was created on 30th October 2022, 364 days ago. That is on average once every 6 days.

CENSYS's first visit was 30 days after the site was made live.

October 2023
CENSYS

https://censys.io
Attack Surface Management Tool

4 days, 4 hours, 49 minutes, 15 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

4 days, 6 hours, 27 minutes, 20 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

6 days, 7 hours, 1 minute, 28 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

6 days, 7 hours, 17 minutes, 24 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

21 days, 6 hours, 21 minutes, 16 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

21 days, 6 hours, 24 minutes, 15 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

24 days, 15 hours, 21 minutes, 20 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

24 days, 22 hours, 3 minutes, 20 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

26 days, 11 hours, 20 minutes, 23 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

26 days, 15 hours, 35 minutes, 46 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

36 days, 11 hours, 28 minutes, 42 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

37 days, 17 hours, 21 minutes, 48 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

59 days, 17 hours, 16 minutes, 37 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

60 days, 22 minutes, 22 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

63 days, 8 hours, 43 minutes, 49 seconds ago
August 2023
CENSYS

https://censys.io
Attack Surface Management Tool

63 days, 9 hours, 48 minutes, 22 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

65 days, 3 hours, 34 minutes, 16 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

65 days, 23 hours, 22 minutes, 24 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

68 days, 6 hours, 23 minutes, 39 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

68 days, 6 hours, 47 minutes, 32 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

104 days, 16 hours, 10 minutes, 37 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

104 days, 16 hours, 39 minutes, 14 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

105 days, 18 hours, 24 minutes, 36 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

105 days, 19 hours, 38 minutes, 45 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

112 days, 16 hours, 22 minutes, 21 seconds ago
July 2023
CENSYS

https://censys.io
Attack Surface Management Tool

112 days, 19 hours, 44 minutes, 51 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

116 days, 3 hours, 24 minutes, 38 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

116 days, 4 hours, 13 minutes, 22 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

129 days, 21 hours, 36 minutes, 24 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

130 days, 1 hour, 14 minutes, 20 seconds ago
June 2023
CENSYS

https://censys.io
Attack Surface Management Tool

144 days, 6 hours, 35 minutes, 26 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

144 days, 6 hours, 51 minutes, 53 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

175 days, 18 hours, 57 minutes, 34 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

175 days, 23 hours, 42 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

179 days, 20 hours, 59 minutes, 39 seconds ago
May 2023
CENSYS

https://censys.io
Attack Surface Management Tool

181 days, 6 hours, 43 minutes, 22 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

181 days, 6 hours, 51 minutes, 49 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

187 days, 11 hours, 57 minutes, 5 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

187 days, 12 hours, 55 minutes, 54 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

190 days, 1 hour, 8 minutes, 41 seconds ago
April 2023
CENSYS

https://censys.io
Attack Surface Management Tool

190 days, 5 hours, 7 minutes, 29 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

192 days, 5 hours, 34 minutes, 56 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

192 days, 14 hours, 53 minutes, 5 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

203 days, 13 hours, 39 minutes, 2 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

203 days, 19 hours, 2 minutes, 40 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

205 days, 12 hours, 27 minutes, 1 second ago
CENSYS

https://censys.io
Attack Surface Management Tool

205 days, 22 hours, 8 minutes, 48 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

215 days, 10 hours, 32 minutes, 25 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

215 days, 16 hours, 13 minutes, 36 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

217 days, 10 hours, 55 minutes, 27 seconds ago
March 2023
CENSYS

https://censys.io
Attack Surface Management Tool

218 days, 3 hours, 15 minutes, 41 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

220 days, 15 hours, 9 minutes, 37 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

220 days, 17 hours, 55 minutes, 49 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

232 days, 6 hours, 18 minutes, 36 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

232 days, 10 hours, 34 minutes, 36 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

244 days, 14 hours, 29 minutes, 33 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

244 days, 18 hours, 25 minutes, 27 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

273 days, 9 hours, 41 minutes, 38 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

284 days, 10 hours, 46 minutes, 39 seconds ago
CENSYS

https://censys.io
Attack Surface Management Tool

303 days, 11 hours, 19 minutes, 38 seconds ago
December 2022
CENSYS

https://censys.io
Attack Surface Management Tool

334 days, 2 hours, 42 minutes, 34 seconds ago
SSL Added 29 November 2022 Site Launched on 30 October 2022