봇 인터넷 인덱싱

어떤 봇이 웹사이트를 색인화하나요?

이 페이지는 특정 시점부터 당사의 비밀 루트 도메인 엔드포인트를 방문하는 봇 및 자동화 에이전트를 보여줍니다. 18 6월 2026. 찾기 어렵게 하기 위해 도메인 이름은 거의 사용되지 않는 TLD에 있는 모호한 무작위 문자와 숫자 문자열이었습니다. 웹사이트는 JavaScript, 이미지, 미디어 및 프리미엄 웹 기술을 로드하여 사이트가 얼마나 깊이 색인되었는지 확인했습니다.

경쟁자는 어디에 있나요? 😳

우리의 비밀 웹사이트가 만들어진 이래로 18th 6월 2026, 기술 인덱싱 경쟁업체라고 자칭하는 회사 중 단 한 곳도 해당 웹사이트를 찾지 못했습니다. 에서 109 실험이 진행된 일수입니다.

 

BuiltWith는 해당 웹사이트가 생성된 지 이틀 만에 웹사이트를 발견했습니다. 우리 자신에 대한 내부 정보 없이. 즉, 기술 추적을 위해 경쟁사를 이용한다면 경쟁사가 사이트를 찾더라도 매우 큰 차이로 놓칠 것입니다.

424 식별된 사용자 에이전트가 만들어졌습니다. 121,950 다양한 요청에 대한 답변 74,056 세션에서 10,316 IP 주소. 마지막 생성 날짜 05 10월 2026 00:20 UTC.

anthropic.com 다음을 통해 확인되었습니다. 45 사용자 에이전트, 만들기 1,889 요청 80 IP 주소.

첫 방문은 1 시간, 31 분s, 43 두번째s 사이트가 공개된 후.

10월 2026
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Macintosh; Intel Mac OS X 14_6_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.1 Safari/605.1.15; compatible; ClaudeBot/1.0; +claudebot@anthropic.com

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user) Chrome/147.0.2579.34 Safari/537.36

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (X11; Ubuntu; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.4992.118 Safari/537.36 Edg/131.0.4992.118; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Linux; Android 14; Pixel 8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.1981.4 Mobile Safari/537.36; compatible; ClaudeBot/1.0; +claudebot@anthropic.com

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Linux; Android 12; Pixel 6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/148.0.211.19 Mobile Safari/537.36; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Linux; Android 14; Pixel 8) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +Claude-User@anthropic.com) Chrome/85.0.7775.97 Mobile Safari/537.36

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Linux; Android 15; SM-S918B) AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com) Chrome/131.0.4065.144 Mobile Safari/537.36

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Linux; Android 12; Pixel 6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.2852.228 Mobile Safari/537.36; compatible; ClaudeBot/1.0; +claudebot@anthropic.com

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:149.13) Gecko/20100101 Firefox/149.13; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.6707.118 Mobile Safari/537.36; compatible; Claude-SearchBot/1.0; +https://www.anthropic.com/claude-searchbot

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Linux; Android 12; Pixel 6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.5483.216 Mobile Safari/537.36; compatible; Claude-SearchBot/1.0; +https://www.anthropic.com/claude-searchbot

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-SearchBot/1.0; +https://www.anthropic.com/claude-searchbot) Chrome/151.0.8341.51 Safari/537.36

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Windows NT 11.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +Claude-User@anthropic.com) Chrome/124.0.7846.145 Safari/537.36 Edg/124.0.7846.145

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (X11; Linux x86_64; rv:121.6) Gecko/20100101 Firefox/121.6; compatible; ClaudeBot/1.0; +claudebot@anthropic.com

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.6176.102 Mobile Safari/537.36; compatible; Claude-SearchBot/1.0; +https://www.anthropic.com/claude-searchbot

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +Claude-User@anthropic.com)

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.4332.186 Safari/537.36; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (compatible; Claude-User/1.0; +Claude-User@anthropic.com)

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-SearchBot/1.0; +https://www.anthropic.com/claude-searchbot)

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Linux; Android 13; Pixel 7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.4312.22 Mobile Safari/537.36; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.13) Gecko/20100101 Firefox/121.13; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Macintosh; Intel Mac OS X 14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.1098.181 Safari/537.36; compatible; ClaudeBot/1.0; +claudebot@anthropic.com

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Linux; Android 13; Pixel 7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.7183.226 Mobile Safari/537.36; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Linux; Android 12; Pixel 6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.8328.75 Mobile Safari/537.36; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (compatible; Claude-SearchBot/1.0; +https://www.anthropic.com/claude-searchbot)

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Linux; Android 14; Pixel 8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.1848.119 Mobile Safari/537.36; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user)

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Linux; Android 12; Pixel 6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.1126.222 Mobile Safari/537.36; compatible; ClaudeBot/1.0; +claudebot@anthropic.com

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user)

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user) Chrome/91.0.2234.205 Safari/537.36

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.8512.166 Safari/537.36 Edg/134.0.8512.166; compatible; Claude-SearchBot/1.0; +https://www.anthropic.com/claude-searchbot

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.8171.162 Safari/537.36; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Macintosh; Intel Mac OS X 14_5) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-SearchBot/1.0; +https://www.anthropic.com/claude-searchbot) Chrome/85.0.8300.176 Safari/537.36

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Macintosh; Intel Mac OS X 14_5) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user) Chrome/118.0.2246.25 Safari/537.36 Edg/118.0.2246.25

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (X11; Ubuntu; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +Claude-User@anthropic.com) Chrome/148.0.3976.71 Safari/537.36

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Macintosh; Intel Mac OS X 13_6_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Safari/605.1.15; compatible; Claude-User/1.0; +Claude-User@anthropic.com

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Macintosh; Intel Mac OS X 14_6_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.2937.79 Safari/537.36; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ClaudeBot/1.0; +claudebot@anthropic.com

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.137.158 Mobile Safari/537.36; compatible; Claude-User/1.0; +https://www.anthropic.com/claude-user

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (Macintosh; Intel Mac OS X 14_6_1) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-SearchBot/1.0; +https://www.anthropic.com/claude-searchbot) Chrome/130.0.7044.228 Safari/537.36

84 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (compatible; ClaudeBot/1.0; +claudebot@anthropic.com)

2 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 (compatible; Claude-Web/1.0; +https://www.anthropic.com)

0 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ClaudeBot/1.0; +mailto:support@anthropic.com

0 날 사이트 출시 이후
anthropic.com
anthropic.com Artificial Intelligence

Anthropic develops Claude; its crawlers retrieve public web content for Claude search, user-requested browsing, and model development.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)

0 날 사이트 출시 이후
6월 2026사이트 출시일 18 6월 2026