# the following were all observed by BHP 2026-07-03 # https://developers.facebook.com/docs/sharing/webmasters/crawler # https://developer.amazon.com/support/amazonbot # http://www.baidu.com/search/spider.html # http://www.sogou.com/docs/help/webmasters.htm#07 # https://zhanzhang.toutiao.com/ # ttspider-feedback@tiktok.com # http://www.everyfeed.com # https://webmaster.petalsearch.com/site/petalbot # https://www.semrush.com/bot.html # http://www.blekko.com/help/crawl User-agent: meta-externalagent User-agent: Amazonbot User-agent: Amzn-SearchBot User-agent: Baiduspider User-agent: Sogou User-agent: Bytespider User-agent: TikTokSpider User-agent: everyfeed-spider User-agent: PetalBot User-agent: SemrushBot User-agent: Blekkobot Disallow: / User-agent: * Allow: / Sitemap: https://allofphysics.com/sitemap.txt