| Linux hosting5.siteguarding.com 3.10.0-962.3.2.lve1.5.88.el7.x86_64 #1 SMP Fri Sep 26 14:06:42 UTC 2025 x86_64 Path : /home/devsafetybis/verd.dev.safetybis.com/ |
| Current File : /home/devsafetybis/verd.dev.safetybis.com/robots.txt |
User-agent: Scrapy Allow: / User-agent: * Disallow: User-agent: TurnitinBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: TwengaBot Disallow: / User-agent: 008 Disallow: / User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: WotBox Disallow: / User-agent: Sosospider Disallow: / User-agent: Baiduspider Disallow: / User-agent: ZumBot Disallow: / User-agent: YandexBot Disallow: /