#To block Bytespider from crawling: User-agent: Bytespider Disallow: / #To block Bytedance from crawling: User-agent: Bytedance Disallow: / User-agent: ServerHunter Crawl-delay: 20 User-agent: Googlebot User-agent: AdsBot-Google # Directories, disallow directories Disallow: /clients/ # Paths (clean URLs) #Disallow: https://www.temok.com/clients/ #Disallow: http://temok.com/cliets/ Disallow: /clients/ Sitemap: https://www.temok.com/sitemap.xml Sitemap: https://www.temok.com/sitemap_2.xml User-agent: * Disallow: