Robots.txt Checker Free Tool For SEO
Robots.txt tells search crawlers like Google bot or Bing bot how to crawl the website with some instructions. If your website does not have configured robots.txt properly then it will negatively impact on SEO.
Robots.txt tells the user agent which URLs they can follow or not. Search bots first come to robots.txt for sitemap and instructions.