Explain what is robots.txt?

DWQA QuestionsCategory: QuestionsExplain what is robots.txt?
Editor Staff asked 1 month ago

Explain what is robots.txt?

1 Answers
Editor Staff answered 1 month ago

Robots.txt is a text file that instructs web crawlers about the pages on your site to crawl or not to crawl.