Skip to main content

How to Create or Edit robots.txt File For Your Site

In these days blogging is going to be a passion of the people and they use it for the earnings from the Adsense and other ad networks but creating a blog is easy but maintaining the blog is quite difficult so here I am going to discuss the robots.txt file creation as well as editing feature on your site to limit the crawling of the spiders. According to Canada's Top Website Design & Development Company, Search engine spiders crawl your site to index your content whereas sometimes crawlers cannot access your site pages due to the errors on the site which really affects badly on your site as it affects your SEO ranking as well as Your Adsense earnings also. So to limit the crawlers so they cannot access your error files you need to edit your Robots.txt file on your site.

What is a Robots.txt file?

Robots.txt file is robot file on your site which gives them access to the other robots or spiders to your content and which makes your site easily accessible from the spiders(Search engines as well.).

How to create/Make Robots.txt file manually?

Ans. Robots.txt file exists in your site by default and you can access it by going to your dashboard of WordPress and accessing the Setting column and there to the privacy option. But this option is only available in the sites to have WordPress version 3.4 and below. It is not present in 3.5 version of the WordPress. Creating a robot.txt file manually firstly go to notepad then create a note likely to be given below:
This is the best robot.txt file which I have edited below and you have to just change your site name in it and paste it in your notepad and save it as robot.txt.

Sitemap: sitemap link
Sitemap: image sitemap link
user-agent: *
Disallow: 
 User-agent: 
Allow: /

How to edit the robot.txt file by plugin “WordPress SEO By Yoast“?

For the editing of this file to modify something go to the:
  • Plugin settings in SEO column and click on the edit file.
  • There you will get the editor of this file.
  • Then edit and click on the save button your file will be edited.
  • Now check your File by typing yoursite.com/robots.txt

Edkent Media

701 W Georgia St #1500, Vancouver, BC V7Y 1C6
(604) 210-7559

Comments

Post a Comment