Post Reply 
 
Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Robot.txt fles
17-02-2012, 09:44 AM
Post: #1
Robot.txt fles
In SEO What exactly robot.txt file mean and what its work is?
Visit this user's website Find all posts by this user
Quote this message in a reply
17-02-2012, 10:35 AM (This post was last modified: 17-02-2012 10:38 AM by nobluff.)
Post: #2
RE: Robot.txt fles
A robots.txt is a permissions file that can be used to control which webpages of a website a search engine indexes. The file must be located in the root directory of the website for a search engine
The robot.txt contains instruction that are read by spiders.Directives can be written to only target spiders from a specific search engine.
As a website indexing is automated, the program used is often referred to as a ‘robot’ or ‘’bot’
Find all posts by this user
Quote this message in a reply
21-02-2012, 10:58 AM
Post: #3
RE: Robot.txt fles
If the robot.txt file not works properly, then what about my search engine condition? Is it collapse totally?
Find all posts by this user
Quote this message in a reply
08-11-2012, 10:57 AM
Post: #4
RE: Robot.txt fles
Search engines like Google cruise the internet by sending out their spidering software. These are commonly known as spiderbots. The spiderbots visit websites all around the internet to include them in their index listings. The first thing they look for when they visit is a file called the robots.txt file. This file normally is found in the root directory of hosted website.

professional website|web design firms
Visit this user's website Find all posts by this user
Quote this message in a reply
11-01-2013, 07:47 AM
Post: #5
RE: Robot.txt fles
It is a file which is included in website to tell robots ie crawlers not to crawl certain pages which are described in this file.This may contain pages like password , directory of images of your website,etc.


SEO services
Find all posts by this user
Quote this message in a reply
09-10-2013, 11:58 AM
Post: #6
RE: Robot.txt fles
Robots.txt is used to inform Search Engine crawlers that the listed pages are not allowed to crawl in Search Engine. Robots.txt file generally contain pages like directory which contains images, admin side pages, etc. Robots.txt file normally upload on root directory of the websites.

Amazon Clone | Auction Script | Best eBay Clone | eCommerce Script
Visit this user's website Find all posts by this user
Quote this message in a reply
26-06-2015, 07:20 AM
Post: #7
RE: Robot.txt fles
That was really nice explanation.
Find all posts by this user
Quote this message in a reply
07-07-2015, 04:01 PM (This post was last modified: 07-07-2015 04:01 PM by Algernon.)
Post: #8
RE: Robot.txt fles
Hi guys thanks for sharing useful info about robots.txt..
I really appreciate such sharing..I hope you keep sharing such kind of sharing more and more..

___________________
Dubai Entertainment news
Find all posts by this user
Quote this message in a reply
23-02-2016, 06:50 AM
Post: #9
RE: Robot.txt fles
A robots.txt file is a file at the root of your site that indicates those parts of your site you don’t want accessed by search engine crawlers. The file uses the Robots Exclusion Standard, which is a protocol with a small set of commands that can be used to indicate access to your site by section and by specific kinds of web crawlers.

seo company dubai
Find all posts by this user
Quote this message in a reply
18-07-2016, 07:49 AM
Post: #10
RE: Robot.txt fles
Robots.txt is used to select which bot can crawl and index your site

Magento 2 One Step Checkout, Magento 2 Gift Card, Magento 2 Affiliate
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 1 Guest(s)