Robots.txt File Preventing Indexing
2 posters
Page 1 of 1
Robots.txt File Preventing Indexing
Hello,
when I search my forum on Google it will say that the description is not available due to my robots.txt file. Also, if I search the contents of a specific post, it doesn't show up on Google but it does on Bing.
It is allowed for guests to see.
I didn't even install such a file nor do I know how to do it. Please help me to get rid of it. What are the possible causes?
I'm using a meta tag called "google-site-verification" and "google-translate-customization".
when I search my forum on Google it will say that the description is not available due to my robots.txt file. Also, if I search the contents of a specific post, it doesn't show up on Google but it does on Bing.
It is allowed for guests to see.
I didn't even install such a file nor do I know how to do it. Please help me to get rid of it. What are the possible causes?
I'm using a meta tag called "google-site-verification" and "google-translate-customization".
Re: Robots.txt File Preventing Indexing
The Google bot only doesn't have access to some specific pages, like:
The rest of the website should be indexed...
What you are telling should be becouse your sub-forums are protected by login. So only registed users could login. The google bot can access that pages and doesn't show the description of the topic/page..
- Code:
Disallow: /abuse
Disallow: /admgt/
Disallow: /contact
Disallow: /donate
Disallow: /go/
Disallow: /login
Disallow: /modcp
Disallow: /post
Disallow: /privmsg
Disallow: /spa/
Disallow: /sta/
The rest of the website should be indexed...
What you are telling should be becouse your sub-forums are protected by login. So only registed users could login. The google bot can access that pages and doesn't show the description of the topic/page..
tugatech- Forumember
- Posts : 186
Reputation : 8
Language : Portuguese
Location : Portugal
Re: Robots.txt File Preventing Indexing
The sub-forums which don't show up on Google are accessible for guests. Bing does pick them up and I can view the content as guest. So you're saying that there's nothing wrong with the robots.txt file? But why does it say that site description etc. are not available? It worked a while ago, I just don't know what changed and why some sub-forums/ topics are no longer picked up by Google.
Last edited by Bully@Wiiplaza on August 31st 2014, 4:42 pm; edited 1 time in total
Re: Robots.txt File Preventing Indexing
Can you give a example?
tugatech- Forumember
- Posts : 186
Reputation : 8
Language : Portuguese
Location : Portugal
Similar topics
» no indexing at all
» No indexing by search engines
» Google Not Indexing Forum because of Mobile App.
» Google Forum Indexing Issue
» Controlling Bot Access & Indexing of Pages
» No indexing by search engines
» Google Not Indexing Forum because of Mobile App.
» Google Forum Indexing Issue
» Controlling Bot Access & Indexing of Pages
Page 1 of 1
Permissions in this forum:
You cannot reply to topics in this forum