The forum of the forums
Welcome to the Official Support Forum of Forumotion!

To take full advantage of everything offered by our forum, please log in if you are already a member, or join our community if you've not yet.



Create a free forum like this one.

Allow more bots on robots.txt(alexa and ask)

View previous topic View next topic Go down

Allow more search engine bots to crawl out forums

95% 95% 
[ 35 ]
5% 5% 
[ 2 ]
 
Total Votes : 37

Allow more bots on robots.txt(alexa and ask)

Post by thealbertross on Fri Feb 06, 2009 3:23 pm

Id very much like the robots.txt page on all of our forums to be altered. as we dont have ftp access, perhaps admin of forumotion could change the text,

currently, we are only allowing google, msn and yahoo to crawl and index our forums, and blocking dozens of other bots including alexa and ask.

would you please agree to this suggestion so we can perhaps get crawled by more search engines?

Thanks

thealbertross
Forumember

Posts : 95
Reputation : 0
Language : english

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by oleander on Fri Apr 03, 2009 8:59 am

has this been done yet?

oleander
Forumember

Posts : 48
Reputation : 0
Language : english

http://advertiseforfree.forumotion.com/forum.htm

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by Jaffa909 on Sat Aug 28, 2010 8:25 am

I votes yes becuase our forums get more visits and more people that join our forums they might join forumotion! But if they allowed all search engine bots and you submitted to like one of those websites where they send like 500 search bots your forum might not be able to handle 500+ bots indexing andn scanning all the posts at one might overload it! but very nice idea

Jaffa909
Forumember

Posts : 57
Reputation : 0
Language : English
Location : Untied kindom

http://www.clubpenguingang.net

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by StoneRaven on Sat Aug 28, 2010 8:37 am

I vote yes. The more we can get crawled the more we show up on more search engines which ='s more real visitors. I don't see 500+ bots at one time though maybe 3 or 4 at a time.
avatar
StoneRaven
Forumember

Male Posts : 142
Reputation : 0
Language : English

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by Walshy95 on Sat Aug 28, 2010 12:58 pm

Voted yes. Great idea. Would be a great way of publicity.
avatar
Walshy95
Active Poster

Male Posts : 1308
Reputation : 6
Language : English
Location : UK, Liverpool

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by Cadence on Sun Sep 11, 2011 6:55 am

good!! I have bots on all the time (Yahoo) and i wanna make my user a bot so when i log in chatbox it will chat to anyone there... how do i add a bot user to chat box?????
avatar
Cadence
New Member

Posts : 9
Reputation : 1
Language : English

http://multipenguin.free-message-board.com/

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by LoveStudio on Sun Sep 11, 2011 7:33 am

I like this héhé
avatar
LoveStudio
New Member

Male Posts : 23
Reputation : 2
Language : Vietnamese
Location : >"<

http://www.9xstudio.biz

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by Cadence on Mon Sep 12, 2011 6:44 pm

Well, bots are from "official" websites. Usually they make bots so they're website may come back with Info from your forum, brought to theres so they may add to it...You can dis bots, most likely, I wouldn't. More searches CLOSE or FAR OFF searches feature your weblink, so its a far trade Wink bots are useful, but if you seem to not like them I guess not...Oh and the reason you cannot edit that is because, the CSS is the theme, look, style and sorts of functions, it does not add bots. If it were a bot to be added, most likely to be added in the GUEST section and not do any data modifiers left behind, so I recommend not to make/add a bot, you must know your FULL WAY throw coding, plus, it wouldn't be are your website!! 0.0 it go through others, and you cannot see the data...I hope this helped...
avatar
Cadence
New Member

Posts : 9
Reputation : 1
Language : English

http://multipenguin.free-message-board.com/

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by !_NICK_! on Mon Sep 12, 2011 8:02 pm

You could edit a template and make it look like that's how many members are on. I don it once, but did not keep it. I just previewed it though, and didn't publish it.
avatar
!_NICK_!
Active Poster

Male Posts : 1505
Reputation : 69
Language : English, HTML, and CSS
Location : In the middle of no return.

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by brandfiesta on Sun Sep 25, 2011 1:33 am

I totally agree, I understand the issue about overload but come on. Im not sure if it is the culprit but this summer I was placing high in search "paid to post forum" now I cannot even find myself on the first page anymore.

Please do not put this in robots.txt

Disallow: /

High Appreciated,
Team BrandFiesta
Voted #1 PTP Forum

brandfiesta
New Member

Posts : 8
Reputation : 0
Language : English
Location : New York

http://www.brandfiesta.com

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by hanzboyz on Sun Sep 25, 2011 2:08 am

Until now my forum havent show yahoo! bots. only showing bing & google.. T_T
avatar
hanzboyz
Forumember

Male Posts : 269
Reputation : 11
Language : Indonesia
Location : http://ps-3.co.cc

http://playstation3.all-up.com

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by True Blue on Wed Oct 05, 2011 8:39 am

Hello thealbertross

I'm yes on this one, as you've already stated placing a robot notpad in the root directory would work, my forum already has the "noodp" robots written into it by Forumotion.com so I will receive bots from Yahoo!, and MSN as well as the first Google one I've spotted today.

Sometime problems do arise with different crawlers that why its better to change "noodp" to "noodp,noydir" thumright

Liz
avatar
True Blue
Forumember

Female Posts : 63
Reputation : 4
Language : english
Location : N.Ireland

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by SLGray on Fri Mar 06, 2015 12:12 pm

Bumping to move this suggestion to the first page.


When your topic has been solved, ensure you mark the topic solved.
Never post your email in public.

avatar
SLGray
Administrator
Administrator

Male Posts : 38225
Reputation : 2574
Language : English
Location : United States

http://ztwds.forumotion.com/

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by Michael_vx on Sat Mar 07, 2015 8:00 pm

i did not understand the m***f**** logic
im kidding
i did not understand everything
but i feel this will help forumotion members
you got my s*** vote Razz
i voted yes
sorry im just excited about the new suggest`s in forumotion
avatar
Michael_vx
Forumember

Male Posts : 664
Reputation : 29
Language : Arabic and some English
Location : Egypt

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by SLGray on Fri Dec 25, 2015 2:55 pm

Bumping to move this suggestion to the first page.


When your topic has been solved, ensure you mark the topic solved.
Never post your email in public.

avatar
SLGray
Administrator
Administrator

Male Posts : 38225
Reputation : 2574
Language : English
Location : United States

http://ztwds.forumotion.com/

Back to top Go down

View previous topic View next topic Back to top


 
Permissions in this forum:
You cannot reply to topics in this forum