The forum of the forums
Welcome to the Official Support Forum of Forumotion!

To take full advantage of everything offered by our forum, please log in if you are already a member, or join our community if you've not yet.



Create a free forum like this one.

Allow more bots on robots.txt(alexa and ask)

View previous topic View next topic Go down

Allow more search engine bots to crawl out forums

95% 95% 
[ 35 ]
5% 5% 
[ 2 ]
 
Total Votes : 37

Allow more bots on robots.txt(alexa and ask)

Post by thealbertross on February 6th 2009, 11:23 pm

Id very much like the robots.txt page on all of our forums to be altered. as we dont have ftp access, perhaps admin of forumotion could change the text,

currently, we are only allowing google, msn and yahoo to crawl and index our forums, and blocking dozens of other bots including alexa and ask.

would you please agree to this suggestion so we can perhaps get crawled by more search engines?

Thanks

thealbertross
Forumember

Posts : 95
Reputation : 0
Language : english

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by oleander on April 3rd 2009, 4:59 pm

has this been done yet?

oleander
Forumember

Posts : 48
Reputation : 0
Language : english

http://advertiseforfree.forumotion.com/forum.htm

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by Jaffa909 on August 28th 2010, 4:25 pm

I votes yes becuase our forums get more visits and more people that join our forums they might join forumotion! But if they allowed all search engine bots and you submitted to like one of those websites where they send like 500 search bots your forum might not be able to handle 500+ bots indexing andn scanning all the posts at one might overload it! but very nice idea

Jaffa909
Forumember

Posts : 57
Reputation : 0
Language : English
Location : Untied kindom

http://www.clubpenguingang.net

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by StoneRaven on August 28th 2010, 4:37 pm

I vote yes. The more we can get crawled the more we show up on more search engines which ='s more real visitors. I don't see 500+ bots at one time though maybe 3 or 4 at a time.

StoneRaven
Forumember

Male Posts : 142
Reputation : 0
Language : English

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by Walshy95 on August 28th 2010, 8:58 pm

Voted yes. Great idea. Would be a great way of publicity.

Walshy95
Active Poster

Male Posts : 1308
Reputation : 6
Language : English
Location : UK, Liverpool

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by Cadence on September 11th 2011, 2:55 pm

good!! I have bots on all the time (Yahoo) and i wanna make my user a bot so when i log in chatbox it will chat to anyone there... how do i add a bot user to chat box?????

Cadence
New Member

Posts : 9
Reputation : 1
Language : English

http://multipenguin.free-message-board.com/

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by LoveStudio on September 11th 2011, 3:33 pm

I like this héhé

LoveStudio
New Member

Male Posts : 23
Reputation : 2
Language : Vietnamese
Location : >"<

http://www.9xstudio.biz

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by Cadence on September 13th 2011, 2:44 am

Well, bots are from "official" websites. Usually they make bots so they're website may come back with Info from your forum, brought to theres so they may add to it...You can dis bots, most likely, I wouldn't. More searches CLOSE or FAR OFF searches feature your weblink, so its a far trade Wink bots are useful, but if you seem to not like them I guess not...Oh and the reason you cannot edit that is because, the CSS is the theme, look, style and sorts of functions, it does not add bots. If it were a bot to be added, most likely to be added in the GUEST section and not do any data modifiers left behind, so I recommend not to make/add a bot, you must know your FULL WAY throw coding, plus, it wouldn't be are your website!! 0.0 it go through others, and you cannot see the data...I hope this helped...

Cadence
New Member

Posts : 9
Reputation : 1
Language : English

http://multipenguin.free-message-board.com/

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by !_NICK_! on September 13th 2011, 4:02 am

You could edit a template and make it look like that's how many members are on. I don it once, but did not keep it. I just previewed it though, and didn't publish it.

!_NICK_!
Active Poster

Male Posts : 1505
Reputation : 69
Language : English, HTML, and CSS
Location : In the middle of no return.

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by brandfiesta on September 25th 2011, 9:33 am

I totally agree, I understand the issue about overload but come on. Im not sure if it is the culprit but this summer I was placing high in search "paid to post forum" now I cannot even find myself on the first page anymore.

Please do not put this in robots.txt

Disallow: /

High Appreciated,
Team BrandFiesta
Voted #1 PTP Forum

brandfiesta
New Member

Posts : 8
Reputation : 0
Language : English
Location : New York

http://www.brandfiesta.com

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by hanzboyz on September 25th 2011, 10:08 am

Until now my forum havent show yahoo! bots. only showing bing & google.. T_T

hanzboyz
Forumember

Male Posts : 269
Reputation : 11
Language : Indonesia
Location : http://ps-3.co.cc

http://playstation3.all-up.com

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by True Blue on October 5th 2011, 4:39 pm

Hello thealbertross

I'm yes on this one, as you've already stated placing a robot notpad in the root directory would work, my forum already has the "noodp" robots written into it by Forumotion.com so I will receive bots from Yahoo!, and MSN as well as the first Google one I've spotted today.

Sometime problems do arise with different crawlers that why its better to change "noodp" to "noodp,noydir" thumright

Liz

True Blue
Forumember

Female Posts : 63
Reputation : 4
Language : english
Location : N.Ireland

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by SLGray on March 6th 2015, 8:12 pm

Bumping to move this suggestion to the first page.


When your topic has been solved, ensure you mark the topic solved.
Never post your email in public.


SLGray
Administrator
Administrator

Male Posts : 35674
Reputation : 2375
Language : English
Location : United States

http://fmthemes.forumotion.com/

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by Michael_vx on March 8th 2015, 4:00 am

i did not understand the m***f**** logic
im kidding
i did not understand everything
but i feel this will help forumotion members
you got my s*** vote Razz
i voted yes
sorry im just excited about the new suggest`s in forumotion

Michael_vx
Forumember

Male Posts : 656
Reputation : 29
Language : Arabic and some English
Location : Egypt

Back to top Go down

Re: Allow more bots on robots.txt(alexa and ask)

Post by SLGray on December 25th 2015, 10:55 pm

Bumping to move this suggestion to the first page.


When your topic has been solved, ensure you mark the topic solved.
Never post your email in public.


SLGray
Administrator
Administrator

Male Posts : 35674
Reputation : 2375
Language : English
Location : United States

http://fmthemes.forumotion.com/

Back to top Go down

View previous topic View next topic Back to top


 
Permissions in this forum:
You cannot reply to topics in this forum