Jump to content

what is a spider?


Bufoman

Recommended Posts

Thanks for the info. I think their really should be a sticky some wheres about Bots Spiders, and any other odd user id tags (or whatever these are called) we have around here.

Clan Moderator from December 15th 2006- August 20th 2007

Founder of: Terran Gamers, formerly known as Militos Deci

Link to comment
Share on other sites

Thanks for the info. I think their really should be a sticky some wheres about Bots Spiders, and any other odd user id tags (or whatever these are called) we have around here.

 

Here, copypasta from one of my phpBB installations:

 

[hide=bots]AdsBot [Google]

 

Alexa [bot]

 

Alta Vista [bot]

 

Ask Jeeves [bot]

 

Baidu [spider]

 

Exabot [bot]

 

FAST Enterprise [Crawler]

 

FAST WebCrawler [Crawler]

 

Francis [bot]

 

Gigabot [bot]

 

Google Adsense [bot]

 

Google Desktop

 

Google Feedfetcher

 

Google [bot]

 

Heise IT-Markt [Crawler]

 

Heritrix [Crawler]

 

IBM Research [bot]

 

ICCrawler - ICjobs

 

ichiro [Crawler]

 

Majestic-12 [bot]

 

Metager [bot]

 

MSN NewsBlogs

 

MSN [bot]

 

MSNbot Media

 

NG-Search [bot]

 

Nutch [bot]

 

Nutch/CVS [bot]

 

OmniExplorer [bot]

 

Online link [Validator]

 

psbot [Picsearch]

 

Seekport [bot]

 

Sensis [Crawler]

 

SEO Crawler

 

Seoma [Crawler]

 

SEOSearch [Crawler]

 

Snappy [bot]

 

Steeler [Crawler]

 

Synoo [bot]

 

Telekom [bot]

 

TurnitinBot [bot]

 

Voyager [bot]

 

W3 [sitesearch]

 

W3C [Linkcheck]

 

W3C [Validator]

 

WiseNut [bot]

 

YaCy [bot]

 

Yahoo MMCrawler [bot]

 

Yahoo Slurp [bot]

 

Yahoo [bot]

 

YahooSeeker [bot][/hide]

C2b6gs7.png

Link to comment
Share on other sites

Well maybe not a sticky then, but a mention of them some wheres- like in an official topic about the forum.

 

 

 

And what are crawlers, validaters, and the others?

Clan Moderator from December 15th 2006- August 20th 2007

Founder of: Terran Gamers, formerly known as Militos Deci

Link to comment
Share on other sites

They could do a search on the forum, and this topic would pop up. Problem solved. :)

You're accusing me of bigotry, how ironic. It's a nice attempt at argument, but your responses are facile and asinine, if not diatribe. Who's arrogant now?

Link to comment
Share on other sites

And what are crawlers, validaters, and the others?

 

If you're developing a website and want it to be valid XHTML, you go to the WC3 site and give it the address. It sends a bot (Labeled on phpBB as a validater) to the website to get the XHTML, so that you can validate it. That's the idea with a validator.

C2b6gs7.png

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.