Noydir may not be incorrect

[Webnauts]

Bug in the results:

"The content for the robots meta tag may be incorrect. Details: "noydir" is not a recognized instruction.... "

"noydir" may not be incorrect. Read my article about this:
Using NOODP and NOYDIR

[Albert Wiersch]

Great, thanks. I will investigate this for a future update.

[Webnauts]

Great, thanks. I will investigate this for a future update.

Albert in my article there is a link to the blog of Yahoo. So the info is 100% accurate.

[Albert Wiersch]

Thanks. I will soon begin working on the next major version of CSE HTML Validator and will address it at that time.

I may also post an updated config file for v8.0 users if this issue can be corrected with an updated config file.

[Webnauts]

Great Albert. Thanks.

[Albert Wiersch]

I'm currently working on this and future versions of CSE HTML Validator will now recognizes "NOYDIR".

I noticed that CSE HTML Validator is currently only checking the "robots" and "googlebot" meta tags for these search engine related directives. What are other meta tags that it should check in the same way? Should it also be checking "slurp" and "msnbot" meta tags in the same way? Your help and guidance is appreciated!

[Webnauts]

You may have for example the following options when you are exclusively about the ODP and/or Yahoo directories:

<meta name="robots" content="noodp">
<meta name="robots" content="noodp,noydir">

<meta name="googlebot" content="noodp">
<meta name="slurp" content="noodp">
<meta name="slurp" content="noydir">
<meta name="slurp" content="noodp,noydir">
<meta name="msnbot" content="noodp">

[Albert Wiersch]

Thanks but I may not have been clear.

Should the content of "msnbot" and "slurp" meta tags be checked the same way as a "robots" meta tag? I assume "msnbot", "slurp", and "googlebot" meta tags do the same as a "robots" meta tag but they are targeted toward a specific crawler. Therefore, can the content of all these meta tags be checked for the same directives in the same way?

[Webnauts]

Thanks but I may not have been clear.

Should the content of "msnbot" and "slurp" meta tags be checked the same way as a "robots" meta tag? I assume "msnbot", "slurp", and "googlebot" meta tags do the same as a "robots" meta tag but they are targeted toward a specific crawler. Therefore, can the content of all these meta tags be checked for the same directives in the same way?

If I understand your question right, using this <meta name="robots" content="noodp,noydir"> is valid for all 3 bots in advance.

For example if you want to exclude only the Yahoo directory, you can also use <meta name="robots" content="noydir">.

So if I understood you, yes, you are right.

Here it is perfectly described: http://www.free-seo-news.com/newsletter224.htm

[Albert Wiersch]

If I understand your question right, using this <meta name="robots" content="noodp,noydir"> is valid for all 3 bots in advance.

For example if you want to exclude only the Yahoo directory, you can also use <meta name="robots" content="noydir">.

So if I understood you, yes, you are right.

Here it is perfectly described: http://www.free-seo-news.com/newsletter224.htm

I understand what you're saying but I'm still not sure we're on the same page.

CSE HTML Validator checks the syntax of the content of the "robots" tag. I just wanted to confirm that the syntax of the content of the "robots" tag is the same as that for the content of the "googlebot", "slurp", and "msnbot" meta tags as well. Currently, if CSE HTML Validator sees ANY of these meta tags, then it will check the content syntax in the same way.

[Webnauts]

CSE HTML Validator checks the syntax of the content of the "robots" tag. I just wanted to confirm that the syntax of the content of the "robots" tag is the same as that for the content of the "googlebot", "slurp", and "msnbot" meta tags as well.

Exactly. It is.

For example all the meta tags below can be added in your documents head tag:

<meta name="robots" content="noodp" />
<meta name="googlebot" content="index,nofollow" />
<meta name="slurp" content="noydir,noarchive, follow" />
<meta name="msb" content="index,nofollow,noarchive" />

[Albert Wiersch]

<meta name="msb" content="index,nofollow,noarchive" />

Thanks... is that suppose to be msnbot instead of msb? I don't want to miss any! Are there any others besides "robots", "googlebot", "slurp", and "msnbot"?

[Webnauts]

<meta name="msb" content="index,nofollow,noarchive" />

Thanks... is that suppose to be msnbot instead of msb? I don't want to miss any! Are there any others besides "robots", "googlebot", "slurp", and "msnbot"?

Ups? I misspelled msn above. Sorry.

You need a list of bots? http://www.hashemian.com/tools/browser-user-agents.htm

[Albert Wiersch]

You need a list of bots? http://www.hashemian.com/tools/browser-user-agents.htm

Thanks, but those are mostly user agents, not bots. I'm looking for others bots (for meta tags) like "robots", "googlebot", "slurp", and "msnbot".

[Webnauts]

You need a list of bots? http://www.hashemian.com/tools/browser-user-agents.htm

Thanks, but those are mostly user agents, not bots. I'm looking for others bots (for meta tags) like "robots", "googlebot", "slurp", and "msnbot".

Well these are the bots that officially agreed to use the "noodp" attributes.

I will have another look and see what I will come up with.