404 Permission denied (Error #10013) when using batch wizard

[rag_gupta]

Today when I tried to run on my website(on localhost), for initial 1000 or so pages it worked, but many batch of pages getting the following error but finally for the last 18000 pages it printed error for all the urls:

An error occurred for the request "http://localhost/downloadformsindia/index.php?option=com_content&task=view&view=article&id=26948&Itemid=5". Error: 10013, Status: 404, Reason: Permission denied (Error #10013). This target will not be processed.

When I pasted those urls in my browser, it worked.
Can somebody please help?


Secondly the search on your website is not useful, can you not please use "Google custom search" instead? It costs $100 per year.

I wish you incorporated regular expression match for "Don't process these targets"

I'm using the latest version (v10).

[Albert Wiersch]

An interesting tidbit: I had typed a reply using IE, but navigated away to a page by accident and when I came back, my response was gone! Now I will use Firefox instead, even though it is a little slower. I used IE because the pages popped up faster but that was just too annoying! Firefox doesn't lose the form data when you navigate away and come back.

Anyway, I was saying that I do not know what is causing those errors, but what I would first try is slowing down the requests. Please go to the "Limits" tab in the Batch Wizard Options page and try lowering the "max concurrent HTTP requests" value and increasing the "wait time before each URL request" and see if that helps (let me know).

As for regular expression matching, I have considered that, but have had virtually zero requests for it (at least that I can remember) so I haven't really looked into implementing it yet. I'm curious though, what are you wanting to match? What regular expression would you like to use?

I'm sorry you did not find the search useful. I assume you are talking about the search at http://www.htmlvalidator.com/search.html. I have not looked into Google's custom search service but perhaps I should. Do you mind if I ask what you searched for that wasn't helpful?

[rag_gupta]

Thanks for your reply and valuable insight about Firefox.
I'd try to set the time out limit. This may be due to resetting of my external hard-drive on which I've set the Windows temp and tmp environment values but I'm not sure.
About the regular expression matching, my website is having 18000 or so pages it's a joomla CMS based so the pages are identified using id:

http://www.*....&id=40000&...

Here 40000 is unique value identifying unique page.

What I want is to check a range of pages say where id>=30000 and id<50000 in one validator "batch".
Next batch I'd like to run for id between 50000 and 70000.

One reason is due to the memory intake of CSE, secondly wait time for it to finish for full site will be longer


Even in regex it won't be possible to specify range say between 30000 - 50000.

I'd be happy if there were (also) an option for specifying an external perl, php or any script/exe which when passed an url as first parameter would return 0 or 1 indicating whether target should be processed or not. There is good chance I'd be taking up a few other website projects like this so I'd be needing them in future. More so because all CMS based websites would benefit with this type of support, because the there are no keywords in the CMS based urls only numeric codes.

For the search on your website, I've searched about the issues I raised in two posts as well as a few others. I got nothing. It's really not useful at all.

I love this CSE and your website because you've filled them with lot of tips, tricks and insight.

I also want to thank you for suggesting Direct Access utility which is really helpful. Let me restrict off-thread talk here.
Anurag

[Albert Wiersch]

http://www.*....&id=40000&...

Here 40000 is unique value identifying unique page.

What I want is to check a range of pages say where id>=30000 and id<50000 in one validator "batch".
Next batch I'd like to run for id between 50000 and 70000.

One reason is due to the memory intake of CSE, secondly wait time for it to finish for full site will be longer


Even in regex it won't be possible to specify range say between 30000 - 50000.

Thanks. I can see how it would be very useful to do that. While I'm no "regex" expert, it sounds to me like that should be possible to specify - "id=" followed by a '3' or a '4', another 4 digits, and finally a non-digit.

I'll make a note too look into regular expression support for a future version.

I'd be happy if there were (also) an option for specifying an external perl, php or any script/exe which when passed an url as first parameter would return 0 or 1 indicating whether target should be processed or not. There is good chance I'd be taking up a few other website projects like this so I'd be needing them in future. More so because all CMS based websites would benefit with this type of support, because the there are no keywords in the CMS based urls only numeric codes.

I had thought about something like this before, using CSE HTML Validator's own "tag name programming" language. I'll also make a note of this. It sounds like something useful for a future major upgrade.

For the search on your website, I've searched about the issues I raised in two posts as well as a few others. I got nothing. It's really not useful at all.

I understand, but that was probably because there wasn't much discussion about these issues before (at least that I'm aware of).

I love this CSE and your website because you've filled them with lot of tips, tricks and insight.

I also want to thank you for suggesting Direct Access utility which is really helpful. Let me restrict off-thread talk here.
Anurag

Great! I'm glad you like the website and content. Direct Access is a real time saver for me. I think it's great for posting, email, and programming, among other things.

[rag_gupta]

Even inconsistent memory usage by CSE is also problemetic.

I've 4 Gb of RAM, on Windows 7, since it's 32 bit it's further reduced to only 3 GB.
I'll have to move to 64 bit just because of CSE.

When I run CSE batch it may take an hour or so, and my system slows downs because of apache, mysql and cse usage.
Finally I see error message by CSE of no memory and all the validations done till now are gone.

When there is no memory or minimum memory CSE should stop gracefully and come out with the report of validations already done.

Thanks
Anurag

[Albert Wiersch]

Even inconsistent memory usage by CSE is also problemetic.

I've 4 Gb of RAM, on Windows 7, since it's 32 bit it's further reduced to only 3 GB.
I'll have to move to 64 bit just because of CSE.

When I run CSE batch it may take an hour or so, and my system slows downs because of apache, mysql and cse usage.
Finally I see error message by CSE of no memory and all the validations done till now are gone.

When there is no memory or minimum memory CSE should stop gracefully and come out with the report of validations already done.

I'm sorry for the trouble.

I'm curious, how many targets are you trying to validate at once?

Also, what version are you using? I can send you a link to the latest build, which should be more memory efficient.

You're right, CSE HTML Validator should stop if memory gets critically low. I will have to research how to determine that, then I can make the Batch Wizard issue a warning message and stop checking more targets.

UPDATE: The upcoming update, v10.01, now attempts to automatically stop processing when it detects that remaining memory is getting too low.

[Albert Wiersch]

As an update to this old issue, I just encountered this myself during some testing, and it was due to a new anti-malware program I installed. So for anyone getting error 10013 for link checking (Permission denied (Error #10013)), it could be due to some type of anti-malware blocker that is blocking requests to certain IPs (like the software I installed was doing).