NOTE: To limit crawl depth, set the 'Depth limit' option in the Follow Links Tab of an individual target's properties.

Limit Options

Default target extensions to follow - This is a comma separated list of extensions that is used as the default extensions for links to crawl/follow when following the links of a target. Once a target is added and follow links is enabled for that target, you can change the extensions to follow for that specific target. The default extensions to follow for a new target will be the extensions that are in this edit box. The default is "htm,html,shtm,shtml,asa,asp,aspx,cfm,cfml,css,js,jsp,php,php3,php4,wml".

Note that this option is for crawling/following only and does not directly affect the link checker. You do not want to crawl/follow binary files like images so do not specify image extensions like jpg and png here.

Max targets - Helps to control processing in some situations by limiting the processing to a maximum number of targets (processing will attempt to stop around this number of targets, but it may not be exactly the number set here). For instance, when following links, you can set this to help keep the wizard from "going out of control" by validating too many documents. If your computer has trouble with large jobs, consider dividing large jobs into multiple smaller jobs with fewer targets. Note that the more memory and resources a computer has, the better large jobs will be handled. The default value is 2000. Possible values are 1 to 30000.

CSS HTML Validator Enterprise has additional features to increase limits even further for large sites.

The default value was increased from 1000 to 2000 in v20.02.

The upper limit was increased from 9999 to 30000 for 64-bit builds (64-bit is the standard build) in v20.02. The upper limit may be increased.

Max concurrent HTTP requests - Use this option to set the maximum number of concurrent HTTP requests made when the Batch Wizard has to download multiple URLs. Increasing this may speed up the downloading of URLs but may cause other problems, such as more stress on your computer and/or too many concurrent requests to one server. Decreasing this may slow down the downloading of URLs and may put less stress on your computer and the server. We recommended that you use the default value unless there is a good reason to change it. The default is 10. Possible values are 1 to 100.

URL timeout in seconds for fetching targets - Use this option to set the maximum amount of time (in whole seconds--no decimal points) before a timeout occurs when the Batch Wizard is downloading a URL. The default value is recommended unless there is a good reason to change it. Note that this option does not affect the timeout of the link checker (that timeout option is in the 'Link Checking' page of the Validator Engine Options). The default is 60. Possible values are 1 to 1000.

Wait time before each URL request (milliseconds) - Use this option to slow down requests made by the Batch Wizard. Before every URL request, the Batch Wizard will wait the amount of milliseconds specified here (1000 milliseconds is equal to 1 second). This option is useful to reduce the load on the server that the requests are being made on. It can also be used to increase the amount of time that the server web logs will show activity from the Batch Wizard. The default is 0. Possible values are 0 to 1000000. New in v10.9910, this has changed from seconds to milliseconds to provide finer control.