Fortinet 476 FortiWeb 5.0 Patch 6 Administration Guide
Allow Known
Search Engines
Enable to exempt popular search engines’ spiders from DoS sensors,
brute force login sensors, HTTP protocol constraints, and
combination rate & access control (called “advanced protection” and
“custom policies” in the web UI).
This option improves access for search engines. Rapid access rates,
unusual HTTP usage, and other characteristics that may be
suspicious for web browsers are often normal with search engines. If
you block them, your web sites’ rankings and visibility may be
affected.
By default, this option allows all popular predefined search engines.
Known search engine indexer source IPs are updated via FortiGuard
Security Service. To specify which search engines will be exempt,
click the Details link. A new frame will appear on the right side of the
protection profile. Enable or disable each search engine, then click
Apply. See also “Blacklisting content scrapers, search engines, web
crawlers, & other robots” on page 337.
Note: X-header-derived client source IPs (see “Defining your proxies,
clients, & X-headers” on page 266) do not support this feature in this
release. If FortiWeb is deployed behind a load balancer or other web
proxy that applies source NAT, this feature will not work.
URL Rewriting Select the name of a URL rewriting rule set, if any, that will be applied
to matching requests.
For details, see “Grouping rewriting & redirection rules” on page 385.
HTTP
Authentication
Select the name of an authorization policy, if any, that will be applied
to matching requests. For details, see “Offloading HTTP
authentication & authorization” on page 225.
If the client fails to authenticate, it will receive an HTTP 403 Access
Forbidden error message.
Site Publish Select the name of a site publishing policy, if any, that will be applied
to matching requests. For details, see “Single sign-on (SSO)” on
page 243.
File Compress Select the name of an compression policy, if any, that will be applied
to matching requests. For details, see “Configuring compression
offloading” on page 457.
File Uncompress Select the name of a decompression policy, if any, that will be applied
to matching requests. For details, see “Configuring decompression to
enable scanning & rewriting” on page 460.
Redirect URL Type a URL including the FQDN/IP and path, if any, to which a client
will be redirected if:
its request violates any of the rules in this profile, and
•the Action for the rule is set to Redirect.
For example, you could enter:
www.example.com/products/
If you do not enter a URL, depending on the type of violation and the
configuration, the FortiWeb appliance will log the violation, may
attempt to remove the offending parts, and could either reset the
connection or return an HTTP 403 Access Forbidden or 404
File Not Found error message.
Setting name Description