Frequently Asked Questions (FAQ)

Are sitemap indexes supported?

Yes, the XML parser detects if an XML sitemap or a sitemap index is parsed. Referenced XML sitemaps in sitemap indexes are followed until a potentially configured limit is reached.

I can't see any valuable output during cache warmup. How can I debug the process?

There exist various debugging and logging tools to increase verbosity of the cache warmup process. Take a look at the logFile, logLevel and progress configuration options. You may also increase output verbosity by using the -v command option.

Can I limit the number of concurrently warmed URLs?

When using the default crawlers, you can configure the concurrency value using the concurrency crawler option.

Is it possible to crawl URLs with GET instead of HEAD?

Yes, this can be configured by using the request_method crawler option in combination with one of the default crawlers.

How can I configure basic auth credentials?

This is possible by using the request_options crawler option in combination with one of the default crawlers. This crawler option accepts all configurable Guzzle request options such as auth for basic auth.

Example:

CLI

./cache-warmup.phar --crawler-options '{"request_options": {"auth": ["username", "password"]}}'

JSON

{
    "crawlerOptions": {
        "request_options": {
            "auth": ["username", "password"]
        }
    }
}

PHP

use EliasHaeussler\CacheWarmup;

return static function (CacheWarmup\Config\CacheWarmupConfig $config) {
    $config->setCrawlerOption('request_options', [
        'auth' => ['username', 'password'],
    ]);

    return $config;
};

YAML

crawlerOptions:
  request_options:
    auth: ['username', 'password']

.env

CACHE_WARMUP_CRAWLER_OPTIONS='{"request_options": {"auth": ["username", "password"]}}'

Can I use a custom User-Agent header instead of the default one?

Yes, a custom User-Agent header can be configured by using the request_headers crawler option in combination with one of the default crawlers.

What does "default crawlers" actually mean?

The library ships with two default crawlers. Depending on the provided configuration options, one of the crawlers is used for cache warmup, unless you configure a custom crawler by using the crawler configuration option. Read more at Default crawlers.