Recipe Dredger

Most Mealie tooling waits for you to paste a URL. CookDex can go and find recipes for you. In the web UI, Recipe Sources manages the sites it crawls; the crawl itself runs as the recipe-dredger task.

How it finds recipes

Point it at a recipe site and it reads that site’s robots.txt and sitemap to enumerate candidate pages, then checks each one before importing anything. A page has to actually look like a recipe: CookDex looks for schema.org/Recipe JSON-LD with real ingredients and instructions, and cross-checks with recipe-scrapers, so listicles and how-to posts are rejected rather than imported and cleaned up later. Anything that passes is handed to Mealie’s own scraper.

Polite crawling

It is built to be a polite crawler: robots.txt crawl-delay is honored, with a one-second floor that configuration cannot lower. Sitemaps are cached, and URLs already imported or rejected are skipped on later runs.

The Dredger’s settings are in Settings under Dredger:

Setting	Default	What it does
Target Language (`DREDGER_TARGET_LANGUAGE`)	`en`	Recipes in other languages are skipped.
Crawl Delay (`DREDGER_CRAWL_DELAY`)	`2.0` seconds	How long to wait between pages on the same site. A site’s own `robots.txt` wins when it asks for longer.
Sitemap Cache Expiry (`DREDGER_CACHE_EXPIRY_DAYS`)	`7` days	How long a site’s list of recipe pages is reused before it’s read again.

Preview first

Preview mode reports what it would import without writing to Mealie or recording any state. Run a preview, read the log, then run it live.

Task options

Option	Type	Default	Description
`dry_run`	boolean	`true`	Preview imports without writing to Mealie.
`limit`	integer	`50`	Maximum recipes to import per configured site.
`depth`	integer	`1000`	Maximum URLs to scan per site sitemap.
`no_cache`	boolean	`false`	Ignore cached sitemaps and crawl fresh.
`import_workers`	integer	`2`	Concurrent import workers, capped at 4.
`precheck_duplicates`	boolean	`true`	Fetch existing Mealie recipes first and skip duplicates before import.
`language_filter`	boolean	`true`	Skip recipes that do not match `DREDGER_TARGET_LANGUAGE`.
`max_retry_attempts`	integer	`3`	Retry transient failures before marking a URL rejected.