Initial setup

Learn how to get spider templates installed and configured on an existing Scrapy project.

Tip

If you do not have a Scrapy project yet, use zyte-spider-templates-project as a starting template to get started quickly.

Requirements

• Python 3.9+

• Scrapy 2.11+

For Zyte API features, including AI-powered parsing, you need a Zyte API subscription.

Installation

pip install zyte-spider-templates

Configuration

In your Scrapy project settings (usually in settings.py):

1. Configure scrapy-poet.

2. For Zyte API features, including AI-powered parsing, configure scrapy-zyte-api.

3. Configure zyte_common_items.ZyteItemAdapter:

   settings.py

   from itemadapter import ItemAdapter
   from zyte_common_items import ZyteItemAdapter
   
   ItemAdapter.ADAPTER_CLASSES.appendleft(ZyteItemAdapter)
   

4. Add the zyte-spider-templates add-on to your ADDONS setting:

   settings.py

   ADDONS = {
       "zyte_spider_templates.Addon": 1000,
   }
   

For an example of a properly configured settings.py file, see the one in zyte-spider-templates-project.