Scrapyd

• Jobs
• Logs
• Documentation

Scrapy projects:

• SwiggyDailyData
• Swiggy_Page_Rating
• Zomato_Page_Rating
• Zomato_Reviews
• bolt
• client_report
• eazydiner
• glovo
• google_reviews
• magicpin
• swiggy
• swiggy_checkin
• swiggy_data
• swiggy_spider
• zomato_check_in
• zomato_comp_item_analysis
• zomato_competiton_analysis
• zomato_dining
• zomato_order_history

This web UI is for monitoring only. To upload projects and schedule crawls, use the API. For example, using curl:

curl http://localhost:6800/schedule.json -d project=default -d spider=somespider

See the Scrapyd documentation for details.