TLDR
Enter a command. For example: ned, redis server, hg commit.

scrapy

 
Web-crawling framework. More information: https://scrapy.org.
 
  • Create a project:
     
    scrapy startproject {{project_name}}
     
  • Create a spider (in project directory):
     
    scrapy genspider {{spider_name}} {{website_domain}}
     
  • Edit spider (in project directory):
     
    scrapy edit {{spider_name}}
     
  • Run spider (in project directory):
     
    scrapy crawl {{spider_name}}
     
  • Fetch a webpage as Scrapy sees it and print the source to stdout:
     
    scrapy fetch {{url}}
     
  • Open a webpage in the default browser as Scrapy sees it (disable JavaScript for extra fidelity):
     
    scrapy view {{url}}
     
  • Open Scrapy shell for URL, which allows interaction with the page source in a Python shell (or IPython if available):
     
    scrapy shell {{url}}

This is a tldr pages (source, CC BY 4.0) web wrapper for cheat-sheets.org. All commands, popular commands, most used linux commands. Referrals. Progressive Web Application (PWA) version to install on your device.