Nasy Crawler Framework -- Never had such a pure crawler.
Project description
Table of Contents
Prologue
Never had such a pure crawler like this nacf
.
Although I often write crawlers, I don’t like to use huge frameworks, such as scrapy, but prefer
simple requests+bs4
or more general requests_html
. However, these two are inconvenient for a
crawler. E.g. Places, such as error retrying or parallel crawling, need to be handwritten by
myself. It is not very difficult to write it while writing too much can be tedious. Hence I
started writing this nacf (Nasy Crawler Framework), hoping to simplify some error retrying or
parallel writing of crawlers.
Packages
Package | Version | Description |
---|---|---|
requests-html | 0.9.0 | HTML Parsing for Humans. |
Development Process
TODO Http Functions
DONE Get
NEXT Post
TODO Bugs
DONE Fix an error from inspect.Parameter which caused the function parallel down. :err:1:
Epoligue
History
Version 0.1.2
- Data:
- Fixed:
inspect.Parameter
error in last version.
Version 0.1.1
- Data:
- Ignored: An error caused by
inspect.Parameter
- Help Wanted: Can someone help me about the Parameter?
Version 0.1.0
- Date:
- Commemorate Version: First Version
- Basic Functions.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
nacf-0.1.2.tar.gz
(13.5 kB
view hashes)
Built Distribution
nacf-0.1.2-py3-none-any.whl
(36.0 kB
view hashes)