pytest plugin to make testing complicated long string output easy to write and easy to debug
Project description
A (yet unnamed) experiment for a pytest plugin to improve pattern matching for complex/long outputs from CLIs, HTML and maybe textual documents like JSON, YAML, etc.
Long outputs means strings with 10-1000 lines.
When writing tests, this should improve:
- tolerance for variability (
...
is used as a pattern) - tolerance for optional output
- tolerance for output that may appear out of order
- mixing/matching of multiple rules
- writing assumptions in a readable/maintainable way
And most importantly: provide failure reports that are very easy to read and allow pinpointing the cause of a failure quickly.
Playground
I'm using a nix flake to toy around with this:
nix develop
$ pytest -vv test_audit.py
TODO
-
normalization feature
-> json (+whitespace) -> python object causes serialization -> json object causes deserialization + reserialization (mit readable oder so) -> whitespace normalization
-> html (+whitespace) -> parse html, then serialize in a normalized way -> whitespace normalization for both pattern and tested content
-> whitespace (pattern and tested content) -> strip whitespace at beginning and end -> replace tabs with spaces -> fold multiple spaces into single spaces
DONE
-
differentiate between tolerated and expected in status reporting
-
add avoided lines that must not appear
-
allow patterns expectations/tolerations/... to have names and use those to mark up the report why things matched?
- (T) DEBUG | ....
- (X) MIGRATION |
-
how to deal with HTML boilerplate -> use
optional("...")
-
add lines that must appear in order without being interrupted
Later
-
API design for merge alternatives?
-
structlog integration? (via normalization?)
-
add line numbers
-
report line numbers on matched avoidances
Wording
Matches on patterns are adjectives:
- These lines must appear and they must be continuous.
- These lines must appear and they must be in order.
- These lines are optional.
- If those lines appear they are refused.
Modifiers to the pattern itself are verbs:
- Merge the rules from those patterns into this one.
- Normalize the input (and the rules) in this way.s
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pytest_patterns-0.1.0.dev0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 24df49802a621b7bf83a78d322adff01bf3cc862e3dfe9ee0aa061fe60dd4aee |
|
MD5 | 70d7b9102a6e32179dd5fad02c1173ff |
|
BLAKE2b-256 | d33ced098e24e9503186d763d85ad711432a6acb737c0fd3f0e51ac137551d92 |
Hashes for pytest_patterns-0.1.0.dev0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c8b737651b8927bb63010917fbd139e9dc0d1e8be17de345de1e087296be417b |
|
MD5 | fefd3de640c28822f20938da0bdcc6e6 |
|
BLAKE2b-256 | b4ac7f2b0ab92f292cb3277e2bcf678363eb41b6b93646655ebb2f8ca348f76b |