Lo-Fi Python

Dec 05, 2019

A Collection of Software Testing Opinions for Python and Beyond

I am a beginner to testing my code. I wanted to write about testing to better understand it. While shaping this link fest masquerading as an essay, I collected great ideas from people who are way more experienced than me. You'll find a few of my thoughts, a Pytest example I use to monitor files, ideas for unit testing, property testing, test driven development and many other commonly used software tests.

An Introductory Rant on Testing

Over several years as a programmer, I've slowly grasped the landscape of testing in software development. After moving beyond my first few tutorials and projects, it seemed very noisy to sort out. Examples provided are usually simple assertions that seem tough to relate to a real use case. It might be easy test the wrong things. Plus, some don't do it at all! The quality of the tests is more important than the quantity. But what makes a quality test? Where's the balance between testing every minute detail of a program and not at all?

Yeah. And the worst thing that happens is that you get people that just stop thinking about what they’re doing. “This is the principle, to always write unit tests, so I’m always going to write unit tests,” and then they’re just not thinking about how they’re spending their time, and they wind up wasting a lot of it.

Joel Spolsky, Stack Overflow Podcast #38

Implementing software tests is a best practice for maintaining code, but seems ambiguous to someone who has not tested any code before. I guess the best way is to read open source projects with test suites, but those can be tough to find. How do you know a good test suite when you see it? Maybe the maintainers went rogue and off the deep end with tests. Online, everybody says you should test your code, is the emperor wearing any clothes?

As a beginner stumbling across articles on testing, these questions were tough to answer. As with most things in programming, figuring out the right question to ask is a challenge in itself. Codeacademy and Coursera never mentioned anything about writing tests. On the other hand, Django includes testing in its tutorial and documentation. Also, most languages come with built-in testing tools. Python has the unittest library.

Why test at all? First, some solid benefits of software testing:

  • With tests on your code in place, you can implement changes and have confidence the code still works if the tests pass. This gives developers more confidence to iterate and improve an application.
  • Detect problems faster. Passing tests are a good indicator that your programs are actually doing what they're supposed to do. If they don't pass, you likely found a bug you might have missed otherwise.
  • When you find a bug, you either need to amend your code, or your tests. write a test for that bug and then fix it. Either that, or you need to be amend your tests. You've just improved the quality of your test suite.
  • Automation. If you are writing tests, those tests can be automated. If you are manually checking the results of your program, you're missing a chance to automate those checks away. I haven't applied it yet, but have heard the Tox library may be useful to automate tests related to Python packaging. For more on automating tests, see this PyCon talk, Three Excellent Python Tools to Automate Repetitive Tasks.
  • Test Driven Development can decrease the time spent debugging code. This claim sometimes lacks empirical evidence, supporting evidence tends to be anecdotal.

A good unit test, therefore, is one that helps enforce the contract to which the function is committed.

If a good unit test breaks, the contract is violated and should be either explicitly amended (by changing the documentation and tests), or fixed (by fixing the code and leaving the tests as is).

A good unit test is also strict. It does its best to ensure the output is valid. This helps it catch more bugs.

Pytest and Unit Testing in Python

This is where the Python hits the pavement. Unit tests are generally liked, although some prefer property tests or integration tests because they think the scope of unit tests is too narrow. The unittest library is Python's default testing framework. However nowadays, pytest seems to be the preferred unit testing framework for Python. Hypothesis is another popular framework I've read about.

Pytest Testing

Tests start to lose signal when Mock becomes routine instead of a reluctant workaround. - Brandon Rhodes, When Python Practices Go Wrong

Testing in Python \ General Unit Testing Ideas

I think hypothesis is probably underrated—some libraries are hesitant to incorporate it into their testing frameworks, but I think the property-based testing has real potential to catch scenarios humans would have a hard time anticipating, or at least that would take a long time to properly plan for. I find that hypothesis almost always adds a few useful test cases I hadn’t thought of that will require special error handling, for example.

Tyler Reddy, SciPy Release Manager

Integration \ Property Tests

Traditional, or example-based, testing specifies the behavior of your software by writing examples of it—each test sets up a single concrete scenario and asserts how the software should behave in that scenario. Property-based tests take these concrete scenarios and generalize them by focusing on which features of the scenario are essential and which are allowed to vary. This results in cleaner tests that better specify the software’s behavior—and that better uncover bugs missed by traditional testing.

Assertions

Assertions are generally accepted as welcome additions to your code.

In reality, the safety and restraints that these code carabiners provide actually give you more freedom to take risks in your coding. If you want to try out some risky feature, refactoring, or external library, you know something is wrong as soon as one of your assertions or tests fail and can undo back to an earlier working state.

Phillip J. Guo, Code Carabiners, (Link Broken)

Test Driven Development

Eventually, you'll discover the evangelists preaching Test Driven Development. There are certain discussions which polarize us in the software development world, such as the appropriate scenarios to deploy this system of development.

Opinions vary widely on the merits and appropriate application of TDD. I'm admittedly skeptical but do see the merits of TDD. But which flavor? Where do unit tests and integration tests fit in? How many tests should I write? What exactly should I be testing? This essay claims anyone pair programming software with an expected life of 3 or more years should use Test Driven Development.

"Test Driven Development is a tool for continuously evaluating hypotheses."

General Testing Ideas and Principles

Other common tests types:

Unit test: when it fails, it tells you what piece of your code needs to be fixed.

Integration test: when it fails, it tells you that the pieces of your application are not working together as expected.

Acceptance test: when it fails, it tells you that the application is not doing what the customer expects it to do.

Regression test: when it fails, it tells you that the application no longer behaves the way it used to.

Testing maturity level progression:

  1. No tests
  2. Occasional, slow, unreliable tests
  3. Semi-comprehensive integration tests
  4. Fast, comprehensive unit tests comprise the bulk of testing
    • Dependency injection
    • Composable subsystem design
  5. Real-time test feedback (ideally integrated into the editor)
  6. Tests are extremely reliable or guaranteed reliable by the type system
    • With tooling that tracks the reliability of tests and provides that feedback to authors.
  7. Fuzzing, statistically automated microbenchmarking, rich testing frameworks for every language and every platform, and a company culture of writing the appropriate number of unit tests and high-value integration tests.

I recently wrote my first unit tests with pytest. Below is a script named test_file_date.py. It tests if the day of month of the most recently changed file in a directory matches today's day. To install pytest, enter into command prompt or terminal:

python -m pip install pytest

test_file_date.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
import glob
import os
from datetime import datetime, date

# The dir_query format is for a Windows path with Unix style pattern matching.
def test_csv_date_equals_today():
    dir_query = 'C:\\Users\\your_username\\Desktop\\*.csv' # specify csv extension and folder
    file_path = sorted(glob.iglob(dir_query), key=os.path.getmtime)[-1] # get most recent file
    file_timestamp = os.path.getmtime(file_path)
    file_date = datetime.fromtimestamp(file_timestamp)
    print(file_date.day)
    print(date.today().day)
    assert file_date.day == date.today().day

Run the test with pytest by entering:

pytest test_file_date.py

Conclusion I write programs for personal productivity and to automate processes. The scope of problems my code solves has grown with my programming ability. I'm now reaching the point where I can apply tests to my advantage. However, sometimes I'll write a quick-hitter script for which I can't justify writing tests. Beyond those cases, testing can help if you pick the right style for your project. More so for recurring, automated processes.

It feels pretty cool when your tests run and you know with more certainty whether a part of your program is getting the job done or not. After setting up my first test with pytest, I have leveled up to novice tester, instead of blissfully not knowing what I don't know about testing. That's a step in the right direction.

posted at 03:06  ·   ·  pytest  software development  test driven development