Behavioral Testing in Python

It’s no secret that the software industry has a lot of opinionated people, considering all of the different technologies, languages, frameworks, methodologies… the list goes on and on. Every team that I’ve personally worked with has had different approaches to development and different viewpoints on their software development practices. Some believed in having a unit test for each piece of code written, some preferred behavioral tests, and others… well, didn’t test anything (I don’t recommend that approach!).

In this post, I wanted to look at behavioral based testing in which we test an API from the outside, to validate its expected behavior. I wanted to cover this because, in my experience, I’ve gotten a lot of mileage with behavioral based testing, rather than more granular unit based testing. Let me give you an example.

Migrating A Service

Let’s say you’re tasked with migrating a service that has limited testing and limited internal support. How would you approach your migration to ensure that you’re not going to break or alter the API that your customers are relying on?

In this scenario, I’d argue that building a set of behavioral based tests arond the api could provide a great deal of value, especially in gaining confidence that the migrated service does infact work. This would allow the developer to model how customers are currently using the API, validate it against the documenation, and also, possibly learn the API at a deeper level.

The API

Let’s consider a hypothetical weather API. In our example, the data will be generated and fake, in which we will populate a small database. It will be an API that exposes several weather resources in which clients applications may consume.

It has the following documentation:

Authentication

  • API Key: To access the API, an API key is required.

Base URL

http://localhost:5000/api

Endpoints

Current Weather

  • Endpoint: /current/{location}
  • Method: GET
  • Description: Returns current weather conditions for a specified location.
  • Sample Request: 
    curl http://localhost:5000/api/current/boston?key=YOUR_API_KEY

Weather Forecast

  • Endpoint: /forecast/{location}
  • Method: GET
  • Description: Returns weather forecast for the specified location.
  • Sample Request: 
    curl http://localhost:5000/api/forecast/boston?key=YOUR_API_KEY

Historical Weather Data

  • Endpoint: /historical/{location}/{date}
  • Method: GET
  • Description: Returns historical weather data for a specified date and location.
  • Sample Request: 
    curl http://localhost:5000/api/historical/boston/2023-06-01?key=YOUR_API_KEY

Parameters

  • location: Name of the location (string)
  • date: Date in YYYY-MM-DD format (for historical data only)

Data

  • All data is returned in JSON format.

Responses

  • Success: Returns weather data (JSON)
  • Error 404: Location or data not found
  • Error 401: Missing API key
  • Error 405: Method not allowed

Setup

I’ve provided the code for this demo in our repo:

If you want to play with this example yourself, then you can follow the setup instructions in the README.

At this point, what we’ve done is:

  • Setup a light flask api service.
  • Populated a fake database with three resources representing: current, historical, and forecast weather

Behavioral Testing

So, what should we test here?

Well, given that we have some external facing documentation, perhaps we can write tests against the documentation. This will allow us to ensure that our documentation aligns with what we’re advertising. As a side note, this process can also help you learn an api, in particular if you’re thrown into the deep end without support or a codebase lacking tests.

Reponse Codes

You can see that we may write tests against the API and validate that the returned response codes match our case. For example, what happens when we make a request, but don’t include an API key? Obviously we don’t want to release data to unauthorized users, so we might validate that the code is a 401.

Alt text for the image Alt text for the image

We can continue working through the API with tests cases to cover our docs. Eventually, we’ll build up a suite, testing all cases. Then, each time we make changes to the system, we can automatically run the suite to verify regressions haven’t been made.

Alt text for the image

Conclusion

Software testing allows developers to validate that their product conforms to their documentation. In the event that a system is missing validation or testing, then behavioral based testing might help developers to learn the system. In some instances, testing may also be used to help validate service migrations, or help guide a migration to reduce risk in the event testing is absent.

Please checkout the code and play with the test suite. Happy coding!