API TESTING WITH THE KARATE FRAMEWORK

This article has two goals. The first is to find interesting information about API Testing with Karate. The second is to help you discover how interesting test automation is and dive deeper into the knowledge behind the passionate job of testing.

And why testing and not Quality Assurance? In this topic, I follow Michael Bolton when he says “Testers: Get Out of the Quality Assurance Business” Words matter and this is a matter of expectations management: we cannot promise something we cannot accomplish.

Michael Bolton

The Forgotten Layer of the Test Automation Pyramid

When we think and design a testing strategy we should not forget The Forgotten Layer of the Test Automation Pyramid: The service layer.

Here is where we need to mention the test automation pyramid. This concept is introduced by Mike Cohn in his book Succeeding with Agile. He describes three levels of test automation, their relationship, and their relative importance. The base layer consists of unit tests and the top layer corresponds to user interface-level tests.

Many times when we start testing without a proper strategy we end up doing the opposite: investing most of the time and the effort in UI tests, which are more expensive to develop and maintain than unit tests. Also we can forget to develop tests for the service level, that’s why they call it the forgotten layer.

As Joe Collantonio described some years ago now, API testing is becoming more and more important. So even if we agree or not with the test automation pyramid schema, it reminds us that we have to pay attention to API testing as part of our strategy.

Software Development Life Cycle

We’re into a spatial representation of the job. As we move down in the testing pyramid, the tests are cheaper, easier to develop, and can find issues in earlier stages, which makes them easier and cheaper to fix than in later stages.

This can be seen as moving left in the Software Development Life Cycle (SDLC), so many teams are involved in Shift Left Testing. Is that enough? It’s not, but it’s a really good move. Which is the next step? Continuous testing. Involve testing in every stage of the SDLC, doing it efficiently, doing it repeatedly, as part of the whole process. Here is when selecting the right tools becomes important: a testing tool that can integrate with our CI (continuous integration)/CD (continuous delivery) systems is a great value.

API Testing Resources

As API testing becomes more and more important, we can find a lot of sites to help us learn more about it. I can suggest just a couple of them which I found very useful:

1. Test Automation University
2. Ministry of Testing

Learn more about the Karate framework

Why is Karate so attractive? Because it’s an open-source tool that combines API test-automation, mocks, performance-testing, and even UI automation into a single, unified testing framework.

Let’s take a look at some of the benefits:

• Language neutral
• Suitable for non-programmers
• Elegant DSL syntax
• Built on Cucumber / BDD Syntax
• Integrates into existing CI/CD
• Good documentation
• Good demos

This Hello World example gives an idea of how simple yet powerful it is.

Peter Thomas provides a deeper explanation in his Webinar on Micro-services Testing With Karate. I found really good arguments about how we can leverage the goods of Karate for our own project. Learn more in the following example: Karate, the black belt of HTTP API testing?.

Some points that look interesting are:

• Tests = Documentation: tests are so easy to read that you can document your API with your tests. And as you have to maintain your tests as the API changes, you have your documentation always up to date.
• Simple: Karate takes care of complexity so you don’t have to worry about Json processing as Karate takes care of that.
• Re-use Karate tests in Gatling to do performance testing: having the power of Gatling based on the API tests you already have is a big win.
• Comprehensive and detailed reports: Karate brings powerful and detailed reports that makes reading the status of our tests very easy. It’s also helpful to debug and troubleshoot problems when you get failures. If you add the benefit of Gatling reports it’s like having two superpowers.
• Data Driven Testing: you can run your tests through tables.
• Mocks fake HTTP servlet: you can execute your tests with a mock server that implements your API.

Karate vs Rest-assured

Do you need to decide which tool suits better for your needs?

Refer more about the following comparison here.

Extra sources about Karate API Testing

I found very useful the following example in Test Guild site. You can also check this demo.

Conclusions

I found Karate can be a great option for automating the checks in the forgotten layer, integrating it in your CI/CD schema, also re-using your tests as the documentation of your API and for performance testing. Is the right tool for you? It depends on your particular project, your needs, and your team. If you want to add the API testing to your strategy, you don’t have too much time and you want to make it simple, it seems like a great option.

Learn more from our team here, or check out our testing services.