Vinai Kopp

Magento Expert, Developer & Trainer

  • 12. The API Integration Kata

    July 2, 2016

    Mage2Katas

    Almost every Magento instance uses some Web APIs, which means that we developers have to create some code that sends requests and reads responses to those web APIs. How to test such integrations is a frequently asked question.
    The types of APIs vary wildly, some are old-school RPC style APIs, some use SOAP, more recent ones mostly use REST - from a testing perspective it doesn’t make a huge difference.
    For the kata we will integrate with a very simple example REST API that provides a list of short names for a given first name.

    The scenario for this kata is the following: a customers first name should be replaced with a common short name that is retrieved from the API.

    Getting started

    The Magento core class that is used to build the customer name to be rendered is \Magento\Customer\Helper\View.
    The method getCustomerName() takes a customer data model and then combines the prefix, the first name, the middle name, the last name and the suffix into a string.
    To achieve our goal I’ve prepared an around plugin for the method, which injects the short name into the customer data instance before the full name is built, and reverts it back to the original first name afterwards.

    <?php
     
    namespace Mage2Kata\CustomerShortName\Plugin;
     
    use Mage2Kata\CustomerShortName\Api\ShortenFirstNameInterface;
    use Magento\Customer\Api\Data\CustomerInterface;
    use Magento\Customer\Helper\View as CustomerViewHelper;
     
    class CustomerViewHelperShortNamePlugin
    {
        /**
         * @var ShortenFirstNameInterface
         */
        private $shortenFirstName;
     
        public function __construct(ShortenFirstNameInterface $shortenFirstName)
        {
            $this->shortenFirstName = $shortenFirstName;
        }
     
        public function aroundGetCustomerName(
            CustomerViewHelper $subject,
            callable $proceed,
            CustomerInterface $customerDataModel
        ) {
            $originalFirstname = $customerDataModel->getFirstname();
            $customerDataModel->setFirstname($this->shortenFirstName->shorten($originalFirstname));
            $resultName = $proceed($customerDataModel);
            $customerDataModel->setFirstname($originalFirstname);
            return $resultName;
        }
    }

    Of course there are tests for the plugin configuration and the plugin itself, too (but I don’t want to show them here - it would make this post to long).
    If you are interested in test driving plugins, please check out the plugin config, the around interceptor and the plugin integration test kata.

    Back to our current kata.
    Notice the dependency that gets injected into the class, the ShortenFirstNameInterface instance.

    During this kata we will create an implementation of the interface, that fetches a list of first names from a REST API.

    Should we mock the external API?

    The natural thing to do when starting to test code using external resources is to replace the API endpoints with a test double.

    However, that introduces the problem that our tests will still work, even if the external API changes it’s endpoints or response format.

    So should we test our code that queries an API during the test directly?
    This approach also is problematic. It makes our tests slow. It also means the tests would fail if there is a network outage or the API is undergoing maintenance.

    My suggestion is to separate testing the external API and testing the code that uses it.

    To catch changes in the external API, we will write a test that checks those parts of the API our code relies on. These tests can also serve to document our understanding of the behavior of the external resource.

    That is the first test case.
    The second test case we will write TDDing our interface implementation, which now can rely on those parts of the API we covered with the behavior test earlier.

    Testing the behavior of the API

    <?php
     
    namespace Mage2Kata\CustomerShortName;
     
    /**
     * @group external
     */
    class HypocorismsApiBehaviorTest extends \PHPUnit_Framework_TestCase
    {
        public function testIsThisOn()
        {
            $this->fail('HALT!');
        }
    }

    Note that the test class is marked with the @group external annotation. This allows us to exclude potentially slow tests that rely on external services by adding the command line argument to phpunit --exclude-group=external. This can also be done in the testsuite configuration in the phpunit.xml file.

    I usually put external API tests into either a directory Test/Behavior or into Test/Unit, depending on how many are needed. I don’t put them into Test/Integration because they don’t require the Magento runtime environment.

    Once we confirmed that our tests are wired up completely, lets replace the testIsThisOn() test with a check confirming the API returns a JSON response.

    public function testReturnsJSON()
    {
        json_decode(\file_get_contents('http://hypocorisms.vinaikopp.com/name/'));
    $this->assertSame(\JSON_ERROR_NONE, json_last_error(), 'JSON decode error: ' . json_last_error_msg());
    }

    And indeed, this test passes. We want to see it fail at least once, so just temporarily lets add some characters to the response so decoding it will fail:

    json_decode('foo' . \file_get_contents('http://hypocorisms.vinaikopp.com/name/'));

    Very good: JSON decode error: Syntax error. Lets fix that again and move on.

    Lets have a look at the API response format. I usually do that using curl on the command line or the PHPStorm REST client.

    $ curl -s http://hypocorisms.vinaikopp.com/name/Robert | jq .
    {
      "data": {
        "resource": "Robert",
        "hypocorisms": [
          "Dob",
          "Dobbin",
          "Bob",
          "Bobbie",
          "Rob",
          "Robin",
          "Rupert",
          "Hob",
          "Hobkin",
          "Robbie"
        ],
        "links": [
          {
            "rel": "self",
            "uri": "/name/Robert"
          },
          {
            "rel": "more.info",
            "uri": "http://www.behindthename.com/api/lookup.php?name=robert&key=[key] "
          },
          {
            "rel": "more.info.human",
            "uri": "http://www.behindthename.com/names/search.php?terms=Robert"
          }
        ]
      },
      "metadata": {
        "description": "Example API for the #Mage2Kata episode 12: The API integration kata",
        "api-spec": "/name/:name",
        "version": "1.0.0",
        "author": "Vinai Kopp"
      }
    }

    So there is a lot of crap information in there we are not really interested in.

    We only want our test to break if relevant parts change. Lets just focus on the data/hypocorisms array and ignore the rest.

    First lets extract the API URL into a property so we can reuse it in the new test.

    private $apiUrl = 'http://hypocorisms.vinaikopp.com/name/';

    And here is the second test:

    public function testReturnsHypocorismsAsArray()
    {
        $response = json_decode(\file_get_contents($this->apiUrl . 'Robert'), true);
     
        $this->assertInternalType('array', $response);
        $this->assertArrayHasKey('data', $response);
     
        $this->assertInternalType('array', $response['data']);
        $this->assertArrayHasKey('hypocorisms', $response['data']);
     
        $this->assertInternalType('array', $response['data']['hypocorisms']);
        $this->assertContains('Bob', $response['data']['hypocorisms']);
    }

    One more to be safe. Lets ensure the response when no match is found.

    public function testReturnsEmptyArrayIfNoMatchIsFound()
    {
        $response = json_decode(\file_get_contents($this->apiUrl . 'THIS IS NOT A NAME'), true);
     
        $this->assertInternalType('array', $response);
        $this->assertArrayHasKey('data', $response);
     
        $this->assertInternalType('array', $response['data']);
        $this->assertArrayHasKey('hypocorisms', $response['data']);
     
        $this->assertInternalType('array', $response['data']['hypocorisms']);
        $this->assertEmpty($response['data']['hypocorisms']);
    }

    With this we can be confident the test will inform us of any changes in the API. While implementing the ShortenFirstNameInterface we can rely on this behavior.

    TDDing the API Client

    First we create a new unit test case \Mage2Kata\CustomerShortName\Model\HypocorismsApiShortenFirstNameTest.

    <?php
     
    namespace Mage2Kata\CustomerShortName\Model;
     
    use Mage2Kata\CustomerShortName\Api\ShortenFirstNameInterface;
     
    class HypocorismsApiShortenFirstNameTest extends \PHPUnit_Framework_TestCase
    {
        public function testImplementsShortensFirstNameInterface()
        {
            $this->assertInstanceOf(ShortenFirstNameInterface::class, new HypocorismsApiShortenFirstName());
        }
    }

    This test allows us to create the class, implement the interface and add a stub for the shorten() method.

    We will skip input parameter validation again in this kata (but never in real project work), and instead focus on what should happen if the API returns an invalid response.
    In that case the method should simply return the first name that was passed as an argument.

    public function testReturnsTheGivenNameIfApiResponseIsInvalid()
    {
        $invalidResponse = '';
        $mockHttpClient->method('getBody')->willReturn($invalidResponse);
        /** @var HttpClientFactory|\PHPUnit_Framework_MockObject_MockObject $mockHttpClientFactory */
        $mockHttpClientFactory = $this->getMock(HttpClientFactory::class, [], [], '', false);
        $mockHttpClientFactory->method('create')->willReturn($mockHttpClient);
        $this->assertSame('Foo', (new HypocorismsApiShortenFirstName($mockHttpClientFactory))->shorten('Foo'));
    }

    To make it pass, we simply return the input argument from shorten.

    Protected by these two tests, it’s time to test our code can actually return a short name returned by the API. Lets extract the following method to prepare:

    /**
     * @var HttpClient|\PHPUnit_Framework_MockObject_MockObject
     */
    private $mockHttpClient;
     
    /**
     * @param string $expected
     * @param string $firstname
     */
    private function assertShortName($expected, $firstname)
    {
        /** @var HttpClientFactory|\PHPUnit_Framework_MockObject_MockObject $mockHttpClientFactory */
        $mockHttpClientFactory = $this->getMock(HttpClientFactory::class, [], [], '', false);
        $mockHttpClientFactory->method('create')->willReturn($this->mockHttpClient);
        $this->assertSame($expected, (new HypocorismsApiShortenFirstName($mockHttpClientFactory))->shorten($firstname));
    }
     
    protected function setUp()
    {
        $this->mockHttpClient = $this->getMock(HttpClient::class);
    }

    Which changes the previous test to this:

    public function testReturnsTheGivenNameIfApiResponseIsInvalid()
    {
        $invalidResponse = '';
        $this->mockHttpClient->method('getBody')->willReturn($invalidResponse);
        $this->assertShortName('Foo', 'Foo');
    }

    Now we can write the next test:

    public function testReturnsFirstHypocorismIfPresent()
    {
        $response = json_encode(['data' => ['hypocorisms' => ['Bar', 'Baz']]]);
        $this->mockHttpClient->method('getBody')->willReturn($response);
     
        $this->assertShortName('Bar', 'Foo');
    }

    We pass in the traditional first name Foo, the API returns ['Bar', 'Baz'], so we expect to get the return value to be Bar.

    Here is the implementation that makes it pass:

    public function shorten($firstname)
    {
        $httpClient = $this->httpClientFactory->create();
        $response = json_decode($httpClient->getBody(), true);
        if (is_array($response)) {
            return $response['data']['hypocorisms'][0];
        }
        return $firstname;
    }

    Now, we are still missing the call to get() on the HTTP client.
    I don’t think it really justifies adding another test. Lets add that to testReturnsFirstHypocorismIfPresent.

    $this->mockHttpClient->expects($this->once())->method('get')->with($this->stringEndsWith('/Foo'));

    This forces us to extend the code of the shorten method like so:

    public function shorten($firstname)
    {
        $httpClient = $this->httpClientFactory->create();
        $httpClient->get('http://hypocorisms.vinaikopp.com/name/' . $firstname);
        $response = json_decode($httpClient->getBody(), true);
        ...

    Given the behavior established by the API test earlier test this would be good enough.
    It would feel good to make our code a bit more robust though.

    Lets add a data provider to the testReturnsTheGivenNameIfApiResponseIsInvalid test to supply some values that would cause our current code to break or throw some PHP notices.
    This is what I ended up with:

    /**
     * @param mixed $invalidResponse
     * @dataProvider invalidApiResponseDataProvider
     */
    public function testReturnsTheGivenNameIfApiResponseIsInvalid($invalidResponse)
    {
        $this->mockHttpClient->method('getBody')->willReturn($invalidResponse);
        $this->assertShortName('Foo', 'Foo');
    }
     
    public function invalidApiResponseDataProvider()
    {
        return [
            [''],
            [false],
            [null],
            [json_encode([])],
            [json_encode(['data' => ''])],
            [json_encode(['data' => []])],
            [json_encode(['data' => ['hypocorisms' => '']])],
            [json_encode(['data' => ['hypocorisms' => []]])],
     
        ];
    }

    And here is the final code to make all those cases pass:

    /**
     * @param string $firstname
     * @return string
     */
    public function shorten($firstname)
    {
        $httpClient = $this->httpClientFactory->create();
        $httpClient->get('http://hypocorisms.vinaikopp.com/name/' . $firstname);
        $response = json_decode($httpClient->getBody(), true);
        return $this->hasHypocorisms($response) ?
            $response['data']['hypocorisms'][0] :
            $firstname;
    }
     
    /**
     * @param mixed $response
     * @return bool
     */
    private function hasHypocorisms($response)
    {
        return is_array($response)
            && isset($response['data'])
            && is_array($response['data'])
            && isset($response['data']['hypocorisms'])
            && is_array($response['data']['hypocorisms'])
            && count($response['data']['hypocorisms']) > 0;
    }

    And that completes the kata.
    This kata is different in that it isn’t mainly about the code and tools, but rater about how to approach testing code that requires an external API.
    I hope this example clears things up, even though it is much simpler then any real world API.
    If you have any questions, please feel free to ask by leaving a comment or by reaching out to me on twitter.

    Hope you enjoyed this episode! Until next time!

    comments powered by Disqus