Vinai Kopp

Magento Expert, Developer & Trainer

  • Magento 2 Repositories, Interfaces and the Web API

    February 18, 2017

    Magento 2 Repositories

    Magento 2 introduced repositories for most of the core entities like products, orders, customers and so on.
    Often I hear the question “Why?”. That is a very good question. What do repositories offer that Magento 1 style collections can’t?

    This blog post would like to explain the reasons why or why not you would want to create repositories for your custom entities, and hopefully show how create them if you come to the conclusion that they make sense.

    Disclaimer:
    This post only describes a pragmatic approach for creating repositories for custom entities for third party modules.
    The Magento core teams have their own standards with which the description below does not comply.

    In general, the purpose of a repository is to hide the storage related logic.
    A client of a repository should not care whether the returned entity is held in memory in an array, is retrieved from a MySQL database, fetched from a remote API or from a file.
    I assume the Magento core team did this so they are able to change or replace the ORM in future. In Magento the ORM currently consists of the Models, Resource Models and Collections.
    If a third party module use only the repositories, Magento can change how and where data is stored, and the module will continue to work, despite these deep changes.

    Repositories generaly have methods like findById(), findByName(), put() or remove().
    In Magento these commonly are called getbyId(), save() and delete(), not even pretending they are doing anything else but wrap CRUD DB operations.

    Magento 2 repository methods can easily be exposed as API resources, making them valuable for integrations with third party systems or headless Magento instances.

    Should I add a repository for my custom entity?

    A good question. Lets prefix it with “Why” to make it even better:

    Why should I add a repository for my custom entity?”.

    As always, the answer is

    “It depends”.

    To make a long story short, if your entities will be used by other modules, then yes, you probably want to add a repository.

    There is another factor that should be added into the equation: in Magento 2, repositories can easily be exposed as Web API - that is REST and SOAP - resources.

    If that is interesting to you because of third party system integrations or a headless Magento setup, then again, yes, you probably want to add a repository for your entity.

    How do I add a repository for my custom entity?

    Lets assume you want to expose your entity as part of the REST API. If that is not true, you can skip the upcoming part on creating the interfaces and go straight to “Create the repository and data model implementation” below.
    But for the rest of this post I’m assuming you are creating interfaces.

    Create the repository and data model interfaces

    Create the folders Api/Data/ in your module. This is just convention, you could use a different location, but you should not.
    The repository interface goes into the Api/ folder. The Api/Data/ subdirectory we will use later.

    In Api/, create a PHP interface with the methods you want to expose. According to Magento 2 conventions all interface names end in the suffix Interface.
    For example, for a Hamburger entity, I would create the interface Api/HamburgerRepositoryInterface.

    Create the repository interface

    Magento 2 repositories are part of the domain logic of a module. That means, there is no fixed set of methods a repository has to implement.
    It depends entirely on the purpose of the module.

    However, in practice all repositories are quite similar. They are wrappers for CRUD functionality.
    Most have the methods getById, save, delete and getList.
    There may be more, for example the CustomerRepository has a method get, which fetches a customer by email, whereby getById is used to retrieve a customer by entity ID.

    Here is an example repository interface for a hamburger entity:

    <?php
     
    namespace VinaiKopp\Kitchen\Api;
     
    use Magento\Framework\Api\SearchCriteriaInterface;
    use VinaiKopp\Kitchen\Api\Data\HamburgerInterface;
     
    interface HamburgerRepositoryInterface
    {
        /**
         * @param int $id
         * @return \VinaiKopp\Kitchen\Api\Data\HamburgerInterface
         * @throws \Magento\Framework\Exception\NoSuchEntityException
         */
        public function getById($id);
     
        /**
         * @param \VinaiKopp\Kitchen\Api\Data\HamburgerInterface $hamburger
         * @return \VinaiKopp\Kitchen\Api\Data\HamburgerInterface
         */
        public function save(HamburgerInterface $hamburger);
     
        /**
         * @param \VinaiKopp\Kitchen\Api\Data\HamburgerInterface $hamburger
         * @return void
         */
        public function delete(HamburgerInterface $hamburger);
     
        /**
         * @param \Magento\Framework\Api\SearchCriteriaInterface $searchCriteria
         * @return \VinaiKopp\Kitchen\Api\Data\HamburgerSearchResultInterface
         */
        public function getList(SearchCriteriaInterface $searchCriteria);
    }

    Important! Here be timesinks!
    There are a few gotchas here that are hard to debug if you get them wrong:

    1. DO NOT use PHP7 scalar argument types or return types if you want to hook this into the REST API!
    2. Add PHPDoc annotations for all arguments and the return type to all methods!
    3. Use Fully Qualified Class Names in the PHPDoc block!

    The annotations are parsed by the Magento Framework to determine how to convert data to and from JSON or XML. Class imports (that is, use statements above the class) are not applied!

    Every method has to have an annotation with all argument types and the return type. Even if a method takes no arguments and returns nothing, it has to have the annotation:

    /**
     * @return void
     */

    Scalar types (string, int, float and bool) also have to be specified, both for arguments and for the return value.

    Note that in the example above, the annotations for methods that return objects are specified as interfaces, too.
    The return type interfaces are all reside the Api\Data namespace/directory.
    In Magento 2 this is to indicate that they do not contain any business logic. They are simply “bags of data”.
    We have to create these interfaces next.

    Create the DTO interface

    I think Magento calls these interfaces “data models”, a name I don’t like at all.
    This type of class is commonly known as a Data Transfer Object, or DTO.
    These DTO classes only have getter and setter methods for all their properties.

    The reason I prefer to use DTO over data model is that it is less easy to confuse with the ORM data models, resource models or view models… too many things are models in Magento already.

    The same restrictions in regards to PHP7 typing that apply to repository interfaces also apply to the DTO interfaces.
    Also, every method has to have an annotation with all argument types and the return type.

    <?php
     
    namespace VinaiKopp\Kitchen\Api\Data;
     
    use Magento\Framework\Api\ExtensibleDataInterface;
     
    interface HamburgerInterface extends ExtensibleDataInterface
    {
        /**
         * @return int
         */
        public function getId();
     
        /**
         * @param int $id
         * @return void
         */
        public function setId($id);
     
        /**
         * @return string
         */
        public function getName();
     
        /**
         * @param string $name
         * @return void
         */
        public function setName($name);
     
        /**
         * @return \VinaiKopp\Kitchen\Api\Data\IngredientInterface[]
         */
        public function getIngredients();
     
        /**
         * @param \VinaiKopp\Kitchen\Api\Data\IngredientInterface[] $ingredients
         * @return void
         */
        public function setIngredients(array $ingredients);
     
        /**
         * @return string[]
         */
        public function getImageUrls();
     
        /**
         * @param string[] $urls
         * @return void
         */
        public function setImageUrls(array $urls);
     
        /**
         * @return \VinaiKopp\Kitchen\Api\Data\HamburgerExtensionInterface|null
         */
        public function getExtensionAttributes();
     
        /**
         * @param \VinaiKopp\Kitchen\Api\Data\HamburgerExtensionInterface $extensionAttributes
         * @return void
         */
        public function setExtensionAttributes(HamburgerExtensionInterface $extensionAttributes);
    }

    If a method retrieves or returns an array, the type of the items in the array have to be specified in the PHPDoc annotation, followed by an opening and closing square bracket [].
    This is true for both scalar values (e.g. int[]) as well as objects (e.g. IngredientInterface[]).

    Note that I’m using an Api\Data\IngredientInterface as an example for a method returning an array of objects, I’ll won’t add the code of the ingredients to this post tough.

    ExtensibleDataInterface?

    In the example above the HamburgerInterface extends the ExtensibleDataInterface.
    Technically this is only required if you want other modules to be able to add attributes to your entity.
    If so, you also need to add another getter/setter pair, by convention called getExtensionAttributes() and setExtensionAttributes().

    The naming of the return type of this method is very important!

    The Magento 2 framework will generate the interface, the implementation, and the factory for the implementation if you name them just right. The details of these mechanics are out of scope of this post though.
    Just know, if the interface of the object you want to make extensible is called \VinaiKopp\Kitchen\Api\Data\HamburgerInterface, then the extension attributes type has to be \VinaiKopp\Kitchen\Api\Data\HamburgerExtensionInterface. So the word Extension has to be inserted after the entity name, right before the Interface suffix.

    If you do not want your entity to be extensible, then the DTO interface does not have to extend any other interface, and the getExtensionAttributes() and setExtensionAttributes() methods can be omitted.

    Enough about the DTO interface for now, time to return to the repository interface.

    The getList() return type SearchResults

    The repository method getList returns yet another type, that is, a SearchResultsInterface instance.

    The method getList could of course just return an array of objects matching the specified SearchCriteria, but returning a SearchResults instance allows adding some useful meta data to the returned values.

    You can see how that works below in the repository getList() method implementation.

    Here is the example hamburger search result interface:

    <?php
     
    namespace VinaiKopp\Kitchen\Api\Data;
     
    use Magento\Framework\Api\SearchResultsInterface;
     
    interface HamburgerSearchResultInterface extends SearchResultsInterface
    {
        /**
         * @return \VinaiKopp\Kitchen\Api\Data\HamburgerInterface[]
         */
        public function getItems();
     
        /**
         * @param \VinaiKopp\Kitchen\Api\Data\HamburgerInterface[] $items
         * @return void
         */
        public function setItems(array $items);
    }

    All this interface does is it overrides the types for the two methods getItems() and setItems() of the parent interface.

    Summary of interfaces

    We now have the following interfaces:

    • \VinaiKopp\Kitchen\Api\HamburgerRepositoryInterface
    • \VinaiKopp\Kitchen\Api\Data\HamburgerInterface
    • \VinaiKopp\Kitchen\Api\Data\HamburgerSearchResultInterface

    The repository extends nothing,
    the HamburgerInterface extends the \Magento\Framework\Api\ExtensibleDataInterface,
    and the HamburgerSearchResultInterface extends the \Magento\Framework\Api\SearchResultsInterface.

    Create the repository and data model implementations

    The next step is to create the implementations of the three interfaces.

    The Repository

    In essence, the repository uses the ORM to do it’s job.

    The getById(), save() and delete() methods are quite straight forward.
    The HamburgerFactory is injected into the repository as a constructor argument, as can be seen a bit further below.

    public function getById($id)
    {
        $hamburger = $this->hamburgerFactory->create();
        $hamburger->getResource()->load($hamburger, $id);
        if (! $hamburger->getId()) {
            throw new NoSuchEntityException(__('Unable to find hamburger with ID "%1"', $id));
        }
        return $hamburger;
    }
     
    public function save(HamburgerInterface $hamburger)
    {
        $hamburger->getResource()->save($hamburger);
        return $hamburger;
    }
     
    public function delete(HamburgerInterface $hamburger)
    {
        $hamburger->getResource()->delete($hamburger);
    }

    Now to the most interesting part of a repository, the getList() method.
    The getList() method has to translate the SerachCriteria conditions into method calls on the collection.

    The tricky part of that is getting the AND and OR conditions for the collection filters right, especially since the syntax for setting the conditions on the collection is different depending on whether it is an EAV or a flat table entity.

    In most cases, getList() can be implemented as illustrated in the example below.

    <?php
     
    namespace VinaiKopp\Kitchen\Model;
     
    use Magento\Framework\Api\SearchCriteriaInterface;
    use Magento\Framework\Api\SortOrder;
    use Magento\Framework\Exception\NoSuchEntityException;
    use VinaiKopp\Kitchen\Api\Data\HamburgerInterface;
    use VinaiKopp\Kitchen\Api\Data\HamburgerSearchResultInterface;
    use VinaiKopp\Kitchen\Api\Data\HamburgerSearchResultInterfaceFactory;
    use VinaiKopp\Kitchen\Api\HamburgerRepositoryInterface;
    use VinaiKopp\Kitchen\Model\ResourceModel\Hamburger\Collection as HamburgerCollectionFactory;
    use VinaiKopp\Kitchen\Model\ResourceModel\Hamburger\Collection;
     
    class HamburgerRepository implements HamburgerRepositoryInterface
    {
        /**
         * @var Hamburger
         */
        private $hamburgerFactory;
     
        /**
         * @var HamburgerCollectionFactory
         */
        private $hamburgerCollectionFactory;
     
        /**
         * @var HamburgerSearchResultInterfaceFactory
         */
        private $searchResultFactory;
     
        public function __construct(
            Hamburger $hamburgerFactory,
            HamburgerCollectionFactory $hamburgerCollectionFactory,
            HamburgerSearchResultInterfaceFactory $hamburgerSearchResultInterfaceFactory
        ) {
            $this->hamburgerFactory = $hamburgerFactory;
            $this->hamburgerCollectionFactory = $hamburgerCollectionFactory;
            $this->searchResultFactory = $hamburgerSearchResultInterfaceFactory;
        }
     
        // ... getById, save and delete methods listed above ...
     
        public function getList(SearchCriteriaInterface $searchCriteria)
        {
            $collection = $this->collectionFactory->create();
     
            $this->addFiltersToCollection($searchCriteria, $collection);
            $this->addSortOrdersToCollection($searchCriteria, $collection);
            $this->addPagingToCollection($searchCriteria, $collection);
     
            $collection->load();
     
            return $this->buildSearchResult($searchCriteria, $collection);
        }
     
        private function addFiltersToCollection(SearchCriteriaInterface $searchCriteria, Collection $collection)
        {
            foreach ($searchCriteria->getFilterGroups() as $filterGroup) {
                $fields = $conditions = [];
                foreach ($filterGroup->getFilters() as $filter) {
                    $fields[] = $filter->getField();
                    $conditions[] = [$filter->getConditionType() => $filter->getValue()];
                }
                $collection->addFieldToFilter($fields, $conditions);
            }
        }
     
        private function addSortOrdersToCollection(SearchCriteriaInterface $searchCriteria, Collection $collection)
        {
            foreach ((array) $searchCriteria->getSortOrders() as $sortOrder) {
                $direction = $sortOrder->getDirection() == SortOrder::SORT_ASC ? 'asc' : 'desc';
                $collection->addOrder($sortOrder->getField(), $direction);
            }
        }
     
        private function addPagingToCollection(SearchCriteriaInterface $searchCriteria, Collection $collection)
        {
            $collection->setPageSize($searchCriteria->getPageSize());
            $collection->setCurPage($searchCriteria->getCurrentPage());
        }
     
        private function buildSearchResult(SearchCriteriaInterface $searchCriteria, Collection $collection)
        {
            $searchResults = $this->searchResultFactory->create();
     
            $searchResults->setSearchCriteria($searchCriteria);
            $searchResults->setItems($collection->getItems());
            $searchResults->setTotalCount($collection->getSize());
     
            return $searchResults;
        }
    }

    Filters within a FilterGroup must be combined using an OR operator.
    Separate filter groups are combined using the logical AND operator.

    Phew
    This was the biggest bit of work. The other interface implementations are simpler.

    The DTO

    Magento originally intended developers to implement the DTO as separate classes, distinct from the entity model.

    The core team only did this for the customer module though (\Magento\Customer\Api\Data\CustomerInterface is implemented by \Magento\Customer\Model\Data\Customer, not \Magento\Customer\Model\Customer).
    In all other cases the entity model implements the DTO interface (for example \Magento\Catalog\Api\Data\ProductInterface is implemented by \Magento\Catalog\Model\Product).

    I’ve asked members of the core team about this at conferences, but I didn’t get a clear response what is to be considered good practice.
    My impression is that this recommendation has been abandoned. It would be nice to get an official statement on this though.

    For now I’ve made the pragmatic decision to use the model as the DTO interface implementation. If you feel it is cleaner to use a separate data model, feel free to do so. Both approaches work fine in practice.

    If the DTO inteface extends the Magento\Framework\Api\ExtensibleDataInterface, the model has to extend Magento\Framework\Model\AbstractExtensibleModel.
    If you don’t care about the extensibility, the model can simply continue to extend the ORM model base class Magento\Framework\Model\AbstractModel.

    Since the example HamburgerInterface extends the ExtensibleDataInterface the hamburger model extends the AbstractExtensibleModel, as can be seen here:

    <?php
     
    namespace VinaiKopp\Kitchen\Model;
     
    use Magento\Framework\Model\AbstractExtensibleModel;
    use VinaiKopp\Kitchen\Api\Data\HamburgerExtensionInterface;
    use VinaiKopp\Kitchen\Api\Data\HamburgerInterface;
     
    class Hamburger extends AbstractExtensibleModel implements HamburgerInterface
    {
        const NAME = 'name';
        const INGREDIENTS = 'ingredients';
        const IMAGE_URLS = 'image_urls';
     
        protected function _construct()
        {
            $this->_init(ResourceModel\Hamburger::class);
        }
     
        public function getName()
        {
            return $this->_getData(self::NAME);
        }
     
        public function setName($name)
        {
            $this->setData(self::NAME);
        }
     
        public function getIngredients()
        {
            return $this->_getData(self::INGREDIENTS);
        }
     
        public function setIngredients(array $ingredients)
        {
            $this->setData(self::INGREDIENTS, $ingredients);
        }
     
        public function getImageUrls()
        {
            $this->_getData(self::IMAGE_URLS);
        }
     
        public function setImageUrls(array $urls)
        {
            $this->setData(self::IMAGE_URLS, $urls);
        }
     
        public function getExtensionAttributes()
        {
            return $this->_getExtensionAttributes();
        }
     
        public function setExtensionAttributes(HamburgerExtensionInterface $extensionAttributes)
        {
            $this->_setExtensionAttributes($extensionAttributes);
        }
    }

    Extracting the property names into constants allows to keep them in one place. They can be used by the getter/setter pair and also by the Setup script that creates the database table. Otherwise there is no benefit in extracting them into constants.

    The SearchResult

    The SearchResultsInterface is the simplest of the three interfaces to implement, since it can inherit all of it’s functionality from a framework class.

    <?php
     
    namespace VinaiKopp\Kitchen\Model;
     
    use Magento\Framework\Api\SearchResults;
    use VinaiKopp\Kitchen\Api\Data\HamburgerSearchResultInterface;
     
    class HamburgerSearchResult extends SearchResults implements HamburgerSearchResultInterface
    {
     
    }

    Configure the ObjectManager preferences

    Even though the implementations are complete, we still can’t use the interfaces as dependencies of other classes, since the Magento Framework object manager does not know what implementations to use. We need to add an etc/di.xml configuration for with the preferences.

    <?xml version="1.0"?>
    <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:ObjectManager/etc/config.xsd">
        <preference for="VinaiKopp\Kitchen\Api\HamburgerRepositoryInterface" type="VinaiKopp\Kitchen\Model\HamburgerRepository"/>
        <preference for="VinaiKopp\Kitchen\Api\Data\HamburgerInterface" type="VinaiKopp\Kitchen\Model\Hamburger"/>
        <preference for="VinaiKopp\Kitchen\Api\Data\HamburgerSearchResultInterface" type="VinaiKopp\Kitchen\Model\HamburgerSearchResult"/>
    </config>

    How can the repository be exposed as an API resource?

    This part is really simple, it’s the reward for going through all the work creating the interfaces, the implementations and wiring them together.

    All we need to do is create an etc/webapi.xml file.

    <?xml version="1.0"?>
    <routes xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Webapi:etc/webapi.xsd">
        <route method="GET" url="/V1/vinaikopp_hamburgers/:id">
            <service class="VinaiKopp\Kitchen\Api\HamburgerRepositoryInterface" method="getById"/>
            <resources>
                <resource ref="anonymous"/>
            </resources>
        </route>
        <route method="GET" url="/V1/vinaikopp_hamburgers">
            <service class="VinaiKopp\Kitchen\Api\HamburgerRepositoryInterface" method="getList"/>
            <resources>
                <resource ref="anonymouns"/>
            </resources>
        </route>
        <route method="POST" url="/V1/vinaikopp_hamburgers">
            <service class="VinaiKopp\Kitchen\Api\HamburgerRepositoryInterface" method="save"/>
            <resources>
                <resource ref="anonymous"/>
            </resources>
        </route>
        <route method="PUT" url="/V1/vinaikopp_hamburgers">
            <service class="VinaiKopp\Kitchen\Api\HamburgerRepositoryInterface" method="save"/>
            <resources>
                <resource ref="anonymous"/>
            </resources>
        </route>
        <route method="DELETE" url="/V1/vinaikopp_hamburgers">
            <service class="VinaiKopp\Kitchen\Api\HamburgerRepositoryInterface" method="delete"/>
            <resources>
                <resource ref="anonymous"/>
            </resources>
        </route>
    </routes>

    Note that this configuration not only enables the use of the repository as REST endpoints, it also exposes the methods as part of the SOAP API.

    In the first example route, <route method="GET" url="/V1/vinaikopp_hamburgers/:id">, the placeholder :id has to match the name of the argument to the mapped method, public function getById($id).
    The two names have to match, for example /V1/vinaikopp_hamburgers/:hamburgerId would not work, since the method argument variable name is $id.

    For this example I’ve set the accessibility to <resource ref="anonymous"/>. This means the resource is exposed publically without any restriction!
    To make a resource only available to a logged in customer, use <resource ref="self"/>. In this case the special word me in the resource endpoint URL will be used to populate an argument variable $id with the ID of the currently logged in customer.
    Have a look at the Magento Customer etc/webapi.xml and CustomerRepositoryInterface if you need that.

    Finally, the <resources> can also be used to restrict access to a resource to an admin user account. To do this set the <resource> ref to an identifier defined in an etc/acl.xml file.
    For example, <resource ref="Magento_Customer::manage"/> would restrict access to any admin account who is privileged to manage customers.

    Note that both the POST and PUT HTTP methods are mapped to the same repository interface method save().
    REST treats creation (a.k.a. inserts) and updates as separate actions, where in Magento both are handled by the same save() method.

    I hope this post will be useful to help you decide if you want to implement repositories for your custom entities, and if so, how to do that and also how to expose them as API endpoints.

    comments powered by Disqus