April 12, 2020

Adding Middleware To Python Requests

By jobala

I have been working on the Python SDK for Microsoft Graph for the last month or so and I am having so much fun. The sdk is built on top of the requests library preventing us from re-inventing the wheel. However, we need to extend the requests library with our own use cases. In this article, I show you how we did that using middleware.

Let reduce the problem to modifying a request object before it is sent to the server.  For example, adding an authorization header to the request object so that our users don’t have to do this. We can do this by passing the request object through a middleware pipeline, each middleware in the pipeline adding new properties to the request object.

As fate had it, request does not support interceptors so we don’t have access to the request object — bummer. Fortunately, requests has transport adapters which are used to mock HTTP services and the good news is, the adapters implement a send method which receives a request object and returns a response object. If we can get access to the send method we’ll have the request object for free.

Inheriting from a transport adapter class will give us the send method and with that the request object. This means that all middleware should inherit from some transport adapter class, in our case, we will inherit from HTTPAdapter. Just when I was getting excited I learnt of another constraint, you can only attach one adapter — think one middleware — to requests. A soul can’t be happy in this industry.

After some thinking, I realized I could overcome the hurdle with a chain of objects, a linkedlist. Each node in the linked list will be in fact a middleware. 

Enough talking, let’s write some code.

First create a MiddlewarePipeline class. We will attach an instance of this class to requests. The add_middleware method adds, middleware into the pipeline by linking them. This is the entry point of all middleware.

class MiddlewarePipeline(HTTPAdapter):
      def __init__(self):
          super().__init()
          self._middleware = None
      
      def send(self, request, **kwargs):
	  if self._middleware:
   	     return self._middleware.send(request, **kwargs)
          return super().send(request, **kwargs)
   	 
      def add_middleware(self, middleware):
          if self.middleware:
   	     self.middleware.next  = middleware
          else:
   	     self.middleware = middleware

The send method checks if there is any middleware in the pipeline — possible for the pipeline to be empty — If the pipeline is empty it calls HTTPAdapter’s send method which makes the requests to the server. If we have at least one middleware in the pipeline, it calls the middleware’s send method, this kick starts the request object’s journey through the pipeline.

With the pipeline in place, we now create a middleware. For simplicity, lets create an authorization middleware that adds an authorization header to the request object.

class AuthorizationMiddleware(HTTPAdapter):
    def __init__(self):
        self.next = None
    
    def send(self, requests, **kwargs):
        token = 'Bearer {a-token}'
        request.headers.update({Authorization: token})
 
        if self.next is None:
   	     return super().send(request, **kwargs)
        return self.next.send(requests, **kwargs)

The next property holds a  reference to the next middleware in the pipeline. In the send method, we get the token, then update the request object with an authorization header. Then we check whether this middleware is the last in the pipeline, if it is, call the HTTPAdapter’s send method — to make a network request — if not pass the modified request object to the next middleware’s send method.

Finally we attach the middleware pipeline to a requests session

from requests import Session
 
middleware_pipeline = MiddlewarePipeline()
auth_middleware = AuthorizationMiddleware()
middleware_pipeline.add_middleware(auth_middleware)
 
session = Session()
session.mount('https', middleware_pipeline)


If you enjoyed the article, you will love working on the sdk. We’ll appreciate any help we can get, code reviews, documentation and feature pull requests. You can find the repo here.