oe-async-service


oe-async-service is a standalone http service that transforms external synchronous rest apis to asynchronous and takes care of successful execution for you.

Getting Started

Deploying and Configuring in CEP


Deploying in CEP

  1. Create the following docker stack file and name it async-service.yml
version: "3"
services:
  processor:
    entrypoint: ["node", "."]
    image: registry.oecloud.ad.infosys.com/oecloudio-oe-async-service-processor
    depends_on:
      - rabbitmq
    deploy:
      mode: replicated
      replicas: 1
      update_config:
        delay: 60s
    environment:
      ROUTER_HOSTNAME: "router"
      RABBITMQ_HOSTNAME: "rabbitmq"
      MAX_RETRY: "15"
      RETRIABLE_STATUS_CODES: "[503,500]"
      NODE_TLS_REJECT_UNAUTHORIZED: "0"
    networks:
      - async-service_network
      - router_network

  rabbitmq:
    image: registry.oecloud.ad.infosys.com/rabbitmq:3.6-management-alpine
    environment:
      VIRTUAL_HOST: "https://rabbitmq.oecloud.ad.infosys.com,rabbitmq.oecloud.ad.infosys.com"
      SERVICE_PORTS: "15672"
    networks:
      - async-service_network
      - router_network

  server:
    entrypoint: ["node", "."]
    image: registry.oecloud.ad.infosys.com/oecloudio-oe-async-service-server
    depends_on:
      - rabbitmq
    deploy:
      mode: replicated
      replicas: 1
      update_config:
        delay: 60s
    environment:
      VIRTUAL_HOST: "https://async-service.oecloud.ad.infosys.com,async-service.oecloud.ad.infosys.com"
      SERVICE_PORTS: "3000"
      FORCE_SSL: "true"
      ROUTER_HOSTNAME: "router"
      RABBITMQ_HOSTNAME: "rabbitmq"
      NODE_TLS_REJECT_UNAUTHORIZED: "0"
      RETRIABLE_STATUS_CODES: "[503,500]"
    networks:
      - async-service_network
      - router_network

networks:
  async-service_network:
  router_network:
    external: true
  1. Run docker stack deploy -c async-service.yml async-service

Configuration

oe-async-service is configurable through environment variables

  • RABBITMQ_HOSTNAME - Configurable hostname for rabbitmq, port is also configurable for example: RABBITMQ_HOSTNAME=rabbitmq:8888
  • ROUTER_HOSTNAME - haproxy hostname
  • MAX_RETRY - the maximum amount of times to retry a request before sending it to the dead queue
  • RETRIABLE_STATUS_CODES - A JSON array specifying which status codes are retriable for your service "[503,500]"

Using oe-async-service

Prerequisites

oe-async-service will query your service before sending any requests to make sure that it is retryable.

If your service is using oe-cloud as a framework all you have to do is add the retry-support-mixin to your model.

Otherwise if your service is on the uri: /api/serviceName you will have to add the following endpoints:

  • GET on /api/serviceName/isRetryable should return 200 status code

  • GET on /api/serviceName/primaryKeyField should return 200 status code and JSON body formatted like so:

    {
    	name: primaryKeyFieldName
    }
    

    Where primaryKeyFieldName is the field name of the service primary key.

Issuing an async request

oe-async-service supports only post,put and delete requests.

In order to issue an async request to your service issue the same request (path,body,headers) to oe-async-service and add the header

"x-host: your service hostname"

For example for the following request:

curl -X POST "https://myService.oecloud.ad.infosys.com/api/BaseUsers?access_token=5dBlyKvBA7rbFetvJ3IJBHW8DZd5S7fyJZCmTm1MxHSqPFXkTvLY8NbJyYcDBgy1" -H  "accept: application/json" -H  "content-type: application/json" -d "{}"

Generate the following request:

curl -X POST "https://async-service.oecloud.ad.infosys.com/api/BaseUsers?access_token=5dBlyKvBA7rbFetvJ3IJBHW8DZd5S7fyJZCmTm1MxHSqPFXkTvLY8NbJyYcDBgy1" -H  "accept: application/json" -H  "content-type: application/json" -H "X-host: myService.oecloud.ad.infosys.com" -d "{}"

if you request is well formed and your backend supports retries (see here) oe-async-service will return 200 status code and a body that contains the id field.

The value in the id field is the id of the entity that will be generated, you can use this id to query your service and make sure the request was processed.

Querying The Dead Queue

oe-async-service will generate a queue called dead queue in rabbitmq - this queue will hold all requests that failed MAX_RETRY times and will not be retryed again.

You can query this queue with rabbitmq user interface https://rabbitmq.oecloud.ad.infosys.com (in the example docker-compose) or make a script that will consume this queue and do something for each dead message (email, alert, etc).

Diagram

Diagram