oe-common-mixins

Introduction

oeCloud Common Mixins is a set of reusable functionalities which can be declaratively attached to any Model as a mixin. This module implements functionalities that are commonly sought for any enterprise application. These functionalities are,

  • Version Mixin
  • Audit Field Mixin
  • Soft Delete Mixin
  • History Mixin
  • Crypto Mixin

Dependency

  • oe-cloud
  • oe-logger

Development

$ git clone http://evgit/oecloud.io/oe-common-mixins.git
$ cd oe-common-mixins
$ npm install --no-optional
$ npm run grunt-cover

You should see coverage report in coverage folder.

Installation

$ npm install oe-common-mixins --save

Enabling the feature

Using app-list.json

This is most ideal and preferred way of loading any oeCloud node module in application. This guarantees all functionality applies to application and no extra code is required.

The app-list.json is application’s module file which is loaded as part of boot. The oeCloud framework goes through this file and loads modules in sequence as specified in app-list.json. It also applies mixins, runs boot scripts, loads middlewares and so on.

This feature applies mixins to BaseEntity model - which is usually the base model for all the models in oeCloud based application. Thus mixin applies on BaseEntity is also available in your model.

Application developer should have following entry in app-list.json. This will attach all the mixins available in oe-common-mixins to BaseEntity.

{
  "path": "oe-common-mixins",
  "enabled": true
},

If you want only some of the mixins to be enabled by default, you can specify this explicitly in the app-list.json entry. Following example enables only AuditFieldsMixin and CryptoMixin while disabling other mixins.

{
  "path": "oe-common-mixins",
  "enabled": true,
  "AuditFieldsMixin": true,
  "CryptoMixin": true,
  "VersionMixin": false,
  "SoftDeleteMixin": false,
  "HistoryMixin": false
},

Using model-config.json

In application’s model-config.json file, you can have an entry for mixin directory pointing to module’s mixin folder, as shown below. This way mixin will be loaded and made available for usage as part of boot script.

"_meta": {
        "sources": [
            "../server/models",
            "../common/models",
            "./models",
        ],
        "mixins": [
            "../common/mixins",
            "./mixins"
            "oe-common-mixins/common/mixins"
        ]
    },
...
...

This approach provides more control to the application developer as it will not auto-apply these mixins onto BaseEntity. Application developer can selectively apply these mixins to specific models declaratively.

As shown above, oe-common-mixins’s mixin path is declared in application. Once this is done, you can apply these mixins to the model you want.

{
  "name": "Customer",
  "base": "BaseEntity",
  "properties": {
    "name": {
      "type": "string",
      "unique" : true
    }
    ...
  },
  "mixins" : {
      "AuditFieldsMixin" : true
  }
}

This will add AuditFieldsMixin functionality to the Customer model.

AuditFieldsMixin

When application creates or updates records of a particular entity (or model), you may want to store the audit information like, who has created the record, when it was created, who has last modified this record and when it was last modified. The framework has ability to maintain this information in same Model. Once the AuditFieldsMixin is enabled on a particular model, the oeCloud framework will maintain this information. This way, at any point of time, you will always know when record was crated, who created that record and when it was last updated by whom.

AuditFieldMixin creates following properties to Model where it is attached to.

  • _createdBy : who has created this record. This information is taken from context. If it is http request, it is usually logged in user id. If it is called by JavaScript API, you can explicitly assign userId to ctx.remoteUser in options. This field is touched only when record is created.
  • _modifiedBy : this is very similar to _createdBy field except it is populated for both create and update operations.
  • _createdOn : When record is created. This is server’s date time where application is running and not database time.
  • _modifiedOn : Same as above exept it will be populated during update and create operations.

Developer Considerations

  • AuditFieldsMixin operates on context information of remote user. For http requests, this information is populated in context based on AccessToken. If this information is not available, then remote user will be set to system.
  • When call is made from JavaScript code, you either have to pass remoteUser in context or this mixin will set createdBy and updatedBy to system

VersionMixin

Enterprise application should always handle concurrency gracefully. There will always be cases where more than one user is updating same record at same time. This may cause data inconsistency. For example, let us consider the case where two users have fetched same customer address record and start modifying it. First user changes line1 and city of address. The other user changes line2 and country in address. The PUT or POST request would contain entire address record. When second user sends the request it has new values for line2 and country but outdated values for line1 and city. The default loopback behavior is to simply overwrite the record. Accordingly, first user’s changes get overwritten by second user.

Both requests are successfully executed but new state of the record is not right.

To avoid this, VersionMixin plays important role.

  • Version mixin maintains the version of each record.
  • When record gets updated, _version field changes to new value
  • Programmer / caller must always pass current version for update operation
  • Since for every update version gets change, above issue is prevented. In that scenario, second request would get error as version mismatch.

VersionMixin creates following properties to Model where it is attached to.

  • _version : This property maintains current version of the record.
  • _oldVersion : This property maintains previous version of record.
  • _newVersion : This property is temporarily used to give newVersion value explicitly by caller

Developer Considerations

  • Version mixin ensures that _version value is given for any update and delete operation. It changes deleteById http end point by adding version field to it. Therefore, http end point to delete a record would be
  DELETE http://example.com/api/customers/{id}/{version}

This way, when you call model.destroyById programatically, you will have to call destroyById method by passing version

model.destroyById(<id>, <version>, options, function(err, result){
  // see results here
});
  • As a developer, you should be careful as for some models you will pass version while deleting and for some models you will not pass it depending on whether the VersionMixin was enabled for those models or not.

  • If programmer calls updateAll method in javascript, version checking would not be possible as multiple records are getting updated. For such models where version mixin is enabled, you should disable updateAll method. If it is not disabled or it is somehow gets called, concurrent update of same records could be possible.

EmbedsOne Relation

When a model has embedded child model, version mixin behavior is little different.

POST - create parent record

When you are creating record in parent model and passing data of embedded model along, you don’t have to worry about anything special.

Once record is created in parent model, _version field will automatically be generated and same value will be populated for both parent and embedded record. Response of such request will look like,

POST /api/Books

{
  "name": "The BFG",
  "id": "B001",
  "_version": "679d0579-67d2-4e12-83a1-f1c8c20981c9",
  "author": {
    "name": "Roald",
    "id": "A001",
    "_version": "679d0579-67d2-4e12-83a1-f1c8c20981c9"
  }
}

Validation: Child record and parent record both are validated.

PUT - Update parent record

For this operation, _version field must be populated in your request body. The value must be the current version of the record that you are trying to modify. If no record with given version is found then API will return error.

PUT /api/Books/B001

{
 "name": "The Magic Finger",
  "id": "B001",
  "_version": "679d0579-67d2-4e12-83a1-f1c8c20981c9"
}

Response

{
  "name": "The Magic Finger",
  "id": "B001",
  "_oldVersion": "679d0579-67d2-4e12-83a1-f1c8c20981c9",
  "_version": "c289c6d2-4bdc-482a-8adc-76d215a402f5",
  "author": {
    "name": "Roald",
    "id": "A001",
    "_version": "679d0579-67d2-4e12-83a1-f1c8c20981c9"
  }
}

Note that only parent record’s _version field is updated with new value. Embedded child record’s version remains same.

Validation : Only parent data is validated.

PUT - Updating embedded record

Consider, you want to just update embedded child record. But since embedded data is not residing in separate collection (or table), you are really modifying parent record. Therefore you need to supply parent record’s version as well as child record’s version. Parent record’s version is supplied in _parentVersion field.

POST /api/Books/B001/author

{
  "name": "Roald Dahl",
  "id": "A001",
  "_version": "679d0579-67d2-4e12-83a1-f1c8c20981c9",
  "_parentVersion": "c289c6d2-4bdc-482a-8adc-76d215a402f5"
}

As you can see above, both _version and _parentVersion is supplied. This becomes important if parent record’s version and embedded record version are not same (due to an update in previous section).

However when both are same, you may supply any one of _parentVersion or _version. However it is good practice to supply both of these fields.

Response of such request will look like,

{
  "name": "Roald Dahl",
  "id": "A001",
  "_version": "486c7ea9-f1b6-413a-afdf-64210e72e81e",
  "_parentVersion": "c289c6d2-4bdc-482a-8adc-76d215a402f5"
}

You will find that the _version field of both parent and embedded record is same again after this update.

Validation: All fields in embedded model are validated. However _version is validated for parent model also. before save hooks on embedded model and parent model are called.

Embedded model without Version Mixin In above examples, we have assumed that both main as well as embedded model VersionMixin enabled. However, if that is not the case, then it will not be possible to pass _parentVersion as strict flag on model is true by default.

SoftDeleteMixin

In typical dnterprise application, data never gets deleted. Data always invalidated or soft deleted. The default behavior in an oeCloud application, is to hard delete the data permanently from the database whenever model.deleteById or model.destroyAll is called programatically or delete API is invoked.

The SoftDeleteMixin enables the soft or logical delete functionality. If you have enabled SoftDeleteMixin=true on a model, the data from that Model will never gets deleted permanently. Instead, SoftDeleteMixin would maintain a flag named _isDeleted. This flag is set to true for the record which is deleted. When user calls model.destroyById() or model.destroyAll(), this functionality ensures that records are not deleted but they are updated with _isDeleted is set to true for those records.

When application retrieves records using model.find() method, this mixin adds filter _isDeleted = false to ensure that deleted records are never fetched.

SoftDeleteMixin adds following field to the Model schema

_isDeleted: Default value of this field (or property) is false. When application deletes the record, this value is set to true.

Developer Considerations

  • Soft Delete feature works properly when you are deleting single instance. It wraps destroyAll method of connector and then calls update method of connector and thus prevent default behavior of deleting records.

  • This means that the data received by the observer hooks are not the one for delete operation but update operation instead.

    Model.destroyById(id, options, function(err, result){
      // get results of delete operation
    });
    
    connector.observe("execute", function(ctx, next){
      // ctx.req,sql - you will find "update" query when you were expecting "delete"
      return next();
    })
    
  • Soft Delete Mixin with Version mixin: When you are deleting more than one record at time (using model.destroyAll()), it gets converted into “update” method. For all the records, this module will set _isDeleted=true and _version value of all those updated records will be same.

Crypto Mixin

In cryptography, encryption is the process of encoding messages or information in such a way that only authorized parties can read it. Encryption does not prevent interception, but denies the message content to the interceptor. In an encryption scheme, the intended communication information or message, referred to as plaintext, is encrypted using an encryption algorithm, generating ciphertext that can only be read if decrypted. An authorized recipient can easily decrypt the message with the key provided by the originator to recipients, but not to unauthorized interceptors.

In the oeCloud.io based applications, it is possible to encrypt “data at rest”. With this feature, you can declare a model property to be encrypted before it is persisted in the database. For example, you can have model called CreditCard and you can save actual credit card number in encrypted form in database.

Using Cryto Mixin

To use CryptoMixin you must load this mixin into your application. Usually this happens during boot of the application. There are several ways to configure mixin paths for your application. You can do it declaratively or you can do it programatically. That is described in section below.

To enable the functionality, you should

  • Add encrypt: true on the property that you want to encrypt.
  • Add CryptoMixin: true under mixins property on the model.
{
    "name": "CardPayment",
    "properties": {
      "cardNo": {
        "type": "string",
        "encrypt": true
      },
      "amount": {
        "type": "number"
      }
    },
    "mixins": {
      "CryptoMixin": true
    }
}

Upon retrieval of the data (using find / findById etc., ) the response will contain the decrypted data.

Configuration

The encryption algorithm is configurable via config.json. Two related config params are added to config.json:

  • encryptionAlgorithm - Algorithm to be used for encryption. Following values are valid:
    • crypto.aes256
    • crypto.aes-256-ctr (default)
    • crypto.aes-256-cbc
    • crypto.aes128
    • crypto.aes192
  • encryptionPassword: the passphrase or encryption key.

The encryption algorithms are defined under the “crypto” function in lib/encryption.js, hence the “crypto.” prefix. More algorithms can be easily added to lib/encryption.js. You just have to define one function in it and export it.

Design

  • The implementation is done in common/mixin/crypto-mixin.js
  • The algorithms are picked up from lib/encryption.js.
  • The implementation also depends on a changes done in loopback-datasource-juggler – an implementation of an “after access” hook.

The drawback of this implementation is that you can’t query on encrypted fields in the database.