reqclient - Node.js HTTP Client

reqclient uses modulerequest to make requests, but adds Promise supports, and many useful features.

Usage

The module provides the class RequestClient, a wrapper class of the HTTP client module request, but makes requests returning Promise objects to handle the responses without blocking the execution, and removes boilerplate configurations on each request: base URL, time out, content type format, default headers, parameters and query formatting in the URL, authentication, and error handling.

Also support in-memory cache of GET responses, and allows to log all operations in cURL syntax style.

var RequestClient = require("reqclient").RequestClient;

var client = new RequestClient("http://baseurl.com/api/");

// Simple GET with Promise handling
client.get("reports/clients")
  .then(function(response) {
    console.log(response);  // REST responses are parsed as JSON objects
  })
  .catch(function(err) {
    console.error(err);
  });

// POST with JSON body and headers
var p = client.post("order", {"client": 1234, "ref_id": "A987"}, {headers: {"x-token": "AFF01XX"}})
// Do something with the Promise `p` ...

// GET with query (http://baseurl.com/api/orders?state=open&limit=10)
client.get({"uri": "orders", "query": {"state": "open", "limit": 10}})

// DELETE with params (http://baseurl.com/api/orders/1234/A987)
client.delete({
    "uri": "orders/{client}/{id}",
    "params": {"client": "A987", "id": 1234}
}).then(handler).catch(errorHanler);

Allows most common HTTP operations: GET, POST, PUT, DELETE, PATCH.

Options

When the RequestClient class is instantiated, a base URL has to be passed as a parameter, or an object with the following options:

• baseUrl The base URL for all the request
• timeout (optional) The TTL of the request in milliseconds
• contentType (optional, default json) Content type, valid values: json, form or formData
• headers (optional) Object with default values to send as headers. Additional headers values can be added in the request call, even override these values
• auth (optional) HTTP Authentication options. The object must contain:
  • user || username
  • pass || password
  • sendImmediately (optional)
  • bearer (optional)
• oauth2 (optional) OAuth 2 Authorization options. The object must contain:
  • The same options than config object, otherwise inherit from config these options: baseUrl, timeout, debugRequest, debugResponse, logger, auth
  • contentType (default form)
  • tokenEndpoint (default token as recommended by the standard)
  • grantType (default client_credentials if oauth2.user isn't provided, otherwise password) The grant_type parameter provider to the endpoint to specify the authentication type
  • user (optional) Object with the user authentication information for a password grant type authentication. Should contains:
    • username
    • password
• encodeQuery (optional, default true) Encode query parameters replacing "unsafe" characters in the URL with the corresponding hexadecimal equivalent code (eg. + -> %2B)
• cache (optional, default false) If it's set to true, adds cache support to GET requests

Logging options:

• debugRequest (optional) If it's set to true, all requests will logged with logger object in a cURL style.
• debugResponse (optional) If it's set to true, all responses will logged with logger object.
• logger (optional, by default uses the console object) The logger used to log requests, responses and errors

The options timeout, headers, auth and encodeQuery can be overridden when you make a call passing in an object as a last argument:

client.patch("patch", {"name":"Mika"}, {headers: {"x-token": "fake_token"}, timeout: 5000});

URL formatting

reqclient supports format the given URI on each call concatenating with the baseUrl provided in the constructor + query binding with a given object. This is useful mostly for two reasons: avoid boilerplate formatting and URL injection attacks when the URL parameters comes from a user form.

In the first parameter of any call you can specify a simple URI string like this: reports/sales, and in the example if the baseUrl has the value https://api.erp.example.com/v1, then the final URL will be https://api.erp.example.com/v1/reports/sales.

But if you want to provide some URL parameters to the previous example, and the data comes from a user form, the user can inject more parameters than the allowed by the system if you do a simple string concatenation. With reqclient module, you can format the URL from an object containing the URL and the parameters, it's more secure, easy, and reqclient also takes care of encode all characters of the parameters to generate a valid URL.

Supposing your parameters are in an object at req.query:

var client = new RequestClient("https://api.erp.example.com/v1");
client.get({
  "uri": "reports/sales",
  "query": {
    "location": req.query.location, //-> = "Buenos Aires"
    "limit": req.query.limit,       //-> = "20"
    "index": 0
  }
}).then(resp => { /* Do something with the response... */ });
// GET to https://api.erp.example.com/v1/reports/sales?location=Buenos%20Aires&limit=20&index=0

In REST services is also useful to provide resource parameters in the URI, like the ID of a client or an order number. This kind of URI are represented like this: /path/{resourceId}/to/{anotherResourceId}, and have the same issues: repetitive parsing and are exposed to URL injection.

For the previous example, supposing you want just the sales of a given client and for a given "status":

client.get({
  "uri": "reports/{clientId}/sales/{status}",
  "query": {
    "location": req.query.location, //-> = "Güemes"
    "limit": req.query.limit,       //-> = "20"
    "index": 0
  },
  "params": {
    "clientId": clientObj.id,       //-> "1234"
    "status": "done"
  }
}).then(resp => { /* ... */ }).catch(err => /* Oh my God... */);
// GET to https://api.erp.example.com/v1/reports/1234/sales/done?location=G%C3%BCemes&limit=20&index=0

Note that in both cases the "location" parameter have blank spaces or diacritics characters than in the final URL they were encoded. You can avoid the URL parameter encoding passing to the RequestClient config option the value encodeQuery: false (default to true).

When you make a call with a string, or an URI object containing the URI string, if the string starts with "http://" or "https://", then the concatenation with the baseUrl is avoided.

Logging with cURL style

By default reqclient uses the standard console object for the log activity, and only logs error responses. But when the RequestClient object is created, the constructor parameters passed can override this behavior (see above section).

In case the request activity is logged, the _debugRequest() method will print with a cURL syntax format (awesome!). This is really useful in development phase, when you need to know what it's doing your application, and you need to reproduce the calls outside the application.

var RequestClient = require("reqclient").RequestClient;
var client = new RequestClient({
        baseUrl: "http://baseurl.com/api/v1.1",
        debugRequest: true, debugResponse: true
    });

client.post("client/orders", {"client": 1234, "ref_id": "A987"}, {headers: {"x-token": "AFF01XX"}})
/* This will log ...
[Requesting client/orders]-> -X POST http://baseurl.com/api/v1.1/client/orders -d '{"client": 1234, "ref_id": "A987"}' -H '{"x-token": "AFF01XX"}' -H Content-Type:application/json
And when the response is returned ...
[Response   client/orders]<- Status 200 - {"orderId": 1320934} */

NOTE: The logging chosen can affect performance, and most important, it might have information security implications for your deployment, because the logger doesn't filter any sensitive data, like passwords, tokens, and private information. Don't set debugRequest or debugResponse to true in production environments.

Cache

By default reqclient doesn't cache results. You can activate cache of GET responses passing to its constructor config the option cache: true. Then, if you add the {cacheTtl: SECONDS} option in a get() call, the module will cache the result to return the same response the next call without accessing to the endpoint again. If the RequestClient object isn't initialized with the cache option, the cacheTtl option in the request calls will ignored.

var client = new RequestClient({baseUrl:"https://myapp.com/api/v1", cache:true});
// GET to "https://myapp.com/api/v1/orders?state=open&limit=10" and cache for 60 seconds 
client.get({ "uri": "orders", "query": {"state": "open", "limit": 10} }, {cacheTtl: 60})

NOTE: In subsequence calls the response will be read from the cache only if the cacheTtl option is present in the request.

This library use the node-cache module to create the in-memory cache. If you activate this feature, you need to add this dependency in your project.

In the example above, the cache will expire in 60 seconds, but you have to consider that if you make a POST/PUT/PATCH and alter the data (or another system do), the cache will be inconsistent, because the cache is not updated automatically (see bellow how to clean the cache).

Also take in consideration that the cache is saved in a key value store, and the key is the uri object passed to the GET call, so, if you make request passing parameters through header parameters instead of URI parameters, the cache system will be inconsistent with the real result.

Clear the cache manually

if you need to clear the cache manually, you can call deleteFromCache() method, passing the URI as a key of the response to delete. The URI could be a string or an object in the same format as in the get() calls.

// Delete the response cached in the example of the previous section
client.deleteFromCache({ "uri": "orders", "query": {"state": "open", "limit": 10} })
// This will delete the same value cached, but the URI is passed as a string
client.deleteFromCache("orders?state=open&limit=10")

Upload files

To upload files, the RequestClient class has to be initialized with contentType: "formData". If it was initialized with json (the default value), in the upload call can be specified in the header POST parameter with the option "Content-Type": "multipart/form-data".

client.post("profile/upload-photo",
            { "file": fs.createReadStream("mypic.jpg"), "id": 1234 },
            { "headers": {"Content-Type": "multipart/form-data"} } )
  .then(jsonResult => console.log("New photo URL: " + jsonResult.url))
  .catch(err => console.log("Something goes wrong with the upload: " + err));

If the logging with cURL style is activated, it will log something like this:

[Requesting profile/upload-photo]-> -X POST http://localhost:8080/api/profile/upload-photo -F "file=@mypic.jpg" -F "id=1234" -H 'Content-Type:multipart/form-data'
[Response   profile/upload-photo]<- Status 200 - {"url":"http://localhost:8080/api/profile/43535342535/mypic.jpg","success":true}
New photo URL: http://localhost:8080/api/profile/43535342535/mypic.jpg

HTTP Authentication

reqclient inherit the HTTP Authentication mechanism from the request module.

The configuration is passed as an option parameter called auth in the constructor, and should be an object containing the values:

• user || username
• pass || password
• sendImmediately (optional)
• bearer (optional)

var client = new RequestClient({
  baseUrl:"http://localhost:5000",
  auth: {user: "admin", pass: "secret"}
});

client.get("orders").then(...)...

sendImmediately: defaults to true, causes a basic or bearer authentication header to be sent. If sendImmediately is false, then request will retry with a proper authentication header after receiving a 401 response from the server (which must contain a WWW-Authenticate header indicating the required authentication method).

Bearer authentication is supported, and is activated when the bearer value is available. The value may be either a String or a Function returning a String. Using a function to supply the bearer token is particularly useful if used in conjunction with defaults to allow a single function to supply the last known token at the time of sending a request, or to compute one on the fly.

OAuth 2 Authorization

There are many ways to login against an OAuth 2.0 server, this library implements some mechanisms.

The options for the constructor object to configure OAuth2 are set in the object oauth2, and because the server where you will authenticate could be the same server you will consume endpoints or not, this objects can receive the same global options than the constructor: baseUrl, timeout, auth, debugRequest, ... If these options aren't provided, they will taken from the global options.

When you configure the OAuth2 options, reqclient will try to login with the OAuth2 endpoint before consume any endpoint to get the access token, and if a refresh token is provided, it will manage the refreshing of the access token automatically for you, or refresh it using the same grant type method used first.

Also if for some reason your token was invalidated before the expiration time, but an appropriate WWW-Authenticate header is provided in a response (as it's specified by the standard), reqclient will try authenticate one more time automatically.

client_credentials grant type

var client = new RequestClient({
  baseUrl: "http://localhost:8080/myapi" ,debugRequest:true
  ,oauth2: {
    auth: {
      user: 'client123'         // The username, also called "client_id"
      ,pass: 'thePass123'       // The password, also called "client_secret"
    }
  }
});

client.get("home-reports")      // First will try to login with OAuth2, then /home-reports 
.then(client.get("messages"));  // Will reuse the previous token obtained

The code above will log this:

[Requesting token]-> -X POST http://localhost:8080/myapi/token -u ${CLIENT_ID}:${CLIENT_SECRET} -d 'grant_type=client_credentials'
[Requesting home-reports]-> http://localhost:8080/myapi/home-reports -H "Authorization: Bearer ${ACCESS_TOKEN}"
[Requesting messages]-> http://localhost:8080/myapi/messages -H "Authorization: Bearer ${ACCESS_TOKEN}"

As you can see, the first operation was get the token against an endpoint /token, then the call to /home-reports was made with the "bearer" token obtained in the first call, and finally a new call to /messages was made also using the same token.

The default endpoint /token can be changed in the oauth2.tokenEndpoint config object, and also the baseUrl used only for the OAuth2 calls:

  ...
  ,oauth2: {
    baseUrl: 'https://api.example.com/oauth2'
    ,tokenEndpoint: 'login'
    ,auth: {user: 'client123', pass: 'thePass123'}
  } // OAuth against POST https://api.example.com/oauth2/login -u client123:thePass123 ...
  ...

password grant type

To authenticate against an OAuth 2 server with a username/password + client_id/client_secret, the credentials must be set the in a user object inside the oauth2 object with the username and password:

var client = new RequestClient({
  baseUrl: "http://localhost:8080/myapi" ,debugRequest:true
  ,oauth2: {
    auth: {
      user: 'client123'             // client_id
      ,pass: 'thePass123'           // client_secret
    }
    ,user: {
      username: "myname@mail.com"   // The user of a "real" user
      ,password: "password1234"
    }
  }
});

This will log in cURL format something like this:

[Requesting token]-> -X POST http://localhost:8080/myapi/token -u ${CLIENT_ID}:${CLIENT_SECRET} -d 'grant_type=password' -d 'username=myname@mail.com' -d "password=${PASSWORD}"

Requirements

• Node.js 4.4+ (supports Javascript classes).
• request module.
• node-cache if the cache features are used.

About

Source code: https://github.com/mrsarm/reqclient

Author: Mariano Ruiz mrsarm@gmail.com

2016 | Apache-2.0