Tutorial: Build an Interactive Chart on QIX with RxQ


Tutorial: Build an Interactive Chart on QIX with RxQ

Speros Kokenes
December 11, 2016
APIs, Qlik Sense
0

At Axis Group, we love working with the Qlik Analytics Platform’s new APIs, especially the Engine API. With the Engine API, we interface directly with QIX Engine to power custom web applications. This interaction allows us to leverage the speed and scalability of the Engine in applications outside the realm of self-service BI.

While playing with the Engine over the last few years, we’ve learned a lot as we’ve tried to scale up to bigger and more complex web applications. In the process, we’ve found that reactive programming concepts pair well with the reactive nature of the QIX Engine. With this in mind, we created a reactive programming JS interface for the QIX Engine known as RxQ. The following tutorial provides a quick tutorial on how you can use RxQ to build a simple interactive chart on top of the QIX Engine.

tutorial

You can see a live example of the chart we will build at http://opensrc.axisgroup.com/examples/chart-with-filter/. The code is available at https://github.com/axisgroup/RxQ/tree/master/examples/chart-with-filter.

This tutorial is not meant to be a deep-dive on the QIX Engine, the Engine API, or reactive programming. However, it will touch on these concepts lightly and provide links to additional resources should you want or need further reference.

Side note

Want to learn more about reactive programming? Try The introduction to Reactive Programming you’ve been missing
Want to learn more about the QIX Engine? Check out:
QIX Under the Hood

What is RxQ, and how does it work?

You can skip this part if all you care about is trying some code, but it is helpful background.

RxQ brings the Observable-pattern, as implemented by RxJS, to the Qlik Engine API. You can think of it like qsocks / enigma, but instead of wrapping the asynchronous call and responses to the QIX Engine with Promises, it wraps these calls with Observables. So for example, in qsocks to connect to a server and asynchronously get a handle for the global class in a session, you would do:

qsocks.connect();
// -> returns a Promise for a Global instance

in RxQ, you write something similar but receive an Observable instead:

RxQ.connectEngine();
// -> returns an Observable for a QixGlobal instance (more on this later)
Side note

A fundamental difference between Promises and Observables is that Promises are eager, while Observables are lazy. This is an important distinction to understand while coding with them and especially RxQ as opposed to qsocks. qsocks methods execute immediately, regardless of a listener; RxQ methods only execute when a subscriber has been added somewhere down the Observable chain.

For more on Promises vs. Observables, check out http://stackoverflow.com/questions/37364973/angular-2-promise-vs-observable

There are two key components that RxQ is built around: Qix Objects and Qix Observables.

Qix Objects

Qix Objects represent a QIX Engine Class instance and provide methods that return Observables for specific API calls. They map directly to the classes available in the Engine:

  • QixEngine
  • QixApp
  • QixField
  • QixGenericObject
  • etc.
Side note

To learn more about the QIX Engine Classes, check out the Qlik Help Docs at http://help.qlik.com/en-US/sense-developer/3.1/Subsystems/EngineAPI/Content/Classes/classes.htm

An instance of a Qix Object represents a specific handle within a QIX session. All method calls on that object will be against that handle. For example, let’s assume you establish a session with a QIX Engine and get a handle back of -1 for the Global Class. With a QixGlobal instance of that handle, you can create an Observable stream of the product version property by calling:

var myEngine;
// -> assume you somehow magically got a QixGlobal instance for a session you have

var productVersion$ = myEngine.productVersion();
// -> returns an Observable that will stream the product version response
Side note

Throughout these code examples, I will label any variables that hold an Observable with a “$” sign at the end. This is just a convenient marker that helps me keep track of my reactive streams.

So how do you get a Qix Object? Well, any Engine API call that you make via RxQ that returns an Engine Class instance with a handle will return a Qix Object for that instance. In the example above, how would I get the myEngine instance? In RxQ, you can use the connectEngine method to connect to a server and return a QixGlobal instance for the established session via an Observable:

var engine$ = RxQ.connectEngine(); 
// -> returns Observable of QixGlobal instances

This observable will stream QixGlobal instances (in this case, just 1 for the session we connect to here). If I wanted to access the underlying QixGlobal class, I could easily do that with an operator or subscriber, like:

engine$
    .do(function(qG) { 
        console.log(“a qix global instance being streamed: “, qG);
    });

Now recall that Qix Objects have methods on them to return Observables of API calls for that class. With that in mind, I can use my engine$ stream to produce new streams based on calls from the underlying Qix Object. For example, I can create a productVersion$ stream like so:

var productVersion$ = engine$
    .mergeMap(function(qG) {
        return qG.productVersion()
    });

mergeMap is a useful RxJS operator that will take values from an inner Observable and flatten the values it streams to the output of an Observable. With this operator, we can easily create new streams of any API function by chaining these operators together. For example, let’s say you want to get the properties of an app:

var appProp$ = RxQ.connectEngine()
    .mergeMap(function(qG) {
        return qG.openDoc(“<YOUR-DOC-ID>”);
    })
    .mergeMap(function(qA) {
        return qA.getAppProperties();
    });

That’s pretty good, but now you are probably thinking “wait, I have to write this weird mergeMap thing that I don’t understand everywhere? This is annoying.”. You’re right; it is pretty annoying! Enter the Qix Observables.

Qix Observables

Qix Observables are classes that extend the Observable class with custom operators that map to the Engine API methods. Qix Observables exist for all the QIX classes, such as:

  • GlobalObservable
  • AppObservable
  • FieldObservable
  • GenericObjectObservable
  • etc.

Each Qix Observable assumes that the data streaming through it will only be of the associated Qix Class. For example, GlobalObservables will only pipe through QixGlobal objects. It will throw an error if it receives any other data input.

Because these QIX Observables rely solely on one data type, we can map the internal methods of those objects to operators to make development more efficient. For example, the GlobalObservable has an operator called qOpenDoc. This essentially is an alias for doing a mergeMap. The following are equivalent:

engine$
    .mergeMap(function(qG) {
        return qG.openDoc(“<DOC-ID>");
    });
    // -> yields an Observable that passes QixApp objects

engine$
    .qOpenDoc(“<DOC-ID>”);
    // -> yields an AppObservable that passes QixApp objects

The second one is cleaner since you no longer have to write all of those mergeMaps! There is another crucial difference however. When using Qix operators from Qix Observables, the operators will automatically cast the resulting Observable to the appropriate Qix Observable when necessary. In the example above, .qOpenDoc returns an AppObservable so that we can then start calling app operators on it. This does not happen with the mergeMap method.

The result is that we can rewrite our app properties stream as:

var appProp$ = RxQ.connectEngine()
     .qOpenDoc(“<YOUR-DOC-ID>”)
     .qGetAppProperties();

That is more or less how RxQ works. With that in mind, let’s actually build something!

Step 1: Set up the project

In order to build out a chart, I am going to use an existing JS charting library called ChartJS. I’m also going to use RxQ for the QIX Engine interactions, and the RxJS library for reactive programming so that I can use Observables as part of my UI interactions as well. I will load these 3 libraries into the head of my app:

<head>
    <script src="rxq.js"></script>
    <script src="https://unpkg.com/@reactivex/rxjs/dist/global/Rx.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.4.0/Chart.bundle.js"></script>
</head>

Our project is going to have a listbox and a chart, so we need to put containers in place to hold these items. Let’s add a div and ul for our listbox, and a div for our chart. We’ll also include a reference to our JS file where we will execute our code to connect to QIX and render the chart.

     <body>
        <div id="lb-cont">
            <strong>Product Group Filter</strong>
            <ul id="myListbox">
            </ul>
        </div>
        <div id="chart-cont">
            <strong>Revenue by Region</strong>
            <canvas id="myChart" ></canvas>
        </div>
        <script src="main.js"></script>
    </body>

Finally, let’s add some initial css to set up the containers. We’ll apply border-box sizing to all elements (which will come in handy for the listbox later), position, size, and space the containers. Inline the style into the header:

<style>
      * {
                box-sizing: border-box;
                font-family: sans-serif;
                font-size: 12px;
            }

            #lb-cont {
                display: inline-block;
                vertical-align: top;
                margin-right: 30px;
            }

            ul {
                list-style: none;
                padding: 0px;
                font-size: 12px;
                margin-top: 5px;
            }

#chart-cont {
                width: 900px;
                display: inline-block;
                vertical-align: top;
            }
</style>
Step 2: Set up a connection to a server

Now we’re ready to write some JavaScript in our main file. The first thing we need to do is define a server that we want to connect to. This is done with a config object that has specific properties for host, port, etc. The config object parameters are detailed on the RxQ README. For example, here is how you would connect to Axis Group’s server:

// Define a server
var config = {
    host: "sense.axisgroup.com",
    isSecure: false
};
Side note

This example server will probably not work for you as it only whitelists certain domains for websocket traffic. This is purely an example.

If you don’t supply a config object in the next step, it will automatically connect to your Qlik Sense Desktop environment, assuming it is running.

Now that we have a config object, we can create a GlobalObservable that will establish a session with the QIX Engine and stream an instance of the QixGlobal class:

// GlobalObservable that connects to the server and returns QixGlobal
var eng$ = RxQ.connectEngine(config, "warm");

If you are unfamiliar with Observable “temperatures”, you can skip this explanation for now. But for those interested, all Observables returned by RxQ are cold, unless you specify RxQ to connect either “warm” or “hot”. In “hot” mode, Observables will always be published and connected, regardless of subscribers. In “warm” mode, Observables will be published but only connect when they have at least 1 subscriber. In general, we find that RxQ works best in “warm” mode but are open to feedback!

Side note

What’s a Hot or Cold Observable? Cold Observables produce their values within the Observable, so any subscriber gets a new set of values. In Hot Observables, values are produced outside of the Observable, so all subscribers receive the same data feed. Confused? Read more at http://blog.thoughtram.io/angular/2016/06/16/cold-vs-hot-observables.html

Step 3: Open a document in the session

Since we have a GlobalObservable setup, we can use this to open a document within that session. We will use the qOpenDoc operator, which will return an App Observable:

var app$ = eng$
    .qOpenDoc("24703994-1515-4c2c-a785-d769a9226143”);
    // -> returns AppObservable

For this example, I am using the “Executive Dashboard.qvf” file that ships with Qlik Sense by default. The app guid used above is for the deployment of the app on Axis’ server, so you will need to update with your own guid.

Side note

The various Engine API methods are documented on each class at Qlik Help Docs at http://help.qlik.com/en-US/sense-developer/3.1/Subsystems/EngineAPI/Content/Classes/classes.htm

Step 4: Create Your Generic Objects

I have to admit: I have a big ol’ crush on the generic object. They are the main interface for interacting with data in the QIX Engine. They can hold dynamic properties like list boxes and hypercubes that the Engine can populate with data in response to the data model selection state.

In order to render a chart and a listbox on the page, we need dynamic data for them from the server. We can define those data sets as properties of generic objects. First let’s set up the generic objects with their listbox and hypercube definitions:

// GenericObject definition for the chart
var genObjProp = {
    qInfo: {
        qType: "chart"
    },
    qHyperCubeDef: {
        "qDimensions": [
            {
                "qDef": {
                    "qFieldDefs": [
                        "Region"
                    ]
                },
                "qNullSuppression": true
            }
        ],
        "qMeasures": [
            {
                "qDef": {
                    "qDef": "sum([Sales Amount])"
                }
            }
        ],
        "qInitialDataFetch": [
            {
                "qLeft": 0,
                "qTop": 0,
                "qWidth": 2,
                "qHeight": 1000
            }
        ]
    }
};

// Define a generic object for the listbox
var lbProp = {
    "qInfo": {
        "qType": "filter"
    },
    "qListObjectDef": {
        "qDef": {
            "qFieldDefs": [
                "Product Group Desc"
            ]
        },
        "qInitialDataFetch": [
            {
                "qLeft": 0,
                "qTop": 0,
                "qWidth": 1,
                "qHeight": 1000
            }
        ]
    }
};

Defining these dynamic properties can be a bit hairy due to the amount of properties and nesting involved, so you may want to use a tool like the Qix Structure Generator to build the definitions via a more intuitive UI. You can then copy and paste the definition right into your code.

Now that we have these definitions, we can create generic objects using our usual QIX Operator pattern:

// Create the generic object for the chart
var gO$ = app$
    .qCreateSessionObject(genObjProp);

// Create the listbox generic object
var lb$ = app$
    .qCreateSessionObject(lbProp);
Step 5: Get the data and render it

We’re almost at the finish line! Just a couple more steps. First, we need to actually pull feeds of data from our generic objects and render them to the screen.

In the QIX Engine, when you are connected to a generic object, you can pull the layout of the object. The layout of the object will contain metadata and actual data for the dynamic properties. It will also validate the object. A validated object basically just means you have the latest layout for the object, based on what your QIX Engine session has to offer. If your user filters the data, your object will become invalidated since you no longer have the up to date data for it. To get back in sync, you must pull the layout again. That will provide you the latest state of the data and validate the object again. Rinse and repeat.

In RxQ, we offer an operator that will return an auto-updating stream of layouts. This stream will automatically fetch new layouts from objects whenever they are invalidated, allowing you to constantly receive the latest data as changes happen. To use this stream, you can use the .qLayouts() operator like so:

gO$
    .qLayouts();
    // -> streams a layout of our chart generic object whenever it changes

lb$
    .qLayouts();
    // -> streams a layout of our listbox generic object whenever it changes

Now that we have a stream of data, all we have to do is subscribe to those streams to initiate the Observables and react to the resulting data by rendering. I won’t go into depth on what these rendering functions do line by line as you can trace it yourself, but in summary the chart generic object subscriber will update a chart from ChartJS and the listbox subscriber will draw an unordered list of the filter options on the page. In the listbox rendering code, you may notice that we save off the qElemNo onto the nodes we render as an attribute. We do this so that in the next step, we can implement a click function that will consume that attribute to make selection calls properly.

// Initialize the chart with empty data
var ctx = document.getElementById("myChart");
var myChart = new Chart(ctx, {
    type: 'bar',
    data: {
        labels: [],
        datasets: [{
            label: 'Revenue ($)',
            data: [],
            backgroundColor: 'rgba(75, 192, 192, 0.2)',
            borderColor: "rgba(75, 192, 192, 1)",
            borderWidth: 1
        }]
    },
    options: {
        scales: {
            yAxes: [{
                ticks: {
                    beginAtZero: true,
                    callback: function(label){return  ' $' + Math.round(label).toString().replace(/\B(?=(\d{3})+(?!\d))/g, ",");}
                }
            }]
        }
    }
});

// Get an observable stream of generic object layouts when the data changes, and update the chart with latest data
gO$
    .qLayouts()
    .subscribe(function(layout) {
        var data = layout.response.qLayout.qHyperCube.qDataPages[0].qMatrix;
        myChart.data.labels = data.map(function(d) { return  d[0].qText; });
        myChart.data.datasets[0].data = data.map(function(d) { return d[1].qNum; });
        myChart.update();
    });

// Get listbox layouts and update the listbox UI
var ul = document.getElementById("myListbox");
lb$
    .qLayouts()
    .subscribe(function(layout) {
        var data = layout.response.qLayout.qListObject.qDataPages[0].qMatrix;
        ul.innerHTML = data
            .map(function(m) { return "<li class='" +  m[0].qState + "' data-qelemno=" + m[0].qElemNumber + ">" + m[0].qText + "</li>"; })
            .join("");
    });

One more thing: let’s add some CSS back in the style section of our HTML head so that our listbox will have a green-white-grey type styling:

li {
    padding: 5px 10px;
    border: 1px solid rgb(200,200,200);
    margin-top: -1px;
    cursor: pointer;
}

li.S {
    background-color: rgba(75, 192, 192, .5);
}

li.X {
    background-color: rgb(230,230,230);
}
Step 6: Creating Click Events for Selections

The last step, the final countdown! We’ve got our chart and listbox rendering on the page, but the listbox doesn’t have any functionality to make selection calls back to the QIX Engine to filter the data set. There are multiple ways to implement this feature; since this tutorial is centered on reactive programming, we’ll do it the reactive way with RxJS.

First, we need to capture click events on the listbox. This is easy with RxJS’ fromEvent constructor:

var select$ = Rx.Observable.fromEvent(ul,"click")

Next, we want to run a method with the listbox generic object for selecting, based on what was clicked. Therefore, we need to use our click event to create a new list object selection Observable and merge that back in.

The GenericObject class has a method on it called SelectListObjectValues that allows you to give it element numbers of data points you want to select in a listbox. We have a GenericObject Observable for our listbox in the variable `lb$`, so we just need to figure out what element number to feed in for the selection. This can be derived from the click event. We can take the target of the click event and pick out the “data-qelemno” attribute that we established in our rendering code. This value can be used with the `lb$` Observable to create an API call for the appropriate selection. We will use `mergeMap` to merge that selection observable back into our stream:

var select$ = Rx.Observable.fromEvent(ul,"click")
    .mergeMap(function(evt) {
        // Get the element number clicked
        var elemNo = parseInt(evt.target.getAttribute("data-qelemno"));
        // Return an observable of the selection API call
        return lb$.qSelectListObjectValues("/qListObjectDef",[elemNo],true);
    });

However, this won’t actually execute anything. Why? `select$` is an Observable of selection calls; it is not actually executing anything. Observables are by default lazy and do not execute without a subscriber. Let’s add a subscriber to `select$` to execute the selections as they come in. If we wanted to do something with the result from the server, we could do that in this subscriber. For now, we will leave it blank:

// When selection call is created, execute it
select$.subscribe(function(result) {
});

And that’s it, we’re done! I realize that if you haven’t done reactive programming before, a lot of these concepts may seem foreign and overcomplicated, especially this last piece to generate selection calls. Once you understand the underlying concepts of Observables however, the model becomes very powerful for hooking together complex data and UI flows that are scalable and maintainable. Listboxes especially can get complicated once you try to build a full-fledged one with searching, streaming, modal, and multi-select capabilities. Rx really shines in this type of situation involving many interwoven dynamic pieces. That is an example we will write about in a future tutorial.

I hope you enjoyed this tutorial and am looking forwarding to hearing from you in the comments.

– Speros

Leave a Reply

Your email address will not be published. Required fields are marked *