Category Archives: Web

Say What? Live video chat between iOS & WebRTC with Twilio & IBM Watson Cognitive Computing in Real Time

What I’m about to show you might seem like science fiction from the future, but I can assure you it is not. Actually, every piece of this is available for you to use as a service.  Today.

Yesterday Twilio, an IBM partner whose services are available via IBM Bluemix, announced several new SDKs, including live video chat as a service.  This makes live video very easy to integrate into your native mobile or web based applications, and gives you the power to do some very cool things. For example, what if you could add video chat capabilities between your mobile and web clients? Now, what if you could take things a step further, and add IBM Watson cognitive computing capabilities to add real-time transcription and analysis?

Check out this video from yesterday’s Twilio Signal conference keynote, where fellow IBM’ers Damion Heredia and Jeff Sloyer demonstrate exactly this scenario; the integration of the new Twilio video SDK between iOS native and WebRTC client with IBM Watson cognitive computing services providing realtime transcription and sentiment analysis.

If it doesn’t automatically jump to the IBM Bluemix Demo, skip ahead to 2 hours, 15 min, and 20 seconds.

Jeff and Damion did an awesome job showing of both the new video service and the power of IBM Watson. I can also say first-hand that the new Twilio video services are pretty easy to integrate into your own projects (I helped them integrate these services into the native iOS client (physician’s app) shown in the demo)!  You just pull in the SDK, add your app tokens, and instantiate a video chat.   Jeff is pulling the audio stream from the WebRTC client and pushing it up to Watson in real time for the transcription and sentiment analysis services.

Data Management for Apps that Work as Well Offline as They Do Online

Earlier this week I had the privilege of speaking at ApacheCon in Austin, TX on the topic of data management for apps that work as well offline as they do online.  This is an important topic for mobile apps, since, as we all painfully know already, there is never a case when you are always online on your mobile devices.  There always ends up being a time when you need your device/app, but you can’t get online to get the information you need.  Well, this doesn’t always have to be the case. There are strategies you can employ to build apps that work just as well offline as they do online, and the strategy I’d like to highlight today is based upon data management using the IBM Cloudant NoSQL database as a service, which is based upon Apache CouchDB.

Here’s a link to the presentation slides (built using reveal.js) – just use the space bar to advance the presentation slides:

The “couch” in CouchDB is actually an acronym for Cluster of Unreliable Commodity Hardware. At the core of this cluster is the concept of replication, which in the most basic of terms means that  data is shared between multiple sources.  Replication is used to share information between nodes of the cluster, which provides for cluster reliability and fault tolerance.

Replication between Nodes
Replication between Nodes (source)

If you’d like to learn more about replication in Cloudant and CouchDB, you can read more using the links below:

Cloudant is a clustered NoSQL database services that provides an extremely powerful and searchable data store.  It is designed to power the web and mobile apps, and all information is exposed via REST services. Since the IBM Cloudant service is based on CouchDB (and not so coincidentally, IBM is a major contributor to the CouchDB project), replication is also core the the Cloudant service.

With replication, you only have to write your data/changes to a single node in the cluster, and replication takes care of propagating these changes across the cluster.

If you are building apps for the web or mobile, there are options to extend the data replication locally either on the device or in the browser.   This means that you can have a local data store that automatically pushes and/or pulls data from the remote store using replication, and it can be done either via native languages, or using JavaScript.

If you want to have local replication in either a web or hybrid (Cordova/PhoneGap) app, you can use PouchDB.  PouchDB is a local JavaScript database modeled after CouchDB and implements that CouchDB replication API.  So, you can store your data in the browser’s local storage, and those changes will automatically be replicated to the remote Cloudant store.  This works in the browser, in a hybrid (web view) app, or even inside of a Node.js instance. Granted, if you’re in-browser you’ll need to leverage the HTML5 cache to have your app cached locally.

If you are building a native app, don’t worry, you can take advantage of the Cloudant Sync API to leverage the local data store with replication.  This is available for iOS and Android, and implements the CouchDB replication API.

The sample app that I showed in the presentation is a native iOS application based on the GeoPix MobileFirst sample app that I detailed in a previous post.  The difference is that in this case I showed it using the Cloudant Sync API, instead of the MobileFirst data wrapper classes, even though it was pointing at the exact same Cloudant database instance.  You can see a video of the app in action below.

All that you have to do is create a local data store instance, and then use replication to synchronize data between the local store and a remote store.

Replication be either one-way (push or pull), or two-way.  So, any changes between the local and remote stores are replicated across the cluster.  Essentially, the local data store just becomes a node in the cluster.  This provides complete access to the local data, even if there is no network available.  Just save your data to the local store, and replication takes care of the rest.

In the native Objective-C code, you just need to setup the CDTDatastore manager, and initialize your datastore instance.

self.manager = [[CDTDatastoreManager alloc] initWithDirectory:path error:nil];
self.datastore = [self.manager datastoreNamed:@"geopix" error:nil];

Once your datastore is created, you can read/write/modify any data in the local store.  In this case I am creating a generic data object (basically  like a JSON object), and creating a document containing this data.  A document is a record within the data store.

You can add attachments to the document or modify the document as your app needs.  In the code below, I add a JPG atttachment to the document.

//create a document revision
CDTMutableDocumentRevision *rev = [CDTMutableDocumentRevision revision];
rev.body = @{
			 @"sort": [NSNumber numberWithDouble:[now timeIntervalSince1970]],
			 @"clientDate": dateString,
			 @"latitude": [NSNumber numberWithFloat:location.coordinate.latitude],
			 @"longitude": [NSNumber numberWithFloat:location.coordinate.longitude],
			 @"altitude": [NSNumber numberWithFloat:location.altitude],
			 @"course": [NSNumber numberWithFloat:location.course],
			 @"type": @"com.geopix.entry"

//add the jpg attachment
NSData *imageData = UIImageJPEGRepresentation(image, 0.1);
[imageData writeToFile:imagePath atomically:YES];
CDTUnsavedFileAttachment *att1 = [[CDTUnsavedFileAttachment alloc]

rev.attachments = @{ imageName: att1 };

//create a new document from the revision
NSError *error = nil;
CDTDocumentRevision *doc = [self.datastore createDocumentFromRevision:rev error:&error];

if (doc == nil) {
	[logger logErrorWithMessages:@"Error creating document: %@", error.localizedDescription];

[logger logDebugWithMessages:@"Document created ID: %@", doc.docId];

Replication is a fire-and-forget process.  You simply need to initialize the replication process, and any changes to the local data store will be replicated to the remote store automatically when the device is online.

//initialize the replicator factory with the local data store manager
CDTReplicatorFactory *replicatorFactory = 
	[[CDTReplicatorFactory alloc] initWithDatastoreManager:self.manager];


//setup push replication for local->remote changes
NSError *error = nil;
CDTPushReplication *pushReplication = 
	[CDTPushReplication replicationWithSource:self.datastore target:remoteDatabaseURL];

//create the replicator instance
self.replicator = [replicatorFactory oneWay:pushReplication error:&error];
if (!self.replicator) {
	[logger logErrorWithMessages:@"An error occurred: %@", error.localizedDescription];

//assign the replicator delegate
self.replicator.delegate = self;

//auto start replication
error = nil;
if (![self.replicator startWithError:&error]) {
	[logger logErrorWithMessages:@"An error occurred: %@", error.localizedDescription];

By assigning a replicator delegate class (as shown above), your app can monitor and respond to changes in replication state.  For example, you can update status if replication is in progress, complete, or if an error condition was encountered.

- (void)replicatorDidChangeState:(CDTReplicator *)replicator {
    [logger logDebugWithMessages:@"Replicator changed State: %@", [CDTReplicator stringForReplicatorState:replicator.state]];

- (void)replicatorDidChangeProgress:(CDTReplicator *)replicator {
    [logger logDebugWithMessages:@"Replicator progress: %d/%d", replicator.changesProcessed, replicator.changesTotal];
    NSDictionary *userInfo = @{ @"status":[NSString stringWithFormat:@"%d/%d", replicator.changesProcessed, replicator.changesTotal] };
    [[NSNotificationCenter defaultCenter]

- (void)replicatorDidError:(CDTReplicator *)replicator info:(NSError *)info {
    [logger logErrorWithMessages:@"An error occurred: %@", info.localizedDescription];
    self.replicator = nil;
    [[NSNotificationCenter defaultCenter]

- (void)replicatorDidComplete:(CDTReplicator *)replicator {
    [logger logDebugWithMessages:@"Replication completed"];
    self.replicator = nil;
    [[NSNotificationCenter defaultCenter]

If you want to access data from the local store, it is always available within the app, regardless of whether or not the device has an active internet connection.  For example, this method will return all documents within the local data store.

-(NSArray*) getLocalData {
    NSArray *docs = [self.datastore getAllDocuments];
    return docs;

Be sure to review the documentation and/or Cloudant Synch API source code for complete details.

Helpful Links

GeoPix: A sample iOS app powered by IBM MobileFirst for Bluemix

In this post I’d like to show a fairly simple application that I put together which shows off some of the rich capabilities for IBM MobileFirst for Bluemix that you get out of the box – All with an absolute minimal amount of your own developer effort.  Bluemix, of course, being IBM’s platform as a service offering.

GeoPix is a sample application leveraging IBM MobileFirst for Bluemix to capture data and images on a mobile device, persist that data locally (offline), and replicate that data to the cloud. Since it’s built with IBM MobileFirst, we get lots of things out of the box, including operational analytics, user authentication, and much more.

(full source code at the bottom of this post)

Here’s what the application currently does:

  • User can take a picture or select an image from the device
  • App captures geographic location when the image is captured
  • App saves both the image and metadata to a local data store on the device.
  • App uses asynchronous replication to automatically save any data in local store up to the remote store whenever the network is available
  • Oh yeah, can’t forget, the user auth is via Facebook
  • MobileFirst provides all the analytics we need.  Bluemix provides the cloud based server and Cloudant NoSQL data store.
  • All captured data is available on a web based front-end powered by Node.js

Here’s a video of it in action:

… and you can check out the web interface at

(full source code at the bottom of this post)

This is powered by the iOS 8 MobileFirst application boilerplate on Bluemix.  With this application template you can have your backend infrastructure setup within minutes, and it includes:

  • User authentication
  • Usage/operational analytics
  • Cloudant NoSQL DB
  • Simplified Push Notifications
  • Node.js backend

In this sample I’m using everything but the Push Notifications service.  I’m using user authentication, the Cloudant DB (offline/local store and remote/cloud store), and the node.js backend.  You get the operational analytics automatically.

To get started, you just need to create a new iOS 8 mobile application on Bluemix.  See my video series on Getting Started with IBM MobileFirst for Bluemix for a complete walkthrough of creating a new app using MobileFirst for Bluemix, or check out the Getting Started Guide in the official docs.

You need to initialize your app, and make sure you have setup the Facebook identity provider.  You can create your Facebook authentication at  Once the user is authenticated, the client app is fully functional.

The app UI is very simple, basically just two buttons for capturing images (the last captured image shows up in the background):

App's main UI
App’s main UI

There’s also a gallery for viewing local images:

Local gallery view
Local gallery view

Capturing Location

Capturing data is very straightforward.  The geographic location is captured using Apple’s Core Location framework.  We just need to implement the CLLocationManagerDelegate protocol:

- (void)locationManager:(CLLocationManager *)manager
 didUpdateLocations:(NSArray *)locations {

    self.currentLocation = [locations lastObject];
    NSDate* eventDate = self.currentLocation.timestamp;
    NSTimeInterval howRecent = [eventDate timeIntervalSinceNow];
    if (abs(howRecent) < 15.0) {
    // If the event is recent, do something with it.
    locationLabel.text = [NSString stringWithFormat:@" Lat: %+.5f, Lon: %+.5f\n",

Then initialize CLLocationManager using our class as the location manager’s delegate:

if (self.locationManager == nil)
  self.locationManager = [[CLLocationManager alloc] init];
self.locationManager.delegate = self;
self.locationManager.desiredAccuracy = kCLLocationAccuracyBest;
self.locationManager.pausesLocationUpdatesAutomatically = YES;

Capturing Images

Capturing images from the device is also very straightforward.  In the app I leverage Apple’s UIImagePickerController to allow the user to either upload an existing image or capture a new image.  See the presentImagePicker and didFinishPickingMediaWithInfo below. All of this standard practice using Apple’s developer SDK:

- (void) presentImagePicker:(UIImagePickerControllerSourceType) sourceType {
 if ( sourceType == UIImagePickerControllerSourceTypeCamera  && ![UIImagePickerController isSourceTypeAvailable:UIImagePickerControllerSourceTypeCamera]) {
  [logger logErrorWithMessages:@"device has no camera"];
  UIAlertView *myAlertView = [[UIAlertView alloc] initWithTitle:@"Error"
                 message:@"Device has no camera"
             otherButtonTitles: nil];
  [myAlertView show];

 if ( sourceType != UIImagePickerControllerSourceTypeCamera || [UIImagePickerController isSourceTypeAvailable:UIImagePickerControllerSourceTypeCamera] ){
  UIImagePickerController *picker = [[UIImagePickerController alloc] init];
  picker.delegate = self;
  picker.allowsEditing = NO;
  picker.sourceType = sourceType;

  [self presentViewController:picker animated:YES completion:NULL];

- (void)imagePickerController:(UIImagePickerController *)picker didFinishPickingMediaWithInfo:(NSDictionary *)info {

 [logger logDebugWithMessages:@"didFinishPickingMediaWithInfo"];
 UIImage *image = info[UIImagePickerControllerOriginalImage];
 currentImage.image = image;
 [[DataManager sharedInstance] saveImage:image withLocation:self.currentLocation];
 [picker dismissViewControllerAnimated:YES completion:nil];

Persisting Data

If you notice in the didFinishPickingMediaWithInfo method above, there is a call to the DataManager’s saveImage withLocation method. This is where we save data locally and rely on Cloudant’s replication to automatically save data from the local data store up to the Cloudant NoSQL database.  This is powered by the iOS 8 Data service from Bluemix.

The first thing that we will need to do is initialize the local and remote data stores. Below you can see my init method from my DataManager class. In this, you can see the local data store is initialized, then the remote data store is initialized. If either data store already exists, the existing store will be used, otherwise it is created.

-(id) init {
 self = [super init];

 if ( self ) {
  logger = [IMFLogger loggerForName:NSStringFromClass([self class])];
  [logger logDebugWithMessages:@"initializing local datastore 'geopix'..."];

  // initialize an instance of the IMFDataManager
  self.manager = [IMFDataManager sharedInstance];

  NSError *error = nil;
  //create a local data store
  self.datastore = [self.manager localStore:@"geopix" error:&error];

  if (error) {
   [logger logErrorWithMessages:@"Error creating local data store %@",error.description];

  //create a remote data store
  [self.manager remoteStore:@"geopix" completionHandler:^(CDTStore *store, NSError *error) {
   if (error) {
    [logger logErrorWithMessages:@"Error creating remote data store %@",error.description];
   } else {
    [self.manager setCurrentUserPermissions:DB_ACCESS_GROUP_MEMBERS forStoreName:@"geopix" completionHander:^(BOOL success, NSError *error) {
     if (error) {
      [logger logErrorWithMessages:@"Error setting permissions for user with error %@",error.description];

     [self replicate];

  //start replication
  [self replicate];

 return self;

Once the data stores are created, you can see that the replicate method is invoked.  This starts up the replication process to automatically push changesfrom the local data store to the remote data store “in the cloud”.

Therefore, if you’re collecting data when the app is offline, then you have nothing to worry about.  All of the data will be stored locally and pushed up to the cloud whenever you’re back online – all with no additional effort on your part.  When using replication with the Cloudant SDK, you just have to start the replication process and let it do it’s thing… fire and forget.

In my replicate function, I setup CDTPushReplication for pushing changes to the remote data store.  You could also setup two-way replication to automatically pull new changes from the remote store.

-(void) replicate {
 if ( self.replicator == nil ) {
  [logger logDebugWithMessages:@"attempting replication to remote datastore..."];

  __block NSError *replicationError;
  CDTPushReplication *push = [self.manager pushReplicationForStore: @"geopix"];
  self.replicator = [self.manager.replicatorFactory oneWay:push error:&replicationError];
   // Handle error
   [logger logErrorWithMessages:@"An error occurred: %@", replicationError.localizedDescription];

  self.replicator.delegate = self;

  replicationError = nil;
  [logger logDebugWithMessages:@"starting replication"];
  [self.replicator startWithError:&replicationError];
   [logger logErrorWithMessages:@"An error occurred: %@", replicationError.localizedDescription];
   [logger logDebugWithMessages:@"replication start successful"];
 else {
  [logger logDebugWithMessages:@"replicator already running"];

Once we’ve setup the remote and local data stores and setup replication, we now are ready to save the data the we’re capturing within our app.

Next is my saveImage withLocation method.  Here you can see that it creates a new CDTMutableDocumentRevision object (this is a generic object for the Cloudant NoSQL database), and populates it with the location data and timestamp.   It then creates a jpg image from the UIImage (passed in from the UIImagePicker above) and adds the jpg as an attachment to the document revision.  Once the document is created, it is saved to the local data store.   We then let replication take care of persisting this data to the back end.

-(void) saveImage:(UIImage*)image withLocation:(CLLocation*)location {

 [logger logDebugWithMessages:@"saveImage withLocation"];

 //save in background thread
 dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^(void) {

  [logger logDebugWithMessages:@"creating document..."];

  NSDate *now = [NSDate date];
  NSString *dateString = [NSDateFormatter localizedStringFromDate:now

  // Create a document
  CDTMutableDocumentRevision *rev = [CDTMutableDocumentRevision revision];
  rev.body = @{
      @"sort": [NSNumber numberWithDouble:[now timeIntervalSince1970]],
      @"clientDate": dateString,
      @"latitude": [NSNumber numberWithFloat:location.coordinate.latitude],
      @"longitude": [NSNumber numberWithFloat:location.coordinate.longitude],
      @"altitude": [NSNumber numberWithFloat:location.altitude],
      @"course": [NSNumber numberWithFloat:location.course],
      @"type": @"com.geopix.entry"

  [logger logDebugWithMessages:@"creating image attachment..."];

  NSDate *date = [NSDate date];
  NSString *imageName = [NSString stringWithFormat:@"image%f.jpg", [date timeIntervalSince1970]];

  NSString *tempDirectory = NSTemporaryDirectory();
  NSString *imagePath = [tempDirectory stringByAppendingPathComponent:imageName];

  [logger logDebugWithMessages:@"saving image to temporary location: %@", imagePath];

  NSData *imageData = UIImageJPEGRepresentation(image, 0.1);
  [imageData writeToFile:imagePath atomically:YES];

  CDTUnsavedFileAttachment *att1 = [[CDTUnsavedFileAttachment alloc]

  rev.attachments = @{ imageName: att1 };

  [self.datastore save:rev completionHandler:^(id savedObject, NSError *error) {
   if(error) {
    [logger logErrorWithMessages:@"Error creating document: %@", error.localizedDescription];
   [logger logDebugWithMessages:@"Document created: %@", savedObject];

  [self replicate];

If we want to query data from either the remote or local data stores, we can just use the performQuery method on the data store. Below you can see a method for retrieving data for all of the images in the local data store.

-(void) getLocalData:(void (^)(NSArray *results, NSError *error)) completionHandler {

 NSPredicate *queryPredicate = [NSPredicate predicateWithFormat:@"(type = 'com.geopix.entry')"];
 CDTCloudantQuery *query = [[CDTCloudantQuery alloc] initWithPredicate:queryPredicate];

 [self.datastore performQuery:query completionHandler:^(NSArray *results, NSError *error) {

  completionHandler( results, error );

At this point we’ve now captured an image, captured the geographic location, saved that data in our local offline store, and then use replication to save that data up to the cloud whenever it is available.


We did all of this without writing a single line of server-side logic.   Since this is built on top of MobileFirst for Bluemix, all the backend infrastructure is setup for us, and we get operational analytics to monitor everything that is happening.

With the operational analytics we get:

  • App usage
  • Active Devices
  • Network Usage
  • Authentications
  • Data Storage
  • Device Logs (yes, complete debug/crash logs from devices out in the field)
  • Push Notification Usage

Sharing on the web

Up until this point we haven’t had to write any back-end code. However the mobile app boilerplate on Bluemix comes with a Node.js server.  We might as well take advantage of it.

I exposed the exact same data captured within the app on the Node.js service, which you can see at

Web UI
Web UI

The Node.js back end comes preconfigured to leverage the express.js framework for building web applications.  I added the jade template engine and Leaflet for web-mapping, and was able to crank this out ridiculously quickly.

The first thing we need to do is make sure  we have our configuration variables for accessing the Cloudant service from our node app.  These are environment vars that you get automatcilly if you’re running on Bluemix, but you need to set these for your local dev environment:

var credentials = {};

if (process.env.hasOwnProperty("VCAP_SERVICES")) {
 // Running on Bluemix. Parse out the port and host that we've been assigned.
 var env = JSON.parse(process.env.VCAP_SERVICES);
 var host = process.env.VCAP_APP_HOST;
 var port = process.env.VCAP_APP_PORT;

 credentials = env['cloudantNoSQLDB'][0].credentials;
else {

 //for local node.js server instance
 credentials.username = "cloudant username here";
 credentials.password = "cloudant password here";
 credentials.url = "cloudant url here";

Next we’ll add our URL/content mappings:

app.get('/', function(req, res){
  prepareData(res, 'map');

app.get('/list', function(req, res){
  prepareData(res, 'list');

Next you’ll se the logic for querying the Cloudant data store and preparing the data for our UI templates. You can customize this however you want – caching for performance, refactoring for abstraction, or whatever you want. All interactions with Cloudant are powered by the Cloudant Node.js Client

var prepareData = function(res, template) {
 var results = [];

 //create the index if it doesn't already exist
 var sort_index = {name:'sort', type:'json', index:{fields:['sort']}};
 geopix.index(sort_index, function(er, response) {
  if (er) {
   throw er;

  //perform the search
  //we're just pulling back all
  //data captured ("sort" will be numeric)
  var selector = {sort:{"$gt":0}};
  geopix.find({selector:selector, sort:["sort"]}, function(er, result) {
   if (er) {
    throw er;

   //prepare data for template
   for (var x=0; x<; x++) {
    var obj =[x];

    for (var key in obj._attachments) {
     obj.image = credentials.url + "/" + database + "/" + obj._id +"/" + key;

    results.push( obj );
   res.render(template, { results:results});

After the prepareData method has prepared data for formatting in the UI, the template is rendered by invoking Jade’s render method:

res.render(template, { results:results});

This will render whichever template was passed in – I have two: map.jade (the map template) and list.jade (the list template). You can check out the list template below, and see it in action here:

 title GeoPix - powered by Bluemix
 link(href='//' rel='stylesheet')
 link(href='/public/css/index.css' rel='stylesheet')
 meta(name="viewport" content="width=device-width, initial-scale=1")
  h1 GeoPix - Powered by Bluemix
   a(href='/') Map
   a(href='/list') List
  each val, index in results
    div(class="panel panel-default")
      h3= val.clientDate
      p= 'latitude: ' + val.latitude + ", longitude:" + val.longitude + ", altitude:" + val.altitude

In the map view I used the Leaflet map engine and Open Street Map data, along with the Leaflet Marker Cluster plugin for displaying clustered results.

Source Code

You can check out the web interface live at:  If you want to setup the environment on your own, you can grab the complete source code at:

Helpful Links

Ready to start building your own apps on IBM Bluemix?  Just head over to and get a free developer trial today!

UX & Mobile Apps Leveraging the IBM MobileFirst Platform

When you are developing a mobile app (or website, or mobile web, or TV app, etc..) you should always ask yourself “What kind of an impact does this have on the end user?” It doesn’t matter whether you are creating enterprise apps or games, or anything in between. Every development decision that you make should be weighed upon it’s impact to the overall impact it has on the end user. Simply put: if your app sucks, nobody is going to want to use it.

When building mobile apps using IBM MobileFirst you have two options for a user interface layer; you can write a native app, or you can write a hybrid app using HTML, CSS, & JavaScript.

Native VS HTML Hybrid

So, what kind of an impact does the addition of IBM MobileFirst have on the app?

NONE, granted the app’s UX can vary depending upon whether you are developing a native app or a hybrid app.

IBM MobileFirst Platform Foundation is a platform that consists of a server tier and client-side SDK. If you are developing a native app, the SDK/API provides access to MobileFirst platform features, like user authentication, app version management, data access through adapters, encrypted storage, unified push notification, remote log collection, and more. If you are developing a hybrid app, the app’s UX must be developed complete inside of the web view container. The MobileFirst Foundation SDK provides additional functionality just like mentioned above for the native SDK, plus a few classes that enable native dialogs and a few native UI elements, but for the most part, there is very, very minimal impact on the user’s experience.

When building any kind of mobile app, regardless of whether it is native or hybrid, you need to pay attention to what the user experiences. Are you following human interaction or design guidelines for the platform? Are you forcing your user to go through unnecessary or redundant steps? Are you making forms more complex than they need to be? Are you forcing multiple taps for a simple interaction? Can things be simplified?

The IBM MobileFirst Platform does not add any additional overhead for UX processes.

The IBM MobileFirst platform can be used to develop native apps on iOS, Android, Windows Phone, or Java ME platforms. Follow native coding conventions and UX guidelines for each individual platform. Make sure you follow these guidelines, otherwise your app may feel alien within the ecosystem, or may be rejected from app store approval altogether.

What should you think about when building a hybrid app leveraging IBM MobileFirst?

First, it depends on your hybrid approach. If you are using the for Xamarin hybrid approach for MobileFirst apps, then your considerations will be much like writing a native app.

If you are using the HTML/CSS/JavaScript approach (leveraging the Apache Cordova container), then you really want to focus on the user’s experience inside of the HTML container. You want to make sure the UI feels like “an app”, not like “a web page”. There are many client-side frameworks that help address this need. Feel free to use any of them, or roll your own – just keep the UX/human interaction guidelines in mind.

(this is by no means an exhaustive list, there are more out there)

Make sure you’re building for touch interactivity, addressing animations correctly to minimize browser reflow operations, and more… Read this post on Dzone that I wrote last year for a lot more detail on performance considerations for hybrid or mobile web apps.

If you are using any of these (or none of these), also don’t forget an architectural pattern/framework for helping keep your code organized and consistent. Read this post on hybrid/web app architectural patterns and considerations for even more detail.

For both native and hybrid approaches, you also want to consider impacts of perceived performance vs actual raw processing  power. These techniques deliver the appearance that the app is fast and responsive, instead of sluggish or locked while waiting to perform an action. Perceived performance improvements can be achieved simply by providing instant feedback, performing animations during an asynchronous request, or preemptive tasking. Don’t miss this post, where I go into perceived performance in mobile apps in great detail.


IBM Watson, Cognitive Computing & Speech APIs

IBM Watson is a cognitive computing platform that you can use to add intelligence and natural language analysis to your own applications.  Watson employs natural language processing, hypothesis generation, and dynamic learning to deliver solutions for natural language question and answer services, sentiment analysis, relationship extration, concept expansion, and language/translation services. ..and, it is available for you to check out with IBM Bluemix cloud services.

Watson won Jeopardy, tackles genetics,  creates recipes, and so much more.  It is breaking new ground on a daily basis.

The IBM Watson™ Question Answer (QA) service provides an API that give you the power of the IBM Watson cognitive computing system. With this service, you can connect to Watson, pose questions in natural language, and receive responses that you can use within your application.

In this post, I’ve hooked the Watson QA node.js starter project to the Web Speech API speech recognition and speech synthesis APIs. Using these APIs, you can now have a conversation with Watson. Ask any question about healthcare, and see what watson has to say. Check out the video below to see it in action.

You can check out a live demo at:

Just click on the microphone button, allow access to the system mic, and start talking.  Just a warning, lots of background noise might interfere with the API’s ability to recognize & generate a meaningful transcript.

This demo only supports Google Chrome only at the time of writing. You can check out where Web Speech is supported at

You can check out the full source code for this sample on IBM Jazz Hub (git):

I basically just took the Watson QA Sample Application for Node.js and started playing around with it to see what I could do…

This demo uses the Watson For Healthcare data set, which contains information from, the CDC, National Hear Lung, and Blood Institute, National Institute of Arthritis and Musculoskeletal and Skin Diseases, National Institute of Diabetes and Digestive and Kidney Diseases, National Institute of Neurological Disorders and Stroke, and  Just know that this is a beta service/data set – implementing Watson for your own enterprise solutions requires system training and algorithm development for Watson to be able to understand your data.

Using Watson with this dataset, you can ask conditional questions, like:

  • What is X?
  • What causes X?
  • What is the treatment for X?
  • What are the symptoms of X?
  • Am I at risk of X?

Procedure questions, like:

  • What should I expect before X?
  • What should I expect after X?

General health auestions, like:

  • What are the benefits of taking aspirin daily?
  • Why do I need to get shots?
  • How do I know if I have food poisoning?

Or, action-related questions, like:

  • How can I quit smoking?
  • What should I do if my child is obese?
  • What can I do to get more calcium?

Watson services are exposed through a RESTful API, and can easily be integrated into an existing application.  For example, here’s a snippet demonstrating how you can consume the Watson QA service inside of a Node.js app:

var parts = url.parse(service_url +'/v1/question/healthcare');
var options = {
host: parts.hostname,
port: parts.port,
path: parts.pathname,
method: 'POST',
headers: {
  'Content-Type'  :'application/json',
  'X-synctimeout' : '30',
  'Authorization' :  auth

// Create a request to POST to Watson
var watson_req = https.request(options, function(result) {
  var response_string = '';

  result.on('data', function(chunk) {
    response_string += chunk;

  result.on('end', function() {
    var answers = JSON.parse(response_string)[0];
    var response = extend({ 'answers': answers },req.body);
    return res.render('response', response);

Hooking into the Web Speech API is just as easy (assuming you’re using a browser that implements the Web Speech API – I built this demo using Chrome on OS X). On the client side, you just need need to create a SpeechRecognition instance, and add the appropriate event handlers.

var recognition = new webkitSpeechRecognition();
 recognition.continuous = true;
 recognition.interimResults = true;

 recognition.onstart = function() { ... }
 recognition.onresult = function(event) {

   var result = event.results[event.results.length-1];
   var transcript = result[0].transcript;

   // then do something with the transcript
   search( transcript );
 recognition.onerror = function(event) { ... }
 recognition.onend = function() { ... }

To make your app talk back to you (synthesize speech), you just need to create a new SpeechSynthesisUtterance object, and pass it into the window.speechSynthesis.speak() function. You can add event listeners to handle speech events, if needed.

var msg = new SpeechSynthesisUtterance( tokens[i] ); 

msg.onstart = function (event) {
    console.log('started speaking');

msg.onend = function (event) {
    console.log('stopped speaking');


Check out these articles on for more detail on Speech Recognition and Speech Synthesis.

Here are those links again…

You can get started with Watson services for Bluemix at