Category Archives: HTML5

Data Management for Apps that Work as Well Offline as They Do Online

Earlier this week I had the privilege of speaking at ApacheCon in Austin, TX on the topic of data management for apps that work as well offline as they do online.  This is an important topic for mobile apps, since, as we all painfully know already, there is never a case when you are always online on your mobile devices.  There always ends up being a time when you need your device/app, but you can’t get online to get the information you need.  Well, this doesn’t always have to be the case. There are strategies you can employ to build apps that work just as well offline as they do online, and the strategy I’d like to highlight today is based upon data management using the IBM Cloudant NoSQL database as a service, which is based upon Apache CouchDB.

Here’s a link to the presentation slides (built using reveal.js) – just use the space bar to advance the presentation slides:

The “couch” in CouchDB is actually an acronym for Cluster of Unreliable Commodity Hardware. At the core of this cluster is the concept of replication, which in the most basic of terms means that  data is shared between multiple sources.  Replication is used to share information between nodes of the cluster, which provides for cluster reliability and fault tolerance.

Replication between Nodes
Replication between Nodes (source)

If you’d like to learn more about replication in Cloudant and CouchDB, you can read more using the links below:

Cloudant is a clustered NoSQL database services that provides an extremely powerful and searchable data store.  It is designed to power the web and mobile apps, and all information is exposed via REST services. Since the IBM Cloudant service is based on CouchDB (and not so coincidentally, IBM is a major contributor to the CouchDB project), replication is also core the the Cloudant service.

With replication, you only have to write your data/changes to a single node in the cluster, and replication takes care of propagating these changes across the cluster.

If you are building apps for the web or mobile, there are options to extend the data replication locally either on the device or in the browser.   This means that you can have a local data store that automatically pushes and/or pulls data from the remote store using replication, and it can be done either via native languages, or using JavaScript.

If you want to have local replication in either a web or hybrid (Cordova/PhoneGap) app, you can use PouchDB.  PouchDB is a local JavaScript database modeled after CouchDB and implements that CouchDB replication API.  So, you can store your data in the browser’s local storage, and those changes will automatically be replicated to the remote Cloudant store.  This works in the browser, in a hybrid (web view) app, or even inside of a Node.js instance. Granted, if you’re in-browser you’ll need to leverage the HTML5 cache to have your app cached locally.

If you are building a native app, don’t worry, you can take advantage of the Cloudant Sync API to leverage the local data store with replication.  This is available for iOS and Android, and implements the CouchDB replication API.

The sample app that I showed in the presentation is a native iOS application based on the GeoPix MobileFirst sample app that I detailed in a previous post.  The difference is that in this case I showed it using the Cloudant Sync API, instead of the MobileFirst data wrapper classes, even though it was pointing at the exact same Cloudant database instance.  You can see a video of the app in action below.

All that you have to do is create a local data store instance, and then use replication to synchronize data between the local store and a remote store.

Replication be either one-way (push or pull), or two-way.  So, any changes between the local and remote stores are replicated across the cluster.  Essentially, the local data store just becomes a node in the cluster.  This provides complete access to the local data, even if there is no network available.  Just save your data to the local store, and replication takes care of the rest.

In the native Objective-C code, you just need to setup the CDTDatastore manager, and initialize your datastore instance.

self.manager = [[CDTDatastoreManager alloc] initWithDirectory:path error:nil];
self.datastore = [self.manager datastoreNamed:@"geopix" error:nil];

Once your datastore is created, you can read/write/modify any data in the local store.  In this case I am creating a generic data object (basically  like a JSON object), and creating a document containing this data.  A document is a record within the data store.

You can add attachments to the document or modify the document as your app needs.  In the code below, I add a JPG atttachment to the document.

//create a document revision
CDTMutableDocumentRevision *rev = [CDTMutableDocumentRevision revision];
rev.body = @{
			 @"sort": [NSNumber numberWithDouble:[now timeIntervalSince1970]],
			 @"clientDate": dateString,
			 @"latitude": [NSNumber numberWithFloat:location.coordinate.latitude],
			 @"longitude": [NSNumber numberWithFloat:location.coordinate.longitude],
			 @"altitude": [NSNumber numberWithFloat:location.altitude],
			 @"course": [NSNumber numberWithFloat:location.course],
			 @"type": @"com.geopix.entry"
			 };

//add the jpg attachment
NSData *imageData = UIImageJPEGRepresentation(image, 0.1);
[imageData writeToFile:imagePath atomically:YES];
 
CDTUnsavedFileAttachment *att1 = [[CDTUnsavedFileAttachment alloc]
								  initWithPath:imagePath
								  name:imageName
								  type:@"image/jpeg"];

rev.attachments = @{ imageName: att1 };

//create a new document from the revision
NSError *error = nil;
CDTDocumentRevision *doc = [self.datastore createDocumentFromRevision:rev error:&error];

if (doc == nil) {
	[logger logErrorWithMessages:@"Error creating document: %@", error.localizedDescription];
}

[logger logDebugWithMessages:@"Document created ID: %@", doc.docId];

Replication is a fire-and-forget process.  You simply need to initialize the replication process, and any changes to the local data store will be replicated to the remote store automatically when the device is online.

//initialize the replicator factory with the local data store manager
CDTReplicatorFactory *replicatorFactory = 
	[[CDTReplicatorFactory alloc] initWithDatastoreManager:self.manager];

NSURL *remoteDatabaseURL = [NSURL URLWithString:REMOTE_DATABASE_URL];

//setup push replication for local->remote changes
NSError *error = nil;
CDTPushReplication *pushReplication = 
	[CDTPushReplication replicationWithSource:self.datastore target:remoteDatabaseURL];

//create the replicator instance
self.replicator = [replicatorFactory oneWay:pushReplication error:&error];
if (!self.replicator) {
	[logger logErrorWithMessages:@"An error occurred: %@", error.localizedDescription];
}

//assign the replicator delegate
self.replicator.delegate = self;

//auto start replication
error = nil;
if (![self.replicator startWithError:&error]) {
	[logger logErrorWithMessages:@"An error occurred: %@", error.localizedDescription];
}

By assigning a replicator delegate class (as shown above), your app can monitor and respond to changes in replication state.  For example, you can update status if replication is in progress, complete, or if an error condition was encountered.

- (void)replicatorDidChangeState:(CDTReplicator *)replicator {
    [logger logDebugWithMessages:@"Replicator changed State: %@", [CDTReplicator stringForReplicatorState:replicator.state]];
}

- (void)replicatorDidChangeProgress:(CDTReplicator *)replicator {
    [logger logDebugWithMessages:@"Replicator progress: %d/%d", replicator.changesProcessed, replicator.changesTotal];
    
    NSDictionary *userInfo = @{ @"status":[NSString stringWithFormat:@"%d/%d", replicator.changesProcessed, replicator.changesTotal] };
    
    [[NSNotificationCenter defaultCenter]
     postNotificationName:@"ReplicationStatus"
     object:self
     userInfo:userInfo];
}

- (void)replicatorDidError:(CDTReplicator *)replicator info:(NSError *)info {
    [logger logErrorWithMessages:@"An error occurred: %@", info.localizedDescription];
    self.replicator = nil;
    
    [[NSNotificationCenter defaultCenter]
     postNotificationName:@"ReplicationError"
     object:self];
}

- (void)replicatorDidComplete:(CDTReplicator *)replicator {
    [logger logDebugWithMessages:@"Replication completed"];
    self.replicator = nil;
    
    [[NSNotificationCenter defaultCenter]
     postNotificationName:@"ReplicationComplete"
     object:self];
}

If you want to access data from the local store, it is always available within the app, regardless of whether or not the device has an active internet connection.  For example, this method will return all documents within the local data store.

-(NSArray*) getLocalData {
    
    NSArray *docs = [self.datastore getAllDocuments];
    return docs;
}

Be sure to review the documentation and/or Cloudant Synch API source code for complete details.

Helpful Links

IBM Watson QA + Speech Recognition + Speech Synthesis = A Conversation With Your Computer

Back in November I released a demo application here on my blog showing the IBM Watson QA Service for cognitive/natural language computing connected to the Web Speech API in Google Chrome to have real conversational interaction with a web application.  It’s a nice demo, but it always drove me nuts that it only worked in Chrome.  Last month the IBM Watson team released 5 new services, and guess what… Speech Recognition and Speech Synthesis are included!

These two services enable you to quickly add Text-To-Speech or Speech-To-Text capability to any application.  What’s a better way to show them off than by updating my existing app to leverage the new speech services?

So here it is: watsonhealthqa.mybluemix.net!

By leveraging the Watson services it can now run in any browser that supports getUserMedia (for speech recognition) and HTML5 <Audio> (for speech playback).

(Full source code available at the bottom of this post)

You can check out a video of it in action below:

If your browser doesn’t support the getUserMedia API or HTML5 <Audio>, then your mileage may vary.  You can check where these features are supported with these links: <Audio>getUserMedia

Warning: This is targeting desktop browsers – HTML5 Audio is a mess on mobile devices due to limited codec support and immature APIs.

So how does this all work?

Just like the QA service, the new Text To Speech and Speech To Text services are now available in IBM Bluemix, so you can create a new application that leverages any of these services, or you can add them to any existing application.

I simply added the Text To Speech and Speech To Text services to my existing Healthcare QA application that runs on Bluemix:

bluemix-dashboard
IBM Bluemix Dashboard

 

These services are available via a REST API. Once you’ve added them to your application, you can consume them easily within any of your applications.

I updated the code from my previous example in 2 ways: 1) take advantage of the Watson Node.js Wrapper that makes interacting with Watson a lot easier and 2) to take advantage of these new services services.

Watson Node.js Wrapper

Using the Watson Node.js Wrapper, you can now easily instantiate Watson services in a single line of code.  For example:

var watson = require('watson-developer-cloud');
var question_and_answer_healthcare = watson.question_and_answer(QA_CREDENTIALS);
var speechToText = watson.speech_to_text(STT_CREDENTIALS);

The credentials come from your environment configuration, then you just create instances of whichever services that you want to consume.

QA Service

The code for consuming a service is now much simpler than the previous version.  When we want to submit a question to the Watson QA service, you can now simply call the “ask” method on the QA service instance.

Below is my server-side code from app.js that accepts a POST submission from the browser, delegates the question to Watson, and takes the result and renders HTML using a Jade template. See the Getting Started Guide for the Watson QA Service to learn more about the wrappers for Node or Java.

// Handle the form POST containing the question
app.post('/ask', function(req, res){

    // delegate to Watson
    question_and_answer_healthcare.ask({ text: req.body.questionText}, function (err, response) {
        if (err)
            console.log('error:', err);
        else {
          var response = extend({ 'answers': response[0] },req.body);

          // render the template to HTML and send it to the browser
          return res.render('response', response);
        }
    });
});

Compare this to the previous version, and you’ll quickly see that it is much simpler.

Speech Synthesis

At this point, we already have a functional service that can take natural language text, submit it to Watson,  and return a search result as text.  The next logical step for me was to add speech synthesis using the Watson Text To Speech Service (TTS).  Again, the Watson Node Wrapper and Watson’s REST services make this task very simple.  On the client side you just need to set the src of an <audio> instance to the URL for the TTS service:

<audio controls="" autoplay="" src="/synthesize?text=The text that should generate the audio goes here"></audio>

On the server you just need to synthesize the audio from the data in the URL query string.  Here’s an example how to invoke the text to speech service directly from the Watson TTS sample app:

var textToSpeech = new watson.text_to_speech(credentials);

// handle get requests
app.get('/synthesize', function(req, res) {

  // make the request to Watson to synthesize the audio file from the query text
  var transcript = textToSpeech.synthesize(req.query);

  // set content-disposition header if downloading the
  // file instead of playing directly in the browser
  transcript.on('response', function(response) {
    console.log(response.headers);
    if (req.query.download) {
      response.headers['content-disposition'] = 'attachment; filename=transcript.ogg';
    }
  });

  // pipe results back to the browser as they come in from Watson
  transcript.pipe(res);
});

The Watson TTS service supports .ogg and .wav file formats.  I modified this sample is setup only with .ogg files.  On the client side, these are played using the HTML5 <audio> tag.

Speech Recognition

Now that we’re able to process natural language and generate speech, that last part of the solution is to recognize spoken input and turn it into text.  The Watson Speech To Text (STT) service handles this for us.  Just like the TTS service, the Speech To Text service also has a sample app, complete with source code to help you get started.

This service uses the browser’s getUserMedia (streaming) API with socket.io on Node to stream the data back to the server with minimal latency. The best part is that you don’t have to setup any of this on your own. Just leverage the code from the sample app. Note: the getUserMedia API isn’t supported everywhere, so be advised.

On the client side you just need to create an instance of the SpeechRecognizer class in JavaScript and handle the result:

var recognizer = new SpeechRecognizer({
  ws: '',
  model: 'WatsonModel'
});

recognizer.onresult = function(data) {

    //get the transcript from the service result data
    var result = data.results[data.results.length-1];
    var transcript = result.alternatives[0].transcript;

    // do something with the transcript
    search( transcript, result.final );
}

On the server, you need to create an instance of the Watson Speech To Text service, and setup handlers for the post request to receive the audio stream.

// create an instance of the speech to text service
var speechToText = watson.speech_to_text(STT_CREDENTIALS);

// Handle audio stream processing for speech recognition
app.post('/', function(req, res) {
    var audio;

    if(req.body.url && req.body.url.indexOf('audio/') === 0) {
        // sample audio stream
        audio = fs.createReadStream(__dirname + '/../public/' + req.body.url);
    } else {
        // malformed url
        return res.status(500).json({ error: 'Malformed URL' });
    }

    // use Watson to generate a text transcript from the audio stream
    speechToText.recognize({audio: audio, content_type: 'audio/l16; rate=44100'}, function(err, transcript) {
        if (err)
            return res.status(500).json({ error: err });
        else
            return res.json(transcript);
    });
});

Source Code

You can interact with a live instance of this application at watsonhealthqa.mybluemix.net, and complete client and server side code is available at github.com/triceam/IBMWatson-QA-Speech.

Just setup your Bluemix app, clone the sample code, run NPM install and deploy your app to Bluemix using the Cloud Foundry CLI.

Helpful Links

GeoPix: A sample iOS app powered by IBM MobileFirst for Bluemix

In this post I’d like to show a fairly simple application that I put together which shows off some of the rich capabilities for IBM MobileFirst for Bluemix that you get out of the box – All with an absolute minimal amount of your own developer effort.  Bluemix, of course, being IBM’s platform as a service offering.

GeoPix is a sample application leveraging IBM MobileFirst for Bluemix to capture data and images on a mobile device, persist that data locally (offline), and replicate that data to the cloud. Since it’s built with IBM MobileFirst, we get lots of things out of the box, including operational analytics, user authentication, and much more.

(full source code at the bottom of this post)

Here’s what the application currently does:

  • User can take a picture or select an image from the device
  • App captures geographic location when the image is captured
  • App saves both the image and metadata to a local data store on the device.
  • App uses asynchronous replication to automatically save any data in local store up to the remote store whenever the network is available
  • Oh yeah, can’t forget, the user auth is via Facebook
  • MobileFirst provides all the analytics we need.  Bluemix provides the cloud based server and Cloudant NoSQL data store.
  • All captured data is available on a web based front-end powered by Node.js

Here’s a video of it in action:

… and you can check out the web interface at geopix.mybluemix.net.

(full source code at the bottom of this post)

This is powered by the iOS 8 MobileFirst application boilerplate on Bluemix.  With this application template you can have your backend infrastructure setup within minutes, and it includes:

  • User authentication
  • Usage/operational analytics
  • Cloudant NoSQL DB
  • Simplified Push Notifications
  • Node.js backend

In this sample I’m using everything but the Push Notifications service.  I’m using user authentication, the Cloudant DB (offline/local store and remote/cloud store), and the node.js backend.  You get the operational analytics automatically.

To get started, you just need to create a new iOS 8 mobile application on Bluemix.  See my video series on Getting Started with IBM MobileFirst for Bluemix for a complete walkthrough of creating a new app using MobileFirst for Bluemix, or check out the Getting Started Guide in the official docs.

You need to initialize your app, and make sure you have setup the Facebook identity provider.  You can create your Facebook authentication at https://developers.facebook.com/.  Once the user is authenticated, the client app is fully functional.

The app UI is very simple, basically just two buttons for capturing images (the last captured image shows up in the background):

App's main UI
App’s main UI

There’s also a gallery for viewing local images:

Local gallery view
Local gallery view

Capturing Location

Capturing data is very straightforward.  The geographic location is captured using Apple’s Core Location framework.  We just need to implement the CLLocationManagerDelegate protocol:

- (void)locationManager:(CLLocationManager *)manager
 didUpdateLocations:(NSArray *)locations {

    self.currentLocation = [locations lastObject];
    NSDate* eventDate = self.currentLocation.timestamp;
    NSTimeInterval howRecent = [eventDate timeIntervalSinceNow];
    if (abs(howRecent) < 15.0) {
    // If the event is recent, do something with it.
    locationLabel.text = [NSString stringWithFormat:@" Lat: %+.5f, Lon: %+.5f\n",
      self.currentLocation.coordinate.latitude,
      self.currentLocation.coordinate.longitude];
    }
}

Then initialize CLLocationManager using our class as the location manager’s delegate:

if (self.locationManager == nil)
  self.locationManager = [[CLLocationManager alloc] init];
self.locationManager.delegate = self;
self.locationManager.desiredAccuracy = kCLLocationAccuracyBest;
self.locationManager.pausesLocationUpdatesAutomatically = YES;

Capturing Images

Capturing images from the device is also very straightforward.  In the app I leverage Apple’s UIImagePickerController to allow the user to either upload an existing image or capture a new image.  See the presentImagePicker and didFinishPickingMediaWithInfo below. All of this standard practice using Apple’s developer SDK:

- (void) presentImagePicker:(UIImagePickerControllerSourceType) sourceType {
 if ( sourceType == UIImagePickerControllerSourceTypeCamera  && ![UIImagePickerController isSourceTypeAvailable:UIImagePickerControllerSourceTypeCamera]) {
  [logger logErrorWithMessages:@"device has no camera"];
  UIAlertView *myAlertView = [[UIAlertView alloc] initWithTitle:@"Error"
                 message:@"Device has no camera"
                delegate:nil
             cancelButtonTitle:@"OK"
             otherButtonTitles: nil];
  [myAlertView show];
 };

 if ( sourceType != UIImagePickerControllerSourceTypeCamera || [UIImagePickerController isSourceTypeAvailable:UIImagePickerControllerSourceTypeCamera] ){
  UIImagePickerController *picker = [[UIImagePickerController alloc] init];
  picker.delegate = self;
  picker.allowsEditing = NO;
  picker.sourceType = sourceType;

  [self presentViewController:picker animated:YES completion:NULL];
 }
}

- (void)imagePickerController:(UIImagePickerController *)picker didFinishPickingMediaWithInfo:(NSDictionary *)info {

 [logger logDebugWithMessages:@"didFinishPickingMediaWithInfo"];
 UIImage *image = info[UIImagePickerControllerOriginalImage];
 currentImage.image = image;
 [[DataManager sharedInstance] saveImage:image withLocation:self.currentLocation];
 [picker dismissViewControllerAnimated:YES completion:nil];
}

Persisting Data

If you notice in the didFinishPickingMediaWithInfo method above, there is a call to the DataManager’s saveImage withLocation method. This is where we save data locally and rely on Cloudant’s replication to automatically save data from the local data store up to the Cloudant NoSQL database.  This is powered by the iOS 8 Data service from Bluemix.

The first thing that we will need to do is initialize the local and remote data stores. Below you can see my init method from my DataManager class. In this, you can see the local data store is initialized, then the remote data store is initialized. If either data store already exists, the existing store will be used, otherwise it is created.

-(id) init {
 self = [super init];

 if ( self ) {
  logger = [IMFLogger loggerForName:NSStringFromClass([self class])];
  [logger logDebugWithMessages:@"initializing local datastore 'geopix'..."];

  // initialize an instance of the IMFDataManager
  self.manager = [IMFDataManager sharedInstance];

  NSError *error = nil;
  //create a local data store
  self.datastore = [self.manager localStore:@"geopix" error:&error];

  if (error) {
   [logger logErrorWithMessages:@"Error creating local data store %@",error.description];
  }

  //create a remote data store
  [self.manager remoteStore:@"geopix" completionHandler:^(CDTStore *store, NSError *error) {
   if (error) {
    [logger logErrorWithMessages:@"Error creating remote data store %@",error.description];
   } else {
    [self.manager setCurrentUserPermissions:DB_ACCESS_GROUP_MEMBERS forStoreName:@"geopix" completionHander:^(BOOL success, NSError *error) {
     if (error) {
      [logger logErrorWithMessages:@"Error setting permissions for user with error %@",error.description];
     }

     [self replicate];
    }];
   }
  }];

  //start replication
  [self replicate];
 }

 return self;
} 

Once the data stores are created, you can see that the replicate method is invoked.  This starts up the replication process to automatically push changesfrom the local data store to the remote data store “in the cloud”.

Therefore, if you’re collecting data when the app is offline, then you have nothing to worry about.  All of the data will be stored locally and pushed up to the cloud whenever you’re back online – all with no additional effort on your part.  When using replication with the Cloudant SDK, you just have to start the replication process and let it do it’s thing… fire and forget.

In my replicate function, I setup CDTPushReplication for pushing changes to the remote data store.  You could also setup two-way replication to automatically pull new changes from the remote store.

-(void) replicate {
 if ( self.replicator == nil ) {
  [logger logDebugWithMessages:@"attempting replication to remote datastore..."];

  __block NSError *replicationError;
  CDTPushReplication *push = [self.manager pushReplicationForStore: @"geopix"];
  self.replicator = [self.manager.replicatorFactory oneWay:push error:&replicationError];
  if(replicationError){
   // Handle error
   [logger logErrorWithMessages:@"An error occurred: %@", replicationError.localizedDescription];
  }

  self.replicator.delegate = self;

  replicationError = nil;
  [logger logDebugWithMessages:@"starting replication"];
  [self.replicator startWithError:&replicationError];
  if(replicationError){
   [logger logErrorWithMessages:@"An error occurred: %@", replicationError.localizedDescription];
  }else{
   [logger logDebugWithMessages:@"replication start successful"];
  }
 }
 else {
  [logger logDebugWithMessages:@"replicator already running"];
 }
}

Once we’ve setup the remote and local data stores and setup replication, we now are ready to save the data the we’re capturing within our app.

Next is my saveImage withLocation method.  Here you can see that it creates a new CDTMutableDocumentRevision object (this is a generic object for the Cloudant NoSQL database), and populates it with the location data and timestamp.   It then creates a jpg image from the UIImage (passed in from the UIImagePicker above) and adds the jpg as an attachment to the document revision.  Once the document is created, it is saved to the local data store.   We then let replication take care of persisting this data to the back end.

-(void) saveImage:(UIImage*)image withLocation:(CLLocation*)location {

 [logger logDebugWithMessages:@"saveImage withLocation"];

 //save in background thread
 dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^(void) {

  [logger logDebugWithMessages:@"creating document..."];

  NSDate *now = [NSDate date];
  NSString *dateString = [NSDateFormatter localizedStringFromDate:now
                 dateStyle:NSDateFormatterShortStyle
                 timeStyle:NSDateFormatterFullStyle];

  // Create a document
  CDTMutableDocumentRevision *rev = [CDTMutableDocumentRevision revision];
  rev.body = @{
      @"sort": [NSNumber numberWithDouble:[now timeIntervalSince1970]],
      @"clientDate": dateString,
      @"latitude": [NSNumber numberWithFloat:location.coordinate.latitude],
      @"longitude": [NSNumber numberWithFloat:location.coordinate.longitude],
      @"altitude": [NSNumber numberWithFloat:location.altitude],
      @"course": [NSNumber numberWithFloat:location.course],
      @"type": @"com.geopix.entry"
      };

  [logger logDebugWithMessages:@"creating image attachment..."];

  NSDate *date = [NSDate date];
  NSString *imageName = [NSString stringWithFormat:@"image%f.jpg", [date timeIntervalSince1970]];

  NSString *tempDirectory = NSTemporaryDirectory();
  NSString *imagePath = [tempDirectory stringByAppendingPathComponent:imageName];

  [logger logDebugWithMessages:@"saving image to temporary location: %@", imagePath];

  NSData *imageData = UIImageJPEGRepresentation(image, 0.1);
  [imageData writeToFile:imagePath atomically:YES];

  CDTUnsavedFileAttachment *att1 = [[CDTUnsavedFileAttachment alloc]
            initWithPath:imagePath
            name:imageName
            type:@"image/jpeg"];

  rev.attachments = @{ imageName: att1 };

  [self.datastore save:rev completionHandler:^(id savedObject, NSError *error) {
   if(error) {
    [logger logErrorWithMessages:@"Error creating document: %@", error.localizedDescription];
   }
   [logger logDebugWithMessages:@"Document created: %@", savedObject];
  }];

  [self replicate];
 });
}

If we want to query data from either the remote or local data stores, we can just use the performQuery method on the data store. Below you can see a method for retrieving data for all of the images in the local data store.

-(void) getLocalData:(void (^)(NSArray *results, NSError *error)) completionHandler {

 NSPredicate *queryPredicate = [NSPredicate predicateWithFormat:@"(type = 'com.geopix.entry')"];
 CDTCloudantQuery *query = [[CDTCloudantQuery alloc] initWithPredicate:queryPredicate];

 [self.datastore performQuery:query completionHandler:^(NSArray *results, NSError *error) {

  completionHandler( results, error );
 }];
}

At this point we’ve now captured an image, captured the geographic location, saved that data in our local offline store, and then use replication to save that data up to the cloud whenever it is available.

AND…

We did all of this without writing a single line of server-side logic.   Since this is built on top of MobileFirst for Bluemix, all the backend infrastructure is setup for us, and we get operational analytics to monitor everything that is happening.

With the operational analytics we get:

  • App usage
  • Active Devices
  • Network Usage
  • Authentications
  • Data Storage
  • Device Logs (yes, complete debug/crash logs from devices out in the field)
  • Push Notification Usage

Sharing on the web

Up until this point we haven’t had to write any back-end code. However the mobile app boilerplate on Bluemix comes with a Node.js server.  We might as well take advantage of it.

I exposed the exact same data captured within the app on the Node.js service, which you can see at http://geopix.mybluemix.net/.

Web UI
Web UI

The Node.js back end comes preconfigured to leverage the express.js framework for building web applications.  I added the jade template engine and Leaflet for web-mapping, and was able to crank this out ridiculously quickly.

The first thing we need to do is make sure  we have our configuration variables for accessing the Cloudant service from our node app.  These are environment vars that you get automatcilly if you’re running on Bluemix, but you need to set these for your local dev environment:

var credentials = {};

if (process.env.hasOwnProperty("VCAP_SERVICES")) {
 // Running on Bluemix. Parse out the port and host that we've been assigned.
 var env = JSON.parse(process.env.VCAP_SERVICES);
 var host = process.env.VCAP_APP_HOST;
 var port = process.env.VCAP_APP_PORT;

 credentials = env['cloudantNoSQLDB'][0].credentials;
}
else {

 //for local node.js server instance
 credentials.username = "cloudant username here";
 credentials.password = "cloudant password here";
 credentials.url = "cloudant url here";
}

Next we’ll add our URL/content mappings:

app.get('/', function(req, res){
  prepareData(res, 'map');
});

app.get('/list', function(req, res){
  prepareData(res, 'list');
});

Next you’ll se the logic for querying the Cloudant data store and preparing the data for our UI templates. You can customize this however you want – caching for performance, refactoring for abstraction, or whatever you want. All interactions with Cloudant are powered by the Cloudant Node.js Client

var prepareData = function(res, template) {
 var results = [];

 //create the index if it doesn't already exist
 var sort_index = {name:'sort', type:'json', index:{fields:['sort']}};
 geopix.index(sort_index, function(er, response) {
  if (er) {
   throw er;
  }

  //perform the search
  //we're just pulling back all
  //data captured ("sort" will be numeric)
  var selector = {sort:{"$gt":0}};
  geopix.find({selector:selector, sort:["sort"]}, function(er, result) {
   if (er) {
    throw er;
   }

   //prepare data for template
   for (var x=0; x<result.docs.length; x++) {
    var obj = result.docs[x];

    for (var key in obj._attachments) {
     obj.image = credentials.url + "/" + database + "/" + obj._id +"/" + key;
     break;
    }

    results.push( obj );
   }
   res.render(template, { results:results});
  });
 });
};

After the prepareData method has prepared data for formatting in the UI, the template is rendered by invoking Jade’s render method:

res.render(template, { results:results});

This will render whichever template was passed in – I have two: map.jade (the map template) and list.jade (the list template). You can check out the list template below, and see it in action here: http://geopix.mybluemix.net/list

html
  head
 title GeoPix - powered by Bluemix
 link(href='//maxcdn.bootstrapcdn.com/bootstrap/3.3.2/css/bootstrap.min.css' rel='stylesheet')
 link(href='/public/css/index.css' rel='stylesheet')
 meta(name="viewport" content="width=device-width, initial-scale=1")
  body
 div(class='well')
  h1 GeoPix - Powered by Bluemix
  p
   a(href='/') Map
   &nbsp;|&nbsp;
   a(href='/list') List
 div(class="container-fluid")
  each val, index in results
   div(class="col-md-6")
    div(class="panel panel-default")
     div(class="panel-heading")
      h3= val.clientDate
     div(class="panel-body")
      img(src=val.image)
      p= 'latitude: ' + val.latitude + ", longitude:" + val.longitude + ", altitude:" + val.altitude

In the map view I used the Leaflet map engine and Open Street Map data, along with the Leaflet Marker Cluster plugin for displaying clustered results.

Source Code

You can check out the web interface live at: http://geopix.mybluemix.net/.  If you want to setup the environment on your own, you can grab the complete source code at:

Helpful Links

Ready to start building your own apps on IBM Bluemix?  Just head over to http://bluemix.net and get a free developer trial today!

Updated: Parallax Effects in Hybrid/Web Apps

A while back I wrote about adding parallax effects to your HTML/JS experiences to make them feel a bit richer and closer to a native experience.  I’ve just added this subtle (key word *subtle*) effect to a new project and made a few changes I wanted to share here.

If you are wondering what I am talking about with “parallax effects” – Parallax movement is where objects in the background move at a different rate than objects in the foreground, thus causing the perception of depth.  Read more about it if you’re interested.

First, here’s a quick video of this latest app in action.  It’s a hybrid MobileFirst app, but this technique could be used in any Cordova/PhoneGap/MobileFirst/mobile web app experience.  The key is to keep it subtle and not too much “in your face”, and yes, it is very subtle in this video.  You have to watch closely.

The techniques that I wrote about in the previous post still apply – I’ve just added a bit more to cover more use cases.

First let’s look at the CSS.

body {
    background-image: url('../images/cloud_advisor_bkgd.png');
    background-attachment: fixed;
    background-position: center;
    background-size: auto 120%;
}

This sets the background image and default position.  The distinct change here is that I set the background size to “auto” width and 120% height.  In this case, you can have a huge image that shrinks down to just slightly larger than the window size, or a small image that scales up to a larger window size.  This way you don’t end up with seams in a repeated background or a background that is too big to highlight the parallax effect effectively.

Next let’s take a quick look at the JS involved.

var position = "center";
var lastPosition = "center";
var contentCSS = "";
var body = $("body");
var content = $(".content");
window.suspendAnimation = false;

var xMovement = 15;
var yMovement = 30;
var halfX = xMovement/2;
var halfY = yMovement/2;

window.ondeviceorientation = function(event) {
 var gamma = event.gamma/90;
 var beta = event.beta/180;

 var temp = 0;

 //fix for holding the device upside down
 if ( gamma >= 1 ) {
  gamma = 2- gamma;
 } else if ( gamma <= -1) {
  gamma = -2 - gamma;
 }

 // shift values/motion based upon device orientation
 switch (window.orientation) {
  case 90:
   temp = gamma;
   gamma = beta;
   beta = temp;
   break;
  case -90:
   temp = -gamma;
   gamma = beta;
   beta = temp;
   break;

 }

 // update positions to be used for CSS
 var yPosition = -yMovement - (beta * yMovement);
 var xPosition = -xMovement - (gamma * xMovement);
 var contentX = (-xMovement - xPosition)/2;
 var contentY = (-yMovement - yPosition)/2;

 // generate css styles
 position = xPosition.toFixed(1) + "px " + yPosition.toFixed(1) + "px";
 contentCSS = "translate3d( " + (contentX.toFixed(1)) + "px, " + (contentY.toFixed(1)) + "px, " + " 0px)";
}

function updateBackground() {

 if (!window.suspendAnimation) {
  if ( position.valueOf() != lastPosition.valueOf() ) {

   body.css( "background-position", position);
   content.css( "-webkit-transform", contentCSS);
   lastPosition = position;
  }
 } else {
  lastPosition = "translate3d(0px, 0px, 0px)";;
 }

 window.requestAnimationFrame(updateBackground);
}

window.requestAnimationFrame(updateBackground);

The html where this would be used would be something like this:

<body>
  <div class="content">put your foreground stuff here.</div>
</body>

There are some subtle but important changes here:

  1. In the requestAnimationFrame loop, it only applies changes *if* there are changes to apply.  This prevents needless calls to apply CSS even if the CSS styles hadn’t changed.  In this, I also truncate the numeric CSS string so that it isn’t reapplying CSS if the position should shift by 0.01 pixels. Side note: If you aren’t using requestAnimationFrame for HTML animations, you should learn about it.
  2. If you used my old code and were holding the device upside down, it wouldn’t work.  Not even a little bit.  This has that fixed (see comments inline above).
  3. This moves the background in CSS, which doesn’t cause browser reflow operations, and moves the foreground content (inside of a div) using translate3d, which also doesn’t cause browser reflow operations.  This helps keep animations smooth and the UX performing optimally.
  4. I also added a global variable to turn parallax on and off very quickly, if you need it.

The result is a faster experience that is more efficient and less of a strain on CPU and battery.  Feel free to test this technique out on your own.

If you use the code above, you can modify the xMovement and yMovement variables to exaggerate the parallax effect.

Getting Started with IBM Bluemix Mobile Services (MBaaS)

I recently wrote an overview of IBM Bluemix’s Mobile Back-end as a Service offerings. I wanted to elaborate on the offerings, plus provide in-depth technical and implementation details, so I decided to produce this 5 part video series on Getting Started with IBM Bluemix Mobile Services.

This post specifically covers native iOS, though there are also Android and hybrid options available. This should have everything you need to get started. It covers all aspects from creating the app, to updating the back end, to leveraging Cloudant storage, push notifications, and monitoring & logging.

So, without further ado, let’s get started…

Part 1: Getting Started with Bluemix Mobile Services

In this first video I show how to create a new mobile app on Bluemix, connect to the cloud app instance, and implement remote logging from the client application. This process is covered in more detail in the Getting Started docs, but below are the basics from my experience.

You’ll first need to sign into your Bluemix account. If you don’t already have one, you can create a trial account for free. Once you’re signed in, you just need to create a new mobile app instance.

The process is very simple, and there is a “wizard” to guide you. The first thing that you need to do is create a new app by clicking the big “Create an App” button on your bluemix dashboard.

Create a new app from IBM Bluemix Dashboard
Create a new app from IBM Bluemix Dashboard

Next, select which kind of app you’re going to create. For MBaaS, you’ll want to select the “Mobile” option.

Select the type of app
Select the type of app

Next you’ll need to select your platform target. You can choose either “iOS, Android, Hybrid”, or the “iOS 8 beta” target. In this case I chose the iOS 8 beta, but the process is similar for both targets. Hybrid apps are built leveraging the Apache Cordova container.

Select your platform target
Select your platform target

Next, just specify an app name and click “Finish”.

Give your app a name
Give your app a name

Once your app is created, you will be presented with instructions how to connect the app in Xcode. I’ll get to that in a moment…

Now that your app has been created, you’ll be able to see it on your Bluemix dashboard. This app will consist of several components: a Node.js back-end instance, a Cloudant NoSQL database instance, an Advanced Mobile Access instance, and a Push instance. The Advanced Mobile Access component provides you with app analytics, user auth management, remote logging, and more. The Push component gives you the ability to manage and send push notifications (either manually, or with a rest-based API).

You app has been created
You app has been created – here are the components and the activity

Once your app has been created, you will need to setup the mobile app to connect to Bluemix to consume the services. Again, this is a very straightforward process.

The next step is to register your client application. Once your app is created, you will be presented with a screen to do this. If you don’t complete it right away, you can always come back later and register an application. You’ll need to specify the Bundle ID and version of your app, then you can setup any authentication (if you choose).

Register your app's bundle ID and version
Register your app’s bundle ID and version

Once your app has been registered, you need to configure Xcode. You’ll first need to create a new project in Xcode. There are two options for configuring your Xcode project: 1) automated installation using CocoaPods, or 2) manual installation. I used the CocoaPods installation simply because it is easier and manages dependencies for you.

If you aren’t familiar with CocoaPods, it is much like NPM… CocoaPods is a dependency manager for Cocoa projects. It helps you configure the Bluemix libraries and manages dependencies for you.

If you’ve got Xcode up and running, then close it and install CocoaPods, if you don’t already have it. Next open up a terminal/command prompt, go to the directory that contains your Xcode project and initialize CocoaPods using the “setup” command:

pod setup

This will create a new file called “podfile”. Open this file in any text editor and paste the following (note: you can remove any lines that you don’t want to actually use):

source 'https://github.com/CocoaPods/Specs.git'
# Copy the following list as is and
# remove the dependencies you do not need
pod 'IMFCore'
pod 'IMFGoogleAuthentication'
pod 'IMFFacebookAuthentication'
pod 'IMFURLProtocol'
pod 'IMFPush'
pod 'CloudantToolkit'

Save the changes to the “podfile” file, and close the text editor. Then go back to your command promprt/terminal  and run the installation process:

pod install

Your project will be configured, and all dependencies will be downloaded automatically. Once this is complete, open up the newly created .xcworkspace file (Xcode Workspace).

You have to initialize the Bluemix inside of your application to connect to the cloud service to be able to take advantage of any Bluemix features (logging, data access, auth, etc…). The best place to put this is inside of your AppDelegate.m class application didFinishLaunchingWithOptions method because it is the first code that will be run within your application:

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {
    // Override point for customization after application launch.

    // initialize SDK with IBM Bluemix application ID and route
    IMFClient *imfClient = [IMFClient sharedInstance];
    [imfClient
        initializeWithBackendRoute:@"your route"
        backendGUID:@"your guide"];

    return YES;
}

One of the first features I wanted to take advantage of was remote collection of client-side logs. You can do this using the IMFLogger class, in much the same fashion as you do with OCLogger in MobileFirst Foundation server. Once great feature that requires almost no additional configuration is the captureUncaughtExceptions method, which automatically configures the Advanced Mobile Access component to collect information for all app crashes.

// capture and record uncaught exceptions (crashes)
[IMFLogger captureUncaughtExceptions]; 

// change the verbosity filter to "debug and above"
[IMFLogger setLogLevel:IMFLogLevelDebug]; 

// create a logger instance
IMFLogger *logger = [IMFLogger loggerForName:@"AppDelegate"];

// log a message
[logger logDebugWithMessages:@"This is a log message from app launch."];

//send logged message to the server
[IMFLogger send];

Next, launch your app in the iOS simulator, or on a device, and you’ll see everything come together. Log into your Bluemix dashboard, and you’ll be able to monitor app analytics and remote logs.

Note: If you experience any issues connecting to the Bluemix mobile app from within the iOS simulator, just clear the iOS Simulator by going to the menu command “iOS Simulator -> Reset Content and Settings…”, and everything should connect properly the next time you launch the app.

Part 2: Configuring the Node.js Backend

In the next video, I demonstrate how to grab the code for the backend Node.js application, create a git repository on IBM JazzHub, then pull the code for local development.

There are two ways to push new backend Node.js code: 1) Using the Cloud Foundry command line API, or 2) by updating a git repository and leveraging automatic deployment from the git repo commits.

When the app is created, you’ll see an “add git” link under the app name. Using this link, you can create a git repository for the backend code.

Add a git repository
Add a git repository

Once your git repo has been created, you can check out the code using any Git client (I used the CLI). You’ll need to use the “npm install” command to pull down all the app dependencies. The biggest thing you need to know is that it uses express.js for the web application framework.  You can make any changes that you want, and they will be automatically deployed to your Bluemix server instance upon commit. Yes, this workflow is also configurable b/c this process may not work for everyone.

One other thing that you will need to watch out for if you are doing local development: You will want to wrap the following code on line 6 in a try/catch block, otherwise you will hit errors in the local environment which will prevent your app from launching locally:

try {
    passport.use(new ImfBackendStrategy());
} catch ( e ) {
    console.log(e);
}

Protected content behind the /protected url endpoint may not be accessible locally with this workaround, but you’ll be able to work on other pieces of your back end.

You can read more about the backend node instance for Bluemix mobile apps in the developer documentation.

Part 3: Consuming Data from Cloudant

Another part of Bluemix mobile applications is the Cloudant NoSQL database. The Cloudant NoSQL database is a powerful solution that gives you remote storage, querrying, and client-side data storage mechanisms with automatic online/offline synchronization, all with monitoring/analytics capabilities.

By default, objects within the Cloudant data store are treated as generic objects (over-simplification: think of it is an extremely powerful JSON store in the cloud). However you can also serialize your objects to strong data types within the client code configuration.

In your AppDelegate class application didFinishLaunchingWithOptions method, you’ll also want to initialize the IMFDataManager class, which is the class used for interacting with all Cloudant data operations.

IMFDataManager *manager =
    [IMFDataManager sharedInstance];

In my sample, I setup the database manually with open permissions, but you’ll probably want something more secure. Once your database is created, you can create indexes, search for data, create data, etc…

In the following code, I create a search index and query for data from the remote Cloudant database. You really only need to create the index if it doesn’t already exist. You can do this either through the mobile app code, or manually through the Cloudant database’s web interface. I did this inline in the following code, just for the sake of simplicity:

//access the remote data store
[[IMFDataManager sharedInstance] remoteStore:@"insurancedb" completionHandler:^(CDTStore *store, NSError *error) {
    // Remote store will be passed into the control handler if no errors occurred.

    // create a search index
    // this is an asynch operation
    [store createIndexWithName:@"customerNameIndex" fields:@[@"customer"] completionHandler:^(NSError *error) {
        // an error is set if index creation failed.

        // Next, we search...
        // Create a query with an NSPredicate
        NSPredicate *queryPredicate = [NSPredicate predicateWithFormat: @"(customer > '')"];
        CDTCloudantQuery *query = [[CDTCloudantQuery alloc] initDataType:@"Claim" withPredicate:queryPredicate];

        [store performQuery:query completionHandler:^(NSArray *results, NSError *error) {
            // results is an array of CDTMutableDocumentRevision objects that are returned by the query

            // convert to a NSArray of NSDictionary objects
            // you could also serialize this to an array of strongly typed objects
            NSMutableArray *array = [[NSMutableArray alloc] init];

            for (CDTMutableDocumentRevision *revision in results) {
                NSDictionary *body = [revision body];
                [array addObject:body];
            }

            // do something with the data (this is specific to my app)
            claimsData = array;

            // reload my data table in the main thread
            dispatch_async(dispatch_get_main_queue(), ^{
                [self reloadData];
            });

        }];
    }];
}];

Be sure to also read up on more of the Cloudant capabilities:

Part 4: Push Notifications

The IBM Bluemix mobile services app also contains a component for managing push notifications within your mobile applications. With this service, you can send push notifications to a specific device, a group of devices using tags, or all devices, and you can send push notifications either manually via the web interface, or as part of an automated workflow using the REST API.

You will first need to configure your app for push notifications. Apple systems using Apple’s Push Notification Service, and Android systems use Google Cloud Messaging. You must configure these hooks per the service providers.

For iOS apps, here are the basic steps for setting up the Push service. It also helps to be familiar with Local and Remote notifications in iOS.

  1. Create an App ID and enable Push Notifications
  2. Create a server certificate for sending push notifications
  3. Upload the server certificate to the Bluemix Push instance
  4. Setup the Xcode project to receive push notifications

In Xcode, open your AppDelegate class again. First you’ll need to register for remote notifications in application didFinishLaunchingWithOptions:

//register for push notifications (iOS 8-specific)
[[UIApplication sharedApplication] registerUserNotificationSettings:
    [UIUserNotificationSettings settingsForTypes:
        (UIUserNotificationTypeSound |
          UIUserNotificationTypeAlert |
          UIUserNotificationTypeBadge)
        categories:nil]];
[[UIApplication sharedApplication]
    registerForRemoteNotifications];

Next, setup the didRegisterForRemoteNotificationsWithDeviceToken callback to register this device with the Bluemix Push service:

-(void) application:(UIApplication *)application didRegisterForRemoteNotificationsWithDeviceToken:(NSData *)deviceToken{

    // get Push instance
    IMFPushClient* push = [IMFPushClient sharedInstance];

    // set current working environment
    push.environment = @"sandbox";

    [push registerDeviceToken:deviceToken completionHandler:^(IMFResponse *response, NSError *error) {

        IMFLogger *logger = [IMFLogger loggerForName:@"AppDelegate"];

        if (error){
            [logger logErrorWithMessages:@"error registering for push notifications: %@", error.description];
        } else {
            [logger logDebugWithMessages:@"registered for push notifications."];
        }
    }];
}

Next do something with the data whenever you receive a push notification inside of the didReceiveRemoteNotification method:

-(void)application:(UIApplication *)application didReceiveRemoteNotification:(NSDictionary *)userInfo {
    //userInfo dictionary will contain data sent from server.

    NSDictionary *notification = [[userInfo objectForKey:@"aps"] objectForKey:@"alert"];
    NSString *body = [notification objectForKey:@"body"];

    UIAlertView *alert = [[UIAlertView alloc] initWithTitle:@"Notification Received"
                                                    message:body delegate:self cancelButtonTitle:@"OK"
                                          otherButtonTitles:nil, nil];
    [alert show];
}

Part 5: Monitoring and Logging

Did I mention that every action that you perform through Bluemix Mobile Services can be monitored? Analytics are available for the Advanced Mobile Access component, the Cloudant NoSQL data store, and the Push Notifications service. In addition, you also have remote collection of client logs and crash reports. This provides  unparalleled insight into the health of your applications.

Need more info? You can find what you’re looking for here:

…. and of course, don’t forget the full 5-part video series available at https://www.youtube.com/playlist?list=PL0U4cUwfUs26kSKY0X-qdDNO5gKj6Lr6q

Ready to get started? Just head over to bluemix.net and create your first app!