Tag Archives: cloud

Thoughts on Cognitive Computing

You may have heard a lot of buzz coming out of IBM lately about Cognitive Computing, and you might have also wondered “what the heck are they talking about?”  You may have heard of services for data and predictive analytics, services for natural language text processing, services for sentiment analysis, services understand speech and translate languages, but it’s sometimes hard to see the forest through the trees.

I highly recommend taking a moment to watch this video that introduces Cognitive Computing from IBM:

Those services that I mentioned above are all examples of Cognitive Computing systems, and are all available for you to use today.

From IBM Research:

Cognitive computing systems learn and interact naturally with people to extend what either humans or machine could do on their own.

They help human experts make better decisions by penetrating the complexity of Big Data.

Cognitive systems are often based upon massive sets of data and powerful analytics algorithms that detect patterns and concepts that can be turned into actionable information for the end users.  It’s not “artificial intelligence” in the sense that the services/machines act upon their own; rather a system that provides the user tools or information that enables them to make better decisions.

The benefits of cognitive systems in a nutshell:

  1. They augment the user’s experience
  2. They provide the ability to process information faster
  3. They make complex information easier to understand
  4. They enable you to do things you might not otherwise be able to do

Curious where this will lead?  Now take a moment and watch this video talking about the industry-transforming opportunities that Cognitive Computing is already beginning to bring to life”

So, why is the “mobile guy” talking about Cognitive Computing?

First, it’s because Cognitive Computing is big… I mean, really, really big.  Cognitive systems are literally transforming industries and providing powerful analytics and insight into the hands of both experts and “normal people”.  When I say “into the hands”, I again mean this literally; much of this cognitive ability is being delivered to those end users through their mobile devices.

It’s also because cognitive systems fit nicely with IBM’s MobileFirst product offerings.  It doesn’t matter whether you’re using the MobileFirst Platform Foundation server on-premise, or leveraging the MobileFirst offerings on IBM Bluemix, in both cases you can easily consume IBM Watson cognitive services to augment and enhance the interactions and data for your mobile applications. Check out the Bluemix catalog to see how you might start adding Watson cognitive or big data abilities to your apps today.

Last, and this is purely just personal opinion, I see the mobile MobileFirst offerings themselves as providing somewhat of cognitive service for developing mobile apps.  If you look at it from the operational analytics perspective, you have an immediate insight and a snapshot into the health of your system that you would never have seen otherwise.  You can know what types of devices are hitting your system, what services are being used, how long things are taking, and detect issues, all without any additional development efforts on your end. It’s not predictive analytics, but sure is helpful and gets us moving in the right direction.

Video – Smarter Apps with Cognitive Computing

UPDATE 12/22/15:  IBM Recently released a new iOS SDK for Watson that makes integration with Watson services even easier. You can read more about it here.


Last week I had the opportunity to present to a great audience at the MoDev DC meetup group on “Smarter Apps with Cognitive Computing”.   In this session I focused on how you can create a voice-driven experience in your mobile apps. I gave an introduction to IBM Bluemix and IBM Watson services (particularly the Watson language services), and demonstrated how you can integrate them into your native iOS apps. I also covered IBM MobileFirst for operational analytics and remote logging to provide insight into your app’s performance once it goes live.  Check out a recording of the complete presentation in the video below:

https://youtu.be/TGRMmf8e-6s

You can read more detail about how this example works and access source code for the sample application in the links below:

Just create an account on IBM Bluemix and you can get started for free!

This app uses three services available through IBM Bluemix, all of which are available for you to try out:

App Architecture
App Architecture

Feel free to poke around the code to learn more!

Complete Walkthrough and Source Code for “Building Offline Apps”

I recently put together some content on building “Apps that Work as Well Offline as they do Online” using IBM MobileFirst and Bluemix (cloud services).  There was the original blog post, I used the content in a presentation at ApacheCon, and now I’ve opened everything up for anyone use or learn from.

The content now lives on the IBM Bluemix github account, and includes code for the native iOS app, code for the web (Node.js) endpoint, a comprehensive script that walks through every step of of the process configuring the application, and also a video walkthrough of the entire process from backend creation to a complete solution.

Key concepts demonstrated in these materials:

  • User authentication using the Bluemix Advanced Mobile Access service
  • Remote app logging and instrumentation using the Bluemix Advanced Mobile Access service
  • Using a local data store for offline data access
  • Data replication (synchronization) to a remote data store
  • Building a web based endpoint on the Node.js infrastructure

You can download or fork any of the code at:

The repo contains:

  • Complete step-by-step instructions to guide through the entire app configuration and deployment process
  • Client-side Objective-C code (you can do this in either hybrid or other native platforms too, but I just wrote it for iOS).  The “iOS-native” folder contains the source code for a complete sample application leveraging this workflow. The “GeoPix-complete” folder contains a completed project (still needs you to walk through backend configuration). The “GeoPix-starter” folder contains a starter application, with all MobileFirst/Bluemix code commented out. You can follow the steps inside of the “Step By Step Instructions.pdf” file to setup the backend infrastructure on Bluemix, and setup all code within the “GeoPix-starter” project.
  • Backend Node.js app for serving up the web experience.

 

GeoPix: A sample iOS app powered by IBM MobileFirst for Bluemix

In this post I’d like to show a fairly simple application that I put together which shows off some of the rich capabilities for IBM MobileFirst for Bluemix that you get out of the box – All with an absolute minimal amount of your own developer effort.  Bluemix, of course, being IBM’s platform as a service offering.

GeoPix is a sample application leveraging IBM MobileFirst for Bluemix to capture data and images on a mobile device, persist that data locally (offline), and replicate that data to the cloud. Since it’s built with IBM MobileFirst, we get lots of things out of the box, including operational analytics, user authentication, and much more.

(full source code at the bottom of this post)

Here’s what the application currently does:

  • User can take a picture or select an image from the device
  • App captures geographic location when the image is captured
  • App saves both the image and metadata to a local data store on the device.
  • App uses asynchronous replication to automatically save any data in local store up to the remote store whenever the network is available
  • Oh yeah, can’t forget, the user auth is via Facebook
  • MobileFirst provides all the analytics we need.  Bluemix provides the cloud based server and Cloudant NoSQL data store.
  • All captured data is available on a web based front-end powered by Node.js

Here’s a video of it in action:

… and you can check out the web interface at geopix.mybluemix.net.

(full source code at the bottom of this post)

This is powered by the iOS 8 MobileFirst application boilerplate on Bluemix.  With this application template you can have your backend infrastructure setup within minutes, and it includes:

  • User authentication
  • Usage/operational analytics
  • Cloudant NoSQL DB
  • Simplified Push Notifications
  • Node.js backend

In this sample I’m using everything but the Push Notifications service.  I’m using user authentication, the Cloudant DB (offline/local store and remote/cloud store), and the node.js backend.  You get the operational analytics automatically.

To get started, you just need to create a new iOS 8 mobile application on Bluemix.  See my video series on Getting Started with IBM MobileFirst for Bluemix for a complete walkthrough of creating a new app using MobileFirst for Bluemix, or check out the Getting Started Guide in the official docs.

You need to initialize your app, and make sure you have setup the Facebook identity provider.  You can create your Facebook authentication at https://developers.facebook.com/.  Once the user is authenticated, the client app is fully functional.

The app UI is very simple, basically just two buttons for capturing images (the last captured image shows up in the background):

App's main UI
App’s main UI

There’s also a gallery for viewing local images:

Local gallery view
Local gallery view

Capturing Location

Capturing data is very straightforward.  The geographic location is captured using Apple’s Core Location framework.  We just need to implement the CLLocationManagerDelegate protocol:

[objc]- (void)locationManager:(CLLocationManager *)manager
didUpdateLocations:(NSArray *)locations {

self.currentLocation = [locations lastObject];
NSDate* eventDate = self.currentLocation.timestamp;
NSTimeInterval howRecent = [eventDate timeIntervalSinceNow];
if (abs(howRecent) < 15.0) {
// If the event is recent, do something with it.
locationLabel.text = [NSString stringWithFormat:@" Lat: %+.5f, Lon: %+.5f\n",
self.currentLocation.coordinate.latitude,
self.currentLocation.coordinate.longitude];
}
}[/objc]

Then initialize CLLocationManager using our class as the location manager’s delegate:

[objc]if (self.locationManager == nil)
self.locationManager = [[CLLocationManager alloc] init];
self.locationManager.delegate = self;
self.locationManager.desiredAccuracy = kCLLocationAccuracyBest;
self.locationManager.pausesLocationUpdatesAutomatically = YES;[/objc]

Capturing Images

Capturing images from the device is also very straightforward.  In the app I leverage Apple’s UIImagePickerController to allow the user to either upload an existing image or capture a new image.  See the presentImagePicker and didFinishPickingMediaWithInfo below. All of this standard practice using Apple’s developer SDK:

[objc]- (void) presentImagePicker:(UIImagePickerControllerSourceType) sourceType {
if ( sourceType == UIImagePickerControllerSourceTypeCamera && ![UIImagePickerController isSourceTypeAvailable:UIImagePickerControllerSourceTypeCamera]) {
[logger logErrorWithMessages:@"device has no camera"];
UIAlertView *myAlertView = [[UIAlertView alloc] initWithTitle:@"Error"
message:@"Device has no camera"
delegate:nil
cancelButtonTitle:@"OK"
otherButtonTitles: nil];
[myAlertView show];
};

if ( sourceType != UIImagePickerControllerSourceTypeCamera || [UIImagePickerController isSourceTypeAvailable:UIImagePickerControllerSourceTypeCamera] ){
UIImagePickerController *picker = [[UIImagePickerController alloc] init];
picker.delegate = self;
picker.allowsEditing = NO;
picker.sourceType = sourceType;

[self presentViewController:picker animated:YES completion:NULL];
}
}

– (void)imagePickerController:(UIImagePickerController *)picker didFinishPickingMediaWithInfo:(NSDictionary *)info {

[logger logDebugWithMessages:@"didFinishPickingMediaWithInfo"];
UIImage *image = info[UIImagePickerControllerOriginalImage];
currentImage.image = image;
[[DataManager sharedInstance] saveImage:image withLocation:self.currentLocation];
[picker dismissViewControllerAnimated:YES completion:nil];
}[/objc]

Persisting Data

If you notice in the didFinishPickingMediaWithInfo method above, there is a call to the DataManager’s saveImage withLocation method. This is where we save data locally and rely on Cloudant’s replication to automatically save data from the local data store up to the Cloudant NoSQL database.  This is powered by the iOS 8 Data service from Bluemix.

The first thing that we will need to do is initialize the local and remote data stores. Below you can see my init method from my DataManager class. In this, you can see the local data store is initialized, then the remote data store is initialized. If either data store already exists, the existing store will be used, otherwise it is created.

[objc]-(id) init {
self = [super init];

if ( self ) {
logger = [IMFLogger loggerForName:NSStringFromClass([self class])];
[logger logDebugWithMessages:@"initializing local datastore ‘geopix’…"];

// initialize an instance of the IMFDataManager
self.manager = [IMFDataManager sharedInstance];

NSError *error = nil;
//create a local data store
self.datastore = [self.manager localStore:@"geopix" error:&error];

if (error) {
[logger logErrorWithMessages:@"Error creating local data store %@",error.description];
}

//create a remote data store
[self.manager remoteStore:@"geopix" completionHandler:^(CDTStore *store, NSError *error) {
if (error) {
[logger logErrorWithMessages:@"Error creating remote data store %@",error.description];
} else {
[self.manager setCurrentUserPermissions:DB_ACCESS_GROUP_MEMBERS forStoreName:@"geopix" completionHander:^(BOOL success, NSError *error) {
if (error) {
[logger logErrorWithMessages:@"Error setting permissions for user with error %@",error.description];
}

[self replicate];
}];
}
}];

//start replication
[self replicate];
}

return self;
} [/objc]

Once the data stores are created, you can see that the replicate method is invoked.  This starts up the replication process to automatically push changesfrom the local data store to the remote data store “in the cloud”.

Therefore, if you’re collecting data when the app is offline, then you have nothing to worry about.  All of the data will be stored locally and pushed up to the cloud whenever you’re back online – all with no additional effort on your part.  When using replication with the Cloudant SDK, you just have to start the replication process and let it do it’s thing… fire and forget.

In my replicate function, I setup CDTPushReplication for pushing changes to the remote data store.  You could also setup two-way replication to automatically pull new changes from the remote store.

[objc]-(void) replicate {
if ( self.replicator == nil ) {
[logger logDebugWithMessages:@"attempting replication to remote datastore…"];

__block NSError *replicationError;
CDTPushReplication *push = [self.manager pushReplicationForStore: @"geopix"];
self.replicator = [self.manager.replicatorFactory oneWay:push error:&replicationError];
if(replicationError){
// Handle error
[logger logErrorWithMessages:@"An error occurred: %@", replicationError.localizedDescription];
}

self.replicator.delegate = self;

replicationError = nil;
[logger logDebugWithMessages:@"starting replication"];
[self.replicator startWithError:&replicationError];
if(replicationError){
[logger logErrorWithMessages:@"An error occurred: %@", replicationError.localizedDescription];
}else{
[logger logDebugWithMessages:@"replication start successful"];
}
}
else {
[logger logDebugWithMessages:@"replicator already running"];
}
}[/objc]

Once we’ve setup the remote and local data stores and setup replication, we now are ready to save the data the we’re capturing within our app.

Next is my saveImage withLocation method.  Here you can see that it creates a new CDTMutableDocumentRevision object (this is a generic object for the Cloudant NoSQL database), and populates it with the location data and timestamp.   It then creates a jpg image from the UIImage (passed in from the UIImagePicker above) and adds the jpg as an attachment to the document revision.  Once the document is created, it is saved to the local data store.   We then let replication take care of persisting this data to the back end.

[objc]-(void) saveImage:(UIImage*)image withLocation:(CLLocation*)location {

[logger logDebugWithMessages:@"saveImage withLocation"];

//save in background thread
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^(void) {

[logger logDebugWithMessages:@"creating document…"];

NSDate *now = [NSDate date];
NSString *dateString = [NSDateFormatter localizedStringFromDate:now
dateStyle:NSDateFormatterShortStyle
timeStyle:NSDateFormatterFullStyle];

// Create a document
CDTMutableDocumentRevision *rev = [CDTMutableDocumentRevision revision];
rev.body = @{
@"sort": [NSNumber numberWithDouble:[now timeIntervalSince1970]],
@"clientDate": dateString,
@"latitude": [NSNumber numberWithFloat:location.coordinate.latitude],
@"longitude": [NSNumber numberWithFloat:location.coordinate.longitude],
@"altitude": [NSNumber numberWithFloat:location.altitude],
@"course": [NSNumber numberWithFloat:location.course],
@"type": @"com.geopix.entry"
};

[logger logDebugWithMessages:@"creating image attachment…"];

NSDate *date = [NSDate date];
NSString *imageName = [NSString stringWithFormat:@"image%f.jpg", [date timeIntervalSince1970]];

NSString *tempDirectory = NSTemporaryDirectory();
NSString *imagePath = [tempDirectory stringByAppendingPathComponent:imageName];

[logger logDebugWithMessages:@"saving image to temporary location: %@", imagePath];

NSData *imageData = UIImageJPEGRepresentation(image, 0.1);
[imageData writeToFile:imagePath atomically:YES];

CDTUnsavedFileAttachment *att1 = [[CDTUnsavedFileAttachment alloc]
initWithPath:imagePath
name:imageName
type:@"image/jpeg"];

rev.attachments = @{ imageName: att1 };

[self.datastore save:rev completionHandler:^(id savedObject, NSError *error) {
if(error) {
[logger logErrorWithMessages:@"Error creating document: %@", error.localizedDescription];
}
[logger logDebugWithMessages:@"Document created: %@", savedObject];
}];

[self replicate];
});
}[/objc]

If we want to query data from either the remote or local data stores, we can just use the performQuery method on the data store. Below you can see a method for retrieving data for all of the images in the local data store.

[objc]-(void) getLocalData:(void (^)(NSArray *results, NSError *error)) completionHandler {

NSPredicate *queryPredicate = [NSPredicate predicateWithFormat:@"(type = ‘com.geopix.entry’)"];
CDTCloudantQuery *query = [[CDTCloudantQuery alloc] initWithPredicate:queryPredicate];

[self.datastore performQuery:query completionHandler:^(NSArray *results, NSError *error) {

completionHandler( results, error );
}];
}[/objc]

At this point we’ve now captured an image, captured the geographic location, saved that data in our local offline store, and then use replication to save that data up to the cloud whenever it is available.

AND…

We did all of this without writing a single line of server-side logic.   Since this is built on top of MobileFirst for Bluemix, all the backend infrastructure is setup for us, and we get operational analytics to monitor everything that is happening.

With the operational analytics we get:

  • App usage
  • Active Devices
  • Network Usage
  • Authentications
  • Data Storage
  • Device Logs (yes, complete debug/crash logs from devices out in the field)
  • Push Notification Usage

Sharing on the web

Up until this point we haven’t had to write any back-end code. However the mobile app boilerplate on Bluemix comes with a Node.js server.  We might as well take advantage of it.

I exposed the exact same data captured within the app on the Node.js service, which you can see at http://geopix.mybluemix.net/.

Web UI
Web UI

The Node.js back end comes preconfigured to leverage the express.js framework for building web applications.  I added the jade template engine and Leaflet for web-mapping, and was able to crank this out ridiculously quickly.

The first thing we need to do is make sure  we have our configuration variables for accessing the Cloudant service from our node app.  These are environment vars that you get automatcilly if you’re running on Bluemix, but you need to set these for your local dev environment:

[js]var credentials = {};

if (process.env.hasOwnProperty("VCAP_SERVICES")) {
// Running on Bluemix. Parse out the port and host that we’ve been assigned.
var env = JSON.parse(process.env.VCAP_SERVICES);
var host = process.env.VCAP_APP_HOST;
var port = process.env.VCAP_APP_PORT;

credentials = env[‘cloudantNoSQLDB’][0].credentials;
}
else {

//for local node.js server instance
credentials.username = "cloudant username here";
credentials.password = "cloudant password here";
credentials.url = "cloudant url here";
}[/js]

Next we’ll add our URL/content mappings:

[js]app.get(‘/’, function(req, res){
prepareData(res, ‘map’);
});

app.get(‘/list’, function(req, res){
prepareData(res, ‘list’);
});[/js]

Next you’ll se the logic for querying the Cloudant data store and preparing the data for our UI templates. You can customize this however you want – caching for performance, refactoring for abstraction, or whatever you want. All interactions with Cloudant are powered by the Cloudant Node.js Client

[js]var prepareData = function(res, template) {
var results = [];

//create the index if it doesn’t already exist
var sort_index = {name:’sort’, type:’json’, index:{fields:[‘sort’]}};
geopix.index(sort_index, function(er, response) {
if (er) {
throw er;
}

//perform the search
//we’re just pulling back all
//data captured ("sort" will be numeric)
var selector = {sort:{"$gt":0}};
geopix.find({selector:selector, sort:["sort"]}, function(er, result) {
if (er) {
throw er;
}

//prepare data for template
for (var x=0; x<result.docs.length; x++) {
var obj = result.docs[x];

for (var key in obj._attachments) {
obj.image = credentials.url + "/" + database + "/" + obj._id +"/" + key;
break;
}

results.push( obj );
}
res.render(template, { results:results});
});
});
};[/js]

After the prepareData method has prepared data for formatting in the UI, the template is rendered by invoking Jade’s render method:

[js]res.render(template, { results:results});[/js]

This will render whichever template was passed in – I have two: map.jade (the map template) and list.jade (the list template). You can check out the list template below, and see it in action here: http://geopix.mybluemix.net/list

[html]html
head
title GeoPix – powered by Bluemix
link(href=’//maxcdn.bootstrapcdn.com/bootstrap/3.3.2/css/bootstrap.min.css’ rel=’stylesheet’)
link(href=’/public/css/index.css’ rel=’stylesheet’)
meta(name="viewport" content="width=device-width, initial-scale=1")
body
div(class=’well’)
h1 GeoPix – Powered by Bluemix
p
a(href=’/’) Map
&nbsp;|&nbsp;
a(href=’/list’) List
div(class="container-fluid")
each val, index in results
div(class="col-md-6")
div(class="panel panel-default")
div(class="panel-heading")
h3= val.clientDate
div(class="panel-body")
img(src=val.image)
p= ‘latitude: ‘ + val.latitude + ", longitude:" + val.longitude + ", altitude:" + val.altitude[/html]

In the map view I used the Leaflet map engine and Open Street Map data, along with the Leaflet Marker Cluster plugin for displaying clustered results.

Source Code

You can check out the web interface live at: http://geopix.mybluemix.net/.  If you want to setup the environment on your own, you can grab the complete source code at:

Helpful Links

Ready to start building your own apps on IBM Bluemix?  Just head over to http://bluemix.net and get a free developer trial today!

Video: MobileFirst for Bluemix (MBaaS)

Last week I gave a presentation to the NYC Bluemix Meetup Group on IBM MobileFirst for Bluemix. Not familiar with the branding and have no idea what that means?  It is a mobile backend as a service, which gives you analytics, remote logging, user auth, data persistence & offline synch, push notification management, and more for your mobile applications.  Yes, as a service – you can create a Bluemix account today for free and start building your apps very quickly and very efficiently.  No problem if you weren’t able to make it to the meetup.  I recorded my session which you can check out in the embedded video below.

I know – the video quality isn’t fantastic, but it’s the best I had at the time.  (I almost always have a GoPro with me.)  If you want to see the code that makes all of this work in much, much more detail, check out my post on Getting Started with Bluemix Mobile Services – it has code, video tutorials and more.  Enjoy!