Tag Archives: ios

Introducing the new Watson iOS SDK (beta)

watson-header
I’ve written here in the past on both the impact of cognitive computing, and how you can integrate IBM Watson services into your mobile apps to add cognitive language processing capabilities and more.  I’m happy to share that IBM has just recently released a new beta SDK that makes integrating more Watson services into your iOS applications easier than ever.

If you aren’t familiar with cognitive computing, or the transformative impact that it is already having on entire industries, then I strongly suggest checking out this video and related article on IBM DeveloperWorks.

IBM Watson services, which are based on machine learning algorithms, give you the ability to handle unstructured data, like text analysis or translation, speech processing, and more.  This makes consumption, mining, or responding to unstructured data or “dark data” faster, more efficient, and more powerful than ever.

The new Watson iOS SDK provides developers with an API to simplify integration of the Watson Developer Cloud services into their mobile apps, including the Dialog, Language Translation, Natural Language Classifier, Personality Insights, Speech To Text, Text to Speech, Alchemy Language, or Alchemy Vision services – all of which are available today, and can now be integrated with just a few lines of code.

The Watson iOS SDK makes integration with Watson services significantly *really* easy. For example, if you want to take advantage of the Language Translation service, you first have to setup a service instance. Once the translation service is setup, then you’ll be able to leverage translation capabilities within your mobile app:

//instantiate the LanguageTranslation service
let service = LanguageTranslation(username: "yourname", password: "yourpass")

//invoke translation methods
service.translate(["Hello","Welcome"],source:"en",target:"es",callback:{(text:[String], error) in
  //do something with the translated text strings
})

I’ve actually put a sample application together that demonstrates the language translation service integration, which you can access at github.com/triceam/Watson-iOS-SDK-Demo.

swift-translator

Be sure to check out the sample’s readme for additional detail and setup instructions. As with all of the Watson services, You must have a service instance properly configured, with authentication credentials in order to be able to consume it within your app.

The new Watson iOS SDK is written in Swift, is open source, and the team encourages you to provide feedback, submit issues, or make contributions.  You can learn more about the Watson iOS SDK, get the source code, and access the open source project here.

Mobile Apps with Language & Translation Services using IBM Watson & IBM MobileFirst

UPDATE 12/22/15:  IBM Recently released a new iOS SDK for Watson that makes integration with Watson services even easier. You can read more about it here.


I recently gave a presentation at IBM Insight on Cognitive Computing in mobile apps.  I showed two apps: one that uses Watson natural language processing to perform search queries, and another that uses Watson translation and speech to text services to take text in one language, translate it to another language, then even have the app play back the spoken audio in the translated language.  It’s this second app that I want to highlight today.

In fact, it gets much cooler than that.  I had an idea: “What if we hook up an OCR (optical character recognition) engine to the translation services?” That way, you can take a picture of something, extract the text, and translate it.  It turns out, it’s not that hard, and I was able to put together this sample app in just under two days.  Check out the video below to see it in action.

To be clear, I ended up using a version of the open source Tesseract OCR engine targeting iOS. This is not based on any of the work IBM research is doing with OCR or natural scene OCR, and should not be confused with any IBM OCR work.  This is basic OCR and works best with dark text on a light background.

The Tesseract engine lets you pass in an image, then handles the OCR operations, returning you a collection of words that it is able to extract from that image.  Once you have the text, you can do whatever you want from it.

So, here’s where Watson Developer Cloud Services come into play. First, I used the Watson Language Translation Service to perform the translation.  When using this service, I make a request to my Node.js app running on IBM Bluemix (IBM’s cloud platform).  The Node.js app acts as a facade and delegates to the Watson service for the actual translation.

translator

You can check out a sample on the web here:

Translate english to:

On the mobile client, you just make a request to your service and do something with the response. The example below uses the IMFResourceRequest API to make a request to the server (this can be done in either Objective C or Swift). IMFResourceRequest is the MobileFirst wrapper for networking requests that enables the MobileFirst/Mobile Client Access service to capture operational analytics for every request made by the app.

NSDictionary *params = @{
  @"text":text,
  @"source":@"en",
  @"target":language
};

IMFResourceRequest * imfRequest =
  [IMFResourceRequest requestWithPath:@"https://translator.mybluemix.net/translate"
                      method:@"GET" parameters:params];

[imfRequest sendWithCompletionHandler:^(IMFResponse *response, NSError *error) {
  NSDictionary* json = response.responseJson;
  NSArray *translations = [json objectForKey:@"translations"];
  NSDictionary *translationObj = [translations objectAtIndex:0];
  self.lastTranslation = [translationObj objectForKey:@"translation"];
  // now do something with the result - like update the UI
}];

On the Node.js server, it is simply taking the request and brokering it to the Watson Translation service (using the Watson Node.js SDK):

app.get('/translate', function(req, res){
  language_translation.translate(req.query, function(err, translation) {
    if (err) {
      console.log(err)
      res.send( err );
    } else {
      console.log(translation);
      res.send( translation );
    }
  });
});

Once you receive the result from the server, then you can update the UI, make a request to the speech to text service, or pretty much anything else.

To generate audio using the Watson Text To Speech service, you can either use the Watson Speech SDK, or you can use the Node.js facade again to broker requests to the Watson Speech To Text Service. In this sample I used the Node.js facade to generate Flac audio, which I played in the native iOS app using the open source Origami Engine library that supports Flac audio formats.

You can preview audio generated using the Watson Text To Speech service using the embedded audio below. Note: In this sample I’m using the OGG file format; it will only work in browsers that support OGG.

English: Hello and welcome! Please share this article with your friends!

Spanish:
Hola y bienvenido! Comparta este artículo con sus amigos!

app.get('/synthesize', function(req, res) {
  var transcript = textToSpeech.synthesize(req.query);
  transcript.on('response', function(response) {
    if (req.query.download) {
      response.headers['content-disposition'] = 'attachment; filename=transcript.flac';
    }
  });
  transcript.on('error', function(error) {
    console.log('Synthesize error: ', error)
  });
  transcript.pipe(res);
});

On the native iOS client, I download the audio file and play it using the Origami Engine player. This could also be done with the Watson iOS SDK (much easier), but I wrote this sample before the SDK was available.

//format the URL
NSString *urlString = [NSString stringWithFormat:@"https://translator.mybluemix.net/synthesize?text=Hola!&voice=es-US_SofiaVoice&accept=audio/flac&download=1", phrase, voice ];
NSString* webStringURL = [urlString stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding];
NSURL *flacURL = [NSURL URLWithString:webStringURL];

//download the contents of the audio file
NSData *audioData = [NSData dataWithContentsOfURL:flacURL];
NSString *docDirPath = NSTemporaryDirectory() ;
NSString *filePath = [NSString stringWithFormat:@"%@transcript.flac", docDirPath ];
[audioData writeToFile:filePath atomically:YES];

//pass the file url the the origami player and play the audio
NSURL* fileUrl = [NSURL fileURLWithPath:filePath];
[self.orgmPlayer playUrl:fileUrl];

Cognitive computing is all about augmenting the experience of the user, and enabling the users to perform their duties more efficiently and more effectively. The Watson language services enable any app to greater facilitate communication and broaden the reach of content across diverse user bases. You should definitely check them out to see how Watson services can benefit you.

MobileFirst

So, I mentioned that this app uses IBM MobileFirst offerings on Bluemix. In particular I am using the Mobile Client Access service to collect logs and operational analytics from the app. This lets you capture logs and usage metrics for apps that are live “out in the wild”, providing insight into what people are using, how they’re using it, and the health of the system at any point in time.

Analytics from the Mobile Client Access service
Analytics from the Mobile Client Access service

Be sure to check out the MobileFirst on Bluemix and MobileFirst Platform offerings for more detail.

Source

You can access the sample iOS client and Node.js code at https://github.com/triceam/Watson-Translator. Setup instructions are available in the readme document. I intend on updating this app with some more translation use cases in the future, so be sure to check back!

 

 

Wearables & IBM MobileFirst – Video & Sample Code

Last week I attended IBM Insight in Las Vegas. It was a great event, with tons of great information for attendees. I had a few sessions on mobile applications. In particular, my dev@Insight session on Wearables powered by IBM MobileFirst was recorded. You can check it out here:

https://youtu.be/d4AEwCOmvug

Sorry it’s not in HD, but the content is still great! (Yes, I am biased.)

In this session I showed how you can power wearable apps, specifically those on smart watch devices, using either the MobileFirst Platform Foundation Server, or the MobileFirst offerings on IBM Bluemix (cloud).

Key takeaways from the session:

  1. Wearables are the most personal computing devices ever. Your users can use them to be notified of information, search/consume data, or even collect environmental data for reporting or actionable analysis.
  2. Regardless of whether developing for a peripheral device like the Apple Watch or Microsoft Band, or a standalone device like Android Wear, you are developing an app that runs in an environment that mirrors that of a a native app. So, the fundamental development principles are exactly the same. You write native code, that uses standard protocols and common conventions to interact with the back-end.
  3. Caveat to #1: You user interface is much smaller. You should design the user interface and services to acomodate for the reduced amount of information that can be displayed.
  4. You can share code across both the phone/tablet and watch/wearable experience (depending on the target device).
  5. Using IBM MobileFirst you can easily expose data, add authentication, and capture analytics for both the mobile and wearable solutions.

Demos/Code Samples:

In the session I showed 3 sample wearable apps.  Full source code and setup instructions for each app is available at: https://github.com/triceam/MobileFirst-Wearables/

Stocks

A sample WatchKit (Apple Watch) app powered by IBM MobileFirst Platform Foundation Server.

applewatch-stocks

Contacts

A sample WatchKit (Apple Watch) app powered by IBM MobileFirst on Bluemix.

contacts-watch

Heartrate

A simple heart rate monitor using the Microsoft Band, powered by MobileFirst on Bluemix and IBM Cloudant.

heartrate

 

IBM Watson Speech Services Just Got A Whole Lot Easier

UPDATE 12/22/15:  IBM Recently released a new iOS SDK for Watson that makes integration with Watson services even easier. You can read more about it here.


IBM_Watson_avatar_negIBM’s Watson Developer Cloud speech services just got a whole lot easier for mobile developers.  I myself just learned about these two, and can’t wait to integrate them into my own mobile applications.

The Watson Speech to Text and Text to Speech services are now available in both native iOS and Android SDKs, making it even easier to integrate language services into your apps.

These native APIs now include audio streaming back to the Watson Speech to Text service, for lower latency responses to spoken languages.

I can guarantee you that my “voice-drive iOS apps” demo will be updated soon, and I’ll be using this for all future language processing services.

JavaScript All The Things – Or – Why You Should Pay Attention To JavaScript

This post is inspired by all the comments I’ve seen this week about JS in the enterprise. I would have never imagined this 10 years ago, but JavaScript is now pretty much ubiquitous. Here are a few reasons why you need to paying attention to JavaScript if you aren’t already, and why you should definitely not write it off.

First, I think one of the major reasons for JavaScript’s ubiquity is that JavaScript is approachable. It is relatively easy for beginners to learn JavaScript, and powerful enough for advanced users to build complex and reliable systems.

Second, why you need to pay attention, JavaScript is everywhere.

jsatt

You can now use JavaScript to develop on virtually any platform: client side applications, server side logic, embedded chips/IoT devices, manage build scripts and dependencies, and more.

This doesn’t mean you’ll use the exact same code in every case, rather that you can use the same skill set – JavaScript Development – to deliver solutions across multiple paradigms.

The Client Side

JavaScript can be used to power client side apps/user interfaces, and user interactions on numerous platforms and devices.

Web

Of course JavaScript powers the web, this is a given. JavaScript is the primary scripting language for all web browsers. I won’t focus on this much b/c it’s already well known.

Mobile

JavaScript can also be used to power mobile applications that are natively installed on a device.

  1. Apache Cordova/PhoneGap – You can build natively installed apps with web technology using PhoneGap or Cordova. PhoneGap is Adobe’s branded distribution of Cordova, but from the developer’s perspective, they are basically the same thing. Your app runs within a webview on the mobile device, and you build your user interface the same way you you build a dynamic web application. Your user interface is implemented in HTML, styled with CSS, and all interactivity is created with JavaScript.
  2. React Native – JavaScript powered web apps don’t just have to be inside of a a web view. The React Native framework gives developers the ability to write their application using JavaScript and declarative UI elements, and results in a native application running on the mobile device. The logic is interpreted JavaScript at runtime, but everything that the user interacts with (all UI elements) is 100% native, providing a very high quality user experience, and it is now available for both iOS and Android applications.
  3. Unity 3D – You can even develop rich & immersive mobile 3D simulation or gaming experience, entirely powered by JavaScript using the Unity 3D engine. **These can be web, desktop, or mobile, but is often used in mobile gaming.
  4. NativeScript – Framework for building cross-platform native iOS, Android and Windows mobile apps using JavaScript.
Desktop

Yup, desktop apps are not left out of the mix. Most desktop solutions fall into a category similar to Apache Cordova, where the end results is a web view that has access to lower level APIs, whose content is developed with web based technology.

  1. Electron – Node.js + Chromium desktop app container from GitHub
  2. app.js  – Node + Chromium for a desktop app container
  3. nw.js – Another framework for Node +Chromium for a desktop app container
  4. CEF – The Chromium Embedded Framework – a framework for embedding the guts of the chrome browser inside of a desktop app.

… and more… I know Microsoft has a solution for building Windows apps purely out of HTML/JS, and there are more solutions out there that I am forgetting.

In fact, some of my favorite desktop tools, such as SlackAtom and VS Code are actually based on web technology and implemented in HTML/JS. Heck, even Photoshop can be scripted and extended with the generator extensibility layer or have a customized user interface in HTML/JS with design spaces.

The Server Side

Most obviously Node.js – a JavaScript runtime buit on Chrome’s V8 JavaScript Engine – has made huge inroads into server side development and the enterprise. Node.js, powered by frameworks like express.js or loopback.io makes server side development and complex enterprise apps with JavaScript possible.

IoT

Pretty much everything that doesn’t fall in the categories above falls in here. You can develop headless apps that run on Arduino, Raspberry Pi or other small boards completely using JavaScript, you can manage infrastructure and information flow of IoT sensors using JavaScript, you can write on-chip programs for embedded systems using JavaScript, you can control robots with it, and you can even power media-centric connected TV experiences using JavaScript.

Like I said… It’s everywhere.

Ecosystem

It’s not just about where you can build and run JavaScript for your applications. JavaScript has a massive and thriving developer ecosystem.

JavaScript is the #1 most active language on GitHub in both the total number of active repositories and total number of active pushes/commits.

 

http://githut.info/
statistics visualization from http://githut.info/

Here are some stats that show the magnitude of growth and adoption for Node.js/npm.js alone. NPM stats currently shows a total of 186,946 packages available for download, 94,978,032 package downloads in the last day, and 2,451,734,737 package downloads in the last month.

npm
NPM Statistics

 

Node.js adoption is massive, and is still growing.

This doesn’t mean that JavaScript is the best language at everything. It also doesn’t meant that you can take a single piece of source code and run it in every device/context imaginable.

It means that you can use your skills in JavaScript to develop for just about any kind of device/context out there. It’s not going to be write once, run everywhere, rather in the words of the React.js team: learn once, write everywhere.