close

Artificial Intelligence

ArticlesCognitive ComputingIOT

Cognitive IoT Ebook Free Download

“Internet of Things is a vision where every object in the world has the potential to connect to the Internet and provide their data so as to derive actionable insights on its own or through other connected objects“

“Cognitive computing are systems that are designed to make computers think and learn like a human brain. Similar to an evolution of a human mind from a newborn to teenager to an adult, where new information is learned and existing information augmented, cognitive system learn through the vast amount of information fed to it. Such a system is trained on a set of information or data so that it can understand the context and help in making informed decisions.”

Cognitive Internet of Things is about enabling current IoT technologies with human-like intelligence.

This is a short ebook under 15 pages targeted to get an overview on Internet of Things and Cognitive Computing and synergies between them.

The book has 4 sections –
1.Introduction to Internet of Things.
2.Introduction to Cognitive Computing.
3.Cognitive Internet of Things
4.The architecture of a connected world.

Click Here to download this ebook free.

Enjoy reading it.

read more
ArticlesCognitive ComputingCognitive RetailFeatured

Cognitive Retail Platform


Cognitive Retail Platform

Happy to announce Cittin Cognitive Retail Cloud Platform , an end-to-end cognitive, digital marketing and analytical platform for retail.

Checkout Peffy – The personalized cognitive shopping application powered by Cittin Cognitive Retail Cloud Platform, which showcases some of our platform cognitive services.


Checkout the youtube video for more details on the Peffy Application

From a technology perspective, the Peffy application showcases the use of various machine learning algorithms, cognitive computing, natural language processing, personalisation  and visual search through the usage of Cittin Cognitive Retail platform. Checkout the live demo at http://peffy.in/
Stay tuned for more updates.

For queries and questions, reach out to me at naveen@cittin.io

read more
ArticlesCognitive ComputingIOT

Cognitive Computing, IoT and Synergies

Let me first start with an introduction on “Cognitive Computing”. Cognitive computing are systems that are designed to make computer’s think and learn like human brain.Similar to an evolution of a human mind from new born to child to an adult, where new information is learned and existing information augmented, cognitive system’s learn through the vast amount of information fed to it and training on a set of information,so it can understand the context and help in making informed decisions.

For example, if you look at any learning methodology, a human mind learns and understand the context, but its equipped to answer questions in an examination/interview which it might not have seen before, but using the experiences and past learning,a informed judgement can be made. Similarly cognitive systems are modelled to learn from past set of reference data set (or learning) to help users make informed decisions. Cognitive systems can be thought of non programming systems which learns through the set of information, training, interactions and a reference data set.

From a technology perspective, at a very high level, building a cognitive systems requires technologies that can understand the language,context, entities and relationship (NLP), learn through a set of supervised or unsupervised learning (Machine learning methodology), domain adoption through various techniques, technologies to help source, curate and manage content, runtimes to  build out the components together in a loosely coupled manner and wide variety of  tooling’s and methodology to enable making cognitive applications. I envision most of the cognitive capabilities offered as service over a cloud (a marketplace for cognitive and smart apps) which can be used individually or as a composite to create applications. Best way to get started with building cognitive applications, its through the Cognitive Services Zone on bluemix

So where does Internet of Things fits into the cognitive world. I assume you are aware of IoT, if not, please read my introductory blog at – IOT – Are you connected?

Let’s start with high level view of an IoT Cognitive Systems Architecture. The real value of IOT applications is realized by enabling them as part of existing applications and higher value added services to create new innovative business solutions.

IOT-COGNITIVE-COMPUTING-ARCHITECTURE

IoT is not just limited to devices and sensors, but how do you derive real insights from the sensor data, filter and analyze the data and use the information with multiple data sources, understand and infer relationship in context of the use case to augment current application with new offerings or build new innovate solutions. The architecture talks about the evolution of an application, building on the value chain and how various cloud services and offering are used to enable smarter applications.

Let’s take an example of a cognitive IoT application. I have taken a very simple example to get the technology and benefits across.

1. I step out of my home and the home electricity turns into a power saver mode.
2. I step into the car and the car recognizes me.
3. My car seats are automatically adjusted.
4. My favorite music station is set and play list is started.
5. An aggregated news for the day is available and tailored for me on my dashboard and read it out by the dashboard device. If I am travelling, weather forecast and news related to the place I am travelling is available as an add-on.
6. As I pass by malls, my car reminds to purchase stuffs.
7. I sync my digital cart and get my purchases quickly. No need to move around, find things and put it to a physical cart.
8. I come back to my car, I start interacting with the Car in Natural human language instead of typing in numbers and searching for things.
9. I ask for good places for lunch which I haven’t visited. Based on my past experience’s and cuisine preference,ratings from third party sites, a set of recommendations are provided. I choose one of them.
10. GPS devices are synced up based on my response and direction’s changed.
11. Alternative route is automatically selected based on traffic sensor’s which the GPS device is subscribed to.
12. I have lunch, I don’t need to carry cards, I am being recognized. The best credit card (after checking available offers for that restaurant) from my digital wallet is automatically selected and used for the payment. Welcome to smart cashless transactions.
13. I come back home, lights are back up again, all devices started.
14. I say “good night”, lights are dimmed and tell my clock to wake me up at 7:00 am after checking my flight status.

As you see in the above example, the real value is derived from how data from sensors are used as part of the broader ecosystem and how cognitive capabilities and learning are used to provide value added services.

These are just one of the many experiences. The real experiences would be technology touching our every day lives and making it easier and a better place to live. I gave a consumer driven example, but IoT benefits can be leveraged across industries – connected cars, aviation, manufacturing, predictive maintenance, healthcare, insurance etc..

With the context set, I suggest to read my follow on blog on “Architecture of a Connected World“.

read more
Architecture PatternsArticlesCloud ComputingCognitive ComputingIOT

IOT Cognitive Systems Architecture

Following is a high level view on how I visualize an IOT Cognitive Systems Architecture. The real value of IOT applications is realized by enabling them as part of existing applications and higher value added services to create new innovative business solutions.

Over the next few years, most of the capabilities (leaving out some of the hybrid cloud use cases) would be available as cloud services, enabling creating of smart business applications through a plug and play architecture. I evasion an ecosystem of cloud services (much like an apple or an android app/play store), where applications can be assembled from multiple providers to provide higher value solutions.

IOT-COGNITIVE-COMPUTING-ARCHITECTURE

PS- The above are some of my rough drafts on my upcoming book on “Advanced Computing – A Futuristic View” which covers IOT, Cognitive, Big Data, Cloud Services, Analytic and Smart Applications.

read more
ArticlesCloud ComputingCognitive ComputingIOTUpcoming technology trends and reviews

Smart Apps, IOT and Cognitive Computing

Smart Computing – IOT meets Cognitive Computing

A peep in the future of connected world –

1. I step out of my home and the home electricity turns into a power saver mode.
2. I step into the car and the car recognizes me.
3. My car seats are automatically adjusted.
4. My favorite music station is set and play list is started.
5. An aggregated news for the day is available and tailored for me on my dashboard and read it out by the dashboard device. If I am travelling, weather forecast and news related to the place I am travelling is available as an add-on.
6. As I pass by malls, my car reminds to purchase stuffs.
7. I sync my digital cart and get my purchases quickly. No need to move around, find things and put it to a physical cart.
8. I come back to my car, I start interacting with the Car in Natural human language instead of typing in numbers and searching for things.
9. I ask for good places for lunch which I haven’t visited. Based on my past experience’s and cuisine preference,ratings from third party sites, a set of recommendations are provided. I choose one of them.
10. GPS devices are synced up based on my response and direction’s changed.
11. Alternative route is automatically selected based on traffic sensor’s which the GPS device is subscribed to.
12. I have lunch, I don’t need to carry cards, I am being recognized. The best credit card (after checking available offers for that restaurant) from my digital wallet is automatically selected and used for the payment. Welcome to smart cashless transactions.
13. I come back home, lights are back up again, all devices started.
14. I say “good night”, lights are dimmed and tell my clock to wake me up at 7:00 am after checking my flight status.

These are just one of the many experiences. The real experiences would be technology touching our every day lives and making it easier and a better place to live.

read more
ArticlesCognitive Computing

Invoking Alchemy Face Detection service using JQUERY

Here is a sample reference on how to invoke Alchemy Face Detection service using JQUERY.  The details were not available on the website, so I decided to build one quickly. I would use this only for quick experimentation as it exposes the client key on the browser side.

Here is the index.html file –

<!doctype html>
<html lang=”en”>
<head>
<meta charset=”utf-8″>
<meta http-equiv=”X-UA-Compatible” content=”IE=edge,chrome=1″>
<script src=”http://ajax.googleapis.com/ajax/libs/jquery/2.0.3/jquery.min.js”></script>
<script src=”script.js”></script>
</head>
<body>

<h1>File Upload</h1>

<form action=”#” method=”post”>
<ul>
<li>
<label for=”name”>Name:</label>
<input type=”text” name=”name” id=”name” multiple>
</li>

<li>
<label for=”file_upload”>File:</label>
<input type=”file” name=”file_upload” id=”file_upload” multiple>
</li>

<li><input class=”button green” type=”submit” name=”submit” value=”Submit Content”></li>
</ul>
</form>
</body>
</html>

Here is the Javsacript file (script.js) code

$(function()
{

var files;

// Add events
$(‘input[type=file]’).on(‘change’, prepareUpload);
$(‘form’).on(‘submit’, uploadFiles);
function prepareUpload(event)
{
files = event.target.files;
}
function uploadFiles(event)
{
event.stopPropagation();
event.preventDefault();

$.ajax({

url :’http://access.alchemyapi.com/calls/image/ImageGetRankedImageFaceTags?apikey=yourkey&outputMode=xml&imagePostMode=raw’,
type: ‘POST’,
data : files[0],
cache: false,
processData: false,
contentType: ‘application/x-www-form-urlencoded’,
success: function(data, textStatus, jqXHR)
{
if(typeof data.error === ‘undefined’)
{
console.log(‘Data: ‘ + jqXHR.responseText);

}
else
{
console.log(‘ERRORS: ‘ + data.error);
}
},
error: function(jqXHR, textStatus, errorThrown)
{
console.log(‘ERRORS: ‘ + textStatus);

}
});
}

});

read more
ArticlesCognitive Computing

Social Match Making using Watson Personality Insights

Lot of information is shared by the user through various social media posts, blogs and opinions. With so much information being shared, how can this unstructured information be leveraged to tailor useful information to the user. If you start start analyzing, you would come across a variety of applications which are currently untapped. I am presenting one such application called “Social MATCH MAKING”.

Social Match Making is a term I coined to use insights from a user activity (blogs, tweets, usually various matrimonial/finder websites employ about yourself details) to understand the social and physiological aspects and use this to compare traits from other prospects/users. For example, in Indian tradition, usually a “kundli” or astrology birth chart is compared to see if a match is compatible using various scientific calculations and planetary positions. Similarly you could use Insights to compare traits and arrive at a conclusion.

Give it a try, use the Watson Personality Insights and compare traits from users to find like minded or persons from opposite kind from your social media groups. You can easily build an application, that sources profiles from social media groups or geo-locations to analyse and compare profiles based on social and cognitive characteristics. Not only social mediums, this concept can be used within a community, intranet or an enterprise or to to drive events (movies, restaurants, places of interest any more) matching a user interest.

I liked the concept of Watson Personality Insights, but I feel in the current form its provides limited capability, particularly requiring a 2000 words to arrive at proper analytics. Getting those 2000 words is a challenge and the implementation could have been way better.

 

read more
ArticlesCognitive Computing

Watson Personality Insights Service using PHP

Here is a sample reference of  using Watson Personality Insight service with PHP. Since this is not readily available as part of the BlueMix Watson examples, I am sharing this in my blog,

The testservice.php is the sample test client which uses the WatsonService.php. The following code is for reference only.

testservice.php

……………….

<?php

/*
* To change this template, choose Tools | Templates
* and open the template in the editor.
*/

require_once(‘./WatsonService.php’);

echo ” hello”;

$text = “Call me Ishmael. Some years ago-never mind how long precisely-having little or no money in my purse, and nothing particular to interest me on shore, I thought I would sail about a little and see the watery part of the world. It is a way I have of driving off the spleen and regulating the circulation. Whenever I find myself growing grim about the mouth; whenever it is a damp, drizzly November in my soul; whenever I find myself involuntarily pausing before coffin warehouses, and bringing up the rear of every funeral I meet; and especially whenever my hypos get such an upper hand of me, that it requires a strong moral principle to prevent me from deliberately stepping into the street, and methodically knocking people’s hats off-then, I account it high time to get to sea as soon as I can. This is my substitute for pistol and ball. With a philosophical flourish Cato throws himself upon his sword; I quietly take to the ship. There is nothing surprising in this. If they but knew it, almost all men in their degree, some time or other, cherish very nearly the same feelings towards the ocean with me.\r\n\r\nThere now is your insular city of the Manhattoes, belted round by wharves as Indian isles by coral reefs-commerce surrounds it with her surf. Right and left, the streets take you waterward. Its extreme downtown is the battery, where that noble mole is washed by waves, and cooled by breezes, which a few hours previous were out of sight of land. Look at the crowds of water-gazers there.\r\n\r\nCircumambulate the city of a dreamy Sabbath afternoon. Go from Corlears Hook to Coenties Slip, and from thence, by Whitehall, northward. What do you see?-Posted like silent sentinels all around the town, stand thousands upon thousands of mortal men fixed in ocean reveries. Some leaning against the spiles; some seated upon the pier-heads; some looking over the bulwarks of ships from China; some high aloft in the rigging, as if striving to get a still better seaward peep. But these are all landsmen; of week days pent up in lath and plaster-tied to counters, nailed to benches, clinched to desks. How then is this? Are the green fields gone? What do they here?\r\n\r\nBut look! here come more crowds, pacing straight for the water, and seemingly bound for a dive. Strange! Nothing will content them but the extremest limit of the land; loitering under the shady lee of yonder warehouses will not suffice. No. They must get just as nigh the water as they possibly can without falling in. And there they stand-miles of them-leagues. Inlanders all, they come from lanes and alleys, streets and avenues-north, east, south, and west. Yet here they all unite. Tell me, does the magnetic virtue of the needles of the compasses of all those ships attract them thither?\r\n\r\nOnce more. Say you are in the country; in some high land of lakes. Take almost any path you please, and ten to one it carries you down in a dale, and leaves you there by a pool in the stream. There is magic in it. Let the most absent-minded of men be plunged in his deepest reveries-stand that man on his legs, set his feet a-going, and he will infallibly lead you to water, if water there be in all that region. Should you ever be athirst in the great American desert, try this experiment, if your caravan happen to be supplied with a metaphysical professor. Yes, as every one knows, meditation and water are wedded for ever.\r\n\r\nBut here is an artist. He desires to paint you the dreamiest, shadiest, quietest, most enchanting bit of romantic landscape in all the valley of the Saco. What is the chief element he employs? There stand his trees, each with a hollow trunk, as if a hermit and a crucifix were within; and here sleeps his meadow, and there sleep his cattle; and up from yonder cottage goes a sleepy smoke. Deep into distant woodlands winds a mazy way, reaching to overlapping spurs of mountains bathed in their hill-side blue. But though the picture lies thus tranced, and though this pine-tree shakes down its sighs like leaves upon this shepherd’s head, yet all were vain, unless the shepherd’s eye were fixed upon the magic stream before him. Go visit the Prairies in June, when for scores on scores of miles you wade knee-deep among Tiger-lilies-what is the one charm wanting?-Water-there is not a drop of water there! Were Niagara but a cataract of sand, would you travel your thousand miles to see it? Why did the poor poet of Tennessee, upon suddenly receiving two handfuls of silver, deliberate whether to buy him a coat, which he sadly needed, or invest his money in a pedestrian trip to Rockaway Beach? Why is almost every robust healthy boy with a robust healthy soul in him, at some time or other crazy to go to sea? Why upon your first voyage as a passenger, did you yourself feel such a mystical vibration, when first told that you and your ship were now out of sight of land? Why did the old Persians hold the sea holy? Why did the Greeks give it a separate deity, and own brother of Jove? Surely all this is not without meaning. And still deeper the meaning of that story of Narcissus, who because he could not grasp the tormenting, mild image he saw in the fountain, plunged into it and was drowned. But that same image, we ourselves see in all rivers and oceans. It is the image of the ungraspable phantom of life; and this is the key to it all.\r\n\r\nNow, when I say that I am in the habit of going to sea whenever I begin to grow hazy about the eyes, and begin to be over conscious of my lungs, I do not mean to have it inferred that I ever go to sea as a passenger. For to go as a passenger you must needs have a purse, and a purse is but a rag unless you have something in it. Besides, passengers get sea-sick-grow quarrelsome-don’t sleep of nights-do not enjoy themselves much, as a general thing;-no, I never go as a passenger; nor, though I am something of a salt, do I ever go to sea as a Commodore, or a Captain, or a Cook. I abandon the glory and distinction of such offices to those who like them. For my part, I abominate all honourable respectable toils, trials, and tribulations of every kind whatsoever. It is quite as much as I can do to take care of myself, without taking care of ships, barques, brigs, schooners, and what not. And as for going as cook,-though I confess there is considerable glory in that, a cook being a sort of officer on ship-board-yet, somehow, I never fancied broiling fowls;-though once broiled, judiciously buttered, and judgmatically salted and peppered, there is no one who will speak more respectfully, not to say reverentially, of a broiled fowl than I will. It is out of the idolatrous dotings of the old Egyptians upon broiled ibis and roasted river horse, that you see the mummies of those creatures in their huge bake-houses the pyramids.\r\n\r\nNo, when I go to sea, I go as a simple sailor, right before the mast, plumb down into the forecastle, aloft there to the royal mast-head. True, they rather order me about some, and make me jump from spar to spar, like a grasshopper in a May meadow. And at first, this sort of thing is unpleasant enough. It touches one’s sense of honour, particularly if you come of an old established family in the land, the Van Rensselaers, or Randolphs, or Hardicanutes. And more than all, if just previous to putting your hand into the tar-pot, you have been lording it as a country schoolmaster, making the tallest boys stand in awe of you. The transition is a keen one, I assure you, from a schoolmaster to a sailor, and requires a strong decoction of Seneca and the Stoics to enable you to grin and bear it. But even this wears off in time.\r\n\r\nWhat of it, if some old hunks of a sea-captain orders me to get a broom and sweep down the decks? What does that indignity amount to, weighed, I mean, in the scales of the New Testament? Do you think the archangel Gabriel thinks anything the less of me, because I promptly and respectfully obey that old hunks in that particular instance? Who ain’t a slave? Tell me that. Well, then, however the old sea-captains may order me about-however they may thump and punch me about, I have the satisfaction of knowing that it is all right; that everybody else is one way or other served in much the same way-either in a physical or metaphysical point of view, that is; and so the universal thump is passed round, and all hands should rub each other’s shoulder-blades, and be content.\r\n\r\nAgain, I always go to sea as a sailor, because they make a point of paying me for my trouble, whereas they never pay passengers a single penny that I ever heard of. On the contrary, passengers themselves must pay. And there is all the difference in the world between paying and being paid. The act of paying is perhaps the most uncomfortable infliction that the two orchard thieves entailed upon us. But BEING PAID,-what will compare with it? The urbane activity with which a man receives money is really marvellous, considering that we so earnestly believe money to be the root of all earthly ills, and that on no account can a monied man enter heaven. Ah! how cheerfully we consign ourselves to perdition!\r\n\r\nFinally, I always go to sea as a sailor, because of the wholesome exercise and pure air of the fore-castle deck. For as in this world, head winds are far more prevalent than winds from astern (that is, if you never violate the Pythagorean maxim), so for the most part the Commodore on the quarter-deck gets his atmosphere at second hand from the sailors on the forecastle. He thinks he breathes it first; but not so. In much the same way do the commonalty lead their leaders in many other things, at the same time that the leaders little suspect it. But wherefore it was that after having repeatedly smelt the sea as a merchant sailor, I should now take it into my head to go on a whaling voyage; this the invisible police officer of the Fates, who has the constant surveillance of me, and secretly dogs me, and influences me in some unaccountable way-he can better answer than any one else. And, doubtless, my going on this whaling voyage, formed part of the grand programme of Providence that was drawn up a long time ago. It came in as a sort of brief interlude and solo between more extensive performances. I take it that this part of the bill must have run something like this:\r\n\r\n\r\nThough I cannot tell why it was exactly that those stage managers, the Fates, put me down for this shabby part of a whaling voyage, when others were set down for magnificent parts in high tragedies, and short and easy parts in genteel comedies, and jolly parts in farces-though I cannot tell why this was exactly; yet, now that I recall all the circumstances, I think I can see a little into the springs and motives which being cunningly presented to me under various disguises, induced me to set about performing the part I did, besides cajoling me into the delusion that it was a choice resulting from my own unbiased freewill and discriminating judgment.\r\n\r\nChief among these motives was the overwhelming idea of the great whale himself. Such a portentous and mysterious monster roused all my curiosity. Then the wild and distant seas where he rolled his island bulk; the undeliverable, nameless perils of the whale; these, with all the attending marvels of a thousand Patagonian sights and sounds, helped to sway me to my wish. With other men, perhaps, such things would not have been inducements; but as for me, I am tormented with an everlasting itch for things remote. I love to sail forbidden seas, and land on barbarous coasts. Not ignoring what is good, I am quick to perceive a horror, and could still be social with it-would they let me-since it is but well to be on friendly terms with all the inmates of the place one lodges in.\r\n\r\nBy reason of these things, then, the whaling voyage was welcome; the great flood-gates of the wonder-world swung open, and in the wild conceits that swayed me to my purpose, two and two there floated into my inmost soul, endless processions of the whale, and, mid most of them all, one grand hooded phantom, like a snow hill in the air”;
WatsonService::Instance()->getInsights($text);
?>

……………………….

WatsonService.php

……………………….

<?php

/**
* Watson PI Service
*/
final class WatsonService {
private static $inst = null;
private $auth = “”;
private $wsurl = “”;
public static function Instance() {
if (self::$inst === null) {
self::$inst = new WatsonService();
}
return self::$inst;
}

private function __construct() {

$parsedUrl = ‘https://gateway.watsonplatform.net/systemu/service’;
$username = ”;
$authp = ”;
#local dummy test
$password = ”;
if($vcapStr = getenv(‘VCAP_SERVICES’)) {
$vcap = json_decode($vcapStr, true);
foreach ($vcap as $serviceTypes) {
foreach ($serviceTypes as $service) {
if($service[‘name’] == ‘user_modeling’) {
$credentials = $service[‘credentials’];
$username = $credentials[‘username’];
$password = $credentials[‘password’];
$parsedUrl = parse_url($credentials[‘url’]);
$host = $parsedUrl[‘host’];
$port = isset($parsedUrl[‘port’]) ?
$parsedUrl[‘port’] : $parsedUrl[‘scheme’] == ‘http’ ?
’80’ : ‘443’;
break;
}
}
}
}

$auth = base64_encode($username . “:” . $password);
$this->auth = $auth;
$this->wsurl = $parsedUrl . “/api/v2/profile”;

}
/**
* Create request for Watson PI service
*/
public function getInsights($datatext) {
try {
$datax = array(
“contentItems” =>
array(
$datarr = array(
‘userid’ => ‘dummy’,
‘id’ => ‘someid’,
‘sourceid’ => ‘freetext’,
‘contenttype’ => ‘text/plain’,
‘language’ => ‘en’,
‘content’ => $datatext)));

$data_string = json_encode($datax);
$curl = curl_init();

//print “ok”;
//print $this->auth;

curl_setopt($curl, CURLOPT_HTTPHEADER, array(
‘Content-Type: application/json’,
‘X-SyncTimeout: 60’,
‘Authorization: Basic ‘ . $this->auth,
‘Content-Length: ‘ . strlen($data_string))
);

curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($curl, CURLOPT_POST, true);
curl_setopt($curl, CURLOPT_POSTFIELDS, $data_string);
curl_setopt($curl, CURLOPT_URL, $this->wsurl);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($curl);
if(curl_errno($curl))
{
echo ‘error:’ . curl_error($curl);
}
//print $result;
curl_close($curl);
return $result;
} catch(Exception $e) {
echo ‘<p>There Was an Error Accessing Watson User Modelling Service!!!</p>’;
echo $e->getMessage();
}
}
}

…………………………..

read more
ArticlesCognitive Computing

Watson Content Curation & Ingestion Best Practices

The blog contains some of the best practices and guidelines for sourcing and evaluating content for Watson enabled applications. Given below is my personal view and recommendations (not endorsed) and should not to be used as a reference for all set of requirements as each client requirements would be different.

Content Sourcing Considerations

Building an application that’s built with Watson requires the right kind of data and content. Watson system answers the question from the content that is ingested in the system. Based on the use case, you would source content within your enterprise or get the information from externally sources, like crawl the web for information like medical journals, financial reports etc. or a combination of both.

As you ingest content in Watson, you need to ensure that you own the content or the content is publicly available and free to use. Please look at the license requirements associated with the content.

Alternatively, you can also tap into to the Watson Content Marketplace to leverage different sources of content based on your use cases. For instance, if you are building a travel application, you can tap into the heath care content from Watson Content Marketplace which provides details on vaccination as part of a user’s travel. The Watson Content Marketplace makes this possible by bringing together different sources of data for developers and content providers, including general knowledge, industry-specific content and subject matter expertise.

As part of content sourcing strategy, you would also determine the lifecycle strategy for managing the content, for instance frequency of adding new content, handling updates to the existing content and validity of the content.

Watson currently support’s HTML, PDF documents, Word documents as part of its ingestion process. Any other format’s need to be converted into one of these formats. Watson currently doesn’t crawl the content. The content needs to be managed and uploaded in Watson through its user interface (referred to as Watson Experience Manager).. The above statements is true at the time of writing this blog.

 

Content Validation

Content validation is a critical piece in every Watson engagement.  To start with, content needs to be in one of the formats which is supported by Watson.

Here are the general guidelines around content validation –

  • Content should be in english language. (other languages are being supported)
  • Content should be in one of the supported format – Microsoft Word formats (expect 2003), HTML and PDF.
  • The content should be in UT-8 format.
  • If content contains scanned images/text (OCR), it needs to convert into either one of the supported format.
  • The content should not be password protected.
  • Content should not contain any personal identifiable information. Any personal identifiable information needs to be handled by the application outside of Watson.
  • Images are currently not processed by Watson. Identify substitute for images if important information is being conveyed by images, like flowcharts, process flows which needs to be answered. An image centric content, like financial charts only would not work with Watson. You can use the Watson Image detection service to get the details of the images.
  • If the content contains tables, ensure the table has well defined heading. For instance and HTML format would typically have header row and column tag. On other hand, a div css style tables might not be work as expected as relevant met-data for understanding the context of the table might not be available.

Watson strength lies in unstructured data analysis. For instance, you can feed it a medical journal or blog and it will understand what facts, relationship and meaning of the sentence is contained in the documents. On the other hand if you have content only related to financial data and want to do mathematical computation on it, the use case might not be a good fit for some of the Watson products like Watson Engagement Advisor.

 

Content Structuring Practices

Content can be ingested into Watson “as-is” without any modifications, but typically content like HTML would have many noise elements like navigation headers, sidebars of links which would interfere with the actual body of the element and would be contained in the answer presented to the user.

Watson doesn’t modify the content, so it’s better to cleanse and strip off such elements which doesn’t add any value to the content.

In some cases, based on the content evaluation outcome, some content would need to be modified/structured to aid Watson to understand the content with less amount of training time. If you relate the analogy to a book, a book with index, chapters and sections is easy to read, understand, infer information quickly and interpret, rather than a book with only pages of text.

Here are some of the recommended practices on structuring content –

  • The document should contain well defined sections with section title. For example, in HTML, the section title would be through the standard h1, h2, h3.. tags and content within each of the sections would be treated as the body of the content for those section, for word documents if through style formatting (h1, h2.) and PDF is characterized by FONT sizes or FONT style.
  • Organize and structure content into logical section’s, preserving the hierarchy of the content. Think of this as a book example given earlier, with well-defined chapters and sections.
  • Remove noise from the content, examples include navigation links like headers, sidebars etc. which doesn’t add value to the content. If you are crawling external websites, remove the header/footers, navigation links, and menu to include the actual body of content. Cleansing the content would also ensure the user’s would get the relevant information without the extraneous noise.
  • Identify substitute for image’s (Watson doesn’t process images currently) if important information is being conveyed by images, like flowcharts, process flows which needs to be answered.
  • Identify how structured data i.e. data from tables (i.e. html tables) would be used in the context of the use case.
  • If content source contains PDF documents, try to get to the source of document (if available) as important structured/hierarchy information is lost during conversion.
  • Invest in content (source, organize, structure, cleanse, update) as end users get responses to questions from your content sources.

 

read more
ArticlesCognitive Computing

Sample Watson QA Request Response

Here is a sample raw JSON request for Watson Q/A service.

{“question” : {
“evidenceRequest” : {
“items” : 5
},
“questionText” : “what is ebola?”
}
}

You can use client tools like POSTER in the chrome browser to test out invocations. Given below is a sample invocation request with required parameters.

watson-qa

read more
1 3 4 5 6
Page 5 of 6