Category Archives: Natural Language Processing (NLP)
This week, a research group asked me, “For cognitive computing to take off, what advances do you think we need to make the in next five years?” I answered the question, first listing the major components of a cognitive system, and then discussing which ones were still fairly primitive. But the question continues to haunt me. The fact is that we’ve had most of the components for cognitive computing for a very long time. Language understanding, machine learning, categorization, voting algorithms, search, databases, reporting and visualization tools, genetic algorithms, inferencing, analytics, modeling, statistics, speech recognition, voice recognition, haptic interfaces, etc., etc. I was writing about all of these in the 1990’s. As hardware capacity and architectures have advanced, and our understanding of how to use these tools has evolved, we have finally been able to put all these pieces together. But the fact is, that we have had them for decades.
Here’s what we don’t have: an understanding of how people and systems can interact with each other comfortably. We need to understand and predict the process by which people interact to question, remove ambiguity, discuss and decide. Then we need to translate that process into human-computer terms. Even more, we need a change in attitude among developers and users. Today, we tend to think about the applications we develop in a vacuum. The human initiates a process and then stands back. The machine takes the query, the problem statement, and processes it, spitting out the answer at the end. Users, because of their expectations that machines will not be information partners, helping the information problem to evolve and then finally be resolved.
That’s not the way a human information interaction happens. If two people exchange information, they first negotiate what it is they are going to discuss. They remove ambiguity and define scope. They refine, expand or digress. This process certainly answers questions, but it does more: it builds trust and relationships, and it explores an information space rather than confining itself to the original question. That’s what we need to improve human-computer interactions: first, help in understanding the question. Then, we need better design to enable that question to evolve over time as we add more information, resolve some pieces and confront more puzzles.
By their very nature, good mobile applications must be smarter. The physical limitations—the small screen, or the input mechanisms (1-2 fingers or unpredictable voice recognition) mandate that a mobile app anticipate what you want to do and make it easy to get there. No drop down boxes, no multiple queries, not much scrolling. Certainly very little clicking to get to a new screen. Or a chain of multiple queries when the first one is off the mark. Forget cut and paste. For these reasons, mobile applications must be both smarter at understanding what you want and intelligently designed. That’s hard.
Enter cognitive computing. If an application can really understand what the user intends, if it can classify questions and predict the kind of answer or action needed, then there will be less burden placed on the user to adapt to the limitations of the app. But cognitive computing requires real language understanding (NLP) as well as machine learning and classification. It also requires a corpus of examples to learn from. This is a level of technical prowess that would be impossible to develop for most start ups. Enter IBM’s Watson. Watson Foundations was released last quarter. And now IBM has announced the Watson Mobile Developer Challenge. This contest invites app developers to submit a proposal to develop an application on the IBM Watson platform. Developers must make a case for what they propose, demonstrating why it would be valuable. Winning apps will capitalize on Watson’s strengths:
- Have a question and answer interaction pattern, with questions posed in natural language
- Draw on mostly unstructured (text) information for answers
- Return answers that are ranked according to their pertinence to the question
- Benefit by better understanding (analysis) of the type of question being submitted
The catch is that applications are due by March 31st.
This contest brings cognitive computing within the reach of developers. Watson supplies the NLP tools, question analysis, machine learning, and confidence scoring that would otherwise place cognitive computing beyond the reach of most vendors. For more information, see IBMWatson.com. The application and rules can be found at:
Big Data and Cognitive Computing: The Next Industrial Revolution? updates the trends we covered in The Answer Machine, published by Morgan & Claypool last year. This webcast on Jan. 30, 2014 was given to the Cornell Entrepreneur Network, but was open to all. You can listen to the recording at https://cornell.webex.com/cornell/lsr.php?RCID=616468230cc9b30a45ddd07d778325e2.
In updating the book, we found that the nascent trends we discussed in 2012 have quickly exploded. Applications that aggregate information and integrate technologies are becoming common. Task-centered design is almost a requirement. The market, driven by the buzz around big data, and bombarded by information has started to demand what vendors foresaw: there’s immense value in putting together the pieces from disparate sources, and we need help in doing this. IBM’s Watson may have been the first to define cognitive computing, but we see others positioning themselves in this marketplace as the interest grows. We’ll be covering some of these new companies in the months ahead.
During the past year, as we work with vendors and technology buyers, we have found that one of the most difficult concepts to get across is probabilistic computing. Where does it fit in the current IT landscape? Does it replace traditional BI? We expect to explore this topic also in the coming months. Please contact me directly if you’d like to discuss it in depth, or to schedule a briefing for your company. I can be reached at email@example.com.