Big Data and Cognitive Computing: The Next Industrial Revolution? updates the trends we covered in The Answer Machine, published by Morgan & Claypool last year. This webcast on Jan. 30, 2014 was given to the Cornell Entrepreneur Network, but was open to all. You can listen to the recording at https://cornell.webex.com/cornell/lsr.php?RCID=616468230cc9b30a45ddd07d778325e2.
In updating the book, we found that the nascent trends we discussed in 2012 have quickly exploded. Applications that aggregate information and integrate technologies are becoming common. Task-centered design is almost a requirement. The market, driven by the buzz around big data, and bombarded by information has started to demand what vendors foresaw: there’s immense value in putting together the pieces from disparate sources, and we need help in doing this. IBM’s Watson may have been the first to define cognitive computing, but we see others positioning themselves in this marketplace as the interest grows. We’ll be covering some of these new companies in the months ahead.
During the past year, as we work with vendors and technology buyers, we have found that one of the most difficult concepts to get across is probabilistic computing. Where does it fit in the current IT landscape? Does it replace traditional BI? We expect to explore this topic also in the coming months. Please contact me directly if you’d like to discuss it in depth, or to schedule a briefing for your company. I can be reached at firstname.lastname@example.org.
The open source software movement raises difficult questions for CIO’s:
- Is open source software “free”?
- If not, what are its costs and risks?
- Does using open source software save time in deploying an application?
- What uses are best suited to open source software?
The answer to all of these questions is, unfortunately, “it depends”. Using open source software effectively depends on the type of application and on the expertise of the developers. It also requires the same kinds of trade-offs that are necessitated by any choice of software: how customized does it have to be? How accurate? How scalable? How usable and for which types of users? This is particularly true in the realm of search and text analytics because both of these applications are language dependent, with all the nuances, variety and complexity that language brings.
We find widespread use of open source components by commercial software vendors. They use open source search or text analytics as a starting point. Then they add in the vocabularies, domain knowledge, tools and widgets, connectors to other applications and information stores, process knowledge and user interaction design to create usable and scalable applications that are suited to a specific purpose. We also find sophisticated enterprises with enough skilled developers, computational linguists, and interaction designers using open source software to give them the custom applications they need. There is no doubt that as open source applications have become more robust and the tools to use them have become available that they are an attractive alternative for many enterprises. But are they “free?” Not if you consider the time, labor and expertise needed to make them an integral, useful part of the enterprise stack.
I’ll be chairing a one-day program on open source search software on Nov. 6th in Chantilly, VA, near Washington, DC that will discuss these questions. We’ve invited some major open source search developers from Elastic Search, Sphinx, Lucene, Solr, as well as vendors who have embedded open source software in their products. Practitioners will discuss their experience with developing applications using open source as well. Eric Brown, Director of Research for IBM Watson, which embeds multiple open source products, will give the keynote, and Donna Harman from NIST’s TREC will discuss how to evaluate search effectiveness. Government employees can register for the event free. Others will get a discount on the registration fee by entering feldman2013 when they register.
In addition, we are collecting data on use of both commercial and open source search and text analytics and are hoping that you will fill in our survey. Results will be tabulated, and all respondents will receive a summary of what we find. You can find the survey at: https://www.surveymonkey.com/s/Synthexis
I hope to see you in November.