Enterprise search: What is it and where is it headed
Docurated, a document management platform provider, published a remarkable compilation of essays. The collection of industry experts’ views appeared in May 2014 as “Enterprise Search: 14 Industry Experts Predict the Future of Search.” What I found interesting is the role or rather the lack of it for enterprise knowledge management. The phrase appears three times in the analysis.
The first reference is made by Steve Nicolaou, principal consultant at Microsoft. A former Fast Search & Transfer professional, Nicolaou “now architects global solutions with SharePoint Search.” He says, “Elements of knowledge management add meaning to queries and results.”
The second reference comes from Seth Redmore, VP of product management and marketing at Lexalytics, a company engaged in sentiment analysis solutions. Redmore, the media and marketing contact for the firm, points out that enterprise search has to provide functionality beyond typical “search,” extending to facets, true knowledge management and multimedia search.
The third reference appears in the essay by Alex Gorbansky, the CEO of Docurated, who asserts, “Docurated [is] a next-generation visual knowledge management platform that solves the information retrieval problem for leading companies.”
Off its rails?
What struck me in this compilation of experts’ opinions is that knowledge management boils down functions like facets and “meaning.” Have these experts sidestepped knowledge management as a relevant component of search and a core feature of knowledge management?
The question I want to consider is, “What is enterprise search?” The Docurated interviews provide a number of interesting ideas. With mainstream vendors thumping on knowledge management and nudging it to the corners of a utility function, has enterprise search started to come off its rails? Search has been an enterprise application for decades. One of the more startling comments in the Docurated compilation makes clear that enterprise search has some built-in friction.
“Enterprise search is a developing industry,” Nicolaou states. “Until recently, much of the core technology remained unchanged since the 1970s and innovation was fairly limited to niche markets. The general purpose enterprise search offerings were fairly similar in technology and scope.”
The innovations he cites include big data, Google and cloud computing. The future is being invented now. He says, “With all the major software houses directing serious R&D toward enterprise search now, the future will bring shorter innovation cycles, continuous user experience improvements, deeper integration with first- and third-party applications and more ETL-like [extract, transform and load] functionality to handle poor quality content.”
Now and in the future
John Challis, CEO/CTO of Concept Searching, a company providing indexing solutions, takes a different approach. He points out that 15 years ago “enterprise search was ready to move beyond simple keyword and Boolean searching.”
He adds, “The future of enterprise search seems destined to continue with simple keyword and Boolean searching, augmented by faceted navigation based on metadata. The main driver for this is the World Wide Web. Virtually every e-commerce website today offers guided navigation based on metadata. When you enter a simple text query into Amazon or eBay or virtually any other shopping site, you see filters for ‘vendor,’ ‘price range,’ ‘color,’ ‘size,’ etc. This ubiquitous model now appears in most of the leading enterprise search products and users immediately understand how a simple text query can quickly be focused to a specific domain by clicking on a metadata filter.”
Lexalytics’ Redmore points to the impact of open source search solutions such as Lucene. He sees an opportunity in relieving the user of the need to formulate a query. He says, “Why should you have to ask first? Search has been traditionally driven by the searcher (duh), but interesting projects that allow for integrated understanding of where/what the user is doing allows for proactive intervention. Why wait for the slow brain to catch up to the fast machine, when the machine can push out what the user needs right then? Yes, this is functionality you’re starting to see with Google and other companies, but there are certainly interesting use cases for the larger enterprises, particularly internally to start, and then helping customers as they grow in their relationship with the company.”
I would characterize this view as “search without search,” an approach that I interpret as the Google self-driving automobile method applied to enterprise information retrieval.
Jim Jackson, product manager at search appliance producer MaxxCAT, highlights vertical applications of search. The devices allow for rapid deployment and easy scalability. His view is that of a professional who wants a user to have access to needed information. His view of the future is aligned toward systems that are capable of “returning finer-grained results that are not documents, but the exact sentence, the exact spreadsheet cell or exact information the user is looking for.”
He adds, “To facilitate our vision of implicit, contextual search, MaxxCAT is continuing to enhance our API to allow people and machines to leverage the tremendous search and performance capabilities of our information platform so that solutions can be built that connect people to the information they want.”
David Murgatroyd, VP of engineering at Basis Technology, provides language components to search vendors. He perceives the enterprise search sector as “spurred on by both commercial needs and pressing governmental security challenges.” Analytics play an ever-larger role. Addressing the future, he notes, “Search will be increasingly entity-centric and collaborative.”
Murgatroyd says, “The user and the system collaborate best when they do so around a shared inventory of real-world entities. Analyzing those sometimes ambiguous entities accurately is best done with the added information provided by rich collaboration.”
Nick De Toustain, director of sales at LTU Technologies, points out, “Clearly there’s value to be had in being able to sort through a company’s disparate data sources and making them accessible so as to deliver actionable business intelligence. The questions for an enterprise search provider then become: How easy is your solution to implement?”
LTU Technologies, a vendor of image matching systems, takes a different posture toward enterprise search, specifically: “Whether it’s websites or social media, everything is becoming increasingly visual. There’s a corresponding need to ‘make sense’ of all that online imagery, which is where image recognition technology comes in. Future enterprise search tools will need to include image recognition capability to keep up with the massive amount of imagery being tweeted and posted every second.”
Otis Gospodnetic, founder of Sematext, places enterprise search in the context of “search of one enterprise’s content ... or large-scale search.” He singles out Elasticsearch as an open source search vendor that can solve big data problems. For him, the future is “not really about pure search any more.”
Gospodnetic says, “We’ll see full-text search embedded in more applications and devices. We’ll see the line between non-search and search software and servers blurred even more. I suspect we may see people putting query languages with familiar syntax, such as SQL, on top of search engines to enable people to write powerful queries more easily while hiding the original query syntax people typically use with search engines today.”
Edward Ross, a solutions architect at the e-commerce search vendor Exorbyte, stresses a growing awareness among customers that “accessing data sources like address and product data often requires a different solution than classical full-text enterprise search solutions.” He views the future as including “more semantic understanding of both content and queries ... I see search engines handling more business process automation tasks based on the search index.”
Jordi Prats, CTO of Inbenta, approaches enterprise search from the perspective of artificial intelligence and natural language processing. He has learned that enterprise search “is often underestimated.” He adds, “Nowadays, the market is flooded with solutions that try to efficiently tackle enterprise search (sometimes enterprise CMS and search at the same time), but more often than not companies end up facing costly integrations which only provide partially effective results.”
Looking toward the future, Prats predicts, “On one hand, the future of search goes through natural language processing ... while on the other hand, it’ll entail the capability of providing advanced information analysis during indexation time ... The future of search still remains an enigma.”
The pace of change
John Felahi, chief strategy officer for Content Analyst Company, views enterprise search based on some of his experiences working at Microsoft. Content Analyst offers content processing technology based on technology originally developed by SAIC (saic.com) for use in selected U.S. government projects. Content Analyst licenses that technology to third parties and provides other services to organizations worldwide. He points to progress made by enterprise search vendors in machine learning and visualization. Looking toward the horizon, he senses that “search is following the path of business intelligence (BI) in terms of visualization, but hopefully search gets there at a much faster pace.”
For Jones, the future will merge Web search and document search. He says, “The information is out there … on Facebook, in public records, on LinkedIn. Search will be harnessing these data signals and driving much stronger insights based on big data patterns and predictions.”
Simon Bain, CEO of SearchYourCloud, has a background in extensible markup language, project management and security for use on interactive digital television employed for voting in the United Kingdom. His view of enterprise search is that it is “an industry in flux.” The problems he identifies in enterprise search include “the complexity of the interfaces and the lack of any real relevant information being returned.” The result is that users are moving to “more relevant applications and the fragmentation of search once again.” When he peers into the future, he responds to search becoming “more personal.”
Bain says, “With users being able to add and delete their own search sources, true federation will come in to play, and the ‘super index’ will start to take a back seat to ‘click-time’ information access. This change will mean that users gain power to control their own results, bringing in cloud stores, internal applications, such as CRM and doc management, as well as pulling in external non-corporate content from websites, such as Linked-In, Facebook and other social networks. This will then give the user a 100 percent view of their data and information points.”
DtSearch, a Microsoft-centric search vendor, provided information without attributing the company’s viewpoint to a particular person. dtSearch points out that “enterprise users expect instant concurrent search of all content-based data applications.” For this company, the future pivots on “cutting-edge implementations by independent programmers ... using extensible APIs.”
Docurated’s Gorbansky stresses that “enterprise search has had to redefine itself once again.” For him, the future makes enterprise search “essential.” Looking toward the future, he says, “Enterprise search will become instant and intuitive, paving the way for increased productivity across the enterprise.”
Several observations occurred to me as I worked through the compilation of expert opinions in “Enterprise Search: 14 Industry Experts Predict the Future of Search”.
First, the confusion about what enterprise search is characterizes the experts’ approach to findability. A knowledge management professional would set about gathering other writings by these individuals and attempting to provide a context for their “information” and opinions. Without a knowledge framework, the collection of opinions is confusing.
Second, the selection of companies represented provides a wide spectrum of starting points. The inclusion of search engine optimization experts mixed with vendors of primary systems and component vendors provides a surprising consensus. Automation is likely to be more important with each passing day. Also, users want to be relieved of the burden of formulating a query or will be given systems that reduce the user’s dependence on keywords and formal queries. The approach is likely to be given considerable attention because automation reduces some of the costs associated with finding information. Will automated search provide knowledge management systems with appropriate inputs? If not, perhaps the discontinuity between enterprise search and knowledge management becomes another challenge for both disciplines to resolve.
Third, the vendors, with the sole exception of LTU in Paris, focus on text. The data about the volume of content by file type is not definitive. The need to be able to search audio, images and video within an organization is increasing. Videos posted onYouTube, Vimeo or other file sharing systems are proliferating. IBM creates big data podcasts each week, distributing them viaApple iTunes. Videos about search, content processing and analytics systems are key parts of the marketing efforts of Attensity,MarkLogic, Oracle and other firms. The future of enterprise search is more than text. The Docurated analysis makes clear that enterprise search vendors and experts may be their own worst enemy.
There may be some challenges for enterprise search in the organizations of tomorrow. Without innovation, enterprise search is likely to find itself marginalized as enterprise knowledge management solutions proliferate. Search without search may be shorthand for who needs old-fashioned search?