Fusion AI

Lucidworks Fusion AI integrates advanced machine learning and artificial intelligence to enable Fusion to dynamically adapt and respond based upon user behaviour.

Through integration with Apache Spark, Fusion offers numerous modes of interaction to deliver next generation application delivery.  Processing of signals generated with Fusion (and potentially other corporate applications), recommendations are developed which ensure that the most appropriate information is presented.  Where content changes, user interaction will allow Fusion to adapt to the content change.

For large organisations, the certified agreement is often a much sought, but hard to find document.  This is because the document itself does not widely use the term certified agreement.  A normal search for certified agreement is likely to have the actual certified agreement as the fourth, or perhaps fifth on the search results.  However, as users click on the certified agreement document in response to the query, Fusion will learn that this is the most relevant document for the query certified agreement.  And, unlike with a best bets based solution, when the new version of the certified agreement document is made available, users will begin to click on the latest version and the latest version of the certified agreement document will then progress to beome the top document on the search results.  This is a simple example of the power of Fusion AI.

Recommenders

Recommenders use machine learning to predict future user behaviour based upon previous interactions.  Amazon, NetFlix and YouTube use collaborative filtering to recommend products/videos that might be of interest based upon previous users behaviour.  Fusion is able to use similar techniques to:

• Increase the relevance ranking (score) of certain documents based upon other users viewing these documents after performing the same search, or,
• Recommend documents for you based upon your individual interaction.

 NLP, NER, Phrases, POS

Natural Language Processing (NLP) gains insights from the individual words contained in a document.  These insights can be classified as Named Entity Recognition (NER), and Part of Speech (POS).  Fusion can use the Apache OpenNLP project to enhance document text for better document searching and analytics.  Using NER, specific names, such as customers, suppliers, projects, divisions or organisations can be extracted from the document text and then used to filter seach results or be presented as additional metadata for result documents.  Part of Speech analysis allows the identification of statistically significant words in documents.

Query Intent and Document Classification

By analysing past user queries and determining the kinds of documents users have viewed, an insight, or intent can be gained from user behaviour.  This intent can then be used to narrow the documents returned in future to assist users in finding the documents they need.  Similarly, documents can be classified allowing users to narrow to specific classes of document being sought.  Through its integration with Apache Spark, Fusion can process indexed documents and user queries to distill insights for subsequent use.

Both supervised and unsupervised classification algorithms are available.  Available algorithms include Random Forest, Logistic Regression, Bisecting k-Means and Word2Vec.  Data distillation algorithms supporting classification include Term Frequency (TF), Term Frequency-Inverse Document Frequency (TF-IDF) and Word2Vec.

Anomaly Detection

Which of these three things are not like the others?  That's the essence of anomaly detection.  Through analysing what is normal or average behaviour Fusion can alert to behaviour, or documents which are anomalous.

Signals and Query Analytics

Through capturing and processing user interactions (called Signals by Fusion), user behaviour can be identifiied and event predicted.  Query Analytics can determine what documents are being sought, and importantly, what documents are being sought which are unable to be found.  This can identify important knowledge gaps in the organisation.

Clustering

Clustering is a technique to group by similarity, items such as documents or users (characterised by queries they issue or documents they view).  Meaningful insights can be gained on users or documents from applying clustering. 

Topic Detection

Algorithms such as Word2Vec can be used to identify topics within documents or user queries which can then be fed back into the query process as part of query intent or document classification.

A/B Testing

A/B Testing provides a framework to evaluate the effectiveness of application changes.  For example, the addition of query intent to your Fusion Enterprise Search environment can be measured to see if users benefit, compared with the existing search activity.  Experiments can be defined and objectively evaluated using the Fusion Experiments API.

Resources

The following resources provide more information about Fusion AI:

• Lucidworks Fusion AI Datasheet
• Lucidworks Understanding Intention
• Lucidworks Flightplan Cognitive Search

Eager to know more?  Perhaps a demonstration is the next step.  Get in touch below.