Knowledge Graphs and Generative AI

Hi everyone, today we are continuing our Graph Databases series and will start by introducing a new concept: Knowledge Graph. 

We will cover what a knowledge graph is, how graph databases are related, what research companies are saying about it, their benefits, and how we can take advantage of them by integrating them into a RAG application. 

First of all, if you didn’t read our previous post, go for it (https://folderit.net/graph-databases/) and then go back here, you will have a better understanding of the basic concepts around Graph Databases that will be useful to understand how they apply here. 

Now, let’s start with “Knowledge Graphs”. 

What are Knowledge Graphs?

Let’s review a few definitions from some well-known industry leaders: 

IBM says that “knowledge graph, also known as a semantic network, represents a network of real-world entities—i.e. objects, events, situations, or concepts—and illustrates the relationship between them. This information is usually stored in a graph database and visualized as a graph structure, prompting the term knowledge graph.”  

Neo4j defines a “knowledge graph is an insight layer of interconnected data enriched with semantics, so you can reason with the underlying data and use it confidently for complex decision-making”.  

On the other hand, Ontotext GraphDB describes that “The heart of the knowledge graph is a knowledge model: a collection of interlinked descriptions of concepts, entities, relationships and events. Knowledge graphs put data in context via linking and semantic metadata and this way provide a framework for data integration, unification, analytics and sharing.”  

In simpler terms, from our perspective, a knowledge graph is a way to model your data with the following characteristics: 

• Data needs to be integrated from various sources. 

• Data is organized in a graph structure, with nodes and edges.  

• The schema is flexible, allowing for easier evolution (LPG/RDF models). 

• You can query this structure using a query language. 

• Data ingestion comes from different silos or sources, such as APIs, SQL databases, datasets, CSV files, Elasticsearch, and other NoSQL databases. 

• It enables you to answer complex questions. 

• It can build a knowledge base where data can be interpreted and new facts can be inferred. 

As you can imagine, one of the most challenging tasks is defining a model that integrates all the existing concepts across your data silos. Fortunately, graph schemas are easy to model and adapt. 

What is a Graph Database?

Please remember to visit our previous post (https://folderit.net/graph-databases/), but if you don’t have too much time now, here’s an oversimplified explanation. 

A graph database stores data in a graph-like structure, where nodes represent entities and edges represent their relationships. This approach offers greater flexibility since it doesn’t rely on rigid table schemas. 

A knowledge graph lets us map all our existing knowledge in one place. 

Here is a simple but powerful example that illustrates how Cypher queries are more readable and simpler than SQL. We intend to list all the friends of Alice’s friends. Note: Cypher is the graph query language used by Neo4j.

This query:

• Starts at the node representing Alice.
• Finds all nodes directly connected to Alice by a FRIEND relationship (Alice’s friends).
• From each of these friends, it finds other nodes connected by a FRIEND relationship (friends of Alice’s friends).
• Returns the names of these friends of friends.

As you can see, Cypher queries are more intuitive, concise, and efficient for querying graph-based data, making them a superior choice for applications that involve complex relationships and traversals. SQL, while powerful for tabular data, can become unwieldy and less performant when dealing with similar graph-like queries.

By using Cypher and a graph database like Neo4j, you can leverage the natural structure of your data, making it easier to build and maintain applications that involve intricate relationships and data interconnections.

The Rise of Knowledge Graphs: Insights from Gartner

Ok, knowledge graphs. Before integrating them with Generative AI, let’s hear what top research companies say about them.

In recent years, knowledge graphs have emerged as a critical technology for businesses, enabling them to map and interconnect complex data in meaningful ways.

According to Gartner, knowledge graphs are one of the 30 emerging technologies that will significantly impact business decisions in the coming years. This designation underscores the growing importance of knowledge graphs in the tech landscape.

Gartner highlights knowledge graphs as a “Critical Enabler” for various advanced technologies and business processes. They allow organizations to consolidate and leverage their existing knowledge, facilitating better decision-making and more efficient data management.

Moreover, Svetlana Sicular, Research VP, AI at Gartner, has pointed out that knowledge graphs have reached the “Slope of Enlightenment” phase in the 2024 Gartner Hype Cycle for Artificial Intelligence. This phase indicates that the technology is becoming more widely understood and adopted, with real-world benefits and implementations becoming clearer. As knowledge graphs mature, they offer practical applications that drive significant value for businesses.

Knowledge Graph Benefits

Last non-techy step I promise, let’s discuss real benefits. Essentially, a knowledge graph offers a more interconnected, flexible, and adaptable structure for data. It’s a valuable tool for integrating diverse data sources and extracting deep insights from them. Here we have a summarized list of the benefits that the KG (Knowledge Graph) offers us:

• Graph Structure: Data is organized into nodes (entities) and edges (relationships), capturing complex and non-linear relationships. This makes it easier to spot and visualize connections.
• Flexible Schema Modeling: Knowledge graphs feature flexible schemas, which can evolve without significant rework. This adaptability meets changing requirements or newly identified relationships, reducing maintenance effort and making updates easier.
• Data Integration: Knowledge graphs unify diverse data sources, offering a single, consolidated view. This means data from APIs, SQL databases, datasets, CSV files, Elasticsearch, and other NoSQL databases can be integrated.
• Query Capability: The structure supports specific query languages (like Cypher), allowing for straightforward extraction and manipulation of data based on complex conditions. Compared to SQL’s nested queries for interconnected data, graph queries are usually more direct and intuitive.
• Data Source Diversity: Knowledge graphs can ingest and harmonize data from various sources, ensuring a comprehensive dataset.
• Answering Complex Questions: With detailed interconnected data, users can explore the data to gain insights and answer complex questions that might be challenging for traditional data structures.
• Knowledge Base Creation: A knowledge graph not only stores data but also interprets it, inferring new facts or relationships from the existing data. This enhances the dataset’s depth and richness over time, employing Graph Data Science algorithms such as centrality, community detection, link prediction, shortest path, and similarity.

Knowledge Graphs and RAG applications

Ok, you may think “all of that sounds great but what about generative AI and RAG applications”? Let’s dig into that.

We will assume that you know a little bit about LLMs and Generative AI, but don’t worry you will also have a TLDR (https://en.wikipedia.org/wiki/TL;DR) on this same section along with a well-explained video.

The majority of the Chatbots based on Generative AI use a framework called RAG (Retrieval-Augmented Generation – https://research.ibm.com/blog/retrieval-augmented-generation-RAG), for retrieving facts from an external knowledge base to ground large language models (LLMs) on the most accurate, up-to-date information and to give users insight into LLMs’ generative process.

If you want some extra background you could take some minutes to watch this video: https://www.youtube.com/watch?v=T-D1OfcDW1M

What does RAG do?

Large language models can be unpredictable. At times, they provide precise answers to questions, but other times they may produce random facts from their training data. If they sometimes appear confused or incoherent, it’s because they lack true understanding. LLMs grasp the statistical relations between words, but not their meanings.

Retrieval-augmented generation (RAG) is an AI framework designed to enhance the quality of responses generated by LLMs. It does this by grounding the model with external sources of knowledge, supplementing the LLM’s internal representation of information.

RAG combines an information retrieval component with a text generator model.

In summary, instead of directly asking the model for an answer, we include some enhanced context coming from an external or internal source that will help the model answer.

Now let’s say that we have a bunch of PDF documents with several technology definitions. When creating RAG applications, we can generally say that there are two general types of questions:

• 1) Questions Answerable by Parsing Content/Documents Metadata: These are questions where the answers can be found within the metadata of the PDFs that our company has mapped. In example:
  • What is artificial intelligence?
  • What is a graph database?
• 2) Questions that are more aggregated or have more than one dimension: In a summary, the answer is not present in our PDFs and we need to traverse our entities, and in some cases group data. In example:
  • Question: Which blog posts by machine learning engineers received the most comments in 2024?
    • Process
      • Select blog posts
      • Filter by published date “2024”
      • Join comments related to these posts
      • Group by blog posts and count the number of comments
      • Order the results by the number of comments
      • Select the top posts
  • Question: What are the trending topics in blog posts by machine learning engineers in 2024?
    • Process:
      • Select blog posts
      • Filter by published date “2024”
      • Join the tags or categories related to these posts
      • Group by tags and count the number of posts associated
      • Order the results by the number of posts per tag
      • Select the top trending topics

In these examples, the first kind of question leverages existing metadata, while the second requires traversing and aggregating data. Knowledge Graphs streamline the process for both types of questions, making complex queries more manageable and the results more insightful.

By integrating Knowledge Graphs, we can enhance our ability to answer complex, multi-dimensional questions efficiently, providing deeper insights and more comprehensive data analysis.

Now, how can we integrate the knowledge graph to tackle this kind of question? There are two approaches that we will cover today:

• Text2Cypher (we will use cypher, but it could be Text2Sparql, Text2GQL, etc)
• Semantic Layer

In this first approach, instead of searching for relevant information with traditional RAG, which involves searching across chunks of relevant text – coming from our documents-, the smart search uses an LLM to generate an appropriate Cypher statement to retrieve relevant information from a knowledge graph. The relevant information is then passed to another LLM call, which uses the original question and the provided information (cypher query response) to generate an answer.

It is important to note that a good prompt should include the question, the graph schema and some cypher query examples.

An example prompt, to achieve the first step could be something like this one:

Generate a Cypher (Neo4j) statement to query a graph database.

Instructions:

• Use only the provided relationship types and properties from the schema.
• Highlight key elements in the query.
• Follow the structured query format for consistency.

Steps:

1. Analyze the input question to extract possible date filters, name filters, topic filters, etc.
2. Review the provided query examples.
3. Construct the query, considering all elements.

Note: Do not include any explanations or apologies in your responses.

Do not include any text except the generated Cypher statement.

Question: {question}

Schema: {schema}

Query examples: {examples}

If we consider the previous two questions, the generated cypher queries from ChatGPT 4o and the corresponding query response look like this:

Which blog posts by machine learning engineers received the most comments in 2024?

Query:

Query response:

What are the trending topics in blog posts by machine learning engineers in 2024?

Query:

Query response:

Now in the last step, we ask the LLM to answer the original question by using the cypher query response:

Which blog posts by machine learning engineers received the most comments in 2024?

In 2024, blog posts written by machine learning engineers at Folder that generated the most engagement included “Deep Learning Techniques,” which led the list with 120 comments. This was followed closely by “Understanding Transformer Models,” which received 105 comments. Other highly engaging posts included “AI Ethics and Responsibility” with 98 comments, “Natural Language Processing in 2024” with 87 comments, and “Advancements in Computer Vision” with 80 comments.

What are the trending topics in blog posts by machine learning engineers in 2024?

The top trending topics in blog posts by machine learning engineers at Folder in 2024 were dominated by deep learning, which emerged as the most frequently discussed subject. Natural language processing and AI ethics also featured prominently, reflecting the community’s focus on both technical advancements and the ethical considerations of AI.

And just like that, by integrating all our relevant existent data into a single knowledge graph and using Generative AI to create a cypher query that answers the question, we can answer the original question by using that query response in a second LLM interaction.

Text2Cypher + Few shot example

Let’s say that you added a few cypher query examples to your prompt but after some time you were able to log and store the top 50 most relevant queries that the users are asking.

Could we include all those 50 query examples in the prompt? Probably not, because models have something called Context Window, which is a limit in the number of words we can send to the LLM.

An improvement that we could implement before asking the LLM to generate a cypher query is:

• Generate the right cypher query for all those 50 most relevant questions that I collected
• Store them in a vector database and index them (and yes, you could also store them in the graph!)

Now every time a new question comes in, before we ask the LLM to generate the cypher query we:

• Search for the top n closer questions to the one the user is now asking
• Collect those n questions along with the related cypher queries
• Include those n questions and related cypher query as part of the cypher examples in the prompt

By strategically storing and indexing the most relevant Cypher queries and selectively incorporating them into the prompt, we can maximize the effectiveness of the LLM without exceeding its token limit. This method ensures that we maintain a high level of accuracy and relevance in the generated queries, ultimately leading to better performance and user satisfaction.

Semantic Layer

Now let us discuss the second approach, a semantic layer. When is useful, how and why.

Knowledge graphs offer a versatile data representation with a flexible schema that accommodates both structured and unstructured information.  While leveraging LLMs to generate Cypher statements offers great flexibility, base LLMs often struggle to consistently produce precise Cypher queries. Thus, we need a more reliable alternative to ensure consistency and robustness.

Instead of having LLMs generate Cypher statements directly, they can extract parameters from user input and utilize predefined functions or Cypher templates based on user intent. Essentially, this involves providing the LLM with a set of predefined tools and instructions on when and how to use them, known as the semantic layer.

The semantic layer serves as an intermediate step, enhancing the accuracy and robustness of LLM interactions with a knowledge graph. It comprises various tools an LLM can use to interact with the knowledge graph, each acting as a function of varying complexity.

An example prompt, to achieve the first step could be something like this one:

Task: Select a function and generate the right input parameters.

Instructions:

• Highlight key elements in the question, such as names or topics.
• Focus on date filters, topic filters, and name filters.

Steps:

• Analyze the input question and select the semantic layer function (from the ‘Function List’ section) that answers the question.
• Identify possible date filters, topic filters, and name filters (people) based on the input parameters.
• Generate a well-formed JSON response with the properties “function_found”, “function”, “message”, and “input_parameters”.
• If no function answers the question, set “function_found”: false.

Note: Do not include explanations or apologies in your responses. Only include the generated JSON structure with the specified parameters.

Function List: {function_list}

Question: {question}

In many architectures and RAG applications, this is often called function calling. It is the main concept or glue that powers the Agent-based systems. You can read more about this on the Open AI Platform:  https://platform.openai.com/docs/guides/function-calling

Those function_list could look like this:

				
					[
  {
    "name": "report_summary",
    "description": "Generates a summary of a specific report based on its title.",
    "input_parameters": [
      {
        "name": "report_title",
        "type": "string"
      }
    ]
  },
  {
    "name": "top_reports_by_region",
    "description": "Lists the top reports read in a specific region during a given timeframe.",
    "input_parameters": [
      {
        "name": "region",
        "type": "string"
      },
      {
        "name": "year",
        "type": "integer"
      }
    ]
  },
  {
    "name": "popular_topics",
    "description": "Identifies the most popular topics covered in reports for a given year.",
    "input_parameters": [
      {
        "name": "year",
        "type": "integer"
      }
    ]
  }
]

				
			

And then, if we have the following question: Which were the top 10 read reports in South America in 2022?

An expected output from the LLM will be the following JSON response:

				
					{
  "function_found": true,
  "function": "top_reports_by_region",
  "message": "Generating a list of top reports.",
  "input_parameters": {
    "region": "South America",
    "year": 2022
  }
}

				
			

We can see how the LLM selected the top_reports_by_region function, and generated also the right input parameters with South America as the region and 2022 as the year.

To leverage the semantic layer effectively, we need to code the corresponding functions, for example, using Python. By doing so, we can utilize the preferred Cypher queries in a deterministic and controlled manner. A simplified example here:

				
					def top_reports_by_region(region, year):
    # Function logic to fetch top 10 reports for the given region and year
    query = f"""
    MATCH (r:Report)-[:PUBLISHED_IN]->(p:PublicationDate),
          (r)-[:WRITTEN_BY]->(a:Author),
          (r)-[:BELONGS_TO]->(c:Category),
          (r)-[:PUBLISHED_BY]->(pub:Publisher),
          (r)-[:TAGGED_WITH]->(t:Tag),
          (r)<-[:CITES]-(cited_by:Report),
          (r)-[:RELATED_TO]->(related:Report)
    WHERE r.region = '{region}' AND p.year = {year}
    RETURN r.title AS title,
           r.views AS views,
           a.name AS author,
           collect(c.name) AS categories,
           pub.name AS publisher,
           collect(t.name) AS tags,
           count(cited_by) AS citations,
           collect(related.title) AS related_reports
    ORDER BY r.views DESC
    LIMIT 10
    """
    results = graph.run(query).data()
    return results

region = "South America"
year = 2022
top_reports = top_reports_by_region(region, year)

				
			

Then, each time a new question comes in, we follow these steps:

·       Ask the LLM to select a semantic layer function: The LLM analyzes the question and determines which predefined function can best address the user’s query and generates the input parameters.

·       Parse the JSON response and extract the function and input parameters: The LLM returns a structured JSON response specifying the function to be used and the necessary input parameters.

·       Execute the semantic layer function: Using the extracted parameters, we execute the appropriate semantic layer function to retrieve or process the required data using the specific Cypher query.

·       Use the semantic layer function response and ask the LLM to answer the original question with that response as the context: The output from the semantic layer function is then provided as context to the LLM, which uses it to generate a precise and accurate answer to the original question.

The function response could look like this:

				
					[
    {
        "title": "The Future of Salesforce in South America",
        "views": 1250,
        "author": "Maria Gomez",
        "categories": ["CRM", "Salesforce"],
        "publisher": "Tech Publishing Co.",
        "tags": ["Salesforce", "CRM", "Enterprise Solutions"],
        "citations": 20,
        "related_reports": ["Salesforce Adoption in Brazil", "CRM Trends in 2022"]
    },
    {
        "title": "Building Mobile Apps with React Native",
        "views": 1180,
        "author": "Carlos Mendez",
        "categories": ["Mobile Development", "React Native"],
        "publisher": "Dev Journal",
        "tags": ["React Native", "Mobile Development", "JavaScript"],
        "citations": 15,
        "related_reports": ["React Native vs Flutter", "Cross-Platform Development in 2022"]
    },
    {
        "title": "Java in Modern Microservices Architecture",
        "views": 1120,
        "author": "Luisa Fernandez",
        "categories": ["Programming", "Java", "Microservices"],
        "publisher": "Code Weekly",
        "tags": ["Java", "Microservices", "Architecture"],
        "citations": 18,
        "related_reports": ["Microservices with Spring Boot", "Java Trends in 2022"]
    },
    {
        "title": "Generative AI: Opportunities and Challenges",
        "views": 1090,
        "author": "Pedro Martinez",
        "categories": ["Artificial Intelligence", "Generative AI"],
        "publisher": "AI Insights",
        "tags": ["Generative AI", "Machine Learning", "AI Research"],
        "citations": 25,
        "related_reports": ["AI in South America", "Generative Models Explained"]
    },
    {
        "title": "Implementing Microservices with Kubernetes",
        "views": 1060,
        "author": "Ana Silva",
        "categories": ["DevOps", "Microservices", "Kubernetes"],
        "publisher": "Cloud Tech",
        "tags": ["Microservices", "Kubernetes", "DevOps"],
        "citations": 22,
        "related_reports": ["Kubernetes Best Practices", "Scalability in Microservices"]
    },
    {
        "title": "Top Java Frameworks for Enterprise Applications",
        "views": 1030,
        "author": "Juan Perez",
        "categories": ["Programming", "Java", "Enterprise"],
        "publisher": "Enterprise Dev",
        "tags": ["Java", "Frameworks", "Enterprise Development"],
        "citations": 19,
        "related_reports": ["Spring vs Java EE", "Enterprise Java Trends"]
    },
    {
        "title": "Scaling Salesforce Implementations",
        "views": 1000,
        "author": "Sofia Rodriguez",
        "categories": ["CRM", "Salesforce"],
        "publisher": "CRM Insights",
        "tags": ["Salesforce", "Scalability", "CRM"],
        "citations": 14,
        "related_reports": ["Salesforce for Large Enterprises", "CRM Scalability Challenges"]
    },
    {
        "title": "React Native Performance Optimization",
        "views": 980,
        "author": "Diego Ramirez",
        "categories": ["Mobile Development", "React Native"],
        "publisher": "Mobile Dev Journal",
        "tags": ["React Native", "Performance", "Optimization"],
        "citations": 16,
        "related_reports": ["React Native Best Practices", "Optimizing Mobile Apps"]
    },
    {
        "title": "Advanced Java Techniques for Microservices",
        "views": 960,
        "author": "Elena Torres",
        "categories": ["Programming", "Java", "Microservices"],
        "publisher": "Java Monthly",
        "tags": ["Java", "Microservices", "Advanced Techniques"],
        "citations": 21,
        "related_reports": ["Java and Cloud Native", "Microservices Design Patterns"]
    },
    {
        "title": "AI-Powered Chatbots for Customer Service",
        "views": 940,
        "author": "Ricardo Sanchez",
        "categories": ["Artificial Intelligence", "Customer Service"],
        "publisher": "AI Today",
        "tags": ["AI", "Chatbots", "Customer Service"],
        "citations": 17,
        "related_reports": ["Chatbot Development", "AI in Customer Service"]
    }
]

				
			

Finally, when we ask the LLM to answer the original question by using the semantic layer response:

Which were the top 10 read reports in South America in 2022?

In 2022, the top 10 read reports in South America showcased a diverse range of IT-related topics, highlighting the region’s growing interest and expertise in areas such as Salesforce, React Native, Java, Microservices, and Generative AI.

“The Future of Salesforce in South America” by Maria Gomez was the most popular report with 1250 views. It explored the evolving role of Salesforce in the region’s CRM landscape, emphasizing its potential and future trends.

“Building Mobile Apps with React Native” by Carlos Mendez garnered 1180 views. This report provided insights into the advantages of using React Native for mobile development, particularly focusing on its application in South America.

“Java in Modern Microservices Architecture” by Luisa Fernandez, with 1120 views, examined the integration of Java within microservices architecture, reflecting on best practices and modern approaches to software development.

…….

These reports collectively underscore South America’s active engagement with cutting-edge IT topics, demonstrating a keen interest in leveraging modern technologies to drive innovation and efficiency across various industries.

Beautiful! As we can see, by implementing this workflow, we ensure that the LLM generates accurate and contextually relevant answers, leveraging the power of the semantic layer to handle complex data interactions effectively.

Conclusion

The integration of Retrieval-Augmented Generation (RAG) with Knowledge Graphs, Text2Cypher, and the Semantic Layer represents a significant advancement in the accuracy and reliability of AI-driven query responses. By leveraging RAG, we can ground language models with external knowledge sources, ensuring that the responses are not only statistically coherent but also contextually relevant and factually accurate. This approach mitigates the inherent unpredictability of LLMs, providing a robust framework for generating precise Cypher queries and improving interactions with graph databases.

Also, incorporating a semantic layer further enhances this framework by acting as an intelligent intermediary. It interprets user queries, selects the appropriate predefined functions, and generates the necessary parameters, ensuring that the generated Cypher queries align perfectly with the user’s intent. This multi-faceted approach, combining the strengths of RAG, Knowledge Graphs, and the Semantic Layer, leads to a more reliable, efficient, and user-centric solution for querying complex datasets, ultimately paving the way for more sophisticated and trustworthy AI applications.

Acknowledgements

I would like to thank Tomaz Bratanic for his valuable contributions to the integration of Knowledge Graphs and Generative AI. His insights have been instrumental in shaping our approach and understanding of these technologies.

Next Post

In our next post on this series, we will delve deeper into the concept of GraphRAG (Graph Retrieval-Augmented Generation) and explore the powerful technique of Graph Patterns. GraphRAG extends the principles of RAG by specifically using LLMS to go from unstructured text to graph schemas. We will examine how this advanced framework can further improve the accuracy and contextual relevance of AI-driven responses.

Additionally, we will explore Graph Patterns, a method for identifying and utilizing recurring structures within graph data. Understanding these patterns is crucial for optimizing queries and enhancing data retrieval processes. Stay tuned as we uncover these advanced techniques and demonstrate their applications in real-world scenarios, pushing the boundaries of what AI can achieve in data-driven environments.

Why you should augment your team with Folder IT

Outsourcing or Augmenting your AI Team with Folder IT professionals is a cost effective solution that does not sacrifice on quality nor communication effectiveness. Our teams are qualified for working with all the latest technologies and for joining you right away.

Request a quote now for outsourcing your project or staff augmentation services to Argentina.

References

https://neo4j.com/use-cases/knowledge-graph/

https://www.ibm.com/topics/knowledge-graph

https://www.ontotext.com/knowledgehub/fundamentals/what-is-a-knowledge-graph/

https://www.gartner.com/en/articles/30-emerging-technologies-that-will-guide-your-business-decisions

https://www.linkedin.com/posts/svetlana-sicular-415549_ai-genai-responsibleai-activity-7208519032365305857-NHR8

https://research.ibm.com/blog/retrieval-augmented-generation-RAG

https://towardsdatascience.com/enhancing-interaction-between-language-models-and-graph-databases-via-a-semantic-layer-0a78ad3eba49

https://medium.com/neo4j/knowledge-graphs-llms-real-time-graph-analytics-89b392eaaa95