Log In to continue learning
Data Storage and Queries
22h24m
Graph databases store data within a mathematical graph structure using nodes and edges. Nodes represent your data items, which are typically entities such as people, products, or locations, among other things. And edges represent the relationship or connection between these data items. With modern applications, you might encounter graph databases if you're building a data system for a use case that involves complex connections between data entities. For instance, if you work at an e-commerce company, you might use a graph database like this one to store data about the relationships between users and products. You could, of course, store this data in a relational database like this one on the right, but graph databases treat relationships as first-class citizens, so you can very easily see the relationships between entities using a graph database compared to using a relational database. When you want to query the data in a graph database, meaning you want to query the relationships between nodes and edges, you can traverse the graph structure. For example, let's say you want to recommend to a user the products that were purchased by the friends. Starting with user 1, you can traverse the graph by following the edges labeled friend to identify all the friends of this user. Then for each friend, you can follow the edges labeled purchased to create a list of the products that these friends have bought in the past and that you want to recommend to user 1. If the same data was stored in a relational database like I have here on the right, then to create the list of recommendations for user 1, you would join the friendship table with the purchase table where the friend and friendship matches with the user in the purchase so that the joined results contain rows of users along with their friends and products purchased by their friends. Then you would filter this query so that you would only get information for user 1. Then you could select the distinct products to get a list of the products that you should recommend to user 1. Now you can imagine that if you wanted to recommend even more products to a user, say by considering the products that were purchased by the friends of their friends, then querying a relational database like this would require even more joins, and that can get out of hand very quickly. Aside from product recommendation applications, there are many other use cases for graph databases, including modeling social networks, representing network and IT operations, simulating supply chain logistics, and tracing data lineage. Another not-so-obvious use case is in fraud detection, say in e-commerce transactions. You can build a graph to model the relationship between entities, such as customers, the products they purchased, the credit card used for the purchases, and their IP addresses. Then you can identify suspicious activity by comparing these relationships with known fraudulent ones. For example, suppose that you know that many fraudulent transactions include a credit card number that's already associated with one being used by a new user at a new IP address. Then, by analyzing the relationships between users, credit cards, and IP addresses in this graph, you can flag all users who are using a credit card that's already associated with another user, but now from a new IP address, as being suspicious. You can also use graph databases to create knowledge graphs to connect data from disparate sources for a variety of use cases, like to enhance the accuracy and reliability of chatbots. So, for example, here you could create a knowledge graph to connect product, customer, and shipping data across an e-commerce company. When a user chats with a chatbot, you can search the knowledge graph for relevant information to provide additional context to the underlying large language model through a technique called Retrieval Augmented Generation, or RAG. This technique gives the LLM access to fresh data that's relevant and specific to the e-commerce company itself, improving the query results. We won't be getting into specific generative AI topics in these courses, but it's a very exciting time for generative AI. And as a data engineer, I would encourage you to learn more about these topics by checking out the links to additional resources at the end of this week. And so, with all these potential use cases for graph databases, you need to develop some experience working with them. Nowadays, you can choose from many different graph databases, including Neo4j, ArangoDB, and Amazon Neptune. With these databases, you'll use specialized languages like Cypher, Gremlin, and SparQL to query data. In the lab at the end of this lesson, you'll get to play around with Neo4j and the Cypher query language. Aside from being a graph database, Neo4j also has integrated vector search capabilities commonly found in vector databases. So, before you jump into the Neo4j lab, in the next video, I'll show you a bit more about vector databases and how vector search works. I'll see you there.