return to table of content

GraphRAG is now on GitHub

sansseriff
9 replies
2d23h

I find it interesting that their entity extraction method for building a knowledge graph does not use or require one of the 'in-vogue' extraction libraries like instructor, Marvin, or Guardrails (all of which build off of pydantic). They just tell the llm to list graph nodes and edges in a list, and do some basic delimiter parsing, and load the result right into a networkx graph [1]. Is this because GPT-4 and the like have become very reliable at following specific formatting instructions, like a certain .json schema?

It looks like they just provide in the prompt a number of examples that follow the schema they want [2].

[1] https://github.com/microsoft/graphrag/blob/main/graphrag/ind...

[2] https://github.com/microsoft/graphrag/blob/main/graphrag/ind...

swyx
4 replies
2d15h

regardless how reliable llms are they will never be perfect, so graphrag will error when the RNG gods feel like ruining your day. use pydantic.

darkteflon
1 replies
2d12h

Was looking forward to this. Haven’t looked at the code yet, but yeah - it’s a bit of a red flag that it doesn’t use an established parsing and validation library.

swyx
0 replies
2d3h

i mean dont throw baby out with bathwater. its research code, take the good chuck the bad.

Dowwie
1 replies
2d8h

Have you any insights about whether these python libraries are using extensions for the heavy lifting? Current situation involves layers upon layers of parsing and serializing/deserializing in Python.

swyx
0 replies
2d3h

sorry what kind of extensions do you mean? im not aware of any. this stuff is just boilerplate

yunohn
1 replies
2d11h

Using a library like Instructor adds a significant token overhead. While that overhead can be justified, if their use case performs fine without it, I don’t see any reason to add such a dependency.

sansseriff
0 replies
2d

That's what I was wondering. How the token overhead of function calling with, say, Instructor compares with a regular chat response that includes few-shot learning (schema examples).

Maybe Instructor makes the most sense only when you're working with potentially malicious user data

inhumantsar
0 replies
2d3h

anecdotally i found gpt3.5 was decently reliable at returning structured data but it took some prompt tuning and gpt4 has never returned invalid JSON when asked for it, though it doesn't always return the same fields or field names. it performs better when the schema is explicit and included in the prompt, but that can add a non trivial number of tokens to each request.

ftkftk
0 replies
2d23h

LLMs are very good at knowledge extraction and following instructions, especially if you provide examples. What you see in the prompt you linked is an example of in-context learning, in particular a method called Few-Shot prompting [1]. You provide the model some specific examples of input and desired output, and it will follow the example as best it can. Which, with the latest frontier models, is pretty darn well.

[1] https://www.promptingguide.ai/techniques/fewshot

piotrrojek
4 replies
2d22h

Does anyone know how to run it against Ollama?

piotrrojek
1 replies
2d9h

Replying to my own comment - it's trivially possible to replace `api_base` in settings.yaml. The only problem is that Ollama doesn't support OpenAI style embedding endpoint yet. But looking through PRs to Ollama, this is very close to being merged. All steps before actual embedding work fine with e.g. Llama3-8B.

goofystasy
0 replies
2d5h

You can use litellm with drop_params for embedding endpoint

ftkftk
0 replies
2d20h

You will have to do a fair amount of refactoring.

dweinus
4 replies
3d

If I understand the paper right...

At indexing time:

- run LLM over every data point multiple times ("gleanings") for entity extraction and constructing a graph index

- run an LLM over the graph multiple times to create clusters ("communities")

At query time:

- Run the LLM across all clusters, creating an answer from each and score them

- Run the LLM across all but the lowest scoring answers to produce a "global answer"

...aren't the compute requirements here untenable for any decent sized dataset?

malux85
2 replies
2d23h

It depends on your latency requirements, not every RAG task has a user waiting for an immediate response, for my use case it doesn't matter if an answer takes even 10's of minutes to generate

hirako2000
1 replies
2d21h

Given the cost of running an LLM for 10 minutes, yes it does matter. That's about 15 dollars.

The overall answer better be very good and relevant every time for this to tech to make sense.

malux85
0 replies
2d21h

Oh that’s a good point, I have my own GPU rack and run locally because it’s cheaper to do so, so I hadn’t considered that…

glesperance
0 replies
2d23h

It really depends on the job you're trying to accomplish. I'd venture saying that it's way too early for horizontal / massive scale RAG apps.

Most solutions will want to focus on a very specific vertical application where the dataset is much more constrained. That we're this makes more sense.

Also a lot of alpha in data augmentation.

laborcontract
2 replies
3d

I've been waiting for this.

Knowledge graphs don't replace traditional semantic search, but they do unlock a whole new set of abilities when performing RAG, like both traversing down extremely long contexts and traversing across different contexts in a coherent, efficient way.

The only thing about KGs is that it's garbage-in-garbage-out and I've found my feeble attempts at using LLMs to generate graphs sorely lacking.. I can't wait to try this out.

gkorland
1 replies
2d15h

Indeed, if you build the Knowledge Graph in a "naive" way but just asking the LLM to generate a Knowledge Graph you'll probably end up with a very "dirty" Knowledge Graph full with duplications.

falcor84
0 replies
2d5h

Can this be handled by a second (adversarial?) LLM that searches for bad entries?

Doorknob8479
1 replies
1d19h

Do you plan to introduce the workflow of GraphRAG into txtai?

dmezzetti
0 replies
1d7h

Perhaps some of the entity extraction workflows. But otherwise what's in GraphRAG that's not in txtai?

yard2010
0 replies
2d22h

Excuse me, how is it not?

throwaway4aday
0 replies
2d21h

Faiss is for similarity search over vectors via k-NN. GraphRAG is, well, a graph. More precisely, GraphRAG has more in common with old school knowledge graph techniques involving named entity extraction and the various forms of black magic used to identify relationships between entities. If you remember RDF and the semantic web it's sort of along those lines. One of the uses of Faiss is in a k-NN graph but the edges between nodes in that graph are (similarity) distance based.

Looking at an example prompt from GraphRAG will make things clear https://github.com/microsoft/graphrag/blob/main/graphrag/pro...

especially these lines:

Return output in English as a single list of all the entities and relationships identified in steps 1 and 2.

Format each relationship as a JSON entry with the following format:

{{"source": <source_entity>, "target": <target_entity>, "relationship": <relationship_description>, "relationship_strength": <relationship_strength>}}

michaelnny
1 replies
10h55m

Anyone know how to use GraphRAG to build the knowledge graph on a large collection of private documents, where some might have complex structure (tables, links to other docs), and the content or terms in one document could be related to other documents as well?

gkorland
0 replies
8h57m

This is exactly why we're working on the GraphRAG-SDK to ease the process. You might want to check out https://github.com/FalkorDB/GraphRAG-SDK/ and we would love to hear your feedback

fumeux_fume
1 replies
2d15h

Knowledge graph or just a graph? My fear is that the term is being borrowed to help hype more AI products.

gkorland
0 replies
2d15h

These are Knowledge Graphs, perhaps not RDF but Knowledge Graph can also be based on properties Graphs

a_wild_dandan
1 replies
3d

I am ecstatic that Microsoft open sourced this. After watching the demo video[1], my mind raced with all of the possibilities that GraphRAG unlocks. I'm planning to try GraphRAG + Llama3 on my MacBook, since it has 96GB of unified (V)RAM. I think this tool could be a legit game changer.

[1] https://www.youtube.com/watch?v=r09tJfON6kE

k__
0 replies
2d19h

How does this work?

I understand the issues with "baseline RAG", as I experienced them myself. They make sense to me, if I query "what are the major themes of this dataset" on a list of articles, and the RAG tries to find content that's similar to that query, it probably won't find much.

Why does GraphRAG find something here? What's happening with my query that it suddenly "matches" the dataset?

throwaway4aday
0 replies
2d21h

This is awesome! I've done a lot of little projects exploring the use of graphs with LLMs and it's great to see that this approach really pays off. Stupid me for trying to prematurely optimize when the solution is just prompt engineering and burning a bunch of tokens on multiple passes. Going to give this a try and see if my jaw drops. If it's as good as it looks then I'll have to put in the work to get it out of Python-land.

shreezus
0 replies
3d

This is great - I have been interested in KG-enhanced RAG for some time and think there is a lot of potential in this space!

ravi1krkr
0 replies
23h38m

can we use the Graph RAG with ollama and other opensource embedding models instead of openai and azureopenai

malux85
0 replies
3d

I’m the creator of https://atomictessellator.com

While building the backend of this, I have focused on building a composable set of APIs suitable for machine consumption - i.e to act as agentic tools.

I was looking for a good RAG framework to process the large amount of pdfs I have crawled, so the agents can then design and run simulations. This comes at just the right time! I am looking forward to trying it out

loufe
0 replies
2d19h

I find the choice of the Russo-Ukrainian war an interesting choice of topic as an example. I could see it being an intentional choice as a means to target military data analysis contracts.

justanotheratom
0 replies
2d22h

does it support multi-tenancy?

glesperance
0 replies
2d23h

For those like me that were looking for something more substantial on the GraphRag Method -> https://arxiv.org/pdf/2404.16130

gkorland
0 replies
2d15h

The GraphRAG project is great and really shows the why Vector Databases can provide a full RAG solution when it comes to none trivial search queries. But in order to build a full an accurate Knowledge Graph we found out you need more than just loading the text to LLM.

For that we wrote the GraphRAG-SDK that is also generating a stable Ontology. https://github.com/FalkorDB/GraphRAG-SDK

ftkftk
0 replies
2d23h

I've been looking forward to playing with this since reading the paper. I was considering implementing it myself based on the paper but I figured the code would just be a few weeks behind and patience did indeed pay off :)

darksaints
0 replies
2d22h

LlamaIndex has something called the Knowledge Graph RAG Query engine. Is this related in any way?

bitsinthesky
0 replies
2d21h

So can someone explain how this is different/superior to Raptor RAG? I don't have the current concentration to figure it out for myself...