This is an automated archive made by the Lemmit Bot.

The original was posted on /r/machinelearning by /u/Seankala on 2024-04-09 03:38:37.

I’m someone who conducted research in NLP a few years ago, stopped and joined industry, and am recently trying to get back on top of things. I’ve taken an interested into RAG-related work and have started reading some papers.

My understanding is that for RAG you have the retriever and the generator. For the generator it seems like using various LLMs is standard but the retriever also seems to be set to using something like BM25 or the DPR that was originally used. I would think that the performance of RAG would rely heavily on the retriever but am also a little surprised to see that there doesn’t seem to be a lot of research being done in that direction.

Am I just mistaken and haven’t looked in the right direction(s)? Or is there a reason why the retriever doesn’t seem to be getting as much attention?

Come to think of it, I haven’t really seen a lot of work being done for encoder models in general.