This is an automated archive made by the Lemmit Bot.
The original was posted on /r/machinelearning by /u/PMMEYOURSMIL3 on 2024-10-17 19:04:55+00:00.
Hi all,
I downloaded the chat messages from a discord server on AI and they amounted to ~500k messages over 2-3 years. My reason for doing this is that I’d like to extract insights/tips & tricks on the subject that you might not find in a tutorial online (I’ve always found being in discord servers where people help each other to be much more densely informative than reading various blog posts/tutorials).
They amount to around 8m tokens which would cost 1-2$ using gpt-4o-mini, or 20-30$ using gpt-4o, which is pretty reasonable.
However I’m trying to figure two things out:
-
whether I can use a local llm for part of the process. That’d be preferred since while gpt-4o-mini would only cost between 1-2$, that’s per prompt, and I might want to query/process the data in multiple ways.
-
what exactly could I do to extract the most valuable insights? Probably 95% of the chat is just banter but 5% is probably full of useful advice. What sort of prompts could I use? And how would I handle the fact that I’d need to chunk the input to fit into the context window?
I’m open to learning and exploring any new topic to go about this, as I’m excited to take it on as a project to get my hands dirty with LLMs.