I have been reading up on how you make LLM-mediated chatbots that do things like TIPPING CRYPTOCURRENCY not hallucinate important values. It's not actually that hard.
Timeline
Post
Remote status
Replies
10
@sun are you looking at the h-neuron paper, the PPO/GRPO for attention fixing, or the self-checking classification stuff? There’s a ton of work in preventing those kinds of things on all levels right now and they’re all awesome
@vii simpler in most cases, I extract certain values from the text before llm processing. I use this to create a whitelist of values. Then I feed the text into a model using ollama that constrains the output to a json command, ensuring it has the right values
@vii this won't work if for example I ask the bot to do some kind of fanct calculation
@vii I bookmarked your post to learn more, thanks
yeah I think you just install OpenClaw and don't give it access to your email, right?
@vic openclaw is very bad for security
@sun even more awesome, and yeah this is what a lot of companies are doing in one form or another for cheap precision. There's edge cases of course, but you'll find those when people use it XD
@vii since I can extract mentions and amounts directly from the post I almost don't even need the LLM, basically I am using it to give me a fuzzy confirmation that it's actually a tip command and not just something that accidentally has those values
@i @vii right now what I mashed up is:
1. extract mention urls from html via a.mention css selector
2. pass the plain text into something that converts english numbers to regular numeric values.
3. regex match and extract the numeric values
4. convert the html to plain text using a few special conversion rules
5. pass that plain text into the llm that has its output constrained to a grammar that forces json in the form of commands for my bot
6. double check and replace values if needed
7. ignore messages that can't be construed as a tip
I am modifying this so it can detect multiple tips in a note and ignores text and mentions unrelated to the tips
1. extract mention urls from html via a.mention css selector
2. pass the plain text into something that converts english numbers to regular numeric values.
3. regex match and extract the numeric values
4. convert the html to plain text using a few special conversion rules
5. pass that plain text into the llm that has its output constrained to a grammar that forces json in the form of commands for my bot
6. double check and replace values if needed
7. ignore messages that can't be construed as a tip
I am modifying this so it can detect multiple tips in a note and ignores text and mentions unrelated to the tips