Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
RLHF Is Cr*P, It's a Paint Job on a Rusty Car: Geoffrey Hinton (officechai.com)
16 points by gitremote 8 months ago | hide | past | favorite | 6 comments


Of course it has fundamental problems. Likely they are more prevalent then in other techniques, regarding LLM training or otherwise.

But one also needs to acknowledge how immensely useful and powerful the resulting models have become, achieving results pure supervised fine-tuning could likely not have achieved at all. Whether fundamental alignment of pre-trained models is even possible is a very different question.


Related: 'Fine-tuning on human preferences is a fool’s errand'

https://lifearchitect.ai/alignment/


> Hinton said in a recent interview. “You design a huge piece of software that has gazillions of bugs in it. And then you say what I’m going to do is I’m going to go through and try and block each and put a finger in each hole in the dyke,” he said.

While I agree with the underlying sentiment, that transcription, uh... I change that to "dike".

Anyway, it seems while we can round off a lot of rough edges from an LLM, it's fundamentally so much of a dream-machine that there's always something else, a never-ending game of whack-a-mole.

It gets even worse when it comes to malicious or adversarial stuff, and not just "ensure it doesn't say generate a rude word most of the time."


> uh... I change that

The variant spellings are all standard English, and, notably, in the British English that Hinton speaks, dyke is a common spelling for a levee built against water. As in, Offa's Dyke, a large earthwork in the UK.


See also: Dick Van Dyke, which is another example of a perfectly normal usage without any possibilities of euphemistic interpretation.


As lampooned in Austin Powers: https://www.youtube.com/watch?v=CpiP_jN1Pv4




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: