Of course it has fundamental problems. Likely they are more prevalent then in other techniques, regarding LLM training or otherwise.
But one also needs to acknowledge how immensely useful and powerful the resulting models have become, achieving results pure supervised fine-tuning could likely not have achieved at all. Whether fundamental alignment of pre-trained models is even possible is a very different question.
> Hinton said in a recent interview. “You design a huge piece of software that has gazillions of bugs in it. And then you say what I’m going to do is I’m going to go through and try and block each and put a finger in each hole in the dyke,” he said.
While I agree with the underlying sentiment, that transcription, uh... I change that to "dike".
Anyway, it seems while we can round off a lot of rough edges from an LLM, it's fundamentally so much of a dream-machine that there's always something else, a never-ending game of whack-a-mole.
It gets even worse when it comes to malicious or adversarial stuff, and not just "ensure it doesn't say generate a rude word most of the time."
The variant spellings are all standard English, and, notably, in the British English that Hinton speaks, dyke is a common spelling for a levee built against water. As in, Offa's Dyke, a large earthwork in the UK.
But one also needs to acknowledge how immensely useful and powerful the resulting models have become, achieving results pure supervised fine-tuning could likely not have achieved at all. Whether fundamental alignment of pre-trained models is even possible is a very different question.