Manually RLHF-ing a Llama2-13B checkpoint to post is proving hilariously hard
cephaloform · x · 2026-11-10
One user is manually RLHF-ing a Llama2-13B checkpoint so it can post on its own. The attached screenshot shows the classic two-response preference-labeling interface — one reply reads "crunchies were made to be eaten", the other "falls from a tree" — with the human asked which is better. The author's verdict: "this is so hard," a vivid look at how painful hand-built preference data is.
Related event: Hobbyist Attempts DIY RLHF on Llama2-13B to Make It Post by Itself(2 posts)→