The human in the loop is the differentiator....

The human in the loop is the differentiator. The top models are all being trained with RLHF: they're using this too! When it comes to productionizing software using LLMs, human evaluation is where you imbue your product with your specialized context. " "" "We need systems to help automate some checks, though, so we can continue to build and scale our products, but having human judgment shape the prompts and workflows you build is critical. " "" "Just like any of the eras of software, there needs to be a balance between manual and automated evaluations, and while we could automate all evaluations with LLMs, some really do benefit from human judgment (at least for now)!

---

Original post on LinkedIn