Productionizing LLM-based applications takes a lot more work...

Productionizing LLM-based applications takes a lot more work than most people realize. It depends on how much polish you want and how responsive your product will be to new models coming out, but ensuring consistency in outputs requires validation, which can leverage LLMs in and of itself. However, all LLM usage requires careful design and testing. " "" "Furthermore, it requires designing around flexibility, so different (and new) models can be tried to improve/speed up/lower costs for you. Having a human in this loop is currently necessary, and just like Google employs data annotators to ensure high-quality search results, the best LLM-based applications will build highly effective and robust systems that still include humans in the loop for the foreseeable future. " "" "In certain validated cases, the human can be replaced by an LLM judge, but the LLM judge needs to be validated, too! As these models continue to get better, there will be areas where more can be handled by LLMs, but it will take a fair bit more time before fully unsupervised systems perform at the quality level we truly want.

---

Original post on LinkedIn