From LLM as oracle to LLM as translator - our journey from theory to everyday’s practice in a corporate setting with dmGPT (and python)

One of the biggest challenges of working in such a large organization like dm is finding the information you need to accomplish your tasks: distributed organization units, multiple knowledge sources, and different tools make it very challenging to know where to find information whose location you don’t know. Most of the times, the best way to find something out is to ping a more experienced colleague and ask them. But what if you could ping your AI-Powered copilot and find out? Not only that… What if it also helped you create content for your specific product without you telling it everything about the product? What if it was able to help you write code using internal tools? What if it could help you have an insight of your internal data?

After its first steps in summer 2023, our vision for dmGPT quickly developed to it becoming a truly helpful assistant for every coworker of dm. Since then, we have contributed to the design and implementation of an LLM-powered platform that aims to achieve this goal. To come a step closer, we had to rethink the role of the LLM, picturing it as a translator between natural languages and software systems and back. Now, it helps us map an instruction in natural language to a set of tools needed to accomplish the given task and construct a coherent answer based on the provided data.

In the design we had to face multiple challenging questions, such as:

• How to connect multiple, heterogenic data sources?
• How to pick an LLM for a given task?
• Which LLM do we support?
• How do we build a user friendly, dynamic and configurable user interface?
• How to measure the system’s quality?

In this talk we would like to provide a technical insight to our journey, discussing architectural decisions as well as implementation dilemmas, and engage in a discussion with the community about the steps to come.