Podcast: Play in new window | Download
Subscribe: Apple Podcasts | Spotify | Amazon Music | Android | Youtube Music | RSS
The promise of agentic AI is being realized in systems like the Service Copilot that Zeiss microscopes provides for its field service engineers.
The system integrates technical documentation, subject matter expertise, and user-generated insights which are orchestrated and shared with a suite of AI agents.
While it relies heavily on modern LLM technology, it’s the system’s solid knowledge graph and metadata foundation that make it a success.
We talked about:
- Max’s work “turning information into value” at PANTOPIX, a technical documentation and information processes consultancy based in Germany
- a recent client project working with Zeiss to help their field service engineers operate more efficiently
- how their prior knowledge management and machine learning work helped them not only cope, but thrive, at the arrival of ChatGPT and LLMs
- the immediate positive stakeholder feedback they received as they incorporated LLMs into their knowledge architecture
- how they extended the iiRDS standard with a custom ontology and taxonomies and integrated topic mappings into their system and workflows
- an overview of the system architecture and tooling, which includes both a graph database and a vector store, an ontology and taxonomy management tool, and documentation of best practices
- their evolution from simple prompt engineering and RAG approach to an agentic orchestration architecture
- a few of the agents in their architecture:
- a planning agent that organizes and orchestrates
- a content agent that replaces the original RAG system
- a troubleshooting agent which surfaces past solutions
- the good problem they experienced of managing enthusiastic user adoption of the new system
- the unexpected benefits to the Zeiss sales team of the system
- how subject matter expertise, user generated content, and other insights are captured and used
- the crucial role of knowledge management practices, structured content, and semantic technology in building the foundation for an organization’s AI capabilities
Max’s bio
Maximilian Gärber is Partner and Principal Technical Consultant at PANTOPIX. Max has been working in the field of technical communication for over 15 years.
As a Partner and Technical Consultant at PANTOPIX, he is responsible for the technical consultation and implementation of projects. In addition to project management, Max is responsible for data modelling and process optimization in relation to product information (migration, publication, translation) and product catalogues. He is also responsible for product development and ensures that innovative solutions for our customers are continuously developed and optimized.
Connect with Max online
Resources mentioned in this episode
- Industrial Knowledge Graph meets Agentic AI: Service Copilot at ZEISS RMS slide deck
- Service Copilot from ZEISS article
Video
Here’s the video version of our conversation:
Podcast intro transcript
This is the Knowledge Graph Insights podcast, episode number 45. When you’re a field service engineer dealing with both the typical challenges of information overload and the need to maintain complex machinery like a high-end Zeiss microscope, you’d really benefit from an intelligent knowledge management system, one that integrates technical documentation, subject matter expertise, and user-generated insights. That’s exactly what Max Gärber has built – an agentic AI system grounded in a solid knowledge graph foundation.
Interview transcript
Larry:
Hi everyone. Welcome to episode number 45 of the Knowledge Graph Insights podcast. I am really excited today to welcome to the show Max Garber. Max did a really interesting presentation at the Semantics conference in Vienna last fall, and I’ve been trying to get him on the show ever since. So here he is. I’m excited to have him here. Max, he’s a partner and a technical consultant at PANTOPIX, a consultancy based here in Germany. Welcome, Max. Tell the folks a little bit more about what you’re doing these days.
Max:
Yeah, thanks Larry. Thanks for having me. Yeah, great show. And yeah, we are mostly concerned with helping mainly our industrial customers structure their content and integrate it from various sources into their systems, delivery systems, wherever it is needed. So yeah, it’s mainly consultancy on data modeling, on how to do information processes and how to get the best out of your data, so to say. So our mission here is literally turning information into value.
Larry:
Oh, I love that. That’s a great tagline for a consultancy. Well, you did the use case, the case study you talked about in Vienna was really interesting to me. This issue of Zeiss microscopes, in particular their research microscopy solutions arm, which is these big, expensive, complex machines that require a lot of service. Can you talk a little bit about how you got involved with Zeiss and what you do to help them? In particular, the thing you talked about in Vienna was about the system to help their field service engineers. Can you talk a little bit about that project?
Max:
Yeah, exactly. The main objective there was helping the field service engineer to get the information in that situation when they need it and in the format they need it. That is essentially the bottom line of it. And it started essentially as a knowledge management project. Zeiss, RMS, they have been really into structuring, getting structured content, adding proper metadata to it so it can be used in various cases. The idea has been to integrate from various sources, spare part system, for example, or the manuals from the technical documentation or ticket information and get them into one system so there’s a single point of access for the service technicians. So they don’t need to spend a lot of time in all of the different systems that there are to get the information about that case they are currently working on because there’s a lot they need to consider when servicing or troubleshooting a microscope.
Max:
And yeah, that project evolved into what is now the Service Copilot because I think it was in early ’22 when we started the project. And one part of it was to not only integrate all of that information in one place, but also recommend content to the service technician. So, if you were working on a specific case, so the ticket was known, the product was known, you should get a recommendation of articles, “Hey, this is how you install this and that component,” for example. So we actually worked a lot on labeling tickets. We actually had a custom labeling interface and used, let’s say, classical machine learning approaches to get that recommendations done.
Max:
And it worked not so good, but that was also the same time when GPT, I think it was 3.0 or 3.5 came out. And yeah, we were faced with that situation that there was a new technology available that looked like it could do everything and much more what we were currently doing without much effort. So we really faced the situation there to either stop the project or reinvent ourselves, I would say.
Larry:
I love that juncture. We were talking a little bit before we went on the air about you were really concerned at that point as this arose, but then it turns out that the prior work you had done, the knowledge management work you had done and the machine learning skills and workflows and things you developed, it turns out you ended up being, to my mind, it looks like from that demo I saw in Vienna, at the leading edge of hybrid AI architectures and agentic AI.
Max:
Yeah, I mean, totally. It evolved really quickly. At the point where we looked into GPT and what language models could do, we asked for, “Hey, can we do some quick prototyping research on this and see if we can replace, let’s say, the machine learning pipeline that we had with language models?” And it worked really well from the start. So in the beginning, we had 15 service technicians as pilot users that were constantly evaluating the system and giving us feedback, “Hey, that’s good, that’s not good.” And they said immediately, “Well, this is working really well.” I mean, they tried, of course, at the very beginning to trick the system and ask the hard questions. And if you look at the content that they are provided, a service manual, it has hundreds of pages and the products that they are servicing, they look quite similar, but they are quite different.
Max:
So there’s a lot of variants in what components you can use, how you configure the system, how you buy it. So it’s really important that if you have a certain product variant, you don’t mix that up. And if you look at how the content is, it is very similar. So of course they have the same structure or a very similar structure and certain, let’s say, chapters or topics, they are always very similar. So how you install electron microscope A is very similar to how you install electron microscope B, but it’s the little differences that are really important if you are doing that installation procedure. If you forget one of those steps, of course, you will fail or you could even do some harm to the system. So it’s really important that you not only have similar content or similarity in, let’s say, the retrieval of the content, but you can actually know, “This is content for product A and this is content for product B.”
Max:
So all of the work that went into structuring the content, adding metadata to each of the topics and connecting the metadata based on what entities are linkable, the RAG system that we implemented then, it could actually filter out all of the content that was not relevant to the specific question or use case. So the answers were quite good from the beginning.
Larry:
Yeah. I want to elaborate a bit on the evolution of your RAG architecture, and for folks who don’t… I think everybody knows at this point, but I’ll just say that RAG stands for Retrieval Augmented Generation, which is a really common practice in AI architectures these days. But one of the things I’m really curious about is you mentioned the common formatting and structure of these documents you’re drawing on. And you mentioned in an earlier conversation or maybe in your presentation that you relied on the iiRDS standard and the metadata that you developed through the machine learning stuff.
Larry:
Can you talk a little bit about how all that comes together? And I guess I’m also curious about that transition from old-fashioned machine learning to the LLMs. Had you done things that hindered things or did it all turn out to be pretty beneficial to the evolution of the system?
Max:
I mean, all that we did upfront, except maybe the labeling of tickets, that was very beneficial to what we did afterwards when we introduced the LLM. And standards like iiRDS, for us, it’s the baseline. We are looking at a technical product, a complex product industrial context. So iiRDS brings already quite some of the entities that you would typically see there. I mean, iiRDS is both. It is an ontology, so it defines entities like topic or product, for example, and it is also a taxonomy. So it provides, for example, concepts or categories if you want for what document types do you know? There is a manual, there is a service bulletin, for example, but there’s also a data sheet, for example. So it gives this taxonomy as well.
Max:
And what we have done, and what you usually also need to do then is, of course, there’s a little bit more, which is not in the standard. It doesn’t cover everything that you would typically see in your use case. So we added or extended iiRDS and added also parts of our, let’s say, custom ontology or customer specific ontology that covered the parts that were not in the standard. And using these ontologies and the taxonomies that are based on them, you would add all of the metadata to the content sources or map them. Either the authors while creating the content would label the information based on the taxonomy, or if you are mapping, for example, from a system that gives you structured information like SAP, for example, there would have been specific mapping and say, yeah, “If it is called an SAP in this way and has this SAP category, then this is in the taxonomy, I don’t know, family X, Y.” And now we have it integrated in the general data model that we then use for everything else.
Max:
So after that, bringing all this together in a single source of truth, in our case, it’s simply a graph database, a knowledge graph, bringing that all together. And from there on, only querying it and using it based on the general data model. That is all of the work that went into it.
Larry:
And you mentioned, and is this the main resource in your new RAG architecture? Is that knowledge graph? Yeah, I guess maybe just talk a little bit about high level overview of the architecture we’re talking about here.
Max:
Yeah. I mean, the architecture consists of, at the core, it’s the graph and the graph DB. We also used an ontology management tool to create the anthology and offer the taxonomy, but this also goes into the graph DB in the end. So all of the instances, classes, and concepts, they all meet, let’s say, in the graph DB in the end. And next to it, we also build a vector store for the similarity search and of course to have the embeddings and support the chat, the natural language interaction with that data. And both, so the graph DB is based on the metadata model and the RAG vector store it would also have these metadata fields. So, yeah, you could use both. And depending on what query was being processed, you would either ask both the vector store and the graph DB or maybe only the graph DB if you you know you’re just searching for a specific entity based on some condition, then of course that’s easy to do by just querying the graph.
Max:
And main benefit of that is also if you are retrieving information directly from the graph, you know it is unaltered. If you are going through the RAG and through the LLM, of course you would always have to have this 15%, maybe even a bit more, sometimes 20% where your result could be, of course, altered in a way that you have not intended. So that’s also limited on that end if you not purely rely on the answer from what the RAG system gives.
Larry:
I’m curious, how does the system decide? If you get to that point of, “No, I just want to get the actual fact out of that graph database.” Versus, was there an optimization process or something in figuring out what generated the best results?
Max:
Yeah. I mean, essentially it evolved. We started with, let’s say RAG plus metadata system, and we mainly had the source of the user manuals, service manuals in topics and some other technical data that we would use, but soon we got new sources into the system. So we wanted to include best practices, for example, or solution nodes, so what worked before? So what did a service technician do, for example, that solved a certain issue? So adding more sources brought us into the, how do we orchestrate this essentially?
Max:
In the beginning, it was just done inside of the prompt. So the prompt would have knowledge of the ontology mainly and could decide what parts of the graph, for example, to query based on does the question fit to the entities that I have in the ontology here, but that quickly turned out to be too static. So we needed to get into the process of allowing more flexible orchestration. And that is where we started to look into the agentic or agents as an architecture and converted basically what had been initially the prompt plus RAG approach into an agent-based approach.
Larry:
We talked a little bit before we went on the air about the number of agents you have, but there’s two that really seem to rise to the top in your hierarchy of agents, the planning agent and the content agent. Can you talk a little bit about those two? And then maybe if it’s relevant, some of the other agents in your architecture? And I guess is the planning agents the sort of orchestration mechanism?
Max:
Exactly. Yeah. I mean, at that time we were reading a lot of papers, of course, and how do you approach such a thing and RAG only solutions, it was all the rage about, “You need to do chain of thought or you need to do this and that.” And we looked into, hey, how can we implement these ideas, but in a little bit different way, because we had not only RAG, we also had, of course, the knowledge graph. So the planner agent is indeed the one that knows the other agents. Each agent defines itself, it describes itself, and says what content, for example, it can fetch or what type of questions by really giving examples, it can answer. And it has knowledge of the ontology. So from the, let’s say, simple prompt approach, the ontology moved into the planner agent. And so it is really able to select one or multiple of, it could be multiple agents needed to answer a question, select the agent based on what the agent is capable of, and what it knows from what entities exist in that domain it is currently in.
Max:
What used to be the RAG system was converted into so called content agent. So the agent that was capable of retrieving most of the service documentation or even transcribed videos of some examples of a service procedure or other information like some technical data sheets or even marketing information. So it’s really one of the bigger agents in terms of capabilities, but that is also how we introduced more and more sources.
Max:
The troubleshooting case is quite interesting. So there’s also a troubleshooting agent, and that is the one that knows about past solutions. If a ticket has information in it, how that issue was solved, we have that information and we can compare a current question, which could be related to, or mostly is related to a current ticket, to what past similar issues were solved. And then this agent can give ideas about, “Hey, did you try this and that? Others have done this.”
Max:
And just give a summary of steps you can consider in the troubleshooting case, which is really different from the content retrieval because if you know what to do, but not how, you would just search for, “Hey, give me the steps from that manual.” But if you are still questioning, “Is it this issue or something else?” The troubleshooting agent is really better suited. And by giving also examples of how questions look like in this or the other case, the planner agent coming back to that can then differentiate and say, “Okay, this looks more like I need to use the troubleshooting agent.” And get the answer from that one.
Larry:
Nice.
Max:
There’s more agents. I don’t know if you want to know-
Larry:
No. And I want to make sure, I would love to talk about all the agents, but a couple things about that. One, I want to just do a little check-in here back to the, you mentioned you had 15 pilot field service engineers kind of working with this. You mentioned in your presentation in Vienna that you had 1000 customers, 1000 users at that point. What was the evolution of this? How long did it take you to go from 15 to 1,000? And I assume you grow over 1000 by now?
Max:
Much sooner than we wanted. That was actually the, yeah, let’s say the unexpected part since it worked so well and it was a much easier access to all these different types of information and sources that they wanted, the pilot users just said, “Hey, can we not just use it now? It’s good enough. We see how it works.” We also implemented a, of course, feedback system, so you could always rate answers and flag, for example, wrong answers to see, “Okay, is this something that we can improve in a systematic way or is it a one-off?” And similar things. So they wanted to use it right away. And yeah, after we scaled up, of course, a little bit of the backend, it went into production quite quickly.
Max:
I mean, it needed to pass quality gates at size. So that’s really great at size that they have policies there, how an internal software, what this now is, can be implemented in the company. But once you have passed that quality gate, all the security checks, like the Workers Council was involved really early on because of course you could say, “Yeah, what can you know about your users?” Then if they use such a system, for example, but once we had all these concerns solved, it went into production, after, it was early ’23, I guess, a year after we started. And from then on, we increased it in batches.
Max:
So the first batch, I think it was 200 and something users that we allowed in the system, and we actually had to implement a wait list there because they told colleagues, “Hey, we have this new system, here’s everything in. It will just answer you directly.” And yeah, we had to work quite hard to onboard all of the users in a very short time. And yeah, I think after another six to eight months, we had the total number of users, maybe short of 1000, but yeah, that’s right about the number, potential users. And yeah, of course that’s not the daily active users, but there are hundreds of daily active users, so it’s really in use.
Larry:
Is that 1000, is that sort of the cap, that’s how many field service engineers there are? Is that sort of…
Max:
It’s the ballpark number, you could say that.
Larry:
I’m loving this and I would love to get into some of the… I mean, do any of the other agents really jump out at you as really an interesting innovation or something that you’d like to talk about? Or the other thing I was curious about is you mentioned the wait list and the adoption of this. Have other people at Zeiss or elsewhere seen this and go like, “Hey, I want that.”
Max:
Yeah, actually they did. And one of the, let’s say probably also taken from real world situations is that not only service technicians will be asked technical questions or questions about maintenance, also the sales representative are asked that. We also had actually really one pilot user that moved from service to sales, and he also told us some stories how a customer, meeting the sales representative, they would talk about some upgrade or maintenance plan, for example, but then the customer would say, “Hey, and by the way, we have that issue here since two months. It’s a minor one. We didn’t open a ticket or anything, but can you tell me something about it?”
Max:
And then they are also in that position of researching or best case, maybe even giving a hint of, “Okay, I can look into the system and it could be something about whatever the cooling system or so, I will just arrange that it will be fixed.” Or have some, let’s say, workaround, maybe even ready to say, “Okay, just drive down the power here and then it will work until the technician comes.” So yeah, enabling sales as well with technical information, but also enabling sales with their main use case, of course, with sales and marketing material, that was the next user group and also a different type of content that had to be onboarded to the system.
Larry:
Hey, you mentioned a couple things here that I’m really curious about now. One, those kinds of edge use cases were something that’s not escalated to the level of being articulated in a ticket, and then just the general learning about as the system improves, is that captured in further documentation or within your architecture itself? Or how does your system learn, I guess?
Max:
It is not learning in a direct way. We are mostly learning from feedback and from, let’s say, adapting how the ontology and the taxonomy looks like. One example, for example, so if you are in the sales perspective, and for example, if you go on the Zeiss website, you will see quite different categories for the product than what the technicians have. Technicians think in, we have light or laser microscopes, electron, and x-ray, and on the website they are talking about, yeah, it’s about life sciences, it’s about material science, so completely different viewpoint. So the system actually learns by learning from the experts and from how they view the same thing. And the beauty of all of this is that you just extend the model. You just add either an entity or you just add a mapping in the taxonomy which says, “Okay, everything from this product family, this is suitable for life sciences and for material sciences.” And then you can start answering questions even though it’s not directly connected somewhere.
Max:
So you could use that information from the technical perspective, translate it more or less into the language and the viewpoint of the salesperson, and it works. And of course, working on the feedback that we collect in the system that is being evaluated and based on that, the behavior might be changed.
Max:
Maybe the last thing, I don’t know if you would call it learning, but of course having user generated content. For example, a best practice description or a solution node, it is user generated content and having that in the system also allows the system to get better if users provide that information. And that’s maybe also a point worth to mention. It actually made an impact on service technicians, they said, “Well, for a long time we have been providing solution nodes, but we cannot really use them. There are hundreds, thousands of them. It’s the first time now that we see that information is being reused and we can actually benefit from the content that we created and the knowledge that we brought into the system.” So that was really cool to see.
Larry:
I might be going to KMWorld in London in April, and that should be a case study there, the way you just discussed it. I realize now that my question was just, it’s really just about how knowledge management systems evolve and you just improve as you go.
Larry:
Hey, I can’t believe it. We’re already coming up close to time. I could talk about, I would love to hear about all the other agents and stuff, but I know you’ve published some case studies about this as well. I’ll add that to the show notes so that folks can explore more if they want to. But before we wrap up, is there anything last, Max, anything that we haven’t talked about or that you want to revisit from the conversation?
Max:
Maybe not from the conversation, but I think what we have seen here is also a showcase of how, let’s say the basics of knowledge management and the benefits of structured content and mainly also the benefits of semantics really help to improve a system or even at the start, make it useful at all. We are a lot in other projects involved in the same discussions that probably everybody has right now in the field is, “Hey, is the language model not enough? It is getting better every half a year or so. Do we really need to put that effort in our content?” And I would just say, yes, it is clearly a benefit. Please don’t forget about the best practices that have been learned and evolved over years and years. You really benefit from it if you build on a solid foundation. And yeah, I hope that is something that will become, again, a driver. And also if you are talking about AI readiness, then yeah, I would say look at semantics, look at knowledge management really.
Larry:
Yeah, that’s music to my ears as a longtime knowledge architect and content architect and information architect, and the things I’ve been saying for 30 years, it’s like you justified it all. Thank you.
Max:
You’re welcome.
Larry:
But hey, one thing-
Max:
We are in the same boat, I guess.
Larry:
Yeah, exactly. Hey, one very last thing, Max. If folks want to follow you online or connect, what’s the best place to find you?
Max:
I’m on LinkedIn, of course. Please find me and connect or of course on our website, pantopix.com. Yeah, you will also find the articles you mentioned, they’re all published. We really, if we are at conferences, if we have something to share, we put it there so you can just download it and have all of this summarized.
Larry:
Cool. Well, I’ll link to some of the ones that look most relevant to this conversation and put them in the show notes. Well, thank you so much, Max. I really enjoyed this conversation.
Max:
Cool. Yeah. Thanks again for inviting me. It’s an honor, I must say, I’m a big fan and yeah, hope we can meet at the next conference or have another conversation.
Larry:
Thanks. Yeah, KGC is coming up in May. Anyhow, great to see you, Max.
Max:
Cool. Thank you. Bye-bye.
