Podcast: Play in new window | Download
Subscribe: Apple Podcasts | Spotify | Amazon Music | Android | Youtube Music | RSS
Where content, knowledge management, and AI converge, you’ll find Michael Iantosca.
As many in the AI world flock to probabilistic models like LLMs, Michael takes a deterministic approach to content management and knowledge engineering, using ontologies and knowledge graphs to ground content in a concrete facts.
This approach embodies his insight that content and the models that describe it are not static information but rather valuable, ever-evolving enterprise IP assets.
We talked about:
- his 44-year career in content, knowledge management and localization/globalization roles
- the three pillars of his work: content, knowledge management, and engineering
- the need he sees in his work to move away from probabilistic, vector-based models to deterministic, neuro-symbolic models like knowledge graphs
- how he decides which models are appropriate to use with each of the varied kinds of data he works with
- his explorations of how to automatically construct a knowledge graph to use to power generative AI solutions
- how he acquires and develops ontology skills in his team
- how graph technology supports the “total content experiences” he builds
- how the non-static nature of content makes it a poor candidate to be managed in a static system like a vector-based model
- the relative merits and utility of 1) deterministic retrieval for structured content and 2) probabilistic retrieval for unstructured content
- the power of combining content models, knowledge models, and ontologies and how they can become crucial enterprise IP assets
- his belief that we are entering a golden age of content and knowledge engineering
Michael’s bio
Michael Iantosca is the Senior Director of Knowledge Platforms and Engineering at Avalara, a sales tax automation company. With over four decades of leadership in technical content management, Michael has been a pioneer in advancing the profession, driving innovations in structured content, intelligent authoring, and scalable knowledge platforms. Renowned for bridging engineering and content teams, he has championed the adoption of AI and cutting-edge technologies to enhance user experience. A thought leader and mentor, Michael continues to shape the future of technical communication through his expertise and passion for innovation.
Connect with Michael online
Video
Here’s the video version of our conversation:
Podcast intro transcript
This is the Knowledge Graph Insights podcast, episode number 16. A lot of work in the AI world these days is about vectorizing giant collections of static, unstructured content and data for LLMs. Michael Iantosca has worked for decades in a world where content is dynamic, always precisely structured, and contextualized with rich metadata. So he has a different take on architectural innovations like graph RAG, favoring knowledge-based deterministic retrieval of content over vector-based models and probabilistic methods.
Interview transcript
Larry:
Hi, everyone. Welcome to episode number 16 of the knowledge graph Insights podcast. I am really delighted today to welcome to the show, Michael Iantosca. Michael is currently the Senior Director of Knowledge Platforms and Engineering at Avalara, the big tax-compliance automation software company. He’s also got a long history. He’s spent a couple of decades, a few decades at IBM prior to his role at Avalara. Welcome to the show, Michael. Tell the folks a little bit more about what you’re up to these days.
Michael:
Larry, thank you for having me. It’s a pleasure and an honor to get a few minutes to talk to you today. Yeah, I have just started my 44th year primarily in the professional content space, but also in the knowledge management and localization globalization space as well. I have been involved with content since the early days of SGML that began the structured content revolution and worked my way up through the professional content ranks. I’m also an engineer in IBM’s Grand Wisdom. I was trained for years as a developer, so I do span both worlds.
Michael:
My responsibility is to develop some of the world’s most advanced content supply chains for creating content for customers, delivering it, as we like to say, “Deliver the right content to the right person at the right time and in the right experience.” That’s my north star that drives that. I lead an engineering team that builds those platforms that services multiple groups throughout the enterprise for their content creation and delivery needs, whether that’s in product, user assistance, contextual help, knowledge centers, help centers, support sites, and a litany of other channels by consolidating that entire supply chain, so that we can write content once and deliver it in many different channels, including chatbots, generative AI.
Larry:
Cool. I love that … I think, I’m trying to remember only 16 episodes in, but I think you’re the most content-ey person I’ve had on the knowledge graph Insights podcast. One of the things we talked about before we went on the air was this notion of you’re like, “Hey, guys, dear engineers, it’s not just about data, it’s equally about knowledge.” And in this world, the knowledge graph engineers and ontologists, they’re on board with that, but you also bring this content perspective to it. I’m really curious how those three concerns combine in your work. Do you approach data stuff differently with a content lens on? And especially the knowledge management part, because each of those is its whole other thing, but you’re working, your title, you’re a knowledge platforms guy, you’re combining all three of those. Tell me how that manifests in your work.
Michael:
Yes, that’s critical. We see content and knowledge management, and when I say knowledge management, I’m talking more about taxonomies and ontologies and knowledge graphs and engineering, the actual coding and infrastructure of building out models and solutions in the AI, especially generative AI space, as a holy trio, if you will, that have to have equal footing. Developing really advanced generative AI solutions is not just a coding problem. It is equally as much of a knowledge management problem and equally as much of a content challenge.
Michael:
Content isn’t generic. Content constantly is changing. The management of that constant is constantly changing. We can’t treat content any more generically than we can treat data that changes daily, sometimes weekly. The state of that content, the purpose of that content is not static. We need the content teams involved because they have the very fuel of our generative AI solutions, but we also need the people that understand advanced semantic knowledge management that can help power both that content and make it intelligent and then feed that to the generative AI models, so that these models can be far better than they are today.
Larry:
Yeah, and when you say today, that’s going to be way different even tomorrow from when we drop this. We were talking, again, before we went on about trends in the development that’s developing around the implementation of graph technologies across all of this stuff, but in particular to content. I wonder if you could talk about how you see those trends. One of the ways I’ve seen is from LLMs to RAG to graph RAG and now, Tony Seale and folks talking about neurosymbolic loops and hybrid AI architectures. How was that unfolding in your world?
Michael:
That’s a really good question too. I think almost everybody who starts out in the generative AI space follows the same basic path. About three and a half years ago, I think it was, it took us about a week to take a simple vector database, Pinecone, I think we used, and maybe a couple of hundred lines of Python code, and we built a RAG, retrieval augmented generation model, because we didn’t want to use the general large language model that uses the public content, and we didn’t want to feed our public content to train a large language model. It was natural that we wanted to have a private data model of our own and use a vector database to do that. But that’s really an old, old model at this stage. It is a probabilistic retrieval model and therein lies its core weakness as well. What we wanted to do was move away from probabilistic … oh, are you there?
Larry:
Yeah. Oh, do we have an internet thing? I lost you. Let me check my internet.
Michael:
I apologize. I had a burp in connectivity.
Larry:
Oh, no worries. Okay.
Michael:
Let me pick up again.
Larry:
Yeah. Go ahead.
Michael:
What we want to do is move away from probabilistic models like relying completely on vector-based retrieval and move toward deterministic models, sometimes what we call neuro-symbolic models and use mechanisms such as knowledge graphs, which are far better at providing true reasoning and true inference based on a concrete set of facts or what we call the ground truth. I think what you’re seeing now in the marketplace is the initial models that are being deployed are good. They’re yielding value, people are excited. They’re not perfect, but as development teams reach those plateaus, they want to get better. They want better than where they are. This is what I call the precision paradox. The precision paradox says that as models improve, the tolerance for errors and lack of accuracy or relevance or contextual truth declines. We’re partying right now with these models, and then, eventually that party is going to end and we’re going to have to get down and do some of the serious work necessary to move to these deterministic, reliable models that are based on ground truth that we control, not that in LLM controls.
Michael:
That’s what I see the trend going to. I think, every morning I wake up, I think I read at least 10 to 20 articles, all different models and variations. Unfortunately, what I’m seeing too often are new teams coming on board building the old traditional vector-based RAG model, or if they have a RAG model, they’re so invested in it that they try to continue to scaffold more and more and more around it. It’s kind of putting lipstick on a vector pig, as I like to say. There’s a ceiling to what you can do on a pure probabilistic model, and the answer is to either use either pure deterministic models or a hybrid between both deterministic and probabilistic models, where you can get the best of both worlds being specific as to which model you apply to, what kind of data to get the most mileage out of your model.
Larry:
Yeah. I think a lot of people are talking about that these days. It sounds like you’ve actually done a lot of it, and a lot of people have, of course. Going from the lipstick on a pig to going to the gym for six months, and I don’t know whatever the analogy there is, but the idea of going from just gussying up a model that’s fundamentally unable, like a probabilistic model, just can’t do things that deterministic approaches can. What do you see in those hybrid architectures? Are you swapping out parts of the architectures? How does that unfold?
Michael:
That’s a really good question, because we have all different kinds of data that come into play when we’re talking about generative solutions like chatbots. Some of it is from documents. A lot of it is from documents, existing bodies of content that … It’s very common that an enterprise, AIML team, will first come after the help corpus because it’s highly-curated, refined, usually componentized kind of content. But there’s also other kinds of data. There is unstructured data out there, big blobs of content that need to be handled. Then, there’s relational data. You might have a CRM database that can answer questions about customer’s account that need to be factored, and so you have all of these different kinds of data. No single model is going to apply for all of them. You’re going to have to apply the right model on the right kind of data, so there’s no silver bullet here.
Michael:
However, what we found was in moving to a deterministic model that we built based on knowledge graph, the challenge was how to build that knowledge graph efficiently and maintain it, because that’s the biggest little dirty secret that very few people talk about. When I say dirty little secret, it’s because there’s really no way to use humans to construct a knowledge graph that represents thousands or hundreds or millions of nodes representing individual pieces of content. You can’t do that manually. You need to do that automatically, which is why you’ve been seeing different models come out to try to automate that.
Michael:
I think Microsoft Graph RAG was probably one of the poster children of a solution intending to do that, and it can do that, and so can some other models. It may work really well on unstructured data, but we have a lot of structured content too. We have a lot of topically oriented documentation that is already chunked into the right sizes and has all the right relationships classically defined. What we wanted to do was see could we automatically construct a knowledge graph to use to power a generative AI solution and could we do it automatically?
Michael:
What we learned was if you have been using graphs like I’ve been since about 2015 or 2016 or so, this is not a unique thing to do in the relational database world. If anybody that’s worked with a relational database knows that the relational database has a schema, and the schema defines the structure of the data. A schema, a well-structured schema, is in fact a sort of ontology in itself, and they translate almost one-to-one from a schema to an ontology. What we learned was that, just as folks in the past were able to take in our relational database, schema and convert it to say an RDF or OWL ontology, we could take something like the DITA XML schema and we could convert that to an ontology and then we could use that to automatically map all of our content objects into a graph completely automatically. And, oh, by the way, update it overnight every single night if we want to get the latest content and the latest metadata.
Michael:
We solved the problem of how do we create a controlled and managed knowledge graph at scale that we could then query and retrieve from. In fact, we did it without needing a vector database at all to retrieve from. We had a pure deterministic retrieval model. But then, once we had that, we realized that in addition to the nodes of the graph representing the actual topics and the relationship between them and having the URIs directly to the graph database, where those objects could be retrieved from, we realized that we could overlay a domain concept ontology. I happen to work in the sales tax space, so we separately could add in the conceptual relationships of that domain, so take that as an example. That is a tax system that is applicable in Europe and say, Germany is in Europe, so we can define those triple relationships and build those up.
Michael:
Over time even, we didn’t even have to have a complete ontology. We were able to query that graph with very minimal ontology and get the related content even if it was two, three, four or five hops or far away from the original topics. We were able to get reasoning and inferencing immediately with nothing more than the structural graph, and then the graph only gets better and better and better over time as we overlay more and more of the conceptual ontology on top of it. That takes time to build out the latter half, and that’s where a lot of AIML teams, either they don’t have the experience in doing that, it’s a skill, or the time because they’re under pressure. Eventually, they’re going to need to move to a fact-based model if they’re going to need to get to the levels of precision and accuracy that a generative AI is going to demand.
Larry:
As you talk about that, one of my intents or my main intent in a lot of the almost all my podcasts is democratization. I’m well aware of the hazards of democratization that you need experts, but in this area in particular, it seems like we’re hitting an era where there’s probably going to be more need for ontological skills than there are going to be ontologists. Have you had any success in upscaling or onboarding ML AI, NLP engineers to the knowledge graph, the RDF stack, or just even graph thinking?
Michael:
I wrote in an article, I think over three years ago, three or four years ago, I wrote in an article for the Center for Information Development. I think I concluded the article making an assertion that eventually here that undergraduates are going to be recruited by companies like the NFL draft eventually. That’s how scarce and valuable they are. One of the very first people that I helped go out and recruit here at Avalara was an ontologist who previously worked at a large company and has decades of experience. I knew instantly that this is not a job for novices.
Michael:
However, that ontologist is brilliant and a good teacher and has become the enterprise ontologist even outside of the content space that I work and teaches other taxonomists and ontologists and brings them together across the enterprise, because ultimately, we’re talking about enterprise knowledge graphs here, not just content knowledge graphs and it all plays together. We have to know about the products, we have to know about the product matrices. We have to know about the content ontologies and the sales and marketing and CRM worlds. They all play into this world that we used to live in. This nice isolated silos is no more, and that applies to knowledge management completely in addition to bringing all this content in from all these different places into these generative AI solutions.
Larry:
Yeah. I’ve talked to a couple of guests about types of knowledge graphs, like the enterprise knowledge graphs versus product graphs, like Amazon’s Product Graph or LinkedIn’s Economic Graph. The way you just talked about, and another insight I’ve had the different points in my career is content is everywhere in the enterprise. I’m just wondering, what’s the relationship between understanding the content in your enterprise in relation to an enterprise knowledge graph, but then also just that subject, not subject matter, but the craft and departmental expertise that sales and marketing and HR and people like that bring to the game, how do those overlap and connect?
Michael:
Well, they all do, because as a content professional, my view of content is I take the view, the empathetic view as that from the customer. The customer doesn’t care that you’re siloed. They don’t care that your marketing team is producing content and you need to go to the marketing website or your developer support organization is developing developer guides for APIs and connectors or that your learning group. They don’t see your company as those departments. They want whatever content they need and they want all of it integrated and they want it orchestrated. They want what I call the total content experience, and that total content experience evolves across the entire customer journey. From the very beginning of discovery of your company and product, to learning about the products, to using them, to trying them out, to purchasing them, then implementing and supporting and even advocating. That’s the big customer journey from a very high macro level.
Michael:
When you look at all of your content across the business, you want to lay it out irrespective of those functional boundaries, so that you know have developed all of the content that is necessary and will be needed at different stages by the customer at all these different life cycle points. The problem that we have today is that most companies are fragmented. They have no integrated content operations that span those areas. Everybody’s working in their own silo. That leads to how do you manage the documentation from a much bigger level and also within the individual areas. Do you have a system of record of all of that content? Do you have the metadata about all of that content to manage it?
Michael:
People tend to think too often, especially in the generative AI world, that content is static. It’s not. If you have a corpus of content, it’s changing daily. Products add features, change how products work. Marketing is changing their strategies. To think that we could just take our content and put it in a vector database and consider it static is plain heresy. How do you know how to manage that content? Well, that has everything to do with content management and metadata and tracking it and updating it. That’s a big weakness of most models. There is no state data associated with those chunks of data that have been ripped into shreds into these vector databases, and that’s where knowledge graphs shine, because we can clearly associate the metadata, associate it with all of these different content objects on the graph nodes with the graph nodes themselves and update them on a regular, frequent basis. Whereas, we could potentially do that in a vector model. I could create vector embeddings of the metadata and associate it with every single chunk of content that’s been fragmented or there’s several models to do that.
Michael:
But that becomes unwieldy both from a compute perspective and a maintenance perspective. I ask the questions such as, how did these models, how are they able to unpublish content in these vector models? How are they able to retire that content? How are they able to deliver only certain content to certain people, what we call entitlement? Say customers, only customers that are licensed for that product get this information, or we have both internal and external versions of that content and we need to know that only the internal users get answers from the bot without having to create 50 different vector databases to maintain?
Michael:
How do we determine version control? If I have a product, I remember one of my old products, IBM DB-II. Big product, very big flagship enterprise software that had, I don’t know, 5, 10, 12, 15 different versions. Enterprise, personal, small business, so on and so forth. You dump all of that into a vector database and have no state information about what version you’re giving an answer for, you’ve got some big problems on your hands. We need to be thinking much more about not just the content as a static resource like it’s generic gasoline, but each one is a piece of specialized content with its own metadata and its own state data that is necessary to do professional level content management, which has been an afterthought, if at all, in most models.
Larry:
Yeah, and as you talk about that, I’m thinking of both in the data world and the content world, the need for governance of that kind of stuff, especially in regulated or other similar industries where accuracy is so important, I want to revisit maybe a little bit those architectures of the relationship between, because there’s benefits of the probabilistic approach to things, but that inability to go back in, it’s not like a database. You can’t go in and do anything with that, and you don’t have any of that. Like you said, the state information or the ability to do anything with it once the model has learned from it. What do you feel like, maybe at a very high level that kind of, and I know it’s evolving all the time, but how are these architecture is going to look? What are the real benefits of the probabilistic models? How should you think about content in those versus the symbolic knowledge representation side of it?
Michael:
That’s a great question, because I think there’s multiple answers to that question. In the graph model that we built with Semantic Web Company, and it’s now Graphwise. They’ve merged with Ontotext recently. What we did was we didn’t want to retrieve, we didn’t want to use a vector database to do the retrieval, the actual retrieval, the content. We want it to be purely deterministic retrieval coming out of the graph database. But we realized that the vector database could give us semantic, probabilistic recommendations to inform the graph. It’s almost the reverse of what you’re seeing going on in the industry today. It’s a much more advanced approach.
Michael:
Today, what people are doing is building a graph and then building vector embeddings of the graph to try to retrieve content. This model reverses that. It says, “No. We’re going to use the graph as the retrieval, as the sole retrieval source, but we’ll use a probabilistic vector model of the content to identify the kind of content that we want to deliver. An example would be we like to, in the professional content world, it is not uncommon for doc teams to write their content in what we call topical format. Those topics are often typed, such as one’s a concept, one’s a task, one’s a reference, maybe one is a Q&A, one’s an API doc kind of topic. We like to combine them into variations, so that we can deliver, say just the how-to information, deliver just the task procedure to the user just in time to help them do their task.
Michael:
We can use the vector model to help the graph as a graph helper to make it better, but we’re never retrieving the content from a vector database. We’re retrieving the actual source content from a graph database, and then we feed that to the large language model and just utilize the large language model as a summarizer, if you will. It really minimizes, absolutely minimize the probabilistic characteristic of that system. Now, I should say, and I want to be careful here, that doesn’t obsolete the use of a vector database for content retrieval. There’s a lot of content that is out there that is big, monstrous blobs of content that’s never going to be in a topical organization, whether it be XML, DITA, Markdown, RST, ASCII Doc, which our model supports all of those, but it might be some giant PDFs or really megabytes or gigabytes of unstructured content.
Michael:
It’s still useful to utilize a vector model in that case or a graph-assisted vector model, and you can combine it with this graph model that we’ve modeled or other graph models that are pure, true graph models, true deterministic and get the most best of both worlds in a hybrid situation. There’s no silver bullet in this world. It’s a matter, as I said, of using the right kind of retrieval model for the right kind of content.
Michael:
Now, there’s one addition to that. I argue that if your content is so important to be in your retrieval model, you should think twice about restructuring that content and putting it into an architecture that is going to live on and be useful. Why would we invest all this effort and time in these models and then treat the content as a stepchild of this whole thing? It is the very fuel. Then, when you combine that with the knowledge model, enrich that content with these taxonomies, and then you have the ontologies to disambiguate and create and understand and retrieve the direct and indirect relationships, it becomes more important than the code of the solution itself, the actual generative bot or whatever your application happens to be. Your intellectual property then becomes your ontology and your content, and that becomes your competitive advantage.
Larry:
Yeah. That is so powerful. Several other guests have said essentially the same thing, that deep ontological understanding of your enterprise is the most precious IP that anybody could have. Hey, Michael, I can’t believe we’re coming up on time already, and I could literally talk about this stuff all day. Before we wrap up, is there anything last, anything you want to revisit from the conversation or just make sure we share before we wrap up?
Michael:
Thank you so much for having me today. I’ve been watching your podcasts and the brilliant people that you’ve interviewed. I just get so much from them, so I can’t wait to see the next one. But I will tell you that I really believe that when it comes to content, when it comes to knowledge engineering, when it comes to AI, I believe we are in a golden age right now of content where content and knowledge engineering and AI all converge together. I encourage all of these teams, the content professionals, to talk and to the AIML engineering team and the knowledge management people who know their taxonomies and ontologies and knowledge models, to get together in the same room and talk about requirements, talk about what they can do collaboratively to go beyond these basic stochastic predictive models in the future. I think that, if they do that, they’re going to discover very quickly that their future lies in these deterministic neuro-symbolic, or at least neuro-symbolic augmented models that over which they have control, that they’re not handing that control over to somebody or something else.
Larry:
Nice. Well, that’s a perfect note to end on. Oh, one very last thing, Michael. If folks want to connect with you or follow you online, what’s the best place to find you?
Michael:
Great. I spend a lot of time on LinkedIn, so people can find me on LinkedIn. I also have a personal website, where I publish some significant articles both on medium.com. but my personal website is thinkingdocumentation.com, and I think folks might find some of the articles there interesting and entertaining.
Larry:
Cool. I’ll put those in the show notes as well. Well, thank you so much, Michael. This was amazing conversation.
Michael:
I appreciate it, Larry. I hope we get to talk again soon.
