Tony Seale: "The Knowledge Graph Guy"

photo of Tony Seale, knowledge graph and data product expert — Tony Seale

With ten years of semantic data experience and an endless stream of insightful posts on LinkedIn, Tony Seale has earned the moniker “The Knowledge Graph Guy.”

Tony says there’s precious little time for enterprises to prepare their data with the interconnectedness and semantic meaning that it needs to be ready for the coming wave of more powerful AI technology.

We talked about:

his 10-year history of applying academic knowledge graph insights to commercial work, mostly in the finance industry
the yin-yang relationship in his “neuro-symbolic loop” concept that connects creative, generative LLMs and the reliable, structured knowledge provided by knowledge graphs
the contrast in reasoning capabilities between LLMs and knowledge graphs
how neither formal logic nor probabilistic systems are rarely the right answer on their own, hence the yin-yang analogy
the crucial role of understanding and consolidating data, the gold mine on which every enterprise is sitting that describes any organization’s unique value
the power of understanding the “ontological core” of your business and then projecting it, selectively and strategically, to the world
the urgent threat posed by snake oil salesmen and other opportunists coming into the graph world and derailing enterprises’ chances to properly exploit their unique data advantage
the two crucial characteristics of AI-ready data: connectedness and semantic meaning
his work chairing the Data Product Ontology (DPROD) working group, an effort to provide a semantic definition of what a data product is

Tony’s bio

For over a decade, Tony has been passionate about linking data. His creative vision for integrating Large Language Models (LLMs) and Knowledge Graphs in large organisations has gained widespread attention, particularly through his popular weekly LinkedIn posts, earning him the reputation of ‘The Knowledge Graph Guy.’

Tony’s journey into AI and knowledge graphs began as a secret side project, working from a computer under his desk while employed at an investment bank. What started as a personal passion quickly evolved into an area of deep expertise. Over the past decade, Tony has successfully delivered several mission-critical Knowledge Graphs into production for Tier 1 investment banks, helping these institutions better organise and leverage their data.

Now, Tony has just founded The Knowledge Graph Guys, a brand-new consultancy dedicated to making knowledge graphs accessible to organisations of all sizes. Through this venture, he aims to empower businesses with the tools and strategies needed to harness this powerful technology.

Connect with Tony online

Resources mentioned in this podcast

Video

Here’s the video version of our conversation:

Podcast intro transcript

This is the Knowledge Graph Insights podcast, episode number 11. Whether they realize it or not, every business on the planet is sitting on a gold mine, the precious data that uniquely positions them in their industry and market. With ten years of AI practice and an endless stream of insightful social media posts, Tony Seale has earned the moniker “The Knowledge Graph Guy.” Tony argues that enterprises that fail to grasp the urgent need to consolidate and understand their data will not survive the coming wave of more powerful AI.

Interview transcript

Larry:
Hi, everyone. Welcome to episode number 11 of the Knowledge Graph Insights podcast. I am really delighted today to welcome to the show Tony Seale. Tony is the Knowledge Graph Guy. That’s how everybody knows him on LinkedIn, and I think he’s earned that moniker. He also does a lot of consulting and work for big investment banks and things in the financial world. But welcome, Tony. Tell the folks a little bit more about what you’re up to these days.

Tony:
Hi, Larry. Thanks for having me on here. Yeah, so as I was saying, I’ve basically been doing knowledge graphs now for the last 10 years, largely within investment bank, investment, large investment banks. So I’m kind of at the rubber meets the road. Well, that’s where I have been. So taking the technology that’s been largely in the academic space and actually applying that into production settings. And then I’ve started trying to share that knowledge and I’ve become obsessed with the idea of what it would be to connect an organization together. So what would be possible if most of the information within a given organization was connected together? And I’m on a mission to push that forward. As you say, I’ve just started doing a consultancy to try and accelerate that effort. The last really, I guess, two and a bit years now have been focused really on the space between large language models and knowledge graphs. And hopefully we can talk a bit about that.

Larry:
Yeah, because I attended the Semantics Conference last week and that’s pretty much all anybody was talking about. I mean, there was other conversations of course, but you have a take, and I think one of the things that is becoming clear in my mind is this sort of evolution from just math and LLMs to graph to RAG to graph RAG. And then you have this concept of the neuro-symbolic loop which, is that the evolution of the integration of these technologies? Tell us some more about that.

Tony:
Yeah, so I guess maybe to frame it at a conceptual level, you can think of the large language models existing in this continuous space. So they’re slightly fuzzy, they’re probabilistic, they’re generative. So they’re always guessing at what the next right thing to do should be. They’re not structured. And that has within it a huge amount of power because they can explore different pathways. They can be in at least some limited sense of the word creative and imaginative. They can write poetry and do everything that we are familiar with them doing. But you could sort of contrast that type of intelligence with I guess what you would call the existing and what people will rather derogatorily call good old-fashioned AI. The existing approach, which is sort of formal deductive logic and formal reasoning. And really that’s what knowledge graphs represent from a data side, like the most flexible structure for doing reasoning over your data.

Tony:
So what then becomes interesting, it’s like, well, can large language models actually do that formal reasoning? And obviously the big AI houses are trying really hard in order to make that happen. They’re chucking a lot of money into making that happen. But I think to a certain extent, maybe one day they will kind of get so close that we won’t know the difference between it. But to a certain extent it’s just a kind of different paradigm, if you like. There an uncertainty there within a generative model, which is just very different from what deductive reasoning is going to be.

Tony:
So the idea of the neuro-symbolic loop is to try to bring these two systems like a yin and yang, bring the two systems in together closely. So that as close as possible and as lower grain possible level, you are looping between this system one and system two. So system one being the large language model, being a bit creative, being generative, very, very quick. And then the system two being the formal representation of the system in which you working through steps, being able to do structured querying, always getting back reliable results.

Larry:
I love the visual of the yin and yang symbol as the representation of that tightening loop, because that’s a perfect way to look at that. But back to, I want to revisit the notion of reasoning a little bit because you mentioned that and that seems really important. And a lot of the AI fans on LinkedIn these days and the big companies themselves of course have been talking a lot about the reasoning that they can do. Can you contrast that? I heard it referred to at Semantics as pseudo-reasoning, versus the real reasoning that you just mentioned, that deductive logic based systems can bring. Can you talk a little bit more about that, the relative reasoning capabilities and whether… Are they just faking it till they make it or what’s going on there?

Tony:
Yeah, so I mean you’re always working in analogies with this stuff because at the end of the day, nobody, not even the people who are very close to this truly understand what’s going on inside of a large language model. It’s a bit of a mysterious thing what’s happening in there. But here’s one way of conceptualizing it, that effectively it’s kind of doing not a database lookup, but almost like that. This very sophisticated lookup of its training data. So it’s seen a huge number of samples of training data. It’s able to a certain extent, it’s kind of mapped all of that kind of training information as you’re doing the different layers within the large language model. It’s compressed some of the concepts in there, not in a way that we would necessarily be able to understand in this kind of vector representation of it. But so it’s able to use that to go and retrieve these kind of patterns, and to a certain extent sort of merge them together.

Tony:
So what recently has been done with the kind of Q-star stroke, Strawberry stroke 01, and really all of the others are doing the same thing as well, is that you will take some domain where you have right and wrong answers, for instance coding or mathematics. And then what you will do is you’ll get the large language model to simulate loads of different answers to that and then have its kind of reasoning steps of how it got to that particular answer. And then you’ll go out to an external verifier, which in the coding thing it’s like, “Well, okay, run this unit test. Does the unit test pass or with the maths? Okay well, the answer to this question is 42.” And then you’ll prune the reasoning paths that actually led to the correct answers. And then you feed that back into the larger language model and you train it on that.

Tony:
And that’s kind of given them the kind of boost up of, the boost up of their reasoning scores. But to a certain extent that there’s a Turing machine, which is the kind of computers that we built, and there’s symbolic logic, where it’s a kind of formal system of it. We create symbols and then we piece those symbols together in a graph. We do it with edges, but in maths you would do it with functions. And within that kind of formal symbolic system that you create, it’s almost got this kind of infinite power. Once I’ve defined the function for x plus y, I can pump any number in as x, and any number as y. And reliably, it’s always going to add those two numbers together. Or I can do the function to times them. So once you’ve created that sort of symbolic representation, and then also with those symbolic representations, what I can do is I can compose them. So I can grab one and I can now have two functions together. I’m going to add, and then I’m going to times and then I’ll sum the whole lot together.

Tony:
And that formal structure is just kind of never going to change. So that “never going to change” aspect gives it this rigidity, which when it comes to knowledge capture can be quite a bad thing. Because it’s like, computer says “no,” the kind of classic thing of, oh, you’ve just kind of fallen out of its hard and fast rules. But it can also be really great. I’m calculating the distance to the Moon and I don’t want to be taking probabilistic guesses about what that thing is. So there’s this way that these two systems are, to a certain extent, just fundamentally different.

Tony:
The large language model is probabilistic. It’s kind of doing this sort of pattern matching where it’s looking at these N-grams and it’s combining them in a very, very sophisticated way. But there’s a certain extent to which it’s always going to be within the distribution of its training set. So it’s never going to be able to go out of that distribution. Because it’s curve fitting at the end of the day. That’s what a large language model is doing.

Tony:
And on the other side, then we have kind of formal systems, all formal systems which are large and would include mathematics and computer languages. But would also include knowledge graphs as a knowledge representation system. And those formal systems, they’re discrete as well. So you have to create a symbol to represent stuff, and once you’ve got that symbol and you’ve nailed your colors to that mast, then you can rely upon it and you can compose it and you can build it up.

Larry:
And that reminds me of the, I got to say the math of all this stuff goes away over my head. I became a journalism major because of fear of all that. But I get it conceptually, I think. But that need to have a symbol to represent those notions reminds me of the work I do in my modeling work to model abstract things and then attach labels to them that let machines and humans get what you’re talking about and do stuff with it. And it seems like, well, I guess so there’s that sort of conceptual thing about it. The other thing about it is just the practical business part of this. I know that we’re seeing this with all the critique of AI around the new data center every day and $50 billion a quarter being spent on that, which kind of hints at the computational intensiveness of this. How does it compare in… Because you’ve done this architecturally, you’ve built systems that deal with both of these things. Does it come down to practical cost-benefit analysis of which task to do with which technology?

Tony:
Yeah, I mean the cost thing is a thing as well. Obviously it’s much more computationally expensive to run something all the way through a generative model with however many billion parameters than it is to just do a kind of classic straight information look up in discrete space to a particular place. There is the extent to which that it’s cheaper. I think the cost is relevant. The cost is relevant, and these things are expensive. And as you’re kind of doing more complicated things with them, it’s really easy to churn through a lot of money. But from my experience within large organizations it’s, at least at this stage, that’s not really the major issue. The major issue is this large language model looks really cool and powerful. It’s like having a Ferrari, but the Ferrari is parked in my front garden. And I can’t actually take it out because the engine could just kind of blow up at any point, or suddenly the autopilot on it decides it’s going to drive off to France or something.

Tony:
So this I think is really the major issue. It’s like how do I get the large language model to actually be doing something practical and useful and reliable? That’s not trivial. It’s easy to apply these things largely. “Oh, I’m going to create a proof of concept. Wow, look, it does x, y, and z.” Now take that into production in a situation where actually be doing something where you can completely rely upon it, then that turns out to be a much harder problem. But it would be a mistake to say, “Oh, well. That means that the good old-fashioned AI, the formal logic is that’s got all of the answers.” Again, that’s wrong. That’s why I think the kind of analogy at the yin and yang is very useful. It’s not like one is right and one is wrong. They’re both useful, but they need to be combined together because one on its own is not going to work.

Tony:
Now will that be forever? I don’t know. Maybe not. Maybe a new kind of AI architecture comes along that unifies the two. Maybe transformers get it in the end and they just get so much reasoning data that basically they can effectively reason. They’re not really doing it, but they’re faking it so well that we can never tell the difference. I don’t know. But again, my philosophy here is that any organization whose strategy is, “Well, I’m just going to sit and wait until that happens. And then I’m going to apply the AI fairy to come and sort everything out for me,” is I think, that’s a kind of category error in a business sense. If you wait until that point to apply this technology in the way that it needs to be applied, then you’ve already waited too long.

Larry:
Yeah. Well, that’s the top reason I wanted to get you on the podcast is like, you’re just, every other post of yours on LinkedIn is about pragmatic solutions to these issues. Like, “Okay, here we are, we got the Ferrari in the driveway. We’re kind of staring at it trying to figure out what to do with it.” You’re in there, building systems that are actually dealing with this. And I know one of the things you’ve talked about is the importance of getting on top of your data. Because that’s such an important, that’s kind of the fuel for all this. Well, I guess, how do you put on that pragmatic business-problem-solving hat and look at this crazy, extremely dense, fast moving thing and do useful stuff with it?

Tony:
Yeah, this for me is absolutely, this is completely fundamental. This is so important. And if anybody’s listening to your podcast and takes one thing away, they’re running an organization, then this really is the essence of it, is that AI needs your data. So your first AI strategy is to basically get your data sorted out. Interestingly, it’s to take the AI and it’s to turn the AI back on the data organization problem. So you’re going to use the AI to sort out your data. Because the truth is that not very many people are going to be developing these foundational models. Well, I don’t know how many we’ll have of them in the end, but you need billions of pounds. I saw something the other day where they’re putting, talking about building data centers up in space in order to have the solar panels, in order to be able to meet the energy needs just to run these things.

Tony:
So it’s massive investment. So it’s going to be a small number of very big, experienced, wealthy organizations that are building these foundational models. That’s not the kind of game that most of the organizations in the world are going to be in on. Most of the organizations in the world will be buying that intelligence in. So you’ll be buying that foundational model in as we do now. You subscribe to your favorite foundational model and then you’ve got intelligence on tap. So in that world then, what is it that each organization is going to do? Because one scenario here is that these foundational models, which are already sucking in, they’ve trained on the internet and now they’re training on private data sets as well. There’s one scenario where basically these things just hoover all of the information into them. And then we have a few absolutely massive foundation models which kind of know everything. And then we will eke out intelligence to the rest of us if we’re nice.

Tony:
And you can kind of already see that developing within the content management space and everything. Where it’s almost like you’re renting out your customers to… Your customers aren’t really yours. You are sort of giving your data over about what those customers are. And then information is given back to you by the kind of major platforms which are holding that information. And for me, that’s quite… I don’t feel that comfortable with that sort of AI future. So then the question comes, “Well, what is it that organizations can do?” And this is where it’s the concentration on data. So returning back to what we were talking about before, with what is it that an LLM is doing at a fundamental level? Well, it’s curve fitting. So it’s bringing in a huge amount of data and then it’s fitting to distribution. But there’s a certain extent to which the LLM is always going for the middle of the road there.

Tony:
And you can kind of see that with when it’s being somewhat, when you get the LLM to do something and it’s just somewhat mediocre or bland. It’s kind of taken the beige, it’s taken the beige approach. So all of this is kind of a roundabout way of saying, “Well, what is it the organization needs to do?” Each organization, the way I see it, is already sitting on top of a gold mine of its own information. But that gold mine is currently completely unrefined. All the information is kind of scattered and siloed in these different systems, particularly the big organizations which I’m familiar with working with. And the ones where I’ve been exploring these patterns.

Tony:
You could be looking at 50 to even a hundred thousand separate applications, each one with its own databases, with its hundreds of tables. Just a complete mess, a hodgepodge of different technologies for sharing information between them. Really no organization around that at all.

Tony:
And what we now have, the possibility, I believe, with this kind of the neuro-symbolic loop pattern is for the first time organizations actually being able to properly distill out what is it that makes this organization unique? What are the concepts that are at the core of this business? What are the natural semantics that our people are discussing about what’s meaningful and valuable within this organization? And then actually having the kind of determination and the guts to nail the colors to the mast there and say, Well, these are the concepts.” And begin to actually kind start to map those out. And then once you’ve mapped those out using this kind of the neuro-symbolic loop, so basically using the large language model to help you understand what your domain is but then kind compiling out into this graph-based format. Then what the next job to do is to connect those concepts back to actual data sources.

Tony:
So when you’re saying what makes this organization unique? We’ve kind of gone out of the… The branding people have known this for ages, but we’re not talking about it in terms of that kind of slightly nefarious branding sense. We’ve now got to drop down to this next level, which is actually a data science sense of what is this organization about? So conceptually, what is it about? What is, I call it like the ontological core. What is the ontological core of this business? And then how can we map that, those core concepts back to real concrete data, which is the instantiation of those concepts? You do that thing. It’s easy words to say, but you do that thing and then you can take the foundational model. And you can inject your own intelligence via the context of the concepts that are specific to your business into that.

Tony:
And suddenly we end up with a possibility of a different AI future where basically… And at this point, this is just kind of theoretical… But basically this kind of ontological core at the center of the organization with the core concepts and trained up around with the machine learning. I predict we’ll see, but I predict we’ll actually kind of project out onto the boundary of holographically almost, onto the boundary of an organization. And its data within there can be then be isolated and protected. And then you’re only sharing the concepts and information that you want to share. And that potentially leads us to a different AI feature, I believe, where we then get this kind of network of networks. Like each individual organization has connected and consolidated its information. It’s able to take a holistic systemic view over how it’s operating and then it networks up with other organizations.

Tony:
And to me, that’s the AI future that I feel more comfortable living in. But we don’t have very long actually to make this happen. This is why I’m kind of a bit on the soapbox about this, and so motivated and keen to kind of get the message out. I mean, there’s so much work to do. We have technology to build, we have patterns of how to do this. There’s so many organizations that need to be helped on their path. And I mean, as knowledge graphs are coming up in the hype cycle, we’re also going to get kind of snake oil salesmen, which are coming in, that are just interested in earning a quick buck on the latest.

Tony:
All of the crypto bros will kind now start piling in on knowledge graphs because “I can make a bit of money here on this kind of latest hot topic.” Which could be very dangerous because an organization could go, “Oh, we will bring this guy in, and he’ll be like, ‘Here you go, bro. You just got to link your data like this.'” And actually that organization maybe just gets one shot, or perhaps two, at doing this within this kind of compressed timeframe before we get really powerful AIs. Because if you haven’t consolidated your data before that point happens, your organization is probably not going through to the other side of that. That is the harsh reality as I see it.

Larry:
Yeah, that’s so urgent, that need. Because just the way you described that. First off, we’ve always like, “Wow, something like the semantic web is finally coming.” But no, the tech bros will get out in front of it like they have with Facebook and all the social media profiles. They’ll enshittify everything. And how can we prevent the enshittification of this beautiful vision that you have?

Tony:
Yeah, I mean, I don’t know. I think it’s about community. And it’s about the people who care about this and are passionate about it getting involved and working together, not fighting amongst each other. Silly things of like, “Oh, this is a property graph. This is a triple store.” And some of the silly things that people can argue that aren’t really relevant to this much bigger picture.

Larry:
It sounds like you would argue the two things to the core of that is, get on top of your data, make it shareable, findable, connectable, all those things… And have an ontological understanding of it, and then project that out. Is that sort of it?

Tony:
That’s it, yeah. Yeah, that’s it. So if I was to summarize it, I would say that your data needs to be in a certain format to be used with AI. And it needs two things, or there are two things that are important to it, I think. One is connections at the end of it. The more time you spend thinking about data, data is basically all about relationships. So your data needs to be richly connected, and that’s not connected in a relational database. Since I’ve got a primary key and a foreign key between these two tables. No, it’s like meaningful semantic relationships between items. Where I’m saying, “This person is a friend of this person,” or, “This person really doesn’t like this other guy.” And so I have this kind of rich network.

Tony:
That’s one thing. And then I guess the other thing I’m leading onto by already saying that meaningful relationships is the semantics. So it’s the business. It’s the kind of concepts in the language of that business. And those are two aspects that are, I think, fundamental about the shape of the data, the shape that the data needs to be in order to be able to effectively use AI with it.

Larry:
I love that you shaped, that you framed the solution to this around the community. That’s what I’m all about, just talking to people and building community. And I just want to make sure, and I’ll link to the Knowledge Graph Conference, Connected Data London, and the Semantics Conference, Amy’s new Graph Geeks Podcast. Are there other resources or other places for people to gather and figure this stuff out?

Tony:
Yeah. I mean, LinkedIn as well. There’s quite an active community of different people on LinkedIn. I don’t know exactly. That’s kind of what comes to mind at the moment. I think maybe we are lacking with more. I mean, there’s podcasts like you are doing. Maybe we are lacking some stuff.

Tony:
I mean, one thing that I’ve been behind recently, chaired a semantic data products working group, and we’ve released a specification, which is called DPROD. And it’s basically a semantic definition of what a data product is. And my big hope with that is that this perhaps is a kind of foundational point, that the community can have something to sort of work with at the very basic level. So if you are into doing stuff within this space, then when you are exposing data out into the network with rich semantics and connectivity, then you could use the DPROD specification to say, “Here you go. I have a data product, a semantic data product that I’m making available.” So that also is one, I think, important first step.

Tony:
The kind of hope is that the graph community can then combine together by using that. Different vendors can map into the semantic data product specifications. So when they bring their products in, they can do that. Different people like LSEC for instance, are publishing within the semantic data their information in the data product. And then certain large investment banks with tight control over their PR relationships are also mapping into that, internally within that specification. So I’m interested to see where that goes and how that takes on, because that’s possibly a way of the various different people who are interested from a vendor, from an organizational perspective, mapping into something very basic.

Larry:
That sounds like a really good service. Because I’ve heard people talk about data products all the time, but I get the sense that there’s not a lot of disambiguation of that term. So to have a formal working group come together and work over a definition of semantic data products, that seems really foundational. Good. Yeah, I’ll definitely link to that.

Larry:
Hey, Tony. I can’t believe that we’re coming up close to time. But before we wrap up, is there anything last that you want to make sure we either reiterate or share from earlier in the conversation? Or just something you want to make sure the folks know before we leave?

Tony:
I feel like we’ve covered some good stuff here. I guess a shameless plug that a lot of these ideas are very detailed. So if you’re interested in that, then come follow me on LinkedIn where I publish information about it. And also in shameless plug mode, I’m just about to start up a consultancy. So if people are interested in having a chat, then very happy to do that.

Larry:
Excellent, yeah. Oh, and that’s the very last thing too. So I’ll definitely include your LinkedIn profile. Is there any other contact info you want to share? Or is that the best place to find you?

Tony:
Well, yeah. In a couple of weeks I’m launching a website, which will be like the Knowledge Graph Guys website. So maybe I’ll give you the link to that and hopefully it’ll be online by the time-

Larry:
Yeah, as soon as that’s up, I’ll add it to the… Is it up now? It’s not up yet?

Tony:
Yeah, a bit soon. Yeah.

Larry:
Okay. As soon as it’s up, I’ll definitely add that to the episode webpage.
Well, thanks so much, Tony. I’m both inspired and terrified and motivated for sure. So thank you.

Tony:
Thanks.

Tony Seale: The Knowledge Graph Guy – Episode 11

Tony’s bio

Connect with Tony online

Resources mentioned in this podcast

Video

Podcast intro transcript

Interview transcript

About The Author

Larry Swanson

Leave a Comment Cancel Reply