Podcast: Play in new window | Download
Subscribe: Apple Podcasts | Spotify | Amazon Music | Android | Youtube Music | RSS
Yaakov Belch is an AI researcher with strong ideas about the role of humans in AI systems. Instead of “human in the loop,” he argues, we should put “humans in control.”
Yaakov’s research looks at business contracts and how knowledge graphs and AI systems can both capture their meaning more accurately and help managers make better business decisions.
We talked about:
- his assertion that we need humans in control, not just in the loop
- his research on applying AI technology to business contracts, in particular the issue of resolving inconsistencies in language model results
- reasons to put human concerns ahead of any particular technology
- the importance of having humans in control when interpreting ambiguous business decisions
- the importance of both accounting for business intent and asking the right questions of your data and how the loop between the two tightens over time
- the responsibility of human users to understand how LLMs work and to prompt and otherwise interact with them accordingly
- why he doesn’t use the term “hallucination” when talking about LLM outputs
- the role and implications of applying different kinds of logic in the use of knowledge graphs
- an analogy that shows how the concept of a Git fork can help knowledge graph engineers account in their models for different versions of reality
- the real-world applications of his research, especially how the practices he is exploring can create new business value
- the importance of building any model off of real data and always thinking about which human needs to be in control
Yaakov’s bio
As a mathematician and data scientist, Yaakov Belch brings a unique perspective to the world of AI and knowledge graphs. With a strong background in mathematics, including participation in prestigious International Mathematical Olympiads, Yaakov went on to earn a Ph.D. in pure mathematics from the University of Cambridge.
Yaakov’s career has spanned both research and industry roles. He has worked as an Algorithm Programmer, collaborating with researchers in bioinformatics and economics, co-authoring academic papers. Yaakov also served as a Senior Data Scientist at Israeli e-commerce startups, where he tackled challenges in symbolic and semantic search from different angles.
Currently, Yaakov is on a sabbatical, working as an independent Data Scientist to develop his method of reliable business reasoning, precise contract understanding, and humans-in-control.ai. He sees an interesting connection between the problems from the International Mathematics Olympiads and taming the inconsistencies of large language models: “At one hand, the problems are hard and just don’t open up to the known, standard methods of the field. But you know that there is a beautiful solution. You need to find the right perspective to appreciate the problem and to see that beautiful solution.”
Connect with Yaakov online
Video
Here’s the video version of our conversation:
Podcast intro transcript
This is the Knowledge Graph Insights podcast, episode number 17. Machine learning architects often talk about the “human in the loop.” Yaakov Belch thinks that when it comes to language models the right approach is to put “humans in control.” Yaakov’s research looks at how knowledge graphs and large language models can help put humans in control of business contracts, capturing the actual intent that underlies them and facilitating better business decision-making based on the discoveries that they enable.
Interview transcript
Larry:
Hi everyone. Welcome to episode number 17 of the Knowledge Graph Insights podcast. I am really delighted today to welcome to the show Yaakov Belch. Yaakov is an independent senior data scientist and he’s made this really provocative statement about … There’s all this talk in the AI world in general about humans in the loop. And Yaakov says, “No. We need humans in control.” So Yaakov, I would love to talk about that today.
Yaakov:
We don’t need humans in the loop. We need humans in control. This is a paradigm which crystallized over time in the research which I’m doing, about how to apply language models for contract understanding in the business context. Let’s unpack that and see where the human in control comes in. A business contract is a memorization, a written-down expression of agreement between two or more parties; with the intention to fulfil that agreement in the future. It contains provisions for what the parties promise to do, the rights and obligations they have. And also descriptions what you do when things go wrong. After you make an agreement, you actually want to execute it, you want to fulfill it.
Yaakov:
When you make a large contract, some people negotiate the contract and other people will be doing the work. There are situations where the people who actually do the work are not really aware about what exactly has been promised in the contract. The huge contract is written in a way as contracts need to be written. But it can’t be understood on the spot when the person needs to decide: Is this promised or not? There can be a disconnect between the expectations of the customer and the provider based on just not being aware of what’s in the contract. In a more advanced setting, you may want to understand the risks which are in the contract in hypothetical situations. When you draft a contract you want to make sure that you’re not setting up yourself for problems when things go in a specific way. Yet more advanced: When you purchase a property, you may acquire contracts which go with them, like liens, leases and bills. In your due diligence work, you want to check that every liability has been properly addressed, so that you don’t acquire additional risks with the property. And if there are problems, you want to resolve them.
Yaakov:
I want to understand contracts from a business perspective. I want to use language models to understand their language. It’s not a surprise anymore that language models can help with that. You put in part of a contract. You can ask a question. You can get an answer. What is problematic is that if you don’t do it right, these answers will not be consistent. You may ask the same question twice and get two different answers. Or you get an answer that is justifiable, but it’s not consistent with the intention of your question. It is against your business goals. Some answers may be completely disconnected from reality. The essence of this research project is how do we deal with this inconsistency? How do we create a reliable system where language models are just a part of it? We have knowledge graphs, we have logic, and we have humans in control … So that businesses can rely on it for their business needs. How do we detect and how do we deal with all the mistakes which happen with language models? But also in your data you have mistakes. And how do we detect them? How do we deal with them? That’s the research.
Yaakov:
In this research, I find that you really need to be careful to use the right paradigms of what you’re doing. If you just apply a playbook from machine learning or from expert systems and just try to do hard work, you may break yourself against language models being different. It’s a new technology with new characteristics. You need to adjust your goals, your paradigms of work, and the structure of your work to the capabilities and to the limitations of the problems of language models. One key point is to understand the right role that humans must play in the whole scheme. It’s not technology alone, it’s technology and people. We don’t need people in the loop. We need people in control.
Larry:
Okay. We actually had a long conversation last week to set this up and I was beginning to really get it, but that was an excellent summary of your research and the business insight and the technical challenge that comes into this. I’d love to really focus on how knowledge graphs figure into this. I’m inferring that there’s a business ontology and stuff that drives… But can you talk about the specific role of knowledge graph in how you see it helping address these new paradigms that we need?
Yaakov:
For sure. This is an excellent question, especially in the audience of Knowledge Graph Conference and knowledge graph podcast. I suggest, however, that before we talk about the knowledge graph, which is basically a technology which supports this work, to first understand: What does this really mean, human control? How does it work? What do we want to achieve? What is the goal? Then, let’s understand the mistakes of language models. Their answers may be inconsistent with what you want. But they are not just making random mistakes. Where does this come from? And then we’ll see how knowledge graphs and logic play an important role. So maybe we go into three steps and get the knowledge graphs at the end.
Larry:
That sounds perfect, because that makes knowledge graph the culmination, which is what we’re looking for. It’s a knowledge graph podcast. But, no. So the first thing you said is, what does that really mean, humans in control? Yeah. I’d love to hear those three steps you just mentioned.
Yaakov:
Okay. Humans in control versus Humans in the loop. When you have a machine learning system, you know you need training data. When you get incorrect results, you want to double-check and correct them with a human in the loop. As a benefit, you create training data that will hopefully make your model more precise. But this usually doesn’t work. It works only when you do it right: When you do this with one human in control in addition to many humans in the loop. Without that, the people in the loop either don’t know the right answers, because they’re not experts. Or they are experts and don’t want to answer your question: you overload them with repetitive work, which is not their job.
Yaakov:
But, when we deal with large language models, we don’t need to create training data. We need to give the language model the right context, the right instructions on what to do. We need to decompose problems into the right small pieces, in order to have explainability, in order to get a handle about when and how things go wrong. That’s a different challenge. So how does “humans in control” work?
Yaakov:
Let’s look at one example. You get a customer request to do some additional work. You have a licensing agreement with the customer and it includes support. You think you have a clear picture of what kind of support you give to your customers. But this customer does something out of the normal. He wants your support for his innovation, something which you didn’t really think of when you created your technology. How do you respond? If you promised this kind of support in your contract, you have to supply. If not, what do you do? Your field engineer has to decide yes or no: “No. That’s not part of this contract. We don’t usually do this. We don’t support you.” Or: “Yes. It’s a new problem. Tell me about the details. We get right into it.” This decision is not a decision for a language model and it’s not a decision for a data scientist. It is a business decision. I want to create a system which helps you to give the right answer.
Yaakov:
Let’s dig deeper. One business makes high profit out of scale at low costs: They have a product and sell the same product to many customers. And every special request is a burden. A burden you need to reduce in order to get to the low cost, in order to make your profit. You have to streamline your customers as you streamline your business. On the other hand, you may want to expand into new markets. Or you may have a service business: A customer shows that your product may have a new use case, may have potential to create additional value … You see this as an opportunity, you want to jump on it. If your lawyer tells you, “You don’t need to provide the support, you don’t need to get into that opportunity. You didn’t promise it, don’t do it.” You don’t listen to this lawyer. You make a decision about our business, where do you want to get into and where not.
Yaakov:
The right answer to this customer request really depends on a business decision. Sometimes, the CEO makes the core decision, and then delegates the details to somebody else. He says, “This is a key growth product. We want to develop it and we want to support as many applications as possible.” Or, “This is a product that we want to streamline or maybe even we want to phase it out of the market in order to promote different, newer products.” If you don’t connect this customer request with the intent of the CEO or the right person in the organization who makes the business decision … you cannot know the right answer. Whether it’s a human, who is not connected, or whether it’s a language model which is not connected, or whether it’s a data scientist who guesses what should be the right answer. Without that connection, you make wrong decisions and damage the business relationship with your customers. You damage the business. You want the CEO to be in control. The CEO decides, “This is what we want and this is what we do.”
Yaakov:
Now you have a requirement: You cannot put the CEO or his intermediate manager into the loop to double check every contract. You must take and accumulate questions and feedback. You must articulate the right problem, to the right person. In a way that the right decision maker can make the right decision. You may need to gather information from many clients before you ask the question and make a decision. You may need to work with past data, you need to compare similar situations in the market and give the decision maker the right background to make a decision.
Yaakov:
You need to let him make the decision in the way he thinks, not necessarily as a one by one, yes, no decision, or by a rule in a specific rule language. You have to connect with the brain of this decision maker. How does he make his decisions? You have to capture this and put it into the system. This is what organizations do manually. With language models, we can involve computers, to make that process more efficient and also more clear. So the idea-
Larry:
I was just going to say, so the mechanism there, there’s a lot going on in there. And it’s like the ultimate goal being to support the overarching strategy and goals of the business. But especially that, and you were just getting to … And that’s what I’d love to elaborate on a little bit, that it sounds like knowledge capture, good old-fashioned knowledge management and ontology design capturing not just subject matter expertise but capturing business intent in some way, so that you can do more with it. Is that the point?
Yaakov:
Yes … That’s half of it. It’s really a two-way conversation. Yes: capturing the real intent of the decision maker is really one important half. The other important half is that the decision maker learns from the system. The system asks the right questions to the right person, prompts from the decision makers the right insights with the right question, which comes from your data. So once you have data, you have user interactions and you get a user request which nobody thought of before. You bring this to the attention of the right person in the right way and prompt for the right answer. And what actually happens is, when you develop a system like this, initially, you have a lot of questions for the decision maker. He’s teaching you what’s important to him, how he thinks, how he wants and should be engaged with the system. You learn from the decision maker, you capture from him.
Yaakov:
But, because you capture the information the right way, you won’t ask the same question over and over again. And you will get to a point where most of the things which you can learn now, and what the decision maker, the expert can teach you, are already answered. This work of teaching the system gets less and less over time. Even when you work for more customers. The common questions are automatically answered. But the one thing which happens now. Instead of the expert teaching the system what he knows from his past experience, he now starts to learn, because he gets exposed to all the new things which happen in his field of expertise. New things he may not yet be aware of. Things which just haven’t come up before. There is a new type of question. The engine concludes, “I don’t know, I have to ask the expert.” This expert gets real-time curated insights in what’s new, what’s happening in the market.
Yaakov:
There’s this flip where initially the expert is teaching and he’s providing value and then the expert is gaining value because he strengthens his expertise and control by being directly connected to what’s going on. This is the second half: the conversation. And maybe when we get to talk about knowledge graphs, we will talk about the conversations between the experts.
Larry:
Okay. Yeah. I would love to get to that. But, it sounds like what you’re beginning to talk about that’s really intriguing that you’re talking about prompting to get the right … It’s asking the right questions to get the right answers and that’s kind of an accumulation of business intelligence it’s building through this process. But then, you talked earlier about dealing with the inconsistent nature of LLMs. What you just said sounded a little bit like prompt engineering and tailoring your prompts to get better answers. Can you do that systematically? Is that what’s going on there?
Yaakov:
Okay. I was making a bit of a pun when I said “I prompt the expert with the right question to get his answer.” But there is a similarity in which we can get right now between prompting humans experts and prompting the language model. And so, let’s speak about what’s the right paradigm to think about inconsistent answers, wrong answers from language models. You may realize that I very much try to avoid the word “hallucinate.” Why? Because I think at the moment when I say that word, when I put the blame on the language model, when I say “The language model makes the mistake. It made up something.” And when I just use a standard word for it that this is some random error, then I am not doing my responsibility. As a prompt engineer, as a system engineer, I have a responsibility to deal with these problems. I need to take responsibility. When I put the blame on the language model, I just gave up. So I try never to say that word.
Yaakov:
But when I say, “Well, the language model makes up some words. The language model gives the right answer to the wrong question” … Then I have an opening to fix the problem: If it makes up information, I know I have to provide it with the right information. If it gives the right answer to the wrong question, I have to clarify my question. If the language model is overwhelmed and focuses on the wrong part of the question, or ignores a part of my question, I need to break things down into smaller pieces. This is a part now where it gets technical. We are now the CEO and the business people can tune out of the talk and engineers want to tune in. “How do I decompose? How do I deal with language model errors?”
Yaakov:
And there is a paradigm which helps me a lot: “anthropomorphism”. To think of a language model like a human organization. There is a wrong way to do that and there’s a right way to do that. The wrong way is to say: “A human has two hands, two legs and he needs to sleep and eat. So for sure if we think of the language model like a human, then, for sure, it also needs two legs. It needs to eat and sleep.” This is the wrong way. “Well, it needs eyes and he needs a mouth” … This really does not capture what’s going on here. This is the wrong way of drawing conclusions from comparing new language models to human organizations. That’s the right way.
Yaakov:
Here is the right way. Let me take an example: “The White House says that it’s important to do something.” What does this mean, “the White House says something” The meaning is: There’s an organization which is led and controlled by the president of the United States. This organization has rules and problems, political problems, biases, … and here’s this person who speaks on behalf of that organization. An opinion that should be in line with the decision of the president of the United States. When “the White House says something”, a real human speaks … on behalf of this organization. There is no surprise in that capacity of speech. A real human speaks. What is interesting is how the “speech of the White House“ is a mixture of the many people involved. You could even say that the president himself acts like such an organization: Many of the themes and words which he’s saying are actually written by other people and he’s just connecting them in the right way to create this message. That is a human organization.
Yaakov:
Let’s look at language models. Their training data is written by real humans and the answers to our questions are really a combination of these answers written by real people. And the preferences that select which answers should be preferred are shaped by humans in the fine-tuning of the language model. Can you look just at this GPU, and think there’s language coming out of this GPU? No. The language comes out of the training data. The training data comes from people, from real people with real sentences, reactions, thoughts and values. These real humans define what you get out of the language model.
Yaakov:
Here is another analogy, from the legal world: You have the concept of a legal person, which can be one person or an organization. In contract theory, you almost never restrict yourself to a real person. A legal person can be represented by a real person, by an organization or by a one-person organization. And then you have a “legal entity”: an organization without the people. You can say a trust is powered by a trustee. But it’s a thing which exists independently, even if you exchange the trustees. You could say the language model is like a trust. The GPU is powered intellectually by real people, from the training data. When you want to understand the answers of the language model, you need to understand what these real people want, what these people do.
Yaakov:
For example, you know that there are different systems of law, like common law and civil law. In the training data, real people answer one question in similar but subtly different ways. You must be prepared that you get an answer from either from here or from there … unless you specify in your prompt what legal system you want. To select the right direction for your answer, you need to understand how the experts write about this kind of question, and how they think about it. What kinds of options do they have? Either you provide the information and framework you want in the prompt, and guide the language model to answer consistently with your goals. Or if you leave out that information, and the language model will make up. It will guess for you. And if you leave out factual information then it will guess reality, which is … making things up … or that one word, which I don’t use anymore.
Yaakov:
But, that’s just one aspect. There are many more technical aspects like how to understand when you need to break down complicated questions and tasks into smaller pieces, to make things explainable. Let’s not go into this right now, but this paradigm of understanding things as a human organization, what it is, to point to real people, makes a huge difference in understanding and dealing with misunderstandings.
Larry:
I love that notion of anthropomorphism as the foundation. Well, if you’re talking about humans in control, it’s like you got to understand them correctly and use metaphors and analogies about them in the right way in this whole technical and business infrastructure that you’re talking about. I think that’s fascinating. And when we talked last week, you mentioned paradigms, I think you said. We’re coming close to time, but I wonder if you can summarize the paradigm shifts we need to make in our brains to support this kind of business insight you’re talking about?
Yaakov:
We covered paradigms about the right role of humans in the picture. There are also technical paradigms. So let’s dive into your question about knowledge graphs. How do we need to think about knowledge graphs? A contract … is very structured information. You have obligations, you have parties, you have rights. And you have explicit references to other documents and parties. The lawyer who writes the contract really puts a lot of emphasis on making this a really structured piece of information. It’s an interesting question, how to capture the structure into a graph that a computer can actually analyze and “think about it”.
Yaakov:
Knowledge graphs. The real question is not how to represent your information as a graph of cells. The real question is: What do you do with it? How do you query it? What logic do you use? How do you make inferences? A common mistake … I think … is that people say, “Well, I have a graph engine, I have a graph database and it has some reasoning capabilities. When I speak about reasoning, it must be that I use the logic which is just hardwired into my engine. And I have to find a way to make everybody use this kind of logic.” And this is a mistake, because there are different types of logic … If you study mathematics, you’ll see that there are different types of logic, each with their own purposes. And you need to use the right logic for the right purpose for instance.
Yaakov:
Predicate logic, which is often hardwired into knowledge systems, is about … amplification. If you have correct axioms, you derive more correct statements, more correct answers from a few axioms. So you amplify what you know. But if you have incorrect axioms, if you have some incorrect piece of information, then you amplify that incorrect statement and you get many incorrect statements as a result. This is the power of predicate logic. And it’s one reason why mathematicians are careful when they inference extensively with predicate logic because they know that it has a specific realm of application.
Yaakov:
Now, what we’re really interested in is that the real world has inconsistencies. They are there and the language models are adding to it. We have inconsistent answers from language models and we need to detect them. So we need to capture the messy world in some way and we find out where things are inconsistent. And then we try to ask and answer: how do we correct this? Predicate logic is not really helping us for that goal.
Yaakov:
There are other technologies like SHACL that detect violations of expectations and reject them, which is closer to what we want. Detect inconsistencies and reject them … But, really, we don’t want to reject them, we want to manage and handle them, we want to refine the knowledge. We need a paradigm of capturing in one knowledge graph and in one data structure, different versions of reality, of different views of different people and comparing them to find out what we need to deal with.
Yaakov:
If you are a programmer, you know Git. Before Git, “fork” was a four-letter word. So you don’t fork, because that breaks the attention of people into smaller projects. But once you have Git and you have ways to compare different branches, and you have ways to reconcile and to merge them, then everything that you do is: you make a fork. You don’t change any project anymore. You make a fork, you may change that and you merge it back. And you live easily with many different versions of reality in your project. And we need to use the same paradigm for allowing different views of the world. And to handle them and to eventually, also to produce sometimes one main version, which we base our decisions on.
Larry:
I got to say I love that idea that you can use knowledge graphs, because so many ontology and knowledge graph projects start with an attempt to capture a version of reality. But, the same mechanism can be used to manage multiple versions. And I love the Git analogy. I think that’s a way that I think everybody gets that model and can imagine you’re overlaying it on the whole ontology and knowledge-graph development process.
Larry:
But, hey Yaakov, I can’t believe we’re coming up close to time already. But I want to make sure, is there anything last, before we wrap up that you want to revisit from the conversation, or just make sure that we share?
Yaakov:
I want to look at the practicalities. This research is aimed to actually apply to real organizations, real companies. And we are now getting to that. I have built this communication engine, which connects these pieces, some contract-understanding engines and I put this all together. And I’m starting to work with companies, organization experts of them, to see, “Well, tell me what’s important to you? And how do you interact with the system? How do you interact with other people in the same system?” And I’m just now in the beginning of December starting to actually switch this on. And I would like to stress something about the expectations which my customers have, and should have to succeed.
Yaakov:
So one thing which is very important is that you have to make a choice whether you want to cut costs or whether you want to create new value. If you want to cut costs and leave as much as possible the same, just replace expensive people by cheap language models, you will not put people into control. It’s a different world. But, if you need to create value, for instance, getting control of the agreements which you have at the right time, then you have the right setting to make the changes about how people communicate through technology. And you may end up with a situation which also saves you a lot of money. But, the right approach is to identify a way to create new value, which motivates change.
Yaakov:
Second point already explains the expectations that initially the experts will contribute their knowledge to the system to connect with the right way. But after a tipping point, the expert will actually learn from the system what’s new, what’s exciting. And the expert will benefit from it. And you have to know that this is what’s going to happen.
Yaakov:
The third point is a consequence of this. We need relevant, real data to start this process and push it over this tipping point. We need real contracts that are relevant to you. We need real-business scenarios to start this whole process of conversation between the expert and with the model. And to get to that tipping point fast. So that we don’t want to wait for a lot of customers to provide this data until we get to the tipping point where the expert is connected and he benefits from the system. So these are things which customers of this technology need to validate. Do we want to create value? Who are the people who need to be in control? And are they willing to make some changes for gaining this additional value? Do we have or can we get the data?
Larry:
Great. I love the emphasis at the start on value creation over cost-cutting. Both will happen, of course, but it seems like the real opportunities may lie in more creative thinking about how to do that. And I’m sure this will inspire conversation desires on the part of my listeners. If folks want to connect to you, Yaakov, what’s the best way to find you online or connect?
Yaakov:
My LinkedIn contact is in the notes below. I am on the Knowledge Graph Conference Slack, so you can contact me over there as well. I’m very slow at writing things on Medium, so maybe put a link there as well, but don’t expect too much right now. But, it’ll grow in the future hopefully. You can also shoot me an email. And if you’re really into applying legal understanding to your business, then we can set up a video call and really have this conversation. What is the control which you need to put into place? How can contracts, legal understanding help you for that?
Larry:
Oh, great. Well, I’ll put all that in the show notes as well. Well, thank you so much Yaakov. I always enjoy talking with you and this is a particularly enlightening conversation.
Yaakov:
Thanks. The same from my side. You are a very great conversation partner.
