We Mapped 115 Microservices for Our Coding Agents — Kamalakannan Nandagopal, Postman

Today I'll talk about Today I'll talk about how to take your how to take your how to take your agents agents agents beyond beyond beyond code generation and how API code generation and how API context can be context can be context can be the answer to this the answer to this the answer to this question. I am Kamal. I am a question. I am Kamal. I am a question. I am Kamal. I am a lead engineer at lead engineer at lead engineer at Postman. And Postman, as you Postman. And Postman, as you Postman. And Postman, as you know, is a know, is a know, is a comprehensive API comprehensive API platform. But in addition to the platform. But in addition to the platform. But in addition to the desktop application desktop application desktop application that allows you that allows you that allows you to test and to test and to test and create REST APIs, Postman create REST APIs, Postman create REST APIs, Postman also has many also has many also has many cloud features. All of cloud features. All of cloud features. All of these cloud capabilities are these cloud capabilities are these cloud capabilities are built on a built on a built on a distributed distributed distributed microservices microservices microservices architecture. We have architecture. We have architecture. We have over 150 over 150 over 150 microservices and microservices and microservices and thousands of REST API thousands of REST API thousands of REST API endpoints. So I'll endpoints. So I'll endpoints. So I'll draw on our draw on our draw on our real-world engineering real-world engineering real-world engineering cases and cases and cases and workflows, our experience workflows, our experience workflows, our experience with with with coding agents, and how coding agents, and how coding agents, and how we improve their we improve their we improve their performance performance performance using the API using the API context graph. Our context graph. Our context graph. Our journey with journey with journey with coding agents began the same way as coding agents began the same way as many of you did. many of you did. Starting with Starting with Starting with autocompletion at the autocompletion at the autocompletion at the line and line and line and function levels—this is when function levels—this is when function levels—this is when Copilot appeared. Copilot appeared.

Copilot appeared. Moving on to Moving on to Moving on to refactoring the entire refactoring the entire refactoring the entire codebase and codebase and codebase and implementing implementing implementing one-click solutions—this is the era of one-click solutions—this is the era of one-click solutions—this is the era of Composer and cloud Composer and cloud Composer and cloud development environments. development environments. development environments. We are now moving We are now moving We are now moving towards fully towards fully towards fully autonomous autonomous autonomous agent workflows. agent workflows. agent workflows. We have agents who We have agents who We have agents who fully perform fully perform fully perform the tasks: from starting the the tasks: from starting the the tasks: from starting the workflow to workflow to workflow to making changes, making changes, making changes, checking them, sending checking them, sending checking them, sending pull requests, and even pull requests, and even pull requests, and even verifying them. We verifying them. We verifying them. We see that see that see that coding agents do a great coding agents do a great coding agents do a great job with job with job with projects from scratch projects from scratch projects from scratch or with fixes or with fixes or with fixes and improvements within the and improvements within the and improvements within the same project. same project. But production But production systems are usually systems are usually systems are usually a bit more complex. These are typically a bit more complex. These are typically a bit more complex. These are typically distributed distributed distributed systems divided into systems divided into systems divided into many microservices many microservices . They are often . They are often . They are often deployed in different deployed in different deployed in different environments, and even environments, and even environments, and even in different versions for in different versions for in different versions for each of them. We have each of them. We have each of them. We have several applications: several applications: several applications: frontend, CLI, API that frontend, CLI, API that frontend, CLI, API that interact with our interact with our interact with our infrastructure, and infrastructure, and infrastructure, and all of this is distributed all of this is distributed all of this is distributed across hundreds of across hundreds of across hundreds of repositories. When we repositories. When we repositories. When we think about think about think about improving improving agent productivity, the first area agent productivity, the first area we look at we look at we look at is is is documentation.

documentation. documentation. Documentation is a great Documentation is a great Documentation is a great place to start, but place to start, but place to start, but we all know that we all know that we all know that developers developers developers hate writing hate writing hate writing and maintaining it. This is an and maintaining it. This is an and maintaining it. This is an additional waste of additional waste of additional waste of time. It's expensive, and over time. It's expensive, and over time. It's expensive, and over time it becomes increasingly time it becomes increasingly time it becomes increasingly difficult to keep it up to date difficult to keep it up to date difficult to keep it up to date . . . But if you entrust But if you entrust But if you entrust the writing and the writing and the writing and maintenance of maintenance of maintenance of documentation documentation documentation exclusively to LLM, exclusively to LLM, exclusively to LLM, studies studies studies show that such show that such show that such documentation documentation documentation quickly loses quickly loses quickly loses quality over time. Some form of quality over time. Some form of quality over time. Some form of human participation is human participation is human participation is definitely needed definitely needed definitely needed to make it to make it to make it effective. Skills and MCP effective. Skills and MCP are a great are a great are a great alternative, but alternative, but alternative, but we all know that they are we all know that they are we all know that they are only as effective as the only as effective as the quality of the context they are given quality of the context they are given . . . Agent memory is actually a Agent memory is actually a Agent memory is actually a very good solution very good solution very good solution for this scenario. for this scenario. for this scenario. In terms of In terms of In terms of real-world experience, real-world experience, real-world experience, most of this most of this most of this knowledge is probably already knowledge is probably already knowledge is probably already informal informal informal knowledge in knowledge in knowledge in people's heads, and now it is people's heads, and now it is people's heads, and now it is being transformed into being transformed into being transformed into agent memory agent memory agent memory within the encoding within the encoding within the encoding agent. The biggest agent. The biggest agent. The biggest limitation we limitation we limitation we see is that see is that see is that agent memory is still agent memory is still agent memory is still very limited to an very limited to an very limited to an individual user individual user , and there are currently no , and there are currently no standard models standard models for sharing for sharing for sharing agent memory across broader agent memory across broader agent memory across broader teams or teams or teams or your entire engineering your entire engineering your entire engineering organization. While we organization. While we organization. While we were trying to solve were trying to solve were trying to solve this problem for this problem for this problem for coding agents coding agents coding agents on the development side, our on the development side, our on the development side, our go-to- go-to- go-to- market teams were also market teams were also market teams were also building business building business agents for their agents for their agents for their end-to-end end-to-end end-to-end execution tasks.

execution tasks. execution tasks. So, they created So, they created So, they created specialized specialized specialized agents designed agents designed agents designed using customer data from using customer data from multiple sources, and they multiple sources, and they went through multiple went through multiple went through multiple iterations to iterations to iterations to organize all of that organize all of that organize all of that unstructured data unstructured data unstructured data into structured into structured into structured information that information that information that these these these agents could operate on. As a result, agents could operate on. As a result, agents could operate on. As a result, they got a highly they got a highly they got a highly optimized optimized optimized RAG pipeline plus a RAG graph, RAG pipeline plus a RAG graph, RAG pipeline plus a RAG graph, which serves as a which serves as a which serves as a centralized centralized centralized context layer that context layer that context layer that forces all these forces all these forces all these individual business individual business agents to work agents to work agents to work efficiently and efficiently and efficiently and solve common solve common solve common tasks. So we tasks. So we tasks. So we started thinking about how started thinking about how started thinking about how to learn from this and to learn from this and to learn from this and what can we do what can we do what can we do to significantly improve to significantly improve to significantly improve the performance of the the performance of the the performance of the encoding agent? encoding agent? encoding agent? And when we And when we And when we analyzed this, we analyzed this, we analyzed this, we realized that in realized that in realized that in distributed distributed distributed systems and systems and systems and microservice microservice microservice architecture, APIs are a architecture, APIs are a architecture, APIs are a context level for you. context level for you. Complex engineering Complex engineering architecture is often architecture is often architecture is often broken down into broken down into broken down into domains and models, domains and models, domains and models, each microservice each microservice each microservice is responsible for one is responsible for one is responsible for one specific function specific function specific function according to the model or according to the model or according to the model or domain, and APIs domain, and APIs domain, and APIs define what define what define what that that that responsibility is. And responsibility is. And responsibility is. And API calls between API calls between API calls between systems are systems are systems are typically how you typically how you typically how you define define define workflows and workflows and workflows and user journeys that user journeys that user journeys that span multiple span multiple span multiple systems. Therefore, systems. Therefore, systems. Therefore, the effectiveness of agents the effectiveness of agents the effectiveness of agents depends only on depends only on depends only on the context they are the context they are the context they are given. So we set given. So we set given. So we set out on a journey out on a journey out on a journey to create the to create the to create the best possible level of best possible level of best possible level of context for context for context for agents. We started agents. We started agents. We started building an building an building an API context graph for what the API context graph for what the Postman engineering architecture looks like.

Postman engineering architecture looks like. The journey began with The journey began with The journey began with identifying and identifying and identifying and cataloging cataloging cataloging every microservice every microservice that was in the that was in the that was in the production production production environment. and then environment. and then environment. and then we identified we identified we identified each endpoint each endpoint each endpoint exposed by each of these exposed by each of these exposed by each of these REST API services. and then we REST API services. and then we REST API services. and then we documented how documented how documented how each of each of each of them was implemented, down to the them was implemented, down to the them was implemented, down to the specific line of specific line of specific line of code where it was code where it was code where it was implemented. We also implemented. We also implemented. We also mapped how each mapped how each mapped how each service communicates service communicates service communicates with each other and how with each other and how with each other and how each endpoint each endpoint each endpoint connects to every connects to every connects to every other other other endpoint. We also endpoint. We also endpoint. We also tracked how data tracked how data tracked how data moves through the entire moves through the entire moves through the entire stack, including stack, including stack, including databases, caches, and everything databases, caches, and everything databases, caches, and everything else throughout the system. else throughout the system. We also defined We also defined how applications such as how applications such as how applications such as frontends, CLIs, and APIs frontends, CLIs, and APIs frontends, CLIs, and APIs interact with the entire interact with the entire backend architecture and infrastructure. We collect backend architecture and infrastructure. We collect all this data and all this data and all this data and index it index it index it using LLM to using LLM to using LLM to create a create a create a centralized and centralized and centralized and efficient efficient efficient context graph layer. But context graph layer. But context graph layer. But one of the key one of the key one of the key principles we principles we principles we always followed always followed was: every was: every was: every data point that enters the data point that enters the data point that enters the context graph context graph context graph must be based must be based must be based on irrefutable on irrefutable on irrefutable facts—either a line of facts—either a line of facts—either a line of code implemented in code implemented in code implemented in our database or an our database or an our database or an accurate trace from accurate trace from accurate trace from product product product telemetry. With telemetry. With telemetry. With that in mind, we that in mind, we that in mind, we started by started by started by filling in our filling in our filling in our context graph, and this is a context graph, and this is a context graph, and this is a snapshot of what snapshot of what our engineering our engineering architecture roughly looks like. So, architecture roughly looks like. So, some very common some very common patterns are starting to emerge that you would patterns are starting to emerge that you would patterns are starting to emerge that you would expect. You expect. You expect. You start to see a start to see a start to see a central cluster of central cluster of central cluster of core services, core services, core services, very tightly very tightly very tightly woven APIs and woven APIs and woven APIs and services that have services that have services that have strong dependencies on strong dependencies on strong dependencies on each other. You each other. You each other. You find one or

find one or find one or two very bloated two very bloated two very bloated services that services that services that are on the are on the are on the outer boundaries of these outer boundaries of these outer boundaries of these systems. Those in systems. Those in systems. Those in the center are usually the center are usually the center are usually classified as classified as classified as zero- zero- zero- or first-level services. or first-level services. So, now that So, now that we have a context graph, we have a context graph, we have a context graph, we start we start we start to measure exactly what to measure exactly what to measure exactly what improvements it provides improvements it provides improvements it provides for for for programming agents. We programming agents. We programming agents. We took real-world took real-world took real-world examples from examples from our our our organization's Git repositories in Postman. We organization's Git repositories in Postman. We organization's Git repositories in Postman. We took real took real took real pull requests (PRs) created pull requests (PRs) created pull requests (PRs) created in GitHub repositories and, in GitHub repositories and, in GitHub repositories and, for each one, for each one, for each one, determined what determined what the developer was trying to do and what the the developer was trying to do and what the actual result was actual result was actual result was after the PR was completed. after the PR was completed. after the PR was completed. We also took We also took We also took important projects and important projects and important projects and key design key design key design decisions, as well as decisions, as well as decisions, as well as conclusions that conclusions that conclusions that were discussed during were discussed during design discussions. We design discussions. We mapped each of mapped each of mapped each of them into an evaluation (eval) and them into an evaluation (eval) and them into an evaluation (eval) and mapped the actual mapped the actual mapped the actual decision made decision made decision made retrospectively as the retrospectively as the retrospectively as the truth we wanted to truth we wanted to truth we wanted to confirm with confirm with confirm with these evaluations. these evaluations. To perform To perform the checks, we the checks, we the checks, we connected the Postman AI agent connected the Postman AI agent connected the Postman AI agent to the to the to the API context graph and were able to API context graph and were able to API context graph and were able to validate all of this validate all of this validate all of this data using data using data using the same AI agent. But the same AI agent. But the same AI agent. But to make the comparison to make the comparison to make the comparison fair, we also fair, we also fair, we also tested it tested it tested it using a generic using a generic using a generic programming agent programming agent , in this case Claude , in this case Claude , in this case Claude Code, and tried Code, and tried Code, and tried to verify these to verify these to verify these use cases use cases use cases simply through a direct simply through a direct simply through a direct search of the code in GitHub. We search of the code in GitHub. We search of the code in GitHub. We also took the same also took the same also took the same context graph, context graph, context graph, made it made it made it available as a skill, available as a skill, available as a skill, and also compared and also compared and also compared it to a generic it to a generic programming agent, in programming agent, in this case Claude Code.

this case Claude Code. this case Claude Code. So, you'll see So, you'll see So, you'll see the results the results the results displayed on the displayed on the displayed on the graph like this. We graph like this. We graph like this. We rated each test rated each test rated each test on a scale of zero to on a scale of zero to on a scale of zero to five, and also five, and also five, and also measured the exact measured the exact measured the exact number of tokens number of tokens number of tokens spent on each spent on each spent on each iteration. So, if you iteration. So, if you iteration. So, if you see the result see the result see the result displayed in the displayed in the displayed in the top right top right top right corner, it means the corner, it means the corner, it means the agent did well. agent did well. agent did well. He was able to get a He was able to get a He was able to get a high score, but he high score, but he high score, but he spent a lot of spent a lot of spent a lot of tokens doing it. If tokens doing it. If tokens doing it. If the result is in the result is in the result is in the upper left the upper left the upper left corner, the agent not only corner, the agent not only corner, the agent not only received a high score, received a high score, received a high score, but also did it but also did it but also did it much more efficiently much more efficiently . If something falls . If something falls . If something falls outside the dotted outside the dotted outside the dotted line, it simply does not line, it simply does not line, it simply does not meet the level meet the level meet the level required for a required for a required for a good score on good score on good score on that test. So we've that test. So we've that test. So we've grouped together some of the grouped together some of the grouped together some of the typical typical typical use cases and use cases and use cases and scenarios we scenarios we scenarios we see in see in see in everyday everyday everyday development and defined what development and defined what they they they belong to. One belong to. One belong to. One key scenario key scenario key scenario we always we always we always see is see is see is API discovery. API discovery. Any developer Any developer Any developer making changes to the making changes to the making changes to the frontend or backend, frontend or backend, frontend or backend, integrating with integrating with integrating with other systems, other systems, other systems, asks the question: asks the question: asks the question: which API is better which API is better which API is better to call? In real to call? In real to call? In real systems, you systems, you systems, you probably have hundreds of probably have hundreds of probably have hundreds of different APIs, and most likely different APIs, and most likely different APIs, and most likely there are multiple APIs that there are multiple APIs that there are multiple APIs that do the same do the same do the same thing. So in this thing. So in this thing. So in this simplified example, simplified example, simplified example, I'll look at which API I'll look at which API I'll look at which API is best to call is best to call is best to call to get the to get the to get the profile name of a specific profile name of a specific profile name of a specific user. So, in user. So, in user. So, in this test we are this test we are this test we are dealing with pure dealing with pure dealing with pure search, and the results search, and the results search, and the results are as are as are as expected.

expected. The agent did very The agent did very well with the context graph compared to well with the context graph compared to well with the context graph compared to pure pure pure code search, where agents do code search, where agents do code search, where agents do not perform not perform not perform well. well. well. There is a slight There is a slight There is a slight difference between Postman difference between Postman difference between Postman and Claude, but it is and Claude, but it is and Claude, but it is not significant enough for not significant enough for not significant enough for us to us to us to delve into it in more delve into it in more delve into it in more detail. The second detail. The second detail. The second use case use case is where it gets is where it gets is where it gets really interesting. These are what really interesting. These are what really interesting. These are what we call we call API design and redesign scenarios. API design and redesign scenarios. In this scenario, In this scenario, In this scenario, imagine a developer imagine a developer imagine a developer creating a new creating a new creating a new feature, and to feature, and to feature, and to implement it, he implement it, he implement it, he designed a new API designed a new API designed a new API and input schema and input schema and input schema for the query. for the query. for the query. They defined They defined They defined the implementation of this API. the implementation of this API. the implementation of this API. They tested They tested They tested it on their end, it on their end, it on their end, then integrated it with the then integrated it with the then integrated it with the product, product, product, were were were satisfied with satisfied with satisfied with the result, and were ready the result, and were ready the result, and were ready for release. At the for release. At the for release. At the last minute, last minute, last minute, someone from the someone from the someone from the dependent team comes along and dependent team comes along and dependent team comes along and says, "Hey, it would be says, "Hey, it would be says, "Hey, it would be nice if I nice if I nice if I had one more had one more had one more extra extra extra property that I property that I property that I could pass to the API could pass to the API could pass to the API for for for future use." So future use." So future use." So the developer is like, " the developer is like, " Maybe we don't have time Maybe we don't have time Maybe we don't have time to evaluate all the different to evaluate all the different to evaluate all the different ways this ways this ways this can be done." What can be done." What can be done." What scheme would be scheme would be scheme would be correct for this? correct for this? correct for this? So they just So they just So they just add one add one add one additional context additional context —an arbitrary JSON —an arbitrary JSON object—and leave object—and leave object—and leave a note in the code: “ a note in the code: “ Come back to this Come back to this Come back to this and add the schema in 2 and add the schema in 2 and add the schema in 2 weeks.” The feature weeks.” The feature weeks.” The feature is released to production.

is released to production. is released to production. Everyone is satisfied. After Everyone is satisfied. After Everyone is satisfied. After 2 weeks, the developer 2 weeks, the developer 2 weeks, the developer returns and returns and returns and finds that that finds that that finds that that particular particular particular property that was supposed to property that was supposed to property that was supposed to pass one pass one pass one attribute now attribute now attribute now contains 15, and is being contains 15, and is being contains 15, and is being passed by not one passed by not one passed by not one service, but perhaps service, but perhaps service, but perhaps five. Maybe five. Maybe five. Maybe even ten even ten even ten services. Now services. Now services. Now the developer has no idea the developer has no idea the developer has no idea in what form in what form in what form the request is passed to the the request is passed to the the request is passed to the API and in what different API and in what different API and in what different ways each ways each ways each individual consumer individual consumer individual consumer sent data. So this is what sent data. So this is what sent data. So this is what I call I call I call analyzing the analyzing the analyzing the input structure for input structure for input structure for each API and the reasons each API and the reasons each API and the reasons why they are why they are why they are used the way they are used the way they are used the way they are . This can be . This can be . This can be difficult for many difficult for many difficult for many reasons. First, reasons. First, reasons. First, tracking every tracking every tracking every consumer of your API in consumer of your API in consumer of your API in real-world conditions real-world conditions real-world conditions can be impractical. can be impractical. can be impractical. Even if you have Even if you have Even if you have dependencies and dependencies and dependencies and service maps, figuring out the service maps, figuring out the service maps, figuring out the exact input structure exact input structure used by used by each consumer each consumer each consumer can be very can be very can be very difficult. Engineers difficult. Engineers difficult. Engineers often lack often lack often lack detailed information detailed information detailed information in production logs, and for in production logs, and for privacy reasons, it privacy reasons, it may sometimes may sometimes may sometimes be unavailable altogether. be unavailable altogether. be unavailable altogether. Even if Even if Even if the structures are defined, the structures are defined, the structures are defined, understanding why understanding why understanding why the consumer decided to the consumer decided to the consumer decided to transmit data in this way transmit data in this way transmit data in this way is a very, very is a very, very is a very, very difficult problem. But difficult problem. But difficult problem. But in this scenario, in this scenario, in this scenario, when we ran this when we ran this when we ran this evaluation using the evaluation using the evaluation using the Postman agent, we were Postman agent, we were Postman agent, we were surprised that the surprised that the surprised that the Postman agent Postman agent Postman agent handled it almost handled it almost handled it almost perfectly. And what perfectly. And what perfectly. And what helped in this helped in this helped in this case was, first, a case was, first, a case was, first, a context graph that context graph that context graph that knew about each knew about each knew about each API consumer, and also API consumer, and also API consumer, and also an understanding of where an understanding of where an understanding of where exactly each of those APIs exactly each of those APIs exactly each of those APIs was implemented in the was implemented in the was implemented in the code, and the context code, and the context code, and the context around that code, around that code, around that code, which helped the agent which helped the agent which helped the agent understand the reason for the understand the reason for the understand the reason for the changes. Very quickly, it

changes. Very quickly, it changes. Very quickly, it was able to calculate was able to calculate was able to calculate the exact unique the exact unique the exact unique structures for structures for structures for each consumer each consumer each consumer accessing accessing accessing this API, and you were able to this API, and you were able to this API, and you were able to make changes quite make changes quite make changes quite reliably and confidently. reliably and confidently. reliably and confidently. You can see You can see You can see that regular that regular that regular code search doesn't work very code search doesn't work very code search doesn't work very well, but with Claude and well, but with Claude and well, but with Claude and skills it skills it skills it does a bit does a bit does a bit better, although it often better, although it often better, although it often gives up after a gives up after a gives up after a certain number of certain number of certain number of steps, waiting for the steps, waiting for the user's confirmation on what to user's confirmation on what to do next. So do next. So do next. So another very common another very common use case that we're use case that we're starting starting starting to see is to see is to see is impact assessment. How do you impact assessment. How do you impact assessment. How do you determine what the determine what the determine what the impact of one impact of one impact of one particular change will be? particular change will be? particular change will be? So, if I change this So, if I change this So, if I change this API, what would be the API, what would be the API, what would be the potential potential potential use cases? Or use cases? Or use cases? Or maybe there is an maybe there is an maybe there is an existing API, version V2, and existing API, version V2, and existing API, version V2, and I want to implement the API I want to implement the API I want to implement the API version V3. How do I version V3. How do I version V3. How do I first understand first understand first understand how how how API V2 is used so I can API V2 is used so I can API V2 is used so I can design a better design a better design a better API V3? This is one of the API V3? This is one of the API V3? This is one of the scenarios where we scenarios where we scenarios where we see the see the see the context graph context graph context graph really show really show really show its benefits, and you its benefits, and you its benefits, and you see that Postman Agent not see that Postman Agent not see that Postman Agent not only shows only shows only shows great quality great quality great quality results, but it does so while results, but it does so while results, but it does so while consuming consuming consuming almost half or almost half or almost half or even three times less even three times less even three times less resources compared resources compared resources compared to regular to regular to regular code search. But not code search. But not code search. But not everything always goes smoothly. Sometimes everything always goes smoothly. Sometimes everything always goes smoothly. Sometimes we see crashes, and in we see crashes, and in we see crashes, and in this particular this particular this particular case, one of the case, one of the case, one of the tasks we tasks we tasks we set for him was to set for him was to set for him was to determine the impact of determine the impact of determine the impact of changes in a certain PR, where changes in a certain PR, where changes in a certain PR, where changes were made to changes were made to changes were made to an endpoint, and we needed to an endpoint, and we needed to an endpoint, and we needed to find all the find all the find all the downstream services associated with it downstream services associated with it downstream services associated with it , and we started , and we started , and we started seeing crashes, seeing crashes, seeing crashes, and the graph and the graph and the graph looked almost looked almost looked almost inverted. We inverted. We inverted. We see that Postman not see that Postman not see that Postman not only lowers the score, only lowers the score, only lowers the score, but also consumes a significant but also consumes a significant but also consumes a significant amount of tokens amount of tokens amount of tokens while performing while performing while performing this task. When this task. When this task. When we we we investigated and investigated and investigated and tried to understand tried to understand why

why why the errors were occurring, we found the errors were occurring, we found the errors were occurring, we found that new consumers had appeared that new consumers had appeared between the time the test was written between the time the test was written and the time it was and the time it was and the time it was run. So it is run. So it is run. So it is important not only to important not only to important not only to populate the populate the populate the context graph, but also context graph, but also context graph, but also to keep it up to to keep it up to to keep it up to date, date, date, as any as any as any delay in delay in delay in synchronizing the graph synchronizing the graph synchronizing the graph with real data with real data with real data creates a gap in creates a gap in creates a gap in results. And if results. And if results. And if there are gaps in the results there are gaps in the results there are gaps in the results , this immediately , this immediately , this immediately leads to a loss of leads to a loss of leads to a loss of trust on the part of trust on the part of trust on the part of the developer, which in the developer, which in the developer, which in turn leads to a turn leads to a turn leads to a lack of lack of lack of use. It use. It is important not only is important not only to identify new APIs and to identify new APIs and to identify new APIs and new dependencies, but also to new dependencies, but also to new dependencies, but also to remove APIs and remove APIs and remove APIs and dependencies, dependencies, dependencies, as this as this as this can also lead to can also lead to can also lead to confusion and problems. confusion and problems. confusion and problems. You might think You might think You might think these these these use cases might use cases might use cases might be atypical for you be atypical for you be atypical for you , but when , but when , but when we analyzed we analyzed we analyzed all the PRs created in the top 10 all the PRs created in the top 10 repositories, we repositories, we noticed that almost 75% of the PRs noticed that almost 75% of the PRs noticed that almost 75% of the PRs had some impact on the had some impact on the had some impact on the API. So, either these changes API. So, either these changes API. So, either these changes make adjustments to the make adjustments to the make adjustments to the external surface of the API, external surface of the API, external surface of the API, or they or they or they introduce a new API introduce a new API , and both of these options , and both of these options , and both of these options can have serious can have serious can have serious cascading consequences cascading consequences cascading consequences for subsequent systems. for subsequent systems.

These These use cases allowed use cases allowed use cases allowed us us us to clearly compare to clearly compare to clearly compare “before” and “after” with “before” and “after” with “before” and “after” with and without the context graph and without the context graph and without the context graph , and once we , and once we , and once we had this had this had this information, we were able to information, we were able to information, we were able to ask questions ask questions ask questions that were previously that were previously that were previously unavailable to us. These unavailable to us. These unavailable to us. These questions allowed questions allowed questions allowed us to ask the us to ask the us to ask the Postman agent: "Hey, Postman agent: "Hey, Postman agent: "Hey, do a full do a full do a full engineering analysis and engineering analysis and engineering analysis and end-to-end end-to-end end-to-end architectural review of architectural review of architectural review of our entire our entire our entire ecosystem." It was ecosystem." It was ecosystem." It was then that we had the then that we had the then that we had the same "enlightenment." When same "enlightenment." When same "enlightenment." When we asked Agent we asked Agent we asked Agent Postman to do this, he Postman to do this, he Postman to do this, he prepared a 21- prepared a 21- page page page engineering report, engineering report, engineering report, detailing detailing detailing every weak link and every weak link and every weak link and vulnerability. He vulnerability. He vulnerability. He discovered interesting discovered interesting discovered interesting insights, including insights, including insights, including dependency loops dependency loops dependency loops that can be very that can be very that can be very problematic during problematic during problematic during incidents; incidents; incidents; risky long-term risky long-term risky long-term migrations, indicating migrations, indicating migrations, indicating technical debt; technical debt; technical debt; single points of failure single points of failure single points of failure that could become a that could become a that could become a serious problem serious problem serious problem or hidden or hidden or hidden threats; “blind threats; “blind threats; “blind spots” in telemetry, spots” in telemetry, spots” in telemetry, where implementation does not where implementation does not where implementation does not match match match monitoring data; and also monitoring data; and also monitoring data; and also provided clear provided clear provided clear recommendations for CEOs, recommendations for CEOs, recommendations for CEOs, CTOs, and lead CTOs, and lead CTOs, and lead engineers on engineers on engineers on next steps. In next steps. In next steps. In addition to all addition to all addition to all this, when considering this, when considering this, when considering opportunities for opportunities for opportunities for further improvements, further improvements, further improvements, one of the key one of the key one of the key areas is areas is areas is to implement a to implement a to implement a semantic layer semantic layer semantic layer that describes that describes that describes business activities and business activities and business activities and connect it to the connect it to the connect it to the context layer that context layer that context layer that your your your AI agents use in AI agents use in AI agents use in workflows. Another workflows. Another workflows. Another level of improvement level of improvement level of improvement we are exploring is we are exploring is we are exploring is improving our understanding of improving our understanding of improving our understanding of why a why a why a particular data model exists.

particular data model exists. particular data model exists. Why does the domain Why does the domain Why does the domain structure look like this structure look like this structure look like this ? Why does ? Why does ? Why does this API exist and not others? Why this API exist and not others? Why this API exist and not others? Why should you should you should you use one use one use one API instead of another? All API instead of another? All API instead of another? All this helps to this helps to this helps to make further make further programming decisions and programming decisions and translates into a translates into a translates into a truly effective truly effective truly effective result in your result in your result in your daily work. daily work. daily work. So, the main So, the main So, the main conclusions. We conclusions. We conclusions. We realized that the realized that the realized that the API context graph serves as a API context graph serves as a API context graph serves as a context layer for context layer for programming agents in programming agents in real real real distributed distributed distributed systems. It provides a systems. It provides a systems. It provides a high-level overview high-level overview high-level overview that helps that helps that helps answer answer engineering engineering architecture questions that could architecture questions that could architecture questions that could not be not be not be answered within a answered within a answered within a single repository single repository single repository or microservice. This or microservice. This or microservice. This reduces the time it takes to find the reduces the time it takes to find the reduces the time it takes to find the right approach right approach right approach to solving a to solving a to solving a specific problem. specific problem. And all this means that And all this means that you can do it either with you can do it either with you can do it either with proper quality, or proper quality, or proper quality, or much more efficiently much more efficiently . So efficiency . So efficiency . So efficiency means either saving means either saving means either saving you money, but we you money, but we you money, but we see it this way: if we see it this way: if we see it this way: if we can do a can do a can do a particular task particular task particular task much more efficiently much more efficiently , it means we , it means we , it means we can do it can do it can do it more often, and that more often, and that more often, and that allows us allows us allows us to do the same to do the same to do the same assessments much more assessments much more assessments much more often, particularly at the often, particularly at the often, particularly at the PR level or even PR level or even PR level or even directly on the directly on the directly on the workstation or in the workstation or in the workstation or in the development environment development environment development environment where the engineer works.

where the engineer works. where the engineer works. So, the earlier we So, the earlier we So, the earlier we can make can make can make decisions and identify decisions and identify decisions and identify issues early in the issues early in the lifecycle, the more we lifecycle, the more we believe believe believe code generation agents code generation agents code generation agents will provide will provide will provide developers with peace of mind developers with peace of mind developers with peace of mind and help and help and help them more confidently them more confidently them more confidently push changes push changes push changes into production in into production in into production in the future. So, that's the future. So, that's the future. So, that's all from my side. All of all from my side. All of all from my side. All of these features these features these features are available within the are available within the are available within the Postman platform. They are Postman platform. They are Postman platform. They are also available for also available for also available for use as use as use as early access. I will be early access. I will be early access. I will be outside immediately after outside immediately after outside immediately after the performance if you have any the performance if you have any the performance if you have any questions questions questions or suggestions. We or suggestions. We or suggestions. We also have stand S32 also have stand S32 also have stand S32 right across the hall. right across the hall. right across the hall. That's all from my side. That's all from my side. That's all from my side. Thank you very much. Thank you very much.