Stop Renting Intelligence: The Train-to-Deploy Loop for Specialized AI — Fireworks AI

Hello everyone. Before Hello everyone. Before we start, I have a we start, I have a we start, I have a quick question for quick question for quick question for all of you. How many of all of you. How many of all of you. How many of you you you are using a are using a are using a closed- closed- closed- source model like ChatGPT source model like ChatGPT source model like ChatGPT or Anthropic today? Good. or Anthropic today? Good. or Anthropic today? Good. How many of you How many of you How many of you feel that it is feel that it is feel that it is quickly becoming very quickly becoming very quickly becoming very expensive? Good. That's a expensive? Good. That's a expensive? Good. That's a good percentage of good percentage of good percentage of you. And just as all of you. And just as all of you. And just as all of you may be thinking you may be thinking you may be thinking about moving from a about moving from a about moving from a closed closed closed source model to source model to source model to open source models for open source models for open source models for cost cost cost and performance reasons, there are and performance reasons, there are and performance reasons, there are other customers and other customers and other customers and users users users going through the same going through the same going through the same journey. And today I'm journey. And today I'm journey. And today I'm going to talk about where these going to talk about where these going to talk about where these open open open source models shine, and how to source models shine, and how to source models shine, and how to make make make the performance or the performance or the performance or quality of these quality of these quality of these open source models open source models open source models match the match the match the closed closed closed source models themselves. And before I source models themselves. And before I source models themselves. And before I get to that, let me get to that, let me get to that, let me briefly introduce myself briefly introduce myself . I work at Firework. I . I work at Firework. I . I work at Firework. I manage part of the manage part of the manage part of the applied applied applied machine learning team. machine learning team. machine learning team. My name is Jatishri, My name is Jatishri, My name is Jatishri, but you can call but you can call but you can call me Jat if that's me Jat if that's me Jat if that's easier. Let's easier. Let's easier. Let's talk about the entire talk about the entire talk about the entire deployment chain deployment chain that that that closed closed closed source models go through today as source models go through today as source models go through today as users users users move from move from move from closed closed closed source models to source models to source models to open source models. But open source models. But open source models. But before we before we before we delve into that, delve into that, delve into that, let's talk let's talk let's talk about some about some about some fundamental fundamental fundamental truths that are truths that are truths that are relevant in the relevant in the relevant in the AI ​​ecosystem today AI ​​ecosystem today AI ​​ecosystem today . So, three . So, three . So, three fundamental fundamental fundamental truths that are truths that are truths that are true: first, true: first, true: first, open open open source models are mostly source models are mostly production-ready. If production-ready. If any of you any of you any of you have used GLM 5.5.2, have used GLM 5.5.2, have used GLM 5.5.2, you know that this is you know that this is you know that this is especially true

especially true especially true for for for code use cases. code use cases. Second, if Second, if Second, if open source models don't open source models don't open source models don't tend to tend to tend to outperform or outperform or outperform or match the quality of match the quality of match the quality of closed closed closed source models, then the differentiating source models, then the differentiating source models, then the differentiating factor is how you factor is how you factor is how you tune, tune, tune, train, and train, and train, and refine those refine those refine those open open open source models to source models to source models to match match match the performance and the performance and the performance and quality of quality of quality of closed source models. And closed source models. And third, the AI ​​stack itself for third, the AI ​​stack itself for third, the AI ​​stack itself for most of our most of our most of our customers and customers and customers and users is users is users is moving from " moving from " let's just let's just let's just service the model service the model service the model once and it's once and it's once and it's done," to a more done," to a more done," to a more continuous loop continuous loop continuous loop that improves that improves that improves from training to from training to from training to outputting the same outputting the same open source model. So , this is where Fireworks comes in , this is where Fireworks comes in , this is where Fireworks comes in . It greatly . It greatly . It greatly simplifies the entire process of simplifies the entire process of training these training these open source models, as open source models, as open source models, as well as deploying them at well as deploying them at well as deploying them at scale, and scale, and scale, and also makes it also makes it also makes it much more much more cost-effective. And I'll cost-effective. And I'll talk about how we talk about how we talk about how we actually deliver actually deliver actually deliver training and the different training and the different training and the different types of training that we types of training that we types of training that we offer towards the offer towards the offer towards the end of the presentation, end of the presentation, end of the presentation, but I also want to but I also want to but I also want to talk briefly talk briefly talk briefly about the about the about the conclusion before we conclusion before we conclusion before we get to the get to the get to the training part, training part, training part, which has become very which has become very which has become very popular with popular with popular with our customers and our customers and our customers and users. And users. And users. And before we also before we also before we also talk about talk about talk about inference and inference and inference and learning, I just learning, I just learning, I just wanted to give you a wanted to give you a wanted to give you a quick overview of what quick overview of what inference looks like, especially at inference looks like, especially at scale, and scale, and scale, and what kind of impact on what kind of impact on what kind of impact on production we're production we're production we're seeing among seeing among seeing among some of our some of our some of our customers and the customers and the customers and the user base itself.

user base itself. user base itself. So, today, at So, today, at So, today, at scale, we are scale, we are scale, we are servicing about servicing about servicing about 30 trillion tokens, 30 trillion tokens, 30 trillion tokens, and that number is growing. This and that number is growing. This and that number is growing. This number was actually number was actually number was actually before the release of GLM 5.2. I'm before the release of GLM 5.2. I'm before the release of GLM 5.2. I'm sure with sure with sure with the release of GLM 5.2 it the release of GLM 5.2 it the release of GLM 5.2 it has grown by at least has grown by at least has grown by at least 5 trillion tokens 5 trillion tokens 5 trillion tokens per day. We per day. We per day. We support over support over support over 10,000 companies 10,000 companies 10,000 companies worldwide and process worldwide and process worldwide and process about 200,000 about 200,000 about 200,000 requests every second. And the requests every second. And the requests every second. And the impact on production impact on production impact on production that we see in that we see in that we see in our customer and our customer and our customer and user base user base user base is very significant. So, is very significant. So, is very significant. So, Cursor, for those of you Cursor, for those of you Cursor, for those of you who are already using who are already using who are already using Cursor and Cursor models, Cursor and Cursor models, Cursor and Cursor models, today the Composer 2 model for Cursor was today the Composer 2 model for Cursor was today the Composer 2 model for Cursor was trained and deployed trained and deployed trained and deployed on our platform on our platform . And, um, it's . And, um, it's able to achieve 13 able to achieve 13 times faster times faster times faster output because of the output because of the output because of the Fire Attention mechanism Fire Attention mechanism Fire Attention mechanism that we have, and I'll that we have, and I'll that we have, and I'll talk a little bit about that talk a little bit about that talk a little bit about that on the next slide. on the next slide. Uh, Vercel is also a Uh, Vercel is also a big client of ours. We big client of ours. We big client of ours. We helped them helped them helped them set up and set up and set up and deploy the model on deploy the model on deploy the model on our platform, and our platform, and our platform, and they saw a they saw a 40x improvement in 40x improvement in 40x improvement in latency. Another latency. Another latency. Another important important important use case is GenSpark.

use case is GenSpark. use case is GenSpark. GenSpark Lava builds GenSpark Lava builds GenSpark Lava builds are in a very are in a very are in a very similar industry where similar industry where similar industry where they allow you they allow you they allow you to develop systems to develop systems to develop systems with just hints and with just hints and with just hints and quality is a big quality is a big quality is a big bottleneck or bottleneck or bottleneck or was a big was a big was a big bottleneck for them when bottleneck for them when bottleneck for them when they were trying to they were trying to they were trying to move from move from move from closed source to closed source to closed source to open open open source models and we helped source models and we helped source models and we helped them adjust. We them adjust. We them adjust. We did an RFT for them. We did an RFT for them. We did an RFT for them. We helped them helped them helped them clean the dataset, clean the dataset, clean the dataset, define the define the define the estimator functions, and at the end of the estimator functions, and at the end of the estimator functions, and at the end of the RFT training itself, we RFT training itself, we RFT training itself, we saw that the quality saw that the quality saw that the quality for the for the for the GenSpark use case GenSpark use case GenSpark use case improved by 12%. improved by 12%. We are also a trusted We are also a trusted platform among platform among platform among all our customers all our customers all our customers and users, and and users, and and users, and some time ago we some time ago we some time ago we raised about 327 raised about 327 raised about 327 million CDC, and Fireworks million CDC, and Fireworks million CDC, and Fireworks employs about 200 employs about 200 employs about 200 people. This brings people. This brings people. This brings me to the next me to the next me to the next part. I know on the part. I know on the part. I know on the previous slide previous slide previous slide I mentioned that we I mentioned that we I mentioned that we made the latency made the latency made the latency faster for a few of faster for a few of faster for a few of our customers like our customers like our customers like Vercel. And how we Vercel. And how we Vercel. And how we do it is quite simple do it is quite simple do it is quite simple at a high level. In at a high level. In at a high level. In Fireworks, we have our own Fireworks, we have our own Fireworks, we have our own inference engine inference engine inference engine called called called Fire Attention. So you have a Fire Attention. So you have a Fire Attention. So you have a model, you have model, you have model, you have GPUs, and GPUs, and GPUs, and the model sits the model sits the model sits on top of the on top of the on top of the GPUs through a GPUs through a GPUs through a translation layer translation layer translation layer called called called CUDA cores, and as CUDA cores, and as CUDA cores, and as part of Fire Attention we part of Fire Attention we part of Fire Attention we built our own built our own built our own CUDA core. So when a customer CUDA core. So when a customer CUDA core. So when a customer comes to us with a comes to us with a workload and a performance target, we performance target, we can optimize can optimize can optimize that particular that particular that particular inference engine inference engine inference engine and the entire and the entire and the entire AI stack, AI stack, AI stack, including that CUDA core, including that CUDA core, including that CUDA core, for that for that for that particular particular particular use case. So

use case. So use case. So most customers, most customers, most customers, like We're Cell on the like We're Cell on the like We're Cell on the previous slide, previous slide, previous slide, come to us and come to us and come to us and say they have X say they have X say they have X input tokens, Y input tokens, Y input tokens, Y output tokens, Z output tokens, Z query cache hit ratio, and query cache hit ratio, and they want to hit they want to hit they want to hit certain certain certain latency targets. latency targets. latency targets. And we help them And we help them And we help them optimize that entire optimize that entire optimize that entire inference stack inference stack inference stack , including the , including the , including the CUDA core, so that we can CUDA core, so that we can CUDA core, so that we can serve that serve that serve that model at a lower model at a lower model at a lower cost, but also at a cost, but also at a cost, but also at a price that meets price that meets price that meets their their their performance target and performance target and performance target and their specific their specific workload. As workload. As part of part of part of attention optimization and logical attention optimization and logical attention optimization and logical inference, we also inference, we also inference, we also implement very implement very implement very well-known methods in the well-known methods in the well-known methods in the field of artificial field of artificial field of artificial intelligence, such as intelligence, such as intelligence, such as speculative speculative speculative decoding, decoding, decoding, disaggregation, disaggregation, disaggregation, quantization. As quantization. As quantization. As for for for speculative speculative speculative decoding itself, we also decoding itself, we also decoding itself, we also allow customers allow customers allow customers to train their to train their to train their draft models on draft models on draft models on our platform. So our platform. So our platform. So they can even they can even they can even load a load a load a dataset very similar dataset very similar dataset very similar to how they to how they to how they do for fine- do for fine- do for fine- tuning to tuning to tuning to train the draft train the draft train the draft model, which helps model, which helps model, which helps them make the main them make the main them make the main model much model much model much faster, meet faster, meet faster, meet latency requirements, and latency requirements, and latency requirements, and also provides also provides also provides better better better throughput if the throughput if the throughput if the draft model is very draft model is very draft model is very well trained. And so once well trained. And so once well trained. And so once people people people reach this reach this reach this stage, it's mainly stage, it's mainly when they're when they're when they're satisfied with the quality of the satisfied with the quality of the satisfied with the quality of the model. But, as model. But, as model. But, as customers move customers move customers move from from from closed-source to closed-source to closed-source to open- open- open- source models, their source models, their source models, their biggest biggest biggest bottleneck or gap is bottleneck or gap is bottleneck or gap is quality. And we quality. And we quality. And we are trying to solve are trying to solve are trying to solve this problem with this problem with this problem with our our our educational product educational product . Our training

. Our training . Our training product itself is available in product itself is available in product itself is available in three different three different three different versions for different versions for different versions for different segments of our segments of our segments of our client and client and client and user base. user base. user base. One of them is a One of them is a learning agent. It is learning agent. It is learning agent. It is very popular very popular very popular among among among product managers and product managers and product managers and developers who have developers who have developers who have recently entered recently entered recently entered this field of artificial this field of artificial this field of artificial inference and inference and inference and learning. So, learning. So, learning. So, the agent can the agent can the agent can be accessed be accessed be accessed through our UI through our UI through our UI , and you , and you , and you just need to just need to just need to connect to it the connect to it the connect to it the dataset that you dataset that you dataset that you uploaded to the uploaded to the uploaded to the Fireworks platform, Fireworks platform, Fireworks platform, as well as a description of what you as well as a description of what you as well as a description of what you are trying to do are trying to do are trying to do within this within this within this training run itself. training run itself. training run itself. It will even let It will even let It will even let you choose the model. you choose the model. you choose the model. It will clean the It will clean the It will clean the dataset for you. It will dataset for you. It will dataset for you. It will also help you also help you also help you perform fine- perform fine- perform fine- tuning of the tuning of the tuning of the hyperparameter, hyperparameter, hyperparameter, as it is not always as it is not always as it is not always easy to get the easy to get the easy to get the right right right hyperparameter when hyperparameter when hyperparameter when you are you are you are training for your training for your training for your specific specific specific workload. The workload. The next option, next option, also one of the also one of the also one of the most popular most popular most popular among our clients among our clients among our clients and users, is and users, is and users, is our guided our guided our guided learning. And it is this that learning. And it is this that most of our most of our clients use today. They can clients use today. They can clients use today. They can use our use our use our managed managed managed learning platform once learning platform once learning platform once through the API and a second time through the API and a second time through the API and a second time through the user interface itself through the user interface itself through the user interface itself . It's as . It's as . It's as simple as simple as simple as entering entering entering the hyperparameters the hyperparameters the hyperparameters you think will be you think will be you think will be useful for your useful for your useful for your model, selecting the model, selecting the model, selecting the model, loading the model, loading the model, loading the dataset itself, and dataset itself, and dataset itself, and then clicking a then clicking a then clicking a button. That's how button. That's how button. That's how simple we make it in simple we make it in simple we make it in Fireworks for self- Fireworks for self- Fireworks for self- guided learning.

guided learning. guided learning. And I know that one of the And I know that one of the And I know that one of the problems that problems that problems that many many many people face is people face is people face is observability observability observability after the training itself begins after the training itself begins after the training itself begins . So we . So we . So we also have a nice UI dashboard also have a nice UI dashboard that that shows how shows how the training is going, how the training is going, how well your model well your model well your model is learning at each is learning at each is learning at each epoch, what the epoch, what the epoch, what the loss function looks like. loss function looks like. loss function looks like. Additionally, you can also Additionally, you can also Additionally, you can also connect your connect your connect your Weights & Biases API key to the Weights & Biases API key to the supervised learning service itself supervised learning service itself and export and export and export some some some observability metrics to observability metrics to observability metrics to platforms like platforms like platforms like Weights&Biases if you are already Weights&Biases if you are already Weights&Biases if you are already using it. And using it. And using it. And the third, the third, the third, most advanced most advanced most advanced type of training type of training type of training that we offer that we offer is the training API itself. And the is the training API itself. And the is the training API itself. And the learning API is very learning API is very learning API is very popular among popular among popular among some some some machine learning engineers in machine learning engineers in machine learning engineers in different organizations different organizations different organizations or in our or in our or in our client base. And in client base. And in client base. And in this particular this particular this particular product offering, product offering, product offering, we allow we allow we allow machine learning engineers machine learning engineers machine learning engineers to define their own to define their own to define their own training loops and training loops and training loops and loss functions. And this loss functions. And this loss functions. And this becomes important when becomes important when becomes important when you want to you want to you want to customize your customize your customize your learning task learning task learning task to suit your to suit your to suit your needs. This goes needs. This goes needs. This goes far beyond the far beyond the far beyond the managed managed managed learning service we learning service we learning service we offer. And these offer. And these offer. And these different kinds of different kinds of different kinds of products in the products in the products in the learning ecosystem learning ecosystem learning ecosystem that we offer that we offer that we offer are offered for are offered for are offered for controlled controlled controlled fine-tuning fine-tuning , reinforcement fine- , reinforcement fine- , reinforcement fine- tuning, tuning, tuning, and and and even DPO. Today, I even DPO. Today, I even DPO. Today, I think RFT has become think RFT has become think RFT has become extremely extremely extremely popular. And if

popular. And if popular. And if you ever you ever you ever need help need help need help defining a good defining a good defining a good estimator function, we've estimator function, we've estimator function, we've worked with worked with worked with clients like clients like clients like GenSpark, which I GenSpark, which I GenSpark, which I told you about on the told you about on the told you about on the previous slide, previous slide, previous slide, to help them to help them to help them define the define the define the estimator function, clean the estimator function, clean the estimator function, clean the dataset, and also dataset, and also dataset, and also choose the right choose the right choose the right hyperparameters, hyperparameters, hyperparameters, especially for RFT. especially for RFT. especially for RFT. Unfortunately, the training Unfortunately, the training Unfortunately, the training agent itself does not support RFT, agent itself does not support RFT, agent itself does not support RFT, so it will not help so it will not help so it will not help you clean the you clean the you clean the dataset or even dataset or even dataset or even perform perform hyperparameter tuning, but hyperparameter tuning, but we would be happy we would be happy we would be happy to help you to help you to help you do that if you do that if you do that if you have a large have a large have a large use case. So, use case. So, use case. So, now that we know now that we know now that we know all about the different all about the different all about the different learning offerings learning offerings we offer, I we offer, I we offer, I would also like to would also like to would also like to show you a quick show you a quick show you a quick demo of each demo of each demo of each of them: the learning of them: the learning of them: the learning agent, the guided agent, the guided agent, the guided learning, and the learning, and the learning, and the learning API itself. But learning API itself. But learning API itself. But before we get before we get before we get to that, I just to that, I just to that, I just wanted to finish wanted to finish wanted to finish by telling you what the by telling you what the data flywheel looks like on data flywheel looks like on our platform today. I our platform today. I our platform today. I know that on the first know that on the first know that on the first slide I talked about slide I talked about slide I talked about the realities of modern the realities of modern the realities of modern AI, AI, AI, where most people where most people where most people prefer a prefer a prefer a continuous loop continuous loop continuous loop where they learn, where they learn, where they learn, deploy, train, deploy, train, deploy, train, deploy, deploy, deploy, getting more and more getting more and more getting more and more data and feedback from data and feedback from data and feedback from their users. And that's what their users. And that's what their users. And that's what we're we're we're trying to make trying to make trying to make more seamless more seamless more seamless on our platform, and that's why on our platform, and that's why on our platform, and that's why we have we have we have different training different training different training products, different products, different products, different offerings, and also making it offerings, and also making it offerings, and also making it easier for you easier for you easier for you to deploy, and also to deploy, and also to deploy, and also making it cheaper and making it cheaper and making it cheaper and faster for you to faster for you to faster for you to deploy these deploy these deploy these models at models at models at scale. And the way scale. And the way scale. And the way this this this flywheel works is very flywheel works is very flywheel works is very simple. Once simple. Once simple. Once these models are trained, these models are trained, these models are trained, I'll show you I'll show you I'll show you a demonstration on the a demonstration on the a demonstration on the next slide where the next slide where the next slide where the models are trained. You are

models are trained. You are models are trained. You are just one click away from just one click away from just one click away from deploying these deploying these deploying these models in a production models in a production models in a production environment. And after environment. And after environment. And after deployment, you deployment, you deployment, you can control it. can control it. can control it. We have metrics We have metrics We have metrics monitoring the monitoring the monitoring the number of tokens, the number of tokens, the cache hit ratio, cache hit ratio, how your how your how your speculator is performing if you speculator is performing if you speculator is performing if you have trained a speculator. have trained a speculator. We help you We help you collect data, gather collect data, gather collect data, gather feedback from all the feedback from all the metrics you have collected, and if you metrics you have collected, and if you choose, also choose, also choose, also retrain on the retrain on the retrain on the same platform, from same platform, from same platform, from the end of the the end of the the end of the checkpoint you were checkpoint you were checkpoint you were using to the using to the using to the newest newest newest checkpoint. checkpoint. So, some of the platform capabilities that platform capabilities that make this data make this data make this data possible are the possible are the possible are the presence of multiple presence of multiple presence of multiple GPUs GPUs . We partner with . We partner with . We partner with various cloud and various cloud and various cloud and non-cloud non-cloud non-cloud providers, making it providers, making it providers, making it easier for us to provide easier for us to provide easier for us to provide compute as well as compute as well as compute as well as deploy these deploy these deploy these models at models at models at scale. Another scale. Another scale. Another thing we thing we thing we offer is offer is offer is hot loading, hot loading, hot loading, and this has become very and this has become very and this has become very popular, especially popular, especially popular, especially when we were training the when we were training the when we were training the model for Cursor. And this model for Cursor. And this model for Cursor. And this was actually a was actually a was actually a feature that we feature that we feature that we implemented implemented implemented specifically for Cursor, and specifically for Cursor, and specifically for Cursor, and then we thought then we thought then we thought it would be useful it would be useful it would be useful for the general public for the general public for the general public as long as they were also as long as they were also as long as they were also training the model.

training the model. training the model. Hot booting Hot booting Hot booting essentially means that essentially means that essentially means that once you have a new once you have a new once you have a new checkpoint, you checkpoint, you checkpoint, you can can can go directly to the actual go directly to the actual go directly to the actual deployment without deployment without any downtime. So, any downtime. So, any downtime. So, if you have a if you have a if you have a deployment and you've deployment and you've deployment and you've gone through this gone through this gone through this continuous loop and continuous loop and continuous loop and trained a new trained a new trained a new endpoint, you endpoint, you endpoint, you can just can just can just click a button and click a button and click a button and replace the weights you replace the weights you replace the weights you used used used before with the new before with the new before with the new endpoints. endpoints. endpoints. Another thing that simplifies Another thing that simplifies Another thing that simplifies learning on our learning on our learning on our platform is numerical platform is numerical platform is numerical accuracy. So often accuracy. So often accuracy. So often customers customers customers use use use platform A for platform A for platform A for training and training and training and deploy it to deploy it to deploy it to platform B. But when platform B. But when platform B. But when you do that, you will have you do that, you will have you do that, you will have problems with problems with problems with model quality. And the model quality. And the model quality. And the reason for these problems is reason for these problems is reason for these problems is that the that the that the numerical metrics are not numerical metrics are not numerical metrics are not always compatible when always compatible when always compatible when you learn on you learn on you learn on one platform and one platform and one platform and host it on host it on host it on another platform. That is another platform. That is another platform. That is why we offer why we offer why we offer training and training and training and inference on a single inference on a single inference on a single platform and platform and platform and guarantee that you will not guarantee that you will not guarantee that you will not find any find any find any numerical numerical numerical differences after differences after differences after training is complete training is complete training is complete and after you and after you and after you decide to deploy decide to deploy decide to deploy that particular model that particular model that particular model on the platform itself. And on the platform itself. And on the platform itself. And another thing that another thing that another thing that makes it more makes it more makes it more seamless is seamless is seamless is that the training and that the training and that the training and inference inference inference happen on the happen on the happen on the same platform, you are same platform, you are same platform, you are just one click away from just one click away from just one click away from deploying this deploying this deploying this model at production model at production model at production scale. Now that scale. Now that scale. Now that we've learned all about we've learned all about we've learned all about inference and learning, inference and learning, inference and learning, I just wanted to I just wanted to I just wanted to show you a quick show you a quick show you a quick demo of the different demo of the different demo of the different learning products learning products learning products and how they and how they and how they work on our work on our work on our platform. The first platform. The first platform. The first part of the demo part of the demo part of the demo shows how you can shows how you can shows how you can use the use the use the learning agent itself. Here learning agent itself. Here learning agent itself. Here you can enter where you can enter where you can enter where your

your your training training training dataset is located in Fireworks, what it's dataset is located in Fireworks, what it's dataset is located in Fireworks, what it's called. You called. You called. You can also provide it with the can also provide it with the can also provide it with the eval dataset itself. I eval dataset itself. I eval dataset itself. I know people have know people have know people have separate datasets separate datasets separate datasets for training and for training and for training and evaluation or evaluation or evaluation or testing. So, you testing. So, you testing. So, you provide provide provide both sets of data to the agent. both sets of data to the agent. both sets of data to the agent. And then you ask And then you ask And then you ask it to fine-tune it to fine-tune it to fine-tune and and and even choose even choose even choose the hyperparameters and the hyperparameters and the hyperparameters and model for you. model for you. model for you. The agent then moves on to the The agent then moves on to the The agent then moves on to the next step and next step and next step and tells you the tells you the tells you the steps it will steps it will steps it will take to take to take to complete the task complete the task complete the task you gave it. So you gave it. So , he describes the , he describes the , he describes the SFT plan in detail. He explains SFT plan in detail. He explains SFT plan in detail. He explains why he chose the basic why he chose the basic why he chose the basic model. In this model. In this model. In this case, he chose quant case, he chose quant case, he chose quant 38B. And will also give 38B. And will also give 38B. And will also give you some information you some information you some information about hyperparameters about hyperparameters about hyperparameters and what the and what the and what the whole whole whole learning process might look like at a learning process might look like at a learning process might look like at a high level. And once high level. And once high level. And once you're done, you're done, you're done, uh, once you're, uh, uh, once you're, uh, uh, once you're, uh, happy with happy with happy with the result that the the result that the the result that the agent has provided, you agent has provided, you agent has provided, you can approve whatever the can approve whatever the can approve whatever the agent has outlined. And agent has outlined. And agent has outlined. And once you once you once you approve it, it will approve it, it will approve it, it will actually go through the actually go through the actually go through the different steps and different steps and different steps and let you know let you know how it completes the how it completes the how it completes the second step, what will it second step, what will it second step, what will it look like? If he look like? If he look like? If he successfully completes successfully completes successfully completes step three, what will it step three, what will it step three, what will it look like now? And look like now? And look like now? And at the end of the entire at the end of the entire at the end of the entire training run, training run, training run, it will also provide you with a it will also provide you with a it will also provide you with a summary of summary of summary of the configuration, the the configuration, the the configuration, the base model, and the base model, and the various various hyperparameters and hyperparameters and hyperparameters and how they are how they are how they are affected. And once affected. And once affected. And once all the model training

all the model training all the model training is complete, it will is complete, it will is complete, it will actually give you actually give you actually give you metrics like metrics like metrics like accuracy and F1 score. Uh, accuracy and F1 score. Uh, accuracy and F1 score. Uh, and it also and it also and it also summarizes, summarizes, summarizes, sort of re- sort of re- sort of re- summarizes and gives you an summarizes and gives you an summarizes and gives you an overview of everything that, uh, overview of everything that, uh, overview of everything that, uh, was done. And one was done. And one was done. And one cool feature cool feature cool feature is that it is that it is that it also gives you also gives you also gives you recommendations for the recommendations for the recommendations for the next step as next step as next step as part of the part of the part of the training run itself. training run itself. training run itself. Now, the next Now, the next Now, the next part of this part of this part of this demo is demo is demo is actually actually actually about how we can about how we can about how we can use our use our use our guided learning guided learning guided learning platform. I mentioned platform. I mentioned that you just that you just that you just need to select the need to select the need to select the model dataset and model dataset and model dataset and hyperparameters, and hyperparameters, and hyperparameters, and then you then you then you can start can start can start training the model with just one click. training the model with just one click. Let's say you've Let's say you've loaded a loaded a loaded a dataset, selected a model dataset, selected a model , and all you , and all you , and all you need to do is need to do is need to do is go to the go to the estimator settings, select the estimator dataset, and also estimator dataset, and also select a select a select a hyperparameter as hyperparameter as hyperparameter as part of the advanced part of the advanced part of the advanced settings if you settings if you settings if you decide to have more decide to have more decide to have more detailed detailed detailed hyperparameters for the hyperparameters for the hyperparameters for the training training training run itself. And once run itself. And once run itself. And once you are happy with you are happy with you are happy with all the configurations all the configurations , including the model and the , including the model and the , including the model and the dataset itself, you dataset itself, you dataset itself, you just need to just need to just need to create, you just create, you just create, you just need to click the need to click the need to click the "create" button. And "create" button. And "create" button. And as soon as you as soon as you as soon as you click the " click the " create" button, the create" button, the learning task will begin. And learning task will begin. And you can see that the you can see that the user interface looks user interface looks similar when the similar when the similar when the learning task is learning task is learning task is actually completed. And actually completed. And actually completed. And if you click on if you click on if you click on that specific that specific that specific training task, training task, training task, it will give you it will give you it will give you information about the information about the information about the dataset used

dataset used dataset used , the , the , the hyperparameters used, what the hyperparameters used, what the model loss itself looked like when model loss itself looked like when training the model. And training the model. And training the model. And then you then you then you can even clone these can even clone these can even clone these training runs and training runs and training runs and replay them with replay them with replay them with different different different settings and settings and settings and configurations. And configurations. And configurations. And once you are once you are once you are happy with the fine- happy with the fine- happy with the fine- tuned model tuned model , you can go to the , you can go to the , you can go to the deployment section, deployment section, deployment section, select the select the select the custom model custom model that you just that you just that you just trained on our trained on our trained on our platform, and then platform, and then platform, and then click click Deploy when you are Deploy when you are Deploy when you are satisfied with the satisfied with the deployment configurations. Or you deployment configurations. Or you can choose from a set of can choose from a set of deployment configurations that deployment configurations that we have, such as we have, such as we have, such as bandwidth, bandwidth, bandwidth, fast, minimal. And fast, minimal. And fast, minimal. And it's kind of a pattern it's kind of a pattern it's kind of a pattern that works well for that works well for that works well for common common common workloads that we've workloads that we've workloads that we've optimized based on optimized based on throughput, latency, throughput, latency, time to first time to first time to first token, and other token, and other token, and other metrics like that. metrics like that. And unfortunately, I And unfortunately, I don't have a full don't have a full don't have a full demo of our demo of our demo of our training API, training API, training API, as the setup itself as the setup itself differs from differs from client to client. client to client.

client to client. But we have many But we have many But we have many guides on our guides on our guides on our websites that you websites that you websites that you can view can view can view and follow and follow , especially if you are , especially if you are , especially if you are interested in interested in interested in setting up your own setting up your own setting up your own learning loops and learning loops and learning loops and loss functions yourself. loss functions yourself. loss functions yourself. And that's all I have for And that's all I have for And that's all I have for you today. I you today. I you today. I just wanted to give you a just wanted to give you a just wanted to give you a quick overview of what the quick overview of what the quick overview of what the entire entire entire learning cycle for learning cycle for learning cycle for inference looks like. Thank you very much inference looks like. Thank you very much . And if you have any . And if you have any . And if you have any questions, I'd be questions, I'd be questions, I'd be happy happy happy to answer them to answer them to answer them within the next within the next within the next minute or two that minute or two that minute or two that we have. we have.