WEBVTT

00:00.000 --> 00:07.000
Hi, folks.

00:07.000 --> 00:12.000
My name is Homsa, and this is Si, and together, we're the main tanners of Swift Open

00:12.000 --> 00:13.000
API Generator.

00:13.000 --> 00:19.000
Today, we'll show you how to build a proxy server in just about 15 minutes, all using Swift

00:19.000 --> 00:20.000
Open API.

00:20.000 --> 00:23.000
And we're going beyond a hell world service here.

00:23.000 --> 00:29.000
What are going to be calling real world, chatGPTAPI, and we'll even include

00:29.000 --> 00:30.000
the streaming here.

00:30.000 --> 00:34.000
We'll be covering a lot, and we'll be going pretty fast, but don't worry, we've already

00:34.000 --> 00:36.000
opened source of the code in the C today.

00:36.000 --> 00:40.000
It's part of the Swift Open API Generator examples directory, so you can check it out

00:40.000 --> 00:41.000
right after the session.

00:41.000 --> 00:45.000
Also, we're doing a live demo here with a lot of move-in bits.

00:45.000 --> 00:49.000
We'll be calling a real third party API, and one that's backed by an LLM.

00:49.000 --> 00:51.000
So, who knows what's going to happen.

00:51.000 --> 00:54.000
If something goes wrong, please bear with us.

00:54.000 --> 00:57.000
So, before we start voting, let's prepare our table.

00:57.000 --> 00:59.000
Here are the ingredients.

00:59.000 --> 01:03.000
We're going to be using Swift Open API to generate the client to talk to the

01:03.000 --> 01:07.000
chatGPTAPI, and we'll also generate the things that APIs server that uses the

01:07.000 --> 01:10.000
chatGPT client to make upstream calls.

01:10.000 --> 01:15.000
The escrowed is our editor for today, and we have the Swift and Open API extensions

01:15.000 --> 01:16.000
installed.

01:16.000 --> 01:20.000
We'll be using Def containers to build and run the server on Linux.

01:20.000 --> 01:25.000
And since Swift Open API clients and transports have a workable transport layer,

01:25.000 --> 01:30.000
we'll be using vapor and a single HTTP client on the Linux server,

01:30.000 --> 01:35.000
and we'll be using your all session when calling APIs from macOS.

01:35.000 --> 01:38.000
But first, why do we need to proxy in the first place?

01:38.000 --> 01:43.000
Why not just make the calls directly from the client up to the chatGPT?

01:43.000 --> 01:46.000
Well, there are a few reasons you might want to consider this approach.

01:46.000 --> 01:49.000
First, you don't want to hand out the credentials for your LM service API.

01:49.000 --> 01:55.000
Second, you might want to customize the prompt and evolve it pretty frequently.

01:55.000 --> 02:00.000
Third, you want to be able to switch the model you're using, or maybe even the backing

02:00.000 --> 02:02.000
LM service.

02:02.000 --> 02:06.000
And finally, you might want to provide a more tailored experience for your users,

02:06.000 --> 02:09.000
by transforming the request and responses.

02:09.000 --> 02:13.000
The proxy we're building today will take advantage of all of these benefits.

02:13.000 --> 02:16.000
So, what should we build?

02:17.000 --> 02:21.000
Well, finally, that's the goal, friends.

02:21.000 --> 02:26.000
But when we attend live games, we're often a bit underwhelmed by the creativity of the chance.

02:26.000 --> 02:32.000
The user go, like, let's go, team name, we can be better.

02:32.000 --> 02:39.000
We want something more engaging and something creative that includes specifics about our team and its players.

02:39.000 --> 02:43.000
Enter, chant VTT.

02:44.000 --> 02:47.000
That tailored API, thank you.

02:47.000 --> 02:52.000
And we're really proud of this one.

02:52.000 --> 02:59.000
It's a tailored API backed by CADGPT that provides team specific songs to sing a games.

02:59.000 --> 03:04.000
So, let's start building.

03:04.000 --> 03:10.000
First, we're going to switch over to terminal, and we're using the OpenAI API today,

03:11.000 --> 03:16.000
and specifically, CADGPT, which is the CADGPT API.

03:16.000 --> 03:20.000
We provide authentication credentials in the header,

03:20.000 --> 03:26.000
we choose the model, and we just take a load.

03:26.000 --> 03:31.000
And we actually get a reply back here with the LM standing pillow back.

03:31.000 --> 03:34.000
So, how do we go from Swift now?

03:34.000 --> 03:40.000
Well, fortunately, OpenAI publishes an Open API document on their official GitHub.

03:40.000 --> 03:47.000
And the Open API document contains over 100 operations, including the CADGPT API.

03:47.000 --> 03:51.000
And while we're only going to use the single chat completions operation today,

03:51.000 --> 03:56.000
since we're going to be generating all the data model and networking codes from the Open API document,

03:56.000 --> 04:02.000
later, we could add more functionality very easily, such as streaming audio or generating images.

04:02.000 --> 04:09.000
So, we can get started working on the client, let me hand it over to Si.

04:09.000 --> 04:11.000
Well, thanks, Monza.

04:11.000 --> 04:16.000
I'm going to switch over to DSCO, where we've already downloaded the Open API document.

04:16.000 --> 04:20.000
And the VS code, Open API plugin, comes with slightly UI preview,

04:20.000 --> 04:23.000
which allows you to explore the API interactively.

04:23.000 --> 04:26.000
This is that chat completions endpoint that we're going to use.

04:26.000 --> 04:30.000
And if I expand the operations, see the parameters it takes,

04:30.000 --> 04:34.000
you can see it takes the messages that make up the front, the model,

04:34.000 --> 04:39.000
and there's also the stream parameter here to expand the documentation.

04:39.000 --> 04:42.000
It says if we enable streaming the server is going to send tokens back

04:42.000 --> 04:47.000
as a data-only service end-to-end stream, terminates the biologic data payment.

04:47.000 --> 04:53.000
Okay, Swift Open API can generate all the tokens we need to call this API.

04:53.000 --> 04:59.000
So, I'm going to head over to package Monza first, and I'm going to add a generated chatGPT file.

05:00.000 --> 05:03.000
We'll start by adding the package dependencies that we need.

05:03.000 --> 05:08.000
And because we're going to use this generated clients from a proxy server target later,

05:08.000 --> 05:10.000
which will have its own Open API documents,

05:10.000 --> 05:17.000
I'm going to generate the chatGPT clients in a dedicated target.

05:17.000 --> 05:20.000
So, this target has depend on the Open API plugin,

05:20.000 --> 05:22.000
and it also depends on the Open API runtime library.

05:22.000 --> 05:26.000
You provide some of the common types used by the generated code.

05:26.000 --> 05:29.000
I'm going to add this target as a dependency as a CLI,

05:29.000 --> 05:32.000
along with the URL session transport.

05:32.000 --> 05:34.000
Now, I'm going to create a source directory for the target,

05:34.000 --> 05:38.000
and I'm going to remove that Open API document into that directory.

05:43.000 --> 05:46.000
And this plugin requires a company file.

05:46.000 --> 05:48.000
I'm going to create that too.

05:48.000 --> 05:57.000
Yeah, when they think you serve filtering configuration,

05:57.000 --> 06:00.000
those are just the operations we've introduced in today,

06:00.000 --> 06:03.000
which can speed up the build as well as a large API.

06:03.000 --> 06:08.000
The one more thing we need is, because this API is final communications,

06:08.000 --> 06:09.000
we're going to need a middleware.

06:09.000 --> 06:11.000
So, I'm going to create that now.

06:11.000 --> 06:21.000
So, this middleware adds a header field to every HTTP request,

06:21.000 --> 06:25.000
and the value for the header field contains an environment variable,

06:25.000 --> 06:27.000
and we'll use that to provide the API token.

06:27.000 --> 06:33.000
So, we have all the pieces we need to start calling the chatGPT API from the CLI.

06:33.000 --> 06:37.000
Over here, and I'll start by importing the modules we're going to use,

06:37.000 --> 06:41.000
including the chatGPT module for the generative files.

06:41.000 --> 06:47.000
And our free compliance, using the URL session transport,

06:47.000 --> 06:50.000
and the middleware that we just created.

06:50.000 --> 06:54.000
Next, I'm going to prepare the messages that make up the prompt.

06:54.000 --> 06:56.000
First, we have the system prompt.

06:56.000 --> 06:58.000
Next, a little odd thing.

06:58.000 --> 07:03.000
And we're going to ask the LN to build a fun and witty chance for a team,

07:03.000 --> 07:08.000
and then we'll determine which team I'm going to make my mind open for.

07:08.000 --> 07:11.000
Next, I'm going to print a placeholder to the console,

07:11.000 --> 07:14.000
and I also don't want to say we're buffering on standard output.

07:14.000 --> 07:16.000
That means, whatever we've printed the console,

07:16.000 --> 07:19.000
which will be our part for requests, we'll see them immediately.

07:19.000 --> 07:21.000
It's an important first streaming network.

07:21.000 --> 07:24.000
Now, if you want to make the request,

07:24.000 --> 07:30.000
the generative client will contain one method for every API operation.

07:30.000 --> 07:33.000
And the parameters that method exists will be generated data model time.

07:33.000 --> 07:35.000
So here we've got those messages.

07:35.000 --> 07:39.000
We also have the model, which is a generated enum value.

07:39.000 --> 07:41.000
We're going to enable streaming.

07:41.000 --> 07:43.000
And we're going to use this extra parameter here,

07:43.000 --> 07:45.000
to temperature, which we're going to set the zero,

07:45.000 --> 07:48.000
because we don't want any hot takes during the demo.

07:48.000 --> 07:53.000
So the response type itself is also a generated enum.

07:53.000 --> 07:56.000
So it allows you to exhaustively switch over all the documented responses

07:56.000 --> 07:57.000
to serve a good return.

07:58.000 --> 08:00.000
Or if you know what you're expecting,

08:00.000 --> 08:04.000
you can use a type say throwing API to extract the content type.

08:04.000 --> 08:07.000
You want here, we expect text event stream,

08:07.000 --> 08:10.000
which is the content type and serve a sense event.

08:10.000 --> 08:13.000
And it returns an async sequence of bytes.

08:13.000 --> 08:15.000
So there's no buffering in the client.

08:15.000 --> 08:19.000
The open API runtime library provides convenient helper functions

08:19.000 --> 08:23.000
for common content types, the video was streaming, including certain sense events.

08:23.000 --> 08:27.000
So with this one line, we can decode that async sequence of bytes

08:27.000 --> 08:31.000
into an async sequence of type values.

08:31.000 --> 08:35.000
Now all this left is for us to iterate over the async sequence

08:35.000 --> 08:39.000
and print the partial responses as they come in.

08:39.000 --> 08:42.000
So this is all the code you need to write by hand

08:42.000 --> 08:45.000
to make a streaming API call to catch easy.

08:45.000 --> 08:48.000
All the serialization of the data model types,

08:48.000 --> 08:51.000
as well as the streaming HTTP request and response logic,

08:51.000 --> 08:53.000
is all handled by the iterative code.

08:53.000 --> 08:57.000
So only doesn't make it serve well and faster to get started on a project.

08:57.000 --> 09:00.000
It also removes a whole class of bugs that have comforted PhDs

09:00.000 --> 09:03.000
and thought it's right to put by hand every time.

09:03.000 --> 09:05.000
So let's give it a go.

09:09.000 --> 09:12.000
So let's say you're, again, you don't even really know much about the team,

09:12.000 --> 09:14.000
but because we're using an LLM,

09:14.000 --> 09:16.000
we can use an actual language to identify the team.

09:16.000 --> 09:18.000
Here, I'm asking for a example,

09:18.000 --> 09:20.000
that team with the bull logo,

09:20.000 --> 09:24.000
and we get a streaming response with a chance for the Chicago.

09:24.000 --> 09:26.000
That's right.

09:26.000 --> 09:28.000
Let's try something else.

09:28.000 --> 09:30.000
So once you give it to your favorite player.

09:30.000 --> 09:31.000
Clay Johnson.

09:31.000 --> 09:32.000
Clay Johnson.

09:32.000 --> 09:34.000
Let's play Johnson.

09:34.000 --> 09:35.000
Let's play Johnson.

09:35.000 --> 09:38.000
So we'll ask for the team,

09:38.000 --> 09:40.000
Clay Johnson.

09:40.000 --> 09:41.000
Thanks.

09:43.000 --> 09:45.000
Can we get?

09:45.000 --> 09:49.000
Oh, we have a Golden State Warriors chapter.

09:49.000 --> 09:53.000
So I say a few people have a chat about that, which is great.

09:53.000 --> 09:55.000
And this is a wrong team.

09:55.000 --> 09:58.000
And for those of you who are not basketball fans, let me explain.

09:58.000 --> 10:01.000
This player moves teams in the Golden State Warriors.

10:01.000 --> 10:02.000
Two Dallas map Chris.

10:02.000 --> 10:05.000
But they only move teams at the start of this season.

10:05.000 --> 10:08.000
So what's happened here is the LLM is on the best kickhand

10:08.000 --> 10:10.000
with the day to extreme them.

10:10.000 --> 10:12.000
They don't have access to the LLM team monsters.

10:13.000 --> 10:14.000
That's okay.

10:14.000 --> 10:16.000
Because we're controlling a system prompt,

10:16.000 --> 10:20.000
we can provide more context as part of the request.

10:20.000 --> 10:22.000
So I'm going to head over to BS code.

10:22.000 --> 10:26.000
I'm going to head a new file for players.txt.

10:26.000 --> 10:30.000
And I'm going to pay for all the current players

10:30.000 --> 10:33.000
and the team will pay for that.

10:33.000 --> 10:37.000
Then I'm going to provide the contents of this file

10:37.000 --> 10:39.000
as part of the system for.

10:42.000 --> 10:45.000
Now we're going to head back to console.

10:45.000 --> 10:47.000
We can run the same example.

10:47.000 --> 10:51.000
Hopefully we get back what we want.

10:51.000 --> 10:54.000
That Dallas map is done.

10:54.000 --> 10:56.000
Okay, let's check in with whatever else.

10:56.000 --> 11:04.000
So we've written a streaming chat GPT client using Swift Open API.

11:04.000 --> 11:07.000
So we've reached the point where we'd like to build all of them.

11:07.000 --> 11:09.000
With data that we maintain.

11:09.000 --> 11:11.000
I'm probably want to update.

11:11.000 --> 11:13.000
So I'm going to hand out on the.

11:13.000 --> 11:16.000
We'll start moving some of this logic into the proxy.

11:16.000 --> 11:17.000
Thank you.

11:17.000 --> 11:21.000
So we'll start by creating a new target for the proxy server.

11:27.000 --> 11:30.000
And this target depends on the chat GPT client we already had.

11:30.000 --> 11:32.000
And it uses Swift Open API generator.

11:32.000 --> 11:36.000
So we're going to need to provide an Open API documents and a config file.

11:36.000 --> 11:41.000
So create the target directory first.

11:41.000 --> 11:44.000
And the Open API document.

11:49.000 --> 11:51.000
Open the preview.

11:51.000 --> 11:55.000
And here we have the tailored API for trend GPT.

11:55.000 --> 11:59.000
It has one input for the user prompt.

11:59.000 --> 12:03.000
And a streaming format for output called JSON lines.

12:04.000 --> 12:18.000
So now that we have the API we need to set up the config file.

12:18.000 --> 12:21.000
And online the config file we saw on the client.

12:21.000 --> 12:24.000
This one will actually generate the server step for us.

12:24.000 --> 12:26.000
So another way to the plugin configure.

12:26.000 --> 12:31.000
We can create the entry file.

12:31.000 --> 12:34.000
And we can create a new dependencies.

12:34.000 --> 12:37.000
And define a type that conforms to this generated protocol.

12:37.000 --> 12:39.000
Called API protocol.

12:39.000 --> 12:44.000
This protocol defines one function for each operation in the Open API document.

12:44.000 --> 12:47.000
And the input and output types are all generated.

12:47.000 --> 12:51.000
And the handled with deceleration and validation of the request.

12:51.000 --> 12:54.000
And the serialization of the response.

12:54.000 --> 12:58.000
So for now this handler just goes an unemployed error.

12:58.000 --> 13:04.000
And we do that by calling this generated function called register handlers.

13:04.000 --> 13:10.000
Which route is the incoming requests to our type save and the functions here above.

13:10.000 --> 13:12.000
So now that we have the scaffolding ready.

13:12.000 --> 13:15.000
Let's start filling in the business logic in the handler.

13:15.000 --> 13:18.000
We step by unwraping the request content in the body.

13:18.000 --> 13:22.000
And returning an error in an unexpected content was provided.

13:22.000 --> 13:27.000
Next I'm going to switch over the command line code base.

13:27.000 --> 13:30.000
I'm going to take all of this code.

13:30.000 --> 13:35.000
I'm going to copy it and paste it in the handler.

13:35.000 --> 13:41.000
And we're already now we're updating this code that used to run in our macOS CI to instead run

13:41.000 --> 13:43.000
in our Linux server.

13:43.000 --> 13:48.000
So first I'm going to update a transport from URL session to a single HTTP client.

13:48.000 --> 13:52.000
And again notice that this is the same generated touch HTTP client we saw before.

13:52.000 --> 13:56.000
Just go strap with a different underlying HTTP library.

13:56.000 --> 14:00.000
So next I'm going to update the input instead of taking the command line arguments.

14:00.000 --> 14:04.000
That we're going to take it from the request by.

14:04.000 --> 14:11.000
And next I'm going to delete all of this code that used to run to the console.

14:11.000 --> 14:16.000
So what we have here is this async sequence of these shared GPT payload elements.

14:16.000 --> 14:21.000
So we need to map over to when async sequence of these proxy payload elements,

14:21.000 --> 14:23.000
which are simplified version of them.

14:23.000 --> 14:28.000
And finally, we take this map async sequence.

14:28.000 --> 14:30.000
We encode it as JSON lines.

14:30.000 --> 14:35.000
And we will turn it as the 200 okay response from our server.

14:35.000 --> 14:44.000
So I'm going to build and run the server now.

14:44.000 --> 14:46.000
And in a second when it finishes building,

14:46.000 --> 14:51.000
it will start running on localhost port 8080.

14:52.000 --> 14:57.000
That's great, that's running. So let's give it a try.

14:57.000 --> 15:00.000
So we're using crawl here in streaming mode,

15:00.000 --> 15:06.000
and we're calling the chant endpoint on our local server.

15:06.000 --> 15:09.000
And once the HTTP starts streaming the response to the proxy,

15:09.000 --> 15:13.000
we see forwarded to crawl as this individual JSON line events.

15:13.000 --> 15:15.000
So let's recap what we are.

15:15.000 --> 15:18.000
We've got this Linux proxy server offering a tailored API,

15:18.000 --> 15:22.000
and of course, tragedy, PT, and it returns a streaming response.

15:22.000 --> 15:24.000
And since we give it a few more minutes left,

15:24.000 --> 15:28.000
let me head over to side to update the climbs to call this proxy server now.

15:31.000 --> 15:33.000
Okay, thanks Sansa.

15:33.000 --> 15:37.000
So I'm going to leave this proxy server running

15:37.000 --> 15:39.000
and inside the Linux container,

15:39.000 --> 15:42.000
listing on localhosts, and we'll update the CLI to call that,

15:42.000 --> 15:45.000
instead of calling CHAPGPT directly.

15:46.000 --> 15:49.000
Now, because the proxy server was also defined using Open API,

15:49.000 --> 15:51.000
we're going to use Swift Open API one more time,

15:51.000 --> 15:54.000
but this time we're going to be generating a proxy client.

15:54.000 --> 15:56.000
So I'm going to start by taking the Open API document,

15:56.000 --> 15:59.000
and the config file from the proxy server,

15:59.000 --> 16:02.000
and copy it into the client CLI target directory.

16:02.000 --> 16:06.000
I'll update the config file to generate the client code.

16:08.000 --> 16:11.000
So I'll update the package manifest so that the client CLI

16:11.000 --> 16:14.000
or no longer depends on the CHAPGPT module,

16:14.000 --> 16:16.000
because it doesn't need anymore.

16:16.000 --> 16:20.000
And instead, it uses Open API plugin to generate the proxy client.

16:20.000 --> 16:24.000
So now I'll update the code for the CLI.

16:24.000 --> 16:28.000
I'll start by removing the CHAPGPT import,

16:28.000 --> 16:32.000
and then replace CHAPGPT.client with just client.

16:32.000 --> 16:35.000
Because this client is now target local,

16:35.000 --> 16:38.000
it's the generated client for the proxy server.

16:38.000 --> 16:41.000
We're going to use a local first URL,

16:41.000 --> 16:43.000
and we don't need this middle way more,

16:43.000 --> 16:47.000
because authentication is being handled by the proxy server.

16:47.000 --> 16:50.000
The system point is also being handled by the proxy server,

16:50.000 --> 16:52.000
along with the updated team roasters.

16:52.000 --> 16:53.000
So we'll give it to that,

16:53.000 --> 16:59.000
and any code relates to creating the request or making the request.

16:59.000 --> 17:03.000
And I'll replace it with this updated code to make a request

17:03.000 --> 17:04.000
to the proxy server.

17:04.000 --> 17:07.000
So here we're using the Create CHAPGPT endpoint,

17:07.000 --> 17:09.000
providing just a user input,

17:09.000 --> 17:12.000
and we're iterating over much simplified JSON lines

17:12.000 --> 17:16.000
async sequence for the contains only the information that we're interested in.

17:16.000 --> 17:18.000
So now if I run our example one more time,

17:18.000 --> 17:20.000
we'll play Thompson,

17:20.000 --> 17:23.000
which hopefully still get what we want,

17:23.000 --> 17:25.000
which is a data smaver exchange because the service handling

17:25.000 --> 17:27.000
those updated roasters in the system prompt,

17:27.000 --> 17:30.000
and should all be streamed ends at end.

17:31.000 --> 17:33.000
So we've got to be as code,

17:33.000 --> 17:37.000
we can see that vapor handled the chant request here.

17:37.000 --> 17:41.000
So we've covered a lot of ground here, so let me recap.

17:41.000 --> 17:43.000
We've built a proxy server,

17:43.000 --> 17:46.000
and a CLI for a tailored use case backed by CHAPGPT,

17:46.000 --> 17:48.000
and it had ends streaming.

17:48.000 --> 17:51.000
And we didn't need to write a single line of networking code,

17:51.000 --> 17:53.000
or serialization logic, because of Swift open API.

17:53.000 --> 17:55.000
In just over 15 minutes,

17:55.000 --> 17:56.000
we've written three targets,

17:56.000 --> 17:59.000
using three different underlying transports for two platforms.

18:00.000 --> 18:03.000
Thank you so much for joining us for this session.

18:03.000 --> 18:05.000
If you have any questions about this demo,

18:05.000 --> 18:07.000
we'll split the open API project,

18:07.000 --> 18:08.000
we can just follow and say hi,

18:08.000 --> 18:11.000
and we'll be out in the hallway after this session.