Create An OpenAI Proxy Server using Express and Https
Introduction
OpenAI API does not support some countries. That is the barrier for some third-country developers to access and explore this technology.
In this article, we will make OpenAI API more accessible using a proxy to transport back and forth the requests and responses between OpenAI and end users
As you may know, the proxy server is an intermediate server that forwards the request and response between the client and the target server. So let's create that
Setup the project
Let's create a new folder name openai-proxy and install express (to receive requests and responses from clients) and https (to send requests and get responses from OpenAI). We will also create a new index.js file to handle the proxy logic
mkdir openai-proxy
cd openai-proxy
npm install express https
touch index.js
Host client requests and responses
In index.js file, let's create a basic express server to receive the client request and response with basic hello world sentence:
const express = require('express');
const app = express(); // create an instance of express
app.use(express.json()); // for parsing application/json
// Setup the server to listen on port 3000 and print the log in the console
app.listen(3000, () => {
console.log('listening on port 3000');
});
// Route to return hello world
app.get('/', (req, res) => {
res.send('hello world');
})
Run the server, we should see the console log
node index.js
# listening on port 3000
Caution*: Every time we change the code, we need to restart the server to apply the new code!*
When testing with the first request, we should see the hello world sentence
curl "localhost:3000"
# hello world%
Make requests and get responses from OpenAI
Now it's time to call OpenAI API. To send a request to OpenAI we will need the OpenAI API key which can be created from
To ask OpenAI to generate the "Hello, World!" response, we will use the payload:
{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "user",
"content": "Say exactly this: Hello, World!"
}
],
"temperature": 0.7
}
We will create the POST request to the https://api.openai.com/v1/chat/completions domain. I created it in the curl so you can test it in your terminal, be noted to replace the OPENAI_API_KEY with your real key
curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer OPENAI_API_KEY" -d '{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "user",
"content": "Say exactly this: Hello, World!"
}
],
"temperature": 0.7
}' "https://api.openai.com/v1/chat/completions"
Run this curl in bash terminal and you will get this result:
{
"id": "chatcmpl-829hRBwMu5JT2cu00kKOLrPIMshQr",
"object": "chat.completion",
"created": 1695524633,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello, World!"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 15,
"completion_tokens": 4,
"total_tokens": 19
}
}
The value in the path Response.choices[0].message.content is Hello, World! which is what we expected from OpenAI API
Now instead of using curl, we transfer this to our code in index.js. Add the following code to our index.js
// ...
const https = require('https'); // Add this to the top of index.js
// ...
app.get("/", (req, res) => {
res.send("hello world");
callOpenAI(); // Add this
});
// Add this new function
const callOpenAI = async () => {
const options = {
hostname: "api.openai.com",
path: "/v1/chat/completions",
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
// We not put OPENAI_API_KEY in the code because it is secret.
// We will add it in the terminal later
},
};
// Payload is what we will send to OpenAI API
const payload = {
model: "gpt-3.5-turbo",
messages: [
{
role: "user",
content: "Say exactly this: Hello, World!",
},
],
temperature: 0.7,
};
// Create a request object with the options
// It will be called by the request.end() function
const openaiRequest = https.request(options, (openaiResponse) => {
let data = "";
// We receive the response in chunks via the "data" event
// Don't worry much. If you don't understand,
// just treat it as the boilerplate of https package
// to send request and receive response.
openaiResponse.on("data", (chunk) => {
data += chunk;
});
// We are done receiving the response when the "end" event is emitted
openaiResponse.on("end", () => {
console.log("Full data response: ", JSON.parse(data));
console.log("Content response: ", JSON.parse(data).choices[0].message.content);
});
});
openaiRequest.write(JSON.stringify(payload)); // Add the payload to the request
openaiRequest.end(); // Activate the request
};
Let's restart our server, this time we also provide the OPENAI_API_KEY environment variable. Be noted to replace the sk-X with your OpenAI API key:
OPENAI_API_KEY=sk-XXXXXXXXXXXXXX node index.js
# listening on port 3000
Making a request to our server and we will see the result in our terminal as below:
curl localhost:3000
# {
# id: 'chatcmpl-829uYKIQ5TRtqUwGdtqTjkK24Xg0p',
# object: 'chat.completion',
# created: 1695525446,
# model: 'gpt-3.5-turbo-0613',
# choices: [ { index: 0, message: [Object], finish_reason: 'stop' } ],
# usage: { prompt_tokens: 15, completion_tokens: 4, total_tokens: 19 }
# }
# Content response: Hello, World!
Now let's do a final step: Instead of hardcoding the options and payload, we will get it from the client request, and then return the OpenAI response to the client response. This time we act exactly like a proxy
Proxy client request and response to OpenAI server
First, refactor the callOpenAI function so that its parameters will be also the req and res variables from the express router. The idea is like this:
// Change from this
app.get("/", (req, res) => {});
const callOpenAI = () => {};
// Change to this, so we can pass the `req` and `res` variable to the
// callOpenAI function. Later we can read and write data from them.
// Also note that we use app.all("*") to accept both GET and POST request and accept all endpoints
app.all("*", (req, res) => {
callOpenAI(req, res);
});
const callOpenAI = async (clientRequest, clientResponse) => {};
We will change options data (responsible for domain, request headers) from hard coding to use the client request (req):
// Change from this
const options = {
hostname: "api.openai.com",
path: "/v1/chat/completions",
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
},
};
// Change to this
const options = {
hostname: "api.openai.com",
path: clientRequest.path, // use client request path
method: clientRequest.method, // use client request method
headers: {
"Content-Type": clientRequest.headers['content-type'], // use client request headers
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`, // Only OPEN AI API key is provided by the proxy
},
};
We also remove the hardcoded payload and use payload from client request instead
// Remove this
const payload = {
model: "gpt-3.5-turbo",
messages: [
{
role: "user",
content: "Say exactly this: Hello, World!",
},
],
temperature: 0.7,
};
// Change from this
openaiRequest.write(JSON.stringify(payload));
// Change to this
openaiRequest.write(JSON.stringify(clientRequest.body)); // The hardcode payload is removed
Finally, we change the openaiRequest object to this:
// Change from this
const openaiRequest = https.request(options, (openaiResponse) => {
let data = "";
// We receive the response in chunks via the "data" event
openaiResponse.on("data", (chunk) => {
data += chunk;
});
// We are done receiving the response when the "end" event is emitted
openaiResponse.on("end", () => {
console.log("Full data response: ", JSON.parse(data));
console.log("Content response: ", JSON.parse(data).choices[0].message.content);
});
});
// Change to this. We do nothing than immediately transport the data packages from OpenAI to our client
const openaiRequest = https.request(options, (openaiResponse) => {
// Forward the header from OpenAI to clientResponse for correct response format
clientResponse.setHeader("Content-Type", openaiResponse.headers["content-type"]);
// We receive the response in chunks via the "data" event
openaiResponse.on("data", (chunk) => {
clientResponse.write(chunk);
});
// We are done receiving the response when the "end" event is emitted
openaiResponse.on("end", () => {
clientResponse.end();
});
});
Here is the full code
const express = require("express");
const https = require("https");
const app = express(); // create an instance of express
app.use(express.json()); // for parsing application/json
// Setup the server to listen on port 3000 and print the log in the console
app.listen(3000, () => {
console.log("listening on port 3000");
});
// Route to return hello world
app.all("*", (req, res) => {
callOpenAI(req, res);
});
const callOpenAI = async (clientRequest, clientResponse) => {
const options = {
hostname: "api.openai.com",
path: clientRequest.path, // use client request path
method: clientRequest.method, // use client request method
headers: {
"Content-Type": clientRequest.headers['content-type'] || "application/json", // use client request headers
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`, // Only OPEN AI API key is provided by the proxy
},
};
// Create a request object with the options
// It will be called by the request.end() function
const openaiRequest = https.request(options, (openaiResponse) => {
clientResponse.setHeader("Content-Type", openaiResponse.headers["content-type"]);
// We receive the response in chunks via the "data" event
openaiResponse.on("data", (chunk) => {
clientResponse.write(chunk);
});
// We are done receiving the response when the "end" event is emitted
openaiResponse.on("end", () => {
clientResponse.end();
});
});
openaiRequest.write(JSON.stringify(clientRequest.body)); // Add the payload to the request
openaiRequest.end(); // Activate the request
};
Let's restart our server and test it with curl. This time instead of using OpenAI domain, we use our proxy domain. Be noted to replace the OPENAI_API_KEY with your OpenAI API key:
curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer OPENAI_API_KEY" -d '{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "user",
"content": "Say exactly this: Hello, World!"
}
],
"temperature": 0.7
}' "http://localhost:3000/v1/chat/completions"
You will get the response
{
"id": "chatcmpl-82BAhc8dzYwA4GtwmNgm0DJVqwNpa",
"object": "chat.completion",
"created": 1695530291,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello, World!"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 15,
"completion_tokens": 4,
"total_tokens": 19
}
}
Recap
We finished proxy the client request to OpenAI API. Now the client and use OpenAI API without needing the key.
Full sourcecode can be found here: https://github.com/votanlean/openai-proxy
We can develop further features:
Create the api key system to manage the client access to our proxy
Implement tiktoken library to calculate the token usage from client request prompt and from OpenAI response
Thank you for reading